Hard disk failure prediction method and device, storage medium and electronic equipment

Abstract

The application discloses a hard disk fault prediction method and device, a storage medium and an electronic equipment, and relates to the technical field of computer storage. The method comprises the following steps: acquiring historical state data of a plurality of storage hard disks recorded in a server data center, wherein the historical state data comprises hard disk fault data and hard disk non-fault data; performing trajectory guidance processing on the hard disk fault data by using a target diffusion generation model to obtain virtual fault data; determining a training data set based on the virtual fault data and the historical state data, training a target prediction model by using the training data set, and obtaining a hard disk prediction model; and in the case of determining to predict a target storage hard disk, inputting real-time hard disk state data of the target storage hard disk in a previous period into the hard disk prediction model, and predicting fault information of the target storage hard disk in a next period by using the hard disk prediction model. The problem of low accuracy of hard disk fault prediction is solved.

Description

Technical Field

[0001] This application relates to the field of computer storage technology, and more specifically, to a method, apparatus, storage medium, and electronic device for predicting hard disk failures. Background Technology

[0002] With the rapid development of information technology, data storage centers have become a key component of modern digital infrastructure, undertaking the storage and processing of massive amounts of data. Hard disk drives (HDDs) and solid-state drives (SSDs), as the primary media for data storage, directly impact the stability and security of data centers. However, hard disk failures are often sudden and unpredictable, leading not only to data loss but also triggering chain reactions that threaten the entire system's operation. Currently, most hard disks incorporate self-monitoring, analysis, and reporting technologies, periodically reading sensor data to assess their health. However, due to the low frequency of hard disk failures, the actual number of collected failure samples is extremely small compared to data from normally functioning hard disks. This results in a severe imbalance in the training dataset, making it difficult for machine learning-based predictive models to effectively learn failure modes, often leading to false positives or false negatives. Secondly, hard disk failures are often the result of multiple factors, involving complex correlations between various hard disk attributes over time. While existing predictive models can handle time-series data to some extent, they struggle to capture long-term time-series dependencies and cross-attribute correlations, resulting in limited understanding and predictive accuracy of failure modes.

[0003] There is currently no effective solution to the problem of low accuracy in hard drive failure prediction in related technologies. Summary of the Invention

[0004] This application provides a method, apparatus, storage medium, and electronic device for predicting hard disk failures, in order to at least solve the problem of low accuracy in hard disk failure prediction in related technologies.

[0005] According to one embodiment of this application, a method for predicting hard disk failure is provided, comprising: acquiring historical state data of multiple storage hard disks recorded in a server data center, wherein the historical state data includes: hard disk failure data and hard disk non-failure data; performing trajectory guidance processing on the hard disk failure data through a target diffusion generation model to obtain virtual failure data; wherein the target diffusion generation model is used to virtually expand the hard disk failure data through a first loss function and a conditional embedding mechanism; determining a training dataset based on the virtual failure data and the historical state data, and using the training dataset to train a target prediction model to obtain a hard disk prediction model, wherein the hard disk prediction model is used to perform cross-attribute attention analysis on the hard disk state data; when it is determined that a target storage hard disk is to be predicted, inputting the real-time hard disk state data of the target storage hard disk in the previous period into the hard disk prediction model, and predicting the failure information of the target storage hard disk in the next period through the hard disk prediction model, wherein the failure information includes: hard disk failure probability information and hard disk remaining lifespan information output by the hard disk prediction model.

[0006] According to another embodiment of this application, a hard disk failure prediction device is provided, comprising: an acquisition module, configured to acquire historical state data of multiple storage hard disks recorded in a server data center, wherein the historical state data includes: hard disk failure data and hard disk non-failure data; a processing module, configured to perform trajectory guidance processing on the hard disk failure data through a target diffusion generation model to obtain virtual failure data; wherein the target diffusion generation model is used to virtually expand the hard disk failure data through a first loss function and a conditional embedding mechanism; a determination module, configured to determine a training dataset based on the virtual failure data and the historical state data, and use the training dataset to train a target prediction model to obtain a hard disk prediction model, wherein the hard disk prediction model is used to perform cross-attribute attention analysis on the hard disk state data; and a prediction module, configured to, when it is determined that a target storage hard disk is to be predicted, input the real-time hard disk state data of the target storage hard disk in the previous period into the hard disk prediction model, and predict the failure information of the target storage hard disk in the next period through the hard disk prediction model, wherein the failure information includes: hard disk failure probability information and hard disk remaining lifespan information output by the hard disk prediction model.

[0007] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0008] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0009] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0010] This application collects historical state data of storage hard drives from server data centers. This data includes not only known hard drive failure instances but also a large number of records of normal hard drive operation. Using a target diffusion generation model, guided by the "failure trajectory" in the real failure data, a series of virtual failure data are generated. This process optimizes the model's performance through a first loss function and combines a conditional embedding mechanism to ensure that the generated data maintains statistical consistency with the real failure data. The generated virtual failure data is merged with non-failure data from the original historical hard drive state data to construct a balanced training dataset. Based on this dataset, a target prediction model is trained to obtain a hard drive prediction model. This model can perform in-depth analysis of the real-time hard drive state data of the input target storage hard drive in the previous period, capture the complex dependencies between different hard drive attributes through a cross-attribute attention mechanism, and predict the failure information of the target storage hard drive in the next period, thereby achieving accurate prediction of hard drive failure probability and remaining lifespan. The above technical solution solves the problem of low accuracy in hard drive failure prediction. Furthermore, by using a target diffusion generation model to create virtual fault data that matches the real fault trajectory, and merging it with normal operation records to construct a balanced training dataset, a prediction model with a cross-attribute attention mechanism was trained to achieve accurate prediction of hard drive failure probability and remaining service life. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a hardware structure block diagram of a server device for a hard disk failure prediction method according to an embodiment of this application.

[0013] Figure 2 This is a flowchart of a hard disk failure prediction method according to an embodiment of this application;

[0014] Figure 3This is a schematic diagram of an offline training process for a cross-attribute attention Transformer model according to an embodiment of this application;

[0015] Figure 4 This is a schematic diagram of an online inference process for a cross-attribute attention Transformer model according to an embodiment of this application;

[0016] Figure 5 This is a structural block diagram of a hard disk failure prediction device according to an embodiment of this application; Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0018] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0019] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] As an optional implementation, the method embodiments provided in this application can be executed on a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of a server device according to an embodiment of the present application for a method for predicting hard disk failures. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the image. A processor 102 (which may include, but is not limited to, a microprocessor MPU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0021] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the hard disk failure prediction method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0022] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0023] This embodiment provides a method for predicting hard drive failures, applied to the aforementioned server equipment. Figure 2 This is a flowchart of a hard disk failure prediction method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0024] Step S202: Obtain historical status data of multiple storage hard drives recorded in the server data center, wherein the historical status data includes: hard drive failure data and hard drive non-failure data;

[0025] Step S204: The hard disk fault data is processed by a target diffusion generation model to obtain virtual fault data; wherein, the target diffusion generation model is used to virtually expand the hard disk fault data through a first loss function and a conditional embedding mechanism.

[0026] Optionally, 100 real hard drive failure data points were extracted from the data center's maintenance logs. Each hard drive's data included attribute values ​​from SMART (Self-Monitoring, Analysis, and Reporting Technology) time-series data for the 90 days preceding the failure. These 100 real hard drive failure data points were used to train a target diffusion generation model. The conditional embedding mechanism, by learning the SMART data sequences of these hard drives before the failure, guided the target diffusion generation model with failure trajectories. Through iterative optimization of the first loss function, the model learned how to evolve from a "healthy" state to a "failed" state. After model training, one or more real failure trajectory prototypes were input, and 1000 virtual failure data points were generated through the conditional diffusion process. These samples maintained statistical properties consistent with real failure data while also exhibiting different failure evolution paths, enriching the diversity of failure modes.

[0027] Step S206: Determine a training dataset based on the virtual fault data and the historical state data, and use the training dataset to train the target prediction model to obtain a hard disk prediction model, wherein the hard disk prediction model is used to perform cross-attribute attention analysis on the hard disk state data.

[0028] Optionally, SMART data for all hard drives over the past year was extracted from the data center's historical operational records, including the status of drives that had already failed. Using a "failure trajectory-guided conditional diffusion model," thousands of virtual failure data points were generated, templated with known failed hard drive SMART data. These data points maintained statistical characteristics consistent with real failure data, but differed in specific values ​​and failure trajectories, increasing data diversity and failure scenario coverage. The generated virtual failure data was mixed with a large amount of historical status data from non-failed hard drives to construct a balanced training dataset. In each sample, in addition to time-series data containing SMART attributes, a failure label (whether failure is imminent) and Remaining Useful Life (RUL) were explicitly labeled. Using the constructed balanced training dataset, a hard drive failure prediction model based on a "cross-attribute attention Transformer" was trained. This model gradually optimized its ability to predict failures and evaluate RUL by learning the complex relationships between SMART attributes and the long-term temporal dependencies of failure modes. The above embodiments, by constructing a training dataset based on virtual fault data and historical state data, and combining it with a hard disk prediction method based on cross-attribute attention analysis, can significantly improve the performance of the prediction model and bring substantial benefits to the operation and maintenance of data centers.

[0029] Step S208: When it is determined that the target storage hard disk is to be predicted, the real-time hard disk status data of the target storage hard disk in the previous period is input into the hard disk prediction model. The hard disk prediction model predicts the fault information of the target storage hard disk in the next period. The fault information includes: hard disk fault probability information and hard disk remaining lifespan information output by the hard disk prediction model.

[0030] Optionally, when operations and maintenance personnel identify a storage hard drive that requires health status analysis, that hard drive is used as a prediction target. After identifying the target hard drive, real-time hard drive status data from the previous period (e.g., the past 24 hours) is automatically collected from that hard drive. This data includes, but is not limited to, SMART data, and is input into the hard drive prediction model. The hard drive prediction model calculates the probability of hard drive failure and remaining lifespan in the next 24 hours based on a dataset generated by cross-attribute attention mechanisms and conditional diffusion.

[0031] The above method collects historical state data of storage hard drives from server data centers. This data includes not only known hard drive failure instances but also a large number of records of normal hard drive operation. Using a target diffusion generation model, guided by the "failure trajectory" in the real failure data, a series of virtual failure data are generated. This process optimizes the model's performance through a first loss function and combines a conditional embedding mechanism to ensure that the generated data maintains statistical consistency with the real failure data. The generated virtual failure data is merged with non-failure data from the original historical hard drive state data to construct a balanced training dataset. Based on this dataset, the target prediction model is trained to obtain a hard drive prediction model. This model can perform in-depth analysis of the real-time hard drive state data of the input target storage hard drive in the previous period, capture the complex dependencies between different hard drive attributes through a cross-attribute attention mechanism, and predict the failure information of the target storage hard drive in the next period, thereby achieving accurate prediction of hard drive failure probability and remaining lifespan. This technical solution solves the problem of low accuracy in hard drive failure prediction. Furthermore, by using a target diffusion generation model to create virtual fault data that matches the real fault trajectory, and merging it with normal operation records to construct a balanced training dataset, a prediction model with a cross-attribute attention mechanism was trained to achieve accurate prediction of hard drive failure probability and remaining service life.

[0032] In an exemplary embodiment, virtual fault data is obtained by trajectory-guided processing of hard disk fault data through a target diffusion generation model, including: extracting feature codes corresponding to hard disk attributes before and after the fault in the hard disk fault data, and converting the feature codes into conditional embedding vectors through a conditional encoder; adding noise to the hard disk fault data to obtain noisy fault data; and inputting the noisy fault data, the conditional embedding vectors, and the duration into the target diffusion generation model for denoising to obtain virtual fault data.

[0033] Optionally, a large amount of SMART data on hard drives is collected from the data center's monitoring system. For each hard drive sample confirmed to have failed, the data before and after the failure is analyzed to extract key feature change patterns and construct corresponding feature codes. These feature codes are then transformed into conditional embedding vectors using a small network with a Gated Recurrent Unit (GRU) or Transformer structure. These conditional embedding vectors serve as guiding conditions for the model to generate virtual failure data. The collected real-world failed hard drive data is then positively noise-added to obtain noisy failure data. This process controls the relationship between noise intensity and time to ensure that the data maintains a natural noise evolution at different time points. Finally, the noisy failure data, the conditional embedding vectors, and the current time step information are input into the target diffusion generation model. The model uses a noise prediction network to progressively remove noise, restoring virtual failure data that matches the original failure pattern.

[0034] Through the above process, a large amount of high-quality virtual fault data can be generated. This data follows the characteristic patterns of real faulty hard drives, enabling the model to understand the fault occurrence mechanism more accurately and comprehensively, thus demonstrating higher accuracy and early warning capabilities in the final hard drive fault prediction.

[0035] In one exemplary embodiment, adding noise to hard disk failure data includes: adding noise to the data and weighted mixing the hard disk failure data according to a first formula, wherein the first formula is: , Add noise to the data. To preset the noise control coefficient, x t x0 represents the noise-inducing fault data, and x0 represents the hard disk fault data.

[0036] Optionally, a weighted mixture of noise-added data and hard drive failure data is applied. First, a progressively decaying noise scheduling coefficient strategy is determined. For example, the noise coefficient is made close to 1 in the initial time step t, indicating almost no noise. As time progresses, the noise coefficient gradually decreases, and the proportion of noise increases until the data is almost entirely composed of noise at the last moment. For hard drive failure data x0, according to the first formula in the above embodiment, a series of noisy data x are generated by adjusting the noise coefficient. t This series of data begins with the almost noise-free data from the faulty hard drive x0. As the time step t increases, noise is gradually added, forming a data set with different states ranging from clear to blurry.

[0037] By employing the above process and applying the first formula to add noise to hard drive failure data, the noise during data acquisition and the natural variability in the hard drive failure process can be simulated. Preset noise scheduling coefficients control the intensity of noise at different time points, enabling the model to adapt to noise environments ranging from mild to severe during training. This makes the model more robust in actual deployment, capable of handling various types of signal interference and data imperfections.

[0038] In an exemplary embodiment, inputting noisy fault data, conditional embedding vectors, and duration into a target diffusion generation model for denoising to obtain virtual fault data includes: predicting the noisy fault data using a noise prediction sub-model in the target diffusion generation model to obtain a prediction result, wherein the noise prediction sub-model minimizes the difference between the noisy fault data and the noisy fault data in the prediction result using a first loss function, the first loss function being: E represents the expected value. Add noise to the data. θ For the noise prediction sub-model, x t The noisy fault data is defined as follows: t is the time step encoding, and c is the conditional embedding vector. The prediction results are then denoised based on the added noise to the data to obtain virtual fault data.

[0039] Optionally, the generated noisy fault data x t The conditional embedding vector c and the current time step t are combined and used as inputs to the target diffusion generation model. The noise prediction sub-model in the diffusion generation model... θ For x t The noise prediction sub-model is used to predict noise in the data. θ The model is trained to minimize the first loss function L, which is the mean square error between the noise prediction and the actual noise. At each time step t, the model uses the noise prediction sub-model. θ The prediction results are used to gradually eliminate x t The noise in the data is used to obtain virtual fault data.

[0040] The above process generates highly realistic virtual fault data, which not only significantly increases the amount of training data, but also ensures that the model can deeply understand the dynamic changes of hard disk faults and the complex relationships between attributes, thus demonstrating better robustness and accuracy in prediction tasks.

[0041] In an exemplary embodiment, training a target prediction model using a training dataset to obtain a hard disk prediction model includes: transposing the dimensions of the training dataset using a preset transpose function, wherein the transpose process adjusts the dimensional order of the training dataset to determine a target dimensional order that fits the training requirements of the target prediction model; determining a target training dataset based on the transpose process result, and training the target prediction model using the target training dataset to obtain the hard disk prediction model.

[0042] Optionally, the original layout of the training dataset is (N, W, F), where N is the total number of samples, W is the time window length, and F is the number of attributes. To enable the model to learn the interactions between attributes more effectively, a predefined transpose function is needed to transform the dimensions of the training dataset. In Python, this is done using the `transpose` function from the NumPy library. Assuming `data` is the original dataset with shape = (N, W, F), `transposed_data = np.transpose(data, axes = (0, 2, 1))`, setting the `axes` parameter to (0, 2, 1) indicates that the total number of samples N remains unchanged. The dimensions F and W are swapped, resulting in a dataset layout of (N, F, W), representing the number of samples, the number of attributes, and the time window length. Using the transposed dataset, the data is divided into a series of subsequences according to the sliding time window principle. Each subsequence contains data from F attributes and W time points, serving as input for model training. Simultaneously, each subsequence is labeled with either a fault or normal state, constructing the final training dataset. The cross-attribute attention Transformer model is trained using a pre-constructed target training dataset. The model can directly perform self-attention computation on the attribute dimension F, learning the associations between different SMART attributes and the trends of these associations over time. During training, the model continuously optimizes its predictive ability by minimizing the loss function, ultimately resulting in a hard drive prediction model that can accurately predict the probability of hard drive failure and remaining lifespan.

[0043] The above process demonstrates how a simple dimension transpose operation can optimize the data structure to meet the training requirements of the target prediction model, thereby improving the accuracy and practicality of hard disk failure prediction.

[0044] In an exemplary embodiment, transposing the dimensions of the training dataset using a preset transpose function includes: determining a first dimension order of the training dataset, wherein the first dimension order is arranged in the order of batch size dimension, time dimension, and attribute dimension; and using the preset transpose function to swap the relative positions of the time dimension and attribute dimension in the first dimension order to obtain a second dimension order.

[0045] Optionally, the original training dataset has a layout of (N, W, F), where N represents the batch size (the number of disk data samples processed simultaneously), W represents the time window length (the number of consecutive time points contained in each sample), and F represents the number of SMART attributes. In the original dataset layout, the first dimension is ordered by default as [batch size, time window length, number of attributes], meaning the data is arranged with batch size as the outermost dimension, followed by the time dimension, and finally the attribute dimension. To enable the model to directly apply cross-attribute attention mechanisms on the attribute dimension, the relative positions of the time and attribute dimensions are swapped, i.e., a transpose operation. This adjusts the dataset layout from (N, W, F) to (N, F, W), making [batch size, number of attributes, time window length] the new second dimension order.

[0046] Through the above process, the model can perform calculations directly on the attribute dimension, which helps the model learn more deeply the nonlinear relationships and long-term dependencies between attributes.

[0047] In an exemplary embodiment, after determining the target training dataset based on the transpose processing result, the method further includes: using multiple weight matrices to transform the feature data of each hard disk attribute corresponding to the attribute dimension of the target training dataset to obtain multiple target vectors, wherein the multiple weight matrices include: a query matrix, a key matrix, and a value matrix; and determining the attention weights between different hard disk attributes based on the multiple target vectors.

[0048] Optionally, for each disk attribute in the target training dataset, a linear transformation is performed using the corresponding query matrix, key matrix, and value matrix. Specifically, the query matrix converts attribute features into query vectors, the key matrix into key vectors, and the value matrix into value vectors. After obtaining the query vectors, key vectors, and value vectors, attention weights between different disk attributes are determined based on these vectors.

[0049] The above process uses query matrices, key matrices, and value matrices to process the transposed training dataset, enhancing the performance of the hard drive prediction model in hard drive failure prediction. These matrices help the model more effectively learn the interdependencies and attention weights between different hard drive attributes, thereby improving the accuracy of failure prediction and the precision of remaining lifespan estimation.

[0050] In one exemplary embodiment, multiple weight matrices are used to transform the feature data of each hard disk attribute corresponding to the attribute dimension of the target training dataset to obtain multiple target vectors, including: using the query matrix in the multiple weight matrices to transform the feature data of each hard disk attribute to obtain a first target vector; using the key matrix in the multiple weight matrices to transform the feature data of each hard disk attribute to obtain a second target vector; and using the value matrix in the multiple weight matrices to transform the feature data of each hard disk attribute to obtain a third target vector.

[0051] Optionally, by transforming the feature data of each attribute using a weight matrix, three target vector sets can be obtained. Specifically, the query matrix Q is used to perform a linear transformation on the attribute feature data to obtain the first target vector (query vector set), the key matrix K is used to perform a linear transformation to obtain the second target vector (key vector set), and the value matrix V is used to perform a transformation to obtain the third target vector (value vector set).

[0052] Through the above process, the feature data of the hard disk SMART attributes are transformed using query matrix, key matrix and value matrix to generate target vector set, which effectively enhances the model's ability to learn the dependencies between attributes, thereby significantly improving the accuracy of hard disk failure prediction and the precision of remaining service life assessment.

[0053] In an exemplary embodiment, determining attention weights between different hard disk attributes based on multiple target vectors includes: calculating the dot product of a first target vector and a second target vector among the multiple target vectors to obtain an attention score; generating an attention score matrix given multiple attention scores corresponding to the training dataset; normalizing the attention score matrix; and determining the attention weights between different hard disk attributes based on the value corresponding to each element in the normalization result.

[0054] Optionally, for each time step and each failed hard drive sample, the dot product between the query vector and the key vector is calculated along the attribute dimension to obtain an attention score. Multiple calculated attention scores are combined to form an attention score matrix. To ensure a reasonable allocation of attention weights, the attention score matrix is ​​normalized. Typically, this is done by applying the softmax function. In the normalized matrix, the sum of the elements in each column is 1, ensuring that the total weight of all attributes is consistent at each time point. Each element in the normalized attention weight matrix represents the degree of attention one hard drive attribute gives to another attribute.

[0055] Through the above process, the model gains a deeper understanding of the intrinsic relationships between hard drive SMART attributes and how these relationships evolve over time, affecting the hard drive's state. This enables the model to more accurately identify failure modes and predict the remaining lifespan of hard drives when processing real-world hard drive data, providing strong support for data center hardware management and effectively reducing maintenance costs and the risk of data loss.

[0056] In an exemplary embodiment, after generating an attention score matrix based on multiple attention scores corresponding to the training dataset, the method further includes: determining the target attention weights in the attention score matrix corresponding to each of the multiple target vectors; and performing a weighted summation of the multiple target vectors based on the target attention weights to obtain a target attribute feature vector, wherein the target attribute feature vector is used to determine the correlation between different hard disk attributes.

[0057] Optionally, a training dataset containing SMART attributes of hard drives is provided, with each hard drive's attribute value encoded as a multi-dimensional vector. An attention mechanism is used to determine the correlation between different attributes and to generate a more comprehensive target attribute feature vector that better reflects the health of the hard drives. For each input hard drive attribute feature vector, the attention mechanism calculates an attention score matrix. Each row of this matrix corresponds to an attribute, and each column represents a time point, with matrix elements indicating the importance of the attribute at that time point. From the attention score matrix, the relative importance of all attributes at each time point is calculated using a softmax function to obtain the target attention weights. This weight vector guides the subsequent weighted summation process. The attention weights are multiplied by each attribute feature vector, and then all attribute feature vectors are summed to obtain a target attribute feature vector that more emphasizes key attribute information.

[0058] The process described above, which involves determining the target attention weights and performing a weighted summation to obtain the target attribute feature vector, not only enhances the model's understanding and prediction capabilities but also improves the interpretability of the prediction results. This approach has significant advantages for complex tasks such as hard drive failure prediction.

[0059] In an exemplary embodiment, after weighted summation of multiple target vectors based on target attention weights to obtain a target attribute feature vector, the method further includes: performing multi-layer processing on the target attribute feature vector through a preset feedforward network to obtain a target comprehensive feature vector, wherein the preset feedforward network includes two fully connected layers, the first fully connected layer is used for nonlinear transformation of the target attribute feature vector, and the second fully connected layer reintegrates the nonlinearly transformed features into the same dimension as the target attribute feature vector.

[0060] Optionally, in a hard disk failure prediction system, a target attribute feature vector is obtained through a cross-attribute attention mechanism. Next, this vector is processed in multiple layers using a feedforward network to obtain a comprehensive target feature vector. Specifically, a feedforward network with two fully connected layers is created. The first layer handles non-linear transformations, and the second layer integrates the features, restoring them to the same dimension as the target attribute feature vector. The activation function and number of neurons in the first fully connected layer are set, and the output dimension of the second fully connected layer is the same as the dimension of the target attribute feature vector. The target attribute feature vector is input into the feedforward network and processed sequentially through the two fully connected layers. After forward propagation, the output comprehensive target feature vector contains information after non-linear transformation and integration, which can be used for subsequent hard disk failure prediction or remaining lifespan estimation.

[0061] By employing the above process and performing multi-level processing on the target attribute feature vectors through a feedforward network, the performance of the model can be significantly improved, including the accuracy of predicting hard drive health status and the ability to generalize to different failure modes.

[0062] In an exemplary embodiment, real-time hard disk status data of the target storage hard disk in the previous period is input into a hard disk prediction model. The hard disk prediction model then predicts fault information indicating that the target storage hard disk will fail in the next period. This includes: performing cross-attribute attention analysis on the real-time hard disk status data in the previous period using the hard disk prediction model to obtain analysis results; and determining the failure probability information and remaining lifespan information of the target storage hard disk based on the analysis results. The failure probability information is used to determine the likelihood of the target storage hard disk failing, and the remaining lifespan information is used to assess the healthy lifespan of the target storage hard disk.

[0063] Optionally, real-time hard drive status data of the target storage hard drive in the previous period (such as daily SMART data from the past week) can be input into the hard drive prediction model. The cross-attribute attention mechanism in the hard drive prediction model performs attention calculations on each attribute in the input data. At each time step, the model assigns an attention score to each attribute, thus focusing more on attributes closely related to failure prediction. Based on the results of the cross-attribute attention analysis, the probability of failure and the remaining lifespan of the hard drive are estimated through the model's output layer. The output of the hard drive prediction model is a continuous value representing the failure probability, and a numerical value representing the predicted remaining usage time.

[0064] Through the above process, the application of cross-attribute attention analysis in hard disk failure prediction enables the model to process multi-attribute data more intelligently, accurately predict failures, provide early warnings, avoid business interruptions and data loss caused by hard disk failures, and improve the stability and reliability of data storage systems.

[0065] In an exemplary embodiment, after training the target prediction model using a training dataset to obtain a hard disk prediction model, the method further includes: optimizing the hard disk prediction model using a second formula, wherein the second formula is: , where L BCE Let L be the binary cross-entropy loss function. RMSE Let λ be the root mean square error loss function, λ be the hyperparameter, P be the hard drive failure probability, R be the remaining lifespan of the hard drive, and Y be the root mean square error loss function. prob Y is a label for the probability of hard drive failure. rul This is a label indicating the remaining lifespan of the hard drive.

[0066] Optionally, collect data containing hard drive failure probability tags Y. prob and remaining useful life label Y rul The training dataset. During model training, a composite loss function L is defined. total The binary cross-entropy loss function L BCE Used to optimize fault probability prediction, while the root mean square error loss function L RMSE This is used to optimize RUL prediction. The value of λ determines the weight of the two loss functions in the total loss. The training dataset is input into the hard disk prediction model, and the model outputs the failure probability P and remaining useful life R. Then, L... BCE and L RMSE The two predicted values ​​are compared with their corresponding labels separately, and finally the two loss values ​​are combined according to the second formula to obtain the total loss L. total The model parameters are then adjusted by backpropagation until the model converges.

[0067] Through the above process and by using a composite loss function, the hard drive prediction model can not only predict hard drive failures but also accurately assess the healthy lifespan of the hard drive, thereby providing the maintenance team with a more comprehensive basis for decision-making and optimizing hard drive management strategies.

[0068] In an exemplary embodiment, after determining the failure probability information and remaining lifespan information of the target storage hard disk based on the analysis results, the method further includes: obtaining the failure probability value corresponding to the failure probability information and the lifespan value corresponding to the remaining lifespan information; performing a first comparison between the failure probability value and a preset first threshold to obtain a first comparison result; performing a second comparison between the remaining lifespan value and a preset second threshold to obtain a second comparison result; determining whether to trigger a first warning based on the first comparison result, and determining whether to trigger a second warning based on the second comparison result.

[0069] Optionally, based on historical data analysis, a first warning threshold for the probability of failure is set to 0.7, and a second warning threshold for the remaining lifespan is set to 30 days. These thresholds are based on the premise that hard drives typically begin to exhibit problems before the probability of failure reaches 0.7, and that the risk of hard drive instability increases sharply when the remaining lifespan is less than 30 days. The probability of failure (P) and the remaining lifespan (R) for a specific hard drive are obtained from the hard drive failure prediction model. P is compared with the first threshold of 0.7 to obtain a first comparison result; R is compared with the second threshold of 30 days to obtain a second comparison result. The first comparison result determines whether a failure warning should be triggered, and the second comparison result determines whether a lifespan warning should be triggered.

[0070] By following the above process and setting appropriate early warning thresholds, the health status of hard drives can be effectively monitored in real time, allowing for timely action to prevent data loss and ensure the continuous and stable operation of the data center. This early warning mechanism improves the security and reliability of data storage.

[0071] In one exemplary embodiment, determining whether to trigger a first warning based on a first comparison result includes: determining to trigger a first warning if the first comparison result indicates that the fault probability value is greater than a preset first threshold; and determining not to trigger a first warning if the first comparison result indicates that the fault probability value is less than or equal to the preset first threshold.

[0072] Optionally, a fault probability warning threshold can be set to 0.8, meaning a warning is triggered when the fault probability exceeds 80%. The hard drive fault probability value P is obtained from the model. The model output is typically a probability value between 0 and 1. The predicted fault probability value P is compared with the preset fault probability warning threshold. If the fault probability value is greater than the preset first threshold, a first warning is triggered. If the fault probability value is less than or equal to the preset first threshold, a first warning is not triggered, and the system continues monitoring without taking emergency action.

[0073] By setting precise failure probability warning thresholds through the above process, and triggering warnings when the predicted results exceed the thresholds, potential hard drive failures can be detected in a timely manner, and resources can be effectively avoided being wasted on low-risk situations.

[0074] In one exemplary embodiment, determining whether to trigger a second warning based on a second comparison result includes: determining to trigger a second warning if the second comparison result indicates that the remaining useful life value is less than a preset second threshold; and determining to trigger a second warning if the second comparison result indicates that the remaining useful life value is greater than or equal to the preset second threshold.

[0075] Optionally, a preset remaining lifespan threshold of 30 days is provided. This means that when the predicted remaining lifespan of the hard drive is less than 30 days, the system should trigger a second warning. The remaining lifespan value R is obtained from the hard drive failure prediction model; assuming R is 25 days, R is compared to the set remaining lifespan threshold. If R is less than the remaining lifespan threshold (25 days < 30 days in this example), the system determines to trigger the second warning. If R is greater than or equal to the remaining lifespan threshold (under normal circumstances), the system determines not to trigger the second warning, and the hard drive maintains normal monitoring status.

[0076] By following the above process, we can ensure that we take timely action when the remaining lifespan of the hard drive drops to a low level. This not only improves the reliability and availability of the system, but also achieves the goals of cost-effective and preventative maintenance.

[0077] To facilitate understanding of the implementation methods of this application, relevant scenarios are explained below, but these explanations do not limit the scope of this application.

[0078] In related technologies, the scarcity of hard drive failure instances in the overall data makes it difficult for machine learning-based prediction models to obtain sufficient information on failure modes, affecting the model's learning efficiency and prediction accuracy, especially when distinguishing the states of hard drives nearing failure. Secondly, hard drive failure is not a linear deterioration of a single attribute, but rather the result of nonlinear interactions among multiple attributes over a long period. This requires prediction models to effectively capture these complex cross-attribute relationships and the evolutionary characteristics of failures over time, areas where traditional time series analysis methods fall short.

[0079] To avoid the aforementioned problems, as an optional implementation, this application proposes a hard disk failure prediction method and system based on a fault trajectory-guided diffusion model and a cross-attribute attention Transformer. Specifically, it consists of two stages. The first stage constructs high-quality virtual fault data based on a fault trajectory-guided diffusion model. By analyzing the SMART time-series data of real faulty hard disks, a large number of virtual fault samples are generated using the diffusion model to address the scarcity of fault data in real-world scenarios. The second stage achieves high-precision fault prediction and remaining lifespan estimation based on a cross-attribute attention Transformer model. Building upon the balanced dataset constructed in the first stage, a cross-attribute attention mechanism is designed using the Transformer architecture to capture the complex nonlinear dependencies between hard disk SMART attributes. The model transposes the input data to allow the attention mechanism to operate along the attribute dimensions, thereby more accurately modeling the interactions between attributes. Combined with time-series information processing, it achieves accurate prediction of hard disk failure probability and reliable estimation of remaining lifespan.

[0080] Optionally, in practical application scenarios, the SMART time-series data X_real_fail of a real faulty hard drive has dimensions (N, W, F), where N is the number of samples, W is the time window length, and F is the number of attributes. X_real_fail is used as the input to the diffusion model. While maintaining the consistency of the fault feature distribution, the diffusion model generates more virtual fault data X_synthetic_fail with more diverse forms. The dimensions of X_synthetic_fail are (M, W, F), where M is the number of generated data.

[0081] Optional, a cross-attribute attention Transformer model architecture is used for hard drive failure prediction and remaining hard drive lifespan estimation based on SMART time-series data. This model is based on the balanced training dataset X constructed in the first stage. train (Dimensions are (B, W, F), where B is the batch size, W is the time window length, and F is the number of attributes) and the corresponding label Y label =(Y prob ,Y rul Training will be conducted. prob Y is a label for the probability of hard drive failure. rul This is the label for the remaining lifespan of the hard drive. After the model is trained, it can receive real-time SMART data (dimension (1,W,F)) of a single in-use hard drive and output the failure probability P and remaining lifespan R of that hard drive.

[0082] Optional, Figure 3This is a schematic diagram of an offline training process for a cross-attribute attention Transformer model according to an embodiment of this application, specifically including the following steps:

[0083] Step 1: Data Collection and Preprocessing. This includes the following steps:

[0084] Step 1: Collect a large amount of SMART raw data from the data center, mark known normal hard drives and faulty hard drives, and separate the data of real faulty hard drives and real normal hard drives from the SMART raw data.

[0085] Step 2: Filter the raw SMART data to identify attributes highly correlated with hard drive failures, such as SMART 5 (number of read error retries), SMART 187 (number of disk spin-start retries), SMART 188 (disk spin duration), SMART 197 (performance read error rate), and SMART 198 (performance write error rate). Normalize the filtered data to ensure that the data input into the model is on the same scale.

[0086] Step 3: Use the sliding window method to divide the entire lifecycle data of each hard drive into multiple time windows, with each window having a length of W days. This allows you to capture information about how the hard drive's health status changes over time.

[0087] Step Two: Training a Fault Trajectory-Guided Diffusion Model. A diffusion model is trained using the isolated real faulty hard drive data. This model can generate virtual fault data with similar trajectories to the real faulty hard drive data. Specifically, this includes the following steps:

[0088] Step 1: Forward Noise Addition Process. Forward noise addition is performed on the selected real faulty hard drive data, gradually adding noise to simulate the "blurred" state of the data until the data is completely transformed into noise. Specifically, multiple steps of forward noise addition are performed on the real faulty hard drive data x0 (dimension (W, F)), at each time step... Noisy data x is generated using the following formula. t :

[0089] ;

[0090] in, For noise sampled from a standard normal distribution, To preset the noise control coefficient, x t The data is noisy fault data, while x0 represents the actual faulty hard drive data.

[0091] Step 2: Conditional Embedding Vector Generation. Input the SMART data of the real failed hard drive into a conditional encoder (such as a GRU or Transformer network structure) to generate conditional embedding vectors. Specifically, select a real failed hard drive sample x. c As a prototype of the fault trajectory, it is input into the condition encoder E. cond The conditional embedding vector c is obtained as follows:

[0092] ;

[0093] Optionally, the aforementioned conditional embedding vectors will be used as guiding information in the subsequent denoising process to ensure that the generated data conforms to a specific fault mode.

[0094] Step 3: Reverse Denoising Process. In the reverse denoising process, the noise prediction network in the diffusion model... θ At each time step t, receive the current noise-adding fault data x. t The time-step encoding and the aforementioned conditional embedding vector c are used to output the prediction result for noise:

[0095] ;

[0096] Optionally, the original sample features can be gradually restored through the above reverse denoising process, and virtual fault data that conforms to the fault trajectory features can be generated under conditional constraints.

[0097] Optionally, the above noise prediction network θ The mean squared error loss function is used to minimize the difference between predicted noise and actual noise, thereby ensuring that the generated data is both realistic and diverse. The mean squared error loss function is as follows:

[0098] ;

[0099] Wherein, the expected value E is given by all training datasets, time step t, and sampled noise. Take the average.

[0100] Step 4: Construct a balanced dataset. Using the trained diffusion model, generate a large amount of virtual failure data (M times the number of real failure samples) based on real failure trajectories, ensuring that this data covers multiple dimensions and trends of the SMART attributes of the failed hard drive. Next, randomly select a number of data points from the real normal hard drive data that match the virtual failure data to form a normal dataset. Merge the virtual failure data and the normal dataset to form a balanced dataset where the number of failure data points and normal data points are approximately equal.

[0101] Step 5: Train the cross-attribute attention Transformer model. Design and train a cross-attribute attention Transformer model based on a balanced dataset. This model can capture the complex associations and long-term temporal dependencies between hard disk SMART attributes. Specifically, this includes the following steps:

[0102] Step 1: Input Dimension Transpose. To effectively utilize the cross-attribute attention mechanism, the input SMART attribute data is transposed, making the attribute dimensions the main dimensions of the sequence. This facilitates the model's learning of the interactions between different attributes, as shown below:

[0103] ;

[0104] Among them, X in For the input SMART attribute data, X transposed To balance the training dataset.

[0105] Optionally, a cross-attribute attention mechanism can be introduced on the attribute dimension F to capture the correlation between different SMART attributes. The specific calculation is as follows:

[0106] ;

[0107] ;

[0108] Among them, W Q W K W V Let Q be the learnable projection matrix, K be the query vector, V be the key vector, and d be the value vector. k is the dimension of the key vector.

[0109] Optional, W Q W K W V It can be used to transform input data into different representations of queries, keys, and values. Specifically, W Q W K W V Used for projection X transposed This generates query vectors, key vectors, and value vectors. These matrices are trained to learn the relationships between various SMART attributes of the hard drive, enabling the model to quantify the importance and mutual influence of different attributes, thereby improving the accuracy of hard drive failure prediction.

[0110] Optionally, after completing the cross-attribute feature modeling, this application further processes the time dimension W of each attribute through a feedforward neural network (FFN) or a one-dimensional convolutional layer to extract the temporal evolution features of the attribute and enhance the model's ability to represent the dynamic changes of the fault.

[0111] Step 2: Model Training. Train the model using a balanced training dataset, optimizing the loss function to minimize the fault prediction error (via L). BCE ) and remaining useful life prediction error (via L RMSE The specific formula is as follows:

[0112] ;

[0113] Among them, L BCE Let L be the binary cross-entropy loss function. RMSE Let Y be the root mean square error loss function, P be the hard drive failure probability, R be the remaining lifespan of the hard drive, and Y be the remaining lifespan of the hard drive. prob Y is a label for the probability of hard drive failure. rul λ is the label for the remaining lifespan of the hard drive, and λ is a hyperparameter used to balance the importance of classification and regression tasks.

[0114] Optional, Figure 4 This is a schematic diagram of an online inference process for a cross-attribute attention Transformer model according to an embodiment of this application, specifically including the following steps:

[0115] Step 1: Obtain recent SMART data. First, collect the SMART data sequence within the most recent time window W (e.g., 90 days) from a currently used hard drive. This step ensures that the model can make predictions based on the latest data.

[0116] Step Two: Data Input. The collected SMART data is input into the previously trained "Cross-Attribute Attention Transformer" model. This model has been trained offline and is capable of understanding the complex relationships between hard drive health indicators and failure evolution patterns.

[0117] Step 3: Output Prediction Results. After processing the input data, the model will output two key prediction values: the hard drive failure probability P and the remaining lifespan R. The failure probability P is used to determine whether the hard drive is about to fail; the remaining lifespan R is used to estimate the healthy lifespan of the hard drive.

[0118] Step 4: Triggering an Alert / Replacement Decision. The model's output failure probability and predicted remaining lifespan are compared to preset thresholds. If the predicted failure probability exceeds a certain threshold, or the remaining lifespan falls below a critical point, an alert will be automatically triggered, prompting maintenance personnel to take action on the hard drive, such as conducting further inspections or planning replacement.

[0119] In summary, this invention develops a fault trajectory-guided diffusion model. By learning the SMART timing characteristics of real faulty hard drives, it generates a large amount of highly realistic virtual fault data, enhancing the model's training effect. Secondly, it proposes a "cross-attribute attention Transformer" model. Based on hard drive SMART data, this model utilizes a cross-attribute attention mechanism to analyze and predict hard drive faults and remaining useful life (RUL), more effectively capturing the complex relationships between attributes, thereby improving the accuracy of fault prediction and the precision of RUL estimation.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0121] This embodiment also provides a fault prediction system for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0122] Figure 5 This is a structural block diagram of a hard disk failure prediction device according to an embodiment of this application, such as... Figure 5 As shown, the device includes:

[0123] The acquisition module 52 is used to acquire historical status data of multiple storage hard drives recorded in the server data center, wherein the historical status data includes: hard drive failure data and hard drive non-failure data;

[0124] Processing module 54 is used to perform trajectory guidance processing on the hard disk fault data through a target diffusion generation model to obtain virtual fault data; wherein, the target diffusion generation model is used to virtually expand the hard disk fault data through a first loss function and a conditional embedding mechanism;

[0125] The determination module 56 is used to determine a training dataset based on the virtual fault data and the historical state data, and to train the target prediction model using the training dataset to obtain a hard disk prediction model, wherein the hard disk prediction model is used to perform cross-attribute attention analysis on the hard disk state data.

[0126] The prediction module 58 is used to input the real-time hard disk status data of the target storage hard disk in the previous period into the hard disk prediction model when it is determined that a prediction is to be made for the target storage hard disk, and to predict the fault information of the target storage hard disk in the next period through the hard disk prediction model. The fault information includes: hard disk fault probability information and hard disk remaining lifespan information output by the hard disk prediction model.

[0127] The aforementioned device collects historical state data of storage hard drives from server data centers. This data includes not only known hard drive failure instances but also a large number of records of normal hard drive operation. Using a target diffusion generation model, guided by the "failure trajectories" in the real failure data, a series of virtual failure data are generated. This process optimizes the model's performance through a first loss function and combines a conditional embedding mechanism to ensure that the generated data maintains statistical consistency with the real failure data. The generated virtual failure data is merged with non-failure data from the original historical hard drive state data to construct a balanced training dataset. Based on this dataset, a target prediction model is trained to obtain a hard drive prediction model. This model can perform in-depth analysis of the real-time hard drive state data of the input target storage hard drive in the previous period, capture the complex dependencies between different hard drive attributes through a cross-attribute attention mechanism, and predict the failure information of the target storage hard drive in the next period, thereby achieving accurate prediction of hard drive failure probability and remaining lifespan. This technical solution solves the problem of low accuracy in hard drive failure prediction. Furthermore, by using a target diffusion generation model to create virtual fault data that matches the real fault trajectory, and merging it with normal operation records to construct a balanced training dataset, a prediction model with a cross-attribute attention mechanism was trained to achieve accurate prediction of hard drive failure probability and remaining service life.

[0128] In an exemplary embodiment, the above-mentioned processing module is further configured to extract feature codes corresponding to the hard disk attributes before and after the failure in the hard disk failure data, and convert the feature codes into conditional embedding vectors through a conditional encoder; add noise to the hard disk failure data to obtain noisy failure data; and input the noisy failure data, the conditional embedding vector, and the duration into the target diffusion generation model for denoising to obtain virtual failure data.

[0129] In an exemplary embodiment, the above-described processing module is further configured to perform a weighted mixing of the noise-added data and the hard disk failure data according to a first formula, wherein the first formula is: , Add noise to the data. To preset the noise control coefficient, x t The data is for adding noise to fault data, and x0 is for hard drive fault data.

[0130] In an exemplary embodiment, the above-described processing module is further configured to predict the noising fault data using a noise prediction sub-model in the target diffusion generation model to obtain a prediction result, wherein the noise prediction sub-model minimizes the difference between the noising fault data and the noising fault data in the prediction result by employing a first loss function, the first loss function being: E represents the expected value. Add noise to the data. θ For the noise prediction sub-model, x t For the noisy fault data, t is the time step encoding and c is the conditional embedding vector; the prediction results are denoised based on the added noise to the data to obtain virtual fault data.

[0131] In an exemplary embodiment, the determining module is further configured to transpose the dimensions of the training dataset using a preset transpose function, wherein the transpose process adjusts the dimensional order of the training dataset to determine the target dimensional order that is suitable for the training requirements of the target prediction model; determine the target training dataset based on the transpose process result, and use the target training dataset to train the target prediction model to obtain the hard disk prediction model.

[0132] In an exemplary embodiment, the determining module is further configured to determine the first dimension order of the training dataset, wherein the first dimension order is arranged in the order of batch size dimension, time dimension, and attribute dimension; and to use a preset transpose function to swap the relative positions of the time dimension and the attribute dimension in the first dimension order to obtain the second dimension order.

[0133] In an exemplary embodiment, the determining module further includes: a transformation unit, configured to, after determining the target training dataset based on the transpose processing result, use multiple weight matrices to transform the feature data of each hard disk attribute corresponding to the attribute dimension of the target training dataset to obtain multiple target vectors, wherein the multiple weight matrices include: a query matrix, a key matrix, and a value matrix; and determine the attention weights between different hard disk attributes based on the multiple target vectors.

[0134] In an exemplary embodiment, the above-mentioned conversion unit is further configured to use the query matrix in the multiple weight matrices to convert the feature data of each hard disk attribute to obtain a first target vector; use the key matrix in the multiple weight matrices to convert the feature data of each hard disk attribute to obtain a second target vector; and use the value matrix in the multiple weight matrices to convert the feature data of each hard disk attribute to obtain a third target vector.

[0135] In an exemplary embodiment, the aforementioned conversion unit is further configured to calculate the dot product of the first target vector and the second target vector among multiple target vectors to obtain an attention score; generate an attention score matrix when multiple attention scores corresponding to the training dataset are determined; normalize the attention score matrix and determine the attention weights between different hard disk attributes based on the numerical values ​​corresponding to each element in the normalization result.

[0136] In an exemplary embodiment, the above-mentioned conversion unit further includes: a determining subunit, configured to, after generating an attention score matrix based on multiple attention scores corresponding to the training dataset, determine the target attention weight in the attention score matrix corresponding to each of the multiple target vectors; and perform a weighted summation of the multiple target vectors based on the target attention weights to obtain a target attribute feature vector, wherein the target attribute feature vector is used to determine the correlation between different hard disk attributes.

[0137] In an exemplary embodiment, the aforementioned determining subunit further includes: a processing subunit, configured to perform weighted summation on multiple target vectors based on target attention weights to obtain a target attribute feature vector, and then perform multi-layer processing on the target attribute feature vector through a preset feedforward network to obtain a target comprehensive feature vector, wherein the preset feedforward network includes two fully connected layers, the first fully connected layer is used for nonlinear transformation of the target attribute feature vector, and the second fully connected layer re-integrates the nonlinearly transformed features into the same dimension as the target attribute feature vector.

[0138] In an exemplary embodiment, the prediction module is further configured to perform cross-attribute attention analysis on the real-time hard disk status data in the previous period using a hard disk prediction model to obtain analysis results; and determine the failure probability information and remaining lifespan information of the target storage hard disk based on the analysis results, wherein the hard disk failure probability information is used to determine the possibility of the target storage hard disk failing, and the remaining lifespan information is used to assess the healthy lifespan of the target storage hard disk.

[0139] In one exemplary embodiment, the apparatus further includes: an optimization module, configured to train a target prediction model using a training dataset to obtain a hard disk prediction model, and then optimize the hard disk prediction model using a second formula, wherein the second formula is: , where L BCE Let L be the binary cross-entropy loss function. RMSE Let λ be the root mean square error loss function, λ be the hyperparameter, P be the hard drive failure probability, R be the remaining lifespan of the hard drive, and Y be the root mean square error loss function. prob Y is a label for the probability of hard drive failure. rul This is a label indicating the remaining lifespan of the hard drive.

[0140] In an exemplary embodiment, the prediction module further includes: a comparison unit, configured to, after determining the failure probability information and remaining service life information of the target storage hard disk based on the analysis results, obtain the failure probability value corresponding to the failure probability information and the service life value corresponding to the remaining service life information; perform a first comparison between the failure probability value and a preset first threshold to obtain a first comparison result; perform a second comparison between the remaining service life value and a preset second threshold to obtain a second comparison result; determine whether to trigger a first warning based on the first comparison result, and determine whether to trigger a second warning based on the second comparison result.

[0141] In an exemplary embodiment, the comparison unit is further configured to determine whether to trigger a first warning if the first comparison result indicates that the fault probability value is greater than a preset first threshold; and to determine whether to not trigger a first warning if the first comparison result indicates that the fault probability value is less than or equal to the preset first threshold.

[0142] In an exemplary embodiment, the comparison unit is further configured to determine whether to trigger a second warning based on a second comparison result, including: determining to trigger a second warning when the second comparison result indicates that the remaining service life value is less than a preset second threshold; and determining to trigger a second warning when the second comparison result indicates that the remaining service life value is greater than or equal to the preset second threshold.

[0143] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0144] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0145] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0146] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0147] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0148] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0149] The embodiments described herein also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0150] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0151] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0153] The above provides a detailed description of a hard disk failure prediction method, apparatus, storage medium, and electronic device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for predicting hard disk failures, characterized in that, include: Obtain historical status data of multiple storage hard drives recorded in the server data center, wherein the historical status data includes: hard drive failure data and hard drive non-failure data; The hard disk fault data is processed by a target diffusion generation model to obtain virtual fault data; wherein, the target diffusion generation model is used to virtually expand the hard disk fault data through a first loss function and a conditional embedding mechanism. A training dataset is determined based on the virtual fault data and the historical state data, and the target prediction model is trained using the training dataset to obtain a hard disk prediction model. The hard disk prediction model is used to perform cross-attribute attention analysis on the hard disk state data. When it is determined that a target storage hard drive is to be predicted, the real-time hard drive status data of the target storage hard drive in the previous period is input into the hard drive prediction model. The hard drive prediction model predicts the fault information of the target storage hard drive in the next period. The fault information includes: hard drive fault probability information and hard drive remaining lifespan information output by the hard drive prediction model. The target diffusion generation model includes: adding noise to the hard disk fault data, generating a conditional embedding vector by combining a conditional encoder, and iteratively predicting and removing noise based on a first loss function through a noise prediction sub-model to generate virtual fault data consistent with the real fault trajectory. The hard disk prediction model includes: transposing the training dataset in dimensions, and using a query matrix, a key matrix, and a value matrix to perform linear transformations on the features of each attribute in the transposed data to obtain a query vector, a key vector, and a value vector; calculating the dot product of the query vector and the key vector, normalizing it to obtain attention weights between different attributes, and performing a weighted summation of the value vector based on the attention weights to generate a cross-attribute attention feature vector.

2. The method for predicting hard disk failures according to claim 1, characterized in that, The hard disk fault data is processed using a target diffusion generation model to obtain virtual fault data, including: Extract the feature codes corresponding to the hard disk attributes before and after the failure from the hard disk failure data, and convert the feature codes into conditional embedding vectors through a conditional encoder; The hard drive fault data is subjected to noise processing to obtain noisy fault data; The noisy fault data, the conditional embedding vector, and the duration are input into the target diffusion generation model for denoising processing to obtain the virtual fault data.

3. The method for predicting hard disk failures according to claim 2, characterized in that, Adding noise to the hard drive failure data includes: The data is weighted and mixed with noise and the hard drive failure data according to the first formula, wherein the first formula is: , Add noise to the data. To preset the noise control coefficient, x t x0 represents the noise-inducing fault data, and x0 represents the hard disk fault data.

4. The method for predicting hard disk failures according to claim 2, characterized in that, The noisy fault data, the conditional embedding vector, and the duration are input into the target diffusion generation model for denoising processing to obtain the virtual fault data, including: The noise prediction sub-model in the target diffusion generation model predicts the added noise fault data to obtain a prediction result. The noise prediction sub-model minimizes the difference between the noise fault data and the added noise fault data in the prediction result using a first loss function, which is: E represents the expected value. Add noise to the data. θ is the noise prediction sub-model, xt is the noisy fault data, t is the time step encoding, and c is the conditional embedding vector; The prediction results are denoised by adding noise to the data to obtain the virtual fault data.

5. The method for predicting hard disk failures according to claim 1, characterized in that, The target prediction model is trained using the training dataset to obtain a hard disk prediction model, including: The dimensions of the training dataset are transposed using a preset transpose function, wherein the transpose process adjusts the dimensional order of the training dataset to determine the target dimensional order that is suitable for the training requirements of the target prediction model. The target training dataset is determined based on the transpose processing result, and the target prediction model is trained using the target training dataset to obtain the hard disk prediction model.

6. The method for predicting hard disk failures according to claim 5, characterized in that, The dimensions of the training dataset are transposed using a preset transpose function, including: Determine the first dimension order of the training dataset, wherein the first dimension order is arranged in the order of batch size dimension, time dimension, and attribute dimension; The relative positions of the time dimension and the attribute dimension in the first dimensional order are swapped using the preset transpose function to obtain the second dimensional order.

7. The method for predicting hard disk failures according to claim 1, characterized in that, After generating a cross-attribute attention feature vector by weighted summation of the value vector based on the attention weights, the method further includes: The cross-attribute attention feature vector is processed in multiple layers by a preset feedforward network to obtain a target comprehensive feature vector. The preset feedforward network includes two fully connected layers. The first fully connected layer is used for nonlinear transformation of the cross-attribute attention feature vector, and the second fully connected layer reintegrates the nonlinearly transformed features into the same dimension as the cross-attribute attention feature vector.

8. The method for predicting hard disk failures according to claim 1, characterized in that, The real-time hard drive status data of the target storage hard drive in the previous period is input into the hard drive prediction model. The hard drive prediction model then predicts fault information for hard drive failures in the target storage hard drive in the next period, including: The hard disk prediction model is used to perform cross-attribute attention analysis on the real-time hard disk status data in the previous period to obtain the analysis results. Based on the analysis results, the failure probability information and remaining lifespan information of the target storage hard drive are determined. The hard drive failure probability information is used to determine the likelihood of the target storage hard drive failing, and the remaining lifespan information is used to assess the healthy lifespan of the target storage hard drive.

9. The method for predicting hard disk failures according to claim 1, characterized in that, After training the target prediction model using the training dataset to obtain the hard disk prediction model, the method further includes: The hard disk prediction model is optimized using a second formula, wherein the second formula is: , where L BCE Let L be the binary cross-entropy loss function. RMSE Let λ be the root mean square error loss function, λ be the hyperparameter, P be the hard drive failure probability, R be the remaining lifespan of the hard drive, and Y be the root mean square error loss function. prob Y is a label for the probability of hard drive failure. rul This is a label indicating the remaining lifespan of the hard drive.

10. The method for predicting hard disk failures according to claim 8, characterized in that, After determining the failure probability information and remaining lifespan information of the target storage hard drive based on the analysis results, the method further includes: Obtain the fault probability value corresponding to the fault probability information and the service life value corresponding to the remaining service life information; The fault probability value is compared with a preset first threshold to obtain a first comparison result; The remaining service life value is compared with a preset second threshold to obtain a second comparison result; Whether to trigger a first warning is determined based on the first comparison result, and whether to trigger a second warning is determined based on the second comparison result.

11. The method for predicting hard disk failures according to claim 10, characterized in that, Determining whether to trigger the first warning based on the first comparison result includes: If the first comparison result indicates that the fault probability value is greater than the preset first threshold, it is determined that a first warning will be triggered; If the first comparison result indicates that the fault probability value is less than or equal to the preset first threshold, it is determined that the first warning will not be triggered.

12. The method for predicting hard disk failures according to claim 10, characterized in that, Determining whether to trigger a second warning based on the second comparison result includes: If the second comparison result indicates that the remaining service life value is less than the preset second threshold, a second warning is triggered. If the second comparison result indicates that the remaining service life value is greater than or equal to the preset second threshold, a second warning is triggered.

13. A device for predicting hard disk failures, characterized in that, include: The acquisition module is used to acquire historical status data of multiple storage hard drives recorded in the server data center, wherein the historical status data includes: hard drive failure data and hard drive non-failure data; The processing module is used to perform trajectory guidance processing on the hard disk fault data through a target diffusion generation model to obtain virtual fault data; wherein, the target diffusion generation model is used to virtually expand the hard disk fault data through a first loss function and a conditional embedding mechanism; The determination module is used to determine a training dataset based on the virtual fault data and the historical state data, and to train the target prediction model using the training dataset to obtain a hard disk prediction model, wherein the hard disk prediction model is used to perform cross-attribute attention analysis on the hard disk state data. The prediction module is used to input the real-time hard disk status data of the target storage hard disk in the previous period into the hard disk prediction model when it is determined that a prediction is to be made for the target storage hard disk, and to predict the hard disk failure information of the target storage hard disk in the next period through the hard disk prediction model. The failure information includes: hard disk failure probability information and hard disk remaining lifespan information output by the hard disk prediction model. The target diffusion generation model includes: adding noise to the hard disk fault data, generating a conditional embedding vector by combining a conditional encoder, and iteratively predicting and removing noise based on a first loss function through a noise prediction sub-model to generate virtual fault data consistent with the real fault trajectory. The hard disk prediction model includes: transposing the training dataset in dimensions, and using a query matrix, a key matrix, and a value matrix to perform linear transformations on the features of each attribute in the transposed data to obtain a query vector, a key vector, and a value vector; calculating the dot product of the query vector and the key vector, normalizing it to obtain attention weights between different attributes, and performing a weighted summation of the value vector based on the attention weights to generate a cross-attribute attention feature vector.

14. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the method for predicting hard disk failure as described in any one of claims 1 to 12 when executing the computer program.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the hard disk failure prediction method as described in any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the hard disk failure prediction method as described in any one of claims 1 to 12.

Images