Hard disk fault prediction method, lightweight prediction model construction method and electronic equipment

Abstract

The invention discloses a hard disk fault prediction method, a lightweight prediction model construction method and electronic equipment, and relates to the technical field of servers, and the method comprises the steps: collecting sensing data sets of sensors corresponding to a plurality of hard disks, and carrying out the preprocessing of the sensing data sets; constructing a time space structure, and converting the preprocessed sensing data set into a multi-dimensional array; inputting the multi-dimensional array into a lightweight prediction model, and obtaining prediction states corresponding to the plurality of hard disks through a convolutional long-short-term memory network layer in the lightweight prediction model; the prediction layer in the lightweight prediction model processes the prediction states corresponding to the plurality of hard disks, and outputs the fault prediction results of the plurality of hard disks, thereby solving the technical problems of lack of edge autonomy and multi-parameter trend analysis capability, slow response to complex faults, untimely early warning and easy false report and missing report. And the technical effect of remarkably improving the early sensing capability and the local autonomy level of the server system on the hard disk fault is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of servers, and in particular to a hard disk failure prediction method, a lightweight prediction model construction method and an electronic device. BACKGROUND

[0002] Hard disk health monitoring mechanisms such as self-monitoring, analysis and reporting technology generally rely on static thresholds and centralized processing, which are difficult to effectively identify the evolution trend of parameters and the coupling relationship between multiple parameters, react slowly to sudden failures, and are prone to false positives or false negatives under complex working conditions, and cannot capture early "soft anomalies" and systematic degradation characteristics such as thermal interference and aging between multiple hard disks. At the same time, alarms and judgments are heavily dependent on host systems or central servers, lack autonomous capabilities on the edge side, and cannot achieve independent health assessment, trend modeling and proactive warning at the hard disk backplane level, resulting in high delay in failure response, strong system dependency, and problems such as prediction failure, poor timeliness, low stability and high network pressure when edge devices are offline, bandwidth is limited or central nodes fail.

[0003] Therefore, in view of the shortcomings of the prior art, the present application provides a hard disk failure prediction method. SUMMARY

[0004] The present application provides a hard disk failure prediction method, a lightweight prediction model construction method and an electronic device to at least solve the problem of slow response to complex failures, untimely warning and false positives and false negatives in related technologies due to reliance on static thresholds and centralized processing, lack of edge autonomy and multi-parameter trend analysis capabilities.

[0005] The present application provides a hard disk failure prediction method, which is suitable for a hard disk backplane connected to multiple hard disks, and the method comprises: collecting a sensor data set of sensors corresponding to the multiple hard disks, and preprocessing the sensor data set; constructing a time-space structure, converting the preprocessed sensor data set into a multi-dimensional array; inputting the multi-dimensional array into a lightweight prediction model, obtaining a prediction state corresponding to the multiple hard disks through a convolutional long short-term memory network layer in the lightweight prediction model; processing the prediction state corresponding to the multiple hard disks through a prediction layer in the lightweight prediction model, and outputting a failure prediction result of the multiple hard disks.

[0006] The application provides a lightweight prediction model construction method, which comprises the following steps: obtaining an initial prediction model according to a training data set; replacing standard convolution in a convolutional long short-term memory network layer in the initial prediction model with deep separable convolution to obtain a first convolutional long short-term memory network layer; training the first convolutional long short-term memory network layer according to a preset gating mechanism and a loss function, removing output channels with gating variables less than a preset value and fine-tuning to obtain a first prediction model; determining a quantization strategy according to an application scenario; quantizing the first prediction model according to the quantization strategy to obtain a second prediction model; and evaluating the second prediction model, and obtaining a lightweight prediction model in response to the second prediction model passing the evaluation.

[0007] The application also provides an electronic device, comprising a memory for storing a computer program and a processor for executing the computer program to implement the steps of any of the hard disk failure prediction methods.

[0008] The application also provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of any of the hard disk failure prediction methods.

[0009] The application also provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the steps of any of the hard disk failure prediction methods.

[0010] According to the application, the sensor data sets of the plurality of hard disks are collected, the sensor data sets are preprocessed, the time-space structure is constructed, the preprocessed sensor data sets are converted into multi-dimensional arrays, the multi-dimensional arrays are input into the lightweight prediction model, the prediction states of the plurality of hard disks are obtained through the convolutional long short-term memory network layer in the lightweight prediction model, and the prediction states of the plurality of hard disks are processed through the prediction layer in the lightweight prediction model to output the failure prediction results of the plurality of hard disks. Therefore, the lightweight prediction model is deployed on the hard disk backboard, the time-space joint modeling of the multi-channel sensor data and the intelligent prediction of the hard disk health trend are realized, the advanced perception ability and the local autonomy level of the server system to the hard disk failure are significantly improved, the system can independently run under abnormal conditions such as main system failure or network interruption, more fine-grained and higher-precision predictive maintenance is supported, the risk of data loss and the operation and maintenance cost are effectively reduced, and the stability, the intelligent degree and the safety performance of the system are comprehensively improved. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described below only illustrate some of the embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0012] Figure 1 A flowchart of a hard disk failure prediction method provided by an embodiment of the present application is shown in FIG. 2. Figure 2 A system architecture diagram of a hard disk failure prediction method provided by an embodiment of the present application is shown in FIG. 3. Figure 3 A flowchart of a prediction model data processing of a hard disk failure prediction method provided by an embodiment of the present application is shown in FIG. 4. Figure 4 A structural block diagram of a hard disk failure prediction device provided by an embodiment of the present application is shown in FIG. 5. Figure 5 An internal structure diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 6. DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort fall within the protection scope of the present application.

[0014] It should be noted that, in the description of the present application, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0015] It should be noted that the terms "S1", "S2" and the like are only used for the purpose of describing the steps, and do not mean to indicate the order or sequence, nor to limit the present application. They are only used to facilitate the description of the method of the present application, and cannot be understood as indicating the sequence of the steps. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the fact that the technical solutions can be realized by those skilled in the art. When the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the protection scope claimed by the present application.

[0016] In order to enable the person skilled in the art to better understand the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0017] The embodiment of the present application provides a hard disk fault prediction method, which is suitable for a hard disk backboard, the hard disk backboard is connected with a plurality of hard disks, and the hard disk fault prediction method is described in detail in combination with an execution flow of the method.

[0018] Here, the hard disk backboard is a key component for connecting a plurality of hard disks in a server or a storage device, and is often integrated with a SATA (Serial Advanced Technology Attachment), a SAS (Serial Attached SCSI), a PCIe (Peripheral Component Interconnect Express) and the like, and communicates with a BMC (Baseboard Management Controller) through an I2C (Inter-Integrated Circuit) / SMBus (System Management Bus) line to realize management functions such as hard disk temperature monitoring and state indication.

[0019] Specifically, the hardware of the hard disk backboard can include a plurality of hard disk interfaces (such as SATA / SAS / PCIe), a plurality of sensor modules (temperature, current, voltage, vibration and the like), a lightweight prediction model, a local storage unit (for storing recent sensor data and model parameters), a communication interface (I2C / SMBus, for communicating with the mainboard BMC), an indication / alarm unit (LED lamp, buzzer and the like) and a power management and protection circuit.

[0020] S101: Collecting a sensor data set of a plurality of hard disks corresponding to a sensor, and pre-processing the sensor data set.

[0021] Here, the sensor can include a temperature sensor, a Hall current sensor, a voltage sampling point and a vibration MEMS (Micro-Electro-Mechanical Systems) sensor. The temperature sensor can be an NTC (Negative Temperature Coefficient) thermistor or a digital thermometer (such as TMP75).

[0022] Each hard disk corresponds to one or more sensors for collecting different data.

[0023] The sensing data set includes different sensing data corresponding to a plurality of hard disks.

[0024] The preprocessing can include denoising, missing compensation, outlier rejection, normalization, etc.

[0025] Specifically, the acquisition module accesses the MCU module through an ADC (Analog-to-Digital Converter) or an I2C / SPI (Serial Peripheral Interface) mode.

[0026] In one embodiment, the sampling frequency can also be determined according to user configuration, for example, 1Hz~0.1Hz.

[0027] S102: Construct a time-space structure to convert the preprocessed sensing data set into a multi-dimensional array.

[0028] Here, the space structure can be a spatial layout corresponding to the physical layout of the hard disk backboard and the plurality of hard disks.

[0029] Here, the time structure can be a sliding window determined according to user requirements.

[0030] Each row in the multi-dimensional array represents a time point, and each column represents a sensing value of a hard disk.

[0031] Specifically, the sensing data set can be preprocessed by the MCU.

[0032] S103: Input the multi-dimensional array into a lightweight prediction model to obtain the predicted state of the plurality of hard disks through a convolutional long short-term memory network layer in the lightweight prediction model.

[0033] Here, ConvLSTM (Convolutional Long Short-Term Memory Network) is a deep learning model combining convolutional neural networks and long short-term memory networks.

[0034] The hidden state represents the state calculated by the ConvLSTM, which is an abstract representation formed after the network understands, filters and compresses all historical frames / snapshots (from t=0 to t) up to the current time t, captures the spatiotemporal dependence relationship (such as time series pattern and spatial correlation), and is used for subsequent prediction or passed to the next time step.

[0035] Here, lightweight refers to the process of reducing the size of a deep learning model, lowering its computational complexity, reducing memory usage and energy consumption, without significantly compromising its performance (such as accuracy), so that it is more suitable for deployment and operation on resource-constrained devices.

[0036] Among them, the lightweight method can include model pruning, knowledge distillation, parameter quantization, lightweight network design, low-rank decomposition, and model sharing and compression.

[0037] In one embodiment, the lightweight prediction model supports updating model weights or parameters through an I²C channel.

[0038] S104: Process the predicted states corresponding to the plurality of hard disks through the prediction layer in the lightweight prediction model, and output the failure prediction results of the plurality of hard disks.

[0039] Here, the failure prediction result can include the next time prediction value and / or health score of the plurality of hard disks.

[0040] In one embodiment, it can also include a gated heat map and channel contribution, combined with the failure prediction result, to locate the abnormal hard disk.

[0041] Specifically, the prediction model outputs the health score and trend label of the plurality of hard disks every preset time period (such as ten minutes), triggers the prompt of the indication unit according to the health score (MCU controls the indication unit to display green / orange / red corresponding to the health, warning, and failure states), and reports the health status, current score, and abnormal log to the server motherboard through the SMBus.

[0042] In one embodiment, the hard disk backplane can communicate with the server motherboard or BMC system in real time through buses such as I2C and SMBus, realize the uploading of backplane prediction results, synchronization of abnormal records, management of state information, and other functions. Support multi-backplane device address configuration and dynamic recognition mechanism, suitable for batch deployment in high-density server scenarios. Among them, it can support multi-backplane communication address recognition in EEPROM address mapping mode.

[0043] It should be noted that the present application deploys a lightweight prediction model on the hard disk backplane, realizes spatio-temporal joint modeling of multi-channel sensing data and intelligent prediction of hard disk health trends, significantly improves the early perception ability and local autonomy level of the server system for hard disk failure, not only can run independently in the case of main system failure or network interruption, but also supports more fine-grained and high-precision predictive maintenance, effectively reduces the risk of data loss and operation and maintenance cost, and improves the stability, intelligence level and security performance of the system.

[0044] In some embodiments, a plurality of sensor data sets corresponding to sensors of a plurality of hard disks are collected, and the sensor data sets are preprocessed, including: According to a preset data range, an abnormal value in the sensor data set is determined, the abnormal value is converted into a missing value, and a first sensor data set is obtained; According to a length of the missing segment, the missing value in the first sensor data set is processed, and a second sensor data set is obtained; The second sensor data set is denoised by three-point median filtering combined with exponential moving average, and a third sensor data set is obtained; The third sensor data set is normalized to obtain a preprocessed sensor data set.

[0045] Here, the three-point median filtering is a nonlinear digital filtering technique used to filter out sharp, bursty impulse noise and outliers in signals.

[0046] Here, the exponential moving average is a data processing method commonly used for time series smoothing, which performs a weighted average of historical data, so that recent data has a larger weight and far-off data has an exponentially decaying weight.

[0047] For example, the preset data range can include ranges corresponding to a plurality of data, such as temperature ranges and current ranges, and the temperature range can be -40℃-125℃.

[0048] Here, normalization can include standardization z=(x-μ) / σ or robust standardization.

[0049] Specifically, three-point median filtering is performed on each point in the second sensor data set to obtain a median result, the median result is taken as input, exponential moving average is performed, and a denoising output value is obtained.

[0050] In this way, the original data is converted into high-quality and high-credibility analysis base data.

[0051] In some embodiments, according to the length of the missing segment, the missing value in the first sensor data set is processed to obtain the second sensor data set, including: In response to the length of the missing segment being less than or equal to a preset length, the missing value in the missing segment is filled by linear interpolation; In response to the length of the missing segment being greater than the preset length, the mask of the missing value in the missing segment is set to zero.

[0052] Here, the preset length can be three consecutive points, and less than or equal to three consecutive points is classified as a short missing, and greater than three consecutive points is classified as a long missing. Specifically, for a short missing, according to the boundary valid point, the missing value is filled by linear interpolation, and the mask is updated to be valid; for a long missing, the data is set to zero, and the mask is kept invalid.

[0053] In one embodiment, when the missing segment is located at the beginning of the data sequence: find the first valid data point in the sequence, fill all the starting boundary missing points with the valid value, and keep 0 if the entire sequence is invalid; when the missing segment is located at the end of the data sequence: find the last valid data point in the sequence, fill all the ending boundary missing points with the valid value, and keep 0 if the entire sequence is invalid.

[0054] In this way, the data authenticity can be maintained, and false data trends introduced by excessive interpolation can be avoided.

[0055] In some specific embodiments, a time-space structure is constructed, and the preprocessed sensor data set is converted into a multi-dimensional array, including: According to the layout of the hard disk backplane and the plurality of hard disks, a two-dimensional space grid is constructed; According to the two-dimensional space grid, an index mapping table is created; According to the layout of the plurality of hard disks, a vacancy mask is determined; Obtain the sensor data corresponding to a plurality of time points, map the sensor data corresponding to the plurality of time points on the two-dimensional space grid according to the index mapping table and the vacancy mask, and arrange according to the time sequence to obtain an image sequence; The image sequence is intercepted by a sliding window to obtain a multi-dimensional array.

[0056] Here, the sliding window analyzes or extracts features from local data by moving a fixed-length window frame by frame on a sequence.

[0057] Wherein, according to user demand, the step size and window size of the sliding window are determined.

[0058] Wherein, the two-dimensional space grid can be a 3*4 grid.

[0059] Specifically, the multi-dimensional array can be [T, H, W, C], where C is the number of sensor channels (for example: temperature + current = 2), T is the time window length (for example: 72 time points), H and W are consistent with the two-dimensional space grid of the actual backplane physical layout.

[0060] Specifically, the plurality of hard disk slots are mapped to a two-dimensional grid (such as 3*4) consistent with the physical layout of the backplane to generate an index mapping table and a vacancy mask, and then divided according to the sliding window to obtain a multi-dimensional array.

[0061] For example, assuming that sampling is performed every 5 minutes, the last 6 hours (72 points) are used to constitute an input multi-dimensional array, the backplane has a layout of three rows and four columns, and temperature and current information is collected, then the multi-dimensional array is: [72, 3, 4, 2].

[0062] In this way, the spatial and temporal dependencies of the sensor data can be preserved.

[0063] In some embodiments, the predicted state corresponding to the plurality of hard disks is obtained by a light-weight convolutional long short-term memory network layer in the prediction model, including: The multi-dimensional array, the hidden state of the previous time step, and the cell state of the previous time step are taken as inputs, and a first deep separable convolution and a second deep separable convolution are performed to obtain an input gate and a forget gate. The multi-dimensional array, the hidden state of the previous time step, the input gate, and the forget gate are taken as inputs, and a third deep separable convolution is performed to calculate a candidate cell state and an updated cell state. The multi-dimensional array, the hidden state of the previous time step, and the updated cell state are taken as inputs, and a fourth deep separable convolution is performed to obtain an output gate. The updated cell state and the output gate are taken as inputs, and an activation function is performed to obtain the predicted state corresponding to the plurality of hard disks.

[0064] The deep separable convolution ensures that the gated memory is recursively in time and can capture long-term / short-term trends (such as temperature rise slope, periodic vibration).

[0065] The number of input channels is determined by the number of sensor data of each hard disk, such as current and temperature, and the number of hidden channels is determined by the computing power.

[0066] For example, the light-weight convolutional long short-term memory network layer processing process is as follows: ; ; ; ; ; In the above formula, σ represents a sigmoid activation function, tanh(·) represents a hyperbolic tangent activation function, X t represents the input of the current time step, H t-1 represents the hidden state of the previous time step, C t-1 represents the memory unit of the previous time step, f t , i t , C t , o t respectively represent the forget gate, the input gate, the memory unit, and the output gate, represents a Hadamard product.

[0067] Among them, W in the gate is the local convolution kernel (such as 3x3), H t and C t They are all spatial feature maps, which share and expand the receptive field of adjacent hard disks. The processing process of the convolutional long short-term memory network layer includes local neighborhood (3×3 / 5×5 / void convolution), which can encode spatial dependencies such as thermal coupling and current linkage between hard disks. At the same time, the memory element C t Cross-time accumulation, forget gate f t AND input gate i t Adaptively retaining long-term information or introducing new evidence can capture slow warming, periodic fluctuations or sudden jumps.

[0068] For example, when a hard disk at (2,3) is identified and its temperature slowly rises due to blocked air duct, the gate corresponding to the current hard disk will spread the abnormal signal to the neighboring gates, indicating a potential "heat island".

[0069] In this way, the temporal change trend of the hard disk operating status and the spatial correlation characteristics between different hard disks can be effectively identified, significantly improving the accuracy and advance time of fault prediction.

[0070] In some specific embodiments, the convolutional long short-term memory network layer includes a first convolutional long short-term memory network layer and a second convolutional long short-term memory network layer, and the convolutional long short-term memory network layer in the lightweight prediction model is used to obtain the predicted states corresponding to the multiple hard disks, further comprising: According to the multidimensional array, first hidden states corresponding to the multiple hard disks are obtained through the first convolutional long short-term memory network layer; According to the first hidden state, the predicted states corresponding to multiple hard disks are obtained through the second convolutional long short-term memory network layer.

[0071] The prediction model includes a first convolutional long short-term memory network layer, a second convolutional long short-term memory network layer and a prediction layer.

[0072] In this way, by stacking convolutional long short-term memory network layers, the receptive field can be expanded.

[0073] In some specific implementations, the prediction layer in the lightweight prediction model processes the prediction states corresponding to the multiple hard disks and outputs the failure prediction results of the multiple hard disks, including: Obtain health scores for multiple hard drives based on their corresponding predicted status, historical data, and current data. Output failure prediction results of multiple hard disks based on their health scores.

[0074] Among them, the health score ranges from 0 to 100 and is graded according to the threshold. A score greater than or equal to 80 is classified as healthy, a score greater than or equal to 60 and less than 80 is classified as sub-healthy, a score greater than or equal to 40 and less than 60 is classified as warning, and a score less than 40 is classified as fault.

[0075] Specifically, the variance of the prediction sequence is obtained based on the prediction state, the historical slope is obtained based on the historical data, and the prediction error is obtained based on the current data. Based on the variance, historical slope and prediction error of the prediction sequence, the abnormality of the hard disk is calculated, the abnormality is normalized, and the health score is obtained.

[0076] In one embodiment, when the health score of the current hard disk is in a warning state, the adjacent hard disks of the current hard disk are identified. When it is determined that the abnormality of the current hard disk affects the health scores of the adjacent hard disks, the health score of the current hard disk is lowered and the hard disk enters a warning state.

[0077] For example, assume that the system is configured with 12 hard drives. The monitoring points include temperature and current, with sampling once per minute. The MCU runs the inference process every 10 minutes to score all hard drives. If a sudden temperature rise and unstable current trend are detected on hard drives 3 and 7, the system will downgrade their scores to "sub-healthy", the slot LED will turn orange, and the BMC records will be uploaded for system operation and maintenance analysis.

[0078] In this way, a health scoring mechanism based on trend judgment is implemented, which supports early warning and graded prompts, provides a proactive and predictable maintenance path for the server, and significantly improves system availability and fault prevention capabilities.

[0079] In some specific implementations, based on the predicted states, historical data, and current data corresponding to the multiple hard disks, an expression for obtaining the health scores of the multiple hard disks is specifically as follows: ; in, is the absolute error between the predicted value and the actual value at time t, α is the weight coefficient, is the predicted value sequence for the next K time steps, is the variance of the predicted sequence, β is the weight coefficient, is the actual observation value at the past k time points, is the slope obtained after linear fitting of the historical series, and γ is the weight coefficient.

[0080] Specifically, is the prediction error term, α is the weight coefficient, which controls the importance of this term; is the forecast uncertainty term, β is the weight coefficient, which controls the impact of future forecast fluctuations; For the historical trend item, γ is a weight coefficient, and controls the influence of the trend item.

[0081] In some embodiments, the method further comprises: According to the failure prediction results of the plurality of hard disks, a failure trend and an influence range are determined. According to the failure trend and the influence range, a control operation is dynamically adjusted.

[0082] The control operation can include a fan, data migration, or a dispatching strategy, etc.

[0083] In this way, predictive maintenance is achieved.

[0084] Embodiments of the present application also provide a method for constructing a lightweight prediction model. The method is described in detail in combination with the execution process of the method for constructing a lightweight prediction model.

[0085] In some embodiments, the method comprises: According to the training data set, an initial prediction model is obtained. The standard convolution in the convolutional long short-term memory network layer in the initial prediction model is replaced by a depth separable convolution to obtain a first convolutional long short-term memory network layer. According to a preset gating mechanism and a loss function, the first convolutional long short-term memory network layer is trained, the output channels with a gating variable less than a preset value are removed and fine-tuned to obtain a first prediction model. According to an application scenario, a quantization strategy is determined. According to the quantization strategy, the first prediction model is quantized to obtain a second prediction model. The second prediction model is evaluated, and in response to the second prediction model passing the evaluation, a lightweight prediction model is obtained.

[0086] Here, the depth separable convolution significantly reduces the amount of calculation and model parameters while maintaining the similar performance of the standard convolution. The depth separable convolution decomposes the standard convolution into two steps, depth convolution and point-wise convolution.

[0087] Here, the gating mechanism is to add a trainable gating variable g (scalar) to each output channel in the first convolutional long short-term memory network layer.

[0088] Here, the loss function is the loss function of the original task.

[0089] Specifically, the first convolutional long short-term memory network layer is trained by a preset gating mechanism and a loss function. During the training process, the gating variables are optimized, and most of the gating variables tend to be close to 0, indicating that the corresponding channels are not important. After the training is completed, all channels are sorted according to the absolute value of the gating variable, a pruning threshold or a target pruning rate (such as pruning 30% of the smallest g channels) is set, and the convolution kernel weight corresponding to the channel whose gating variable is lower than the threshold and the corresponding channel dimension in the input of the next layer are removed.

[0090] Here, the quantization strategy can include post-training quantization and quantization-aware training. Among them, the post-training quantization includes only weight quantization and full integer quantization.

[0091] Among them, the application scenarios can include model accuracy sensitivity, hardware acceleration demand, development time cost, data preparation difficulty, model structure complexity and output layer protection demand, etc.

[0092] Specifically, the parameters in the application scenarios constitute a decision matrix, and the quantization strategy is determined according to the decision matrix.

[0093] Specifically, the quantization-aware training includes: inserting a Fake Quantization node (simulating quantization error) in the pruned model, fine-tuning using original training data (or part), small learning rate, and after training, converting the FakeQuant node to a real int8 operation to generate a lightweight prediction model.

[0094] Among them, the evaluation can include evaluating the accuracy (such as Top-1 / Top-5 accuracy, mAP) of the model, the parameter quantity, the calculation quantity (FLOPs) and the inference speed, etc.

[0095] In one embodiment, the prediction layer in the lightweight prediction model includes a prediction head, and different hard disks are predicted through the same prediction head.

[0096] In one embodiment, a lightweight channel attention (such as a Squeeze-and-Excitation module) or a spatial attention mechanism can be introduced, combined with the convolutional long short-term memory network layer, so that the prediction model can focus on more important features with fewer channels and parameters, thereby reducing the total parameter quantity while maintaining performance.

[0097] In this way, the lightweight prediction model can adapt to resource-constrained chip environments, support local storage, circular buffer, power-off protection and other mechanisms to ensure the continuity and safety of model inference, and balance accuracy and efficiency.

[0098] In one embodiment, Figure 2 The system architecture in the embodiments of the present application is shown in the following figure: Figure 2As shown, the system architecture in the present application includes: hard disk slot group HDD1-HDDn, sensor acquisition module, prediction module, local buffer storage, I2C communication interface, indicator light / buzzer control unit.

[0099] Specifically, the sensor acquisition module is used to collect multi-point temperature, current and vibration data.

[0100] Specifically, the prediction module is used for data preprocessing, model inference and scoring, and risk level judgment.

[0101] Specifically, the local buffer storage is used for ring storage of N hours of raw data.

[0102] Specifically, the I2C communication interface is used to connect the BMC.

[0103] In one embodiment, Figure 3 The flowchart of the prediction model data processing in the embodiment of the present application is as shown in Figure 3 As shown, the flow of the prediction model data processing in the present application includes: S1: sensor data sampling (multi-channel); S2: data normalization and windowing (time series formatting); S3: constructing tensor (i.e. multi-dimensional array) input lightweight prediction model; S4: model inference; S5: output score (health degree 0-100); S6: determine alarm level; S7: local control feedback and state reporting.

[0104] It should be understood that, although Figures 1-3 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figures 1-3 At least part of the steps in

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0106] The embodiment of the application also provides a hard disk fault prediction device, which is suitable for a hard disk backboard connected with a plurality of hard disks, and comprises: a first processing module 401 configured to collect a sensor data set of sensors corresponding to the plurality of hard disks, and pre-process the sensor data set; a second processing module 402 configured to construct a time-space structure, and convert the pre-processed sensor data set into a multi-dimensional array; a third processing module 403 configured to input the multi-dimensional array into a lightweight prediction model, and obtain a prediction state corresponding to the plurality of hard disks through a convolutional long short-term memory network layer in the lightweight prediction model; and a fourth processing module 404 configured to process the prediction state corresponding to the plurality of hard disks through a prediction layer in the lightweight prediction model, and output a fault prediction result of the plurality of hard disks.

[0107] As a preferred implementation, in the embodiment of the application, the first processing module 401 is specifically configured to: determine an abnormal value in the sensor data set according to a preset data range, convert the abnormal value into a missing value, and obtain a first sensor data set; process the missing value in the first sensor data set according to the length of the missing segment, and obtain a second sensor data set; perform denoising processing on the second sensor data set through three-point median filtering combined with exponential moving average, and obtain a third sensor data set; and perform normalization on the third sensor data set, and obtain the pre-processed sensor data set.

[0108] As a preferred implementation, in the embodiment of the application, the first processing module 401 is specifically further configured to: in response to the length of the missing segment being less than or equal to a preset length, fill the missing value in the missing segment through linear interpolation; and in response to the length of the missing segment being greater than the preset length, set the mask of the missing value in the missing segment to zero.

[0109] As a preferred implementation, in the embodiment of the application, the second processing module 402 is specifically configured to: construct a two-dimensional space grid according to the layout of the hard disk backboard and the plurality of hard disks; create an index mapping table according to the two-dimensional space grid; determine a vacancy mask according to the layout of the plurality of hard disks; obtain sensor data corresponding to a plurality of time points, map the sensor data corresponding to the plurality of time points on the two-dimensional space grid according to the index mapping table and the vacancy mask, arrange according to time sequence, and obtain an image sequence; and obtain a multi-dimensional array by sliding window interception of the image sequence.

[0110] As a preferred implementation, in the embodiment of the present application, the third processing module 403 is specifically configured to: take the multi-dimensional array, the hidden state of the previous time step and the cell state of the previous time step as inputs, obtain the input gate and the forget gate through the first deep separable convolution and the second deep separable convolution; take the multi-dimensional array, the hidden state of the previous time step, the input gate and the forget gate as inputs, calculate the candidate cell state and the updated cell state through the third deep separable convolution; take the multi-dimensional array, the hidden state of the previous time step and the updated cell state as inputs, obtain the output gate through the fourth deep separable convolution; take the updated cell state and the output gate as inputs, obtain the predicted state corresponding to the plurality of hard disks through the activation function.

[0111] As a preferred implementation, in the embodiment of the present application, the convolutional long short-term memory network layer includes a first convolutional long short-term memory network layer and a second convolutional long short-term memory network layer, and the third processing module 403 is specifically further configured to: obtain the first hidden state corresponding to the plurality of hard disks through the first convolutional long short-term memory network layer according to the multi-dimensional array; obtain the predicted state corresponding to the plurality of hard disks through the second convolutional long short-term memory network layer according to the first hidden state.

[0112] As a preferred implementation, in the embodiment of the present application, the fourth processing module 404 is specifically configured to: obtain the health scores of the plurality of hard disks according to the predicted state corresponding to the plurality of hard disks, the historical data and the current data; and output the failure prediction result of the plurality of hard disks according to the health scores of the plurality of hard disks.

[0113] As a preferred implementation, in the embodiment of the present application, the fourth processing module is specifically further configured to: ; wherein, is the absolute error between the predicted value and the actual value at time t, and a is a weight coefficient, is the predicted value sequence for K future time steps, is the variance of the predicted sequence, and β is a weight coefficient, is the actual observation value at the past k time points, is the slope obtained after linear fitting of the historical sequence, and γ is a weight coefficient.

[0114] The embodiment of the application further provides a hard disk failure prediction device, the device comprising: a first construction module, configured to obtain an initial prediction model according to a training data set; a second construction module, configured to replace standard convolution in a convolutional long short-term memory network layer in the initial prediction model with a depth separable convolution to obtain a first convolutional long short-term memory network layer; a third construction module, configured to train the first convolutional long short-term memory network layer according to a preset gating mechanism and a loss function, remove an output channel with a gating variable less than a preset value and fine-tune to obtain a first prediction model; a fourth construction module, configured to determine a quantization strategy according to an application scenario; a fifth construction module, configured to quantize the first prediction model according to the quantization strategy to obtain a second prediction model; and a sixth construction module, configured to evaluate the second prediction model, and obtain a lightweight prediction model in response to the second prediction model passing the evaluation.

[0115] The features of the embodiment of the hard disk failure prediction device can be understood with reference to the related description of the embodiment of the hard disk failure prediction method, which will not be repeated here.

[0116] The embodiment of the application further provides an electronic device, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 5 The electronic device comprises a processor, a memory, a network interface, a display screen and an input device connected through a system bus. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the electronic device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a hard disk failure prediction method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0117] Those skilled in the art can understand that Figure 5 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the application, and does not constitute a limitation on the electronic device to which the scheme of the application is applied. The specific electronic device can comprise more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0118] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implements the following steps when executing the computer program: S1: collecting a sensor data set of a plurality of hard disks corresponding sensors, and preprocessing the sensor data set; S2: constructing a time-space structure, and converting the preprocessed sensor data set into a multi-dimensional array; S3: inputting the multi-dimensional array into a lightweight prediction model, and obtaining a plurality of hard disk corresponding predicted states through a convolutional long short-term memory network layer in the lightweight prediction model; and S4: processing the plurality of hard disk corresponding predicted states through a prediction layer in the lightweight prediction model, and outputting a plurality of hard disk failure prediction results.

[0119] In one embodiment, the processor further implements the following steps when executing the computer program: determining an abnormal value in the sensor data set according to a preset data range, converting the abnormal value into a missing value, and obtaining a first sensor data set; processing the missing value in the first sensor data set according to a length of a missing segment, and obtaining a second sensor data set; performing denoising processing on the second sensor data set through three-point median filtering combined with exponential moving average, and obtaining a third sensor data set; and normalizing the third sensor data set, and obtaining the preprocessed sensor data set.

[0120] In one embodiment, the processor further implements the following steps when executing the computer program: in response to the length of the missing segment being less than or equal to a preset length, filling the missing value in the missing segment through linear interpolation; and in response to the length of the missing segment being greater than the preset length, setting a mask of the missing value in the missing segment to zero.

[0121] In one embodiment, the processor further implements the following steps when executing the computer program: constructing a two-dimensional space grid according to a layout of a hard disk backboard and a plurality of hard disks; creating an index mapping table according to the two-dimensional space grid; determining a vacancy mask according to the layout of the plurality of hard disks; obtaining sensor data corresponding to a plurality of time points, mapping the sensor data corresponding to the plurality of time points on the two-dimensional space grid according to the index mapping table and the vacancy mask, and arranging according to time sequence to obtain an image sequence; and obtaining the multi-dimensional array by sliding window intercepting the image sequence.

[0122] In one embodiment, the processor, when executing the computer program, further implements the following steps: taking the multi-dimensional array, the hidden state of the last time step and the cell state of the last time step as inputs, obtaining the input gate and the forget gate through the first deep separable convolution and the second deep separable convolution; taking the multi-dimensional array, the hidden state of the last time step, the input gate and the forget gate as inputs, calculating the candidate cell state and the updated cell state through the third deep separable convolution; taking the multi-dimensional array, the hidden state of the last time step and the updated cell state as inputs, obtaining the output gate through the fourth deep separable convolution; taking the updated cell state and the output gate as inputs, obtaining the predicted state corresponding to the plurality of hard disks through the activation function.

[0123] In one embodiment, the processor, when executing the computer program, further implements the following steps: obtaining the first hidden state corresponding to the plurality of hard disks through the first convolutional long short-term memory network layer according to the multi-dimensional array; obtaining the predicted state corresponding to the plurality of hard disks through the second convolutional long short-term memory network layer according to the first hidden state.

[0124] In one embodiment, the processor, when executing the computer program, further implements the following steps: obtaining the health scores of the plurality of hard disks according to the predicted state corresponding to the plurality of hard disks, the historical data and the current data; outputting the failure prediction results of the plurality of hard disks according to the health scores of the plurality of hard disks.

[0125] In one embodiment, the processor, when executing the computer program, further implements the following steps: ; wherein, is the absolute error between the predicted value and the actual value at time t, α is a weight coefficient, is the predicted value sequence for the future K time steps, is the variance of the predicted sequence, β is a weight coefficient, is the actual observation value at the past k time points, is the slope obtained after linear fitting of the historical sequence, γ is a weight coefficient.

[0126] In one embodiment, a computer device is also provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: S1: obtaining an initial prediction model according to a training data set; S2: replacing a standard convolution in a convolutional long short-term memory network layer in the initial prediction model with a depth separable convolution to obtain a first convolutional long short-term memory network layer; S3: training the first convolutional long short-term memory network layer according to a preset gating mechanism and a loss function, removing an output channel with a gating variable less than a preset value and fine-tuning to obtain a first prediction model; S4: determining a quantization strategy according to an application scenario; S5: quantizing the first prediction model according to the quantization strategy to obtain a second prediction model; and S6: evaluating the second prediction model, and obtaining a lightweight prediction model in response to the second prediction model passing the evaluation.

[0127] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program, the computer program being executable by a processor to implement the following steps: S1: collecting a sensor data set of sensors corresponding to a plurality of hard disks, and preprocessing the sensor data set; S2: constructing a time-space structure, and converting the preprocessed sensor data set into a multi-dimensional array; S3: inputting the multi-dimensional array into a lightweight prediction model, and obtaining a predicted state of the plurality of hard disks through a convolutional long short-term memory network layer in the lightweight prediction model; and S4: processing the predicted state of the plurality of hard disks through a prediction layer in the lightweight prediction model, and outputting a failure prediction result of the plurality of hard disks.

[0128] In one embodiment, the computer program is further executable by the processor to implement the following steps: determining an outlier in the sensor data set according to a preset data range, and converting the outlier into a missing value to obtain a first sensor data set; processing the missing value in the first sensor data set according to a length of a missing segment to obtain a second sensor data set; performing denoising processing on the second sensor data set through three-point median filtering combined with exponential moving average to obtain a third sensor data set; and normalizing the third sensor data set to obtain the preprocessed sensor data set.

[0129] In one embodiment, the computer program is further executable by the processor to implement the following steps: in response to the length of the missing segment being less than or equal to a preset length, filling the missing value in the missing segment through linear interpolation; and in response to the length of the missing segment being greater than the preset length, setting a mask of the missing value in the missing segment to zero.

[0130] In one embodiment, the computer program is further implemented when executed by the processor to perform the following steps: constructing a two-dimensional spatial grid according to the layout of the hard disk backplane and the plurality of hard disks; creating an index mapping table according to the two-dimensional spatial grid; determining a vacancy mask according to the layout of the plurality of hard disks; obtaining sensing data corresponding to a plurality of time points, mapping the sensing data corresponding to the plurality of time points on the two-dimensional spatial grid according to the index mapping table and the vacancy mask, and obtaining an image sequence according to the time sequence arrangement; and obtaining a multi-dimensional array by sliding window interception of the image sequence.

[0131] In one embodiment, the computer program is further implemented when executed by the processor to perform the following steps: taking the multi-dimensional array, the hidden state of the previous time step, and the cell state of the previous time step as inputs, obtaining an input gate and a forget gate through the first deep separable convolution and the second deep separable convolution; taking the multi-dimensional array, the hidden state of the previous time step, the input gate, and the forget gate as inputs, calculating a candidate cell state and an updated cell state through the third deep separable convolution; taking the multi-dimensional array, the hidden state of the previous time step, and the updated cell state as inputs, obtaining an output gate through the fourth deep separable convolution; and taking the updated cell state and the output gate as inputs, obtaining the predicted state corresponding to the plurality of hard disks through the activation function.

[0132] In one embodiment, the computer program is further implemented when executed by the processor to perform the following steps: obtaining the first hidden state corresponding to the plurality of hard disks through the first convolutional long short-term memory network layer according to the multi-dimensional array; and obtaining the predicted state corresponding to the plurality of hard disks through the second convolutional long short-term memory network layer according to the first hidden state.

[0133] In one embodiment, the computer program is further implemented when executed by the processor to perform the following steps: obtaining the health scores of the plurality of hard disks according to the predicted state corresponding to the plurality of hard disks, historical data, and current data; and outputting the failure prediction results of the plurality of hard disks according to the health scores of the plurality of hard disks.

[0134] In one embodiment, the computer program is further implemented when executed by the processor to perform the following steps: ; wherein, is the absolute error between the predicted value and the actual value at time t, and a is a weight coefficient, is the predicted value sequence for the future K time steps, is the variance of the predicted sequence, and β is a weight coefficient, is the actual observation value at the past k time points, is the slope obtained after linear fitting of the historical sequence, and γ is a weight coefficient.

[0135] In one embodiment, a computer readable storage medium is also provided, and the computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the following steps: S1: obtaining an initial prediction model according to a training data set; S2: replacing a standard convolution in a convolutional long short-term memory network layer in the initial prediction model with a depth separable convolution to obtain a first convolutional long short-term memory network layer; S3: training the first convolutional long short-term memory network layer according to a preset gating mechanism and a loss function, removing an output channel with a gating variable less than a preset value and fine-tuning to obtain a first prediction model; S4: determining a quantization strategy according to an application scenario; S5: quantizing the first prediction model according to the quantization strategy to obtain a second prediction model; and S6: evaluating the second prediction model, and obtaining a lightweight prediction model in response to the second prediction model passing the evaluation.

[0136] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0137] Any combination of the technical features in the above embodiments can be made, and to make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present disclosure.

[0138] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the scope of the patent. It should be pointed out that, for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the scope of protection of the present application.

Claims

1. A hard disk failure prediction method, characterized in that: Applicable to a hard disk backplane, the hard disk backplane is connected to multiple hard disks, and the method includes: Collecting sensor data sets from sensors corresponding to multiple hard disks, and preprocessing the sensor data sets; Constructing a time-space structure to convert the preprocessed sensor data set into a multidimensional array; Inputting the multidimensional array into a lightweight prediction model, and obtaining prediction states corresponding to the multiple hard disks through a convolutional long short-term memory network layer in the lightweight prediction model; The prediction states corresponding to the multiple hard disks are processed through the prediction layer in the lightweight prediction model, and the fault prediction results of the multiple hard disks are output.

2. The hard disk failure prediction method according to claim 1, wherein: The collecting of sensor data sets from sensors corresponding to the plurality of hard disks and preprocessing the sensor data sets includes: Determining outliers in the sensor data set according to a preset data range, converting the outliers into missing values, and obtaining a first sensor data set; Processing missing values ​​in the first sensor data set according to the length of the missing segment to obtain a second sensor data set; Denoising the second sensor data set by using three-point median filtering combined with exponential moving average to obtain a third sensor data set; The third sensor data set is normalized to obtain a preprocessed sensor data set.

3. The hard disk failure prediction method according to claim 2, characterized in that: The processing of missing values ​​in the first sensor data set according to the length of the missing segment to obtain a second sensor data set includes: In response to the length of the missing segment being less than or equal to a preset length, filling the missing value in the missing segment by linear interpolation; In response to the length of the missing segment being greater than the preset length, a mask of missing values ​​in the missing segment is set to zero.

4. The hard disk failure prediction method according to claim 1, wherein: The construction of the time-space structure converts the pre-processed sensor data set into a multi-dimensional array, including: Constructing a two-dimensional spatial grid according to the layout of the hard disk backplane and the multiple hard disks; Creating an index mapping table according to the two-dimensional spatial grid; Determining a space mask according to the layout of the multiple hard disks; Acquire sensor data corresponding to a plurality of time points, map the sensor data corresponding to the plurality of time points onto the two-dimensional spatial grid according to the index mapping table and the vacancy mask, and arrange them in chronological order to obtain an image sequence; The image sequence is intercepted through a sliding window to obtain the multidimensional array.

5. The hard disk failure prediction method according to claim 1, wherein: Obtaining the predicted states corresponding to the multiple hard disks through the convolutional long short-term memory network layer in the lightweight prediction model includes: Taking the multidimensional array, the hidden state of the previous time step, and the cell state of the previous time step as input, and obtaining an input gate and a forget gate through a first depth-wise separable convolution and a second depth-wise separable convolution; Taking the multidimensional array, the hidden state of the previous time step, the input gate, and the forget gate as input, calculating the candidate cell state and updating the cell state through a third depthwise separable convolution; Taking the multidimensional array, the hidden state of the previous time step and the updated cell state as input, performing a fourth depth-wise separable convolution to obtain an output gate; The updated cell state and the output gate are used as inputs, and the predicted states corresponding to the multiple hard disks are obtained through an activation function.

6. The hard disk failure prediction method according to claim 1, wherein: The convolutional long short-term memory network layer includes a first convolutional long short-term memory network layer and a second convolutional long short-term memory network layer. The predicted states corresponding to the multiple hard disks are obtained by the convolutional long short-term memory network layer in the lightweight prediction model, and further includes: Obtaining first hidden states corresponding to the plurality of hard disks through a first convolutional long short-term memory network layer according to the multidimensional array; According to the first hidden state, a second convolutional long short-term memory network layer is used to obtain predicted states corresponding to the multiple hard disks.

7. The hard disk failure prediction method according to claim 1, wherein: The processing of the prediction states corresponding to the multiple hard disks by the prediction layer in the lightweight prediction model and outputting the failure prediction results of the multiple hard disks includes: Obtaining health scores of the multiple hard disks according to the predicted states, historical data, and current data corresponding to the multiple hard disks; Outputting failure prediction results of the multiple hard disks according to the health scores of the multiple hard disks.

8. The hard disk failure prediction method according to claim 1, wherein: The expression for obtaining the health scores of the multiple hard disks according to the predicted states, historical data, and current data corresponding to the multiple hard disks is specifically as follows: ； in, is the absolute error between the predicted value and the actual value at time t, α is the weight coefficient, is the predicted value sequence for the next K time steps, is the variance of the predicted sequence, β is the weight coefficient, is the actual observation value at the past k time points, is the slope obtained after linear fitting of the historical series, and γ is the weight coefficient.

9. A lightweight prediction model construction method, characterized in that: The method comprises: Based on the training data set, an initial prediction model is obtained; Replacing the standard convolution in the convolutional long short-term memory network layer in the initial prediction model with a depthwise separable convolution to obtain a first convolutional long short-term memory network layer; Training the first convolutional long short-term memory network layer according to a preset gating mechanism and loss function, removing output channels whose gating variables are less than a preset value, and fine-tuning the network to obtain a first prediction model; Determine the quantitative strategy based on the application scenario; quantizing the first prediction model according to the quantization strategy to obtain a second prediction model; The second prediction model is evaluated, and in response to the second prediction model passing the evaluation, the lightweight prediction model is obtained.

10. An electronic device, characterized in that: include: Memory for storing computer programs; A processor for implementing the steps of the hard disk failure prediction method as described in any one of claims 1 to 8 when executing the computer program, or for implementing the steps of the lightweight prediction model construction method as described in claim 9 when executing the computer program.

Images