Prediction method and device of disk life, electronic equipment and storage medium

Abstract

The invention provides a disk life prediction method and device, electronic equipment and a storage medium, and relates to the field of computer hardware. The method comprises the following steps: acquiring data of each feature item influencing the health of the disk, wherein the data comprises SMART attribute data, IOPS data, basic information data, file type data and operating system data; carrying out data processing on the data of each feature item, and carrying out normalization and feature extraction through data normalization and data coding; aligning the processed feature item data according to timestamps, and constructing a time sequence data set; training a machine learning model according to the time sequence data set to obtain a disk life prediction model; deploying the disk life prediction model to a disk health condition monitoring system, monitoring the health condition of the disk in real time, and calculating the residual life of the target disk. The health condition of the disk is predicted by collecting various data influencing the life of the disk in real time and inputting the data into the prediction model, the health change of the disk can be predicted more accurately, and disk faults can be found in advance.

Description

Technical Field

[0001] The present invention relates to the technical field of computer hardware, and in particular to a method, device, electronic equipment and storage medium for predicting the life of a disk. Background Art

[0002] Disks are the main storage medium of computers and are the core hardware components of computers. They can store a large amount of binary data and keep the data intact even after a power outage. Since the development of disks in the 1950s, the data storage capacity of disks has increased from MB to TB, with a sequential data read speed of 2400MB / s and a sequential write speed of 1200MB / s. From the initial giant mechanical disks to today's micro high-speed solid-state disks, each technological breakthrough has greatly promoted the progress of the information age and brought great convenience to people's lives. With the in-depth research of the current society in the field of artificial intelligence, massive amounts of data need to be stored, processed and analyzed, so the performance requirements for disk products are getting higher and higher.

[0003] At present, disks are very important in data centers and various enterprise storage. The data stored in them is an important resource that cannot be easily leaked or lost. As the disk usage period is extended or damaged by external forces, the disk will gradually wear out and eventually fail or become directly outdated. The stored data is likely to be lost or formatted. Predicting the disk life in advance can avoid data loss. In the prior art, only the tool smartctrl provided by the manufacturer can obtain the disk self-detection analysis and reporting technology (SMART) to analyze the disk status. By monitoring the integrity of the disk device driver and reporting potential problems, the failure of the disk device can be predicted in advance. However, this method can only issue a warning when the disk is about to fail, and it is difficult to foresee the trend that the disk may have problems in advance. In addition, the analysis results are not accurate because the factors affecting the disk life cannot be fully analyzed. When the disk device cannot provide reliable data access capabilities, the security and reliability of its stored data will be significantly reduced. Therefore, it is necessary to propose a solution that can accurately predict the disk life to improve the security and reliability of disk use. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, electronic device and storage medium for predicting the life of a disk. By performing feature analysis on multiple factors that affect the life of the disk, a disk life prediction model based on machine learning is used to predict the current health status of the disk. The disk life is predicted on the premise of determining the current health status of the disk, and the prediction result is more accurate.

[0005] To achieve the above object, the present invention proposes the following technical solutions:

[0006] Firstly, a method for predicting disk life is proposed, including:

[0007] Acquire various feature item data that affect disk health in real time, wherein the various feature item data include disk SMART attribute data, disk IOPS data, disk basic information data, disk file type data and disk operating system data;

[0008] Processing the data of each feature item, normalizing and extracting features through data normalization and data encoding;

[0009] Align the feature item data after data processing according to the timestamp to construct a time series data set;

[0010] Divide the time series data set into a training set, a validation set and a test set, train, optimize and evaluate the machine learning model, and obtain a disk life prediction model; wherein the disk life prediction model is a model with the highest prediction accuracy among a random forest model, a support vector machine model and a long short-term memory network model based on a machine learning algorithm;

[0011] Deploy the disk life prediction model to a disk health monitoring system to monitor the disk health in real time; wherein the disk health status is to determine the disk health level;

[0012] After monitoring the health status of the disk to determine the health level of the target disk, the remaining life of the target disk is calculated; the process of calculating the remaining life of the target disk is as follows:

[0013] Remaining life = theoretical life * (health level) α - Elapsed time*(health level) β *(1-health level), where α and β are decay factors, representing the nonlinear effect of health level on remaining life.

[0014] Further, the method further comprises: determining a health level corresponding to the health status of the disk monitored in real time according to the output of the machine learning model and a preset health level;

[0015] The input of the machine learning model is time series data, and the output is a numerical score, which is mapped to different health levels;

[0016] The numerical score range of the machine learning model output is [0, 1.0], 0 indicates that the disk is damaged, and 1 indicates that the disk is completely healthy. The preset health levels and corresponding numerical score labels include: "very healthy": (0.8, 1.0], "good": (0.5, 0.8], "warning": (0.2, 0.5], "failure": [0, 0.2].

[0017] Furthermore, the method further comprises:

[0018] After the disk life prediction model monitors the disk health status to determine the health level of the target disk, selecting several historical time series data of the target disk with accurate prediction as time series data samples;

[0019] Performing association rule mining on the time series data samples to determine the importance ranking of the characteristic items that affect the disk life;

[0020] Based on the accurately predicted health level of the target disk and the importance ranking, a regression analysis is performed on all time series data samples corresponding to the target disk to calculate the weight of each feature item in disk life prediction;

[0021] The time series data set is reconstructed based on the weights, and the disk life prediction model is updated according to the reconstructed time series data set.

[0022] Furthermore, the disk life prediction model is a random forest model;

[0023] The random forest model uses a grid search method to adjust parameters, and selects the best parameter combination after cross-validation; wherein the parameters adjusted by the grid search method include the number of decision trees, the maximum depth of the decision tree, and the minimum number of samples of leaf nodes, the number of decision trees ranges from 10 to 100, the maximum depth of the decision tree ranges from [3, 10, 20], and the minimum number of samples of leaf nodes ranges from [2, 5, 10, 20, 50].

[0024] Furthermore, the values ​​of the decay factors corresponding to the numerical score labels of the health levels are: "very healthy": (0.8, 1.0], α = 2, β = 1.1; "good": (0.5, 0.8], α = 3, β = 5; "warning": (0.2, 0.5], α = 5, β = 7; "fault": [0, 0.2), α = 7, β = 10.

[0025] Furthermore, the method also includes: after determining that the life of the target disk predicted by the machine learning model reaches a preset life threshold, outputting an alarm message.

[0026] In a second aspect, a disk life prediction device is provided, comprising:

[0027] A data acquisition module is used to acquire various feature item data that affect the health of the disk in real time, wherein the various feature item data include disk SMART attribute data, disk IOPS data, disk basic information data, disk file type data and disk operating system data;

[0028] A data processing module is used to process the data of each feature item, and perform standardization and feature extraction through data normalization and data encoding;

[0029] A construction module, used to align the feature item data after data processing according to timestamps to construct a time series data set;

[0030] A training module is used to divide the time series data set into a training set, a validation set and a test set, train, optimize and evaluate the machine learning model, and obtain a disk life prediction model; wherein the disk life prediction model is a model with the highest prediction accuracy among a random forest model, a support vector machine model and a long short-term memory network model based on a machine learning algorithm;

[0031] A deployment monitoring module is used to deploy the disk life prediction model to the disk health monitoring system to monitor the disk health in real time; wherein the disk health is to determine the disk health level;

[0032] The calculation module is used to calculate the remaining life of the target disk after monitoring the health status of the disk to determine the health level of the target disk; the process of calculating the remaining life of the target disk is:

[0033] Remaining life = theoretical life * (health level) α - Elapsed time*(health level) β *(1-health level), where α and β are decay factors, representing the nonlinear effect of health level on remaining life.

[0034] Furthermore, it also includes: a selection module, which is used to select historical time series data of several target disks with accurate predictions as time series data samples after the disk life prediction model monitors the disk health status to determine the health level of the target disk;

[0035] A mining and determination module is used to perform association rule mining on the time series data samples to determine the importance ranking of the characteristic items that affect the disk life;

[0036] An analysis and calculation module, used for performing regression analysis on all time series data samples corresponding to the target disk based on the accurately predicted health level of the target disk and the importance ranking, and calculating the weight of each feature item in disk life prediction;

[0037] An updating module is used to reconstruct the time series data set based on the weights, and update the disk life prediction model according to the reconstructed time series data set.

[0038] In a third aspect, an electronic device is proposed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the disk life prediction method as described above.

[0039] In a fourth aspect, a computer-readable storage medium is proposed, wherein the computer-readable storage medium is used to store a computer program. When the computer program is run on a computer, the computer executes the disk life prediction method as described above.

[0040] It can be seen from the above technical solutions that the technical solutions of the present invention have the following beneficial effects:

[0041] The present invention discloses a disk life prediction method, device, electronic device and storage medium, the method comprising: obtaining various feature item data that affect the health of the disk, including disk SMART attribute data, disk IOPS data, disk basic information data, disk file type data and disk operating system data; performing data processing on each feature item data, and performing normalization and feature extraction through data normalization and data encoding; aligning each feature item data after data processing according to timestamps to construct a time series data set; training a machine learning model based on the time series data set to obtain a disk life prediction model; deploying the disk life prediction model to a disk health status monitoring system, monitoring the disk health status in real time, and calculating the remaining life of the target disk. The present invention predicts the health status of the disk by real-time collection of various data that affect the disk life and inputting them into a prediction model based on machine learning, which can more accurately predict the health changes of the disk and detect disk failures in advance.

[0042] In addition, the present invention further considers the impact of the disk health status on its remaining life when predicting the disk life. The worse the health status of the disk, the greater the impact of its health level on its remaining life. Therefore, when predicting the remaining life of the disk, the present invention not only considers the various characteristic data that affect the disk life, but also considers the current health level of the disk, so that the remaining life of the disk can be predicted more accurately. In addition, in order to improve the accuracy of the prediction model, the present invention also determines the weight of each characteristic item that affects the remaining life of the disk by performing correlation analysis and regression analysis on the sample data of the disk with accurate prediction, and updates the prediction model in real time to further improve the prediction accuracy of the prediction model.

[0043] It should be appreciated that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, may be considered to be part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent.

[0044] The foregoing and other aspects, embodiments and features of the present invention can be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as the features and / or beneficial effects of the exemplary embodiments, will be apparent from the following description or learned from the practice of the specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are not drawn to scale according to actual reference objects. In the accompanying drawings, each identical or nearly identical component shown in various figures may be represented by the same reference numeral. For the sake of clarity, not every component is labeled in each figure. Now, embodiments of various aspects of the present invention will be described by way of example and with reference to the accompanying drawings, in which:

[0046] Figure 1 A flow chart of a method for predicting disk life disclosed in an embodiment of the present application;

[0047] Figure 2 A flowchart for updating the disk life prediction model disclosed in the embodiment of the present application;

[0048] Figure 3 A schematic diagram of a framework of a disk life prediction device disclosed in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meanings understood by people with ordinary skills in the field to which the present invention belongs.

[0051] The words "first", "second" and similar words used in the specification and claims of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular forms of "a", "an" or "the" and similar words do not indicate a quantitative limitation, but indicate the existence of at least one. The words "include" or "comprise" and similar words mean that the elements or objects appearing before "include" or "comprise" include the features, wholes, steps, operations, elements and / or components listed after "include" or "comprise", and do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0052] Traditional disk life prediction methods usually rely on some simple rules or regular inspections to evaluate disk health. These methods can only detect obvious problems and are powerless against hidden potential failures. For example, traditional methods based on SMART data can only issue warnings when the disk is about to have problems, and it is difficult to foresee the trend that the disk may have problems in advance. Therefore, the present invention introduces machine learning technology to analyze more disk usage data, discover the potential rules therein, and more accurately predict disk health changes, identify potential disk failures in advance, and provide a reference for timely replacement and maintenance of the disk.

[0053] The following is a further detailed introduction to the disk life prediction method, device, electronic device and storage medium disclosed in the present invention in conjunction with specific embodiments and drawings.

[0054] Combination Figure 1 As shown, the method for predicting the disk life disclosed in the embodiment includes the following steps:

[0055] Step S102, acquiring various feature item data that affect the health of the disk in real time, wherein the various feature item data include disk SMART attribute data, disk IOPS data, disk basic information data, disk file type data, and disk operating system data;

[0056] SMART attribute data is the main source of information for monitoring disk health. The key SMART attributes that have a significant impact on disk health include: Power-On Hours: records the cumulative working hours of the disk; Temperature: the temperature of the disk's working environment; Raw Read Error Rate: measures the reliability of reading data; Reported_Uncorrect: the number of uncorrectable read errors, indicating that the disk may be physically damaged; Command_Timeout: an increase in the number of disk command timeouts may mean that the disk's response performance has decreased; Current_Pending_Sector: an increase in the pending sector count indicates an increased risk of bad sectors and that the disk is aging; Offline_Uncorrectable: the number of uncorrectable bad sectors found during offline testing, an increase means that data integrity may be compromised; Reallocated Sector Count: the number of times disk bad sectors have been reallocated. When spare space is insufficient, reliability is significantly reduced. SMART attribute data is determined based on disk usage, including load, read and write frequency, and usage environment, and can be collected daily, hourly, or minutely.

[0057] Disk IOPS data includes IOPS, read and write latency, and throughput, which are collected once every minute; disk basic information data includes disk brand information, such as Seagate, Western Digital, Toshiba, etc., and specific disk model information, so as to distinguish and compare different models of disks in life prediction, and disk basic information data is only collected once; disk file type data, that is, the file system type used on the disk, such as ext4, xfs, ntfs; disk operating system data, such as Linux, Windows Server, macOS, etc.; disk file type data and disk operating system data are only collected once, which helps to more accurately analyze the causes of disk wear and life prediction. Among them, the disk file type affects the disk's space management and read and write performance, and thus affects the disk's wear; the disk operating system determines the disk's usage mode, scheduling, load and resource allocation method, thereby affecting the disk's actual usage and life.

[0058] Step S104, processing the data of each feature item, normalizing and extracting features through data normalization and data encoding;

[0059] Among them, before normalizing the data of each feature item, data cleaning is also included, including: missing value processing. Disk life monitoring is a long-term data collection and analysis process, and it is inevitable that some records will be missing. The missing data is filled with the average value of the corresponding data before and after the time point; abnormal data detection and processing. Abnormal data is found through threshold filtering and statistical monitoring. For example, data that exceeds the set threshold and data fluctuation exceeds 50% can be replaced with the average value of the corresponding data before and after the time point;

[0060] Data normalization is mainly for disk SMART attribute data and disk IOPS data. These two types of data have different statistical methods and large data volumes. Direct use cannot fairly consider the effects of all features, so over-normalization processing is performed to [0,1]. For example, the number of disk reads and writes (IOPS) and temperature. The IOPS value may be between 50 and 1000, while the temperature may be between 20 and 60°C. This difference will make the model training pay more attention to the impact of IOPS. When the IOPS value is 600, and all IOPS vary between 50 and 1000, the normalized IOPS value is about 0.58.

[0061] One-Hot encoding is used for data encoding and feature value extraction of One-Hot encoded data to learn the impact of different file system types on disk health.

[0062] For example, after One-Hot encoding the disk file type data and the disk operating system data, the final feature vector is shown in Table 1: the feature vector [1,0,0,1,0,0] of data 1 indicates that the file type is ext4 and the operating system is Linux; the feature vector [0,1,0,0,1,0] of data 2 indicates that the file type is xfs and the operating system is Windows; the feature vector [0,0,1,0,0,1] of data 3 indicates that the file type is ntfs and the operating system is macOS.

[0063] Table 1 One-hot encoding of disk file type data and disk operating system data

[0064] data File Type operating system Eigenvector Data 1 Ext4 Linux [1,0,0,1,0,0] Data 2 xfs Windows [0,1,0,0,1,0] Data 3 ntfs macOS [0,0,1,0,0,1]

[0065] Step S106, aligning the feature item data after data processing according to the timestamp to construct a time series data set;

[0066] The feature data acquired in real time are aligned according to the timestamps to ensure that all data at each time point, including SMART attribute data, IOPS data, file type data, and operating system data, are in the same row, as shown in Table 2 below, to facilitate the correct merging of data. The timestamps should have the same time resolution, such as seconds, minutes, hours, etc.

[0067] Table 2 Several data samples obtained by merging after timestamp alignment

[0068] Year … Second Power-on time temperature Error rate … IOPS File Type operating system 2024 … 45 100 0.2 0.001 … 0.04 1 0 2024 … 10 150 0.2 0.002 … 0.04 1 0 2024 … 35 200 0.5 0.003 … 0.08 1 0 2024 … 55 250 1.0 0.005 … 0.16 0 1

[0069] Since the frequency of data collection for each feature item is different, the high-frequency data timestamps are aggregated into time points consistent with other data frequencies, or missing values ​​are interpolated and filled to complete the continuous data.

[0070] In order to preserve the data meaning of the timestamp, the timestamp in Table 2 above needs to be transformed into a vector before application for model recognition. For example, the feature vector of the timestamp "2024-11-08 10:00:00" after transformation is: [2024, 11, 8, 10, 0, 0].

[0071] Step S108, dividing the time series data set into a training set, a validation set and a test set, training, optimizing and evaluating the machine learning model, and obtaining a disk life prediction model; wherein the disk life prediction model is a model with the highest prediction accuracy among a random forest model, a support vector machine model and a long short-term memory network model based on a machine learning algorithm; the random forest model, the support vector machine model and the long short-term memory network model can all process and predict time series data, and this embodiment selects the model with the highest prediction accuracy after training prediction as the disk life prediction model;

[0072] In this embodiment, the time series data set is divided into a training set, a validation set, and a test set according to 70%:15%:15%.

[0073] Step S110, deploying the disk life prediction model to a disk health monitoring system to monitor the disk health in real time; wherein the disk health is to determine the disk health level;

[0074] Step S112, after monitoring the health status of the disk and determining the health level of the target disk, calculate the remaining life of the target disk; the process of calculating the remaining life of the target disk is as follows:

[0075] Remaining life = theoretical life * (health level) α - Elapsed time*(health level) β *(1-health level), where α and β are decay factors, indicating the nonlinear effect of health level on remaining life; theoretical life is the expected service life of the disk under normal conditions, such as 20,000 hours;

[0076] Among them, the input of the machine learning model is time series data, and the output is a numerical score, which is used to map to different health levels; the numerical score interval of the health level mapping is [0,1.0], 0 indicates that the disk is damaged, and 1 indicates that the disk is completely healthy, then the numerical score labels of the health level include: "very healthy": (0.8,1.0], "good": (0.5,0.8], "warning": (0.2,0.5], "failure": [0,0.2).

[0077] As the working time of the disk increases and the amount of data increases, the disk state may change. The above process of the present invention can timely collect various data that affect the disk life, obtain the disk time series data collected in real time, and input the prediction model to obtain the current disk health status, and then further consider the impact of the disk health status on its remaining life, and calculate the remaining life of the disk; according to the above remaining life calculation formula: theoretical life × (health level) αIndicates the positive impact of the disk's health level on its remaining lifespan. The higher the health level, the longer the remaining lifespan. Elapsed time × (health level) β ×(1-health level): indicates the negative impact of the disk's running time on its remaining life. The lower the health level, the greater the impact of the used time on the remaining life. That is, the worse the health of the disk, the greater the impact of its health level on its remaining life. Therefore, when predicting the remaining life of the disk, the present invention not only considers the various characteristic data that affect the disk's life, but also considers the current health level of the disk, so that the remaining life of the disk can be predicted more accurately. Compared with the prior art, this solution can more accurately predict the health changes of the disk and foresee the trend that the disk may have problems.

[0078] Optionally, the method further includes: determining the health level corresponding to the real-time monitored disk health status according to the output of the machine learning model and the preset health level; wherein the input of the machine learning model is time series data, and the output is a numerical score, and the numerical score is mapped to different health levels; the numerical score interval of the machine learning model output is [0,1.0], 0 indicates that the disk is damaged, and 1 indicates that the disk is completely healthy, then the preset health level and the corresponding numerical score label include: "very healthy": (0. 8,1.0], "good": (0.5,0.8], "warning": (0.2,0.5], "fault": [0,0.2]. When calculating the remaining life, the decay factor values ​​corresponding to the numerical score labels of the health levels are: "very healthy": (0.8,1.0], α=2, β=1.1; "good": (0.5,0.8], α=3, β=5; "warning": (0.2,0.5], α=5, β=7; "fault": [0,0.2), α=7, β=10.

[0079] In the above remaining life calculation formula: theoretical life × (health level) α Indicates the positive impact of the disk's health level on its remaining lifespan. The higher the health level, the longer the remaining lifespan. Elapsed time × (health level) β ×(1-health level): indicates the negative impact of the disk's running time on its remaining life. The lower the health level, the greater the impact of the used time on the remaining life. That is, the present invention further considers the impact of the disk's health status on its remaining life when predicting the disk's life. The worse the health status of the disk, the greater the impact of its health level on its remaining life. Therefore, when predicting the remaining life of the disk, the present invention not only considers the various characteristic data that affect the disk's life, but also considers the current health level of the disk, so that the remaining life of the disk can be predicted more accurately.

[0080] In addition to considering the impact of the current health level of the disk on the remaining life of the disk to improve the remaining life estimation accuracy as mentioned above, the present invention also improves the prediction accuracy of the model for the health status of the disk by updating the disk life prediction model.

[0081] Combination Figure 2 As shown, the process of updating the disk life prediction model includes: step S202, after the disk life prediction model monitors the disk health status to determine the health level of the target disk, select several historical time series data of the target disk with accurate prediction as time series data samples; step S204, perform association rule mining on the time series data samples to determine the weight ranking of the characteristic items that affect the disk life; wherein the Apriori algorithm is used to mine the association rules between each characteristic item and the disk health level; step S206, based on the accurately predicted health level of the target disk and the weight ranking, all time series data samples corresponding to the target disk are subjected to regression analysis to calculate the weight of each characteristic item in the disk life prediction; in implementation, the least squares method is used to calculate the weight value of each characteristic item; step S208, reconstruct the time series data set based on the weight, and update the disk life prediction model according to the reconstructed time series data set. wherein, the process of updating the disk life prediction model according to the reconstructed time series data set is retrained by re-dividing the training set, the validation set and the test set; in addition, the time series data set is also kept updated in real time during the model update process.

[0082] In addition, when the disk life prediction model selects the random forest model, the random forest model adopts the grid search method to adjust parameters during training, and selects the best parameter combination after cross-validation; wherein, the parameters for adjusting parameters by the grid search method include the number of decision trees, the maximum depth of the decision tree and the minimum number of samples of leaf nodes, the value range of the number of decision trees is 10 to 100, the value interval of the maximum depth of the decision tree is [3, 10, 20], and the value interval of the minimum number of samples of leaf nodes is [2, 5, 10, 20, 50]; in this embodiment, the best parameter combination of the random forest model is: number of decision trees, 53; maximum depth of decision trees, 10; minimum number of samples of leaf nodes, 20.

[0083] As an optional implementation, the disk life prediction method disclosed in the present invention also includes: classifying all disks according to the disk storage content, sorting and encoding any type of disk according to the length of the remaining life; receiving a data storage request, and judging the category of the disk to be stored according to the data information; when there is a storage directory corresponding to the data information stored in a disk under the corresponding category, storing the data in the disk and recording the disk code; when none of the disks under the corresponding category contains the storage directory corresponding to the data information, storing the data in the disk with the longest remaining life and establishing a corresponding storage directory, and recording the disk code. By distributing data storage, the storage reliability of the disk storage system is improved, the number of times the entire storage system is shut down for modification due to the exhaustion of the storage life of a disk is reduced, and the service life of the storage system is improved as a whole.

[0084] As an optional embodiment, the disk life prediction method disclosed in the present invention also includes, during disk health status monitoring: after determining that the life of the target disk predicted by the machine learning model reaches a preset life threshold, outputting an alarm message to prompt the staff to replace the storage disk immediately, thereby improving the security and reliability of disk data storage.

[0085] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a disk life prediction device, such as Figure 3 The figure shows a schematic diagram of a disk life prediction device. Figure 3 It can be seen that the device includes: a data acquisition module, which is used to acquire various feature item data that affect the health of the disk, and the various feature item data include disk SMART attribute data, disk IOPS data, disk basic information data, disk file type data and disk operating system data; a data processing module, which is used to process the various feature item data, and perform normalization and feature extraction through data normalization and data encoding; a construction module, which is used to align the various feature item data after data processing according to timestamps to construct a time series data set; a training module, which is used to divide the time series data set into a training set, a validation set and a test set, and train, optimize and Evaluate the machine learning model to obtain a disk life prediction model; wherein the disk life prediction model is a model with the highest prediction accuracy among the random forest model, support vector machine model and long short-term memory network model based on the machine learning algorithm; deploy a monitoring module, which is used to deploy the disk life prediction model to the disk health monitoring system to monitor the disk health status in real time; wherein the disk health status is to determine the disk health level; a calculation module, which is used to calculate the remaining life of the target disk after determining the health level of the target disk by monitoring the disk health status; the process of calculating the remaining life of the target disk is: remaining life = theoretical life * (health level) α - Elapsed time*(health level)β *(1-health level), where α and β are decay factors, representing the nonlinear effect of health level on remaining life.

[0086] Optionally, the device deploys a monitoring module to monitor the health status of the disk in real time and determines the health level of the disk as follows: according to the output of the machine learning model and the preset health level, determine the health level corresponding to the health status of the disk monitored in real time; wherein the input of the machine learning model is time series data, and the output is a numerical score, and the numerical score is mapped to different health levels; the numerical score interval of the machine learning model output is [0,1.0], 0 indicates that the disk is damaged, and 1 indicates that the disk is completely healthy, then the preset health level and the corresponding numerical score label include: "very healthy": (0.8,1.0], "good": (0.5,0.8], "warning": (0.2,0.5], "fault": [0,0.2).

[0087] Optionally, in the process of calculating the remaining life of the target disk, the values ​​of decay factors α and β are determined according to the health level, and the values ​​of decay factors corresponding to the numerical score labels of the health level are: "very healthy": (0.8, 1.0], α = 2, β = 1.1; "good": (0.5, 0.8], α = 3, β = 5; "warning": (0.2, 0.5], α = 5, β = 7; "fault": [0, 0.2), α = 7, β = 10.

[0088] As an optional implementation, the disk life prediction device also includes a module for updating the prediction model to improve the model prediction accuracy, specifically including: a selection module, which is used to select several historical time series data of the target disk with accurate prediction as time series data samples after the disk life prediction model monitors the disk health status to determine the health level of the target disk; a mining and determination module, which is used to perform association rule mining on the time series data samples to determine the importance ranking of the various feature items that affect the disk life; an analysis and calculation module, which is used to perform regression analysis on all time series data samples corresponding to the target disk based on the accurately predicted health level of the target disk and the importance ranking, and calculate the weight of each feature item when predicting the disk life; an updating module, which is used to reconstruct the time series data set based on the weight, and update the disk life prediction model according to the reconstructed time series data set.

[0089] The present invention deeply explores the impact of various data collected in real time that affect the disk life on the disk life, and evaluates the remaining disk life from two perspectives: prediction model training and updating, and evaluation of the impact of disk health level on disk life, with high prediction accuracy. Compared with the existing technology, it can not only comprehensively analyze the impact of disk use on its life, but also predict disk failures in advance based on changes in disk use, thereby improving the security and reliability of disk storage.

[0090] Based on the same inventive concept as the above-mentioned method embodiment, an electronic device is also provided in an embodiment of the present application, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the electronic device implements the control method in the above-mentioned embodiment.

[0091] In one embodiment, the electronic device may be a server. In this embodiment, the structure of the electronic device may be as follows: Figure 4 As shown, it includes a memory 2001 , a communication module 2003 and one or more processors 2002 .

[0092] The memory 2001 is used to store computer programs executed by the processor 2002. The memory 2001 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0093] The memory 2001 may be a volatile memory, such as a random-access memory (RAM); the memory 2001 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 2001 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2001 may be a combination of the above memories.

[0094] The processor 2002 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 2002 is used to implement the above-mentioned audio data processing method when calling the computer program stored in the memory 2001 .

[0095] The communication module 2003 is used to communicate with terminal devices and other servers.

[0096] The specific connection medium between the memory 2001, the communication module 2003 and the processor 2002 is not limited in the embodiment of the present application. Figure 4 In the embodiment, the memory 2001 and the processor 2002 are connected via a bus 2004. The bus 2004 is Figure 4 The arrows in the figure are used to describe the connection between other components, which is only for illustrative purposes and is not intended to be limiting. Bus 2004 can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 4 The diagram only uses one arrow, but does not describe only one bus or one type of bus.

[0097] Based on the same inventive concept as the above method embodiment, an embodiment of the present invention also provides a computer-readable storage medium, the computer-readable storage medium is used to store a computer program, when the computer program is run on a computer, the electronic device implements the control method in the above embodiment. The computer-readable storage medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a magnetic disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0098] Based on the same inventive concept as the above-mentioned method embodiment, an embodiment of the present invention further provides a computer program product, the computer program product including a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the control method according to various exemplary embodiments of the present application described above in this specification. The program product may adopt any combination of one or more readable media. These computer program commands may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the steps in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. A method for predicting disk life, characterized in that: include: Acquire various feature item data that affect disk health in real time, wherein the various feature item data include disk SMART attribute data, disk IOPS data, disk basic information data, disk file type data and disk operating system data; Processing the data of each feature item, normalizing and extracting features through data normalization and data encoding; Align the feature item data after data processing according to the timestamp to construct a time series data set; Divide the time series data set into a training set, a validation set and a test set, train, optimize and evaluate the machine learning model, and obtain a disk life prediction model; wherein the disk life prediction model is a model with the highest prediction accuracy among a random forest model, a support vector machine model and a long short-term memory network model based on a machine learning algorithm; Deploy the disk life prediction model to a disk health monitoring system to monitor the disk health in real time; wherein the disk health status is to determine the disk health level; After monitoring the health status of the disk to determine the health level of the target disk, the remaining life of the target disk is calculated; the process of calculating the remaining life of the target disk is as follows: Remaining life = theoretical life * (health level) α - Elapsed time*(health level) β *(1-health level), where α and β are decay factors, representing the nonlinear effect of health level on remaining life.

2. The method for predicting disk life according to claim 1, characterized in that: The method further comprises: determining a health level corresponding to the health status of the disk monitored in real time according to the output of the machine learning model and a preset health level; The input of the machine learning model is time series data, and the output is a numerical score, which is mapped to different health levels; The numerical score range of the machine learning model output is [0, 1.0], 0 indicates that the disk is damaged, and 1 indicates that the disk is completely healthy. The preset health levels and corresponding numerical score labels include: "very healthy": (0.8, 1.0], "good": (0.5, 0.8], "warning": (0.2, 0.5], "fault": [0, 0.2].

3. The method for predicting disk life according to claim 1, characterized in that: The method further comprises: After the disk life prediction model monitors the disk health status to determine the health level of the target disk, selecting several historical time series data of the target disk with accurate prediction as time series data samples; Performing association rule mining on the time series data samples to determine the importance ranking of the characteristic items that affect the disk life; Based on the accurately predicted health level of the target disk and the importance ranking, a regression analysis is performed on all time series data samples corresponding to the target disk to calculate the weight of each feature item in disk life prediction; The time series data set is reconstructed based on the weights, and the disk life prediction model is updated according to the reconstructed time series data set.

4. The method for predicting disk life according to claim 1, characterized in that: The disk life prediction model is a random forest model; The random forest model uses a grid search method to adjust parameters, and selects the best parameter combination after cross-validation; wherein the parameters adjusted by the grid search method include the number of decision trees, the maximum depth of the decision tree, and the minimum number of samples of leaf nodes, the number of decision trees ranges from 10 to 100, the maximum depth of the decision tree ranges from [3, 10, 20], and the minimum number of samples of leaf nodes ranges from [2, 5, 10, 20, 50].

5. The method for predicting disk life according to claim 2, characterized in that: The values ​​of the decay factors corresponding to the numerical score labels of the health levels are: "very healthy": (0.8, 1.0], α = 2, β = 1.1; "good": (0.5, 0.8], α = 3, β = 5; "warning": (0.2, 0.5], α = 5, β = 7; "fault": [0, 0.2), α = 7, β = 10.

6. The method for predicting disk life according to claim 1, characterized in that: The method also includes: after determining that the life of the target disk predicted by the machine learning model reaches a preset life threshold, outputting an alarm message.

7. A device for predicting disk life, characterized in that: include: A data acquisition module is used to acquire various feature item data that affect the health of the disk in real time, wherein the various feature item data include disk SMART attribute data, disk IOPS data, disk basic information data, disk file type data and disk operating system data; A data processing module is used to process the data of each feature item, and perform standardization and feature extraction through data normalization and data encoding; A construction module, used to align the feature item data after data processing according to timestamps to construct a time series data set; A training module is used to divide the time series data set into a training set, a validation set and a test set, train, optimize and evaluate the machine learning model, and obtain a disk life prediction model; wherein the disk life prediction model is a model with the highest prediction accuracy among a random forest model, a support vector machine model and a long short-term memory network model based on a machine learning algorithm; A deployment monitoring module is used to deploy the disk life prediction model to the disk health monitoring system to monitor the disk health in real time; wherein the disk health is to determine the disk health level; The calculation module is used to calculate the remaining life of the target disk after monitoring the health status of the disk to determine the health level of the target disk; the process of calculating the remaining life of the target disk is: Remaining life = theoretical life * (health level) α - Elapsed time*(health level) β *(1-health level), where α and β are decay factors, representing the nonlinear effect of health level on remaining life.

8. The disk life prediction device according to claim 7, characterized in that: Also includes: A selection module is used to select accurate historical time series data of a number of target disks as time series data samples after the disk life prediction model monitors the disk health status to determine the health level of the target disk; A mining and determination module is used to perform association rule mining on the time series data samples to determine the importance ranking of the characteristic items that affect the disk life; An analysis and calculation module, used for performing regression analysis on all time series data samples corresponding to the target disk based on the accurately predicted health level of the target disk and the importance ranking, and calculating the weight of each feature item in disk life prediction; An updating module is used to reconstruct the time series data set based on the weights, and update the disk life prediction model according to the reconstructed time series data set.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the method for predicting the life of a magnetic disk according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program. When the computer program is executed on a computer, the computer is enabled to execute the method for predicting the disk life according to any one of claims 1 to 6.

Images