Method, device and system for predicting execution policy based on storage device health state

Abstract

The application discloses an execution strategy prediction method, device and system based on a storage device health state, and belongs to the technical field of computer storage technology, which comprises the following steps: performing fault prediction and residual life prediction according to state data of the storage device at a current time to obtain a health state of the storage device, inputting the health state into a multi-agent reinforcement learning model to obtain a corresponding execution strategy; the execution strategy is to maintain the status quo, backup data or replace the storage device; performing mutation point detection on the state data at each time within a sliding time window before the current time, if the execution strategy is to maintain the status quo but a mutation point is detected, false negative is determined; if the execution strategy is to backup data or replace the storage device but no mutation point is detected, false positive is determined; in other cases, the execution strategy is reported to a storage system, so that the storage system performs corresponding operations. The application can effectively solve the technical problems of lack of decision processing, high misjudgment cost and poor interpretability of existing methods.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer storage, and more particularly relates to an execution strategy prediction method, device and system based on a storage device health state. BACKGROUND

[0002] With the continuous development of computer technology, the growth of users and the diversification of applications have led to an explosive growth in data volume, which has also led to continuous upgrades in capacity, quantity and performance of storage devices. In an era of continuous strengthening of data integrity and high availability of services, storage device failures still cannot be avoided and occur from time to time, which also increases the risk of data loss and economic loss. Storage device failure prediction technology helps to guarantee the reliability and availability of storage system data.

[0003] Based on the storage device, the failure prediction method is constantly innovating in multiple dimensions such as data preprocessing, feature selection, modeling algorithm, feature category, and prediction output. For example, the modeling algorithm gradually transitions from being based on SMART threshold judgment and statistical analysis to machine learning, deep learning and transfer learning methods; the feature category adds other dimensional effective information on the basis of single SMART feature; the prediction output develops to failure prediction, remaining life prediction, small sample prediction, single sample detection and other aspects.

[0004] The above methods focus on how to improve the prediction accuracy of the model, ignoring the research on storage device processing decisions. There is an inevitable error in the prediction result, and the misclassification cost is large. Specifically, if data backup or replacement of the storage device is not performed in time when the storage device is unreliable, data loss and economic loss may occur; and if data backup or replacement of the storage device is performed too early when the storage device is operating normally, resource waste will occur. In addition, the existing failure prediction method has independent prediction results before and after, which is only related to the current data to be tested, and the model has poor interpretability. SUMMARY

[0005] In view of the defects and improvement needs of the prior art, the present application provides an execution strategy prediction method, device and system based on a storage device health state, which aims to propose an execution strategy suitable for the health state of the storage device to solve the technical problems of lack of processing decisions, large misclassification cost and poor interpretability of the existing failure prediction method.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, an execution strategy prediction method based on a storage device health state is provided, comprising:

[0007] The storage device is predicted for failure and remaining life according to state data of the storage device at a current time, and a health state of the storage device is formed by failure prediction information and remaining life prediction information; the state data includes multiple features associated with the health state of the storage device;

[0008] The health state of the storage device is input into the multi-agent reinforcement learning model, and a corresponding execution strategy is output by the multi-agent reinforcement learning model; the execution strategy is to maintain the status quo, backup data or replace the storage device;

[0009] Each feature in a state sequence formed by state data of the storage device at the current time and at each time within a sliding time window before the current time is detected for a mutation point, if the current execution strategy is to maintain the status quo, but a mutation point is detected, it is determined that the current execution strategy belongs to a false negative; if the current execution strategy is to backup data or replace the storage device, but no mutation point is detected, it is determined that the current execution strategy belongs to a false positive; in other cases, it is determined that the current execution strategy is correct, and the execution strategy is reported to the storage system, so that the storage system performs corresponding operations.

[0010] The execution strategy prediction method based on the health state of the storage device provided by the application simultaneously uses failure prediction information and remaining life prediction information of the storage device to construct a health state of the storage device, which not only more comprehensively and accurately evaluates the health state of the storage device from multiple angles, but also avoids storage resource waste and data loss caused by premature backup / late replacement of the storage device due to single-dimensional false prediction;

[0011] The three execution strategies, i.e., maintaining the status quo, backing up data and replacing the storage device, can maximize the coverage of each stage of the life cycle of the storage device, specifically, in the early and middle stages of the use of the storage device, the failure rate is low, and the remaining service life is the highest, at this time, the health state of the storage device is good, and the storage device should be selected to be retained without other processing; in the transition stage from the middle stage to the late stage of the storage device, the storage device gradually wears out, the stability decreases, and the probability of failure increases, but still has a long remaining life, at this time, the backup data operation should be selected to avoid storage resource waste caused by premature replacement of the hard disk; in the late stage of the storage device, the device wears and ages seriously, the remaining life is short, and the probability of failure is extremely high, at this time, it needs to be replaced in time to avoid data loss and economic loss caused by failure. Therefore, based on the above execution strategy design, the selection of the execution strategy in the application is more flexible and more accurate, and greater benefits can be obtained;

[0012] The application makes a decision on post-processing of a prediction result of a storage device, uses a multi-agent reinforcement learning model to predict a corresponding execution strategy according to a health state of the storage device, can accurately evaluate a decision risk according to the health state of the storage device and give an execution strategy with the minimum risk and the best benefit, and can comprehensively consider historical prediction results, so that the prediction result is more accurate and has higher interpretability.

[0013] The application detects a mutation point of state data in a current time window and a sliding time window of the storage device, verifies the execution strategy predicted by the multi-agent reinforcement learning model in combination with a detection result of the mutation point, and sends the corresponding execution strategy to a storage system to perform a corresponding operation only when the execution strategy is verified to be correct, so that the application can filter false positive and false negative prediction results, reduce false positive rate and false negative rate, and improve reliability of the storage device.

[0014] Overall, the above technical solution provided by the application uses failure prediction information and residual life prediction information of a storage device to construct a health state of the storage device, maintains a status quo, backs up data, and replaces the storage device as optional execution strategies, uses a multi-agent reinforcement learning model to predict a corresponding execution strategy according to the health state of the storage device, verifies the execution strategy predicted by the multi-agent reinforcement learning model in combination with a detection result of a mutation point of state data in a current time window and a sliding time window, and sends only the execution strategy verified to be correct to a storage system, so that the application can ensure that the execution strategy is adapted to the health state of the storage device, effectively solve technical problems of lack of processing decision, high misjudgment cost, and poor interpretability of an existing failure prediction method, and improve reliability of the storage device.

[0015] Further, the execution strategy prediction method based on a health state of a storage device provided by the application further includes: if the current execution strategy is false negative or false positive, the current health state of the storage device is returned to the multi-agent reinforcement learning model, and the multi-agent reinforcement learning model is retrained.

[0016] Further, in the multi-agent reinforcement learning model, for the execution strategy of maintaining the status quo, a feedback function Reward Keep As follows:

[0017]

[0018] For the execution strategy of backing up data, a feedback function Reward Backup As follows:

[0019]

[0020] For the execution strategy of replacing the storage device, a feedback function Reward Replace As follows:

[0021]

[0022] wherein bestKeep represents the retention time point, bestBackup represents the best data backup time point, bestReplace represents the best replacement time point, x represents the actual time point; a, b, c, d, f all represent the change rate; maxKeep, maxBackup and maxReplace respectively represent the maximum feedback values corresponding to maintaining the status quo, data backup and replacing the storage device.

[0023] Further, the failure prediction information in the health status of the storage device is a binary tuple:

[0024]

[0025] wherein, represents the failure prediction information of the storage device m at time t i ; represents the failure prediction result of the storage device m at time t i , represents the probability that the storage device m is predicted to be normal at time t i , represents the probability that the storage device m is predicted to be failure at time t i , and α and β respectively represent the weights of each contribution factor in the binary tuple. and are output by the trained failure prediction model according to the state data of the storage device m at time t i .

[0026] Further, the remaining life prediction information in the health status of the storage device is a binary tuple:

[0027]

[0028] wherein, represents the remaining life prediction information of the storage device m at time t i ; represents the remaining life prediction result of the storage device m at time t i , which is output by the trained remaining life prediction model according to the state data of the storage device m at time t i ; represents the exponential moving average value; RUL max represents the maximum predicted remaining life; and γ and λ respectively represent the weights of each contribution factor in the binary tuple.

[0029] Further, the state data comprises a reliability index of the running state of the storage device, performance information of the storage device, and error information associated with the failure.

[0030] Further, the reliability index of the running state of the storage device is extracted from SMART data of the storage device, the performance information of the storage device is extracted from I / O state information of the storage device, and the error information associated with the failure is extracted from an error log of the storage device.

[0031] Further, the method for predicting an execution strategy based on the health state of the storage device further comprises: when it is determined that the current execution strategy is correct, sending the feature of the detected mutation point to the storage system.

[0032] According to another aspect of the present application, there is provided a device for predicting an execution strategy based on the health state of a storage device, comprising:

[0033] a computer readable storage medium for storing a computer program;

[0034] and a processor for reading the computer program stored in the computer readable storage medium and executing the method for predicting an execution strategy based on the health state of the storage device.

[0035] According to still another aspect of the present application, there is provided a storage system comprising a storage device and the device for predicting an execution strategy based on the health state of the storage device.

[0036] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0037] (1) The above technical solutions provided by the present application utilize both the failure prediction information and the remaining life prediction information of the storage device to construct a health state of the storage device, maintain the present situation, backup data and replace the storage device as optional execution strategies, utilize a multi-agent reinforcement learning model to predict a corresponding execution strategy according to the health state of the storage device, and verify the execution strategy predicted by the multi-agent reinforcement learning model in combination with the mutation point detection result of the state data within the current time and the sliding time window, only send the execution strategy verified correctly to the storage system, which can ensure that the execution strategy is adapted to the health state of the storage device, effectively solve the technical problems of lack of processing decision, high misjudgment cost and poor interpretability existing in the existing failure prediction method, and improve the reliability of the storage device.

[0038] (2) The application can return the current health state of the storage device to the multi-agent reinforcement learning model in the case of false positives and false negatives, retrain the multi-agent reinforcement learning model, optimize the multi-agent reinforcement learning model, and improve the prediction accuracy of the multi-agent reinforcement learning model.

[0039] (3) The application proposes a multi-element feedback mechanism for the multi-agent reinforcement learning model, that is, the feedback function is designed for each execution strategy. In the life cycle of the storage device, the actions of "maintaining the status quo", "backing up data" and "replacing the storage device" have their own optimal execution intervals, and the corresponding actions can obtain the highest feedback within a specific interval. The application can make the agent make the best decision for the various states of the storage device through the multi-element feedback mechanism.

[0040] (4) The failure prediction result indicating whether the storage device fails is an extreme value, and direct introduction will lead to a less smooth storage device health state evaluation result. The application introduces a value-continuous and Specifically, in addition to , the application introduces as another contribution factor of the storage device health state, which can more comprehensively and more granularly represent the health state of the storage device.

[0041] (5) In addition to the remaining life prediction result , the application introduces the exponential moving average of to construct another contribution factor of the storage device health state, which can effectively reduce the volatility of the remaining life prediction result.

[0042] (6) The application collects the reliability indicators of the storage device running state, the performance information of the storage device, and the error information associated with the failure as the state data of the storage device, which is used for subsequent prediction, can comprehensively and accurately represent the state of the storage device, and ensures the prediction accuracy of the subsequent health state and execution strategy.

[0043] (7) Generally, when the state of the storage device changes, the mutation feature can be considered as the cause of the change of the storage device state. When the application determines that the current execution strategy is correct, the execution strategy is sent to the storage system, and the detected mutation point feature is also sent to the storage system, so that the storage system can know the cause of the change of the storage device state, and further improve the interpretability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The execution strategy prediction method flowchart based on the health state of the storage device provided by the embodiment of the application;

[0045] Figure 2 A method for executing strategy based on health state of storage device is provided in the embodiment of the present application.

[0046] Figure 3 A multi-element feedback function diagram is provided in the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as there is no conflict.

[0048] In the present application, the terms "first", "second", etc. (if any) in the present application and the accompanying drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0049] Embodiment 1:

[0050] A method for predicting the execution strategy based on the health state of the storage device, as shown in Figure 1 and Figure 2 , comprising:

[0051] First, according to the state data of the storage device at the current time, the storage device is predicted for failure and remaining life, and the health state of the storage device is composed of the failure prediction information and the remaining life prediction information.

[0052] In order to comprehensively and accurately characterize the current state of the storage device, in the present embodiment, the collected state data of the storage device specifically includes: the reliability index of the running state of the storage device, the performance information of the storage device, and the error information associated with the failure as the state data of the storage device; optionally, in the present embodiment, the three types of information in the state data of the storage device are respectively extracted from the SMART data, I / O state information and error log of the storage device.

[0053] SMART technology refers to the "self-monitoring, analysis and reporting technology" of the storage device, which monitors, records and compares the running conditions of the storage device hardware such as the head, disk, motor and circuit through detection instructions in the storage device with the threshold value set by the storage device manufacturer, and if the monitoring record value exceeds the threshold value, the host monitoring hardware or software automatically alarms the user and performs coarse-grained automatic repair to ensure the data security of the hard disk in advance. These SMART data records a number of reliability indicators of the running state of the hard disk, making it possible to predict hard disk failures with high accuracy and high coverage. The reliability indicators extracted from the SMART data in the embodiment specifically include: timestamp, hard disk manufacturer, hard disk serial number, underlying data read error rate, disk startup time, repositioning magnetic area count, seek error rate, hard disk power-on duration, report uncorrectable error, and sector count waiting for repositioning. Each item of data constitutes a feature in the state data.

[0054] The I / O state information contains a variety of information reflecting the performance of the storage device, and frequent and continuous I / O operations can reduce the service life of the hard disk and may cause failure. Therefore, the embodiment extracts performance information from the I / O state information, which specifically includes: number of read / write sectors per second, number of read / write kilobytes per second, I / O average sector number, I / O average time required, read / write operation time required, and busy ratio of the storage device, etc. Each item of data constitutes a feature in the state data.

[0055] The gradually aging storage device will have various drive layer and system layer errors before the final failure, so the embodiment extracts error information associated with failure from the error log of the storage device, which specifically includes: read / write error, correctable and uncorrectable ECC error, erase error, response timeout error, system event, blue screen failure, etc. Each item of data constitutes a feature in the state data.

[0056] It should be noted that each feature in the above state data is only an exemplary description and should not be understood as the only limitation of the present application. In the case of ensuring accurate representation of the state of the storage device, the state data of the storage device can also use other combinations of features.

[0057] The specific way of the embodiment for predicting the failure of the storage device according to the state data of the storage device at the current time is as follows:

[0058] At the current time t i The state data of the storage device m is input into the trained failure prediction model, and the failure prediction result is output by the failure prediction model 0 represents that the storage device is normal, and 1 represents that the storage device is faulty.

[0059] The fault prediction model is specifically a classification model, and the trained fault prediction model can be obtained by training the state data of the storage device with a known fault state.

[0060] Considering that the fault prediction result indicating whether the storage device fails is an extreme value, directly introducing it will lead to an insufficiently smooth storage device health state evaluation result, therefore, a value-continuous and are introduced to more comprehensively and more finely represent the health status of the storage device, wherein, represents the probability that the storage device m is predicted to be normal at time t i , represents the probability that the storage device m is predicted to fail at time t i , and are also predicted and output by the fault prediction model according to the state data of the storage device m at time t i ; specifically, in addition to , the present application introduces as another contribution factor of the storage device health status, when the value of is greater than 0, it is considered that the device tends to fail, and vice versa, it is considered that the device tends to be healthy, , the value range of is [-1, 1], and the present embodiment realizes the normalization of

[0061] by corresponding calculation, so that the value range of is [0, 1], through the processing, the storage device health state evaluation result can be made more smooth, and through the normalization, the subsequent data processing is facilitated and the program running convergence is accelerated. Specifically, in the present embodiment, the fault prediction information of the storage device m at time t i is a binary tuple:

[0062] wherein α and β respectively represent the weights of the contribution factors in the binary tuple, which can be flexibly adjusted in actual application; the introduction of the weights is helpful to reduce the negative impact of false positives, and also makes the contribution factors more smooth.

[0063] Optionally, in the present embodiment, the specific manner of predicting the remaining life of the storage device according to the state data of the storage device at the current time is as follows:

[0064] At the current time t i , the state data of the storage device m is input into the trained remaining life prediction model, and the remaining life prediction result

[0065] The remaining life prediction model is specifically a regression model, which is trained using status data of storage devices with known fault states to obtain a trained remaining life prediction model.

[0066] In order to reduce the adverse effects of the volatility of the regression model results on the health status assessment of the storage device, as a preferred implementation, this embodiment uses the remaining life prediction results to In addition, the introduction Exponential moving average of Another contributing factor to the health of storage devices is the exponential sliding average, which is calculated using the following formula:

[0067]

[0068] in, represents the exponential sliding average of the remaining life prediction results of the storage device at the previous moment, and w represents the decay weight;

[0069] In this embodiment, based on Another contribution factor constructed is The RUL max Indicates the predicted maximum remaining life, using RUL max The value range of the contribution factor is normalized to the interval [0,1]. Normalization can facilitate subsequent data processing and accelerate the convergence of the program.

[0070] Specifically, in this embodiment, the remaining life prediction information in the health status of the storage device is a two-tuple:

[0071]

[0072] Among them, γ and λ represent the weights of each contribution factor in the binary group, which can be flexibly adjusted in practical applications. The introduction of weights helps to reduce the negative impact of false positives and also makes the contribution factors smoother.

[0073] Based on the above fault prediction information and remaining life prediction information, in this embodiment, for any storage device m, at time t i health status It can be represented as the following four-tuple:

[0074]

[0075] The embodiment simultaneously utilizes the failure prediction information and the residual life prediction information of the storage device to construct a health state of the storage device, more comprehensively and accurately evaluates the health state of the storage device from multiple angles, and can avoid the waste of storage resources and the loss of data caused by the premature backup / late replacement of the storage device due to single-dimensional false prediction. Meanwhile, the embodiment constructs a new contribution factor in addition to the failure prediction result and the residual life prediction result, can ensure that the evaluation result of the health state of the storage device is smoother, and the normalized processing can facilitate subsequent data processing and accelerate the convergence of the program.

[0076] After obtaining the health state of the storage device, the embodiment further comprises: Figure 1 Figure 2 inputting the health state of the storage device into the multi-agent reinforcement learning model, outputting a corresponding execution strategy from the multi-agent reinforcement learning model; the execution strategy is to maintain the status quo, backup data or replace the storage device.

[0077] inputting the health state of the storage device into the multi-agent reinforcement learning model, outputting a corresponding execution strategy from the multi-agent reinforcement learning model; the execution strategy is to maintain the status quo, backup data or replace the storage device.

[0078] Based on the health state characteristics of the storage device in different stages of the whole life cycle, the embodiment correspondingly proposes three execution strategies, namely maintaining the status quo, backing up data and replacing the storage device. Specifically, in the early and middle stages of the use of the storage device, the failure rate is low and the residual service life is the highest, at this time the health state of the storage device is good, and the storage device should be selected to be retained without other processing; in the transition stage from the middle stage to the late stage of the storage device, the storage device device gradually wears, the stability decreases, and the probability of failure increases, but still has a long residual life, at this time the backup data operation should be selected to avoid the waste of storage resources caused by the premature replacement of the hard disk; in the late stage of the storage device, the device wears and ages seriously, the residual life is short, and the probability of failure is extremely high, at this time the storage device needs to be replaced in time to avoid the loss of data and economic loss caused by failure. Based on the above execution strategy design, the selection of the execution strategy in the application is more flexible and more accurate, and greater benefits can be obtained. The three execution strategies designed in the embodiment are also used as decision actions of the multi-agent reinforcement learning model, and Keep, Backup and Replace represent maintaining the status quo, backing up data and replacing the storage device, respectively. Therefore, the decision action Action of the multi-agent reinforcement learning model can be represented as the following triple:

[0079] Action=(Keep,Backup,Replace)

[0080] ​The embodiment starts from the decision of post-processing of the prediction result of the storage device, and uses a multi-agent reinforcement learning model to predict the corresponding execution strategy according to the health state of the storage device. The execution strategy with the minimum risk and the best benefit can be obtained according to the health state of the storage device. The historical prediction results can be considered comprehensively, and the prediction result is more accurate and has higher interpretability. In the life cycle of the storage device, the actions of "maintaining the status quo", "backing up data" and "replacing the storage device" have their own best execution intervals. The corresponding actions can obtain the highest feedback in a specific interval. Based on this, the embodiment proposes a multi-element feedback mechanism for the multi-agent reinforcement learning model. In the multi-element feedback mechanism, a specific feedback function is designed for the decision action and the current state of each storage device:

[0081] Reward=F(s i ,a i )→r i

[0082] Where s i and a i represent the state and action at the current time, and r i represents the current feedback.

[0083] Specifically, for the execution strategy of maintaining the status quo, the feedback function Reward Keep is as follows:

[0084]

[0085] For the execution strategy of backing up data, the feedback function Reward Backup is as follows:

[0086]

[0087] For the execution strategy of replacing the storage device, the feedback function Reward Replace is as follows:

[0088]

[0089] Where bestKeep represents the retention time point, bestBackup represents the best data backup time point, bestReplace represents the best replacement time point, and x represents the actual time point. a, b, c, d, and f represent the change rate. maxKeep, maxBackup, and maxReplace represent the maximum feedback values corresponding to maintaining the status quo, data backup, and replacing the storage device, respectively. The feedback functions for each action are as shown in Figure 3 Figure 3 ​In the above multi-element feedback mechanism, the feedback functions of the actions of "backup data" and "replace storage device" adopt a segmented design, the purpose of which is to make the feedback function more flexible, and thus make the agent learn the trend of the gradual transition of the action, and make it more sensitive to the change of the action execution timing.

[0090] In the multi-agent reinforcement learning model, the simulation environment is a whole formed by integrating the actions, states and feedback functions of each storage device in the storage system, and is used to describe the logic and rules of the interaction between each other. It is mainly responsible for receiving an action, calculating the feedback result based on the health status of the current storage device through the feedback function and transferring to the next state, which can be represented by a triple:

[0091] Environment=(State,Action,Reward)

[0092] Specifically, its role can be represented by the following formula:

[0093] Environment(s i ,a i )→s i+1 ,r i

[0094] s i and a i represent the state and action at the current time, r i represents the current feedback, and s i+1 represents the state at the next time;

[0095] The agent selects the action with the highest return from the action space according to the current strategy and the input state data, and the environment gives feedback after the action is executed, and then the agent receives the feedback and optimizes its own strategy according to the interaction process. The core of the agent is the strategy function, which is as follows:

[0096] Agent=P(State→Action

[0097] where Agent represents the agent, P represents the strategy function, and P(S→A represents that the input of the strategy function is the state State and the output is the action Action.

[0098] Specifically, the process of optimizing the strategy of the agent according to the interaction process is as follows:

[0099] Agent(a i ,s i ,r i ,s i+1 ,P)→P′

[0100] P′ represents the action selection strategy of the optimized storage device, s i and a i respectively represent the health state and action of the storage device at the current time, r i represents the current feedback, s i+1 represents the state at the next time, which together constitute an interaction process, and the agent optimizes its strategy function P according to the interaction information;

[0101] The multi-agent reinforcement learning model is fully trained based on the health state of each storage device, a simulated environment, a multi-element feedback mechanism, and a formulated action decision, so as to finally realize the maximization of the total income of a series of decisions of the agent.

[0102] By taking the real-time health state of the storage device as input, the corresponding execution strategy of the storage device at the next stage, i.e., maintaining the status quo, backing up data, or replacing the storage device, can be obtained through the multi-agent reinforcement learning model.

[0103] The present application starts from the decision-making of post-processing of the prediction result of the storage device, and uses the multi-agent reinforcement learning model to predict the corresponding execution strategy according to the health state of the storage device, so as to accurately evaluate the decision-making risk according to the health state of the storage device and give the execution strategy with the best risk-minimizing income, and to comprehensively consider the historical prediction results, so that the prediction result is more accurate and has higher interpretability.

[0104] Referring to Figure 1 and Figure 2 , after predicting the execution strategy corresponding to the current health state of the storage device by using the multi-agent reinforcement learning model, the embodiment further comprises: detecting a mutation point for each feature in a state sequence composed of state data of the storage device at the current time and at each time within a sliding time window before the current time, if the current execution strategy is to maintain the status quo, but a mutation point is detected, it is determined that the current execution strategy belongs to a false negative, if the current execution strategy is to back up data or replace the storage device, but no mutation point is detected, it is determined that the current execution strategy belongs to a false positive, and in other cases, it is determined that the current execution strategy is correct, and the execution strategy is reported to the storage system, so that the storage system performs corresponding operations.

[0105] In the early and middle stages of a storage device's use, actions to maintain the status quo are required, and during this period, its state data typically does not mutate. In the middle and late stages of a storage device's use, actions such as data backup or replacement are required, and during these stages, the state data of the storage device typically mutates. Based on this principle, this embodiment verifies the execution strategy predicted by the multi-agent reinforcement learning model using the mutation point detection results of the state data, identifying false positives and false negatives. Only when the execution strategy is correct is it reported to the storage system for execution of the corresponding action. This can filter out false positive and false negative prediction results, reduce the false positive and false negative rates, and improve the reliability of the storage device.

[0106] Optionally, in this embodiment, for any storage device m, obtain the current time t i and the previous sliding time window θ(t i ~t i-θ ) state data, obtain the state data sequence, extract each feature from the state sequence, and obtain the feature data value sequence n represents the total number of features in the state data, and θ represents the length of the sliding time window. For any j-th feature, a heuristic mutation point detection algorithm is used to detect mutation points. The specific process is as follows:

[0107] a. For the eigenvalue sequence A subsequence of a certain eigenvalue m in The data at a certain time point k

[0108] b. The data on the left and right sides are divided into two new feature group sequences and

[0109] The sequence lengths are denoted as N1 and N2 respectively;

[0110] c. Calculate the mean μ1, μ2 and standard deviation σ1, σ2 of the two new sequences respectively;

[0111] d. Calculate the combined deviation at time point k:

[0112]

[0113] e. Calculate the t-test statistic T at time point k k :

[0114]

[0115] f. For the eigenvalue sequence Repeat the above process for each point in the calculation, and calculate all the t-test statistics T iT sequence, calculate the maximum value T in t sequence max Statistical significance of T

[0116]

[0117] Characteristic value subsequence Length is theta, v = theta-2, delta = 0.4, eta = 0.2

[0118] 4.19(n)-11.54.If its statistical characteristics P(T max ) >= P0(P0 is a preset parameter), the point is the mutation segmentation point of the sequence, otherwise, it is not segmented.Statistical characteristic value subsequence

[0119] All mutation time points exist;

[0120] For each feature in the state data sequence, after performing the above steps, the set of all characteristic values of the existing mutation points and the respective mutation time points can be obtained.

[0121] It should be noted that the heuristic mutation point detection algorithm is selected for mutation point detection here, which is only one optional implementation of the embodiment and should not be understood as the only limitation of the present application. Other methods that can be used for mutation point detection of the sequence can also be applied to the present application.

[0122] In order to further improve the prediction accuracy of the multi-agent reinforcement learning model, as shown in Figure 2 If the current execution strategy belongs to a false alarm or a false alarm, the current health status of the storage device is returned to the multi-agent reinforcement learning model, and the multi-agent reinforcement learning model is retrained.

[0123] In order to further improve the interpretability of the model prediction, as shown in Figure 2 When it is determined that the current execution strategy is correct, the characteristic of the detected mutation point is sent to the storage system; usually, when the state of the storage device changes, the characteristic of the mutation can be regarded as the reason for the state transition of the storage device; in this embodiment, when it is determined that the current execution strategy is correct, the characteristic of the detected mutation point is sent to the storage system in addition to the execution strategy, so that the storage system can know the reason for the state transition of the storage device, and the interpretability of the model is further improved.

[0124] Overall, the above technical solutions provided by the embodiment simultaneously utilize the failure prediction information and the remaining life prediction information of the storage device to construct a health state of the storage device, maintain the present situation, backup data and replace the storage device as optional execution strategies, utilize the multi-agent reinforcement learning model to predict the corresponding execution strategy according to the health state of the storage device, and verify the execution strategy predicted by the multi-agent reinforcement learning model in combination with the mutation point detection result of the state data in the current moment and the sliding time window, and only send the execution strategy verified correctly to the storage system, which can ensure that the execution strategy is adapted to the health state of the storage device, effectively solve the technical problems of lack of processing decision, high misjudgment cost and poor interpretability of the existing failure prediction method, and improve the reliability of the storage device.

[0125] Embodiment 2

[0126] An execution strategy prediction device based on a health state of a storage device, comprising:

[0127] A computer readable storage medium for storing a computer program;

[0128] and a processor for reading the computer program stored in the computer readable storage medium, executing the execution strategy prediction method based on the health state of the storage device provided in the above embodiment 1.

[0129] Embodiment 3

[0130] A storage system comprising a storage device and the execution strategy prediction device based on the health state of the storage device provided in the above embodiment 2.

[0131] Those skilled in the art will readily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for predicting execution strategies based on the health status of storage devices, characterized in that: include: Performing a fault prediction and a remaining life prediction on the storage device based on the current state data of the storage device, wherein the fault prediction information and the remaining life prediction information constitute a health state of the storage device; the state data includes a plurality of features associated with the health state of the storage device; Inputting the health status of the storage device into a multi-agent reinforcement learning model, and having the multi-agent reinforcement learning model output a corresponding execution strategy; the execution strategy is to maintain the status quo, back up data, or replace the storage device; Perform mutation point detection on each feature in the state sequence composed of the state data of the storage device at the current moment and at each moment in the previous sliding time window. If the current execution strategy is to maintain the status quo but a mutation point is detected, the current execution strategy is determined to be a false alarm; if the current execution strategy is to back up data or replace the storage device but no mutation point is detected, the current execution strategy is determined to be a false alarm; in other cases, the current execution strategy is determined to be correct and reported to the storage system so that the storage system performs the corresponding operation; In the multi-agent reinforcement learning model, the feedback function Reward for the execution strategy of maintaining the status quo is Keep as follows: The execution strategy for backup data, its feedback function Reward Backup as follows: The execution strategy for replacing storage devices, its feedback function Reward Replace as follows: Among them, bestKeep represents the retention time point, bestBackup represents the best data backup time point, bestReplace represents the best replacement time point, and x represents the actual time point; a, b, c, d, and f all represent the change rate; maxKeep, maxBackup, and maxReplace represent the maximum feedback values ​​corresponding to maintaining the status quo, data backup, and replacing the storage device, respectively.

2. The execution strategy prediction method based on the health status of a storage device according to claim 1, characterized in that: Also includes: If the current execution strategy is a missed alarm or a false alarm, the current health status of the storage device is transmitted back to the multi-agent reinforcement learning model to retrain the multi-agent reinforcement learning model.

3. The execution strategy prediction method based on the health status of a storage device according to claim 1, characterized in that: The fault prediction information in the health status of the storage device is a two-tuple: in, Indicates storage device m At time t i Fault prediction information; Indicates storage device m At time t i The fault prediction results, Indicates storage device m At time t i The probability of being predicted as normal, Indicates storage device m At time t i The probability of being predicted to fail, α and β Respectively represent the weight of each contribution factor in the binary group; 、 and All fault prediction models are trained based on the storage device m At time t i The state data prediction output.

4. The execution strategy prediction method based on the health status of a storage device according to claim 1 or 3, characterized in that: The remaining life prediction information in the health status of the storage device is a two-tuple: in, Indicates storage device m At time t i Remaining life prediction information; Indicates storage device m At time t i The remaining life prediction result is obtained by the trained remaining life prediction model based on the storage device m At time t i The state data prediction output; express The exponential moving average of represents the predicted maximum remaining life; γ and λ Represent the weight of each contribution factor in the binary group.

5. The execution strategy prediction method based on the health status of a storage device according to claim 1, characterized in that: The status data includes: reliability indicators of the storage device's operating status, performance information of the storage device, and error information associated with the fault.

6. The execution strategy prediction method based on the health status of a storage device according to claim 5, characterized in that: The reliability index of the storage device operating status is extracted from the storage device's SMART data, the performance information of the storage device is extracted from the storage device's I / O status information, and the error information associated with the fault is extracted from the storage device's error log.

7. The execution strategy prediction method based on the health status of a storage device according to claim 1 or 2, characterized in that: Also includes: When it is determined that the current execution strategy is correct, the features of the detected mutation point are sent to the storage system.

8. An execution strategy prediction device based on the health status of a storage device, characterized in that: include: a computer-readable storage medium for storing a computer program; and a processor, configured to read the computer program stored in the computer-readable storage medium and execute the execution strategy prediction method based on the health status of the storage device according to any one of claims 1 to 7.

9. A storage system, characterized in that: include: A storage device, and the execution strategy prediction device based on the health status of the storage device as described in claim 8.

Images