Storage resource maintenance method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a storage resource maintenance method and device, electronic equipment and a readable storage medium, and relates to the technical field of storage. The method comprises the following steps: inputting respective running state parameter information and corresponding state trend prediction information of a first type of storage equipment and a second type of storage equipment into a tree model for predicting faults through an online learning mode to obtain storage resource fault prediction information. Whenever it is determined that storage resources to be maintained exist according to the operation state parameter information and / or the storage resource fault prediction information, tasks to be maintained are generated, and task priorities and task deadlines are determined based on the storage resource fault prediction information of the tasks to be maintained and the original states of the tasks; and according to the task priority and the task deadline of each to-be-maintained task, determining a maintenance sequence of each to-be-maintained task. The problem that related technologies are difficult to popularize on a large scale in a production environment can be solved, storage resources can be efficiently maintained, and the deployment and reasoning cost is low.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a storage resource maintenance method, device, electronic device and computer-readable storage medium. Background Art

[0002] As information technology becomes widely used in our daily lives, to meet the demand for secure electronic data storage, related technologies use machine learning models to train models with predictive failure capabilities. This allows for early prediction of storage device failures in computer systems, enabling timely maintenance and ensuring the security of electronic data. However, this approach is costly to deploy and infer, making it difficult to scale in production environments. Summary of the Invention

[0003] The present invention provides a storage resource maintenance method, device, electronic device, computer-readable storage medium and computer program product. By accurately predicting storage device failures, storage resource maintenance can be efficiently completed, thereby effectively improving data security. In addition, the deployment and inference costs are low, making it suitable for large-scale promotion.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: In one aspect, the present invention provides a storage resource maintenance method, comprising: The operating status parameter information of each of the first type of storage device and the second type of storage device and their corresponding status trend prediction information are input into a tree model that predicts faults through an online learning mode to obtain storage resource fault prediction information; wherein the status trend prediction information is the data smoothing result of the operating status parameter information.

[0005] Whenever it is determined based on the operating status parameter information and / or the storage resource fault prediction information that there are storage resources to be maintained, a maintenance task is generated, and the task priority and task deadline are determined based on the storage resource fault prediction information and the original status of each maintenance task.

[0006] Determine the maintenance order of each maintenance task based on its priority and deadline.

[0007] Another aspect of the present invention provides a storage resource maintenance device, comprising: The fault prediction module is used to input the operating status parameter information of the first type storage device and the second type storage device and their corresponding status trend prediction information into a tree model that predicts faults through an online learning mode to obtain storage resource fault prediction information.

[0008] The storage resource maintenance module generates a task to be maintained whenever it is determined based on the operating status parameter information and / or the storage resource fault prediction information that there are storage resources to be maintained, and determines the task priority and task deadline based on the storage resource fault prediction information and the original status of each task to be maintained; and determines the maintenance order of each task to be maintained based on the task priority and task deadline of each task to be maintained.

[0009] The present invention also provides an electronic device comprising a memory and a processor, wherein the processor is configured to implement the steps of any of the above-mentioned storage resource maintenance methods when executing a computer program stored in the memory.

[0010] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the steps of any of the above-mentioned storage resource maintenance methods.

[0011] The advantage of the technical solution provided by the present invention is that, by using the operating status parameter information of each storage device to perform real-time trend prediction and failure probability assessment, potential storage device failures can be discovered hours to days in advance, and maintenance can be scheduled in a timely manner before the failure occurs, effectively extending the service life of the storage device, effectively reducing unplanned downtime, and ensuring the storage security and stability of electronic data. Using real-time trend prediction and incremental decision trees to predict storage device failures can not only accurately predict storage device failures, but also has lower deployment and inference costs than deep learning network models, which is conducive to large-scale promotion in production environments. Maintenance work orders are dynamically allocated based on failure probability, effectively avoiding large-scale blind inspections and maintenance, avoiding resource waste, and reducing computing and deployment costs by approximately 25%. The optimal maintenance sequence is generated by integrating failure risk, task priority, and task urgency. This not only improves the efficiency of storage resource maintenance, but also effectively improves resource utilization and response speed.

[0012] In addition, the present invention also provides corresponding implementation devices, electronic devices and computer-readable storage media for the storage resource maintenance method, further making the method more practical, and the devices, electronic devices and computer-readable storage media have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1A schematic diagram of the hardware composition framework applicable to the storage resource maintenance method provided by the present invention; Figure 2 A flowchart of a storage resource maintenance method provided by the present invention; Figure 3 A flowchart of another storage resource maintenance method provided by the present invention; Figure 4 A structural framework diagram of an exemplary embodiment of the storage resource maintenance device provided by the present invention; Figure 5 This is a structural diagram of an exemplary embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. The terms "first," "second," "third," "fourth," etc. in the specification and the accompanying drawings are used to distinguish different objects rather than to describe a specific order. Furthermore, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. The term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0016] With the rapid development of cloud computing, big data, artificial intelligence, and the Internet of Things, data centers and various business information systems are placing increasingly stringent demands on the data security of storage devices. Data security refers to protecting the confidentiality, integrity, and availability of data. Hard drives, as persistent storage media, have a lifespan and performance that directly impacts data security and system availability. Memory, the core of efficient caching and computational acceleration, has a health impact on application throughput and response latency. For example, in downtime-sensitive scenarios such as finance, healthcare, telecommunications, and manufacturing, a failure in any hard drive or memory can result in data loss, transaction interruption, or even production line downtime.

[0017] To ensure the security of stored data, traditional methods rely on regular manual inspections or testing storage devices like hard drives and memory devices to see if their monitoring values ​​exceed manufacturer-recommended thresholds. For example, for hard drives, this can be done by checking whether the SMART (Self-Monitoring, Analysis, and Reporting Technology) indicator exceeds a fixed value. However, this approach suffers from high false positive rates, delayed response times, and frequent false positives. Once a failure occurs, data recovery is often impossible, or the only solution is to shut down the system and replace the drive. Furthermore, triggering alarms or maintenance only when hard drive temperature, SMART indicators, or memory usage exceed static thresholds fails to predict risks, resulting in high false positive and false negative rates and failing to truly guarantee the security of electronic data. With the rapid development of artificial intelligence (AI), related technologies use complex machine learning models, such as deep neural networks, to predict memory or hard drive failures. However, this approach is often difficult to scale in production environments due to its black-box nature, complex deployment, large data requirements, and high resource consumption.

[0018] In view of this, the present invention performs status trend prediction on the operating status parameter information of each of the first type of storage device and the second type of storage device, and uses the operating status parameter information of the two and their status trend prediction as input features, and inputs them together into the incremental decision tree for fault prediction to obtain storage resource fault prediction information. Whenever it is determined that there are storage resources to be maintained based on the operating status parameter information of the storage device and the storage resource fault prediction information, a task to be maintained is generated. Based on the storage resource fault prediction information of the current task to be maintained and the original state of the task, the maintenance order of each task to be maintained is determined according to the task priority and task deadline. By accurately predicting storage device failures, storage resource maintenance can be completed efficiently, thereby effectively improving data security, and is suitable for large-scale promotion. In combination with the specific application environment architecture or specific hardware architecture on which the execution of the storage resource maintenance method depends, the specific application environment architecture or specific hardware architecture is described here. The following is combined with Figure 1 Some possible application scenarios of the technical solution of the present invention are introduced as examples. For servers that handle extremely downtime-sensitive services, a computer program that implements the following technical solution can be implanted in the operating platform or operation and maintenance platform, which may include the following contents: Monitor all hard drives and memory sticks on the server, collect operating status parameter information for each hard drive and memory stick using the same data collection frequency, and store the collected operating status parameter information in a database. Read the operating status parameter information of the hard drives and memory sticks collected at the same time from the database in sequence, perform a status trend prediction on the operating status parameter information of both, and use the operating status parameter information and its status trend prediction as input features, inputting them together into an incremental decision tree for fault prediction to obtain storage resource fault prediction information. Whenever operating status parameter information is read from the database, a determination is made as to whether the storage device corresponding to the read operating status parameter requires maintenance. Whenever storage resource fault prediction information is generated, a determination is made as to whether a storage resource to be maintained exists based on the operating status parameter information of the storage device and / or the storage resource fault prediction information, and a maintenance task is generated. Based on the storage resource fault prediction information of the current task to be maintained and the original status of the task, the maintenance order of each task to be maintained is determined according to the task priority and task deadline. Use EDF (earliest deadline first) for time window allocation: reserve the lowest business hours every day to perform heavy-load maintenance, and handle maintenance tasks in other time periods. Finally, within each maintenance window (such as 02:00-04:00 at night), the maintenance tasks are allocated in sequence on the basis of not exceeding the maximum number of parallel tasks at any time.

[0019] It should be noted that the above application scenarios are only shown to facilitate understanding of the ideas and principles of the present invention, and the embodiments of the present invention are not limited in this respect. On the contrary, the embodiments of the present invention can be applied to any applicable scenario. After introducing the technical solution of the present invention, various non-limiting embodiments of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments. First, please refer to Figure 2 , Figure 2 This is a flowchart of a storage resource maintenance method provided in this embodiment. This embodiment may include the following contents: S201: Inputting the operating status parameter information of the first type storage device and the second type storage device and their corresponding status trend prediction information into a tree model for predicting faults through an online learning mode to obtain storage resource fault prediction information.

[0020] In this step, the first type of storage devices and the second type of storage devices refer to different types of storage devices in the computer system, such as hard disks and memory devices. System monitoring tools can be used to continuously collect operating data of the storage devices from sensors and system interfaces, such as hard disk SMART indicators and memory usage data. For ease of description, the collected operating data is defined as operating status parameter information, which may include, for example, the number of remapped sectors (Reallocated_Sector_Ct), the number of pending sectors (Current_Pending_Sector), power-on time (Power_On_Hours), read and write latency (Average_Latency), usage rate (Used / Total, memory percentage), the number of pages loaded per second (Page_Reads / sec, representing hard fault pages), and the number of pages loaded per second (Page_Writes / sec).

[0021] Among them, the state trend prediction information is the result of data smoothing of the operating state parameter information, that is, data smoothing is performed on each state parameter in the operating state parameter information, which can automatically attenuate the impact of old data on the trend, and can also take into account sensitivity to sudden changes and filtering of noise. The smoothing result can be used as the current estimate of the operating state trend of the storage device, and the smoothing result and the original parameter data can be used together as input features for subsequent abnormality or fault detection, effectively improving the prediction accuracy. The tree model in this step can integrate any lightweight tree model that supports online learning models. Its base learner does not need to store historical data, such as online decision trees such as gradient boosting decision trees, incremental decision trees such as Hoeffding trees, and lightweight LightGBM models (boosting trees) suitable for real-time prediction and large-scale online learning. As long as it meets the lightweight requirements and supports online learning, the present invention does not impose any restrictions on this. The collected time series data is first smoothed and de-noised to extract trends. A lightweight tree model is then used to calculate the probability of failure at each moment in real time. This approach not only captures time series trends but also maintains model deployability, eliminating the need for a deep network. This reduces deployment and inference costs, and provides accurate risk input for subsequent maintenance scheduling. Storage resource failure prediction information is the result of dual-characteristic anomaly predictions for first- and second-category storage devices, representing comprehensive failure prediction data. Of course, those skilled in the art may also configure third- and fourth-category storage devices based on actual needs, without affecting the implementation of the present invention.

[0022] S202: Whenever it is determined based on various operating status parameter information and / or storage resource failure prediction information that there are storage resources to be maintained, a maintenance task is generated.

[0023] After collecting the operating status parameter information of various storage devices, the operating status information can be analyzed to determine whether the corresponding storage device has a risk of failure. For example, if the hard disk power_on_hours exceeds 20,000 hours or the memory used_rate is continuously above 90%, it is considered that there is a risk of failure, and a maintenance task can be triggered to be generated. After the storage resource failure prediction information is calculated through the above steps, if the storage resource failure prediction information reflects a high probability of failure, it can also directly trigger the generation of a maintenance task. If the failure prediction probability exceeds 0.8, it is considered to be in a "high risk" state. In addition, the operating status parameter information used to calculate the storage resource failure prediction information can be combined to determine whether there is a failure. For example, if the failure prediction probability exceeds 0.6, the number of remapped sectors accumulates to more than 10, or the number of pending sectors is continuously greater than 5, then the generation of a maintenance task is triggered. For example, the present invention can set corresponding trigger conditions for different trigger scenarios. The trigger conditions can be set according to actual conditions, such as the first type of preset maintenance condition, the second type of preset maintenance condition, and exceeding a specified threshold. The present invention does not impose any restrictions on this. When there is a target storage device whose failure probability value exceeds the preset single device failure threshold, such as 0.8, a corresponding maintenance task is generated for the target storage device; when the storage resource failure prediction information exceeds the preset comprehensive failure threshold, such as 0.6, and the target state parameters in the corresponding operating state parameter information, such as the number of remapped sectors or the number of pending sectors, meet the first type of preset maintenance conditions, such as the cumulative number of remapped sectors exceeds 10 or the number of pending sectors is continuously greater than 5, corresponding maintenance tasks are generated for the first type of storage device and the second type of storage device; when the first target state parameter of the first type of storage device, such as the power-on time of the hard disk, meets the second type of preset maintenance conditions, such as the power-on time exceeds the specified time threshold, a corresponding maintenance task is generated for the first type of storage device; when the second target state parameter of the second type of storage device, such as the memory usage rate, meets the third type of preset maintenance conditions, such as the memory usage rate exceeds the specified usage threshold, a corresponding maintenance task is generated for the second type of storage device.

[0024] S203: Determine the task priority and task deadline based on the storage resource failure prediction information and the original status of the task to be maintained. Determine the maintenance order of each task to be maintained based on the task priority and task deadline of each task to be maintained.

[0025] The above steps will generate multiple tasks to be maintained. This step will determine the corresponding priority and deadline for each task to be maintained. For ease of description, the task to be maintained that is being processed is defined as the current task to be maintained. According to the trigger generation conditions of the tasks to be maintained in the above steps, some tasks to be maintained do not have storage resource failure prediction information. In the steps, for tasks to be maintained that do not have storage resource failure prediction information, a default value such as 0 or other pre-set fill value can be used. The original state of the task refers to some states and configurations of the storage device corresponding to the task to be maintained, such as the maintenance time of this type of task, the urgency of this type of task to be maintained, and the balance between the failure risk of the storage device corresponding to the task to be maintained and the urgency of the task. The task priority and task deadline can be determined based on the original state of the task and the storage resource failure prediction information. After determining the task priority and task deadline, these tasks are sorted. The sorting can be in ascending or descending order, which does not affect the implementation. However, the order of issuing maintenance work orders should be that the tasks with earlier task deadlines and higher task priorities are issued first. The maintenance work of each maintenance task is completed before the task deadline. The maintenance tasks are sorted according to the failure risk, task priority, and deadline, and maintenance work orders are dynamically allocated to avoid large-scale blind inspection and maintenance, thereby improving the accuracy and efficiency of storage resource maintenance.

[0026] In the technical solution provided in this embodiment, real-time trend prediction and failure probability assessment are performed based on the operating status parameter information of each storage device. This allows potential storage device failures to be discovered hours to days in advance, and maintenance can be scheduled in a timely manner before a failure occurs, effectively extending the service life of the storage device, effectively reducing unplanned downtime, and ensuring the storage security and stability of electronic data. Failure prediction for storage devices using real-time trend prediction and incremental decision trees not only accurately predicts storage device failures, but also has lower deployment and inference costs than deep learning network models, facilitating large-scale promotion in production environments. Maintenance work orders are dynamically allocated based on failure probability, effectively avoiding large-scale blind inspections and maintenance, and wasting resources. Computational and deployment costs are reduced by approximately 25%. Failure risk, task priority, and task urgency are integrated to generate an optimal maintenance sequence. This not only improves the efficiency of storage resource maintenance, but also effectively enhances resource utilization and response speed.

[0027] In the above embodiment, there is no limitation on how to process the operating state parameter information into state trend prediction information through smoothing operation. Based on the above embodiment, the present invention also provides an exemplary implementation method, such as Figure 3 As shown, it may include the following: Collect current operating status parameter information of the first type of current storage device and the second type of current storage device at the current moment and historical status trend prediction information at the previous moment; obtain the first historical operating status parameter of the first type of current storage device and its corresponding first historical smoothed prediction information, the second historical operating status parameter of the second type of current storage device and its corresponding second historical smoothed prediction information; determine the first historical prediction error between the first historical smoothed prediction information and the first historical operating status parameter, and the second historical prediction error between the second historical operating status parameter and the second historical smoothed prediction information, to achieve a smoothing coefficient when minimized, and adjust the smoothing coefficient according to data processing requirements to obtain a smoothing factor at the current moment; determine the state smoothing value corresponding to each state parameter according to the smoothing factor, each state parameter of the current operating status parameter information and its corresponding historical state smoothing value, and use the state smoothing value of each state parameter of the current operating status parameter information as the corresponding state trend prediction information.

[0028] In this embodiment, in the initial stage, since there is no state trend prediction information from the previous moment, the current operating state parameter information can be used as the state trend prediction information. For example, the value x0 of the first collected reallocated_sectors is used as s0, that is, s0 = x0. To adapt to the characteristics of different hard disk models and memory, the most suitable smoothing factor can be determined before smoothing. The first historical operating state parameter is the operating state parameter information of the first type of current storage device at a certain moment in the past, such as moment A. The first historical smoothed prediction information is the state trend prediction information corresponding to the operating state parameter at the previous moment of the first historical operating state parameter. That is, the first historical operating state parameter is the actual operating data of the first type of current storage device at moment A. The first historical smoothed prediction information is the data obtained by performing data smoothing processing on the operating state parameter information collected at moment A-1. It is the predicted value of the operating state of the first type of current storage device at moment A. For ease of description, the smoothing factor used in the first historical smoothed prediction information is defined as a smoothing coefficient. The second historical operating state parameter is the operating state parameter information of the second type of current storage device at a certain moment in the past. The second historical smoothed prediction information is the state trend prediction information corresponding to the operating state parameter at the previous moment of the second historical operating state parameter. For example, the historical prediction error SSE can be calculated by the following relationship, and the optimal smoothing coefficient is determined by minimizing the historical prediction error: ; in, For the The actual observation at the moment, that is, the first historical operating state parameter or the second historical operating state parameter, It is the first historical smoothed prediction information or the second historical smoothed prediction information. The smaller the SSE, the more accurate the trend prediction. If the data processing requirement is, for example, to respond to sudden changes faster, the optimal smoothing coefficient can be appropriately increased. If the data processing requirement is, for example, to have stronger noise filtering capabilities, the optimal smoothing coefficient can be reduced to the optimal smoothing coefficient. The optimal smoothing coefficient adjusted by the data processing requirements is defined as the smoothing factor, which takes into account both sensitivity to sudden changes and noise filtering on the basis of adapting to different models of hard disks and memory characteristics. For example, the optimal smoothing coefficient is any value between 0.2-0.3, which can balance the weights of new and old data. If you want to respond to sudden changes faster, you can increase it to 0.4. If you seek stronger noise filtering capabilities, you can reduce it to 0.1. As a more accurate data smoothing process, exponentially weighted moving average can be used for data smoothing. For any state parameter in the current operating state parameter information, it can be based on Smoothing the current operating state parameters, is the smoothing factor, is the value of one of the state parameters in the current operating state parameter information at time t, The historical state smoothing value corresponding to the historical state trend prediction information of the operating state parameter at the previous time t-1, the historical state trend prediction information includes the historical state smoothing value corresponding to each state parameter; is the state smoothing value corresponding to the state parameter, and the state smoothing values ​​of all state parameters constitute the state trend prediction information. This method can automatically reduce the impact of old data on the trend. For example, if pending_sectors has always been 0 and suddenly soars to 5, then It will follow the rise within a few minutes, indicating a possible malfunction.

[0029] In order to further improve the prediction accuracy, for the same type of storage devices, the smoothing factor can be adjusted regularly, such as once a quarter, based on historical data according to the above method.

[0030] As can be seen from the above, this embodiment adjusts the smoothing factor to accommodate the characteristics of different hard drive models and memory, while also balancing sensitivity to sudden changes and noise filtering. Exponential smoothing is first used to remove noise from the collected time series data to extract trends. Then, a lightweight tree fault model is used to calculate the failure probability at each moment in real time, providing accurate risk input for scheduling.

[0031] Based on the above embodiment, the present invention also provides an exemplary method for determining storage resource fault prediction information, such as Figure 3 As shown, in this embodiment, the first type of storage device is a hard disk, and the second type of storage device is a memory. The process of determining the comprehensive fault prediction information of the hard disk and the memory may include the following: Obtain the first operating status parameter information of the current hard disk at the current moment and the second operating status parameter information of the current memory at the current moment; input the first operating status parameter information and its corresponding status trend prediction information into a tree model that predicts faults through an online learning mode, and determine the failure probability value of the current hard disk according to the failure prediction probability of each tree of the tree model; input the second operating status parameter information and its corresponding status trend prediction information into a tree model that predicts faults through an online learning mode, and determine the failure probability value of the current memory according to the failure prediction probability of each tree of the tree model; determine the storage resource failure prediction information according to the hard disk weight factor, the memory weight factor, the failure probability value of the current hard disk and the failure probability value of the current memory.

[0032] In this embodiment, the following SMART raw attributes can be collected from the hard disk at a frequency of 1 minute: the number of remapped sectors (Reallocated_Sector_Ct), the number of pending sectors (Current_Pending_Sector), the power-on duration (Power_On_Hours), and the read and write latency (Average_Latency) and other indicators, and written to the time series database (ULINK Technology). To further improve data processing efficiency, host memory indicators are also collected at a frequency of 1 minute: usage (Used / Total memory percentage), page loads per second (Page_Reads / sec, representing hard fault pages), and page loads per second (Page_Writes / sec). To facilitate processing and improve the accuracy of subsequent data processing, the above-collected data can be globally synchronized, missing values ​​supplemented, and formatted according to timestamps to ensure data accuracy, traceability, and unified format. The data is then stored in a time series database through database tables for subsequent processing and analysis. The time series database includes disk_usage (hard disk usage data table) and memory_usage (memory usage data table). Disk_usage (hard disk usage data table) includes: disk_id (disk ID), timestamp (collection time), reallocated_sectors, pending_sectors, power_on_hours, and avg_latency; memory_usage (memory usage data table) can include host_id (host ID), imestamp, used_rate, page_reads_per_sec, and page_writes_per_sec, as shown in Tables 1 and 2 below: Table 1 Hard disk usage data Table 2 Memory usage data table

[0033] The first operating state parameter information and its corresponding state trend prediction information are used as an input feature. The tree model can be an incremental decision tree, for example. B trees of the incremental decision tree are randomly selected for online update, such as 3-10 trees. When the number of trees is small, the model is lightweight. When the number of trees is large, the accuracy is improved but the resource consumption increases. The selection can be based on actual needs. When predicting, the failure probability of each tree i is output The average value of the final failure probability is taken as the final failure probability, that is, the failure probability value is determined according to the following relationship : .when or If it exceeds 0.5, it is judged as a "high risk" state and can be selected as a candidate for maintenance tasks. and the failure probability value of the memory , obtain their respective weight factors, for example, they can both be 0.5, and the failure probabilities of the hard disk and memory can be combined with equal weights , and obtain storage resource failure prediction information R. Of course, the weight factor can be determined according to the actual situation. If the hard disk data is more important, a larger hard disk weight factor can be set. If the memory is more important, a larger memory weight factor can be set.

[0034] To further improve the effectiveness of storage resource failure prediction, you can further set multiple fault levels, such as R ≥ 0.6 for emergency, R < 0.3 for normal, and 0.3 ≤ R < 0.6 for warning. When the number of reallocated_sectors accumulated from the start to the current moment exceeds 10 or pending_sectors is continuously greater than 5, and R ≥ 0.6, a maintenance task is triggered.

[0035] As can be seen above, this embodiment integrates the failure probabilities of hard disk and memory resources according to preset weights to generate a comprehensive risk score. Alerts are triggered based on three threshold levels: normal, warning, and emergency, and delivered to operations and maintenance personnel in real time, improving storage resource maintenance effectiveness. By performing real-time trend prediction and failure probability assessment on multi-dimensional monitoring data such as the number of remapped sectors, the number of pending sectors, and the rate of memory page in / out, potential failures can be detected hours to days in advance, allowing maintenance to be scheduled before they occur. This effectively reduces unplanned downtime and extends the lifespan of hard disks and memory. By focusing on these two critical resources, prediction accuracy and interpretability are guaranteed.

[0036] In order to improve data processing efficiency and save computing resources and storage resources, based on the above embodiment, the following contents may also be included: Set up an initial data collection window, and collect the operating status parameter information of the first type of storage device and the second type of storage device at each collection moment within the initial data collection window, and the historical status trend prediction information of the previous moment; splice the operating status parameter information and status trend prediction information at each moment into a multi-dimensional feature vector, and input it into the Hoeffding tree, maintain the statistical summary information of each feature of the multi-dimensional feature vector, and calculate the splitting gain of each feature, use the Hoeffding inequality to calculate the splitting threshold, determine the number of target features whose splitting gain is greater than the splitting threshold, and adjust the initial data collection window according to the number of target features. For example, the initial data collection window is the last 5 sampling points, and the status trend prediction information is spliced ​​into the last 5 sampling points. and operating status parameter information Directly concatenate them into a 10-dimensional feature vector and input it into the Hoeffding tree. When the number of target features is determined to be 1, adjust the most recent 4 sampling points as the data collection window. This helps reduce memory usage, save computing and storage resources, and further reduce deployment and inference costs.

[0037] The above embodiment does not impose any limitation on how to determine the task priority and task deadline of each maintenance task. The present invention also provides an exemplary implementation method, which may include the following content: Determine the task urgency parameter, the balance parameter between the task urgency and the storage device risk, and the initial task deadline based on the original task status of the current task to be maintained; determine the risk control offset coefficient of the current task to be maintained based on the storage resource failure prediction information corresponding to the current task to be maintained, and calculate the product of the risk control offset coefficient and the storage resource failure prediction information to obtain a virtual deadline; calculate the difference between the initial task deadline and the virtual deadline as the task deadline of the current task to be maintained; calculate a first difference between the balance parameter and the first target value, and a second difference between the storage resource failure prediction information corresponding to the current task to be maintained and the second target value, and calculate the product of the first difference and the second difference to obtain a priority adjustment value; calculate the product of the balance parameter and the task urgency parameter to obtain an initial priority, and use the sum of the initial priority and the priority adjustment value as the task priority.

[0038] In this embodiment, the task urgency parameter is used to express the importance of a maintenance task. The value range of this value can be between [0, 1], with 1 indicating the highest urgency and requiring an immediate response. Different task urgency parameters can be set in advance for the task to be maintained, or the task urgency parameter can be assigned when the task to be maintained is generated. The task urgency parameter can be determined by any large language model using a pre-set calling interface and pre-set prompt words, or by obtaining a value input by the business department or operation and maintenance personnel through a human-computer interaction page. This does not affect the implementation of the present invention. The balance parameter is used to balance business urgency and equipment risk. The value range of this value can be between [0, 1]. The larger the value, the more emphasis is placed on task urgency, and the smaller the value, the more emphasis is placed on risk priority. It can also be set manually or generated through a prompt large model. The initial task deadline is the deadline of the original task to be maintained, which can also be set manually or generated through a prompt large model. The risk control offset coefficient is a value greater than 0, which is used to advance the virtual deadline based on the value of the storage resource failure prediction information to speed up the scheduling of high-risk maintenance tasks. It can also be set manually or generated by prompting a large model, for example When , the deadline of the full-risk task with R=1 will be moved forward by 2 hours. For example, the relationship Determine the task priority of the i-th maintenance task , is the balance parameter, is the task urgency parameter of the i-th task to be maintained, is the storage resource failure prediction information for the i-th maintenance task, reflecting the probability and potential impact of equipment failure. Indicates that the storage device corresponding to the i-th maintenance task is in good condition. Indicates that the storage device corresponding to the i-th maintenance task has a very high failure risk. Determine the deadline for the i-th maintenance task , is the initial deadline of the i-th task to be maintained, The risk control offset coefficient is used to give high-risk tasks an earlier deadline and thus obtain higher scheduling priority.

[0039] On this basis, the tasks to be maintained can be sorted in ascending order according to the time value based on the task deadline; for the first task to be maintained and the second task to be maintained that are in the same task deadline, the first task to be maintained and the second task to be maintained can be sorted in descending order according to the task priority values ​​of the first task to be maintained and the second task to be maintained; under the condition that the total number of tasks to be maintained processed in each maintenance time window at the same time is less than or equal to the maximum number of parallel tasks, the sorted tasks to be maintained will be sent to each maintenance time window in turn.

[0040] In this embodiment, the earliest deadline first (EDF) can be used to allocate time windows: a low-business period is reserved every day for heavy-load maintenance, and the remaining periods are filled according to the risk order, which can ensure that a solution that meets all task deadlines can always be found when the system is schedulable. Of course, you can also use task priority first. Perform a preliminary sorting of the tasks to be maintained. The larger the value, the higher the priority. After obtaining the task deadline, press Sort from small to large, for The same multiple maintenance tasks can be based on the above The descending order is further differentiated. The high-risk tasks have earlier deadlines in the EDF, thus obtaining higher scheduling priority, where The initial deadline for tasks. During each maintenance window (e.g., 2:00 AM–4:00 AM), tasks are assigned sequentially to ensure that the concurrent capacity of a single maintenance team does not exceed the system's maximum number of concurrent tasks at any given time, preventing resource overload and conflicts of responsibility.

[0041] As can be seen from the above, this embodiment uses EDF to sort maintenance tasks based on the failure risk and task priority of the storage device. It then automatically generates and issues maintenance work orders during appropriate time windows, such as during off-peak hours at night or during reserved flexible periods. This can improve maintenance response time by 30% and ensure that critical business systems are not affected during peak periods, significantly improving system availability and customer satisfaction. Based on priority queues and windowed maintenance time slot allocation, dynamic scheduling of maintenance tasks is achieved, balancing business urgency and failure risk, generating an optimal maintenance sequence, and improving resource utilization and response speed.

[0042] Furthermore, in order to improve the accuracy of fault prediction, based on the above embodiment, as Figure 3 As shown, it may also include the following: Whenever maintenance of the target task to be maintained is completed, the number of remapped sectors of the maintained hard disk or the number of pages loaded per second of the maintained memory corresponding to the target task to be maintained is obtained again; if the value of the number of remapped sectors of the maintained hard disk after multiple maintenance is the same as the value of the number of remapped sectors before maintenance, the smoothing factor value is reduced or the incremental decision tree is reset; if the false alarm rate of the number of pages loaded per second of the maintained memory after multiple maintenance exceeds the false alarm threshold, the smoothing factor value is reduced or the incremental decision tree is reset.

[0043] In this embodiment, after maintenance is completed, the actual occurrence of a failure and the duration of the maintenance are recorded and compared with the storage resource failure prediction information before scheduling to assess accuracy. For example, if the number of hard drive remapped sectors remains unchanged or the false alarm rate of memory page-in per second exceeds 20% after three consecutive maintenance operations, the smoothing factor is automatically lowered, such as from 0.25 to 0.2, or the online tree model is reset to avoid oversensitivity or excessive hysteresis. Through this closed loop, the smoothing factor and tree model are continuously calibrated, making predictive maintenance scheduling more accurate and efficient.

[0044] As can be seen from the above, this embodiment compares the actual execution results of each maintenance with the prediction, adaptively and dynamically adjusts the smoothing factor and tree model, and constructs a closed-loop iterative system of "prediction-execution-evaluation-optimization" to achieve end-to-end continuous optimization. As the data scale grows and the algorithm is iterated, the fault prediction accuracy and scheduling efficiency will continue to improve, achieving long-term sustainable value.

[0045] It should be noted that there is no strict order in which the steps in the present invention are performed. As long as they conform to a logical order, the steps can be performed simultaneously or in a predetermined order. Figure 2-Figure 3 This is just a schematic and does not mean that this is the only execution order.

[0046] The present invention also provides a corresponding device for the storage resource maintenance method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The storage resource maintenance device provided by the present invention is introduced below. The device is used to implement the storage resource maintenance method provided by the present invention. In this embodiment, the storage resource maintenance device may include or be divided into one or more program modules. The one or more program modules are stored in a storage medium and executed by one or more processors to complete the storage resource maintenance method disclosed in Example 1. The program module referred to in this embodiment refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of the storage resource maintenance device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The storage resource maintenance device described below and the storage resource maintenance method described above can be referenced to each other.

[0047] From the perspective of functional modules, see Figure 4 , Figure 4 The following is a structural diagram of a storage resource maintenance device provided in this embodiment in a specific implementation manner. The device may include: The fault prediction module 401 is used to input the operating status parameter information of the first type storage device and the second type storage device and their corresponding status trend prediction information into a tree model that predicts faults through an online learning mode to obtain storage resource fault prediction information.

[0048] The storage resource maintenance module 402 generates a task to be maintained whenever it is determined based on the operating status parameter information and / or storage resource fault prediction information that there are storage resources to be maintained, and determines the task priority and task deadline based on the storage resource fault prediction information and the original status of each task to be maintained; and determines the maintenance order of each task to be maintained based on the task priority and task deadline of each task to be maintained.

[0049] Exemplarily, in some implementations of this embodiment, the above-mentioned fault prediction module 401 can also be used to: collect the current operating status parameter information of the first type of current storage device and the second type of current storage device at the current moment and the historical status trend prediction information at the previous moment; obtain the first historical operating status parameter of the first type of current storage device and its corresponding first historical smoothed prediction information, the second historical operating status parameter of the second type of current storage device and its corresponding second historical smoothed prediction information; determine the first historical prediction error between the first historical smoothed prediction information and the first historical operating status parameter, and the second historical prediction error between the second historical operating status parameter and the second historical smoothed prediction information to achieve the smoothing coefficient when minimized, and adjust the smoothing coefficient according to data processing requirements to obtain the smoothing factor at the current moment; determine the state smoothing value corresponding to each state parameter according to the smoothing factor, each state parameter of the current operating status parameter information and its corresponding historical state smoothing value, and use the state smoothing value of each state parameter of the current operating status parameter information as the corresponding state trend prediction information.

[0050] Exemplarily, in some other implementations of this embodiment, the above-mentioned fault prediction module 401 can also be used for: the first type of storage device is a hard disk, and the second type of storage device is a memory, obtaining the first operating status parameter information of the current hard disk at the current moment and the second operating status parameter information of the current memory at the current moment; inputting the first operating status parameter information and its corresponding status trend prediction information into a tree model that predicts faults through an online learning mode, and determining the failure probability value of the current hard disk according to the failure prediction probability of each tree of the tree model; inputting the second operating status parameter information and its corresponding status trend prediction information into a tree model that predicts faults through an online learning mode, and determining the failure probability value of the current memory according to the failure prediction probability of each tree of the tree model; and determining the storage resource fault prediction information according to the hard disk weight factor, the memory weight factor, the failure probability value of the current hard disk and the failure probability value of the current memory.

[0051] Exemplarily, in some other implementations of this embodiment, the above-mentioned device may also include an optimization module, which can be used to: obtain the number of remapped sectors of the maintained hard disk or the number of pages loaded per second of the maintained memory corresponding to the target task to be maintained each time the maintenance of the target task to be maintained is completed; if the value of the number of remapped sectors of the maintained hard disk after multiple maintenance is the same as the value of the number of remapped sectors before maintenance, then reduce the value of the smoothing factor or reset the incremental decision tree; if the false alarm rate of the number of pages loaded per second of the maintained memory after multiple maintenance exceeds the false alarm threshold, then reduce the value of the smoothing factor or reset the incremental decision tree.

[0052] Illustratively, in some other implementations of this embodiment, the above-mentioned storage resource maintenance module 402 can also be used for: when there is a target storage device whose failure probability value exceeds the preset single device failure threshold, a corresponding maintenance task is generated for the target storage device; when the storage resource failure prediction information exceeds the preset comprehensive failure threshold, and the target state parameter in the corresponding operating state parameter information meets the first type of preset maintenance conditions, corresponding maintenance tasks are generated for the first type of storage device and the second type of storage device; when the first target state parameter of the first type of storage device meets the second type of preset maintenance conditions, a corresponding maintenance task is generated for the first type of storage device; when the second target state parameter of the second type of storage device meets the third type of preset maintenance conditions, a corresponding maintenance task is generated for the second type of storage device.

[0053] Illustratively, in some other implementations of this embodiment, the above-mentioned storage resource maintenance module 402 can also be used to: sort each task to be maintained in ascending order according to the time value based on the task deadline; for the first task to be maintained and the second task to be maintained that are in the same task deadline, sort the first task to be maintained and the second task to be maintained in descending order according to the task priority values ​​of the first task to be maintained and the second task to be maintained; and send the sorted tasks to be maintained to each maintenance time window in turn, under the condition that the total number of tasks to be maintained processed in each maintenance time window at the same time is less than or equal to the maximum number of parallel tasks.

[0054] Exemplarily, in some other implementations of this embodiment, the above-mentioned storage resource maintenance module 402 can also be used to: determine the task urgency parameter, the balance parameter between the task urgency and the storage device risk, and the initial deadline of the task based on the original task status of the current task to be maintained; determine the risk control offset coefficient of the current task to be maintained based on the storage resource failure prediction information corresponding to the current task to be maintained, and calculate the product of the risk control offset coefficient and the storage resource failure prediction information to obtain a virtual deadline; calculate the difference between the initial deadline of the task and the virtual deadline as the task deadline of the current task to be maintained; calculate the first difference between the balance parameter and the first target value, and the second difference between the storage resource failure prediction information corresponding to the current task to be maintained and the second target value, and calculate the product of the first difference and the second difference to obtain a priority adjustment value; calculate the product of the balance parameter and the task urgency parameter to obtain an initial priority, and use the sum of the initial priority and the priority adjustment value as the task priority.

[0055] The storage resource maintenance device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention in one implementation manner. The electronic device includes a memory 501 and a processor 502. The memory 501 stores a computer program, and the processor 502 is configured to run the computer program to perform the steps of any of the above-mentioned storage resource maintenance method embodiments.

[0056] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned storage resource maintenance method embodiments when running.

[0057] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0058] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned storage resource maintenance method embodiments are implemented.

[0059] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned storage resource maintenance method embodiments are implemented.

[0060] The above is a detailed introduction to a storage resource maintenance method, device, electronic device and computer-readable storage medium provided by the present invention. The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. Whether the units and algorithm steps of each example described in each disclosed embodiment are executed in electronic hardware or computer software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, and such implementation should not be considered to exceed the scope of the present invention. Without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A storage resource maintenance method, characterized in that: include: Inputting the operating status parameter information of each of the first type of storage device and the second type of storage device and the corresponding status trend prediction information thereof into a tree model for predicting faults through an online learning mode to obtain storage resource fault prediction information; wherein the status trend prediction information is a result of data smoothing of the operating status parameter information; Whenever it is determined based on the operating status parameter information and / or the storage resource fault prediction information that there is a storage resource to be maintained, a maintenance task is generated, and the task priority and task deadline are determined based on the storage resource fault prediction information and the original status of each maintenance task; Determine the maintenance order of each maintenance task based on its priority and deadline.

2. The storage resource maintenance method according to claim 1, characterized in that: The operating status parameter information of the first type of storage device and the second type of storage device and the corresponding status trend prediction information thereof include: Collecting current operating state parameter information of the first type of current storage device and the second type of current storage device at the current moment and historical state trend prediction information of the previous moment; the historical state trend prediction information includes the historical state smoothing value corresponding to each state parameter; Obtaining a first historical operating state parameter of a first type of current storage device and its corresponding first historical smoothed prediction information, and a second historical operating state parameter of a second type of current storage device and its corresponding second historical smoothed prediction information; determining a first historical prediction error between the first historical smoothed prediction information and the first historical operating state parameter, and a second historical prediction error between the second historical operating state parameter and the second historical smoothed prediction information, to achieve a smoothing coefficient when the coefficients are minimized, and adjusting the smoothing coefficients according to data processing requirements to obtain a smoothing factor at a current moment; According to the smoothing factor, each state parameter of the current operating state parameter information and its corresponding historical state smoothing value, the state smoothing value corresponding to each state parameter is determined, and the state smoothing value of each state parameter of the current operating state parameter information is used as the corresponding state trend prediction information.

3. The storage resource maintenance method according to claim 1, wherein: The first type of storage device is a hard disk, and the second type of storage device is a memory. Operational status parameter information of the first type of storage device and the second type of storage device and their corresponding status trend prediction information are input into a tree model that predicts faults through an online learning model to obtain storage resource fault prediction information, including: Obtaining first operating status parameter information of the current hard disk at the current moment and second operating status parameter information of the current memory at the current moment; Inputting the first operating status parameter information and its corresponding status trend prediction information into a tree model that predicts faults through an online learning mode, and determining a failure probability value of the current hard disk based on the failure prediction probabilities of each tree in the tree model; Inputting the second operating state parameter information and its corresponding state trend prediction information into a tree model that predicts faults through an online learning mode, and determining a fault probability value of the current memory according to the fault prediction probability of each tree in the tree model; Storage resource failure prediction information is determined according to the hard disk weight factor, the memory weight factor, the current hard disk failure probability value, and the current memory failure probability value.

4. The storage resource maintenance method according to claim 1, wherein: The tree model is an incremental decision tree, and further includes: Whenever maintenance of a target task to be maintained is completed, the number of remapped sectors of the maintained hard disk or the number of page loads per second of the maintained memory corresponding to the target task to be maintained is obtained again; If the number of remapped sectors of the maintained hard disk after multiple maintenances is the same as the number of remapped sectors before maintenance, then reduce the value of the smoothing factor or reset the incremental decision tree; If the false alarm rate of the number of page-in per second of the maintained memory after multiple maintenances exceeds a false alarm threshold, the smoothing factor value is reduced or the incremental decision tree is reset.

5. The storage resource maintenance method according to claim 1, wherein: Determining, based on the operating status parameter information and / or the storage resource failure prediction information, that there is a storage resource to be maintained includes: When there is a target storage device whose failure probability value exceeds a preset single device failure threshold, a corresponding maintenance task is generated for the target storage device; When the storage resource fault prediction information exceeds a preset comprehensive fault threshold, and the target state parameter in the corresponding operating state parameter information meets the first type of preset maintenance condition, a corresponding maintenance task is generated for the first type of storage device and the second type of storage device; When the first target state parameter of the first type of storage device meets the second type of preset maintenance condition, generating a corresponding maintenance task for the first type of storage device; When the second target state parameter of the second-type storage device meets the third-type preset maintenance condition, a corresponding maintenance task is generated for the second-type storage device.

6. The storage resource maintenance method according to claim 1, characterized in that: Determine the maintenance order of each maintenance task based on its priority and deadline, including: Determine multiple maintenance time windows based on business processing volume and minimum business reservation period; Based on the task deadlines, sort the tasks to be maintained in ascending order of time values; For a first task to be maintained and a second task to be maintained that are within the same task deadline, sorting the first task to be maintained and the second task to be maintained from largest to smallest according to the values ​​of their task priorities; Under the condition that the total number of tasks to be maintained processed by each maintenance time window at the same time is less than or equal to the maximum number of parallel tasks, the sorted tasks to be maintained are sent to each maintenance time window in sequence.

7. The storage resource maintenance method according to any one of claims 1 to 6, characterized in that: Determine the task priority and deadline based on the storage resource failure prediction information and original task status of each maintenance task, including: Determine the task urgency parameter, the balance parameter between task urgency and storage device risk, and the initial task deadline based on the original status of the task to be maintained; Determining a risk control offset coefficient for the current task to be maintained based on the storage resource failure prediction information corresponding to the current task to be maintained, and calculating a product of the risk control offset coefficient and the storage resource failure prediction information to obtain a virtual deadline; Calculating the difference between the initial deadline of the task and the virtual deadline as the task deadline of the current task to be maintained; Calculating a first difference between the balance parameter and a first target value, and a second difference between the storage resource failure prediction information corresponding to the current maintenance task and a second target value, respectively, and calculating a product of the first difference and the second difference to obtain a priority adjustment value; The product of the balance parameter and the task urgency parameter is calculated to obtain an initial priority, and the sum of the initial priority and the priority adjustment value is used as the task priority.

8. A storage resource maintenance device, characterized in that: include: A fault prediction module is configured to input the operating status parameter information of the first type of storage device and the second type of storage device and the corresponding status trend prediction information thereof into a tree model for predicting faults through an online learning mode, thereby obtaining storage resource fault prediction information; a storage resource maintenance module that generates a task to be maintained whenever it is determined based on the operating status parameter information and / or the storage resource fault prediction information that a storage resource to be maintained exists, and determines a task priority and a task deadline based on the storage resource fault prediction information and the original status of each task to be maintained; Determine the maintenance order of each maintenance task based on its priority and deadline.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the storage resource maintenance method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the storage resource maintenance method according to any one of claims 1 to 7 are implemented.

Images