Disk fault prediction method and device, electronic equipment, storage medium and product
By collecting multi-source heterogeneous data from disk storage nodes, generating feature sets, and performing local predictions, the problem of low prediction accuracy caused by a single data source is solved, and high-accuracy disk failure prediction is achieved in a distributed environment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA UNITED NETWORK COMM GRP CO LTD
- Filing Date
- 2026-01-15
- Publication Date
- 2026-04-28
AI Technical Summary
Existing disk failure prediction methods rely on a single data source, leading to reduced prediction accuracy in distributed disk storage clusters.
Collect multi-source heterogeneous data from disk storage nodes, generate multi-source feature sets, and call the inference model locally on the disk storage nodes to make predictions, generate edge prediction results, and generate disk failure prediction results by summarizing data and comparing with failure thresholds.
It improves the accuracy of disk failure prediction, avoids reliance on centralized machine learning models, and enhances predictive capabilities in distributed environments.
Smart Images
Figure CN121935104A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a disk failure prediction method, apparatus, electronic device, storage medium, and product. Background Technology
[0002] As data centers and cloud storage systems continue to expand, the number of disk devices is growing exponentially. As a core hardware component of storage systems, the reliability of disks directly determines the availability of data storage and business continuity. In storage clusters, a single disk failure can lead to data loss, service interruption, or performance degradation. Therefore, rapid prediction and proactive management of disk failures have become critical requirements for ensuring stable system operation.
[0003] In existing technologies, machine learning methods are used to predict disk failures. By setting thresholds for various disk metrics, a disk failure is determined when a metric exceeds the set threshold.
[0004] However, existing disk failure prediction methods rely on a single data source and the deployed machine learning models depend on centralized training, which reduces the accuracy of disk failure prediction in distributed disk storage clusters. Summary of the Invention
[0005] This application provides a disk failure prediction method, apparatus, electronic device, storage medium, and product to solve the problem of reduced accuracy in disk failure prediction in the prior art.
[0006] In a first aspect, embodiments of this application provide a disk failure prediction method, including:
[0007] Collect multi-source heterogeneous data from disk storage nodes;
[0008] Data processing is performed on the multi-source heterogeneous data of the disk storage node to generate a multi-source feature set;
[0009] The multi-source feature set is sent to the disk storage node, so that the disk storage node calls the local inference model to perform inference prediction on the multi-source feature set and generate edge prediction results.
[0010] The edge prediction results are aggregated to generate disk failure prediction information;
[0011] The disk failure prediction information is compared with the corresponding failure threshold to generate a disk failure prediction result.
[0012] In one possible implementation, the step of processing the multi-source heterogeneous data of the disk storage node to generate a multi-source feature set includes: synchronizing the timestamps of the multi-source heterogeneous data of the disk storage node to obtain time-aligned multi-source heterogeneous data; normalizing the time-aligned multi-source heterogeneous data to generate standardized data; and performing sliding window statistics and frequency domain transformation on the standardized data to obtain a multi-source feature set.
[0013] In one possible implementation, the normalization process for the time-aligned multi-source heterogeneous data to generate standardized data includes: acquiring runtime status data from the time-aligned multi-source heterogeneous data; determining weight adjustment values for disk runtime status features based on the runtime status data; adjusting the feature weights of the disk runtime status features using an online learning algorithm based on the weight adjustment values to obtain adjusted feature weights; and normalizing the time-aligned multi-source heterogeneous data based on the adjusted feature weights to generate standardized data.
[0014] In one possible implementation, after comparing the disk failure prediction information with the corresponding failure threshold to generate a disk failure prediction result, the method further includes: determining faulty disk information based on the disk failure prediction result; obtaining disk data to be migrated based on the faulty disk information; and sending the disk data to be migrated to the target disk to complete the data migration of the faulty disk.
[0015] In one possible implementation, before sending the disk data to be migrated to the target disk to complete the data migration of the faulty disk, the method further includes: obtaining status information of multiple healthy disks; determining the target disk based on the status information of the multiple healthy disks; creating a data transfer task for the disk data to be migrated based on the target disk; and performing a data migration operation based on the data transfer task.
[0016] In one possible implementation, the step of creating a data transfer task for the disk data to be migrated based on the target disk includes: creating a data transfer queue based on the data transfer priority in the disk data to be migrated; generating a preemptive scheduling policy based on the data transfer queue and system resources; and creating a data transfer task based on the preemptive scheduling policy and the data transfer queue.
[0017] Secondly, embodiments of this application provide a disk failure prediction device, comprising:
[0018] The data acquisition module is used to collect multi-source heterogeneous data from disk storage nodes;
[0019] The data processing module is used to process the multi-source heterogeneous data of the disk storage node and generate a multi-source feature set.
[0020] The edge inference module is used to send the multi-source feature set to the disk storage node, so that the disk storage node calls the local inference model to perform inference prediction on the multi-source feature set and generate edge prediction results;
[0021] The data aggregation module is used to aggregate the edge prediction results and generate disk failure prediction information;
[0022] The comparison module is used to compare the disk failure prediction information with the corresponding failure threshold to generate a disk failure prediction result.
[0023] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;
[0024] The memory stores computer-executed instructions;
[0025] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.
[0026] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.
[0027] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.
[0028] The disk failure prediction method, apparatus, electronic device, storage medium, and product provided in this application embodiment acquire multi-source heterogeneous data from disk storage nodes to construct a multi-source feature set, avoiding inaccurate disk failure prediction caused by a single data source; utilize a local inference model deployed on the disk storage node to predict failures locally, generate edge prediction results, summarize the edge prediction results, compare the summarized disk failure prediction information with the corresponding failure threshold, and generate disk failure prediction results. This avoids the problem of centralized training required for centralized machine learning models to predict disk failures, and improves the accuracy of disk failure prediction. Attached Figure Description
[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0030] Figure 1 This is a schematic diagram of the system structure of a computer device provided in an embodiment of this application;
[0031] Figure 2 Flowchart of the disk failure prediction method provided in this application Figure 1 ;
[0032] Figure 3 Flowchart of the disk failure prediction method provided in this application Figure 2 ;
[0033] Figure 4 Flowchart of the disk failure prediction method provided in this application Figure 3 ;
[0034] Figure 5 A schematic diagram of the disk failure prediction device provided in this application;
[0035] Figure 6 A schematic diagram of the structure of the electronic device provided in this application.
[0036] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0037] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0038] As data centers and cloud storage systems continue to expand, the number of disk devices is growing exponentially. As a core hardware component of storage systems, the reliability of disks directly determines the availability of data storage and business continuity. In storage clusters, a single disk failure can lead to data loss, service interruption, or performance degradation. Therefore, rapid prediction and proactive management of disk failures have become critical requirements for ensuring stable system operation. Current technologies use machine learning methods to predict disk failures. By setting thresholds for various disk metrics, a disk failure is determined when these metrics exceed the set thresholds. However, existing disk failure prediction methods rely on a single data source, and the deployed machine learning models depend on centralized training. In distributed disk storage clusters, the accuracy of disk failure prediction decreases.
[0039] To address the aforementioned technical problems, this application proposes the following technical concept: The inventors considered integrating multi-source heterogeneous data and constructing a multi-source feature set based on this data to avoid inaccurate disk fault prediction caused by a single data source. They also considered utilizing a local inference model deployed on disk storage nodes to predict faults locally on the disk, generating edge prediction results. The edge prediction results were then aggregated, and the aggregated disk fault prediction information was compared with the corresponding fault threshold to generate a disk fault prediction result. This avoids the problem of centralized training required for centralized machine learning models to predict disk faults, thus improving the accuracy of disk fault prediction.
[0040] Figure 1 This is a schematic diagram of the system architecture of a computer device provided in an embodiment of this application. Figure 1 As shown, the computer device includes: a receiving device 101, a processing device 102, and a display device 103.
[0041] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the disk failure prediction method. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.
[0042] In the specific implementation process, the receiving device 101 can be an input / output interface or a communication interface, and can collect multi-source heterogeneous data from disk storage nodes.
[0043] The processing unit 102 can generate disk failure prediction results.
[0044] The display device 103 can be used to display the above-mentioned disk failure prediction results, etc.
[0045] The display device can also be a touch screen, used to receive user commands while displaying the above content, so as to realize the operation interaction with the user.
[0046] It should be understood that the above-mentioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.
[0047] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.
[0048] Figure 2 Flowchart of the disk failure prediction method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:
[0049] S201: Collect multi-source heterogeneous data from disk storage nodes.
[0050] In this embodiment, the multi-source heterogeneous data includes, but is not limited to, SMART (Self-Monitoring, Analysis and Reporting Technology) parameters, I / O (Input / Output) performance parameters, distributed file system OSD (Object Storage Daemon) logs, and device temperature data.
[0051] Among them, SMART parameters include, but are not limited to, temperature, read / write error rate, and remapped sector count.
[0052] The contents recorded in the OSD log of the distributed file system include, but are not limited to, I / O request latency, heartbeat status, and object access error records.
[0053] Specifically, the task scheduling module transmits multi-source heterogeneous data to the data processing module.
[0054] In this embodiment, the functions of the task scheduling module include, but are not limited to, task distribution, concurrent execution control, and asynchronous collection of task results.
[0055] S202: Perform data processing on multi-source heterogeneous data of disk storage nodes to generate multi-source feature sets.
[0056] Specifically, timestamp synchronization is performed on the multi-source heterogeneous data of the disk storage nodes, the time-aligned multi-source heterogeneous data is normalized to generate standardized data, and sliding window statistics and frequency domain transformation are performed on the standardized data to obtain a multi-source feature set.
[0057] In this embodiment, the content recorded in the multi-source feature set includes, but is not limited to, static features, temporal features, and health decay index.
[0058] S203: Send the multi-source feature set to the disk storage node so that the disk storage node can call the local inference model to perform inference prediction on the multi-source feature set and generate edge prediction results.
[0059] In this embodiment, an edge inference module is deployed in the agent process of the disk storage node.
[0060] Specifically, the edge inference module periodically loads the model data, performs real-time predictions on local disk operation data, generates edge prediction results, and sends the edge prediction results to the central management node.
[0061] S204: Summarize the edge prediction results to generate disk failure prediction information.
[0062] Specifically, the data aggregation module aggregates the edge prediction results sent by multiple disk storage nodes to generate disk failure prediction information.
[0063] In this embodiment, the data aggregation module includes an anomaly filtering submodule and a fault prediction submodule.
[0064] The anomaly screening submodule detects abnormal disks based on a clustering algorithm.
[0065] The fault prediction submodule uses an ensemble learning algorithm to calculate the fault probability of edge prediction result samples.
[0066] S205: Compare the disk failure prediction information with the corresponding failure threshold to generate a disk failure prediction result.
[0067] Specifically, the fault prediction information for each disk is compared with a preset fault threshold. The fault threshold is a dynamic threshold that depends on factors such as disk type or business importance.
[0068] Specifically, for disks exceeding the "warning threshold", the engine determines them to be in a "warning state"; for disks exceeding the "action threshold", they are determined to be in a "fault state" and the data on the faulty disks needs to be migrated.
[0069] As can be seen from the above embodiments, by acquiring multi-source heterogeneous data from disk storage nodes and constructing a multi-source feature set, the inaccuracy of disk failure prediction caused by a single data source is avoided; the local inference model deployed on the disk storage node is used to predict failures locally and generate edge prediction results; the edge prediction results are aggregated, and the aggregated disk failure prediction information is compared with the corresponding failure threshold to generate disk failure prediction results. This avoids the problem of centralized machine learning models relying on centralized training to predict disk failures and improves the accuracy of disk failure prediction.
[0070] In one embodiment of this application, step S202 includes:
[0071] S2021: Timestamp synchronization is performed on the multi-source heterogeneous data of the disk storage nodes to obtain time-aligned multi-source heterogeneous data.
[0072] Specifically, for raw streaming data with their own collection timestamps, the sampling points of the data are aligned to equally spaced, unified time points. For missing data points, linear interpolation is used to fill in the missing data points, resulting in time-aligned multi-source heterogeneous data.
[0073] S2022: Normalize time-aligned multi-source heterogeneous data to generate standardized data.
[0074] Specifically, the process involves acquiring operational status data, determining weight adjustment values based on the operational status data, adjusting the feature weights of disk operational status features using an online learning algorithm based on the weight adjustment values, and normalizing time-aligned multi-source heterogeneous data based on the adjusted feature weights to generate standardized data.
[0075] S2023: Perform sliding window statistics and frequency domain transformation on the standardized data to obtain a multi-source feature set.
[0076] Specifically, a fixed-length sliding window is defined, which slides across the time series data, and statistical features are calculated within each window.
[0077] Specifically, for signals with obvious periodicity (such as I / O delay fluctuations), a fast Fourier transform is applied to convert from the time domain to the frequency domain, extract frequency domain features such as the main frequency and harmonic components, and then the statistical features and frequency domain features are concatenated into a high-dimensional feature vector to obtain a multi-source feature set.
[0078] As can be seen from the above embodiments, by synchronizing and normalizing the timestamps of multi-source heterogeneous data, the feature fusion deviation problem caused by time offset and dimensional differences in multi-source heterogeneous data is avoided. By using sliding window statistics and frequency domain transformation, the ability to perceive the dynamic change trend of data is enhanced.
[0079] In one embodiment of this application, step S2022 includes:
[0080] S221: Obtain runtime status data from time-aligned multi-source heterogeneous data.
[0081] In this embodiment, the running status data includes, but is not limited to, the current I / O load pressure of the disk, the working mode, and whether it is in the background task period.
[0082] S222: Determine the weight adjustment value of disk operating status characteristics based on operating status data.
[0083] Specifically, appropriate weight adjustment values are assigned to different operating states according to a predefined strategy.
[0084] For example, when the disk is under "heavy load", the weight adjustment value of I / O latency-related features is set to 1.5 (to increase their importance), while the weight adjustment value of temperature features is set to 0.8 (to consider that their fluctuations due to load are within the normal range).
[0085] S223: The feature weights of the disk operating status features are adjusted based on the weight adjustment values of the disk operating status features using an online learning algorithm, resulting in the adjusted feature weights.
[0086] Specifically, when the weight adjustment value of the running status is received, the weights of the corresponding features in the weight vector are updated by a lightweight online learning algorithm (such as gradient-based fine-tuning or exponential weighted averaging) to obtain the adjusted feature weights.
[0087] S224: Normalize the time-aligned multi-source heterogeneous data according to the adjusted feature weights to generate standardized data.
[0088] Specifically, when performing normalization, the adjusted feature weights are introduced simultaneously. When calculating distance or similarity, the contributions of different feature dimensions are scaled according to their weights. The normalization calculation is performed based on the adjusted weights to generate standardized data.
[0089] As can be seen from the above embodiments, by acquiring operating status data, determining the weight adjustment value of disk operating status features based on the operating status data, and adjusting the feature weights based on the weight adjustment value, flexible adaptation to different feature weights can be achieved, avoiding feature redundancy with fixed weights, and improving the accuracy of standardized data generated by normalization processing.
[0090] Figure 3 Flowchart of the disk failure prediction method provided in this application Figure 2 ,like Figure 3 As shown, after step S205, the following steps are also included:
[0091] S301: Determine the information of the faulty disk based on the disk failure prediction results.
[0092] Specifically, the fault arbiter analyzes the disk fault prediction results. When the results determine that a disk is in a "fault state" and the confidence level exceeds the preset limit, the arbiter determines that the disk is about to fail and extracts its fault disk information.
[0093] In this embodiment, the faulty disk information includes, but is not limited to, the disk's unique identifier in the storage pool, the disk's RAID group and erasure coding set, and the list of objects currently hosted by the disk.
[0094] S302: Obtain the disk data to be migrated based on the faulty disk information.
[0095] Specifically, based on the information of the faulty disk, the storage system's metadata server is queried to obtain a list of logical addresses of all data blocks stored on the physical disk, as well as information on the user files and objects to which these data blocks belong.
[0096] S303: Send the disk data to be migrated to the target disk to complete the data migration of the faulty disk.
[0097] Specifically, based on the data block list, the raw data is read from the faulty disk, and simultaneously, according to a predetermined data distribution strategy, the data is written over the network to a pre-selected target disk.
[0098] Specifically, data consistency verification is maintained during the migration process. Once all data blocks have been successfully migrated and verified, the faulty disk is marked as "isolated".
[0099] As can be seen from the above embodiments, by identifying the faulty disk through the disk failure prediction results, the disk data to be migrated is migrated to the target disk, thus avoiding the problem of data loss.
[0100] Figure 4 Flowchart of the disk failure prediction method provided in this application Figure 3 ,like Figure 4 As shown, before step S303, the following steps are also included:
[0101] S401: Obtain status information for multiple healthy disks.
[0102] In this embodiment, the status information of a healthy disk includes, but is not limited to, the disk's remaining available capacity, current I / O load, estimated health score, and location information.
[0103] S402: Determine the target disk based on the status information of multiple healthy disks.
[0104] For example, the policy engine analyzes the status information of all healthy disks and prioritizes disks that are not in the same failure domain as the original disk; under the premise of meeting capacity and load constraints, it prioritizes disks with high health scores and low current load; for RAID reconstruction, it needs to select other member disks in the same group.
[0105] S403: Create a data transfer task for the disk data to be migrated based on the target disk, and perform data migration operations based on the data transfer task.
[0106] Specifically, a data transfer queue is created based on the data transfer priority in the disk data to be migrated, a preemptive scheduling policy is generated based on the data transfer queue and system resources, and a data transfer task is created based on the preemptive scheduling policy and the data transfer queue.
[0107] As can be seen from the above embodiments, by obtaining the status information of multiple healthy disks, the target disk for receiving the data to be migrated is determined based on the status information, a data transfer task is created based on the target disk, and the data migration operation is completed, thus avoiding the problems of data loss and service interruption.
[0108] In one embodiment of this application, step S403 includes:
[0109] S4031: Create a data transfer queue based on the data transfer priority in the disk data to be migrated.
[0110] Specifically, the disk data to be migrated is parsed, the data blocks storing high-priority business volumes are marked as high priority, the data blocks are arranged in descending order of priority, and the data block transfer requests are created as a data transfer queue.
[0111] S4032: Generate a preemptive scheduling strategy based on the data transmission queue and system resources.
[0112] In this embodiment, system resources include, but are not limited to, network bandwidth utilization, target disk I / O wait queue length, and CPU utilization.
[0113] For example, a preemptive scheduling strategy could be to temporarily pause or reduce the bandwidth usage of low-priority tasks that are currently being executed when a high-priority task is ready.
[0114] S4033: Create a data transmission task based on the preemptive scheduling strategy and the data transmission queue.
[0115] Specifically, each request in the data transmission queue is instantiated into an executable data transmission task object by combining it with the specific parameters of the preemptive scheduling strategy.
[0116] As can be seen from the above embodiments, by obtaining the data transmission priority in the disk data to be migrated, creating a data transmission queue according to the priority, ensuring that high-priority data is transmitted first, and generating a preemptive scheduling strategy based on the data transmission queue and system resources, the system's response speed to the migration of faulty disk data is improved.
[0117] Figure 5 A schematic diagram of the disk failure prediction device provided in this application is shown below. Figure 5As shown, the disk failure prediction device 50 provided in this embodiment includes: a data acquisition module 501, a data processing module 502, an edge inference module 503, a data aggregation module 504, and a comparison module 505.
[0118] The data acquisition module 501 is used to acquire multi-source heterogeneous data from disk storage nodes.
[0119] The data processing module 502 is used to process multi-source heterogeneous data from disk storage nodes and generate multi-source feature sets.
[0120] The edge inference module 503 is used to send the multi-source feature set to the disk storage node, so that the disk storage node can call the local inference model to perform inference prediction on the multi-source feature set and generate edge prediction results.
[0121] The data aggregation module 504 is used to aggregate the edge prediction results and generate disk failure prediction information.
[0122] The comparison module 505 is used to compare the disk failure prediction information with the corresponding failure threshold to generate a disk failure prediction result.
[0123] In one embodiment of this application, the data processing module 502 includes:
[0124] The synchronization unit is used to synchronize the timestamps of multi-source heterogeneous data on disk storage nodes to obtain time-aligned multi-source heterogeneous data.
[0125] The normalization unit is used to normalize time-aligned multi-source heterogeneous data to generate standardized data.
[0126] The transformation unit is used to perform sliding window statistics and frequency domain transformation on standardized data to obtain a multi-source feature set.
[0127] In one embodiment of this application, the normalization unit includes:
[0128] The acquisition sub-unit is used to acquire runtime status data from time-aligned multi-source heterogeneous data.
[0129] The determination sub-unit is used to determine the weight adjustment value of disk operating status characteristics based on the operating status data.
[0130] The adjustment subunit is used to adjust the feature weights of disk operating status features based on the weight adjustment values of disk operating status features through an online learning algorithm, so as to obtain the adjusted feature weights.
[0131] The normalization subunit is used to normalize time-aligned multi-source heterogeneous data according to the adjusted feature weights, generating standardized data.
[0132] In one embodiment of this application, the disk failure prediction device 50 further includes:
[0133] The first determination module is used to determine the faulty disk information based on the disk fault prediction results.
[0134] The first acquisition module is used to acquire disk data to be migrated based on the faulty disk information.
[0135] The sending module is used to send the disk data to be migrated to the target disk in order to complete the data migration of the faulty disk.
[0136] In one embodiment of this application, the disk failure prediction device 50 further includes:
[0137] The second acquisition module is used to acquire the status information of multiple healthy disks.
[0138] The second determination module is used to determine the target disk based on the status information of multiple healthy disks.
[0139] The creation module is used to create a data transfer task for the disk data to be migrated based on the target disk, and to perform data migration operations based on the data transfer task.
[0140] In one embodiment of this application, the creation module includes:
[0141] The first creation unit is used to create a data transfer queue based on the data transfer priority in the disk data to be migrated.
[0142] The generation unit is used to generate preemptive scheduling strategies based on the data transmission queue and system resources.
[0143] The second creation unit is used to create data transmission tasks based on the preemptive scheduling strategy and the data transmission queue.
[0144] The disk failure prediction device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.
[0145] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the electronic device 60 further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus.
[0146] In the specific implementation process, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to execute the disk failure prediction method described above.
[0147] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.
[0148] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0149] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0150] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0151] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the disk failure prediction method described above.
[0152] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned disk failure prediction method.
[0153] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0154] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0155] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.
[0157] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0158] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0159] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0160] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. A disk failure prediction method, characterized in that, include: Collect multi-source heterogeneous data from disk storage nodes; Data processing is performed on the multi-source heterogeneous data of the disk storage node to generate a multi-source feature set; The multi-source feature set is sent to the disk storage node, so that the disk storage node calls the local inference model to perform inference prediction on the multi-source feature set and generate edge prediction results. The edge prediction results are aggregated to generate disk failure prediction information; The disk failure prediction information is compared with the corresponding failure threshold to generate a disk failure prediction result.
2. The method according to claim 1, characterized in that, The step of processing the multi-source heterogeneous data of the disk storage node to generate a multi-source feature set includes: The multi-source heterogeneous data of the disk storage node is timestamped to obtain time-aligned multi-source heterogeneous data. The time-aligned multi-source heterogeneous data is normalized to generate standardized data; The standardized data is subjected to sliding window statistics and frequency domain transformation to obtain a multi-source feature set.
3. The method according to claim 2, characterized in that, The normalization process for the time-aligned multi-source heterogeneous data to generate standardized data includes: Obtain the running status data from the time-aligned multi-source heterogeneous data; The weight adjustment values for the disk operating status characteristics are determined based on the operating status data; The disk operating status features are adjusted by an online learning algorithm based on their weight adjustment values to obtain the adjusted feature weights. The time-aligned multi-source heterogeneous data is normalized according to the adjusted feature weights to generate standardized data.
4. The method according to claim 1, characterized in that, After comparing the disk failure prediction information with the corresponding failure threshold to generate a disk failure prediction result, the method further includes: The faulty disk information is determined based on the disk failure prediction results; Obtain the disk data to be migrated based on the faulty disk information; The disk data to be migrated is sent to the target disk to complete the data migration of the faulty disk.
5. The method according to claim 4, characterized in that, Before sending the disk data to be migrated to the target disk to complete the data migration of the faulty disk, the process also includes: Obtain status information for multiple healthy disks; The target disk is determined based on the status information of the multiple healthy disks; A data transfer task is created based on the target disk to be migrated, and a data migration operation is performed based on the data transfer task.
6. The method according to claim 5, characterized in that, The data transfer task for creating the disk data to be migrated based on the target disk includes: Create a data transfer queue based on the data transfer priority in the disk data to be migrated; A preemptive scheduling strategy is generated based on the data transmission queue and system resources; A data transmission task is created based on the preemptive scheduling strategy and the data transmission queue.
7. A disk failure prediction device, characterized in that, include: The data acquisition module is used to collect multi-source heterogeneous data from disk storage nodes; The data processing module is used to process the multi-source heterogeneous data of the disk storage node and generate a multi-source feature set. The edge inference module is used to send the multi-source feature set to the disk storage node, so that the disk storage node calls the local inference model to perform inference prediction on the multi-source feature set and generate edge prediction results; The data aggregation module is used to aggregate the edge prediction results and generate disk failure prediction information; The comparison module is used to compare the disk failure prediction information with the corresponding failure threshold to generate a disk failure prediction result.
8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the disk failure prediction method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the disk failure prediction method as described in any one of claims 1 to 6.
10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the disk failure prediction method according to any one of claims 1 to 6.