Disk status analysis method and device based on software-defined storage, and electronic device
By collecting disk information in software-defined storage, training health prediction models and adjusting data scheduling strategies, the problem of inability to effectively analyze disk health status in the existing technology is solved, and the service life and overall reliability of the hard disk are improved.
Patent Information
- Application Number
- CN202011250284.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-10
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-11-12
AI Technical Summary
Existing software-defined storage technology cannot effectively analyze the health status of the disk, resulting in early damage to some hard disks, affecting overall reliability.
By collecting disk information in historical data, training the disk health prediction model, analyzing the health status of each disk, and adjusting the data scheduling strategy according to the health status, including data allocation, IO access and cache flushing strategy.
Reduce the probability of disk problems, avoid the overall reliability of the cluster caused by poor health status, and improve the overall reliability of software-defined storage.
Smart Images

Figure CN112328453B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of terminal device analysis technology, and in particular to a disk status analysis method and device based on software-defined storage, and an electronic device. Background Art
[0002] In the related technology, software-defined storage builds a storage pool through massive hard disks and provides services. As time goes by, the hard disks show different health states, some hard disks are on the verge of damage, and some hard disks are in good condition. In some expansion or reuse scenarios, new and old hard disks are in service at the same time, and the difference in health status is also large. The existing technology does not effectively identify the health status of the hard disk, nor does it schedule IO pressure according to the health status of the disk. Basically, all hard disks in the storage pool are treated equally, resulting in premature damage to some hard disks, affecting the overall reliability of the product.
[0003] That is, in the current software-defined storage, it is impossible to reasonably analyze the usage status and health status of each disk. Basically, when a disk is found to be in a sub-healthy or error state, the use of the disk is stopped and it is kicked out of the management cluster. This method not only easily causes the disk to be damaged, but also after some disks become healthy, if IO tasks are forcibly assigned, it is easy for the single point pressure to be too high, resulting in continuous damage to the disk health.
[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0005] The embodiments of the present invention provide a disk status analysis method and device, and an electronic device based on software-defined storage, so as to at least solve the technical problem in the related technology that the disk health status cannot be analyzed, which easily leads to partial hard disk damage and affects the overall reliability of the product.
[0006] According to one aspect of an embodiment of the present invention, a disk status analysis method based on software-defined storage is provided, comprising: collecting disk information in historical data, wherein the disk information includes at least: disk failure information; training a disk health prediction model based on the disk information; using the disk health prediction model to analyze the disk health status of each disk in all disk clusters; and adjusting a data scheduling strategy based on the disk health status.
[0007] Optionally, the step of collecting disk information and disk failure information in historical data includes: determining the number of disks and disk identifications in each of the disk clusters; and based on the number of disks and the disk identifications in each of the disk clusters, collecting the disk information and the disk failure information in the historical data corresponding to changes in time points.
[0008] Optionally, the disk information includes at least one of the following: number of disk IO accesses, size of each disk partition, number of disk power-on and power-off times, disk power-on and power-off time, disk bad block information, read and write error rate, disk failure information, and cumulative power-on time, wherein the disk failure information includes at least: disk damage status, failure time point, time point when IO transmission delay is higher than a preset delay threshold, and failure label.
[0009] Optionally, based on the disk information, the step of training a disk health prediction model includes: determining status data corresponding to each disk type in the disk information; performing label classification and formatting on the status data corresponding to each disk type to obtain a status processing result; based on the status processing result, using the basic disk status information in the disk information as training data, and using the disk failure information as label result data, to train the disk health prediction model.
[0010] Optionally, the step of using the disk health prediction model to analyze the disk health status of each disk in all disk clusters includes: using the disk health prediction model to analyze the failure probability value of each disk; and determining the disk health value of each disk based on the failure probability value of each disk.
[0011] Optionally, the data scheduling strategy includes: a data allocation strategy, and the step of adjusting the data scheduling strategy based on the disk health status includes: adjusting the data allocation strategy based on the disk health value of each disk, wherein the data allocation strategy includes at least: the data allocation capacity of each disk.
[0012] Optionally, the data scheduling strategy includes: IO access strategy, and the step of adjusting the data scheduling strategy based on the disk health status also includes: adjusting the IO access strategy based on the disk health value of each disk, wherein the IO access strategy includes at least: IO access channel occupancy of each disk.
[0013] Optionally, the data scheduling strategy includes: a cache flush strategy, and the step of adjusting the data scheduling strategy based on the disk health status also includes: adjusting the cache flush strategy based on the disk health value of each disk, wherein the cache flush strategy includes at least: a frequency of flushing data in the cache.
[0014] According to another aspect of an embodiment of the present invention, a disk status analysis device based on software-defined storage is also provided, including: a collection unit, used to collect disk information in historical data, wherein the disk information at least includes: disk failure information; a training unit, used to train a disk health prediction model based on the disk information; an analysis unit, used to use the disk health prediction model to analyze the disk health status of each disk in all disk clusters; and an adjustment unit, used to adjust a data scheduling strategy based on the disk health status.
[0015] Optionally, the collection unit includes: a first determination module, used to determine the number of disks and disk identifications in each of the disk clusters; a first collection module, used to collect the disk information and the disk failure information corresponding to the changes in time points in the historical data based on the number of disks and the disk identifications in each of the disk clusters.
[0016] Optionally, the disk information includes at least one of the following: number of disk IO accesses, size of each disk partition, number of disk power-on and power-off times, disk power-on and power-off time, disk bad block information, read and write error rate, disk failure information, and cumulative power-on time, wherein the disk failure information includes at least: disk damage status, failure time point, time point when IO transmission delay is higher than a preset delay threshold, and failure label.
[0017] Optionally, the training unit includes: a second determination module, used to determine the status data corresponding to each disk type in the disk information; a first processing module, used to perform label classification and formatting on the status data corresponding to each disk type to obtain a status processing result; a training module, used to train the disk health prediction model based on the status processing result, using the basic disk status information in the disk information as training data, and using the disk failure information as label result data.
[0018] Optionally, the analysis unit includes: a first analysis module, used to analyze the failure probability value of each disk using the disk health prediction model; and a third determination module, used to determine the disk health value of each disk based on the failure probability value of each disk.
[0019] Optionally, the data scheduling strategy includes: a data allocation strategy, and the adjustment unit includes: a first adjustment module, used to adjust the data allocation strategy based on the disk health value of each disk, wherein the data allocation strategy includes at least: a data allocation capacity of each disk.
[0020] Optionally, the data scheduling strategy includes: IO access strategy, and the adjustment unit further includes: a second adjustment module, used to adjust the IO access strategy based on the disk health value of each disk, wherein the IO access strategy includes at least: IO access channel occupancy of each disk.
[0021] Optionally, the data scheduling strategy includes: a cache flush strategy, and the adjustment unit further includes: a third adjustment module, used to adjust the cache flush strategy based on the disk health value of each disk, wherein the cache flush strategy includes at least: a frequency of data flushing in the cache.
[0022] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned disk status analysis methods based on software-defined storage by executing the executable instructions.
[0023] In an embodiment of the present invention, disk information in historical data is collected, wherein the disk information includes at least: disk failure information; based on the disk information, a disk health prediction model is trained; the disk health prediction model is used to analyze the disk health status of each disk in all disk clusters; and based on the disk health status, the data scheduling strategy is adjusted. In this embodiment, a prediction model that can analyze the disk health status based on a large amount of data can be obtained through training, and the data scheduling strategy can be adjusted by analyzing the health status of each disk to reduce the probability of disk problems, avoid the poor health status of some disks leading to a decrease in the overall reliability of the cluster, and improve the overall reliability of software-defined storage, thereby solving the technical problem in related technologies that the disk health status cannot be analyzed, which easily leads to damage to some hard disks and affects the overall reliability of the product. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0025] Figure 1 is a flowchart of an optional disk status analysis method based on software-defined storage according to an embodiment of the present invention;
[0026] Figure 2 is a schematic diagram of an optional method of analyzing disk status and implementing data scheduling according to an embodiment of the present invention;
[0027] Figure 3 It is a schematic diagram of an optional disk status analysis device based on software-defined storage according to an embodiment of the present invention. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0030] To facilitate those skilled in the art to understand the present invention, some terms or nouns involved in the embodiments of the present invention are explained below:
[0031] Software-defined storage, or SDS for short, is a data storage method where all storage-related control work is done only in external software relative to the physical storage hardware. Software-defined storage can ensure that the system's storage access can be managed more flexibly at a precise level, deployed and provisioned through software and management, and can also be further simplified through policy-based automated management. Its core is to leverage the storage capabilities and value of hardware through software. Software-defined storage mostly uses a distributed technology architecture.
[0032] Software-defined storage is a software-defined storage system that deploys storage software on general-purpose servers to build a highly available storage system. It has several notable features: 1. It manages general-purpose servers through software to form storage resources; 2. It adopts a distributed design to support horizontal expansion; 3. It supports large-scale clusters: Through horizontal expansion, it can reach the scale of hundreds or even thousands of servers; 4. It manages the massive hard disks in the server to form a storage resource pool and provide access services to the outside world.
[0033] From the perspective of storage resources, software-defined storage requires the management of a large number of disks. Hundreds, thousands, or even tens of thousands of disks are managed through software control to provide reliable storage services. The stability of the disks largely determines the stability of the storage, and the software system's ability to manage the disks also determines the reliability of the storage.
[0034] This method is aimed at large-scale software-defined storage, and designs a method for improving disk reliability, thereby improving the overall reliability of large-scale disk systems and improving software-defined storage reliability. The present invention is described in detail below in conjunction with various embodiments.
[0035] Embodiment 1
[0036] According to an embodiment of the present invention, an embodiment of a disk status analysis method based on software-defined storage is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0037] Figure 1 is a flowchart of an optional disk status analysis method based on software-defined storage according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:
[0038] Step S102, collecting disk information in historical data, wherein the disk information at least includes: disk failure information;
[0039] Step S104, training a disk health prediction model based on the disk information;
[0040] Step S106, using a disk health prediction model to analyze the disk health status of each disk in all disk clusters;
[0041] Step S108, adjusting the data scheduling strategy based on the disk health status.
[0042] Through the above steps, disk information in historical data can be collected, wherein the disk information at least includes: disk failure information; based on the disk information, a disk health prediction model is trained; the disk health prediction model is used to analyze the disk health status of each disk in all disk clusters; based on the disk health status, the data scheduling strategy is adjusted. In this embodiment, a prediction model that can analyze the disk health status based on a large amount of data can be obtained through training, and the data scheduling strategy can be adjusted by analyzing the health status of each disk to reduce the probability of disk problems, avoid the poor health status of some disks leading to a decrease in the overall reliability of the cluster, and improve the overall reliability of software-defined storage, thereby solving the technical problem in related technologies that the disk health status cannot be analyzed, which easily leads to damage to some hard disks and affects the overall reliability of the product.
[0043] The present invention is described in detail below in conjunction with the above steps.
[0044] Step S102, collecting disk information in historical data, wherein the disk information at least includes: disk failure information.
[0045] Optionally, the step of collecting disk information and disk failure information in historical data includes: determining the number of disks and disk identifications in each disk cluster; based on the number of disks and disk identifications in each disk cluster, collecting disk information and disk failure information in historical data corresponding to changes in time points.
[0046] Historical data can refer to a certain period of time in the historical process (such as data from the past year, month, or week).
[0047] Real-time collection of disk information in each disk cluster (also understood as an online cluster) (for example, disk IO access times, IO model, disk power-on times, disk bad block information, disk failure information (time point when service cannot be provided)), as well as the time correspondence of these information, will be summarized in each cluster information on the network.
[0048] In an embodiment of the present invention, the disk information includes at least one of the following: the number of disk IO accesses, the size of each disk partition, the number of disk power-on and power-off times, the disk power-on and power-off time, the disk bad block information, the read and write error rate, the disk failure information, and the cumulative power-on time, wherein the disk failure information includes at least: the disk damage status, the failure time point, the time point when the IO transmission delay is higher than the preset delay threshold, and the failure label.
[0049] The information collection module is responsible for collecting data of each disk in the disk system, including disk status information and time change data.
[0050] The above-mentioned power-on and power-off times of the disk may include: planned power-on and power-off times of the disk and unplanned power-on and power-off times of the disk.
[0051] The above disk information may also include: IO model, average blocksize size.
[0052] The fault tag in the disk fault information may indicate the following fault time points: 1. IO is inaccessible; 2. IO has high latency.
[0053] The key point of information collection is to mark and record the above information on the operation timeline of the overall system, so as to present the information related to the life cycle status of the disk and the time when the failure occurred.
[0054] Step S104: training a disk health prediction model based on the disk information.
[0055] Optionally, the step of training a disk health prediction model based on disk information includes: determining status data corresponding to each disk type in the disk information; performing label classification and formatting on the status data corresponding to each disk type to obtain a status processing result; based on the status processing result, using basic disk status information in the disk information as training data, and using disk failure information as label result data, to train the disk health prediction model.
[0056] When training the model, training data (disk information over time) and training labels (whether the disk fails and the time point) are input, and the training labels are derived from the training data to continuously train the disk health prediction model.
[0057] In the process of obtaining the disk health prediction model, data cleaning and model training can be performed, wherein data cleaning includes: label classification and formatting of the status data corresponding to each disk type to obtain the status processing result, that is, classifying and formatting the data and labels for different types of disks. Model training includes: based on the status processing result, using the basic status information of the disk in the disk information as training data, and using the disk failure information as label result data, to train the disk health prediction model. For example, according to the data and label training model, the model can calculate the probability value of disk failure based on the disk status information, and set the disk health status according to this probability value.
[0058] Step S106: using the disk health prediction model to analyze the disk health status of each disk in all disk clusters.
[0059] Optionally, the step of using a disk health prediction model to analyze the disk health status of each disk in all disk clusters includes: using the disk health prediction model to analyze the failure probability value of each disk; and determining the disk health value of each disk based on the failure probability value of each disk.
[0060] Based on the disk status information of the disk cluster, calculate the health status of the disk and set a specific value for the health status, for example, from 0 to 100, the higher the value, the healthier it is. This value is updated regularly as the disk status changes.
[0061] Step S108, adjusting the data scheduling strategy based on the disk health status.
[0062] Optionally, the data scheduling strategy includes: a data allocation strategy, and the step of adjusting the data scheduling strategy based on the disk health status includes: adjusting the data allocation strategy based on the disk health value of each disk, wherein the data allocation strategy includes at least: the data allocation capacity of each disk.
[0063] That is, data of different capacities can be allocated according to the different health status of different disks.
[0064] Another optional step is that the data scheduling strategy includes: IO access strategy, based on the disk health status, adjusting the data scheduling strategy, and also includes: adjusting the IO access strategy based on the disk health value of each disk, wherein the IO access strategy includes at least: the IO access channel occupancy rate of each disk.
[0065] That is, when adjusting the IO access strategy, different IO access pressures are allocated according to the different health states of different disks. For example, for a disk in a poor health state, more read and write IOs are allocated to other healthy disks to reduce the pressure on the disk and increase its lifespan.
[0066] In an embodiment of the present invention, the data scheduling strategy includes: a cache flush strategy, and a step of adjusting the data scheduling strategy based on the disk health status, and also includes: adjusting the cache flush strategy based on the disk health value of each disk, wherein the cache flush strategy includes at least: the frequency of data flushing in the cache.
[0067] That is, when adjusting the cache flushing strategy, the frequency of data flushing in the cache is controlled according to the different health states of different disks. For disks in poor health, the frequency of flushing is reduced to increase their service life.
[0068] Figure 2 is a schematic diagram of an optional method for analyzing disk status and implementing data scheduling according to an embodiment of the present invention. Figure 2As shown in the figure, all disks form multiple online clusters and servers, including: online cluster 1, online cluster 2...online cluster n. Each online cluster corresponds to multiple servers, and each server is responsible for collecting information from the corresponding multiple disks. In terms of disk information collection, by collecting information from multiple online clusters and then sending it to the AI training platform, the AI training platform completes data collection and cleaning, then performs model training, and outputs the inference model. After that, the server in the online cluster X can use the model to analyze the health status of each disk and implement data policy adjustments, including data allocation policy adjustments, IO access policy adjustments, and cache flushing policy adjustments.
[0069] Figure 2 In the process, the information collection module collects disk information in each cluster and inputs it into the AI analysis system; after data cleaning, it is input into the model training module; the training outputs the inference model and outputs it to the online cluster; the online cluster infers all disks according to the inference model to obtain the health status of each disk; the data scheduling module adjusts the IO strategy according to the health status of each disk, including data allocation strategy adjustment, IO access strategy adjustment, and cache flushing strategy adjustment; healthy disks will share more IO access, thereby ensuring the uniformity of the entire system and improving overall reliability.
[0070] The embodiment of the present invention obtains a disk health status prediction model by collecting, analyzing and training data of an online cluster. This model is used in an online environment to analyze the health status of each disk, and data access pressure is scheduled according to the health status, thereby achieving the effect of making all disks in the cluster in a uniformly healthy state, avoiding abnormalities in some disks that lead to reduced cluster availability, significantly improving the overall reliability of the cluster, and reducing the probability of cluster unavailability caused by unexpected failures of some disks.
[0071] Embodiment 2
[0072] Figure 3 is a schematic diagram of an optional disk status analysis device based on software-defined storage according to an embodiment of the present invention, such as Figure 3 As shown, the disk status analysis device further includes: a collection unit 31, a training unit 33, an analysis unit 35, and an adjustment unit 37, wherein:
[0073] The collecting unit 31 is used to collect disk information in the historical data, wherein the disk information at least includes: disk failure information;
[0074] A training unit 33, used for training a disk health prediction model based on the disk information;
[0075] An analysis unit 35, configured to analyze the disk health status of each disk in all disk clusters using a disk health prediction model;
[0076] The adjustment unit 37 is used to adjust the data scheduling strategy based on the disk health status.
[0077] The above-mentioned disk status analysis device based on software-defined storage can collect disk information in historical data through the collection unit 31, wherein the disk information at least includes: disk failure information, train the disk health prediction model based on the disk information through the training unit 33, analyze the disk health status of each disk in all disk clusters using the disk health prediction model through the analysis unit 35, and adjust the data scheduling strategy based on the disk health status through the adjustment unit 37. In this embodiment, a prediction model that can analyze the disk health status based on a large amount of data can be obtained through training, and the data scheduling strategy can be adjusted by analyzing the health status of each disk, thereby reducing the probability of disk problems, avoiding the poor health status of some disks leading to a decrease in the overall reliability of the cluster, and improving the overall reliability of software-defined storage, thereby solving the technical problem in the related technology that the disk health status cannot be analyzed, which easily leads to damage to some hard disks and affects the overall reliability of the product.
[0078] Optionally, the collection unit includes: a first determination module, used to determine the number of disks and disk identifications in each disk cluster; a first collection module, used to collect disk information and disk failure information corresponding to time point changes in historical data based on the number of disks and disk identifications in each disk cluster.
[0079] Optionally, the disk information includes at least one of the following: number of disk IO accesses, size of each disk partition, number of disk power on and off, disk power on and off time, disk bad block information, read and write error rate, disk failure information, and cumulative power-on time, wherein the disk failure information includes at least: disk damage status, failure time point, time point when IO transmission delay is higher than a preset delay threshold, and fault label.
[0080] Optionally, the training unit includes: a second determination module, used to determine the status data corresponding to each disk type in the disk information; a first processing module, used to perform label classification and formatting on the status data corresponding to each disk type to obtain a status processing result; a training module, used to train a disk health prediction model based on the status processing result, using the basic disk status information in the disk information as training data, and using the disk failure information as label result data.
[0081] In an embodiment of the present invention, the analysis unit includes: a first analysis module, which is used to analyze the failure probability value of each disk using a disk health prediction model; and a third determination module, which is used to determine the disk health value of each disk based on the failure probability value of each disk.
[0082] Optionally, the data scheduling strategy includes: a data allocation strategy, and the adjustment unit includes: a first adjustment module, used to adjust the data allocation strategy based on the disk health value of each disk, wherein the data allocation strategy includes at least: a data allocation capacity of each disk.
[0083] Alternatively, the data scheduling strategy includes: an IO access strategy, and the adjustment unit further includes: a second adjustment module for adjusting the IO access strategy based on a disk health value of each disk, wherein the IO access strategy includes at least: an IO access channel occupancy rate of each disk.
[0084] Optionally, the data scheduling strategy includes: a cache flush strategy, and the adjustment unit further includes: a third adjustment module, used to adjust the cache flush strategy based on the disk health value of each disk, wherein the cache flush strategy includes at least: a frequency of data flushing in the cache.
[0085] The above-mentioned disk status analysis device may also include a processor and a memory. The above-mentioned acquisition unit 31, training unit 33, analysis unit 35, adjustment unit 37, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.
[0086] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the data scheduling strategy can be adjusted based on the disk health status by adjusting kernel parameters.
[0087] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.
[0088] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned disk status analysis methods based on software-defined storage by executing the executable instructions.
[0089] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: collecting disk information in historical data, wherein the disk information at least includes: disk failure information; training a disk health prediction model based on the disk information; using the disk health prediction model to analyze the disk health status of each disk in all disk clusters; and adjusting a data scheduling strategy based on the disk health status.
[0090] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0091] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0092] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0094] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0096] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A disk status analysis method based on software-defined storage, characterized in that: include: Collecting disk information in historical data, wherein the disk information at least includes: disk failure information; Based on the disk information, training a disk health prediction model; The disk health prediction model is used to analyze the disk health status of each disk in all disk clusters, including: using the disk health prediction model to analyze the failure probability value of each disk; based on the failure probability value of each disk, determining the disk health value of each disk; Adjusting the data scheduling strategy based on the disk health status; The data scheduling strategy includes: a cache flush strategy, and the step of adjusting the data scheduling strategy based on the disk health status also includes: adjusting the cache flush strategy based on the disk health value of each disk, wherein the cache flush strategy at least includes: a frequency of flushing data in the cache.
2. The disk status analysis method according to claim 1, characterized in that: The steps of collecting disk information and disk failure information in historical data include: Determine the number of disks and disk identifiers in each of the disk clusters; Based on the number of disks in each disk cluster and the disk identifier, the disk information and the disk failure information corresponding to the time point changes in the historical data are collected.
3. The disk status analysis method according to claim 2, characterized in that: The disk information includes at least one of the following: the number of disk IO accesses, the size of each disk partition, the number of disk power-on and power-off times, the disk power-on and power-off time, the disk bad block information, the read and write error rate, the disk failure information, and the cumulative power-on time, wherein the disk failure information includes at least: the disk damage status, the failure time point, the time point when the IO transmission delay is higher than the preset delay threshold, and the failure label.
4. The disk status analysis method according to claim 1, characterized in that: The step of training a disk health prediction model based on the disk information includes: Determine status data corresponding to each disk type in the disk information; Performing label classification and formatting processing on the status data corresponding to each disk type to obtain a status processing result; Based on the status processing result, the disk basic status information in the disk information is used as training data, and the disk failure information is used as label result data to train the disk health prediction model.
5. The disk status analysis method according to claim 1, characterized in that: The data scheduling strategy includes: a data allocation strategy, and the step of adjusting the data scheduling strategy based on the disk health status includes: Based on the disk health value of each disk, a data allocation strategy is adjusted, wherein the data allocation strategy at least includes: a data allocation capacity of each disk.
6. The disk status analysis method according to claim 1, characterized in that: The data scheduling strategy includes: IO access strategy, and the step of adjusting the data scheduling strategy based on the disk health status also includes: Based on the disk health value of each disk, the IO access strategy is adjusted, wherein the IO access strategy at least includes: the IO access channel occupancy rate of each disk.
7. A disk status analysis device based on software-defined storage, characterized in that: include: A collection unit, used for collecting disk information in historical data, wherein the disk information at least includes: disk failure information; A training unit, used for training a disk health prediction model based on the disk information; An analysis unit, used to analyze the disk health status of each disk in all disk clusters using the disk health prediction model, the analysis unit comprising: a first analysis module, used to analyze the failure probability value of each disk using the disk health prediction model; a third determination module, used to determine the disk health value of each disk based on the failure probability value of each disk; An adjustment unit, used for adjusting a data scheduling strategy based on the health status of the disk; The data scheduling strategy includes: a cache flush strategy, and the adjustment unit also includes: a third adjustment module, which is used to adjust the cache flush strategy based on the disk health value of each disk, wherein the cache flush strategy at least includes: a data flush frequency in the cache.
8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the disk status analysis method based on software-defined storage as described in any one of claims 1 to 6 by executing the executable instructions.
