A storage management method, device, equipment and readable storage medium

Abstract

The present specification provides a storage management method, device and equipment and readable storage medium, the method comprises: obtaining and / or updating and recording the state information of the state machine of each hard disk in the hard disk pool in advance; in response to a hard disk isolation request, obtaining the hard disk pool to which the hard disk to be isolated associated with the hard disk isolation request belongs, and determining the total number of fault nodes in the current hard disk pool according to the state information of each hard disk in the hard disk pool, wherein the fault node refers to the node associated with the hard disk in the second type state; according to the comparison relationship between the total number of fault nodes in the hard disk pool and the allowed number of fault nodes, the hard disk to be isolated associated with the hard disk isolation request is allowed or rejected. Through the technical scheme of the present specification, the state machine state of each hard disk in the hard disk pool is recorded in advance, the total number of fault nodes is counted based on the global state when the isolation request is received, and the total number of fault nodes is compared with the allowed number of fault nodes, thereby realizing the pre-judgment of the super redundancy before isolation.

Description

Technical Field

[0001] This specification relates to the field of communication technology, and in particular to a storage management method, apparatus, device, and readable storage medium. Background Technology

[0002] To address the unavoidable risk of storage hardware failures, distributed storage systems typically employ redundancy strategies such as multiple replicas or erasure coding. Data is distributed across multiple physically or logically isolated units within the cluster; these isolated units are called fault domains. Common fault domains include, but are not limited to, a single node (Host), an entire rack (Rack), or even a data center. When a storage device in the system, such as a disk, fails, to ensure data redundancy is not reduced and to maintain continuous business operations, the system must quickly isolate the failed disk from the service cluster and immediately initiate a data reconstruction process. This process recreates the data originally stored on the failed disk on other healthy storage nodes, generating replicas or checksum blocks to restore the system to the preset level of security redundancy.

[0003] In real-world large-scale production environments, disk failures do not always occur in isolation. Especially under conditions of hardware aging, batch issues, power fluctuations, or network outages, it is highly likely that multiple disks will report failures successively or even simultaneously within a short period. Such concurrent failures can occur within the same failure domain, such as multiple hard drives on the same server node simultaneously exhibiting slow disk, IO timeout, or pre-failure signals; or they can span different failure domains, such as disks on different nodes in a cluster entering an abnormal state almost simultaneously. Faced with such complex concurrent failure scenarios, traditional disk isolation mechanisms reveal significant shortcomings. The most basic existing technical solution is that when a disk failure is detected, the system does not perform any judgment on the remaining redundancy capacity of the cluster and directly isolates the failed disks sequentially. This solution works normally in most single-point-of-failure scenarios because it assumes that concurrent failures are low-probability events. However, as the cluster size expands rapidly, the probability of concurrent failures increases accordingly. This direct isolation method without redundancy verification is very likely to isolate more disks than the system's redundancy design limit in a short period of time, which may lead to the destruction of data replicas or erasure code stripe integrity, ultimately causing business interruption or even permanent data loss. This is a major incident that cannot be accepted in a production environment. Summary of the Invention

[0004] In view of this, this specification provides a storage management method, apparatus, device, and readable storage medium to improve the problem of business or data damage caused by the above-mentioned excessive redundancy events.

[0005] The specific technical solution is as follows: This specification provides a storage management method applied to a storage management device for a distributed storage cluster. The distributed storage cluster includes a disk pool, and the disk pool includes a plurality of disks. The method includes: pre-acquiring and / or updating and recording the state information of the state machine of each disk in the disk pool, the state information including the current state of the disk state machine, including being in a first type state that allows direct isolation or a second type state that does not allow direct isolation; responding to a disk isolation request, acquiring the disk pool to which the disk to be isolated associated with the disk isolation request belongs, determining the total number of faulty nodes in the disk pool based on the state information of each disk in the disk pool, wherein a faulty node refers to a node whose associated disk is in the second type state; and allowing or rejecting the isolation of the disk to be isolated associated with the disk isolation request based on the comparison between the total number of faulty nodes in the disk pool and the number of allowed faulty nodes.

[0006] As a technical solution, the states belonging to the second type of state include: DOWN state, which indicates that the OSD instance associated with the hard disk is abnormal but reconstruction has not been triggered, and the data on the disk may be incomplete; and / or OUT state, which indicates that the OSD instance associated with the hard disk is abnormal, reconstruction has been triggered but not completed, and the number of backups of the data on the disk on other disks may be insufficient; and / or RECOVER state, which indicates that the hard disk has been rejoined to the cluster from the abnormal state, the data is being reconstructed, and the number of backups of the data on the disk on other disks may be insufficient.

[0007] As a technical solution, the step of responding to a hard disk isolation request, obtaining the hard disk pool to which the hard disk to be isolated associated with the hard disk isolation request belongs, and determining the total number of faulty nodes in the hard disk pool based on the status information of each hard disk in the hard disk pool, includes: obtaining the currently confirmed faulty nodes based on the recorded status information of the state machines of each hard disk in the hard disk pool; obtaining one or more nodes to which one or more hard disks associated with the hard disk isolation request belong, and treating the one or more nodes as faulty nodes; and counting the total number of faulty nodes after deduplicating the faulty nodes in the above two steps.

[0008] As a technical solution, the step of allowing or denying the isolation request associated with the hard disk to be isolated based on the comparison between the total number of faulty nodes in the hard disk pool and the number of allowed faulty nodes includes: obtaining the number of allowed faulty nodes in each storage pool in the hard disk pool, taking the smallest one as the number of allowed faulty nodes, and if the total number of faulty nodes is less than or equal to the number of allowed faulty nodes, then the isolation request associated with the hard disk to be isolated is allowed.

[0009] This specification also provides a storage management device for use in a distributed storage cluster. The distributed storage cluster includes a disk pool, and the disk pool includes a plurality of disks. The device includes: a first module, used to pre-acquire and / or update and record the state information of the state machine of each disk in the disk pool, the state information including the current state of the disk state machine, including being in a first type state that allows direct isolation or a second type state that does not allow direct isolation; a second module, used to respond to a disk isolation request, acquire the disk pool to which the disk to be isolated associated with the disk isolation request belongs, and determine the total number of faulty nodes in the disk pool based on the state information of each disk in the disk pool, wherein a faulty node refers to a node whose associated disk is in the second type state; and a third module, used to allow or refuse to isolate the disk to be isolated associated with the disk isolation request based on the comparison between the total number of faulty nodes in the disk pool and the number of allowed faulty nodes.

[0010] As a technical solution, the states belonging to the second type of state include: DOWN state, which indicates that the OSD instance associated with the hard disk is abnormal but reconstruction has not been triggered, and the data on the disk may be incomplete; and / or OUT state, which indicates that the OSD instance associated with the hard disk is abnormal, reconstruction has been triggered but not completed, and the number of backups of the data on the disk on other disks may be insufficient; and / or RECOVER state, which indicates that the hard disk has been rejoined to the cluster from the abnormal state, the data is being reconstructed, and the number of backups of the data on the disk on other disks may be insufficient.

[0011] As a technical solution, the step of responding to a hard disk isolation request, obtaining the hard disk pool to which the hard disk to be isolated associated with the hard disk isolation request belongs, and determining the total number of faulty nodes in the hard disk pool based on the status information of each hard disk in the hard disk pool, includes: obtaining the currently confirmed faulty nodes based on the recorded status information of the state machines of each hard disk in the hard disk pool; obtaining one or more nodes to which one or more hard disks associated with the hard disk isolation request belong, and treating the one or more nodes as faulty nodes; and counting the total number of faulty nodes after deduplicating the faulty nodes in the above two steps.

[0012] As a technical solution, the step of allowing or denying the isolation request associated with the hard disk to be isolated based on the comparison between the total number of faulty nodes in the hard disk pool and the number of allowed faulty nodes includes: obtaining the number of allowed faulty nodes in each storage pool in the hard disk pool, taking the smallest one as the number of allowed faulty nodes, and if the total number of faulty nodes is less than or equal to the number of allowed faulty nodes, then the isolation request associated with the hard disk to be isolated is allowed.

[0013] This specification also provides an electronic device, including a processor and a readable storage medium storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the aforementioned storage management method.

[0014] This specification also provides a readable storage medium storing machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the aforementioned storage management method.

[0015] The technical solutions provided in this specification offer at least the following beneficial effects: By pre-recording the state of each hard disk state machine in the hard disk pool, and comparing the total number of faulty nodes with the allowed number of faulty nodes based on the global state when an isolation request is made, super-redundant prediction before isolation is achieved. This avoids super-redundant isolation caused by a lack of global awareness in multi-disk concurrent failure scenarios, and ensures the data availability and business continuity of the distributed storage system. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments of this specification or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this specification.

[0017] Figure 1 This is a flowchart of a storage management method according to one embodiment of this specification; Figure 2 This is a schematic diagram of state transition in one embodiment of this specification; Figure 3 This is a flowchart of one embodiment of this specification; Figure 4 This is a structural diagram of a storage management device according to one embodiment of this specification; Figure 5 This is a hardware structure diagram of an electronic device according to one embodiment of this specification.

[0018] Reference numerals: Module 1 21, Module 22, Module 3 23. Detailed Implementation

[0019] The terminology used in the embodiments described herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The singular forms “a,” “described,” and “the” as used in this specification and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0020] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0021] This specification provides a storage management method, apparatus, device, and readable storage medium to at least improve one of the above-mentioned technical problems.

[0022] The specific technical solution is described below.

[0023] In one embodiment, this specification provides a storage management method applied to a storage management device for a distributed storage cluster, wherein the distributed storage cluster includes a disk pool, and the disk pool includes a plurality of disks. The method includes: pre-acquiring and / or updating and recording the state information of the state machine of each disk in the disk pool, the state information including the current state of the disk state machine, including being in a first type state that allows direct isolation or a second type state that does not allow direct isolation; responding to a disk isolation request, acquiring the disk pool to which the disk to be isolated associated with the disk isolation request belongs, determining the total number of faulty nodes in the disk pool based on the state information of each disk in the disk pool, wherein a faulty node refers to a node whose associated disk is in the second type state; and allowing or rejecting the isolation of the disk to be isolated associated with the disk isolation request based on the comparison between the total number of faulty nodes in the disk pool and the number of allowed faulty nodes.

[0024] Specifically, such as Figure 1 This includes the following steps, the order of which can be changed depending on the needs of the actual application scenario: Step S11: Pre-acquire and / or update and record the state information of the state machine of each hard disk in the hard disk pool, the state information including the current state of the state machine of the hard disk.

[0025] In a distributed storage cluster, the storage management device can be a dedicated management node in the cluster, or a distributed coordination service running on multiple nodes, such as the OSDMON module implemented based on the Paxos protocol. This storage management device is responsible for maintaining the status information of all hard drives in the entire cluster, especially the current state of the state machine of each hard drive.

[0026] To achieve fine-grained management of hard drive status, the state machine states of each hard drive are classified into five states: UP, DOWN, OUT, FAULT, and RECOVER. In this context, the states are as follows: UP indicates that the hard drive is running normally and the data stored on it is complete and usable. The hard drive participates in normal read and write services and does not require any isolation. DOWN indicates that the hard drive has experienced a temporary failure, such as IO timeout, brief node downtime, or network interruption. While the hard drive is temporarily unable to provide services, the system has not yet determined it to be a permanent failure, nor has it triggered data reconstruction. Therefore, such hard drives usually wait for automatic recovery within a certain period and should not be directly isolated. OUT indicates that the hard drive has been confirmed as a permanent failure, and the system is about to begin or has already begun reconstructing the data on the hard drive, but the reconstruction has not yet been completed. At this point, the hard drive has lost the ability to continue providing services and needs to be isolated as soon as possible to release resources and initiate the reconstruction process. FAULT indicates that the data on the hard drive is invalid, the hard drive has completely failed, and no longer carries any valid data. Isolating the hard drive at this time will not cause any additional impact on data redundancy. RECOVER indicates that the hard drive has previously failed but has since recovered, or it is a newly added idle hard drive in the cluster currently undergoing data backfilling operations, writing data migrated from other failed hard drives to this drive. During this period, the data on the hard drive is not yet complete, so it cannot be directly isolated.

[0027] Based on these five states, this embodiment divides them into two types: the first type of state, which allows direct isolation, specifically includes the UP and FAULT states; and the second type of state, which does not allow direct isolation, specifically includes the OUT, DOWN, and RECOVER states. The storage management device continuously listens and interacts with messages to pre-acquire and update the state machine information of all disks in the disk pool in real time, and persists this information in a local or distributed database, thereby providing a globally consistent view for subsequent isolation decisions.

[0028] When an OSD in the cluster detects an anomaly in a hard drive it manages, such as multiple consecutive I / O timeouts or a SMART log reporting a pre-faulty sector, the OSD sends a hard drive isolation request to the storage management device. This request contains the unique identifier of the hard drive to be isolated and the node information of the hard drive. Upon receiving the request, the storage management device first resolves which hard drive pool the hard drive to be isolated belongs to. In a distributed storage system, a hard drive pool is a logical management unit consisting of a group of hard drives with the same attributes (such as capacity and performance level) and serving one or more storage pools. Each storage pool defines the number of data replicas (size) and the minimum number of live replicas (min_size). These two parameters together determine the maximum number of fault domains the storage pool can tolerate. For example, a three-replica storage pool, if size=3 and min_size=2, can tolerate the failure of a fault domain containing at most one replica, because as long as two replicas are alive, the data remains readable and writable. If two fault domains fail simultaneously, the data will be unavailable. Therefore, the redundancy tolerance limit of a hard drive pool depends on the minimum tolerance capacity of all its subordinate storage pools.

[0029] Step S12: In response to the hard disk isolation request, obtain the hard disk pool to which the hard disk to be isolated associated with the hard disk isolation request belongs, and determine the total number of faulty nodes in the hard disk pool based on the status information of each hard disk in the hard disk pool.

[0030] Storage management devices need to determine the total number of faulty nodes in a disk pool based on the status information of each disk within the pool. Faulty nodes refer to those nodes whose associated disks are in a second-type state (i.e., a state where direct isolation is not allowed).

[0031] Using the condition that the associated hard drive is in the second type of state as a criterion for determining a faulty node is actually a conservative strategy. It treats any node with a hard drive in any abnormal (non-UP / FAULT) state as a potential risk node, thus leaving a larger safety margin during redundancy assessment. Of course, in another implementation, the division between the first and second type of states can also be adjusted.

[0032] This embodiment adopts a more comprehensive definition, that is, as long as any hard disk on a node is in any of the second type states of OUT, DOWN or RECOVER, the node is counted as a faulty node.

[0033] Step S13: Based on the comparison between the total number of faulty nodes in the hard disk pool and the number of allowed faulty nodes, allow or deny the hard disk to be isolated associated with the isolation request.

[0034] After obtaining the current total number of faulty nodes and the allowed number of faulty nodes, the storage management device compares them. If the current total number of faulty nodes plus the number of faulty nodes that may be added during this isolation (i.e., if the node where the disk to be isolated is located is not currently a faulty node, then add 1; otherwise add 0) is less than or equal to the allowed number of faulty nodes, then isolation is allowed; otherwise isolation is rejected.

[0035] Simultaneously, the storage management device updates the state machine of the isolated disk, changing it from its original state (such as UP or DOWN) to OUT, and triggers the data reconstruction process. After reconstruction, the disk's state becomes FAULT. Subsequently, after the administrator replaces the disk with a new one, the new disk enters RECOVER mode and eventually returns to UP.

[0036] In one implementation, the states belonging to the second type of state include: DOWN state, which indicates that the OSD instance associated with the hard disk is abnormal but reconstruction has not been triggered, and the data on the disk may be incomplete; and / or OUT state, which indicates that the OSD instance associated with the hard disk is abnormal, reconstruction has been triggered but not completed, and the number of backups of the data on the disk on other disks may be insufficient; and / or RECOVER state, which indicates that the hard disk has been rejoined to the cluster from the abnormal state, the data is being reconstructed, and the number of backups of the data on the disk on other disks may be insufficient.

[0037] In one implementation, the step of responding to a hard disk isolation request, obtaining the hard disk pool to which the hard disk to be isolated associated with the hard disk isolation request belongs, and determining the total number of faulty nodes in the hard disk pool based on the status information of each hard disk in the hard disk pool, includes: obtaining the currently confirmed faulty nodes based on the recorded state machine status information of each hard disk in the hard disk pool; obtaining one or more nodes to which one or more hard disks associated with the hard disk isolation request belong, and treating the one or more nodes as faulty nodes; and counting the total number of faulty nodes after deduplicating the faulty nodes in the above two steps.

[0038] In one implementation, the step of allowing or denying the isolation request associated with the hard disk to be isolated based on the comparison between the total number of faulty nodes in the hard disk pool and the number of allowed faulty nodes includes: obtaining the number of allowed faulty nodes for each storage pool in the hard disk pool, taking the smallest one as the number of allowed faulty nodes, and if the total number of faulty nodes is less than or equal to the number of allowed faulty nodes, then the isolation request associated with the hard disk to be isolated is allowed.

[0039] In one implementation, such as Figure 2 The disk state machine has 5 possible states (UP, DOWN, OUT, FAULT, RECOVER) to record the state of each disk in the cluster. The transition between these 5 states is shown in Table 1.

[0040] Table 1 The trigger scenarios for route A include: disk failure scenarios (disk removal, slow disk, IO timeout, suspected bad disk, etc.); node failure scenarios (server shutdown, restart, network outage, etc., heartbeat timeout); and user command-line setting of disk (OSD) DOWN. All of these share the common characteristic of having an OSD instance status that changes from UP to DOWN on the disk during OSDMAP.

[0041] The trigger scenarios for route b include: disk failure scenarios, confirmed bad disk / strong isolation / pre-failure disk (assessment passed) EIO; disk OSD instance DOWN exceeding the threshold for more than 30 minutes; user command line setting disk (OUT) DOWN. The common characteristic of all these is that in OSDMAP, the OSD instance status on the disk is updated from DOWN to OUT.

[0042] The trigger scenarios for route C include: disk failure scenarios, confirmation of a bad disk event / passing of a pre-failure disk isolation assessment / strong isolation of a disk failure, etc.; and OSD instances in the UP state being set to OUT via user commands. The common characteristic of all these is that in OSDMAP, the OSD instance status on the disk is updated from UP to OUT.

[0043] The trigger scenario for route d is that the reconstruction is successfully completed and all data on the disk has been migrated.

[0044] The trigger scenarios for the e-route include: disk failure scenarios, such as successful disk insertion / suspected bad disk repair / successful pre-read and restart; node failure scenarios, such as fault recovery; and OSD instances in a DOWN state being set to UP via user commands. The common characteristic of all these scenarios is that an OSD instance on the disk in OSDMAP has had its state updated from DOWN to UP, and the reconstruction is not yet complete.

[0045] The trigger scenarios for route f include: disk failure scenarios, disk re-insertion, forced restart via user command; node failure recovery; and user command setting of the disk (OSD) to UP. A common characteristic of these scenarios is that during the reconstruction process, the OSD instance status on the disk in OSDMAP is updated from OUT to UP.

[0046] The trigger scenarios for the g route are: successful reconstruction and the cluster recovering to a healthy state.

[0047] The trigger scenarios for the h route include: disk failure scenarios, disk re-insertion, forced restart via user command; node failure recovery; and user command setting of the disk (OSD) UP. All of these characteristics indicate that the refactoring is complete.

[0048] The disk's status indicates whether it is faulty, serving as a basis for isolation assessment. The corresponding state transition scenarios are shown in the figure, based on the meaning of the disk state machine. By recording the state of each disk, if a node has a disk in a state machine state (DOWN / OUT / RECOVER), the node is considered abnormal. This allows for redundancy assessment based on which nodes in the cluster are currently abnormal.

[0049] See Figure 3 OSDMON is introduced as a unified statistician to record the state machines of each disk. Based on the state / PG state of each OSD, it records the current state machine and the current state after transitions for all disks in the cluster. When a faulty disk needs isolation assessment, the OSD requests isolation assessment from OSDMON. After assessment, OSDMON returns the result to the OSD and records the state machine state after transitions for that disk. When two or more disks are simultaneously undergoing isolation assessment, OSDMON processes the state update after each state machine transition sequentially, thus being able to globally determine each isolation assessment based on the latest disk state.

[0050] OSDMON performs isolation assessments based on the state machines of all disks within the cluster, their current fault status, and fault domains. The isolation assessment result is either "isolateable" or "cannot be isolated," and is returned to OSD. The specific isolation method is as follows (taking a host fault domain as an example): Traverse the set of disk pools reporting disk failures. Taking disk pool A as an example, find the number of currently failed disks in disk pool A based on the state machine of each disk, and record the corresponding host count. Add this to the number of hosts belonging to disk pool A reported in the isolation assessment (if the currently failed host and the host reported in the isolation assessment are the same, it is considered the same host and only recorded once) to get the total number of failed hosts. Obtain the minimum allowed number of failed hosts across all storage pools under disk pool A; this minimum value is the allowed number of failed hosts for disk pool A. If the total number of failed hosts is less than or equal to the allowed number of failures, then the disks belonging to disk pool A reported in the isolation assessment are allowed to be isolated. In one implementation, such as Figure 4This specification also provides a storage management device for use in a distributed storage cluster. The distributed storage cluster includes a disk pool, and the disk pool includes a plurality of disks. The device includes: a first module, used to pre-acquire and / or update and record the state information of the state machine of each disk in the disk pool, the state information including the current state of the disk state machine, including being in a first type state that allows direct isolation or a second type state that does not allow direct isolation; a second module, used to respond to a disk isolation request, acquire the disk pool to which the disk to be isolated associated with the disk isolation request belongs, and determine the total number of faulty nodes in the disk pool based on the state information of each disk in the disk pool, wherein a faulty node refers to a node whose associated disk is in the second type state; and a third module, used to allow or refuse to isolate the disk to be isolated associated with the disk isolation request based on the comparison between the total number of faulty nodes in the disk pool and the number of allowed faulty nodes.

[0051] In one implementation, the states belonging to the second type of state include: DOWN state, which indicates that the OSD instance associated with the hard disk is abnormal but reconstruction has not been triggered, and the data on the disk may be incomplete; and / or OUT state, which indicates that the OSD instance associated with the hard disk is abnormal, reconstruction has been triggered but not completed, and the number of backups of the data on the disk on other disks may be insufficient; and / or RECOVER state, which indicates that the hard disk has been rejoined to the cluster from the abnormal state, the data is being reconstructed, and the number of backups of the data on the disk on other disks may be insufficient.

[0052] In one implementation, the step of responding to a hard disk isolation request, obtaining the hard disk pool to which the hard disk to be isolated associated with the hard disk isolation request belongs, and determining the total number of faulty nodes in the hard disk pool based on the status information of each hard disk in the hard disk pool, includes: obtaining the currently confirmed faulty nodes based on the recorded state machine status information of each hard disk in the hard disk pool; obtaining one or more nodes to which one or more hard disks associated with the hard disk isolation request belong, and treating the one or more nodes as faulty nodes; and counting the total number of faulty nodes after deduplicating the faulty nodes in the above two steps.

[0053] In one implementation, the step of allowing or denying the isolation request associated with the hard disk to be isolated based on the comparison between the total number of faulty nodes in the hard disk pool and the number of allowed faulty nodes includes: obtaining the number of allowed faulty nodes for each storage pool in the hard disk pool, taking the smallest one as the number of allowed faulty nodes, and if the total number of faulty nodes is less than or equal to the number of allowed faulty nodes, then the isolation request associated with the hard disk to be isolated is allowed.

[0054] The implementation methods of the apparatus are the same as or similar to the corresponding implementation methods, and will not be described in detail here.

[0055] In one embodiment, this specification provides an electronic device including a processor and a readable storage medium storing machine-executable instructions executable by the processor. The processor executes the machine-executable instructions to implement the aforementioned storage management method. From a hardware perspective, a hardware architecture diagram can be found... Figure 5 As shown.

[0056] In one embodiment, this specification provides a readable storage medium storing machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the aforementioned storage management method.

[0057] Here, a readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, a readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0058] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0059] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0060] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification can take the form of a completely hardware implementation, a completely software implementation, or an implementation combining software and hardware aspects. Furthermore, embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments thereof. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification can take the form of a completely hardware implementation, a completely software implementation, or an implementation combining software and hardware aspects. Furthermore, this specification can take the form of a computer program product implemented on one or more computer-usable storage media (which may include, but are not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0065] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A storage management method, characterized in that, A storage management device applied to a distributed storage cluster, wherein the distributed storage cluster includes a disk pool, and the disk pool includes a plurality of disks, the method comprising: Pre-acquire and / or update and record the state information of the state machine of each hard disk in the hard disk pool. The state information includes the current state of the hard disk state machine, including being in a first type of state that allows direct isolation or a second type of state that does not allow direct isolation. In response to a disk isolation request, obtain the disk pool to which the disk to be isolated associated with the disk isolation request belongs, and determine the total number of faulty nodes in the disk pool based on the status information of each disk in the disk pool. Here, a faulty node refers to a node whose associated disk is in the second type of state. Based on the comparison between the total number of faulty nodes in the disk pool and the number of allowed faulty nodes, the isolation request for the disk to be isolated is allowed or denied.

2. The method according to claim 1, characterized in that, States belonging to the second type of state include: The DOWN status indicates that the OSD instance associated with the hard drive is abnormal but has not triggered a reconstruction, and the data on the disk may be incomplete. And / or, OUT status, used to indicate that the OSD instance associated with the hard disk is abnormal, a reconstruction has been triggered but not completed, or the data on the disk may not be backed up enough on other disks; And / or, RECOVER status, used to indicate that the hard drive has been rejoined to the cluster from an abnormal state, the data is being rebuilt, and the number of backups of the data on the disk on other disks may be insufficient.

3. The method according to claim 1, characterized in that, In response to a hard disk isolation request, the process involves obtaining the hard disk pool to which the hard disk to be isolated belongs, and determining the total number of faulty nodes in the hard disk pool based on the status information of each hard disk within the pool, including: Based on the state information of each hard drive in the recorded hard drive pool, obtain the currently confirmed fault nodes; Based on the disk isolation request, obtain one or more nodes to which one or more disks associated with the disk isolation request belong, and regard the one or more nodes as faulty nodes; After deduplicating the faulty nodes in the above two steps, the total number of faulty nodes is counted.

4. The method according to claim 1, characterized in that, The step of allowing or denying the isolation of hard drives associated with the isolation request based on the comparison between the total number of faulty nodes in the hard drive pool and the number of allowed faulty nodes includes: Obtain the number of allowed faulty nodes for each storage pool within the disk pool, and take the smallest number as the allowed number of faulty nodes. If the total number of faulty nodes is less than or equal to the allowed number of faulty nodes, then the disk to be isolated associated with the isolation request is allowed.

5. A storage management device, characterized in that, A storage management device for use in a distributed storage cluster, wherein the distributed storage cluster includes a disk pool, and the disk pool includes a plurality of disks, the device comprising: The first module is used to pre-acquire and / or update and record the state information of the state machine of each hard disk in the hard disk pool. The state information includes the current state of the hard disk state machine, including being in a first type of state that allows direct isolation or a second type of state that does not allow direct isolation. The second module is used to respond to a hard disk isolation request, obtain the hard disk pool to which the hard disk to be isolated associated with the hard disk isolation request belongs, and determine the total number of faulty nodes in the hard disk pool based on the status information of each hard disk in the hard disk pool. Here, a faulty node refers to a node whose associated hard disk is in the second type of state. The third module is used to allow or deny the isolation request associated with the hard disk to be isolated based on the comparison between the total number of faulty nodes in the hard disk pool and the number of allowed faulty nodes.

6. The apparatus according to claim 5, characterized in that, States belonging to the second type of state include: The DOWN status indicates that the OSD instance associated with the hard drive is abnormal but has not triggered a reconstruction, and the data on the disk may be incomplete. And / or, OUT status, used to indicate that the OSD instance associated with the hard disk is abnormal, a reconstruction has been triggered but not completed, or the data on the disk may not be backed up enough on other disks; And / or, RECOVER status, used to indicate that the hard drive has been rejoined to the cluster from an abnormal state, the data is being rebuilt, and the number of backups of the data on the disk on other disks may be insufficient.

7. The apparatus according to claim 5, characterized in that, In response to a hard disk isolation request, the process involves obtaining the hard disk pool to which the hard disk to be isolated belongs, and determining the total number of faulty nodes in the hard disk pool based on the status information of each hard disk within the pool, including: Based on the state information of each hard drive in the recorded hard drive pool, obtain the currently confirmed fault nodes; Based on the disk isolation request, obtain one or more nodes to which one or more disks associated with the disk isolation request belong, and regard the one or more nodes as faulty nodes; After deduplicating the faulty nodes in the above two steps, the total number of faulty nodes is counted.

8. The apparatus according to claim 5, characterized in that, The step of allowing or denying the isolation of hard drives associated with the isolation request based on the comparison between the total number of faulty nodes in the hard drive pool and the number of allowed faulty nodes includes: Obtain the number of allowed faulty nodes for each storage pool within the disk pool, and take the smallest number as the allowed number of faulty nodes. If the total number of faulty nodes is less than or equal to the allowed number of faulty nodes, then the disk to be isolated associated with the isolation request is allowed.

9. An electronic device, characterized in that, include: A processor and a readable storage medium storing machine-executable instructions that can be executed by the processor to implement the method of any one of claims 1-4.

10. A readable storage medium, characterized in that, The readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method described in any one of claims 1-4.

Images