Distributed storage cluster data rescue method and device

Abstract

The invention relates to the technical field of distributed storage, and discloses a distributed storage cluster data rescue method and device.The method comprises the steps that when it is detected that a storage pool is abnormal, an OSD process related to the storage pool is stopped, and in an OSD running diagram, a target version number before the abnormality occurs is determined; suspending the service of the cluster management process, performing version rollback on the cluster management process, and rolling back the OSD running diagram from the original version number to the target version number; and restarting the service of the cluster management process, pulling up the version number of the OSD running diagram through the cluster management process, and when the pulled-up version number represents the latest version in the distributed storage cluster, restarting the OSD process related to the storage pool, and activating the storage pool to rescue the data in the storage pool. According to the method provided by the embodiment of the invention, the version of the preset database is rolled back and pulled up, so that the data in the storage pool can be recovered and rescued when the storage pool is abnormal, and the fault tolerance and the stability of the system are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of distributed storage technology, and in particular to a distributed storage cluster data rescue method and device. Background Art

[0002] Ceph is a widely used distributed storage system. In the Ceph system, data is managed through storage pools, which are a set of logical partitions used to store data in the cluster. Each storage pool divides data through placement groups (PGs). However, failures are inevitable in distributed storage systems. When anomalies occur in Ceph, serious data loss and service interruption may occur.

[0003] In the related art, data recovery is often performed on a single PG, but this method cannot effectively deal with large-scale failures such as the deletion of a storage pool or the damage of a large number of PGs in the storage pool, which may lead to poor data rescue results and cause a large amount of data loss and damage. Therefore, in the Ceph system, how to still be able to effectively rescue data when facing large-scale failures of abnormal storage pools has become a problem that needs to be solved. Summary of the invention

[0004] In view of this, the present disclosure provides a distributed storage cluster data rescue method and device to solve the problem of how to effectively rescue data when facing a large-scale failure of an abnormal level in a storage pool.

[0005] On the one hand, the present disclosure provides a distributed storage cluster data rescue method, the method comprising: when an abnormality is detected in a storage pool of a distributed storage cluster, stopping the OSD process related to the storage pool, and determining the target version number before the abnormality occurs in the OSD operation map of a preset database of the cluster management process; pausing the service of the cluster management process, and in the OSD operation map, rolling back the cluster management process from the original version number to the target version number; the original version number is used to represent the version number corresponding to the OSD operation map after the abnormality occurs; restarting the service of the cluster management process, raising the version number of the OSD operation map through the cluster management process, and when the raised version number represents the latest version in the distributed storage cluster, restarting the OSD process related to the storage pool, activating the storage pool, and rescuing the data in the storage pool.

[0006] On the other hand, the present disclosure also provides a distributed storage cluster data rescue device, the device includes a determination module, a version rollback module and a data rescue module, wherein: the determination module is used to stop the OSD process related to the storage pool when an abnormality is detected in the storage pool of the distributed storage cluster, and determine the target version number before the abnormality occurs in the OSD operation diagram of the preset database of the cluster management process; the version rollback module is used to suspend the service of the cluster management process, and in the OSD operation diagram, perform a version rollback on the cluster management process, and roll back the OSD operation diagram from the original version number to the target version number; the original version number is used to represent the version number corresponding to the OSD operation diagram after the abnormality occurs; the data rescue module is used to restart the service of the cluster management process, raise the version number of the OSD operation diagram through the cluster management process, and when the raised version number represents the latest version in the distributed storage cluster, restart the OSD process related to the storage pool, activate the storage pool, and rescue the data in the storage pool.

[0007] On the other hand, the present disclosure further provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to implement the above-mentioned distributed storage cluster data rescue method.

[0008] On the other hand, the present disclosure further provides a computer program product, including computer instructions, where the computer instructions are used to enable a computer to execute the above-mentioned distributed storage cluster data rescue method.

[0009] By using the distributed storage cluster data rescue method and device of the above-mentioned embodiment of the present invention, when an abnormality is detected in the storage pool of the distributed storage cluster Ceph, the service of the cluster management process is suspended, and the version number of the cluster management process is backtracked in the OSD operation diagram, so that the cluster management process is backtracked to the target version number before the abnormality occurs, and the service of the cluster management process is restarted, so that the cluster management process can quickly increase the version number by operating the OSD process, and the storage pool is activated to rescue data. The storage pool data can be efficiently restored in the distributed storage cluster, which improves the data integrity and recovery efficiency of the Ceph system under sudden failures, thereby greatly enhancing the fault tolerance and stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1A schematic diagram showing a process flow of a distributed storage cluster data rescue method according to an embodiment of the present disclosure is shown;

[0012] Figure 2a It is a schematic diagram of the OSD operation diagram structure of a distributed storage cluster data rescue method provided by an embodiment of the present disclosure;

[0013] Figure 2b It may be a schematic diagram of an abnormal structure of an OSD operation diagram of a distributed storage cluster data rescue method provided by an embodiment of the present disclosure;

[0014] Figure 2c It may be a schematic diagram of a deletion range of an OSD operation diagram of a distributed storage cluster data rescue method provided by an embodiment of the present disclosure;

[0015] Figure 2d It may be a schematic diagram of the latest version number of an OSD operation diagram of a distributed storage cluster data rescue method provided by an embodiment of the present disclosure;

[0016] Figure 3 It is a structural schematic diagram of a distributed storage cluster data rescue device provided by an embodiment of the present disclosure;

[0017] Figure 4 It is a structural diagram of another distributed storage cluster data rescue device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] Ceph is an open source distributed storage system designed to provide high-availability, scalable and fault-tolerant storage solutions for enterprises and scientific research applications. It uses a no-single-point-of-failure architecture, can effectively distribute and store data, and ensures high data availability through a self-healing mechanism. Ceph can be used as object storage, block storage and file system at the same time, and is widely used in cloud storage, big data processing and other fields.

[0019] In the Ceph system, data is managed through storage pools. A storage pool is a set of logical partitions used to store data in the cluster. Each storage pool uses PG to divide data. PG is the basic unit used by Ceph for data distribution. Data objects are mapped to specific PGs through the CRUSH algorithm, and then the data is distributed to multiple OSDs through the replication mechanism. This distributed architecture ensures data redundancy and high availability. However, especially in large-scale distributed storage systems, failures are inevitable.

[0020] In this context, the following problems often arise:

[0021] 1. The Ceph system does not provide a built-in data recovery mechanism to deal with the situation where the storage pool itself is accidentally deleted. Therefore, after the storage pool is permanently deleted, the data in the storage pool is usually unrecoverable.

[0022] 2. The data rescue methods in the related technologies often repair the data of a single PG. When the entire storage pool is deleted or a large number of PGs can no longer work properly, the related technologies cannot be used. They cannot effectively deal with large-scale failures such as the storage pool being deleted or a large number of PGs in the storage pool being damaged, which may result in poor data rescue results and cause a large amount of data to be lost or damaged.

[0023] To solve the above problems, various embodiments of the present invention provide a distributed storage cluster data rescue method, the method comprising: when an abnormality is detected in a storage pool of a distributed storage cluster, stopping the OSD process related to the storage pool, and determining the target version number before the abnormality occurs in the OSD operation map of a preset database of the cluster management process; pausing the service of the cluster management process, and in the OSD operation map, rolling back the cluster management process from the original version number to the target version number; the original version number is used to represent the version number corresponding to the OSD operation map after the abnormality occurs; restarting the service of the cluster management process, raising the version number of the OSD operation map through the cluster management process, and when the raised version number represents the latest version in the distributed storage cluster, restarting the OSD process related to the storage pool, activating the storage pool, and rescuing the data in the storage pool.

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0025] Please refer to Figure 1 , Figure 1 A schematic diagram of a process of a distributed storage cluster data rescue method according to an embodiment of the present disclosure is shown. The process of the method may include the following steps:

[0026] Step S101, when an abnormality is detected in a storage pool of a distributed storage cluster, the OSD process related to the storage pool is stopped, and the target version number before the abnormality occurs is determined in the OSD operation diagram of the preset database of the cluster management process.

[0027] In this embodiment, the distributed storage cluster may refer to Ceph, which may be a distributed storage platform based on object storage. Data is stored in multiple OSDs in a distributed manner to eliminate the risk of single point failure. The storage pool may refer to a container for data storage in a Ceph cluster, which determines how data is distributed, stored, and protected in the cluster. The storage pool is a virtualized logical concept that abstracts physical storage resources (such as OSDs) so that users can divide multiple independent storage areas according to their needs. Data can be distributed in the form of PGs in the storage pool. Each PG is responsible for storing a portion of data, and then the PG is mapped to a specific OSD.

[0028] Here, the storage pool may have large-scale anomalies during the operation of the Ceph cluster. For example, the large-scale anomalies may include at least one of the following: the storage pool is accidentally deleted, a large number of PGs in the storage pool are damaged, and the capacity of the storage pool exceeds the limit.

[0029] Among them, accidental deletion of a storage pool may refer to a user or administrator accidentally deleting a storage pool during an operation, causing all data in the storage pool to be marked as inaccessible, which may cause all data objects in the storage pool and their corresponding PGs to be cleared, and the data may be permanently lost. Damage to a large number of PGs in a storage pool may refer to an abnormal state of the PGs in the storage pool, resulting in inaccessible data, which may cause incomplete or damaged data distribution, and partial or all data loss. Excess capacity of a storage pool may refer to the storage pool's used capacity exceeding its quota or the total storage capacity of the cluster, resulting in a write failure, which may cause data to be unable to be written to new objects, and the storage pool may stop serving.

[0030] Furthermore, the OSD process associated with the storage pool may refer to the OSD instances associated with the storage pool, which store the object data or metadata in the storage pool. The association here may be determined by the PG mapping of the storage pool and the Controlled Replication Under Scalable Hashing (CRUSH) algorithm.

[0031] Furthermore, in the OSD operation diagram of the preset database of the cluster management process, the cluster management process may refer to a monitor (mon) component, and the mon component may be responsible for managing the state and operation diagram of the cluster and maintaining distributed consistency. Specifically, the mon component may use an internal database to store at least one operation diagram in the cluster. Here, the preset database may refer to an internal database of the mon component, for example, the internal database may include but is not limited to a RocksDB database. The RocksDB database may store metadata of at least one cluster state, for example, an OSD operation diagram may be stored.

[0032] In a Ceph cluster, the version number can be used to identify different versions of the running graph. It can be a monotonically increasing integer to reflect changes in cluster status, configuration, or data.

[0033] Step S102, suspending the service of the cluster management process, and rolling back the version of the cluster management process in the OSD operation map, rolling back the OSD operation map from the original version number to the target version number.

[0034] In this embodiment, the original version number is used to represent the version number corresponding to the OSD operation diagram after the exception occurs.

[0035] Suspending the service of the cluster management process may refer to temporarily shutting down the process of the mon component in the Ceph cluster. The reason for suspending the service of the mon component may be to prevent receiving a new version of the OSD operation map while performing a version rollback, thereby avoiding data inconsistency or operation conflict in the cluster.

[0036] The mon component service can be suspended by using a command line tool. During the suspension of the mon component service, the mon component no longer receives status updates from the OSD or provides new configuration data to other components of the cluster.

[0037] Rolling back the version of the mon component process can make the mon component perceive the state before the error occurs again, and make the mon component recognize the target version number of the OSD operation map after the rollback, and use the target version number as the "latest" version of the current cluster. At this time, the mon component can control the state and configuration in the cluster according to the rolled-back version, so that other components in the cluster can obtain the rolled-back OSD operation map through the mon component, and then restore the normal operation state, thereby recovering the accidentally deleted or damaged data.

[0038] Step S103, restart the service of the cluster management process, and increase the version number of the OSD operation diagram through the cluster management process. When the increased version number represents the latest version in the distributed storage cluster, restart the OSD process related to the storage pool, activate the storage pool, and rescue the data in the storage pool.

[0039] In this embodiment, the service of the mon component is restarted so that it can reload the OSD operation map of the cluster and update the version number of the OSD operation map.

[0040] Here, in the Ceph cluster, the data information broadcast by the mon component to other components includes the version number of the OSD operation map. When other components determine that the version number of the mon component is lower than their own version number, other components will ignore the data information broadcast by the mon component. When other components determine that the version number of the mon component is higher than their own version number, other components will receive the data information broadcast by the mon component.

[0041] Therefore, it is necessary to increase the version number of the OSD operation map through the mon component to ensure that the version number of the OSD operation map maintained by the mon component is higher than the version number held by any running component in the cluster, so that other components in the cluster can obtain the latest OSD operation map from the mon component and synchronize to the latest status in the cluster.

[0042] Furthermore, restarting the OSD process related to the storage pool can ensure that the OSD process is restarted and the data processing function related to the storage pool is restored. Restarting the OSD process can also be completed using a command line tool.

[0043] Furthermore, when all OSD processes related to the storage pool are restarted and are active, the storage pool can be activated by force-create, at least one PG in the storage pool can be recreated, and the data can be reallocated to the corresponding OSD process to verify the data.

[0044] Here, the verification of data may include, but is not limited to: checking the scope of data loss during the recovery process and ensuring that the storage pool can provide normal services after recovery.

[0045] In the distributed storage cluster data rescue method and device of the above-mentioned embodiments of the present invention, the cluster can be restored to the state before the exception occurred through version rollback, thereby minimizing data loss caused by storage pool exceptions; by quickly rolling back to a stable historical version when the storage pool is damaged, the time required for recovery can be reduced and long-term interruption of storage system services can be avoided; during the storage pool recovery process, by ensuring the version synchronization and consistency of all components, unnecessary cleanup or reconstruction of other components in the cluster is reduced, thereby improving the recovery speed and disaster recovery capabilities of the cluster.

[0046] In step S101 of the above embodiment, when an abnormality is detected in a storage pool of a distributed storage cluster, the OSD process related to the storage pool is stopped, and the target version number before the abnormality occurs is determined in the OSD operation diagram of the preset database of the cluster management process, including:

[0047] Detect whether there is an abnormality in the storage pool in the distributed storage cluster. When an abnormality is detected in the storage pool, query the OSD process related to the storage pool and stop the OSD process;

[0048] Determine the time when the exception occurred, and based on the time location, determine the target version number that is before the occurrence time in the OSD operation map.

[0049] In this embodiment, the method for detecting whether there is an abnormality in the storage pool in the distributed storage cluster may include but is not limited to: using the Ceph command line tool to view the overall status of the cluster, using a log monitoring and alarm system, and using an automated inspection robot.

[0050] Among them, using Ceph command line tools to view the overall status of the cluster can refer to using Ceph command line tools (such as ceph status, ceph health) to view the overall status of the cluster, and identifying whether there are abnormalities based on the output information of the health status. Using a log monitoring and alarm system can refer to analyzing the detailed logs generated by the Ceph cluster through a log analysis tool or a centralized monitoring system, or setting alarm notifications in the Ceph cluster to automatically trigger an alarm when an abnormal storage pool status or OSD component status is detected. Using an automated inspection robot can refer to using an automated inspection script to regularly check the health status of the storage pool and the OSD status.

[0051] Furthermore, when an abnormality is detected in the storage pool, the OSD process related to the storage pool is queried, which may include but is not limited to: using the CRUSH mapping algorithm, querying the PG status, and querying the OSD log.

[0052] Among them, using the CRUSH mapping algorithm may refer to using the CRUSH algorithm to determine the OSD to which the data object belonging to the storage pool is mapped. Querying the PG status may refer to obtaining the correspondence between the PG and the OSD through the PG status, thereby finding out all OSDs associated with the storage pool. Querying the OSD log may refer to finding out which OSDs are involved in the target storage pool through log analysis.

[0053] Furthermore, the time when the exception occurred is determined. Based on the time location, the target version number before the time of occurrence is determined in the OSD operation graph. This can be achieved based on the following steps:

[0054] The time when the exception occurred is determined by the preset time acquisition method. The mon component is used to find the target version number closest to the time point in the RocksDB database based on the time when the exception occurred. The located target version number is recorded as the baseline version for subsequent version rollback.

[0055] Among them, the preset time acquisition method may include log analysis, alarm system record extraction and command line query.

[0056] Finding the target version number closest to the time point may include: tracing back from the time point when the exception occurred to find the target version number, wherein the status of the target version number shows that the cluster was healthy or the storage pool had no exception at that time.

[0057] In the distributed storage cluster data rescue method and device of the above-mentioned embodiment of the present invention, when an abnormality occurs in the storage pool of the distributed storage cluster, the OSD process related to the abnormal storage pool can be detected and stopped in real time, so as to quickly isolate the fault and prevent further data damage or loss; through the rapid rollback and recovery operations of the OSD process and the OSD operation diagram, the fault handling process is simplified, the recovery operation is fast and efficient, and the stability and recovery efficiency of the Ceph cluster are significantly improved.

[0058] In a possible implementation of the above embodiment, the method further includes:

[0059] After the target version number is determined in the OSD operation map, at least one version number after the target version number in the OSD operation map and data information corresponding to each version number in the at least one version number are deleted.

[0060] In this embodiment, when the target version number before the exception occurs is determined, the metadata of each version number after the target version number is deleted to ensure that the OSD operation diagram no longer contains versions after the target version number, and the data information of each version number after the target version number is cleared.

[0061] Here, the purpose of deleting all versions after the target version number can be to roll back the cluster state to a known healthy state and clear the impact of the abnormal state.

[0062] Among them, since the version number in the OSD operation map is monotonically increasing, if there are abnormal versions, it may cause confusion or conflict in the operation map record logic. When these versions are deleted, the version number of the operation map can maintain a clear sequence relationship.

[0063] In the distributed storage cluster data rescue method and device of the above-mentioned embodiments of the present invention, by deleting the version number after the target version number and its corresponding data information, it is possible to effectively roll back to a healthy state where the storage pool has no abnormalities, avoiding erroneous operations or data cleanup caused by abnormal versions, thereby improving the effect of data rescue when abnormalities occur in the storage pool.

[0064] In step S102 of the above embodiment, in the OSD operation graph, the cluster management process is rolled back to roll back the OSD operation graph from the original version number to the target version number, which is implemented based on the following steps:

[0065] Modify the preset pointer in the preset database so that the preset pointer points to the target version number in the OSD operation diagram, so that the distributed storage cluster can be traced back to the period corresponding to the target version number.

[0066] In this example, modifying the preset pointer in the preset database so that the preset pointer points to the target version number in the OSD operation diagram so that the distributed storage cluster can be traced back to the time period corresponding to the target version number can include: modifying the three preset pointers of the mon component so that the three preset pointers point to the positions corresponding to the target version number in the OSD operation diagram.

[0067] Specifically, the mon component maintains three pointers in the preset database, namely first_committed, last_committed and full_latest, where first_committed can point to the earliest OSD version number of the mon component, last_committed can point to the latest version number currently maintained by the mon component, that is, the latest version number in the cluster, and full_latest can point to the data content of the OSD operation diagram corresponding to the latest version number.

[0068] Furthermore, pointing the three preset pointers to the position corresponding to the target version number in the OSD run graph may include: pointing last_committed to the position where the target version number is located, and pointing full_latest to the data content corresponding to the target version number.

[0069] In the distributed storage cluster data rescue method and device of the above embodiment of the present disclosure, by modifying the three pointers of the mon component in the preset database to point to the target version number and its corresponding data content, it is possible to quickly roll back to the state when the storage pool is not abnormal. This not only avoids the impact of the abnormal version on the cluster state, but also restores the normal operation logic of the cluster, providing a basic guarantee for subsequent data rescue.

[0070] In step S103 of the above embodiment, restarting the service of the cluster management process and raising the version number of the OSD operation map through the cluster management process includes:

[0071] After the service of the cluster management process is restarted, based on a preset control method, the cluster management process is used to control at least one OSD process in the distributed storage cluster, and the version number of the OSD operation map is raised at least once.

[0072] In this embodiment, the preset control mode may refer to performing a preset operation on the OSD, and the preset operation may include but is not limited to: restarting the OSD, setting the OSD to IN, or setting the OSD to OUT. Among them, setting the OSD to IN may indicate that the OSD is allowed to participate in data storage and reading operations, and setting the OSD to OUT may cause the OSD to temporarily withdraw from the storage and reading tasks of the cluster.

[0073] Here, each preset operation performed on the OSD can update the version number of the OSD operation map once, until the version number in the OSD operation map of the mon component is raised to the latest version number in the entire cluster, that is, the version number of the mon component is greater than the OSD operation map version numbers of all other components in the cluster.

[0074] Furthermore, when the version number in the OSD operation diagram of the mon component is raised to the latest version number in the entire cluster, the previously stopped OSD process related to the storage pool is restarted to trigger the data self-check and recovery process of the OSD process, thereby reducing data loss caused by abnormalities.

[0075] In the distributed storage cluster data rescue method and device of the above-mentioned embodiment of the present invention, by raising the version number, other components in the cluster are prevented from continuing to use the abnormal version of the OSD operation diagram, thereby avoiding potential erroneous operations from further affecting the storage pool and cluster; by performing IN / OUT operations on the OSD or restarting the OSD, the self-healing mechanism of the Ceph system is triggered, thereby restoring the normal operation of the cluster and minimizing data loss caused by abnormalities.

[0076] In a specific embodiment, Figure 2a It is a schematic diagram of the OSD operation diagram structure of a distributed storage cluster data rescue method provided by an embodiment of the present disclosure, wherein: the epoch1 pointed to by first_committed in the OSD operation diagram may refer to the earliest OSD version number of the mon component, the epoch104 pointed to by last_committed may refer to the latest OSD version number of the mon component, and full_latest points to the data content of the latest OSD version number, and each OSD version number corresponds to a data content osdmap.

[0077] Figure 2bIt may be a schematic diagram of the abnormal structure of the OSD operation diagram of a distributed storage cluster data rescue method provided by an embodiment of the present disclosure, wherein, when an abnormality occurs in the storage pool in version 101 and version 102, after other components in the cluster obtain the OSD operation diagram of these versions, they find that the storage pool has been deleted and start to clean up the data on their own disks. At this time, the version number will continue to increase. In this case, a large number of OSDs will be required to clean up the data. If it is not handled in time, the data may be permanently lost.

[0078] Figure 2c It may be a schematic diagram of the deletion range of the OSD operation diagram of a distributed storage cluster data rescue method provided by an embodiment of the present disclosure, wherein the version 101 where the abnormality is first detected and subsequent versions are used as the data range to be deleted.

[0079] Figure 2d It can be a schematic diagram of the latest version number of the OSD operation diagram of a distributed storage cluster data rescue method provided by an embodiment of the present disclosure, wherein the last_committed of the mon component is pointed to the version number before the exception occurred (that is, version 100) as the latest version number in the cluster, and full_latest is pointed to the data content of the OSD operation diagram corresponding to version 100, and the version number is raised and the data is recovered starting from version 100.

[0080] In one embodiment, a distributed storage cluster data rescue device 30 is provided, and the distributed storage cluster data rescue device 30 corresponds one-to-one to the distributed storage cluster data rescue method in the above embodiment. Figure 3 As shown, the distributed storage cluster data rescue device 30 includes a determination module 301, a version rollback module 302 and a data rescue module 303, wherein each functional module is described in detail as follows:

[0081] The determination module 301 is used to stop the OSD process related to the storage pool when an abnormality is detected in the storage pool of the distributed storage cluster, and determine the target version number before the abnormality occurs in the OSD operation diagram of the preset database of the cluster management process;

[0082] The version rollback module 302 is used to suspend the service of the cluster management process, and in the OSD operation diagram, perform a version rollback on the cluster management process, and roll back the OSD operation diagram from the original version number to the target version number; the original version number is used to represent the version number corresponding to the OSD operation diagram after the exception occurs;

[0083] The data rescue module 303 is used to restart the service of the cluster management process. It raises the version number of the OSD operation diagram through the cluster management process. When the raised version number represents the latest version in the distributed storage cluster, it restarts the OSD process related to the storage pool and activates the storage pool to rescue the data in the storage pool.

[0084] In one embodiment, the determination module 301 is used to detect whether there is an abnormality in the storage pool in the distributed storage cluster. When the storage pool is detected to be abnormal, query the OSD process related to the storage pool and stop the OSD process;

[0085] Determine the time when the exception occurred, and based on the time location, determine the target version number that is before the occurrence time in the OSD operation map.

[0086] In one embodiment, the apparatus 30 further includes: a deletion module 304, wherein:

[0087] The deleting module 304 is used to delete at least one version number after the target version number in the OSD running map, and data information corresponding to each version number in the at least one version number after the target version number in the OSD running map.

[0088] In one embodiment, the version rollback module 302 is used to modify a preset pointer in a preset database so that the preset pointer points to a target version number in the OSD operation diagram, so that the distributed storage cluster can be rolled back to a period corresponding to the target version number.

[0089] In one embodiment, the data rescue module 303 is used to control at least one OSD process in the distributed storage cluster using the cluster management process based on a preset control method after the service of the cluster management process is restarted, and to increase the version number of the OSD operation diagram at least once.

[0090] It should be noted that: the distributed storage cluster data rescue device provided in the above embodiment only uses the division of the above program modules as an example to illustrate when implementing the corresponding distributed storage cluster data rescue method. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the above system can be divided into different program modules to complete all or part of the above-described processing. In addition, the system provided in the above embodiment and the corresponding Figure 1 The embodiments of the method shown belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0091] The present disclosure also provides a computer device having the above Figure 3 The distributed storage cluster data rescue device shown.

[0092] Please refer to 4, Figure 4is a schematic diagram of the structure of another distributed storage cluster data rescue device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0093] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0094] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0095] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0097] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0098] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0099] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0100] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0101] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0102] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A distributed storage cluster data rescue method, characterized in that: Methods include: When an abnormality is detected in a storage pool of a distributed storage cluster, the OSD process related to the storage pool is stopped, and the target version number before the abnormality occurs is determined in the OSD operation diagram of the preset database of the cluster management process; Suspending the service of the cluster management process, and in the OSD operation map, rolling back the cluster management process from the original version number to the target version number; the original version number is used to represent the version number of the OSD operation map corresponding to the exception; Restart the service of the cluster management process, and raise the version number of the OSD operation diagram through the cluster management process. When the raised version number represents the latest version in the distributed storage cluster, restart the OSD process related to the storage pool and activate the storage pool to rescue the data in the storage pool.

2. The method according to claim 1, characterized in that When an abnormality is detected in a storage pool of the distributed storage cluster, the OSD process related to the storage pool is stopped, and the target version number before the abnormality occurs is determined in the OSD operation diagram of the preset database of the cluster management process, including: Detect whether there is an abnormality in a storage pool in a distributed storage cluster. When an abnormality is detected in the storage pool, query an OSD process related to the storage pool and stop the OSD process; The occurrence time of the exception is determined, and based on the time positioning, a target version number located before the occurrence time is determined in the OSD operation map.

3. The method according to claim 2, characterized in that The method further comprises: After the target version number is determined in the OSD operation map, at least one version number after the target version number in the OSD operation map and data information corresponding to each version number in the at least one version number are deleted.

4. The method according to claim 3, characterized in that In the OSD operation graph, the cluster management process is rolled back to roll back the OSD operation graph from the original version number to the target version number, which is achieved based on the following steps: Modify the preset pointer in the preset database so that the preset pointer points to the target version number in the OSD operation diagram, so that the distributed storage cluster is traced back to the period corresponding to the target version number.

5. The method according to claim 1, characterized in that The restarting of the service of the cluster management process, raising the version number of the OSD operation graph through the cluster management process, includes: After the service of the cluster management process is restarted, based on a preset control method, the cluster management process is used to control at least one OSD process in the distributed storage cluster, and the version number of the OSD operation diagram is increased at least once.

6. A distributed storage cluster data rescue device, characterized in that: The distributed storage cluster data rescue device includes a determination module, a version rollback module and a data rescue module, wherein: The determination module is used to stop the OSD process related to the storage pool when an abnormality is detected in the storage pool of the distributed storage cluster, and determine the target version number before the abnormality occurs in the OSD operation diagram of the preset database of the cluster management process; The version rollback module is used to suspend the service of the cluster management process, and in the OSD operation diagram, perform a version rollback on the cluster management process, rolling back the OSD operation diagram from the original version number to the target version number; the original version number is used to represent the version number corresponding to the OSD operation diagram after the exception occurs; The data rescue module is used to restart the service of the cluster management process, and to increase the version number of the OSD operation diagram through the cluster management process. When the increased version number represents the latest version in the distributed storage cluster, the OSD process related to the storage pool is restarted, and the storage pool is activated to rescue the data in the storage pool.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the distributed storage cluster data rescue method described in claims 1-5.

8. A computer program product, characterized in that It comprises computer instructions, and the computer instructions are used to enable a computer to execute the distributed storage cluster data rescue method according to any one of claims 1-5.

Images