Metadata recovery method and metadata recovery device

Abstract

The invention relates to a metadata recovery method and a metadata recovery device. The method comprises the following steps: detecting the loss condition of metadata in a plurality of storage spaces in a storage system; and under the condition that the metadata in the target storage space is lost, recovering the metadata in the target storage space. Wherein the plurality of storage spaces are obtained by dividing storage buckets in the storage system; the target storage space is at least one storage space in the storage system. By adopting the method, the introduction of recovery of a large amount of non-lost metadata can be avoided, and the metadata recovery efficiency is further improved.

Description

Technical Field

[0001] The present application relates to the technical field of storage systems, and in particular to a metadata recovery method and a metadata recovery device. Background Art

[0002] In a storage system, metadata is data that describes data. Data and metadata are usually stored separately, and data can be processed by retrieving and managing metadata. In this case, once data or metadata is lost, it needs to be restored in time to ensure data consistency.

[0003] Taking metadata loss as an example, in the related art, when restoring the metadata of a storage system, the storage bucket corresponding to the lost metadata is usually used as the recovery unit, and all the metadata in the storage bucket is rolled back to a certain moment, resulting in the introduction of a large amount of non-lost metadata to be restored when restoring the lost metadata, affecting the recovery efficiency of the lost metadata. Summary of the invention

[0004] Based on this, it is necessary to provide a metadata recovery method and a metadata recovery device to address the above technical problems, which can avoid introducing a large amount of non-lost metadata recovery and thus improve metadata recovery efficiency.

[0005] In a first aspect, the present application provides a metadata recovery method, the method comprising:

[0006] Detecting metadata loss in multiple storage spaces in a storage system; the multiple storage spaces are obtained by dividing storage buckets in the storage system;

[0007] In the case where metadata of a target storage space is lost, the metadata in the target storage space is restored; the target storage space is at least one storage space in the storage system.

[0008] In the technical solution provided in the embodiment of the present application, the loss of metadata in multiple storage spaces in the storage system is detected, and when the metadata of the target storage space is lost, the metadata in the target storage space is restored. Among them, the multiple storage spaces are obtained by dividing the storage buckets in the storage system, and the target storage space is at least one storage space in the storage system. It is equivalent to further subdividing the storage unit of the metadata from the storage bucket into the storage space. In this way, when the metadata is restored, a single target storage space is used as the minimum recovery unit for restoration, so as to avoid introducing the recovery of metadata of non-target storage spaces in the storage bucket to which the target storage space belongs, thereby achieving accurate recovery of the metadata of the target storage space, and thus improving the efficiency of metadata recovery.

[0009] In one embodiment, detecting loss of metadata in a plurality of storage spaces in a storage system includes:

[0010] Get the fragmentation rate of each storage space;

[0011] Determine a storage space whose fragmentation rate is higher than a preset fragmentation rate threshold as a candidate storage space;

[0012] The metadata of the candidate storage spaces is queried in the storage system to determine the loss status of the metadata of each candidate storage space.

[0013] In the technical solution provided in the embodiment of the present application, the fragmentation rate of each storage space is compared with a preset fragmentation rate threshold, and the candidate storage space with a fragmentation rate higher than the preset fragmentation rate threshold is determined, which is equivalent to preliminarily screening the storage space to be cleaned up in the storage system, that is, the storage space for which metadata needs to be retrieved in the embodiment of the present application, thereby reducing the detection scope and improving the detection speed of metadata loss.

[0014] In one embodiment, metadata of each storage space is stored in an object table; querying metadata of candidate storage spaces in a storage system to determine loss of metadata of each candidate storage space includes:

[0015] Querying an object table of candidate storage spaces in a storage system;

[0016] When the query result meets the preset condition, the candidate storage space is determined as the target storage space, and the metadata of the target storage space is determined to be lost;

[0017] The preset condition includes that the object table of the candidate storage space does not exist, or the content of the object table of the candidate storage space is incomplete.

[0018] In the technical solution provided in the embodiment of the present application, for any candidate storage space, the object table of the candidate storage space is queried in the storage system. If the storage system does not include the object table of the candidate storage space, or the content of the object table of the candidate storage space in the storage system is incomplete, the candidate storage space is determined as the target storage space, and it is determined that the metadata of the target storage space is lost, thereby providing a reliable and accurate basis for subsequent recovery of the metadata.

[0019] In one embodiment, restoring metadata in the target storage space includes:

[0020] Enter the target storage space ID into the recovery tool;

[0021] Run the recovery tool to create a metadata recovery task for the target storage space;

[0022] By executing the metadata recovery task, the metadata in the target storage space is restored.

[0023] In the technical solution provided in the embodiment of the present application, the identifier of the target storage space is input into the recovery tool to instruct the recovery tool to create a metadata recovery task for the target storage space, which is equivalent to re-checking the metadata loss status of the target storage space to ensure the validity of subsequent metadata recovery tasks. Then, after the metadata recovery is detected, the metadata recovery task is executed to restore the metadata in the target storage space in a timely manner, thereby improving the metadata recovery efficiency.

[0024] In one embodiment, running a recovery tool to create a metadata recovery task for a target storage space includes:

[0025] According to the identifier of the target storage space, determine the physical node corresponding to the target storage space;

[0026] Check whether additional metadata of the target storage space exists on the physical node;

[0027] If there is additional metadata, query whether there is a metadata recovery task on the physical node;

[0028] If no metadata recovery task exists, create a metadata recovery task on the physical node.

[0029] In the technical solution provided in the embodiment of the present application, the physical node corresponding to the target storage space is determined according to the identifier of the target storage space, and the authenticity of the metadata loss of the target storage space is judged by querying whether there is additional metadata of the target storage space on the physical node, and the situation where the data in the target storage space has been migrated is excluded, thereby ensuring the accuracy of the metadata recovery task. After determining the authenticity of the metadata loss of the target storage space, the situation where the physical node repeatedly creates the metadata recovery task of the target storage space is excluded by querying whether there is a metadata recovery task on the physical node, thereby ensuring the validity of the metadata recovery task created on the physical node.

[0030] In one embodiment, the metadata in the target storage space is restored by executing a metadata restoration task, including:

[0031] Obtain object metadata information of the storage bucket to which the target storage space belongs; the object metadata information is generated during the process of storing data in different storage spaces in the storage bucket in segments;

[0032] Based on the object metadata information, the metadata in the target storage space is restored.

[0033] In the technical solution provided by the embodiment of the present application, the metadata in the target storage space is restored according to the object metadata information of the storage bucket to which the target storage space belongs. Since the object metadata information is generated in the process of storing the data in different storage spaces in the storage bucket, it is equivalent to using the existing metadata for the repair operation, without the need for additional backup of the data, thus saving storage resources.

[0034] In one embodiment, metadata of each storage space is stored in an object table; based on the object metadata information, restoring metadata in a target storage space includes:

[0035] According to the identifier of the target storage space, determining multiple groups of target metadata information matching the identifier from the object metadata information;

[0036] Assembling metadata information of each target to obtain a key-value pair of metadata information of each target;

[0037] Based on each key-value pair, fill the data in the object table of the target storage space;

[0038] The data in the populated object table is determined as metadata of the target storage space.

[0039] In the technical solution provided by the embodiment of the present application, according to the identification of the target storage space, multiple groups of target metadata information matching the identification are determined from the object metadata information as the basis for metadata recovery; then, each target metadata information is assembled to obtain a key-value pair of each target metadata information to ensure the format consistency of the recovered metadata and the data in the object table; then, based on each key-value pair, the data in the object table of the target storage space is filled to recover the missing metadata in the object table of the target storage space, thereby ensuring the integrity of the metadata in the object table of the target storage space.

[0040] In one of the embodiments, the object metadata information is distributedly stored on multiple logical nodes on the physical node to which the target storage space belongs; according to the identifier of the target storage space, multiple groups of target metadata matching the identifier are determined from the object metadata, including:

[0041] On the physical node to which the target storage space belongs, detect whether there is a metadata recovery task for the target storage space;

[0042] If it does not exist, the identifier of the target storage space is broadcast to each logical node on the physical node to instruct each logical node to parse the object metadata stored in itself and determine the target metadata corresponding to the identifier;

[0043] Receive target metadata that matches the identifier and is sent by each logical node.

[0044] In the technical solution provided by the embodiment of the present application, on the physical node to which the target storage space belongs, by detecting whether there is a metadata recovery task for the target storage space, the situation where the physical node repeatedly establishes the metadata recovery task is excluded, and the validity of the metadata recovery task is determined; then, when it is determined that there is no repeated establishment of the metadata recovery task, the identifier of the target storage space is broadcast to each logical node on the physical node to instruct each logical node to parse the object metadata stored in itself, and the target metadata sent by each logical node that matches the identifier is broadcast to ensure the integrity of the target metadata.

[0045] In one embodiment, the method further comprises:

[0046] Enter the target storage space ID into the recovery tool;

[0047] Run the recovery tool to create a recovery mark bit of the target storage space; the recovery mark bit is used to indicate to stop migrating data in the target storage space.

[0048] In the technical solution provided by the embodiment of the present application, the identifier of the target storage space is input into the recovery tool to create a recovery mark of the target storage space. In this way, after the controller recognizes the recovery mark of the target storage space, the data of the target storage space will not be migrated, so as to avoid the problem of waste of processing resources caused by blindly searching for data in the target storage space.

[0049] In a second aspect, the present application further provides a metadata recovery device, comprising:

[0050] A detection module, used to detect the loss of metadata in multiple storage spaces in the storage system; the multiple storage spaces are obtained by dividing the storage buckets in the storage system;

[0051] The recovery module is used to recover the metadata in the target storage space when the metadata of the target storage space is lost; the target storage space is at least one storage space in the storage system.

[0052] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in any one of the embodiments of the first aspect when executing the computer program.

[0053] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method in any one of the embodiments of the first aspect above are implemented.

[0054] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method in any one of the embodiments of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A diagram of an application environment of a metadata recovery method in an embodiment;

[0057] Figure 2 A schematic diagram of a process of recovering metadata in one embodiment;

[0058] Figure 3 A schematic diagram of a flow chart of data storage steps of an object storage system in one embodiment;

[0059] Figure 4 A flowchart of a metadata loss checking step in one embodiment;

[0060] Figure 5 A flowchart of a metadata loss checking step in another embodiment;

[0061] Figure 6 A flowchart of the steps for executing a metadata recovery task in one embodiment;

[0062] Figure 7 A schematic diagram of a process for creating a metadata recovery task in one embodiment;

[0063] Figure 8 A flowchart of steps for executing a metadata recovery task in another embodiment;

[0064] Fig. 9 It is a flowchart of the step of filling metadata of the object table in one embodiment;

[0065] Fig.10 A schematic diagram of a process of obtaining target metadata in one embodiment;

[0066] Fig.11 A schematic diagram of a process of creating a metadata recovery mark in one embodiment;

[0067] Fig.12 An interactive schematic diagram of a metadata recovery method in one embodiment;

[0068] Fig.13 is a structural block diagram of a metadata recovery device in an embodiment;

[0069] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0071] The metadata recovery method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the application environment includes a storage system 102 and a controller 104. The storage system 102 is a system composed of various storage devices, control components, and management information scheduling devices (hardware) and algorithms (software) in a computer system for storing programs and data. It can be a distributed storage system, such as an object storage system, a block storage system, or a centralized storage system. Taking the storage system 102 as an object storage system as an example, the controller 104 is responsible for managing and controlling the operation of each object storage device in the object storage system.

[0072] In a storage system, data and metadata of the data are usually stored separately in a storage medium, and the data can be processed by retrieving and managing the metadata. In this case, once the data or metadata is lost, it needs to be restored in time to ensure the consistency of the data.

[0073] Taking metadata loss as an example, in the related technology, when restoring the metadata of the storage system, at least the following two methods are included: one is to regularly back up the metadata of the entire storage system, and when it is found that the data or the metadata of the data is lost, the backup data is used for recovery. This requires additional storage devices to store the backup metadata, which has low disk utilization, resulting in waste of storage resources and reducing the recovery speed of metadata; the other is to use the storage bucket corresponding to the lost metadata as the minimum repair unit, and roll back the data in the storage bucket to a certain moment. This data recovery method has a large repair range and will introduce the repair of a large amount of non-lost data, affecting the recovery efficiency of the lost metadata.

[0074] In summary, the metadata recovery method in the prior art has the problem of large granularity of metadata recovery range, resulting in low metadata recovery efficiency. Based on this, the embodiment of the present application provides a metadata recovery method and a metadata recovery device, which detects the loss of metadata in multiple storage spaces in a storage system, and recovers the metadata in the target storage space when the metadata of the target storage space is lost, so as to restore the metadata in a targeted manner and improve the metadata recovery efficiency. Among them, the multiple storage spaces are obtained by dividing the storage buckets in the storage system, and the target storage space is at least one storage space in the storage system.

[0075] In order to concisely and clearly explain the technical solution provided by the embodiments of the present application, the controller of the storage system is used as the execution body, and the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail through specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0076] In an exemplary embodiment, Figure 2 As shown, a metadata recovery method is provided, comprising the following steps:

[0077] S201, detecting the loss of metadata in multiple storage spaces in a storage system; the multiple storage spaces are obtained by dividing storage buckets in the storage system.

[0078] The storage system is a system composed of storage devices, control components, and management information scheduling devices and algorithms used to store and manage program data in a computer system. In the embodiments of the present application, the storage system can be a distributed storage system, such as an object storage system, a block storage system, or a centralized storage system, such as a file storage system.

[0079] It should be noted that when uploading data to a storage system, metadata describing the data information will be generated, and the metadata is used to record the attribute information of the data, such as storage location, data size, storage time, etc. Moreover, in a storage system, data and metadata are usually stored separately.

[0080] Taking the object storage system as an example, the container for storing data in the object storage system is a bucket. The storage process of data and metadata is explained as follows: Figure 3 As shown, when the object storage system receives the uploaded data, it divides the data into multiple copies according to the size of the data. Figure 3For example, data is divided into three segments: data 1, data 2, and data 3. The three segments are then stored in three storage spaces. Each storage space can be a group of logical disks or a group of physical disks. Figure 3 In the storage space, a storage bucket can include N storage spaces, where N is a positive integer not less than 3. In addition, each storage space corresponds to a set of metadata and additional metadata. Metadata is used to record the description information of the data stored in the storage space, and additional metadata is used to record the basic information of the storage space. In the process of storing data in the storage space, each time data is stored in the storage space, a set of key-value pairs is generated and recorded as a metadata in the object table. Furthermore, after all the segmented data of the data is written, the object storage system will also write the storage information of the entire data as a record into the object metadata of the object storage system. A record in the object metadata includes multiple sets of metadata information, Figure 3 In the object metadata shown, the storage information about the data includes three sets of metadata information: the starting position and length of data 1 in storage space 1, the starting position and length of data 2 in storage space 2, and the starting position and length of data 3 in storage space n.

[0081] When the object storage system stores the data and metadata, the data in the object storage system can be processed by retrieving the metadata of each storage space, such as garbage collection (GC) processing, data grading processing, and other data migration processing.

[0082] When processing data in an object storage system, the loss of metadata in multiple storage spaces in the storage system is detected. If no metadata is lost, the data is processed directly based on the metadata. If lost, the storage space with the lost metadata is determined as the target storage space, and the metadata of the target storage space is restored, and then the subsequent data processing flow is executed with the restored metadata.

[0083] It should be emphasized that in the embodiment of the present application, detecting the loss of metadata in multiple storage spaces in the storage system refers to detecting whether the metadata of each storage space is complete, rather than detecting the object metadata of the object storage system. If it is complete, it means that the metadata of the storage space is not lost, and the data is directly processed based on the metadata. If it is incomplete, it means that the metadata of the storage space is lost, and the storage space with the lost metadata is determined as the target storage space, and the metadata of the target storage space is restored, and then the subsequent data processing flow is executed with the restored metadata.

[0084] S202: When metadata of a target storage space is lost, restore the metadata in the target storage space; the target storage space is at least one storage space in the storage system.

[0085] In actual scenarios, the metadata of one or more storage spaces in the storage system may be lost. In other words, the number of target storage spaces may be one or more.

[0086] If metadata of multiple target storage spaces is lost, you can restore the metadata of the target storage spaces one by one or in batches.

[0087] Exemplarily, the identifier of each target storage space may be sent to the metadata recovery tool to instruct the metadata recovery tool to recover the metadata of the target storage space based on the object metadata of the object storage system without additional backup.

[0088] For example, when storage resources are sufficient, the metadata of the target storage space may be backed up, so that when it is determined that the metadata of the target storage space is lost, the backup data of the target storage space may be used to quickly and specifically restore the metadata of the target storage space.

[0089] In an embodiment of the present application, the loss of metadata in multiple storage spaces in a storage system is detected, and when the metadata of a target storage space is lost, the metadata in the target storage space is restored. The multiple storage spaces are obtained by dividing the storage buckets in the storage system, and the target storage space is at least one storage space in the storage system. This is equivalent to further subdividing the storage unit of the metadata from the storage bucket into the storage space. In this way, when restoring the metadata, a single target storage space is used as the minimum recovery unit for restoration, so as to avoid introducing the recovery of metadata of non-target storage spaces in the storage bucket to which the target storage space belongs, thereby achieving accurate recovery of the metadata of the target storage space and improving the efficiency of metadata recovery.

[0090] In actual application scenarios, there are various factors that lead to metadata loss of storage space, such as system failure, hardware failure, abnormal power failure, etc. In this case, metadata loss can be detected regularly or in real time, and metadata can be restored regularly or in a timely manner. The following describes a detection scenario for metadata loss.

[0091] In an exemplary embodiment, Figure 4 As shown, detecting the loss of metadata in multiple storage spaces in a storage system includes the following steps:

[0092] S401, obtaining the fragmentation rate of each storage space.

[0093] The fragmentation rate refers to the proportion of invalid data in the storage space to the entire storage space, which to a certain extent reflects the degree to which the storage space is effectively utilized. A low fragmentation rate means that there is less invalid data in the storage space and the storage space is fully utilized; a high fragmentation rate means that there is more invalid data in the storage space and the storage space is not effectively utilized.

[0094] Optionally, the fragmentation rate of each storage space in the storage system may be obtained in response to a garbage collection request of the storage system, or the fragmentation rate of each storage space in the storage system may be scanned regularly at preset time intervals.

[0095] S402: Determine a storage space whose fragmentation rate is higher than a preset fragmentation rate threshold as a candidate storage space.

[0096] The preset fragmentation rate threshold refers to the upper limit of the fragmentation rate of the storage space that needs to be cleaned up.

[0097] If the fragmentation rate is higher than the preset fragmentation rate threshold, it means that there is a lot of invalid data in the storage space and the storage space is not effectively utilized. At this time, it is necessary to further migrate the valid data in the storage space and clean up the invalid data in the storage space so as to store more new and valid data in the future and improve the storage performance of the object storage system.

[0098] In an embodiment of the present application, among multiple storage spaces of the storage system, storage spaces with a fragmentation rate higher than a preset fragmentation rate threshold are determined according to the fragmentation rate of each storage space as candidate storage spaces, that is, storage spaces that require data cleaning.

[0099] S403, querying metadata of candidate storage spaces in the storage system to determine the loss status of metadata of each candidate storage space.

[0100] In the storage system, the metadata of each candidate storage space is queried in turn to see whether it is complete. For any candidate storage space, if the metadata of the candidate storage space is complete, it is determined that the metadata of the candidate storage space is not lost; conversely, if the metadata of the candidate storage space is incomplete or does not exist at all, it is determined that the metadata of the candidate storage space is partially lost or completely lost.

[0101] Optionally, the metadata of the storage space completely corresponds to a metadata identifier. If the metadata identifier of the candidate storage space is queried, it is determined that the metadata of the candidate storage space is not lost; if the metadata identifier of the candidate storage space is not queried, it is determined that the metadata of the candidate storage space is lost.

[0102] In an embodiment of the present application, the fragmentation rate of each storage space is compared with a preset fragmentation rate threshold, and a candidate storage space with a fragmentation rate higher than the preset fragmentation rate threshold is determined, which is equivalent to a preliminary screening of the storage space to be cleaned up in the storage system, that is, the storage space for which metadata needs to be retrieved in an embodiment of the present application, thereby reducing the detection scope and improving the detection speed of metadata loss.

[0103] Please continue to see Figure 3 In the object storage system, an object table is created for each storage space, and the metadata of each storage space is stored in the object table corresponding to the storage space. Figure 5 As shown, querying the metadata of the candidate storage space in the storage system to determine the loss of the metadata of each candidate storage space includes the following steps:

[0104] S501, querying an object table of candidate storage spaces in a storage system.

[0105] The object table corresponds to the storage space one by one and is used to record the metadata of the data stored in the storage space. Specifically, each time a piece of data is stored in the storage space, a new metadata record is added to the object table.

[0106] In the embodiment of the present application, after the candidate storage space is determined, an object table of the candidate storage space is queried in the storage system according to the identifier of the candidate storage space.

[0107] S502: When the query result meets a preset condition, the candidate storage space is determined as the target storage space, and metadata of the target storage space is determined to be lost.

[0108] The preset condition includes that the object table of the candidate storage space does not exist, or the content of the object table of the candidate storage space is incomplete.

[0109] For any candidate storage space, if the object table of the candidate storage space is not found in the storage system, the candidate storage space is determined as the target storage space, and the metadata of the target storage space is determined to be lost.

[0110] For any candidate storage space, if the object table of the candidate storage space is queried in the storage system, but the object table does not contain metadata records covering all data in the candidate storage space, the candidate storage space is determined as the target storage space, and the metadata of the target storage space is determined to be lost.

[0111] In an embodiment of the present application, for any candidate storage space, an object table of the candidate storage space is queried in the storage system. If the storage system does not include the object table of the candidate storage space, or the content of the object table of the candidate storage space in the storage system is incomplete, the candidate storage space is determined as the target storage space, and it is determined that the metadata of the target storage space is lost, thereby providing a reliable and accurate basis for subsequent recovery of the metadata.

[0112] After determining that the target storage space metadata is lost, it is necessary to promptly restore the target storage space metadata in order to process the data in the target storage space. Figure 6 As shown, restoring the metadata in the target storage space includes the following steps:

[0113] S601: Input the identifier of the target storage space into a recovery tool.

[0114] The recovery tool can be a third-party application or a custom program script.

[0115] After the target storage space is determined, the identifier of the target storage space is input into the recovery tool to instruct the recovery tool to recover the metadata of the target storage space.

[0116] It should be noted that, when there are multiple target storage spaces, the identifiers of the target storage spaces may be sent to the recovery tool to instruct the recovery tool to process the metadata recovery steps of the target storage spaces in parallel.

[0117] S602: Run a recovery tool to create a metadata recovery task for the target storage space.

[0118] In an embodiment of the present application, the trigger condition of the recovery tool is to receive the identifier of the target storage space, and the operating logic is: based on the identifier of the target storage space, the authenticity of the metadata loss of the target storage system is verified, and a metadata recovery task for the target storage space is created if the authenticity verification passes.

[0119] Exemplarily, in the case where the storage system is an object storage system, the logical node corresponding to the target storage space is determined according to the identifier of the target storage space, and then the physical node of the target space is determined, and then the authenticity of the metadata loss request of the target storage space is verified on the physical node, and if the authenticity verification passes, a metadata recovery task for the target storage space is created on the physical node.

[0120] In another scenario, if a metadata recovery task has been established before the current moment and is in the metadata recovery process, the authenticity check of the metadata loss of the target storage system will fail, and a task establishment failure indication message will be returned.

[0121] S603: Recover the metadata in the target storage space by executing a metadata recovery task.

[0122] On the physical node corresponding to the target storage space, check whether the metadata recovery task of the target storage space exists. If it is detected that the recovery tool has successfully created the metadata recovery task, it means that the metadata loss of the target storage space is true, and then execute the metadata recovery task of the target storage space.

[0123] If no metadata recovery task is detected, it means that the metadata of the target storage space is not lost or has been recovered, and there is no need to recover the metadata in the target storage space again.

[0124] In an embodiment of the present application, the identifier of the target storage space is input into the recovery tool to instruct the recovery tool to create a metadata recovery task for the target storage space, which is equivalent to re-checking the metadata loss status of the target storage space to ensure the validity of subsequent metadata recovery tasks. Then, after the metadata recovery is detected, the metadata recovery task is executed to restore the metadata in the target storage space in a timely manner, thereby improving the metadata recovery efficiency.

[0125] As can be seen from the above embodiments, the recovery tool is a metadata recovery task created after verifying the authenticity of the metadata loss of the target storage space. The task creation process of the recovery tool is described below through an embodiment in the scenario where the storage system is a distributed storage system, such as an object storage system.

[0126] In an exemplary embodiment, Figure 7 As shown in the figure, run the recovery tool to create a metadata recovery task for the target storage space, including the following steps:

[0127] S701, determining a physical node corresponding to the target storage space according to an identifier of the target storage space.

[0128] In a distributed storage system, the target storage space serves the logical nodes of the storage system, and the logical nodes can be migrated on multiple physical nodes. For example, if the target storage space serves logical node A, after the physical node 1 corresponding to logical node A fails, logical node A can be migrated to physical node 2 to run. At this time, the physical node corresponding to the target storage space is also migrated from physical node 1 to physical node 2.

[0129] Based on this, the controller determines the logical node corresponding to the target storage space at the current moment according to the identifier of the target storage space, and further determines the physical node corresponding to the logical node at the current moment.

[0130] S702, query whether there is additional metadata of the target storage space on the physical node.

[0131] The additional metadata corresponds to the storage space one by one. In the embodiment of the present application, the additional metadata refers to the basic information of the metadata of the target storage space, which migrates with the migration of the data in the target storage space. In other words, if the additional metadata of the storage space is stored on the physical node, it means that the data in the target storage space at the current moment has not been processed; if the additional metadata of the storage space does not exist on the physical node, it means that the data in the target storage space at the current moment has been processed. At this time, the loss of metadata of the target storage space is normal, and there is no need to recover the metadata.

[0132] S703: If there is additional metadata, query whether there is a metadata recovery task on the physical node.

[0133] The existence of additional metadata of the target storage space on the physical node means that the metadata loss of the target storage space determined previously by the controller is true. At this time, in order to prevent repeated establishment of metadata recovery tasks for the same target storage space on the physical node, a further query is made as to whether a metadata recovery task already exists on the physical node.

[0134] If a metadata recovery task exists, a task reconstruction instruction can be returned to inform the current physical node that a metadata recovery task for the target storage space has been established. Alternatively, the metadata recovery task being executed can be deleted and a new metadata recovery task can be created.

[0135] S704: If no metadata recovery task exists, create a metadata recovery task on the physical node.

[0136] If no metadata recovery task exists, a metadata recovery task is created on the physical node so that the controller in the storage system can execute the metadata recovery task in time and complete the recovery of the metadata of the target storage space.

[0137] In an embodiment of the present application, the physical node corresponding to the target storage space is determined based on the identifier of the target storage space, and the authenticity of the metadata loss of the target storage space is judged by querying whether there is additional metadata of the target storage space on the physical node, and the situation where the data in the target storage space has been migrated is excluded, thereby ensuring the accuracy of the metadata recovery task. After determining the authenticity of the metadata loss of the target storage space, the situation where the physical node repeatedly creates the metadata recovery task of the target storage space is excluded by querying whether there is a metadata recovery task on the physical node, thereby ensuring the validity of the metadata recovery task created on the physical node.

[0138] After the recovery tool successfully creates the metadata recovery task and the controller recognizes the metadata recovery task, the controller needs to further execute the metadata recovery task. Figure 8 As shown, by executing the metadata recovery task, the metadata in the target storage space is recovered, including the following steps:

[0139] S801, obtaining object metadata information of the storage bucket to which the target storage space belongs; the object metadata information is generated in the process of storing data in different storage spaces in the storage bucket in segments.

[0140] Taking the storage system as an object storage system as an example, the object metadata information of a bucket refers to a collection of metadata information of all data in the object storage system, and in the object metadata information of the bucket, complete data is recorded as a set of storage information, a set of storage information includes multiple sets of object metadata, and the number of object metadata in a set of storage information matches the number of divisions of a complete data.

[0141] Please continue to see Figure 3 The data storage process shown, Figure 3 The object metadata shown records the storage information of a complete set of data. Figure 3 The data is divided into three sections, accordingly, Figure 3 The set of storage information shown includes three sets of metadata information, namely: the starting position and length of data 1 in storage space 1, the starting position and length of data 2 in storage space 2, and the starting position and length of data 3 in storage space n.

[0142] In an embodiment of the present application, the object metadata information of the bucket and the metadata of each storage space corresponding to the bucket are stored in different storage media, and the security level of the storage medium storing the object metadata information is higher than the security level of the storage medium storing the metadata.

[0143] In actual applications, even if the metadata of the target storage space is lost, the object metadata is safe and complete. Based on this, when it is determined that the metadata of the target storage space is lost, the metadata matching the target storage space can be determined based on the content in the object metadata.

[0144] S802: Restore metadata in the target storage space based on the object metadata information.

[0145] Optionally, based on the identifier of the target storage space, metadata that matches the target storage space is determined from the object metadata, and then the format of the metadata that matches the target storage space is converted according to the format of the metadata of the target storage space, and the metadata of the target storage space is replaced with the metadata after the format conversion to restore the metadata of the target storage space.

[0146] In the embodiment of the present application, the metadata in the target storage space is restored according to the object metadata information of the storage bucket to which the target storage space belongs. Since the object metadata information is generated in the process of storing the data in different storage spaces in the storage bucket, it is equivalent to using the existing metadata for the repair operation, without the need for additional backup of the data, thus saving storage resources.

[0147] In an exemplary embodiment, metadata for each storage space is stored in an object table; Fig. 9 As shown, the aforementioned step S802 "recovering the metadata in the target storage space based on the object metadata information" includes the following steps:

[0148] S901, according to the identifier of the target storage space, determine from the object metadata information a plurality of groups of target metadata information matching the identifier.

[0149] In object metadata, a complete set of data uploaded by the user is used as a record unit, and the metadata of a single data is represented in text form. Since a piece of data is stored in different storage units, a data record can correspond to multiple sets of metadata information, and each set of metadata information corresponds to a storage space.

[0150] Furthermore, in actual applications, the object metadata information includes multiple groups of data records in the object storage system, and the metadata information of each group of data records includes: each segmented data corresponding to the data, and an identifier of a storage space storing each segmented data.

[0151] For any target storage space whose metadata is lost, multiple groups of target metadata information of the target storage space are filtered out from the object metadata according to the identifier of the target storage space.

[0152] For example, an object storage system stores two pieces of data, Data 1 and Data 2, uploaded by a user, in the same bucket, and each piece of data is stored in three segments. The object metadata of the bucket includes two records: (1) the storage information record of Data 1; and (2) the storage information record of Data 2.

[0153] Among them, the storage information record of data 1 includes three sets of metadata information: ① segmented data 11 of data 1 is stored in storage space 1; ② segmented data 12 of data 1 is stored in storage space 2; ③ segmented data 13 of data 1 is stored in storage space n.

[0154] The storage information record of data 2 includes three sets of metadata information: ① segmented data 21 of data 2 is stored in storage space 1; ② segmented data 22 of data 2 is stored in storage space 2; ③ segmented data 23 of data 2 is stored in storage space n.

[0155] If the target storage space is storage space 1, two sets of target metadata are found from the object metadata according to the identifier of storage space 1: segmented data 11 of data 1 is stored in storage space 1, and segmented data 21 of data 2 is stored in storage space 1.

[0156] S902, assembling each target metadata information to obtain a key-value pair of each target metadata information.

[0157] Each storage space corresponds to an object table for storing metadata of each segmented data in the storage space. In addition, in the object table of the storage space, the metadata of each segment of data is represented in the form of a key-value pair.

[0158] After determining the multiple groups of target metadata information corresponding to the target storage space, by executing a preset key-value pair recovery script, the formats of the target metadata are converted respectively to obtain the key-value pairs corresponding to the target metadata information.

[0159] S903: Fill the data in the object table of the target storage space based on each key-value pair.

[0160] When the object table of the target storage space is empty, all the assembled key-value pairs of each target metadata information are filled into the object table; when the object table of the target storage space is not empty, the assembled key-value pairs are compared with the key-value pairs in the object table of the target storage space, the target key-value pairs that are not in the object table are determined, and the target key-value pairs are filled into the object table.

[0161] It should be noted that steps S902 and S903 in the embodiment of the present application can be integrated into the same key-value pair recovery script, and the assembly of the target metadata and the data supplement of the object table of the target storage space can be automatically completed by executing the key-value pair recovery script.

[0162] S904: Determine the data in the filled object table as metadata of the target storage space.

[0163] In an embodiment of the present application, based on the identifier of the target storage space, multiple groups of target metadata information matching the identifier are determined from the object metadata information as a basis for metadata recovery; then, each target metadata information is assembled to obtain a key-value pair of each target metadata information to ensure format consistency between the recovered metadata and the data in the object table; then, based on each key-value pair, the data in the object table of the target storage space is filled to recover the missing metadata in the object table of the target storage space, thereby ensuring the integrity of the metadata in the object table of the target storage space.

[0164] Next, in a scenario where the storage system is a distributed storage system, an implementation method for obtaining target metadata from object metadata is further described.

[0165] In an exemplary embodiment, Fig.10 As shown, the object metadata information is distributedly stored on multiple logical nodes on the physical node to which the target storage space belongs; according to the identifier of the target storage space, multiple groups of target metadata matching the identifier are determined from the object metadata, including the following steps:

[0166] S1001, on the physical node to which the target storage space belongs, detecting whether there is a metadata recovery task for the target storage space.

[0167] The distributed storage system includes multiple physical nodes. Different physical nodes can be accessed by different users at the same time, and the controller can be instructed to run multiple recovery tools to create metadata recovery tasks at the same time. To avoid the above situation where the same metadata recovery task is repeatedly established on the same physical node, the controller needs to further detect whether there is a metadata recovery task for the target storage space on the physical node. If not, the controller continues to execute the subsequent steps. If so, the controller stops executing the subsequent steps.

[0168] S1002: If it does not exist, broadcast the identifier of the target storage space to each logical node on the physical node to instruct each logical node to parse the object metadata stored in itself and determine the target metadata corresponding to the identifier.

[0169] In the case that the physical node does not have a metadata recovery task for the target storage space, the identifier of the target storage space is broadcast to each logical node on the physical node. Each logical node creates a repair event based on the identifier of the target storage space to determine the target metadata that matches the identifier from the object metadata stored in itself, and sends the determined target metadata to the controller.

[0170] S1003: Receive target metadata that matches the identifier and is sent by each logical node.

[0171] After receiving the target metadata matching the identifier sent by each logical node on the physical node, the received target metadata can be assembled to obtain the key-value pair of the target storage space, and the assembled key-value pair is used as the metadata of the target storage space.

[0172] Optionally, the controller automatically completes the steps of assembling the target metadata and supplementing the data of the object table of the target storage space by executing the key-value pair recovery script. The specific implementation method can be found in the description of S902 to S904 in the above embodiment, which will not be repeated here.

[0173] In an embodiment of the present application, on the physical node to which the target storage space belongs, by detecting whether there is a metadata recovery task for the target storage space, the situation in which the physical node repeatedly establishes the metadata recovery task is excluded, and the validity of the metadata recovery task is determined; then, when it is determined that there is no repeated establishment of the metadata recovery task, the identifier of the target storage space is broadcast to each logical node on the physical node to instruct each logical node to parse the object metadata stored in itself, and the target metadata sent by each logical node that matches the identifier is broadcast to ensure the integrity of the target metadata.

[0174] In actual applications, if the metadata of the target storage space is lost, the data of the target storage space cannot be found, which naturally affects the processing of the data of the target storage space. Fig.11 As shown, the method further comprises the following steps:

[0175] S1101: Input the identifier of the target storage space into a recovery tool.

[0176] The recovery tool may be a third-party application or a customized program script.

[0177] After determining the target storage space, the identifier of the target storage space is input into the recovery tool to instruct the recovery tool to verify the authenticity of the metadata loss of the target storage space, and if the verification passes, establish a recovery mark bit of the target storage space.

[0178] It should be noted that, when there are multiple target storage spaces, the identifiers of the target storage spaces may be sent to the recovery tool in a unified manner to instruct the recovery tool to set the recovery flags of the target storage spaces in parallel.

[0179] S1102, running a recovery tool to create a recovery mark of the target storage space; the recovery mark is used to indicate to stop migrating data in the target storage space.

[0180] The recovery tool is run to verify the authenticity of the metadata loss of the target storage space. For the specific verification process, see steps S701 to S704 in the above embodiment. If the authenticity verification passes, a recovery flag of the target storage space is established to instruct the controller to stop migrating the data in the target storage space.

[0181] In an optional embodiment, in addition to creating a recovery mark bit of the target storage space, the metadata loss of the target storage space may also be communicated through page alarms, message notifications, and the like.

[0182] In the embodiment of the present application, the identifier of the target storage space is input into the recovery tool to create a recovery mark of the target storage space. In this way, after the controller recognizes the recovery mark of the target storage space, the data of the target storage space will not be migrated, so as to avoid the problem of waste of processing resources caused by blindly searching for data in the target storage space.

[0183] In an exemplary embodiment, taking the storage system as an object storage system, and the metadata of the storage space being stored in an object table as an example, the metadata recovery method is equivalent to the object table recovery method. The metadata recovery method provided in the embodiment of the present application includes:

[0184] (1) In response to a garbage collection instruction of the object storage system, check whether an object table loss occurs in each storage space.

[0185] (2) If lost, the target storage space identifier of the lost object table will be exposed in the form of log printing.

[0186] (3) Next, the target storage space identifier is input into the repair tool (objscan) of the object storage system, and the repair tool is instructed to use the target storage space identifier as input and mark the target storage space to be repaired with a repair mark. At the same time, the repair tool is instructed to create a metadata recovery task, that is, an object table recovery task.

[0187] (4) When the controller performs garbage collection, if it recognizes a repair mark, it will not process the data in the target storage space.

[0188] (5) After scanning the object table recovery task, the controller scans all object metadata information of the object storage system, obtains the target object metadata information stored in the target storage space, and combines the target object metadata information to rebuild the object table.

[0189] Next, in the scenario where it is determined that the object table of the target storage space is lost, the steps for repairing the metadata of the object table of the target storage space are further described. Fig.12 As shown, Fig.12The diagram is a schematic diagram of the interaction between the controller and the repair tool in the object storage system, including the following steps:

[0190] S1201: The object storage system sends a target storage space identifier to a repair tool.

[0191] The repair tool is a shell script that can input multiple target storage space identifiers that need to be repaired at one time.

[0192] S1202: The repair tool determines the logical node to which the target storage space belongs according to the target storage space identifier.

[0193] S1203: The repair tool sends a first query command to the object storage system, instructing to query whether additional metadata corresponding to the target storage space exists.

[0194] The object storage system queries whether the additional metadata corresponding to the target storage space exists on the physical node corresponding to the logical node to which the target storage space belongs, and obtains a first query result.

[0195] S1204: The object storage system returns the first query result to the repair tool.

[0196] S1205: If the first query result is that no additional metadata exists, the repair tool terminates execution and displays a first error message.

[0197] The first error message is used to prompt that the target storage space does not have additional metadata.

[0198] S1206: If the first query result indicates that additional metadata exists, the repair tool sends a second query command to the object storage system, instructing to query whether a repair task for the target storage space exists.

[0199] The object storage system queries whether there is a repair task for the target storage space on the physical node corresponding to the logical node to which the target storage space belongs.

[0200] S1207: The object storage system returns the second query result to the repair tool.

[0201] S1208: If the second query result indicates that a repair task exists, display a second error message.

[0202] The second error message is used to prompt that the target storage space is repairing the target storage space.

[0203] S1209: If the second query result indicates that there is no repair task, the repair tool temporarily shuts down the garbage collection and classification tasks of the object storage system.

[0204] S1210: Set a recovery mark bit for the target storage space.

[0205] S1211, start garbage collection and classification tasks of the object storage system.

[0206] S1212: The repair tool sends an object table recovery task creation instruction to the object storage system.

[0207] The object table recovery task creation instruction is used to instruct the object storage system to create an object table recovery task on a physical node corresponding to a logical node to which the target storage space belongs.

[0208] S1213: The object storage system queries on the physical node whether an object table recovery task has been created according to the target storage space identifier.

[0209] S1214: If the object table recovery task already exists, the object storage system sends a message to the repair tool indicating that the object table recovery task already exists.

[0210] S1215: The repair tool displays the third error message.

[0211] The third error message is used to prompt the physical node that an object table recovery task exists.

[0212] S1216: If the object table recovery task does not exist, the object storage system creates an object table recovery task.

[0213] S1217: The object storage system broadcasts the target storage space identifier to all logical nodes on the physical node, instructing each logical node to create its own repair event.

[0214] S1218, each logical node on the physical node in the object storage system scans the object metadata stored in itself according to the target storage space identifier.

[0215] S1219, assembling the object metadata parsed from each logical node into key-value pairs.

[0216] S1220: For any assembled key-value pair, use the key in the key-value pair to query in the object table of the target storage space.

[0217] S1221: If it exists, skip it and do not process it.

[0218] S1222: If it does not exist, add the assembled key-value pair to the object table of the target storage space.

[0219] S1223: When the scanning of a single logical node is completed, the repair event of the single logical node is set to a completed state.

[0220] S1224, the repair tool queries the repair progress of the target storage space.

[0221] S1225: The object storage system obtains the repair progress status information of each logical node.

[0222] S1226: The object storage system returns the repair progress status information of each logical node as a repair result to the repair tool.

[0223] If the repair progress status information of all logical nodes has been repaired, it is considered that all metadata of the object table of the target storage space have been repaired.

[0224] In the embodiment of the present application, the object table repair operation of the target storage space is performed using the existing object table metadata, and there is no need to perform additional metadata backup, thus saving storage resources. Moreover, in the process of repairing the metadata in the storage space object table, the storage space is used as the minimum repair granularity, and the object table that has not been lost is not affected or modified. In this way, the controller only needs to identify the repair mark of the storage space and not process it, without locking the entire storage bucket or storage device, and does not affect the foreground business of other storage spaces.

[0225] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0226] Based on the same inventive concept, the embodiment of the present application also provides a storage system for implementing the metadata recovery method involved above, which can be an object storage system. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, and the specifics can be found in the above definition of the metadata recovery method, which will not be repeated here.

[0227] Based on the same inventive concept, the embodiment of the present application also provides a metadata recovery device for implementing the metadata recovery method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more metadata recovery device embodiments provided below can refer to the limitations of the metadata recovery method above, and will not be repeated here.

[0228] In an exemplary embodiment, Fig.13 As shown, a metadata recovery device is provided, including: a detection module 1301 and a recovery module 1302, wherein:

[0229] The detection module 1301 is used to detect the loss of metadata in multiple storage spaces in the storage system; the multiple storage spaces are obtained by dividing the storage buckets in the storage system;

[0230] The recovery module 1302 is used to recover the metadata in the target storage space when the metadata of the target storage space is lost; the target storage space is at least one storage space in the storage system.

[0231] In an exemplary embodiment, the detection module 1301 includes: a fragmentation rate acquisition unit, a candidate space determination unit and a data query unit, wherein:

[0232] A fragmentation rate acquisition unit, used to acquire the fragmentation rate of each storage space;

[0233] A candidate space determination unit, configured to determine a storage space having a fragmentation rate higher than a preset fragmentation rate threshold as a candidate storage space;

[0234] The data query unit is used to query the metadata of the candidate storage space in the storage system and determine the loss status of the metadata of each candidate storage space.

[0235] In an exemplary embodiment, metadata of each storage space is stored in an object table; the data query unit includes: an object table query subunit and a target space determination subunit, wherein:

[0236] An object table query subunit, used to query an object table of a candidate storage space in a storage system;

[0237] The target space determination subunit is used to determine the candidate storage space as the target storage space and determine that the metadata of the target storage space is lost when the query result meets the preset conditions; wherein the preset conditions include that the object table of the candidate storage space does not exist, or the content of the object table of the candidate storage space is incomplete.

[0238] In an exemplary embodiment, the recovery module 1302 includes: an identification input unit, a task creation unit and a task execution unit, wherein:

[0239] An identification input unit, used to input the identification of the target storage space into the recovery tool;

[0240] A task creation unit, used to run a recovery tool to create a metadata recovery task for a target storage space;

[0241] The task execution unit is used to restore the metadata in the target storage space by executing the metadata recovery task.

[0242] In an exemplary embodiment, the task creation unit includes: a physical node determination subunit, an additional metadata query subunit, a task query subunit and a task creation subunit, wherein:

[0243] A physical node determination subunit, used to determine the physical node corresponding to the target storage space according to the identifier of the target storage space;

[0244] The additional metadata query subunit is used to query whether additional metadata of the target storage space exists on the physical node;

[0245] A task query subunit is used to query whether a metadata recovery task exists on a physical node if additional metadata exists;

[0246] The task creation subunit is used to create a metadata recovery task on a physical node when no metadata recovery task exists.

[0247] In an exemplary embodiment, the task execution unit includes: an object metadata acquisition subunit and a metadata recovery subunit, wherein:

[0248] The object metadata acquisition subunit is used to acquire the object metadata information of the storage bucket to which the target storage space belongs; the object metadata information is generated in the process of storing data in different storage spaces in the storage bucket in segments;

[0249] The metadata recovery subunit is used to recover the metadata in the target storage space based on the object metadata information.

[0250] In an exemplary embodiment, the metadata of each storage space is stored in an object table; the metadata recovery subunit is further used to determine, based on the identifier of the target storage space, multiple groups of target metadata information that match the identifier from the object metadata information; assemble each target metadata information to obtain a key-value pair of each target metadata information; based on each key-value pair, fill in the data in the object table of the target storage space; and determine the data in the filled object table as the metadata of the target storage space.

[0251] In an exemplary embodiment, object metadata information is distributedly stored on multiple logical nodes on the physical node to which the target storage space belongs; the metadata recovery subunit is also used to detect whether there is a metadata recovery task for the target storage space on the physical node to which the target storage space belongs; if not, the identifier of the target storage space is broadcast to each logical node on the physical node to instruct each logical node to parse the object metadata stored in itself and determine the target metadata corresponding to the identifier; and receive the target metadata matching the identifier sent by each logical node.

[0252] In an exemplary embodiment, the metadata recovery device also includes a recovery mark bit creation module, which is used to input the identifier of the target storage space into the recovery tool and run the recovery tool to create a recovery mark bit of the target storage space; the recovery mark bit is used to indicate to stop migrating data to the target storage space.

[0253] Each module in the metadata recovery device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0254] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.14 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store metadata recovery data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a metadata recovery method is implemented.

[0255] Those skilled in the art will understand that Fig.14The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0256] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0257] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0258] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0259] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0260] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0261] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0262] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A metadata recovery method, characterized in that: The method comprises: Detecting metadata loss in a plurality of storage spaces in a storage system; the plurality of storage spaces are obtained by dividing storage buckets in the storage system; In the case where metadata of a target storage space is lost, the metadata in the target storage space is restored; the target storage space is at least one storage space in the storage system.

2. The method according to claim 1, characterized in that The detecting loss of metadata in a plurality of storage spaces in the storage system includes: Obtaining a fragmentation rate of each storage space; Determine a storage space whose fragmentation rate is higher than a preset fragmentation rate threshold as a candidate storage space; The metadata of the candidate storage spaces is queried in the storage system to determine the loss status of the metadata of each candidate storage space.

3. The method according to claim 2, characterized in that The metadata of each storage space is stored in the object table; The querying the metadata of the candidate storage space in the storage system to determine the loss of the metadata of each candidate storage space includes: Querying the object table of the candidate storage space in the storage system; In the case that the query result meets the preset condition, determining the candidate storage space as the target storage space, and determining that the metadata of the target storage space is lost; The preset condition includes that the object table of the candidate storage space does not exist, or the content of the object table of the candidate storage space is incomplete.

4. The method according to any one of claims 1 to 3, characterized in that: The restoring of the metadata in the target storage space includes: inputting the identifier of the target storage space into a recovery tool; Running the recovery tool to create a metadata recovery task for the target storage space; The metadata in the target storage space is restored by executing the metadata restoration task.

5. The method according to claim 4, characterized in that The step of running the recovery tool to create a metadata recovery task for the target storage space includes: Determining a physical node corresponding to the target storage space according to the identifier of the target storage space; Querying whether additional metadata of the target storage space exists on the physical node; If the additional metadata exists, query whether the metadata recovery task exists on the physical node; If the metadata recovery task does not exist, create the metadata recovery task on the physical node.

6. The method according to claim 4, characterized in that The step of restoring the metadata in the target storage space by executing the metadata restoration task includes: Obtaining object metadata information of the storage bucket to which the target storage space belongs; the object metadata information is generated in the process of storing data in different storage spaces in the storage bucket in segments; Based on the object metadata information, the metadata in the target storage space is restored.

7. The method according to claim 6, characterized in that The metadata of each storage space is stored in the object table; The restoring the metadata in the target storage space based on the object metadata information includes: According to the identifier of the target storage space, determining from the object metadata information a plurality of groups of target metadata information matching the identifier; Assembling each of the target metadata information to obtain a key-value pair of each of the target metadata information; Based on each of the key-value pairs, filling the data in the object table of the target storage space; The data in the filled object table is determined as metadata of the target storage space.

8. The method according to claim 7, characterized in that The object metadata information is distributedly stored on multiple logical nodes on the physical node to which the target storage space belongs; The step of determining, according to the identifier of the target storage space, a plurality of groups of target metadata matching the identifier from the object metadata comprises: On the physical node to which the target storage space belongs, detecting whether there is a metadata recovery task for the target storage space; If it does not exist, broadcasting the identifier of the target storage space to each of the logical nodes on the physical node to instruct each logical node to parse the object metadata stored in itself and determine the target metadata corresponding to the identifier; Receive target metadata that matches the identifier and is sent by each of the logical nodes.

9. The method according to claim 1, characterized in that: The method further comprises: inputting the identifier of the target storage space into a recovery tool; The recovery tool is run to create a recovery mark of the target storage space; the recovery mark is used to indicate to stop migrating data in the target storage space.

10. A metadata recovery device, characterized in that: The device comprises: A detection module, used to detect the loss of metadata in multiple storage spaces in a storage system; the multiple storage spaces are obtained by dividing storage buckets in the storage system; The recovery module is used to recover the metadata in the target storage space when the metadata of the target storage space is lost; the target storage space is at least one storage space in the storage system.

Images