Stored data recovery method, apparatus and system, electronic device, and storage medium

Abstract

The present application relates to the technical field of servers, and provides a stored data recovery method, apparatus and system, an electronic device, and a storage medium. The method comprises: on the basis of metadata information of a storage pool to be recovered, determining a bitmap flag bit corresponding to each storage space, constructing bitmap data of said storage pool, and on the basis of the bitmap data, recovering the data in said storage pool. According to the stored data recovery method, apparatus and system, the electronic device, and the storage medium provided by the present application, the bitmap data of a storage pool to be recovered is determined by means of metadata information of said storage pool. After the bitmap data of said storage pool is determined, during subsequent recovery of data in said storage pool, it is only necessary to determine whether the bitmap of the storage space to which the data belongs is set, and when it is determined that the bitmap is set, there is no need to perform setting again, thereby reducing the number of operations of setting the bitmap, and improving the recovery efficiency of data of the storage pool.

Description

Method, device, system, electronic device and storage medium for repairing stored data

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on December 27, 2023, with application number 202311818432.9, and entitled “Method, device, system, electronic device and storage medium for repairing stored data”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of server technology, and in particular to a method, device, system, electronic device, and storage medium for repairing stored data. Background Art

[0004] In the all-flash storage industry, storage pools are divided into data and metadata areas. The data area has a very large capacity, potentially reaching hundreds of terabytes (TB) or even petabytes (PB). The data area can be divided into multiple storage spaces based on specific capacity.

[0005] The usage of storage space is generally managed based on a bitmap. In the event of a power outage in the storage system, if there is no battery to save these important and non-lost data, the bitmap information will be lost.

[0006] After bitmap information is lost, it's impossible to determine the usage of each storage space in the storage pool. Each storage space contains numerous physical addresses, and each physical address requires a bitmap to be set when repairing data. This results in numerous bitmap setup operations, which in turn increases the storage pool repair time and reduces the efficiency of data repair.

[0007] Summary of the Invention

[0008] In one aspect, the present application provides a method for repairing stored data, comprising:

[0009] Scanning the metadata storage area of ​​the storage pool to be repaired, extracting metadata information from the metadata storage area, and constructing metadata information of the storage pool to be repaired based on the extracted metadata information;

[0010] Determining, based on metadata information of the storage pool to be repaired, whether each storage space in the storage pool to be repaired contains valid data, where the storage pool to be repaired contains multiple storage spaces;

[0011] Determining a bitmap flag corresponding to each storage space based on whether each storage space in the storage pool to be repaired contains valid data, and constructing bitmap data of the storage pool to be repaired based on the bitmap flag corresponding to each storage space; and

[0012] The data in the storage pool to be repaired is repaired based on the bitmap data.

[0013] According to a method for repairing stored data provided by the present application, data in a storage pool to be repaired is repaired based on bitmap data, including:

[0014] Determine the physical address corresponding to the data in the storage pool to be repaired;

[0015] Based on the bitmap data, determining whether the storage space to which the physical address belongs is marked with a bitmap mark bit in the bitmap data;

[0016] In response to determining that the storage space to which the physical address belongs has a bitmap mark bit marked in the bitmap data, repairing the physical address;

[0017] In response to determining that the bitmap data does not mark the bitmap mark bit corresponding to the storage space to which the physical address belongs, repairing the physical address and marking the bitmap mark bit corresponding to the storage space to which the physical address belongs in the bitmap data; and

[0018] Based on the bitmap mark bit of the storage space to which the physical address belongs in the bitmap data, a mark synchronization message is generated and sent to the peer system, so that after the peer system receives the mark synchronization message, the bitmap data of the corresponding storage space in the peer system is synchronously updated. The peer system is a redundant storage system corresponding to the storage pool to be repaired.

[0019] According to a method for repairing stored data provided by the present application, based on whether each storage space in a storage pool to be repaired contains valid data, determining a bitmap flag corresponding to each storage space includes:

[0020] Traverse each storage space in the storage pool to be repaired as the current storage space;

[0021] In response to determining that the current storage space contains valid data, determining that a bitmap flag bit corresponding to the current storage space is 1; or

[0022] In response to determining that the current storage space does not contain valid data, a bitmap flag bit corresponding to the current storage space is determined to be 0.

[0023] According to a method for repairing storage data provided by the present application, after constructing the bitmap data of the storage pool to be repaired, the method further includes:

[0024] Constructing a bitmap array corresponding to the bitmap data of the storage pool to be repaired; and

[0025] The bitmap array is stored in the memory space of the system corresponding to the storage pool to be repaired.

[0026] According to a method for repairing stored data provided by the present application, a bitmap array is stored in the memory space of a system corresponding to a storage pool to be repaired, and then the method further includes:

[0027] Generate a bitmap data synchronization message based on the bitmap array; and

[0028] The bitmap data synchronization message is sent to the peer system, so that after receiving the bitmap data synchronization message, the peer system synchronously updates the bitmap data of the storage pool to be repaired in the peer system. The peer system is a redundant storage system corresponding to the storage pool to be repaired.

[0029] A method for repairing stored data provided by the present application further includes:

[0030] receiving a synchronization message sent by the peer system, where the synchronization message is generated by the peer system based on a bitmap array stored in memory; and

[0031] Based on the synchronization message, the bitmap data of the storage pool to be repaired is updated.

[0032] According to a method for repairing stored data provided by the present application, a metadata storage area of ​​a storage pool to be repaired is scanned, and metadata information is extracted from the metadata storage area, including:

[0033] Scan the metadata storage area of ​​the storage pool to be repaired, and determine the node information of the data written to the disk in the storage pool to be repaired from the metadata storage area; and

[0034] The metadata in the node information is traversed, and metadata information is extracted from the metadata in the node information.

[0035] According to a method for repairing stored data provided by the present application, each storage space in the storage pool to be repaired is obtained by dividing the storage pool to be repaired based on a preset data block size.

[0036] According to a method for repairing stored data provided by the present application, before scanning the metadata storage area of ​​the storage pool to be repaired and extracting metadata information from the metadata storage area, the method further includes:

[0037] Determining the size of the memory space for storing the bitmap data of the storage pool to be repaired based on the multiple storage spaces included in the storage pool to be repaired; and

[0038] Based on the memory space size, a target memory space is applied for in the memory of the system corresponding to the storage pool to be repaired. The target memory space is used to store the bitmap data of the storage pool to be repaired.

[0039] According to a method for repairing stored data provided by the present application, after repairing the data in the storage pool to be repaired based on bitmap data, the method further includes:

[0040] Perform metadata verification on the repaired data in the storage pool to be repaired to verify whether the repaired data is accurate.

[0041] According to a method for repairing stored data provided by the present application, after repairing the data in the storage pool to be repaired based on bitmap data, the method further includes:

[0042] The repaired data in the storage pool to be repaired is synchronized to the peer system, which is a redundant storage system corresponding to the storage pool to be repaired.

[0043] According to a method for repairing stored data provided by the present application, the steps of synchronizing the repaired data in the storage pool to be repaired to the peer system include:

[0044] The repaired data is synchronized to the peer system through a preset data synchronization method, where the preset data synchronization method includes at least one of mirror synchronization, incremental synchronization, and asynchronous replication.

[0045] According to a method for repairing stored data provided by the present application, the steps of synchronizing the repaired data to the peer system through a preset data synchronization method include:

[0046] Determine synchronization configuration parameters according to a preset data synchronization method; and

[0047] Synchronize the repaired data to the peer system according to the synchronization configuration parameters.

[0048] According to a method for repairing stored data provided by the present application, the synchronization configuration parameters include at least one of a synchronization frequency, a synchronization strategy, and a synchronization rule.

[0049] According to a method for repairing stored data provided by the present application, the step of scanning the metadata storage area of ​​the storage pool to be repaired includes:

[0050] Scan the metadata storage area using the storage system's management tool or preset scanning program.

[0051] According to a method for repairing storage data provided by the present application, after the step of storing the bitmap array in the memory space of the system corresponding to the storage pool to be repaired, and before the step of repairing the data in the storage pool to be repaired based on the bitmap data, it also includes: obtaining the bitmap data from the memory space.

[0052] On the other hand, the present application also provides a storage data repair device, comprising:

[0053] A scanning module, configured to scan the metadata storage area of ​​the storage pool to be repaired, extract metadata information from the metadata storage area, and construct metadata information of the storage pool to be repaired based on the extracted metadata information;

[0054] A data verification module is used to determine whether each storage space in the storage pool to be repaired contains valid data based on metadata information of the storage pool to be repaired, and the storage pool to be repaired includes multiple storage spaces;

[0055] A bitmap construction module is used to determine the bitmap flag corresponding to each storage space in the storage pool to be repaired based on whether each storage space in the storage pool to be repaired contains valid data, and to construct bitmap data of the storage pool to be repaired based on the bitmap flag corresponding to each storage space;

[0056] The repair module is used to repair the data in the storage pool to be repaired based on the bitmap data.

[0057] On the other hand, the present application also provides a storage data repair system, including: a storage pool to be repaired and a repair device applied to the above storage data.

[0058] On the other hand, the present application also provides an electronic device comprising one or more processors; and a memory associated with the one or more processors, the memory being used to store computer-readable instructions, which, when read and executed by the one or more processors, implement any of the above-mentioned methods for repairing stored data.

[0059] On the other hand, the present application also provides a non-transitory computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, they implement any of the above-mentioned methods for repairing stored data.

[0060] The storage data repair method, apparatus, system, electronic device, and storage medium provided herein scan the metadata storage area of ​​a storage pool to be repaired, extract metadata information, and construct metadata information for the storage pool to be repaired. Based on the metadata information of the storage pool to be repaired, the method determines whether each storage space in the storage pool to be repaired contains valid data, thereby determining the bitmap data for the storage pool to be repaired. After determining the bitmap data for the storage pool to be repaired, the data in the storage pool to be repaired is repaired. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0062] FIG1 is a schematic diagram of a flow chart of a method for repairing stored data provided in an embodiment of the present application;

[0063] FIG2 is a schematic diagram of a synchronization process according to an embodiment of the present application;

[0064] FIG3 is a schematic diagram of the structure of a method for repairing stored data provided by an embodiment of the present application;

[0065] FIG4 is a schematic diagram of the structure of a storage data repair device provided in an embodiment of the present application;

[0066] FIG5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0067] FIG6 is a schematic diagram of the structure of a non-transitory computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0069] The present application proposes a method for repairing stored data. FIG1 is a flow chart of the method for repairing stored data provided in an embodiment of the present application. Referring to FIG1 , the method for repairing stored data provided in an embodiment of the present application includes the following steps:

[0070] Step 110: Scan the metadata storage area of ​​the storage pool to be repaired, extract metadata information from the metadata storage area, and construct metadata information of the storage pool to be repaired based on the extracted metadata information;

[0071] Step 120: Determine whether each storage space in the storage pool to be repaired contains valid data based on metadata information of the storage pool to be repaired, where the storage pool to be repaired contains multiple storage spaces.

[0072] Step 130: determining a bitmap flag corresponding to each storage space in the storage pool to be repaired based on whether each storage space contains valid data, and constructing bitmap data of the storage pool to be repaired based on the bitmap flag corresponding to each storage space;

[0073] Step 140: Repair the data in the storage pool to be repaired based on the bitmap data.

[0074] The execution subject of the storage data repair method provided in the present application can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), or a personal computer (PC), etc., and this application does not make specific limitations.

[0075] The following describes the technical solution of the present application in detail by taking an example of a computer executing the method for repairing stored data provided in the present application.

[0076] In step 110, the metadata storage area of ​​the storage pool to be repaired is scanned, metadata information is extracted from the metadata storage area, and metadata information of the storage pool to be repaired is constructed based on the extracted metadata information.

[0077] In a storage system, metadata describes the stored data itself. Metadata includes file name, size, creation time, permissions, and more. The metadata storage area is a dedicated area for storing metadata information, typically a separate storage medium, for quick file retrieval and management.

[0078] A storage pool in a storage system is a collection of storage resources consisting of multiple storage devices used to store user data. If a device in the storage pool is damaged or the storage system corresponding to the storage pool loses power or fails, data recovery is required to ensure data integrity and reliability.

[0079] The metadata storage area is scanned using the storage system's management tools or a designated scanning program to retrieve the metadata information. The corresponding metadata information, including file name, file size, permissions, and storage location, is extracted from the scanned metadata storage area. Based on the extracted metadata, the metadata information for the storage pool to be repaired is reconstructed, including the structure and file distribution of the repaired storage pool.

[0080] In step 120 , based on metadata information of the storage pool to be repaired, it is determined whether each storage space in the storage pool to be repaired contains valid data, and the storage pool to be repaired contains multiple storage spaces.

[0081] Typically, a storage pool has a very large capacity, potentially reaching hundreds of TB or even PB. The storage pool space is divided into specific capacities, such as 128MB (megabyte) per block storage space, resulting in multiple storage spaces within the storage pool. Each storage space is divided into 8KB (kilobyte) blocks, and each 8KB has a physical address. Each physical address points to a storage space. For a 128MB block, there are 128*128 / 4 physical addresses. The resulting multiple physical addresses are shown in Table 1, Physical Address Table, where N+1 is the number of physical addresses.

[0082] Table 1 Physical address table

[0083] By analyzing the metadata of the storage pool to be repaired, the status, allocation, and file information of each storage space in the storage pool to be repaired can be obtained. This metadata may include the storage space number, capacity, used space, file index, etc.

[0084] Based on the metadata information, it can be further determined whether each storage space contains valid data.

[0085] In step 130, based on whether each storage space in the storage pool to be repaired contains valid data, the bitmap flag corresponding to each storage space is determined, and based on the bitmap flag corresponding to each storage space, bitmap data of the storage pool to be repaired is constructed.

[0086] In some embodiments, whether each storage space contains valid data is determined. In response to determining that the data in a storage space is valid, its bitmap flag bit can be set to "1" to indicate that the storage space contains valid data; in response to determining that the data in a storage space is invalid, its bitmap flag bit can be set to "0" to indicate that the storage space does not contain valid data.

[0087] Each flag bit in the bitmap can be represented by a binary bit, where "1" indicates valid and "0" indicates invalid. For example, if there are 10 storage spaces in the storage pool to be repaired, the bitmap will contain 10 binary bits, each bit corresponding to the validity of a storage space.

[0088] By traversing each storage space, the information of each storage space's bitmap flag is written into the bitmap. Based on the storage space number or index, the flag bit at the corresponding position in the bitmap data can be determined, and the corresponding value ("1" or "0") is set based on the validity of the storage space. After setting the bitmap flag bit corresponding to each storage space in the storage pool to be repaired, the bitmap data of the storage pool to be repaired is obtained.

[0089] In step 140, the data in the storage pool to be repaired is repaired based on the bitmap data.

[0090] In the related method, during the process of repairing the storage pool, a bitmap of the storage space to which the physical address belongs needs to be set once for repairing each physical address in each storage space of the storage pool, resulting in a large number of bitmap setting operations, which affects the data repair time of the storage pool and reduces the data repair efficiency of the storage pool.

[0091] After pre-determining the bitmap data of the storage pool to be repaired, in the subsequent process of repairing the data in the storage pool to be repaired, it is only necessary to determine whether the bitmap of the storage space to which the corresponding physical address belongs is set. If it is determined to be set, there is no need to set it again, which reduces the number of bitmap setting operations and improves the data repair efficiency of the storage pool.

[0092] The method for repairing stored data provided in an embodiment of the present application scans the metadata storage area of ​​the storage pool to be repaired, extracts metadata information, and constructs metadata information for the storage pool to be repaired. Based on the metadata information of the storage pool to be repaired, it is determined whether each storage space in the storage pool to be repaired contains valid data, thereby determining the bitmap data for the storage pool to be repaired. After determining the bitmap data for the storage pool to be repaired, the subsequent process of repairing the data in the storage pool to be repaired only requires determining whether the bitmap of the storage space to which the data belongs has been set. If it is determined to be set, there is no need to set it again, thereby reducing the number of bitmap setting operations and improving the efficiency of data repair in the storage pool.

[0093] In some embodiments, based on bitmap data, data in the storage pool to be repaired is repaired, including: determining the physical address corresponding to the data in the storage pool to be repaired; based on the bitmap data, determining whether the storage space to which the physical address belongs is marked with a bitmap mark bit in the bitmap data; in response to determining that the storage space to which the physical address belongs has been marked with a bitmap mark bit in the bitmap data, repairing the physical address; in response to determining that the bitmap mark bit corresponding to the storage space to which the physical address belongs is not marked in the bitmap data, repairing the physical address and marking the bitmap mark bit corresponding to the storage space to which the physical address belongs in the bitmap data; generating a mark synchronization message based on the bitmap mark bit of the storage space to which the physical address belongs in the bitmap data, and sending the mark synchronization message to the opposite system, so that after the opposite system receives the mark synchronization message, it synchronously updates the bitmap data of the corresponding storage space in the opposite system, and the opposite system is a redundant storage system of the corresponding system of the storage pool to be repaired.

[0094] Based on the data in the storage pool to be repaired, the physical address corresponding to each data block can be determined. Furthermore, based on the bitmap data, it can be determined whether the storage space to which each physical address belongs has a bitmap marker bit in the bitmap. If the bitmap marker bit is "1," it indicates that the storage space contains valid data; if the marker bit is "0," it indicates that the storage space does not contain valid data.

[0095] For the storage space to which the physical address belongs, if the bitmap flag is marked in the bitmap data, it means that the bitmap flag corresponding to the storage space has been set. Therefore, there is no need to set it, and the physical address can be repaired directly to repair the data of the physical address.

[0096] For the storage space to which the physical address belongs, if the bitmap flag is not marked in the bitmap data, it indicates that the bitmap flag of the storage space needs to be set. During the repair process of the physical address, the bitmap flag is marked in the bitmap data of the storage space to which the physical address belongs, thereby implementing the bitmap data update process.

[0097] A mark synchronization message can be generated based on the mark bitmap flag of the storage space to which the physical address belongs in the bitmap data. This mark synchronization message records the physical address and flag bit information of the storage space and indicates that the storage space has been repaired. This mark synchronization message is then sent to the peer system, which receives it and synchronously updates the bitmap data of the corresponding storage space.

[0098] After receiving the mark synchronization message, the peer system can synchronize the bitmap data of the corresponding storage space based on the physical address and mark bit information in the message. That is, the storage space is marked as valid in the peer system's bitmap data to keep it consistent with the bitmap data in the storage pool to be repaired.

[0099] In some embodiments, a synchronization process diagram may be shown in FIG2 . Assume that the storage space contains 16,384 physical addresses. After constructing the bitmap data for the storage pool to be repaired, a bitmap array corresponding to the bitmap data is generated. During data recovery for each physical address in the storage space, based on querying the bitmap array, it is only necessary to determine whether the bitmap for the storage space to which the corresponding physical address belongs has been set. If it is determined to be set, there is no need to set it again, thereby reducing the number of bitmap setting operations. After the synchronization operation is completed, the bitmap array is stored.

[0100] Based on the bitmap mark bit in the bitmap data of the storage space to which the physical address belongs, the data is synchronized to the peer system, and the peer system synchronously updates the bitmap data of the corresponding storage space.

[0101] The storage data repair method provided in the embodiment of the present application predetermines the bitmap data of the storage pool to be repaired. In the subsequent process of repairing the data in the storage pool to be repaired, it is only necessary to determine whether the bitmap of the storage space to which the corresponding physical address belongs is set. If it is determined that it is set, there is no need to set it again, which reduces the number of bitmap setting operations and improves the data repair efficiency of the storage pool.

[0102] In some embodiments, based on whether each storage space in the storage pool to be repaired contains valid data, the bitmap mark bit corresponding to each storage space is determined, including: traversing each storage space in the storage pool to be repaired; in response to determining that the storage space contains valid data, determining that the bitmap mark bit corresponding to the storage space is 1; in response to determining that the storage space does not contain valid data, determining that the bitmap mark bit corresponding to the storage space is 0.

[0103] Each storage space in the storage pool to be repaired is traversed to determine whether each storage space contains valid data. Based on the determination of whether the storage space contains valid data, a bitmap flag corresponding to the storage space is determined.

[0104] In some embodiments, in response to determining that the storage space contains valid data, the bitmap mark bit corresponding to the storage space is determined to be 1; in response to determining that the storage space does not contain valid data, the bitmap mark bit corresponding to the storage space is determined to be 0, so as to determine the bitmap mark bit of each storage space in the storage pool to be repaired.

[0105] The storage data repair method provided in the embodiment of the present application determines the bitmap mark bit corresponding to the storage space based on the judgment of whether the storage space contains valid data, thereby realizing the determination of the bitmap mark bit of each storage space in the storage pool to be repaired.

[0106] In some embodiments, after constructing the bitmap data of the storage pool to be repaired, the method further includes: constructing a bitmap array corresponding to the bitmap data of the storage pool to be repaired, and storing the bitmap array in the memory space of the system corresponding to the storage pool to be repaired.

[0107] After constructing the bitmap data of the storage pool to be repaired, a bitmap array corresponding to the bitmap data of the storage pool to be repaired is determined, and the bitmap data is stored in the form of a bitmap array.

[0108] The bitmap array is stored in the memory space of the system corresponding to the storage pool to be repaired. In response to determining that the data in the storage pool to be repaired needs to be repaired later, the bitmap array is quickly obtained directly from the memory space to determine whether the bitmap mark bit of the corresponding storage space needs to be set, thereby improving the judgment efficiency.

[0109] The method for repairing stored data provided in an embodiment of the present application stores a bitmap array in the memory space of the system corresponding to the storage pool to be repaired. In response to determining that the data in the storage pool to be repaired needs to be repaired later, the bitmap array is quickly obtained directly from the memory space to determine whether the bitmap mark bit of the corresponding storage space needs to be set, thereby improving the judgment efficiency.

[0110] In some embodiments, after storing the bitmap array in the memory space of the system corresponding to the storage pool to be repaired, it also includes: generating a bitmap data synchronization message based on the bitmap array, and sending the bitmap data synchronization message to the peer system, so that after the peer system receives the bitmap data synchronization message, it synchronously updates the bitmap data corresponding to the storage pool to be repaired in the peer system, and the peer system is a redundant storage system of the system corresponding to the storage pool to be repaired.

[0111] A mutually redundant storage system is constructed based on the corresponding system of the storage pool to be repaired and the peer system. In this redundant storage system, data is stored simultaneously in two independent storage systems: the primary storage system and the backup storage system. Data is backed up during storage, and different algorithms are used to achieve redundant replication or distributed storage to ensure data integrity and availability in the event of a failure.

[0112] After determining the bitmap array and storing it in memory, it needs to be synchronized with the peer system. Based on the bitmap array, a bitmap data synchronization message is generated. This message is sent to the peer system. Upon receiving the message, the peer system synchronously updates the bitmap data corresponding to the storage pool to be repaired.

[0113] The storage data repair method provided in the embodiment of the present application ensures the storage consistency of the redundant peer system by synchronously updating the bitmap data of the storage pool to be repaired in the peer system.

[0114] In some embodiments, the method further includes: receiving a synchronization message sent by a peer system, and updating the bitmap data of the storage pool to be repaired based on the synchronization message, wherein the synchronization message is generated by the peer system based on the bitmap array stored in the memory.

[0115] During data repair, both the storage pool system to be repaired and the peer system are often repaired simultaneously. Synchronization is required for both systems during the repair process.

[0116] After the peer system completes the partial repair operation, it needs to be synchronized to the corresponding system of the storage pool to be repaired. The synchronization process of the bitmap data of the peer system is realized based on the synchronization message sent by the peer system.

[0117] The storage data repair method provided in the embodiment of the present application ensures the storage consistency of the redundant peer system by synchronously updating the bitmap data in the peer system.

[0118] In some embodiments, the metadata storage area of ​​the storage pool to be repaired is scanned, and metadata information is extracted from the metadata storage area, including: scanning the metadata storage area of ​​the storage pool to be repaired, determining the node information of the disk data in the storage pool to be repaired from the metadata storage area; traversing the metadata in the node information, and extracting metadata information from the metadata in the node information.

[0119] The storage system first scans the metadata storage area and reads out all the node information that has been written to disk. This node information contains various metadata about the storage pool, such as the file system, file blocks, and storage devices. The system then traverses these nodes, extracting metadata information such as file name, size, permissions, and storage location, and reconstructs the metadata based on this information.

[0120] The purpose of metadata reconstruction is to restore the integrity and consistency of the data in the storage pool. By scanning the entire disk and rebuilding the metadata, the system can re-establish the correct data indexes and mapping relationships, so that subsequent data access and data recovery operations can proceed correctly.

[0121] The storage data repair method provided in the embodiment of the present application realizes the metadata information extraction process by traversing the metadata in the node information and extracting metadata information from the metadata in the node information, thereby providing a basis for the subsequent construction of metadata information of the storage pool to be repaired.

[0122] In some embodiments, each storage space in the storage pool to be repaired is obtained by dividing the storage pool to be repaired based on a preset data block size.

[0123] Generally, the space capacity of a storage pool is very large, which may reach hundreds of TB or even PB level. The space of the storage pool is divided according to a certain capacity. For example, when the preset data block size is 128M, the storage pool to be repaired is divided into 128M sizes to obtain multiple storage spaces of the storage pool to be repaired.

[0124] The method for repairing stored data provided by the embodiment of the present application determines a preset data block size, thereby determining multiple storage spaces in the storage pool to be repaired.

[0125] In some embodiments, scanning the metadata storage area of ​​the storage pool to be repaired and extracting metadata information from the metadata storage area also includes: determining the memory space size for storing the bitmap data of the storage pool to be repaired based on multiple storage spaces contained in the storage pool to be repaired; and applying for a target memory space in the memory of the system corresponding to the storage pool to be repaired based on the memory space size, and the target memory space is used to store the bitmap data of the storage pool to be repaired.

[0126] Before reconstructing the metadata information of the storage pool to be repaired, it is necessary to apply for target memory space in the memory of the system corresponding to the storage pool to be repaired, so as to store the metadata information of the storage pool to be repaired.

[0127] The size of the target memory space applied for may be determined based on the number of storage spaces included in the storage pool to be repaired.

[0128] It can be understood that the bitmap array is stored in the memory space of the system corresponding to the storage pool to be repaired. In response to determining that the data in the storage pool to be repaired needs to be repaired later, the bitmap array is quickly obtained directly from the memory space to determine whether the bitmap mark bit of the corresponding storage space needs to be set, thereby improving the judgment efficiency.

[0129] The storage data repair method provided in the embodiment of the present application applies for target memory space in the memory of the system corresponding to the storage pool to be repaired before reconstructing the metadata information of the storage pool to be repaired, thereby providing a basis for the subsequent storage of the metadata information of the storage pool to be repaired.

[0130] In some embodiments, after repairing the data in the storage pool to be repaired based on the bitmap data, the method further includes: performing metadata verification on the repaired data in the storage pool to be repaired to verify whether the repaired data is accurate.

[0131] Extract metadata information, including file name, size, permissions, timestamp, etc. This metadata information can be used in the subsequent verification process.

[0132] After the data in the storage pool to be repaired is repaired, metadata verification is performed on the repaired data based on the metadata information to determine whether the repaired data is accurate, thereby improving the accuracy of data repair.

[0133] The storage data repair method provided in the embodiment of the present application improves the accuracy of data repair by performing metadata verification on the repaired data based on metadata information after repairing the data in the storage pool to be repaired, thereby determining whether the repaired data is accurate.

[0134] In some embodiments, after repairing the data in the storage pool to be repaired based on the bitmap data, the method further includes: synchronizing the repaired data in the storage pool to be repaired to a peer system, which is a redundant storage system corresponding to the storage pool to be repaired.

[0135] After the data in the system to which the storage pool to be repaired belongs is repaired, the repaired data is synchronized to the peer system.

[0136] An appropriate data synchronization method may be selected based on actual needs and the characteristics of the peer system. In some embodiments, the data synchronization method includes mirror synchronization, incremental synchronization, and asynchronous replication.

[0137] Configure the corresponding data synchronization task based on the selected data synchronization method. During synchronization configuration, specify the storage pool to be repaired as the data source and the peer system as the target. Set synchronization configuration parameters such as synchronization frequency, synchronization policy, and synchronization rules.

[0138] Start the configured data synchronization task and begin synchronizing the repaired data to the peer system. During this process, the system will automatically transfer the repaired data to the peer system to ensure data consistency between the two systems.

[0139] The storage data repair method provided in the embodiment of the present application ensures data consistency between the two systems by synchronizing the repaired data to the peer system after repairing the data of the system to which the storage pool to be repaired belongs.

[0140] The following uses FIG3 , a structural diagram of a method for repairing stored data provided by an embodiment of the present application, as an example to illustrate the technical solution provided by the embodiment of the present application:

[0141] The local system and the peer system to which the storage pool to be repaired belongs perform different data repairs respectively.

[0142] Both processes scan the metadata storage area of ​​the storage pool to be repaired, extract metadata information from the metadata storage area, and construct metadata information of the storage pool to be repaired based on the extracted metadata information. Based on the metadata information of the storage pool to be repaired, it is determined whether each storage space in the storage pool to be repaired contains valid data. The storage pool to be repaired includes multiple storage spaces. Based on whether each storage space in the storage pool to be repaired contains valid data, a bitmap flag corresponding to each storage space is determined. Based on the bitmap flag corresponding to each storage space, bitmap data of the storage pool to be repaired is constructed, and a bitmap array is constructed based on the bitmap data.

[0143] In the process of recovering data from each physical address in the storage space, based on the query bitmap array, it is only necessary to determine whether the bitmap of the storage space to which the corresponding physical address belongs is set. If it is determined to be set, there is no need to set it again, thereby reducing the number of bitmap setting operations.

[0144] Based on the bitmap mark bit in the bitmap data of the storage space to which the physical address belongs, it is synchronized to the peer system. The peer system will synchronize the bitmap data of the corresponding storage space to update the bitmap array of the peer system to keep the consistency of the two systems.

[0145] FIG4 is a schematic diagram of the structure of a storage data repair device provided in an embodiment of the present application. Referring to FIG4 , the storage data repair device provided in an embodiment of the present application includes:

[0146] The scanning module 410 is used to scan the metadata storage area of ​​the storage pool to be repaired, extract metadata information from the metadata storage area, and construct the metadata information of the storage pool to be repaired based on the extracted metadata information; the data verification module 420 is used to determine whether each storage space in the storage pool to be repaired contains valid data based on the metadata information of the storage pool to be repaired, and the storage pool to be repaired contains multiple storage spaces; the bitmap construction module 430 is used to determine the bitmap mark bit corresponding to each storage space based on whether each storage space in the storage pool to be repaired contains valid data, and construct the bitmap data of the storage pool to be repaired based on the bitmap mark bit corresponding to each storage space; the repair module 440 is used to repair the data in the storage pool to be repaired based on the bitmap data.

[0147] The storage data repair device provided in the embodiment of the present application scans the metadata storage area of ​​the storage pool to be repaired, extracts metadata information, and constructs metadata information of the storage pool to be repaired. Based on the metadata information of the storage pool to be repaired, it is determined whether each storage space in the storage pool to be repaired contains valid data, thereby determining the bitmap data of the storage pool to be repaired. After determining the bitmap data of the storage pool to be repaired, in the subsequent process of repairing the data in the storage pool to be repaired, it is only necessary to determine whether the bitmap of the storage space to which the data belongs has been set. If it is determined that it has been set, there is no need to set it again, thereby reducing the number of bitmap setting operations and improving the data repair efficiency of the storage pool.

[0148] In some embodiments, the repair module 440 is used to: determine the physical address corresponding to the data in the storage pool to be repaired; based on the bitmap data, determine whether the storage space to which the physical address belongs is marked with a bitmap mark bit in the bitmap data; in response to determining that the storage space to which the physical address belongs has been marked with a bitmap mark bit in the bitmap data, repair the physical address; in response to determining that the bitmap mark bit corresponding to the storage space to which the physical address belongs is not marked in the bitmap data, repair the physical address and mark the bitmap mark bit corresponding to the storage space to which the physical address belongs in the bitmap data; based on the bitmap mark bit of the storage space to which the physical address belongs in the bitmap data, generate a mark synchronization message, and send the mark synchronization message to the peer system, so that after the peer system receives the mark synchronization message, it synchronously updates the bitmap data of the corresponding storage space in the peer system, and the peer system is a redundant storage system of the system corresponding to the storage pool to be repaired.

[0149] In some embodiments, the bitmap construction module 430 is used to: traverse each storage space in the storage pool to be repaired; in response to determining that the storage space contains valid data, determine that the bitmap mark bit corresponding to the storage space is 1; in response to determining that the storage space does not contain valid data, determine that the bitmap mark bit corresponding to the storage space is 0.

[0150] In some embodiments, the bitmap construction module 430 is further configured to: construct a bitmap array corresponding to the bitmap data of the storage pool to be repaired, and store the bitmap array in the memory space of the system corresponding to the storage pool to be repaired.

[0151] In some embodiments, the bitmap construction module 430 is also used to: generate a bitmap data synchronization message based on the bitmap array, and send the bitmap data synchronization message to the peer system, so that after the peer system receives the bitmap data synchronization message, it synchronously updates the bitmap data of the storage pool to be repaired in the peer system, and the peer system is a redundant storage system of the system corresponding to the storage pool to be repaired.

[0152] In some embodiments, the bitmap construction module 430 is further used to: receive a synchronization message sent by the peer system, and update the bitmap data of the storage pool to be repaired based on the synchronization message, where the synchronization message is generated by the peer system based on the bitmap array stored in the memory.

[0153] In some embodiments, the scanning module 410 is used to: scan the metadata storage area of ​​the storage pool to be repaired, determine the node information of the disk data in the storage pool to be repaired from the metadata storage area; traverse the metadata in the node information, and extract metadata information from the metadata in the node information.

[0154] In some embodiments, the data verification module 420 is used to determine that each storage space in the storage pool to be repaired is obtained by dividing the storage pool to be repaired based on a preset data block size.

[0155] In some embodiments, the scanning module 410 is further used to: determine the size of the memory space for storing the bitmap data of the storage pool to be repaired based on the multiple storage spaces contained in the storage pool to be repaired; and based on the size of the memory space, apply for a target memory space in the memory of the system corresponding to the storage pool to be repaired, and the target memory space is used to store the bitmap data of the storage pool to be repaired.

[0156] In some embodiments, the repair module 440 is further configured to perform metadata verification on the repaired data in the storage pool to be repaired to verify whether the repaired data is accurate.

[0157] In some embodiments, the repair module 440 is further configured to synchronize the repaired data in the storage pool to be repaired to a peer system, where the peer system is a redundant storage system corresponding to the storage pool to be repaired.

[0158] The present application also provides a storage data repair system, comprising a storage pool to be repaired and the above-mentioned storage data repair device.

[0159] The storage data repair system provided in the embodiments of the present application extracts metadata information by scanning the metadata storage area of ​​the storage pool to be repaired, constructing metadata information for the storage pool to be repaired. Based on the metadata information of the storage pool to be repaired, it is determined whether each storage space in the storage pool to be repaired contains valid data, thereby determining the bitmap data of the storage pool to be repaired. After determining the bitmap data of the storage pool to be repaired, the subsequent process of repairing the data in the storage pool to be repaired only needs to determine whether the bitmap of the storage space to which the data belongs has been set. If it is determined that it has been set, there is no need to set it again, thereby reducing the number of bitmap setting operations and improving the data repair efficiency of the storage pool.

[0160] The present application also provides an electronic device, as shown in FIG5 , which may include: one or more processors 510, a communication interface 520, a memory 530 associated with the one or more processors 510, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The memory is used to store computer-readable instructions, which, when read and executed by the one or more processors 510, implement a method for repairing stored data, for example, including:

[0161] Scan the metadata storage area of ​​the storage pool to be repaired, extract metadata information from the metadata storage area, and construct the metadata information of the storage pool to be repaired based on the extracted metadata information; based on the metadata information of the storage pool to be repaired, determine whether each storage space in the storage pool to be repaired contains valid data, and the storage pool to be repaired contains multiple storage spaces; based on whether each storage space in the storage pool to be repaired contains valid data, determine the bitmap mark bit corresponding to each storage space, and construct bitmap data of the storage pool to be repaired based on the bitmap mark bit corresponding to each storage space; based on the bitmap data, repair the data in the storage pool to be repaired.

[0162] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0163] On the other hand, the present application further provides a computer program product, which includes computer-readable instructions stored on a non-transitory computer-readable storage medium. When the computer-readable instructions are executed by one or more processors, the method for repairing stored data provided in the above-mentioned method embodiments can be implemented, for example, including:

[0164] Scan the metadata storage area of ​​the storage pool to be repaired, extract metadata information from the metadata storage area, and construct the metadata information of the storage pool to be repaired based on the extracted metadata information; based on the metadata information of the storage pool to be repaired, determine whether each storage space in the storage pool to be repaired contains valid data, and the storage pool to be repaired contains multiple storage spaces; based on whether each storage space in the storage pool to be repaired contains valid data, determine the bitmap mark bit corresponding to each storage space, and construct bitmap data of the storage pool to be repaired based on the bitmap mark bit corresponding to each storage space; based on the bitmap data, repair the data in the storage pool to be repaired.

[0165] In another aspect, the present application further provides a non-transitory computer-readable storage medium, as shown in FIG6 . The non-transitory computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the method for repairing stored data provided in the above-mentioned method embodiments is implemented, for example, including:

[0166] Scan the metadata storage area of ​​the storage pool to be repaired, extract metadata information from the metadata storage area, and construct the metadata information of the storage pool to be repaired based on the extracted metadata information; based on the metadata information of the storage pool to be repaired, determine whether each storage space in the storage pool to be repaired contains valid data, and the storage pool to be repaired contains multiple storage spaces; based on whether each storage space in the storage pool to be repaired contains valid data, determine the bitmap mark bit corresponding to each storage space, and construct bitmap data of the storage pool to be repaired based on the bitmap mark bit corresponding to each storage space; based on the bitmap data, repair the data in the storage pool to be repaired.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. That is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0168] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for repairing stored data, characterized in that, The method includes: Scanning the metadata storage area of the storage pool to be repaired, extracting metadata information from the metadata storage area, and constructing the metadata information of the storage pool to be repaired based on the extracted metadata information; Based on the metadata information of the storage pool to be repaired, determining whether each storage space in the storage pool to be repaired contains valid data, where the storage pool to be repaired includes multiple storage spaces; Based on whether each storage space in the storage pool to be repaired contains valid data, determining the bitmap flag bits corresponding to each storage space, and constructing the bitmap data of the storage pool to be repaired based on the bitmap flag bits corresponding to each storage space; and Repairing the data in the storage pool to be repaired based on the bitmap data.

2. The method for repairing stored data according to claim 1, wherein The repairing the data in the storage pool to be repaired based on the bitmap data includes: Determining the physical address corresponding to the data in the storage pool to be repaired; Based on the bitmap data, determining whether the storage space to which the physical address belongs is marked with a bitmap flag bit in the bitmap data; In response to determining that the storage space to which the physical address belongs has been marked with a bitmap flag bit in the bitmap data, repairing the physical address; In response to determining that the bitmap data does not mark the bitmap flag bit corresponding to the storage space to which the physical address belongs, repairing the physical address and marking the bitmap flag bit corresponding to the storage space to which the physical address belongs in the bitmap data; Generating a marker synchronization message based on the bitmap flag bit of the storage space to which the physical address belongs in the bitmap data; and Sending the marker synchronization message to the peer system, so that after the peer system receives the marker synchronization message, it synchronously updates the bitmap data of the corresponding storage space in the peer system, where the peer system is the redundant storage system of the system corresponding to the storage pool to be repaired.

3. The method for repairing stored data according to claim 1, wherein The determining the bitmap flag bits corresponding to each storage space based on whether each storage space in the storage pool to be repaired contains valid data includes: Traversing each storage space in the storage pool to be repaired as the current storage space; In response to determining that the current storage space contains valid data, determining that the bitmap flag bit corresponding to the current storage space is 1; or In response to determining that the current storage space does not contain valid data, determining that the bitmap flag bit corresponding to the current storage space is 0.

4. The method for repairing stored data according to claim 1, wherein After constructing the bitmap data of the storage pool to be repaired, it further includes: Constructing a bitmap array corresponding to the bitmap data of the storage pool to be repaired; and Storing the bitmap array in the memory space of the system corresponding to the storage pool to be repaired.

5. The method for repairing stored data according to claim 4, wherein, After storing the bitmap array in the memory space of the system corresponding to the storage pool to be repaired, it further includes: Generating a bitmap data synchronization message based on the bitmap array; and Sending the bitmap data synchronization message to the peer system, so that after the peer system receives the bitmap data synchronization message, it synchronously updates the bitmap data of the corresponding storage pool to be repaired in the peer system, where the peer system is the redundant storage system of the system corresponding to the storage pool to be repaired.

6. The method for repairing stored data according to claim 5, wherein It further includes: Receive a synchronization message sent by the peer system, where the synchronization message is generated by the peer system based on a bitmap array stored in memory; And Update the bitmap data of the storage pool to be repaired based on the synchronization message.

7. The method for repairing stored data according to claim 1, wherein, Scan the metadata storage area of the storage pool to be repaired, and extract metadata information from the metadata storage area, including: Scan the metadata storage area of the storage pool to be repaired, and determine the node information of the disk-down data in the storage pool to be repaired from the metadata storage area; and Traverse the metadata in the node information, and extract the metadata information from the metadata in the node information.

8. The method for repairing stored data according to claim 1, wherein Each storage space in the storage pool to be repaired is obtained by dividing the storage pool to be repaired based on a preset data block size.

9. The method for repairing stored data according to claim 1, wherein Before scanning the metadata storage area of the storage pool to be repaired and extracting metadata information from the metadata storage area, it further includes: Determine the memory space size for storing the bitmap data of the storage pool to be repaired based on the multiple storage spaces included in the storage pool to be repaired; and Apply for a target memory space in the memory of the system corresponding to the storage pool to be repaired based on the memory space size, where the target memory space is used to store the bitmap data of the storage pool to be repaired.

10. The method for repairing stored data according to claim 1, wherein After repairing the data in the storage pool to be repaired based on the bitmap data, it further includes: Perform metadata verification on the repaired data in the storage pool to be repaired to verify whether the repaired data is accurate.

11. The method for repairing stored data according to claim 1, wherein After repairing the data in the storage pool to be repaired based on the bitmap data, it further includes: Synchronize the repaired data in the storage pool to be repaired to the peer system, where the peer system is a redundant storage system of the system corresponding to the storage pool to be repaired.

12. The method for repairing stored data according to claim 11, wherein, The step of synchronizing the repaired data in the storage pool to be repaired to the peer system includes: Synchronize the repaired data to the peer system through a preset data synchronization method, where the preset data synchronization method includes at least one of mirror synchronization, incremental synchronization, and asynchronous replication.

13. The method for repairing stored data according to claim 12, wherein The step of synchronizing the repaired data to the peer system through a preset data synchronization method includes: Determine synchronization configuration parameters according to the preset data synchronization method; and Synchronize the repaired data to the peer system according to the synchronization configuration parameters.

14. The method for repairing stored data according to claim 13, wherein, The synchronization configuration parameters include at least one of synchronization frequency, synchronization policy, and synchronization rule.

15. The method for repairing stored data according to claim 1, wherein The step of scanning the metadata storage area of the storage pool to be repaired includes: Scan the metadata storage area through the management tool of the storage system or a preset scanning program.

16. The method for repairing stored data according to claim 4, wherein, After the step of storing the bitmap array in the memory space of the system corresponding to the storage pool to be repaired and before the step of repairing the data in the storage pool to be repaired based on the bitmap data, it further includes: Obtain the bitmap data from the memory space.

17. A repair device for storing data, characterized in that, Includes: A scanning module, configured to scan the metadata storage area of the storage pool to be repaired, extract metadata information from the metadata storage area, and construct the metadata information of the storage pool to be repaired based on the extracted metadata information; A data verification module, configured to determine whether each storage space in the storage pool to be repaired contains valid data based on the metadata information of the storage pool to be repaired, where the storage pool to be repaired contains multiple storage spaces; A bitmap construction module, configured to determine the bitmap flag bits corresponding to each storage space based on whether each storage space in the storage pool to be repaired contains valid data, and construct bitmap data of the storage pool to be repaired based on the bitmap flag bits corresponding to each storage space; And A repair module, configured to repair the data in the storage pool to be repaired based on the bitmap data.

18. A repair system for storing data, characterized in that, It includes a storage pool to be repaired and a repair device for storing data as described in claim 17.

19. An electronic device, characterized in that, It includes: One or more processors; And A memory associated with the one or more processors, where the memory is used to store computer-readable instructions, and the computer-readable instructions, when read and executed by the one or more processors, implement the method for repairing stored data as described in any one of claims 1 to 16.

20. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions, when executed by one or more processors, implement the method for repairing stored data as described in any one of claims 1 to 16.

Images