Data recovery method and device, electronic equipment and storage medium

Abstract

The application discloses a data recovery method and device, electronic equipment and storage medium, and relates to the technical field of digital data processing. The data recovery area reserved in the data disk in advance is used to save the data of the storage system, so that the target data disk can be quickly locked according to the time effectiveness verification and effectiveness verification of the data during recovery, and then the data recovery is performed by using the data stored in the data recovery area of the target data disk, without full-disk scanning. The data verification is completed while the target data disk is found, the recovery period is reduced, and the technical problem that the full-disk scanning needs to be performed on all back-end data disks during the data structure recovery in the related art, thereby leading to a long recovery period and affecting the business continuity is solved. The system and data are recovered by reorganizing the data structure, and the data recovery speed is faster.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric digital data, and in particular to a data recovery method and device, an electronic device and a storage medium. BACKGROUND

[0002] The storage system occupies a very important position in the architecture of the customer application platform, and the storage and access of core data depend on the storage system. Among them, the double redundancy backup failure caused by software or hardware problems, the storage software system downtime, and the loss of cache data of the storage system, at this time, the business continuity will be affected, and a small amount of data loss will occur.

[0003] In the related art, based on the design architecture of mixed storage of metadata and business data, the data structure is reorganized to recover the system and data through the full-disk scanning method of the back-end data disk, but this scheme has a big drawback. If the storage data volume is large, the recovery period caused by the long full-disk scanning period will be relatively long, thereby affecting the business continuity, which needs to be improved. SUMMARY

[0004] The present application provides a data recovery method and device, an electronic device and a storage medium to at least solve the technical problem that in the related art, when the data structure is recovered, all back-end data disks need to be scanned, resulting in a long recovery period, thereby affecting the business continuity.

[0005] The present application provides a data recovery method, which reserves a data recovery area for storing data in the data disk space accessed by the storage system, wherein the method comprises the following steps: in the case that the target storage system meets the preset data recovery condition, in response to the data recovery instruction of the target storage system, scanning at least one data recovery area associated with the target storage system to determine at least part of the data disk that meets the preset data time limit condition; sorting at least part of the data disk using the identifier of at least part of the data disk to obtain a sorting result, and performing data validity verification on at least part of the data disk based on the sorting result to obtain a verification result, and determining the target data disk based on the verification result; and reconstructing the storage system cluster of the target storage system using the storage data in the data recovery area of the target data disk.

[0006] The application further provides a data recovery device, which reserves a data recovery area for storing data in a data disk space accessed by a storage system, wherein the device comprises: a scanning module, configured to scan at least one data recovery area associated with a target storage system to determine at least part of to-be-selected data disks satisfying a preset data time limit condition, in response to a data recovery instruction of the target storage system, if the target storage system meets a preset data recovery condition; an ordering module, configured to order the at least part of to-be-selected data disks by using the identifiers of the at least part of to-be-selected data disks to obtain an ordering result, perform data validity verification on the at least part of to-be-selected data disks based on the ordering result to obtain a verification result, and determine a target data disk based on the verification result; and a recovery module, configured to reconstruct a storage system cluster of the target storage system by using the stored data in the data recovery area of the target data disk.

[0007] The application further provides an electronic device, comprising: a memory configured to store a computer program; and a processor configured to execute the computer program to implement the steps of any of the data recovery methods.

[0008] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of any of the data recovery methods.

[0009] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the steps of any of the data recovery methods.

[0010] According to the application, the data recovery area reserved in advance in the data disk is used to store the data of the storage system, so that the target data disk can be quickly locked according to the time limit verification and validity verification of the data when recovery, and the data recovery is performed by using the data stored in the data recovery area of the target data disk, without full-disk scanning, and the data verification is completed while the target data disk is searched, thereby reducing the recovery period, solving the problem that all the back-end data disks need to be scanned in the related art when the data structure recovery is performed, leading to a long recovery period and affecting the business continuity, and enabling the data recovery speed to be faster by reorganizing the data structure to recover the system and the data. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0012] Figure 1A flow chart of a data recovery method provided by an embodiment of the present application;

[0013] Figure 2 A structural schematic diagram of a data recovery area provided by an embodiment of the present application;

[0014] Figure 3 A schematic diagram of a calculation principle of a hard disk score provided by an embodiment of the present application;

[0015] Figure 4 A structural schematic diagram of a storage system provided by an embodiment of the present application;

[0016] Figure 5 A structural schematic diagram of a data recovery device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0018] It should be noted that, in the description of the present application, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0019] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] It can be understood that the storage system occupies a very important position in the architecture of the customer application platform, and the storage and access of the core data depend on the storage system.

[0021] Taking the topology of a customer's application platform deployment as an example, in the production system, there are multiple application servers at the top layer, which deploy customer application systems. The application systems access the storage system at the bottom layer through two core database servers. The application servers, database servers, and storage systems are connected through network switches or fiber switches. The application servers and database servers have multiple or dual machine backups. If a single or multiple servers fail, the business continuity and data security will not be affected. However, if the storage system fails, or the data security is damaged, the impact on the overall application system is fatal.

[0022] Therefore, the design of centralized storage is basically dual redundancy, or there is a check backup mechanism to ensure the continuity of the storage system and data security. For example, the controller, system disk, power supply, battery, and fan are designed with dual redundancy. The RAID (Redundant Array of Independent Disks) mechanism realizes the check redundancy at the hard disk level. However, even so, some uncontrollable factors may still cause the storage system to be damaged, such as storage system hardware failure caused by abnormal temperature and humidity of the machine room environment, storage system software failure, or customer failure to maintain in a timely manner.

[0023] According to the fault level, it can be classified as:

[0024] T1: There are dual redundancy backups or check redundancy hardware components, such as single fan, single power supply, single hard disk failure, single storage controller failure caused by software or hardware problems, reboot problems, and no impact on business continuity.

[0025] T2: Software or hardware problems cause dual redundancy backup failure, which will affect business continuity, but will not cause data loss. After replacing the hardware components or software system restart, it can be restored.

[0026] T3: Software or hardware problems cause dual redundancy backup failure, and the storage software system is down, which causes the loss of cache data in the storage system. At this time, the business continuity will be affected, and a small amount of data will be lost.

[0027] T4: Software or hardware failure causes the failure or loss of the back-end data disk, the check redundancy fails, and the saved storage system metadata is lost and cannot be recovered. At this time, the business continuity will be affected, and the data will be damaged or lost due to the failure, and cannot be recovered.

[0028] For the T3 fault level, in the related art, based on the design architecture of mixed storage of metadata and service data, the data structure is reorganized to recover the system and data through the method of full-disk scanning of the rear-end data disk, but when the storage data volume is large, the recovery period caused by the long full-disk scanning period will be relatively long.

[0029] Therefore, the embodiment of the present application can reserve the hard disk space, except for the space for storing the customer service data, the storage system reserves the space of each accessed data disk for storing the attributes of the objects of the key modules of the storage system and the metadata, and when the storage system is normal, the data is maintained and written, and when the T3 fault occurs in the storage system, the data can be quickly recovered by using the data service area.

[0030] As shown in Figure 1 The embodiment of the present application provides a data recovery method, which reserves a data recovery area for storing data in the data disk space accessed by the storage system, wherein the method comprises the following steps:

[0031] In step S101, when the target storage system meets the preset data recovery condition, at least one data recovery area associated with the target storage system is scanned in response to the data recovery instruction of the target storage system to determine at least part of the to-be-selected data disk meeting the preset data time limit condition.

[0032] In the actual execution process, the embodiment of the present application can reserve a data recovery area in the data disk (hard disk) to store the attributes of the objects of the key modules of the storage system and the metadata, which is the basis for the rapid reconstruction of the storage system, and the size of the data recovery area can be calculated according to the maximum hardware and software specifications of the storage system, for example, 512MB of space can be reserved.

[0033] Further, the embodiment of the present application can first determine whether the target storage system needs to perform data recovery and whether the data recovery can be performed, for example, after the T3 fault occurs, the embodiment of the present application can first determine whether the target storage system returns to normal after the hardware fault is repaired or the hardware components are replaced, whether the stored software system returns to the initial state, and whether each module and each component of the target storage system operates normally.

[0034] After determining that the storage system meets the data recovery condition, the embodiment of the present application can perform data recovery on the target storage system.

[0035] The embodiment of the present application can scan data recovery areas of all data disks to determine the time of the last data modification recorded by the data recovery area and the time of the last data validity check to obtain a plurality of to-be-selected data disks with timeliness. In order to avoid the single failure problem of the data disk, the embodiment of the present application can select a plurality of data disks to save multiple copies of the data recovery area (if the number of data disks configured by the storage system is less than three, all the existing data disks are selected to be used), and the data of the data recovery areas of all the copies remains completely consistent.

[0036] The embodiment of the present application can scan only the relatively very small data recovery area reserved on each disk, which greatly reduces the scanning range and improves the recovery efficiency.

[0037] In step S102, the identification of at least part of the to-be-selected data disk is used to sort at least part of the to-be-selected data disk to obtain a sorting result, data validity check is performed on at least part of the to-be-selected data disk based on the sorting result to obtain a check result, and the target data disk is determined based on the check result.

[0038] As a possible implementation, the embodiment of the present application can perform cyclic redundancy check value check on the to-be-selected data disk with the latest data scanned according to the scanning result in the order from small to large according to the identification of the to-be-selected data disk, such as ID, serial number, timestamp of joining the cluster, and so on, to confirm that the data is problem-free. The target storage system can select the first data disk checked to be problem-free as the target data disk to perform data recovery of the target storage system, thereby avoiding blind check and repeated check.

[0039] In step S103, the storage system cluster of the target storage system is reconstructed by using the storage data in the data recovery area of the target data disk.

[0040] The embodiment of the present application can read the data recovery area of the target data disk, traverse all data, record and recover object information and metadata information of each module, and perform reconstruction of each object, reorganization of data space structure, and recovery of configuration information and data before failure.

[0041] Based on the above recovery scheme, the target storage system can quickly recover to the consistent state before downtime, and the business can be quickly recovered. Since the "small amount of data" lost in the cache cannot be recovered, but the skeleton (metadata) of the entire system and most of the data are recovered, the continuity of the business is ensured.

[0042] By the pre-reserved design of "reserving space and separating storage and calculation", the key information (metadata) required for recovery is centrally stored and managed. When a fault occurs, instead of the cumbersome method of full scan, a precise, efficient and low-cost path is used to directly read the "recovery map", thereby achieving the improvement of recovery speed.

[0043] Optionally, in an embodiment of the present application, further comprising: setting a super block and a plurality of data blocks in the data recovery area; wherein the super block is used to store a special character identifier of the data recovery area, a logical block address, a version of a current storage cluster system of the target storage system, a time of a last data modification, a time of a last data validity check, a number of used data blocks, and a starting position of idle data blocks, so as to use the special character identifier and the logical block address to locate the data recovery area in the data disk, and use the time of the last data modification and the time of the last data validity check to determine at least part of the data disks to be selected that meet a preset data time condition.

[0044] As shown in Figure 2 , the data recovery area of the embodiment of the present application can set a super block and a plurality of data blocks.

[0045] The super block, as the head information block or control total block of the data recovery area, can be 4k space size, and contains key metadata describing the overall situation of the data recovery area, and the storage content includes: a special character identifier of the data recovery area, a LBA (Logical Block Address) address of the data recovery area, a version of a current storage cluster system, a time of a last data modification, a time of a last data validity check, a number of used data blocks, and a starting position of idle data blocks.

[0046] The special character identifier can be used for quick positioning of the starting position of the data recovery area. The logical block address can record the address of the data recovery area, and is verified with the special character identifier to ensure that the found area is accurate and correct, and the boundary is clear. The version of the current storage cluster system of the target storage system, the time of the last data modification, and the time of the last data validity check can be used to determine the data time condition to locate the data disk to be selected. The number of used data blocks and the starting position of idle data blocks can be used to quickly locate the starting position when data recording is needed.

[0047] Based on the content of the super block record, the embodiment of the application can ensure the safety and compatibility of the recovery process through version information and check time stamps, avoid secondary disasters caused by software upgrades or data silent damage, and make the data recovery area itself a dynamic writable and updateable live structure rather than a one-time static snapshot through internal space management information.

[0048] Optionally, in an embodiment of the application, the plurality of data blocks are used to store data block special character identification, module identification, object identification, offset address, data block length, object attribute, and metadata information and cyclic redundancy check values of data content, so as to reconstruct the storage system cluster by using the data block special character identification, module identification, object identification, offset address, data block length, object attribute, and metadata information and cyclic redundancy check values of data content.

[0049] The data block can be a unit for storing specific metadata information in the data recovery area. Each data block records detailed information of a certain item of user data or system metadata.

[0050] The data block special character identification is used to identify the beginning of a data block, facilitating parsing and checking. The module identification and object identification are used to identify which functional module (such as a metadata service or a data distribution module) or which data object (such as an inode of a certain file or a certain data shard) the metadata belongs to, so as to classify the metadata and facilitate reconstruction by module or object during recovery. The offset address is used to record the location of actual user data corresponding to the metadata on a physical disk. The data block length and offset address can clearly locate the data. The object attribute and metadata information can describe the attributes of the data object itself, and are used to reconstruct the namespace and attributes of a file. The recovered data is not only the data content, but also complete file information. The cyclic redundancy check value is a check code calculated based on the metadata information recorded in the data block and the user data content pointed to by the data block, and is used to ensure the integrity of the data. During recovery, the cyclic redundancy check value can be used to verify whether the metadata information itself is damaged.

[0051] The cooperative design of the super block and the data block together constitutes an efficient, self-contained, and self-verified metadata backup system, and changes the recovery process from blindly, slowly, and speculatively reconstructing a data structure in massive data to accurately, quickly, and reliably reading a pre-prepared and structured data structure. It reduces the recovery time from a complexity related to the total amount of data to a complexity related to the amount of metadata, thereby greatly guaranteeing business continuity.

[0052] Optionally, in an embodiment of the present application, further comprising: maintaining the offset address and data length information of the data block in the hash table in the case that the target storage system meets the preset normal operation condition; and locating the offset address of the data recovery area based on the hash table when the storage system cluster is reconstructed.

[0053] In the case that the target storage system is in normal operation, the embodiment of the present application can maintain the hash table to save the offset address and data length information of the data block area, which is used for quick location of the offset address of the data recovery area. The key of the hash table is the combination of the module ID and the object ID, and the value of the hash table is the offset address and data length information of the data block area.

[0054] Optionally, in an embodiment of the present application, further comprising: resetting the logical volume used for managing the data recovery area in the target storage system; in the case that the target storage system meets the preset normal operation condition, responding to the logical volume expansion instruction to determine at least one data block to be added in the target logical volume and obtain the target logical volume after expansion; updating the hash table based on the offset position of the at least one data block to be added; modifying the data content of the at least one data block to be added by using the information of the target logical volume after expansion, and recalculating the cyclic redundancy check value of the at least one data block to be added.

[0055] In order to ensure the data recovery efficiency, in the embodiment of the present application, the target storage system needs to be maintained when in normal operation. For the maintenance of the logical volume expansion scenario, the embodiment of the present application can ensure that the metadata of the data recovery area is always synchronized with the real physical storage layout, which is the prerequisite and key guarantee for realizing quick and reliable recovery.

[0056] When the created logical volume is expanded, the implementation steps are as follows:

[0057] Step S1, responding to the expansion instruction, quickly locating the offset position of the data block by using the hash table.

[0058] Step S2, modifying the data block content, including the logical volume capacity size and the metadata information of the logical volume.

[0059] Step S3, recalculating the cyclic redundancy check value of the data block content and writing it into the data block space, so that the new metadata is correctly recorded.

[0060] Step S4, updating the time of the last data modification of the superblock space.

[0061] Optionally, in an embodiment of the present application, further comprising: in the case that the target storage system meets the preset normal operation condition, in response to the data write instruction, accessing the superblock of the at least partial data disk to obtain the starting position of the idle data block of the at least partial data disk; performing content writing based on the starting position of the idle data block to update the data content of the idle data block and the information of the logical volume; calculating the cyclic redundancy check value of the updated data content of the idle data block to obtain an updated cyclic redundancy check value, and writing the updated cyclic redundancy check value into the updated idle data block; updating the time of the last data modification, the number of used data blocks, and the starting position of the idle data block stored in the superblock of the at least partial data disk; and updating the corresponding data block information in the hash table.

[0062] The embodiment of the present application can create a new logical volume through a GUI (Graphical User Interface) interface of the target storage system, and the logical volume can call a corresponding interface to realize writing of new data, and the implementation steps are as follows:

[0063] Step S1: in response to a data write instruction, accessing a superblock to obtain the starting position of an idle data block.

[0064] Step S2: starting content writing of the data block, including a special character identifier of the data block, a logical volume module identifier, a logical volume ID, attributes of the logical volume, including a unique identifier, a logical volume name, a logical volume capacity size, a logical volume type, a storage pool ID to which the logical volume belongs, metadata information of the logical volume, and the metadata information can be used to determine the spatial distribution of the logical volume in the storage pool, and if the logical volume is a thin provisioning, the B+ tree metadata of the logical volume (used to record the mapping relationship between LBA and PBA (Physical Block Address)) is further included.

[0065] Step S3: calculating a cyclic redundancy check value of the data block content and writing the cyclic redundancy check value into the data block space.

[0066] Step S4: updating the time of the last data modification, the number of used data blocks, and the starting position of the idle data block in the superblock space.

[0067] Step S5: recording the data block information newly written in the hash table.

[0068] It should be noted that the cyclic redundancy check value of the data content recorded in the data block is avoided to be tampered. In addition, when the storage system is in long-term stable operation and there is no change in configuration and metadata information, the storage system will periodically inspect the data recovery area in the hard disk. The inspection process will recalculate the cyclic redundancy check value of the data block content, compare it with the saved cyclic redundancy check value, and confirm the consistency of the data block content. If there is an inconsistency in the check, the corresponding module will be notified to record the updated data block content. After the inspection is completed, the time of the last data validity check is updated in the super block to always maintain the validity and reliability of the data.

[0069] The embodiment of the application always maintains the recording and maintenance of data in a fixed position when creating a new logical volume, and ensures that the data always meets the check requirements, thereby saving the time required for subsequent recovery.

[0070] Optionally, in an embodiment of the application, the method further comprises: calculating a hard disk score of at least part of the data disks when the target storage system meets a preset normal operation condition; and selecting a plurality of to-be-stored data disks that meet a preset score condition from the at least part of the data disks by using the hard disk score, so as to store data of the target storage system by using the plurality of to-be-stored data disks.

[0071] When storing data, the target storage system can determine which data disk to store data into by calculating a hard disk score of each data disk, and comprehensively evaluate and rank the data disks through a quantitative scoring system, so as to place the data on the most suitable and reliable data disk.

[0072] When the target storage system is in a healthy and stable working state, all monitoring indicators (such as CPU, memory, network, and IO load) are within the normal threshold range, and the system can perform such background evaluation tasks without affecting the performance of the front-end business, the embodiment of the application can calculate the hard disk score of the data disk by using the health degree (SMART (Self-Monitoring, Analysis and Reporting Technology) attributes such as reallocation sector count, seek error rate, and power-on time), performance (current and historical average read / write operation per second, throughput, delay, etc.), utilization (remaining available capacity, historical load), reliability (fault history, cyclic redundancy check value error count), and cost of the data disk, and according to the hard disk score, a data disk with better comprehensive quality is obtained for data storage.

[0073] Based on the foregoing description, when storing data, to avoid the problem of single failure of the hard disk, a plurality of to-be-stored data disks can be selected when selecting the to-be-stored data disk by using the hard disk score.

[0074] Optionally, in an embodiment of the present application, the hard disk score of at least part of the data disk is calculated, comprising: obtaining historical health data of at least part of the data disk, wherein the historical health data comprises at least one of port error count, slow disk record; obtaining usage state data of at least part of the data disk; obtaining media type of at least part of the data disk; obtaining physical location of at least part of the data disk; respectively assigning corresponding weights to the historical health data, usage state data, media type and physical location, to calculate the hard disk score by using the weights, historical health data, usage state data, media type and physical location.

[0075] Wherein, the historical health data (such as port error count, slow disk record, etc.) is used as the fault-related data based on time and performance as a prediction basis; the usage state data can be the current running load indicator of the data disk, to reflect the current pressure and workload of the disk; the different media types have different characteristics, to realize the key of heterogeneous storage intelligent management according to the characteristics; the physical location can describe the location information in the physical server or cabinet, to avoid placing all copies of data in the same physical server, the same power module or the same rack, to prevent single physical failure from causing all data to be lost.

[0076] As shown in Figure 3 The selection strategy of the data disk to be stored can calculate the hard disk score through four levels, and the hard disk with a high score is preferentially selected to save the recovery data of the storage system. The hard disk score is recorded by int integer, and the different 8 bits of the integer are recorded according to the weight of the key factor, including:

[0077] 1. Data disk (hard disk) health state level, whether the data disk is connected to two controllers, whether there is port error count caused by timeout, transmission error and other problems, whether there is slow disk record and other problems. This level involves the health performance of the data disk, and the weight is the highest, which is recorded by the high 8 bits of the integer.

[0078] 2. Usage state of the data disk, the current data disk state is candidate state (the hard disk has passed the initialization check, but is not used), RAID hot spare disk, and RAID member disk. This level considers to preferentially use the data disk without pressure load, and the weight is higher, which is recorded by the 16th-23rd high 8 bits of the integer.

[0079] 3. Media type of the data disk, such as SSD disk, ten-thousand revolutions of HDD disk, 7200 revolutions of near-line disk. This level preferentially considers to use the data disk with better performance, and the weight is lower, which is recorded by the 8th-15th 8 bits of the integer.

[0080] 4. Physical location of the data disk, the hard disk is located in the main control cabinet or the expansion cabinet. The weight of this level is the lowest, which is recorded by the low 8 bits of the integer.

[0081] If any of the factors related to the data disks mentioned above change, the score will be recalculated, and the data disk with the higher score will be used to replace the data disk with the lower score.

[0082] This invention integrates four dimensions: health, performance, type, and location, avoiding the limitations of relying on a single factor. The weighting mechanism allows the strategy to flexibly adapt to different business scenarios. Data disks with low scores can be assigned to cold data storage or have their writes reduced; high-scoring data disks can be used for core, frequently accessed business data storage, achieving refined hierarchical storage and performance optimization.

[0083] Optionally, in one embodiment of the present invention, before calculating the hard disk score of at least some data disks, the method further includes: obtaining error information of the data recovery area of ​​at least some data disks, so as to use the error information to determine whether the data recovery area has an input / output read / write error; setting the data disk corresponding to the data recovery area with the input / output read / write error as a faulty data disk, and copying the data stored in the faulty data disk to a new data disk.

[0084] Understandably, input / output read / write errors are returned by the operating system or driver layer. If such an error occurs, it proves that the physical media of that area of ​​the data disk has been damaged, and the stored data and metadata are extremely unreliable.

[0085] For example, if an I / O read / write error occurs in the reserved data recovery area, the hard drive will score 0, the hard drive with the higher score will replace the hard drive with the lower score, and the data in the data recovery area will be copied to the newly selected hard drive.

[0086] The entire process of this invention embodiment, from detection, judgment, isolation to reconstruction, is completed automatically, forming an automated closed loop. This enables uninterrupted business operation and provides infrastructure support for the continuous and stable operation of core businesses.

[0087] Combination Figure 4 As shown, the working principle of the data recovery method of this invention will be explained in detail with reference to an embodiment.

[0088] like Figure 4 As shown, the storage system provides a unified interface for writing, reading, and modifying the data recovery area. Each module can call the corresponding interface to achieve read and write access. The following example illustrates the implementation methods of writing, modifying, and reading through the interface. The LV module is the logical volume module.

[0089] In this embodiment of the invention, a data recovery area can be reserved in the hard disk, i.e., the data disk, and the data recovery area can be composed of a superblock and multiple data blocks.

[0090] When the storage system is running normally, a hash table is maintained to save the offset address and data length information of the data block area, which is used for quick positioning of the offset address of the data recovery area. The key of the hash table is the combination of the module ID and the object ID, and the value of the hash table is the offset address and data length information of the data block area.

[0091] When the storage system is running normally, data maintenance and data addition operations can be performed for subsequent quick data recovery.

[0092] When a new logical volume is created through the GUI interface of the storage system, the LV logical volume module can call the corresponding interface to implement the writing of new data:

[0093] 1. Access the superblock to obtain the starting position of the free data block.

[0094] 2. Start the content writing of the data block.

[0095] 3. Calculate the cyclic redundancy check value of the data block content and write it to the data block space.

[0096] 4. Update the time of the last data modification, the number of used data blocks, and the starting position of the free data block in the superblock space.

[0097] 5. Record the newly written data block information in the hash table.

[0098] For the expansion operation of the created logical volume:

[0099] 1. Quickly locate the data block offset position through the hash table.

[0100] 2. Modify the data block content, including the logical volume capacity size and the metadata information of the logical volume.

[0101] 3. Recalculate the cyclic redundancy check value of the data block content and write it to the data block space to avoid data tampering.

[0102] 4. Update the time of the last data modification in the superblock space.

[0103] The storage system will periodically inspect the data recovery area in the hard disk, and the inspection process will recalculate the cyclic redundancy check value of the data block content and compare it with the saved cyclic redundancy check value to confirm the consistency of the data block content. If there is an inconsistency in the check, the corresponding module will be notified to re-record and update the damaged data block content. After the inspection is completed, the time of the last data validity check in the superblock is updated.

[0104] When data storage is performed, hard disk score calculation can be performed on multiple data disks to calculate the hard disk score through four layers, and the hard disk with a high score is preferentially selected to save the recovery data of the storage system. The hard disk score is recorded by int integer, and is recorded to different 8 bit spaces according to the weight of the key factors.

[0105] According to the hard disk score, the embodiment of the application can select multiple data disks to save multiple copies of the data recovery area (if the number of data disks configured by the storage system is less than three, all the existing data disks will be selected), and the data of all the copies of the data recovery area remains completely consistent.

[0106] Among them, the hard disk score will change according to the change of related factors, so as to always use the data disk with a higher hard disk score for storage. And according to the error of the data disk, when IO read-write error occurs, the data disk is no longer allowed to store data, and another data disk is selected to store the data of the error data disk.

[0107] Based on the above process, after the storage system fails at the T3 level, the following rapid recovery can be tried:

[0108] Step S1, first, it is ensured that the hardware failure of the storage device has been restored to normal through maintenance or hardware component replacement, the software system of the storage has been restored to the initial state, and each module and each component is running normally.

[0109] Step S2, scan the superblock space of the data recovery area of all data disks, and according to the time of the last data modification and the time of the last data validity check recorded by the superblock, confirm the data disk with the latest data.

[0110] Step S3, according to the scanning result, the data disk with the latest data scanned is checked by cyclic redundancy check value in the order of data disk ID from small to large, to confirm that the data is problem-free. The system selects the first data disk checked without problems to perform storage system and data recovery.

[0111] Step S4, rebuild the storage system cluster, read the data recovery area of the selected data disk, traverse all data blocks, record and restore the object information and metadata information of each module, and perform object reconstruction, data space structure reorganization, and recovery of pre-failure configuration information and data.

[0112] After the failure is recovered, the embodiment of the application can pull up the upper-layer business. The loss of storage cache data caused by T3 failure may cause damage to the upper-layer file system, database and other applications, and further file system or database repair work is required.

[0113] When repairing, the embodiment of the present application can firstly perform file system unloading to ensure no concurrent access during the repairing process and prevent secondary damage. After unloading, the repairing tool is started to check the block and size, path name, connection line, reference count, cluster group and hole.

[0114] When repairing the data block, the built-in crash recovery mechanism can be relied on to complete the corresponding recovery.

[0115] After the above repairing is completed, the storage system of the embodiment of the present application can pull up the upper-layer service and closely monitor whether the log has data error alarm.

[0116] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0117] As shown in Figure 5 The embodiment of the present application also provides a data recovery device 10, which reserves a data recovery area for storing data in a data disk space accessed by a storage system, wherein the device 10 comprises a scanning module 100, a sorting module 200 and a recovery module 300.

[0118] Specifically, the scanning module 100 is configured to, in a case where a target storage system meets a preset data recovery condition, scan at least one data recovery area associated with the target storage system in response to a data recovery instruction of the target storage system, to determine at least part of data disks to be selected that meet a preset data time limit condition.

[0119] The sorting module 200 is configured to sort the at least part of data disks to be selected by using the identifiers of the at least part of data disks to be selected, to obtain a sorting result, to perform data validity verification on the at least part of data disks to be selected based on the sorting result, to obtain a verification result, and to determine a target data disk based on the verification result.

[0120] The recovery module 300 is configured to reconstruct a storage system cluster of the target storage system by using the stored data in the data recovery area of the target data disk.

[0121] Optionally, in an embodiment of the present application, the data recovery device 10 further comprises a first setting module.

[0122] The first setting module is configured to set a super block and a plurality of data blocks in the data recovery area.

[0123] The super block is used for storing a special character identifier of the data recovery area, a logical block address, a version of a current storage cluster system of the target storage system, a time of a last data modification, a time of a last data validity check, a number of used data blocks, and a start position of idle data blocks, so as to use the special character identifier and the logical block address to locate the data recovery area in the data disk, and use the time of the last data modification and the time of the last data validity check to determine at least part of the data disks to be selected which meet a preset data time limit condition.

[0124] Optionally, in an embodiment of the present application, the plurality of data blocks are used for storing a data block special character identifier, a module identifier, an object identifier, an offset address, a data block length, object attribute and metadata information, and a cyclic redundancy check value of data content, so as to use the data block special character identifier, the module identifier, the object identifier, the offset address, the data block length, the object attribute and the metadata information, and the cyclic redundancy check value of the data content to reconstruct the storage system cluster.

[0125] Optionally, in an embodiment of the present application, the data recovery device 10 further comprises a maintenance module and a locating module.

[0126] The maintenance module is configured to maintain the offset address and the data length information of the data block in the hash table in a case where the target storage system meets a preset normal operation condition.

[0127] The locating module is configured to locate the offset address of the data recovery area based on the hash table when the storage system cluster is reconstructed.

[0128] Optionally, in an embodiment of the present application, the data recovery device 10 further comprises a second setting module, a determination module, a first updating module and a modification module.

[0129] The second setting module is configured to reset a logical volume used for managing the data recovery area in the target storage system.

[0130] The determination module is configured to determine at least one data block to be added of the target logical volume and obtain the target logical volume after expansion in response to a logical volume expansion instruction in a case where the target storage system meets a preset normal operation condition.

[0131] The first updating module is configured to update the hash table based on the offset position of the at least one data block to be added.

[0132] The modification module is configured to modify the data content of the at least one data block to be added by using the information of the target logical volume after expansion, and recalculate the cyclic redundancy check value of the at least one data block to be added.

[0133] Optionally, in an embodiment of the present application, the data recovery device 10 further comprises a first obtaining module, a second updating module, a writing module, a third updating module and a fourth updating module.

[0134] The first obtaining module is configured to, in response to a data write instruction, access a superblock of at least part of the data disks to obtain a starting position of a free data block of at least part of the data disks, when the target storage system meets a preset normal operation condition.

[0135] The second updating module is configured to perform content writing based on the starting position of the free data block to update data content of the free data block and information of the logical volume.

[0136] The writing module is configured to calculate a cyclic redundancy check value of the updated data content of the free data block, obtain an updated cyclic redundancy check value, and write the updated cyclic redundancy check value into the updated free data block.

[0137] The third updating module is configured to update a time of a last data modification, a number of used data blocks and the starting position of the free data block stored in the superblock of at least part of the data disks.

[0138] The fourth updating module is configured to update corresponding data block information in the hash table.

[0139] Optionally, in an embodiment of the present application, the data recovery device 10 further comprises a calculating module and a selecting module.

[0140] The calculating module is configured to calculate a hard disk score of at least part of the data disks, when the target storage system meets a preset normal operation condition.

[0141] The selecting module is configured to select a plurality of to-be-stored data disks that meet a preset score condition from at least part of the data disks by using the hard disk score, so as to store data of the target storage system by using the plurality of to-be-stored data disks.

[0142] Optionally, in an embodiment of the present application, the calculating module comprises a first obtaining unit, a second obtaining unit, a third obtaining unit, a fourth obtaining unit and a calculating unit.

[0143] The first obtaining unit is configured to obtain historical health data of at least part of the data disks, wherein the historical health data comprises at least one of a port error count and a slow disk record.

[0144] The second obtaining unit is configured to obtain usage state data of at least part of the data disks.

[0145] The third obtaining unit is configured to obtain a medium type of at least part of the data disks.

[0146] A fourth obtaining unit is configured to obtain the physical location of at least part of the data disks.

[0147] A computing unit is configured to assign respective weights to the historical health data, the usage status data, the medium type and the physical location, and to calculate the hard disk score by using the weights, the historical health data, the usage status data, the medium type and the physical location.

[0148] Optionally, in an embodiment of the present application, the data recovery device 10 further comprises a second obtaining module and a copying module.

[0149] The second obtaining module is configured to obtain error information of the data recovery area of at least part of the data disks, and to determine whether the data recovery area has an input / output read / write error by using the error information.

[0150] The copying module is configured to set the data disk corresponding to the data recovery area having the input / output read / write error as a faulty data disk, and to copy the data stored in the faulty data disk to a new data disk.

[0151] The features of the embodiments of the data recovery device can be referred to the related descriptions of the embodiments of the data recovery method, which will not be repeated here.

[0152] The embodiments of the present application further provide an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned data recovery method embodiments.

[0153] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above-mentioned data recovery method embodiments when running.

[0154] In an exemplary embodiment, the above-mentioned computer readable storage medium can include but is not limited to a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0155] The embodiments of the present application further provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the above-mentioned data recovery method embodiments.

[0156] The embodiment of the present application further provides another computer program product, comprising a nonvolatile computer readable storage medium, the nonvolatile computer readable storage medium stores a computer program, the computer program is executed by a processor to realize the steps in any one of the above data recovery method embodiments.

[0157] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the above description has generally been stated in terms of the functional components and steps of the examples. Whether such functionality is implemented in hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in varying ways for each particular application, but such implementation should not be interpreted as a departure from the scope of the present application.

[0158] The above has introduced in detail a data recovery method, device, electronic equipment and storage medium provided by the present application. The principle and implementation mode of the present application are described by applying specific examples in this paper, and the above embodiment description is only applicable to help understand the method of the present application and its core idea. It should be pointed out that for those skilled in the art, without departing from the principle of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A data recovery method, characterized in that, A data recovery area is reserved in the data disk space accessed by the storage system for storing data, wherein the method includes the following steps: If the target storage system meets the preset data recovery conditions, in response to the data recovery command of the target storage system, at least one data recovery area associated with the target storage system is scanned to determine at least a portion of the selectable data disks that meet the preset data timeliness conditions; The at least part of the selected data disks are sorted by their identifiers to obtain a sorting result. The at least part of the selected data disks are then validated for data validity based on the sorting result to obtain a validation result. The target data disk is then determined based on the validation result. The storage system cluster of the target storage system is rebuilt using the stored data in the data recovery area of ​​the target data disk; A superblock and multiple data blocks are set in the data recovery area; wherein, the superblock is used to store the special character identifier of the data recovery area, the logical block address, the current storage cluster system version of the target storage system, the time of the most recent data modification, the time of the most recent data validity verification, the number of used data blocks, and the starting position of the free data blocks, so as to use the special character identifier and the logical block address to locate the data recovery area in the data disk, and to use the time of the most recent data modification and the time of the most recent data validity verification to determine at least a portion of the candidate data disks that meet the preset data timeliness conditions; Multiple data blocks are used to store special character identifiers, module identifiers, object identifiers, offset addresses, data block lengths, object attributes, metadata information, and cyclic redundancy check values ​​of the data content, so as to reconstruct the storage system cluster using the special character identifiers, module identifiers, object identifiers, offset addresses, data block lengths, object attributes, metadata information, and cyclic redundancy check values ​​of the data content.

2. The data recovery method according to claim 1, characterized in that, Also includes: Under the condition that the target storage system meets the preset normal operating conditions, maintain the offset address and data length information of the data block in the hash table; When rebuilding the storage system cluster, the offset address of the data recovery area is located based on the hash table.

3. The data recovery method according to claim 1, characterized in that, Also includes: Reconfigure the logical volume used to manage the data recovery area in the target storage system; When the target storage system meets the preset normal operating conditions, in response to the logical volume expansion command, at least one data block to be added to the target logical volume is determined, and the expanded target logical volume is obtained. Update the hash table based on the offset position of at least one of the data blocks to be added; The data content of at least one of the data blocks to be added is modified using the information of the expanded target logical volume, and the cyclic redundancy check value of at least one of the data blocks to be added is recalculated.

4. The data recovery method according to claim 3, characterized in that, Also includes: When the target storage system meets the preset normal operating conditions, in response to a data write command, at least a portion of the superblock of the data disk is accessed to obtain the starting position of the free data block of at least a portion of the data disk; Content is written based on the starting position of the free data block to update the data content of the free data block and the information of the logical volume; Calculate the cyclic redundancy check value of the updated data content of the free data block, obtain the updated cyclic redundancy check value, and write the updated cyclic redundancy check value into the updated free data block; Update the time of the last data modification, the number of used data blocks, and the starting position of free data blocks in the superblock storage of at least a portion of the data disks; Update the corresponding data block information in the hash table.

5. The data recovery method according to claim 1, characterized in that, Also includes: Under the condition that the target storage system meets the preset normal operating conditions, calculate the hard disk score of at least some of the data disks; Using the hard disk scores, a plurality of data disks that meet preset scoring conditions are selected from at least some of the data disks to store data of the target storage system using the plurality of data disks to store data.

6. The data recovery method according to claim 5, characterized in that, The calculation of the hard disk score for at least a portion of the data disks includes: Obtain at least a portion of the historical health data of the data disk, wherein the historical health data includes at least one of port error counts and slow disk records; Obtain at least some of the usage status data of the data disks; Obtain the media type of at least some of the data disks; Obtain the physical location of at least a portion of the data disks; The historical health data, the usage status data, the media type, and the physical location are each assigned a corresponding weight, and the hard drive score is calculated using the weights, the historical health data, the usage status data, the media type, and the physical location.

7. The data recovery method according to claim 5, characterized in that, Before calculating the hard drive score for at least a portion of the data disks, the following is also included: Obtain error information from at least a portion of the data recovery area of ​​the data disk, and use the error information to determine whether an input / output read / write error has occurred in the data recovery area; The data disk corresponding to the data recovery area where the input / output read / write error occurred is set as the faulty data disk, and the data stored in the faulty data disk is copied to the new data disk.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data recovery method as described in any one of claims 1 to 7 when executing the computer program.

Images