Data recovery method and device of storage system, electronic equipment and storage medium

Abstract

The invention provides a data recovery method and device of a storage system, electronic equipment and a storage medium, and relates to the technical field of distributed storage. The free storage space is used for generating a locally optimized redundant data block which coexists with an original redundant data block of the data object and can recover data in a smaller fault domain range, and when a storage fault is detected, the corresponding redundant data block is selected and called according to the fault range to execute recovery operation. The problems that in the prior art, due to the fact that the locality of erasure codes is insufficient and the fault domain range of original redundant data block recovery data is large, data needs to be accessed across multiple nodes during fault recovery, the transmission amount is large, the recovery speed is low, and the system load is high can be solved, and the purposes of optimizing the locality of the erasure codes, flexibly adapting to different fault scenes and improving the recovery efficiency are achieved. And the cross-node data transmission overhead is reduced, the fault recovery efficiency is improved, and the overall load of the system is reduced.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular to a data recovery method and apparatus, electronic device and storage medium for a storage system. Background Technology

[0002] Distributed storage systems are core infrastructure in cloud computing and big data, widely used for massive data storage and high-reliability scenarios. Related technologies utilize erasure coding and redundancy strategies to construct a multi-level data protection system covering data sharding, encoding generation, and redundant distribution, including key aspects such as explicit redundant block storage and cross-node data recovery. With the significant increase in storage scale, while traditional erasure coding schemes can guarantee high reliability, their reliance on global data access leads to low fault recovery efficiency, becoming a key issue restricting system performance. Simultaneously, a large amount of idle space in storage systems remains unutilized, and redundancy strategies are mostly statically configured, further exacerbating network load and hardware resource contention during the recovery process.

[0003] Existing erasure coding schemes have several shortcomings: they employ a global redundancy strategy without fully integrating dynamic adjustments to free space, which can easily lead to storage fragmentation and reduce overall system efficiency; they require access to multiple nodes during fault recovery, resulting in large amounts of cross-node data transfer; implicit redundancy mechanisms cannot flexibly adjust according to fault type and system state, making it difficult to balance local data protection with global disaster recovery requirements; the redundancy protection of warm data is interconnected with hot and cold data, exacerbating uneven system load and recovery delays, affecting system availability and cost-effectiveness; in cross-fault domain redundancy protection, the selection of redundant nodes is fixed and not dynamically optimized based on real-time load, resulting in persistent network hotspot bottlenecks and limiting the scalability and stability of large-scale distributed systems. Summary of the Invention

[0004] This application provides a data recovery method and apparatus, electronic device and storage medium for a storage system, in order to at least partially solve one of the technical problems in the related art.

[0005] This application provides a data recovery method for a storage system, including:

[0006] Detect the free storage space within each storage node of a distributed storage system; Using free storage space, at least one locality-optimized redundant data block is generated for the stored data object, wherein the locality-optimized redundant data block coexists with the original redundant data block of the data object and is configured to be used to recover data within a smaller fault domain than the original redundant data block in the event of a failure. When a storage failure is detected, depending on the scope of the failure, either locally optimized redundant data blocks or existing redundant data blocks are selected to perform data recovery operations.

[0007] This application also provides a data recovery device for a storage system, comprising: The detection unit is used to detect the free storage space in each storage node of the distributed storage system. The generation unit is used to generate at least one locally optimized redundant data block for a stored data object using free storage space. The locally optimized redundant data block coexists with the original redundant data block of the data object and is configured to recover data within a smaller fault domain than the original redundant data block in the event of a failure. The recovery unit is used to select either locally optimized redundant data blocks or the original redundant data blocks to perform data recovery operations when a storage failure is detected, depending on the scope of the failure.

[0008] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above methods.

[0009] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above methods.

[0010] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0011] This application provides a data recovery method, apparatus, electronic device, and storage medium for a storage system. By detecting the free storage space in each storage node of a distributed storage system, it generates a locality-optimized redundant data block that coexists with the original redundant data block of the data object and can recover data within a smaller fault domain using the free storage space. When a storage fault is detected, the corresponding redundant data block is selected and called to perform the recovery operation according to the fault range. Therefore, it can solve the problem in the prior art that due to insufficient locality of erasure coding, the fault domain range of the original redundant data block recovery data is large, which leads to the need to access data across multiple nodes and large transmission volume during fault recovery, resulting in slow recovery speed and high system load. It achieves the technical effects of optimizing locality of erasure coding, flexibly adapting to different fault scenarios, reducing cross-node data transmission overhead, improving fault recovery efficiency, and reducing the overall system load.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 A schematic flowchart illustrating a data recovery method for a storage system provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a data recovery device for a storage system provided in an embodiment of this application. Detailed Implementation

[0014] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0015] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] The specific application environment architecture or specific hardware architecture on which the execution of the data recovery method in conjunction with the storage system depends is described here.

[0017] Embodiments of this application provide a data recovery method for a storage system. Figure 1 This is a schematic flowchart of a data recovery method for a storage system provided in an embodiment of this application.

[0018] like Figure 1 As shown, the method includes the following steps: Step 101: Detect the free storage space in each storage node of the distributed storage system.

[0019] In the embodiments of this application, the free space detection module (i.e., the functional module for identifying free storage resources) can be deployed on the control node of the distributed storage system or locally on each storage node. It employs a mechanism combining timed polling and triggered scanning to comprehensively traverse all storage media (including mechanical hard drives, solid-state drives, etc.) within each storage node. This module reads the partition table, file system metadata, and data storage mapping records of the storage media, parses the space occupancy identifiers, and filters out physical storage areas not occupied by actual user data or system core configuration files. Simultaneously, it accurately records the starting address, continuous capacity, disk number, and unique identifier of each free area, forming a structured list of free space. In response to space changes caused by data allocation or reclamation in the storage system, this module updates the list in real time, ensuring the accuracy and timeliness of free space information and providing reliable physical space support for subsequent local optimization of redundant data blocks.

[0020] By accurately and in real-time detecting free space, we can fully understand the distribution of free storage resources in each storage node, avoid the waste of idle space, provide accurate space basis for the local storage of redundant data blocks, ensure the effective implementation of local optimization strategies, and lay the foundation for the system to dynamically adjust the redundancy configuration.

[0021] Step 102: Using the free storage space, generate at least one locality-optimized redundant data block for the stored data object. The locality-optimized redundant data block coexists with the original redundant data block of the data object and is configured to be used to recover data within a smaller fault domain than the original redundant data block in the event of a failure.

[0022] In the embodiments of this application, the data block organization module and the implicit data redundancy block generation module (i.e., the functional module responsible for data reorganization and redundancy generation) work together. For warm data (medium access frequency) already stored in the distributed storage system, based on the free space distribution information within each storage node obtained by the free space detection module, a reorganization strategy within the node or within the same fault domain is adopted. Through a background asynchronous calculation method, the original data blocks of the data object are encoded according to the erasure coding rules to generate at least one implicit data redundancy block (i.e., a locally optimized redundant data block). This implicit data redundancy block is stored in the free storage area within the storage node where the original data object is located or within the same fault domain. It exists in parallel with the default redundant data block (i.e., the default EC check block) originally stored in the explicit data area and does not interfere with each other. At the same time, the storage address of the implicit data redundancy block, the corresponding fault domain range, and the associated original data block identifier are synchronously written into the metadata of the data object to provide data support for rapid location and retrieval during subsequent fault recovery.

[0023] By generating and storing locally optimized redundant data blocks within a smaller fault domain, and having them coexist with the original redundant data blocks, the global disaster recovery capability of the original redundant data blocks is preserved. This also solves the problems of large fault domain range and cross-node transmission during the recovery of traditional redundant data blocks. It provides nearby redundant resources for small-scale fault scenarios, effectively improves the locality of fault recovery, and lays the foundation for subsequent rapid data recovery.

[0024] Step 103: When a storage failure is detected, depending on the scope of the failure, select to call the locality-optimized redundant data block or the original redundant data block to perform the data recovery operation.

[0025] In the embodiments of this application, the fault recovery mechanism module (i.e. the functional module responsible for optimizing the fault recovery process) monitors the operating status of storage nodes and storage media in the distributed storage system in real time, identifies storage faults through hardware status feedback signals, data read and write response timeout detection, etc., and defines the size of the fault domain based on the scope of storage resources affected by the fault (such as a single storage disk, a single storage node, or multiple nodes across the fault domain). Upon detecting a fault, the module first queries the metadata of the data objects involved in the fault to obtain key information such as the storage location of locally optimized redundant data blocks (i.e., implicit data redundancy blocks) and the original redundant data blocks (i.e., default EC check blocks), as well as the corresponding fault domain range. If the fault range is limited to a single storage disk within a single storage node (small fault domain), the locally optimized redundant data blocks within that node are prioritized and combined with the original data blocks within the same node to complete local data reconstruction without cross-node data transfer. If the fault range expands to a single storage node or multiple nodes across fault domains (large fault domain), the original redundant data blocks stored in the explicit data area are called, and cross-node or cross-fault domain data recovery operations are performed according to traditional erasure coding recovery logic to ensure the integrity of data recovery.

[0026] By dynamically selecting appropriate redundant data blocks to perform recovery operations based on the scope of the fault, the fault recovery strategy and the fault scenario are precisely matched. In the case of a small-scale fault, the locality of the redundant data blocks is optimized for rapid recovery, avoiding the latency and load consumption caused by cross-node transmission. In the case of a large-scale fault, the original redundant data blocks are used to ensure disaster recovery reliability, effectively balancing fault recovery efficiency and data protection comprehensiveness, and significantly improving the overall availability of the distributed storage system.

[0027] This application provides a data recovery method for a storage system. By detecting the free storage space in each storage node of the distributed storage system, the free storage space is used to generate a locality-optimized redundant data block that coexists with the original redundant data block of the data object and can recover data within a smaller fault domain. When a storage fault is detected, the corresponding redundant data block is selected and called to perform the recovery operation according to the fault range. Therefore, it can solve the problem in the prior art that due to insufficient locality of erasure coding, the fault domain range of the original redundant data block recovery data is large, which leads to the need to access data across multiple nodes and large transmission volume during fault recovery, resulting in slow recovery speed and high system load. The method achieves the technical effects of optimizing locality of erasure coding, flexibly adapting to different fault scenarios, reducing cross-node data transmission overhead, improving fault recovery efficiency, and reducing the overall system load.

[0028] In the embodiments involved in this application, there are various feasible specific implementation methods. To clearly and completely illustrate the technical solutions of this disclosure, the implementation methods listed below are merely exemplary and do not constitute a limitation on the scope of protection of this disclosure. That is, in addition to the implementation methods described below, other implementation methods that can be obtained by those skilled in the art based on the technical content disclosed in this disclosure through reasonable logical analysis, reasoning, or limited experimentation should also be covered within the scope of protection of this disclosure. The following specifically describes some exemplary implementation methods: As a specific implementation of this application, based on the basic scheme, the detection of free storage space in each storage node of the distributed storage system is further limited, including: dynamically monitoring the capacity status of the storage nodes; when a free area that meets the preset space threshold and is continuously distributed is identified in the node, it is marked as a candidate space that can be used to generate redundant data blocks for locality optimization.

[0029] Specifically, the free space detection module (i.e., the functional module responsible for identifying free storage resources) adopts a real-time dynamic monitoring mechanism to continuously read the capacity occupancy data, space allocation logs, and physical address mapping tables of the storage media within each storage node, and to track the capacity change status of the nodes in real time. The system is pre-configured with the minimum space requirement for generating locally optimized redundant data blocks (i.e., implicit data redundancy blocks), and a preset space threshold is set based on this requirement (e.g., configured as 1.2 times the capacity required for a single locally optimized redundant data block). During the monitoring process, the free space detection module analyzes the physical address distribution information of the storage media to determine whether the unoccupied free areas meet the condition of continuous address distribution. When the continuous capacity of a certain free area is identified to reach the preset space threshold, the area is immediately marked as a candidate space, and its continuous address range, actual capacity, storage disk to which it belongs, and node identifier are recorded simultaneously. This information is then updated to the structured free space list, providing accurate space adaptation support for the subsequent generation and storage of redundant data blocks.

[0030] By combining dynamic monitoring with threshold screening, continuous free space that meets the requirements for redundancy generation can be accurately identified, avoiding the problem of redundant data block storage failure or degraded read / write performance caused by fragmented free space. At the same time, it ensures the sufficient capacity of the candidate space, providing a reliable guarantee for efficient storage and fast retrieval of redundant data blocks through local optimization.

[0031] As a specific implementation of this application, based on the basic scheme, it is further defined to generate at least one locally optimized redundant data block for the stored data object, including: filtering out target data objects according to the access characteristics of the data; logically reorganizing some or all of the data blocks of the target data object with other data blocks located in the same storage node or the same fault domain based on the distribution location of the candidate space to form a new data stripe; and encoding and calculating the new data stripe to generate locally optimized redundant data blocks.

[0032] Specifically, the data block organization module (i.e., the functional module responsible for data reorganization) first analyzes the access logs and data read / write frequency statistics of the distributed storage system to identify data access characteristics and select moderately accessed data as target data objects (i.e., data objects to be generated as locally optimized redundant data blocks). Based on the candidate space distribution location information marked by the free space detection module, this module logically reorganizes and sorts some or all of the original data blocks of the target data objects with the original data blocks of other target data objects located in the same storage node or the same fault domain (such as the same rack), ensuring that the storage location corresponding to the reorganized data block set is in the same node or the same fault domain as the candidate space, forming a new data stripe with a regular structure (i.e., implicit data stripe). Subsequently, the implicit data redundant block generation module (i.e., the functional module responsible for redundancy generation in the technical solution) calls the preset erasure coding algorithm to encode all the original data blocks in the new data stripe, generating the corresponding locally optimized redundant data blocks (i.e., implicit data redundant blocks), and stores the redundant data blocks in the corresponding candidate space, while updating the association information about the new data stripe and redundant blocks in the data object metadata.

[0033] By accurately filtering target data objects based on data access characteristics, redundant optimization resources are focused on high-frequency demand scenarios, improving resource utilization efficiency. Based on the candidate spatial distribution, data blocks within the same node or fault domain are reorganized and encoded, further enhancing the locality of redundant data blocks. This provides more direct redundancy support for subsequent small-scale fault recovery and significantly reduces data transmission overhead across nodes or fault domains.

[0034] As a specific implementation of this application, based on the basic scheme, the redundant data blocks for local optimization are further defined to include at least one of the following types: intra-node redundant blocks, in which all data blocks in the corresponding data stripe are located within the same storage node; and intra-fault domain redundant blocks, in which all data blocks in the corresponding data stripe are located within the same fault domain but distributed across different storage nodes.

[0035] Specifically, the data block organization module and the implicit data redundancy block generation module (i.e., the functional module responsible for data reorganization and redundancy generation) work together to generate two types of locally optimized redundant data blocks (i.e., implicit data redundancy blocks) based on the candidate space and data distribution determined by the free space detection module. For intra-node redundant blocks, the data block organization module selects the original data blocks of the target data object (i.e., warm data) within the same storage node, logically reorganizes them with other target data blocks within the node, ensuring that all data blocks in the reorganized data stripe (i.e., implicit data stripe) are located within the storage node. Then, the implicit data redundancy block generation module generates the corresponding intra-node redundant blocks through erasure coding operations and stores them in the candidate space within the node. For redundant blocks within a fault domain, the data block organization module uses the fault domain (such as the same rack) as a range to filter target data blocks in different storage nodes within the fault domain and perform logical reorganization, so that all data blocks in the data stripe are in the same fault domain but belong to different storage nodes. Then, redundant blocks within the fault domain are generated through encoding operations and stored in the candidate free space within the fault domain. At the same time, the types of the two types of redundant blocks, the range of the corresponding data stripe, and the storage address are synchronously written into the metadata.

[0036] By clarifying the specific generation logic and distribution rules of the two types of locality optimization redundant blocks, it can accurately adapt to small-scale fault scenarios at different levels. The redundant blocks within the node further shorten the spatial range of fault recovery, while the redundant blocks within the fault domain balance the locality and fault tolerance capabilities within the fault domain. Both avoid the cross-large-scale transmission problem of traditional global redundancy, and significantly improve the data recovery efficiency and system stability under different small fault scenarios.

[0037] As a specific implementation of this application, based on the basic solution, the target data objects are further selected according to the access characteristics of the data, including: monitoring the access frequency and access time distribution of the data objects; identifying data objects whose access frequency is within a preset medium frequency range as warm data objects, and prioritizing them as target data objects.

[0038] Specifically, the data block organization module (i.e., the functional module responsible for data reorganization) collects read and write request logs from the distributed storage system in real time, extracts access frequency statistics and timestamp information for each data object, and constructs a data access feature profile. The system pre-configures a medium-frequency range for access frequency (e.g., set to a range of 10-100 accesses per day, which can be dynamically adjusted according to the actual application scenario of the system), and makes a comprehensive judgment based on access time distribution characteristics (such as whether there are scattered accesses in multiple time periods, no long-term idle or continuous high-load access characteristics). When the access frequency of a data object falls within the preset medium-frequency range and the access time shows a scattered distribution characteristic, it is identified as a warm data object and marked as a priority processing identifier in the data access feature profile, making it a priority target object for generating locally optimized redundant data blocks. At the same time, the metadata of the data object is updated synchronously, recording its warm data attributes and priority optimization identifier, providing a clear basis for subsequent data block reorganization and redundancy generation.

[0039] By accurately monitoring access frequency and time distribution, warm data objects are selected and prioritized as optimization targets. This ensures that localized redundant resources are concentrated on scenarios with moderate and continuous access demand, avoiding ineffective redundant consumption of hot and cold data. This improves the targeting and efficiency of system resource utilization and provides accurate object selection criteria for the subsequent generation of localized optimization redundant data blocks, ensuring the practical value of the optimization strategy.

[0040] As a specific implementation of this application, based on the basic scheme, the data recovery operation is further defined according to the scope of the fault, selecting to call locally optimized redundant data blocks or the original redundant data blocks to perform the data recovery operation, including: when a single storage device fails, first querying whether there is a path that can be recovered through redundant blocks within the node, and if so, using that path; when a single storage node fails, first querying whether there is a path that can be recovered through redundant blocks within the fault domain, and comprehensively evaluating the redundancy ratio of each available path and the load of related nodes to select the target path.

[0041] Specifically, the fault recovery mechanism module (i.e., the functional module responsible for optimizing the fault recovery process) first determines the fault type after detecting a storage fault by using storage device status feedback signals and node communication link detection results, clarifying whether it is a single storage device fault or a single storage node fault. When it is determined to be a single storage device fault, this module first queries the metadata of the storage node to which the faulty device belongs, searching for the existence of a recovery path corresponding to a redundant block within the node (i.e., the intra-node type of redundant data blocks in locally optimized redundant blocks). If the metadata records such a path and the intra-node redundant block is in normal status and matches the target data block in the faulty device, then the path is directly adopted, calling the intra-node redundant block and the original data block within the same node to complete local data reconstruction without cross-node data transfer. When it is determined to be a single storage node fault, this module first queries the metadata of the fault domain to which the faulty node belongs, searching for the existence of a redundant block within the fault domain (i.e., the intra-node type of redundant blocks in locally optimized redundant blocks). The system identifies available recovery paths corresponding to the fault domain types within locally optimized redundant data blocks. Simultaneously, it obtains the redundancy ratio (i.e., the ratio of redundant blocks to corresponding data blocks) associated with each available path and the real-time load data of relevant nodes (including CPU utilization, storage bandwidth utilization, network transmission rate, etc.) through the system resource monitoring module. The system then calculates the path priority (the lower the redundancy ratio and the lower the node load, the higher the priority). Finally, it selects the path with the highest priority to perform data recovery. If there is no available path for a redundant block within the fault domain, the system switches to the path corresponding to the original redundant data block (i.e., the default EC check block) to complete the recovery.

[0042] By developing differentiated path query and selection strategies for different fault types, single storage device failures rely on redundant blocks within the node to achieve rapid local recovery, while single storage node failures select the optimal path by comprehensively evaluating the redundancy ratio and node load. This maximizes the advantages of local optimization of redundant data blocks while avoiding performance bottlenecks caused by high redundancy ratios and high-load nodes, significantly improving the targeting and efficiency of data recovery under different fault scenarios and further optimizing the overall availability of the system.

[0043] As a specific implementation of this application, based on the basic solution, the embodiment of this application further includes: maintaining metadata information of locally optimized redundant data blocks, wherein the metadata information records at least the corresponding data object identifier, redundancy type and physical storage location; when the total free space of the system is lower than the recycling threshold, some locally optimized redundant data blocks are selected and deleted according to a preset strategy to release space.

[0044] Specifically, the data block organization module and the implicit data redundancy block generation module (i.e., the functional module responsible for data reorganization and redundancy generation) work together to maintain the metadata information of the locally optimized redundant data blocks (i.e., implicit data redundancy blocks). This metadata is stored in the metadata server of the distributed storage system or the local metadata storage area of ​​each storage node. It records at least the unique identifier of the corresponding data object (such as data object ID), the redundancy type (internal redundant block or fault domain redundant block) and the physical storage location (including the storage node ID, storage disk number, starting physical address and occupied capacity). At the same time, the timestamp of the metadata is updated synchronously to ensure the timeliness of the information. The system pre-configures a free space reclamation threshold (which can be set according to a preset proportion or a fixed capacity value of the total system storage capacity). The free space detection module counts the total free space of the distributed storage system in real time. When the total free space is detected to be lower than the reclamation threshold, the space reclamation mechanism is triggered. Based on preset strategies (such as prioritizing the deletion of localized optimized redundant data blocks with the earliest generation time and the lowest access frequency, or retaining redundant blocks within the fault domain according to redundancy type priority, and prioritizing the deletion of redundant blocks within the node), the redundant data blocks to be deleted are selected. The corresponding data blocks are deleted by calling the space release interface of the storage medium, and the metadata information is updated synchronously to mark the redundant data block as reclaimed, ensuring that the released free space can be used for normal system data storage or high-priority redundant generation.

[0045] By maintaining detailed metadata information, the system can quickly locate and retrieve locally optimized redundant data blocks during fault recovery, thus improving the smoothness of the recovery process. Based on the space reclamation mechanism with reclamation thresholds and preset strategies, non-critical redundant resources can be released in a timely manner when the system's free space is scarce, avoiding the impact of insufficient free space on the system's core storage functions, and achieving a dynamic balance between redundancy optimization and space utilization.

[0046] It should be noted that the embodiments of this disclosure may include multiple steps. For ease of description, these steps are numbered, but these numbers are not a limitation on the execution time slots or execution order between the steps; these steps can be implemented in any order, and the embodiments of this disclosure do not limit this.

[0047] Corresponding to the data recovery method for the storage system described above, this disclosure also proposes a data recovery apparatus for a storage system. Since the apparatus embodiments of this disclosure correspond to the method embodiments described above, details not disclosed in the apparatus embodiments can be referred to the method embodiments described above, and will not be repeated here.

[0048] Figure 2 This is a schematic diagram of the structure of a data recovery device for a storage system provided in an embodiment of this disclosure, as shown below. Figure 2 As shown, it includes: Detection unit 21 is used to detect the free storage space in each storage node of the distributed storage system; The generation unit 22 is used to generate at least one locality-optimized redundant data block for the stored data object using free storage space. The locality-optimized redundant data block coexists with the original redundant data block of the data object and is configured to recover data within a smaller fault domain than the original redundant data block in the event of a fault. The recovery unit 23 is used to select, based on the scope of the fault, either to invoke locally optimized redundant data blocks or the original redundant data blocks to perform data recovery operations when a storage fault is detected.

[0049] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of this embodiment, and the principle is the same, so it is not limited in this embodiment.

[0050] For a description of the features of the data recovery device in the storage system in the corresponding embodiment, please refer to the relevant description of the data recovery method in the storage system in the corresponding embodiment, which will not be repeated here.

[0051] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the data recovery method embodiments of the storage system described above.

[0052] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in the data recovery method embodiments of any of the above storage systems when running.

[0053] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0054] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data recovery method embodiments of the storage system.

[0055] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data recovery method embodiments of the storage system.

[0056] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0057] The data recovery method, apparatus, electronic device, and storage medium of a storage system provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data recovery method for a storage system, characterized in that, Applied to a distributed storage system employing erasure coding, the method includes: Detect the free storage space within each storage node of the distributed storage system; Using the free storage space, at least one locality-optimized redundant data block is generated for the stored data object, wherein the locality-optimized redundant data block coexists with the original redundant data block of the data object and is configured to recover data within a smaller fault domain than the original redundant data block in the event of a failure. When a storage failure is detected, the locality-optimized redundant data block or the original redundant data block is selected to perform a data recovery operation, depending on the scope of the failure.

2. The method according to claim 1, characterized in that, The detection of free storage space in each storage node of the distributed storage system includes: Dynamically monitor the capacity status of storage nodes; When a free region that meets a preset space threshold and is continuously distributed within a node is identified, it is marked as a candidate space that can be used to generate the redundant data block for locality optimization.

3. The method according to claim 2, characterized in that, The step of generating at least one locally optimized redundant data block for a stored data object includes: Filter out target data objects based on data access characteristics; Based on the distribution location of the candidate space, some or all of the data blocks of the target data object are logically reorganized with other data blocks located in the same storage node or the same fault domain to form a new data stripe; The new data stripe is encoded and calculated to generate the locality-optimized redundant data block.

4. The method according to claim 3, characterized in that, The redundant data blocks for locality optimization include at least one of the following types: Redundant blocks within a node, where all data blocks in the corresponding data stripe are located within the same storage node; Redundant blocks within a fault domain are data blocks in the corresponding data stripe that are located within the same fault domain but distributed across different storage nodes.

5. The method according to claim 3, characterized in that, The step of filtering target data objects based on data access characteristics includes: Monitor the access frequency and access time distribution of data objects; Data objects whose access frequency falls within a preset medium frequency range are identified as warm data objects and are given priority as the target data objects.

6. The method according to claim 1, characterized in that, The step of selecting to invoke the locally optimized redundant data block or the original redundant data block to perform data recovery operation based on the fault range includes: When a single storage device fails, the system first checks if there is a path that can be recovered using redundant blocks within the node; if so, that path is used. When a single storage node fails, the system first queries whether there is a path that can be recovered through redundant blocks in the failure domain, and comprehensively evaluates the redundancy ratio of each available path and the load of related nodes to select the target path.

7. The method according to claim 1, characterized in that, Also includes: Maintain the metadata information of the redundant data blocks for locality optimization, wherein the metadata information records at least the corresponding data object identifier, redundancy type and physical storage location; When the total free space of the system is lower than the recycling threshold, some of the redundant data blocks of the locality optimization are selected and deleted according to the preset strategy to release space.

8. A data recovery device for a storage system, characterized in that, Applications include distributed storage systems employing erasure coding, including: The detection unit is used to detect the free storage space in each storage node of the distributed storage system; The generation unit is used to generate at least one locally optimized redundant data block for the stored data object using the free storage space, wherein the locally optimized redundant data block coexists with the original redundant data block of the data object and is configured to recover data within a smaller fault domain than the original redundant data block in the event of a fault. The recovery unit is used to select, based on the scope of the fault, either the locally optimized redundant data block or the original redundant data block to perform a data recovery operation when a storage fault is detected.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the data recovery method of the storage system according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the data recovery method for the storage system according to any one of claims 1-7.

Images