A data recovery method for EC replica hybrid redundant storage

Abstract

The present invention provides a data recovery method for EC replica hybrid redundant storage, belonging to the field of data processing technology, the method comprises the following steps: step S1, unit repair operation and data source mapping: firstly, the data source and data operation required for repairing a specific unit are bound, the input is the unit to be repaired, and the output is repair_mapping; step S2, strip read operation mapping: read the data in repair_mapping, map it to the shard required for recovering the data, and the output is strip_mapping; in the present invention, the operation mapping method is adopted to separate the repair operation and the data source, reduce coupling, and reduce complexity, on this basis, strip cache and shard cache are adopted to improve search efficiency, and IO Aggregator is used to merge small IO to improve write efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a data recovery method for EC replica hybrid redundant storage. Background Art

[0002] Erasure coding (EC) is a data redundancy and error repair technology that improves data reliability and fault tolerance by adding redundant data blocks. It divides the original data into multiple strips, each stripe into multiple blocks, and generates a check data block for the stripe, which is stored in data shards on different devices. When data is damaged or lost, it can be restored using the check data block. However, when modifying data, the check block will be recalculated for the entire modified unit, resulting in excessive computing overhead.

[0003] Currently, in file systems that only support append-write, there is a method to avoid excessive computing overhead, that is, to use EC and replica hybrid redundant storage. This method stores data that does not fill up the strip in the form of replicas, and only performs verification data block calculations when data is continuously appended until the strip is full, thereby saving computing overhead. Figure 1 The diagram shows a structure diagram of a non-strip length aligned file using EC4+2 and replica mixed storage. Strips 1 to 5 are partial data of full stripes, original data units are stored in shards 1 to 4, and check data calculated by EC is stored in shards 5 and shard 6. The data in strip 6 is not full stripe, and all unit n and unit m will be stored in shard 5 and shard 6 in a replica redundancy manner.

[0004] Although the hybrid storage of EC and replicas reduces the cost of EC calculations and saves IO times, it will bring greater complexity when recovering data. First, you need to determine whether the data source is directly reading replica data or EC data, or a mixture of the two. You also need to decide whether the written data needs to be calculated for the check block. The process is much more complicated than traditional EC data recovery. Summary of the invention

[0005] The purpose of the present invention is to provide a data recovery method for EC replica hybrid redundant storage, aiming to solve the problem that the prior art brings greater complexity when performing data recovery. First, it is necessary to determine whether the data source is directly read replica data or EC data or a mixture of the two, and it is also necessary to determine whether the written data needs to be calculated for the check block, etc.; the process is much more complicated than traditional EC data recovery.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A data recovery method for EC replica hybrid redundant storage, comprising:

[0008] Step S1: Unit repair operation and data source mapping:

[0009] First, bind the data source and data operation required to repair a specific Unit. The input is the Unit to be repaired, and the output is repair_mapping.

[0010] Step S2, Strip read operation mapping:

[0011] Read the data in repair_mapping, map it to the shard required for data recovery, and output it as strip_mapping;

[0012] Step S3: IO aggregation and data flushing:

[0013] According to the data and operation set mapped in step S1 and step S2, the original shard data is read, and then EC recovery and data flushing are performed.

[0014] As a preferred solution of the present invention, the step S1 includes a tuple of strip_req, repair_op.

[0015] As a preferred solution of the present invention, in step S2, strip_req data in repair_mapping is read.

[0016] As a preferred solution of the present invention, the input in step S2 is strip_req.

[0017] As a preferred solution of the present invention, strip_req in step S1 is the source data strip required to complete the repair operation, and one strip of data is used to repair one unit; the repair_op data operation in step S1 includes writing raw data WRITE_RAW_DATA and writing verification data WRITE_CODING_DATA.

[0018] As a preferred solution of the present invention, the input in step S2 is strip_req, and the output is strip_mapping, that is, a tuple consisting of shard_req and strip_op, wherein

[0019] Shard_req refers to the shard data required to restore strip data. Restoring a strip requires data from multiple shards.

[0020] As a preferred solution of the present invention, the step of repairing Shard is also included, which is as follows:

[0021] Step a: Determine multiple units to be repaired, bind operation data to these units respectively, and aggregate strip_req;

[0022] Step b, get the strip data from the strip cache, if the cache does not hit, then execute the next step, otherwise jump directly to step g;

[0023] Step c: Input the strip_req obtained in the previous step, obtain the shard operation, and aggregate the shard_req;

[0024] Step d, get shard data from shard_cache. If the cache is not hit, read the Shard data and put it into shard_cache. Otherwise, jump to step f.

[0025] Step e: Read Shard data and put it into shard_cache;

[0026] Step f, restore the Strip data and put it into strip_cache;

[0027] Step g: Perform a repair operation and write the data into the IO Aggregator;

[0028] Step h: If the shard has other shards that need to be repaired, jump to step a; otherwise, flush the data in the IOAggregator.

[0029] As a preferred solution of the present invention, the repair_mapping in step S1 is specifically as follows:

[0030]

[0031]

[0032] As a preferred solution of the present invention, the strip_mapping in step S2 is specifically as follows.

[0033]

[0034] As a preferred solution of the present invention, the strip_op includes recovering from the checksum and reading the original data.

[0035] Yet another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program is configured to implement any of the above methods when executed.

[0036] Yet another embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement any of the above methods.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The present invention adopts the operation mapping method to separate the repair operation from the data source, reduce coupling, and reduce complexity. On this basis, strip cache and shard cache are used to improve search efficiency, and IO Aggregator is used to merge small IOs to improve write efficiency;

[0039] 2. Compared with the existing technology, the present invention completes the mapping of the data to be repaired to the existing data through two mapping operations, isolating the complex data distribution of each Shard encountered in the EC file repair process. It has clear logic and simple implementation, and can reduce the complexity of system implementation. At the same time, the mapping relationship between the target Unit and the logical Strip, and the mapping relationship between the logical Strip and the target Shard output in the two mapping operations facilitates the design of the Cache. When multiple Shards of a file need to be repaired, EC Decode and IO overhead can be saved, improving overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0041] Figure 1 A schematic diagram of a system for hybrid storage of EC4+2 and replicas in a data recovery method for hybrid redundant storage of EC replicas of the present invention;

[0042] Figure 2 A is a data structure schematic diagram of a data recovery method for EC replica hybrid redundant storage according to the present invention;

[0043] Figure 3 Schematic diagram B of the data structure in a data recovery method for EC replica hybrid redundant storage of the present invention;

[0044] Figure 4This is a flowchart of a data recovery method for EC replica hybrid redundant storage in the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Example 1

[0047] See also Figure 1-Figure 4 , the present invention provides the following technical solutions:

[0048] A data recovery method for EC replica hybrid redundant storage, comprising:

[0049] Step S1: Unit repair operation and data source mapping:

[0050] First, bind the data source and data operation required to repair a specific Unit. The input is the Unit to be repaired, and the output is repair_mapping.

[0051] Step S2, Strip read operation mapping:

[0052] Read the data in repair_mapping, map it to the shard required for data recovery, and output it as strip_mapping;

[0053] Step S3: IO aggregation and data flushing:

[0054] According to the data and operation set mapped in step S1 and step S2, the original shard data is read, and then EC recovery and data flushing are performed.

[0055] In a specific embodiment of the present invention, compared with the current technology, the present invention completes the mapping of the data to be repaired to the existing data through two mapping operations, and isolates the complex data distribution of each Shard encountered during the EC file repair process. It has clear logic and simple implementation, and can reduce the complexity of system implementation. At the same time, the mapping relationship between the target Unit and the logical Strip, and the mapping relationship between the logical Strip and the target Shard output in the two mapping operations facilitates the design of the Cache. When multiple Shards of a file need to be repaired, EC Decode and IO overhead can be saved, thereby improving overall performance.

[0056] Unit repair operation refers to the operation of repairing or replacing a unit (or module), which is a common hardware maintenance method. Data source mapping refers to the process of mapping data from one form or structure to another form or structure, which usually involves data conversion or data processing; in Unit repair operation, data source mapping may involve backing up or copying the data of the faulty unit to a spare unit, or obtaining data from other normally operating units and using it to replace the faulty unit. This mapping process can ensure the integrity and consistency of the data and avoid data loss or data inconsistency during the repair process; when performing Unit repair operations, it is recommended to refer to relevant operation guides and processes to ensure the correctness and safety of the operation. At the same time, for Unit repair operations involving data source mapping, it is recommended to back up the data before execution and ensure the integrity and consistency of the data.

[0057] "Strip" is often used in the fields of electrical engineering and computer science to refer to a data or signal that has had certain elements or features removed. In file reading operations, "strip" is generally used to remove whitespace characters or specific characters at both ends of a string. For example, Python's `strip()` function can be used to remove spaces at the beginning and end of a string; data mapping operations, in which data or signals are "stripped" or "trimmed" to remove unwanted elements or features. It includes multiple steps, including reading, parsing, filtering, and converting data.

[0058] IO aggregation and data flushing are two concepts in the field of computer storage:

[0059] IO aggregation mainly refers to combining multiple input / output (IO) requests into one, thereby reducing the number of accesses to storage devices. In computer systems, IO requests usually refer to read and write operations on storage devices (such as hard disks, solid-state drives, memory, etc.). These operations usually require mechanical movement or electronic operation of storage devices, so they are relatively time-consuming and prone to causing performance bottlenecks. By aggregating multiple IO requests into one, the number of accesses to storage devices can be reduced, improving the overall performance of the system.

[0060] Data flushing refers to writing data from the cache to a disk or SSD. In a computer system, cache refers to an area in memory that can quickly store and read data. After the data is written to the cache, the system will write the data from the cache to the disk or SSD according to a certain strategy. This process is called data flushing. During the data flushing process, the system will determine the order and frequency of flushing based on factors such as the access frequency and importance of the data to ensure data reliability and performance.

[0061] In WPS AI, the concepts of IO aggregation and data flushing may involve its internal processing mechanism and optimization of storage devices. For example, WPS AI may reduce the number of accesses to storage devices by aggregating multiple IO requests, thereby improving overall performance. At the same time, after data is written to the cache, WPS AI may flush the data to the disk or SSD according to certain strategies to ensure data reliability and performance.

[0062] Specifically, the EC replica hybrid redundant storage data recovery method mainly includes:

[0063] Unit repair operation and data source mapping

[0064] This step binds the data source and data operation required to repair a specific Unit. The input is the Unit to be repaired, and the output is repair_mapping, which contains a tuple of (strip_req, repair_op).

[0065] Strip read operation mapping

[0066] This step maps the strip_req in repair_mapping to the shard required for data recovery. The output is strip_mapping, which is a tuple consisting of (shard_req[n], strip_op).

[0067] IO aggregation and data flushing

[0068] This step reads the original shard data, restores EC, and flushes data based on the data-operation set mapped in 1 and 2, which involves updating each cache and IO aggregation.

[0069] In this embodiment: The present invention adopts the method of binding the operation data source to split the complex repair logic process into several independent logic processes to make each independent logic as simple and reliable as possible. Since the logical mapping decouples the repair write operation from the repair data source acquisition, it is more convenient to perform read and write IO aggregation and process data caching. The repair process is performed for each Unit. Specifically, there are the following sub-processes:

[0070] Sub-process 1: Unit repair operation and data source mapping

[0071] This step binds the data source and data operation required to repair a specific Unit. The input is the Unit to be repaired, and the output is repair_mapping, which contains a tuple of (strip_req, repair_op). Its structure is as follows

[0072] As shown in Table 1:

[0073]

[0074] Table 1

[0075] Among them, strip_req is the source data strip required to complete the repair operation. Repairing a unit can be completed using at most the data of one strip. There are two types of data operations repair_op: write raw data (WRITE_RAW_DATA) and write verification data (WRITE_CODING_DATA). This part only associates the unit to be repaired with the logical strip, and does not care about how to obtain the strip data.

[0076] like Figure 2 As shown:

[0077] If the repair target is unitA, the repair_mapping obtained by the process mapping is shown in Table 2 below:

[0078]

[0079] Table 2

[0080] If the repair target is unit B, the repair_mapping obtained by the process mapping is shown in Table 3 below:

[0081]

[0082] Table 3

[0083] Subprocess 2: Strip read operation mapping

[0084] The strip_req bound in subprocess 1 needs to rely on the original Strip data, but in actual situations, the Strip data may not be complete and may need to be restored. Therefore, the Strip data is also obtained by binding operations with data. Its input is strip_req, and its output is strip_mapping, which is a tuple consisting of (shard_req[n], strip_op). Its structure is shown in Table 4 below:

[0085]

[0086] Table 4

[0087] Shard_req refers to the shard data needed to restore strip data. Restoring a strip usually requires data from multiple shards, so shard_req is in array form. Strip_op includes the following two types: restore from checksum (EC_DECODE) and read raw data (READ_RAW_DATA).

[0088] As attached Figure 3 As shown:

[0089] If strip_req is 4k-8k data of strip6, it can be directly read through the copy data saved in shard6, and the resulting mapping is shown in Table 5. If strip_req is 4k-12k data of strip 5, because strip5 is missing and needs to be restored through EC, the resulting mapping is shown in Table 6.

[0090]

[0091] Table 5

[0092]

[0093] Table 6

[0094] It should be noted that: in this embodiment, the default size of each unit is 4k

[0095] Specifically, it also includes the steps for repairing Shard, as follows:

[0096] Step a: Determine multiple units to be repaired, bind operation data to these units respectively, and aggregate strip_req;

[0097] Step b, get the strip data from the strip cache, if the cache does not hit, then execute the next step, otherwise jump directly to step g;

[0098] Step c: Input the strip_req obtained in the previous step, obtain the shard operation, and aggregate the shard_req;

[0099] Step d, get shard data from shard_cache. If the cache is not hit, read the Shard data and put it into shard_cache. Otherwise, jump to step f.

[0100] Step e: Read Shard data and put it into shard_cache;

[0101] Step f, restore the Strip data and put it into strip_cache;

[0102] Step g: Perform a repair operation and write the data into the IO Aggregator;

[0103] Step h: If the shard has other shards that need to be repaired, jump to step a; otherwise, flush the data in the IOAggregator.

[0104] In this embodiment: Repair overall process

[0105] In the present invention, the main process of Shard repair is performed as described below:

[0106] Step 1: Determine multiple units to be repaired, execute sub-process 1 to bind the operation data for these units respectively, and aggregate strip_req;

[0107] The second step is to get the strip data from the strip cache. If the cache does not hit, execute the next step, otherwise jump to step 7;

[0108] The third step calls sub-process 2, inputs the strip_req obtained in the previous step, obtains the shard operation, and aggregates shard_req;

[0109] Step 4: Get shard data from shard_cache. If the cache is not hit, execute the next step, otherwise jump to step 6.

[0110] Step 5: Read Shard data and put it into shard_cache;

[0111] Step 6: Restore the Strip data and put it into strip_cache;

[0112] Step 7: Perform the repair operation and write the data into the IO Aggregator;

[0113] Step 8. If the shard has other shards that need to be repaired, jump to the first step; otherwise, flush the data in the IOAggregator.

[0114] Compared with the existing technology, the present invention completes the mapping of the data to be repaired to the existing data through two mapping operations, and isolates the complex data distribution of each Shard encountered in the EC file repair process. It has clear logic and simple implementation, and can reduce the complexity of system implementation. At the same time, the mapping relationship between the target Unit and the logical Strip, and the mapping relationship between the logical Strip and the target Shard output in the two mapping operations facilitate the design of the Cache. When multiple Shards of a file need to be repaired, EC Decode and IO overhead can be saved, thereby improving overall performance.

[0115] In Elasticsearch, a Mapping defines how documents are indexed and searched. It allows you to specify which fields are indexed, how they should be analyzed, and whether they should be stored for full-text search.

[0116] Basic operations of Mapping:

[0117] 1. Create a Mapping: To create an index, you need to define a Mapping. For example, the following code creates an index called "customer" with two fields: "name" (which will be analyzed and stored for full-text search) and "age" (only stored without analysis):

[0118]

[0119]

[0120] 2. Update Mapping: If you need to update an existing Mapping, you can use the PUT method. For example, the following code adds a new field named "email" to the Mapping of the existing "customer" index:

[0121] ```bash

[0122] PUT / customer / _mapping{"properties":{"email":{"type":"text"}}}

[0123] ```

[0124] 3. Batch query: You can use curl to batch query all Mappings. For example, the following code can query the Mappings of all indexes:

[0125] ```bash

[0126] curl-X GET"localhost:9200 / _mapping"

[0127] ```

[0128] If you need to query a specific index and its mapping, you can specify the index in the curl command. For example, the following code can query the mapping of the index named "customer":

[0129] ```bash

[0130] curl-X GET"localhost:9200 / customer / _mapping"

[0131] ```

[0132] The method in the present invention has been used in the Lava storage engine to implement EC replica hybrid redundant Chunk repair in a distributed system. The main process is executed on one node, and the nodes interact with each other through RPC, and play a core role in the Chunk Seal, PG Recovery and other processes; the operation mapping method is used to separate the repair operation and the data source, reduce coupling and complexity; on this basis, strip cache and shard cache are used to improve search efficiency; IO Aggregator is used to merge small IOs to improve write efficiency; the implementation is simple, the logic is clear, and read and write IO aggregation is supported to reduce IO overhead. Multi-level Cache can improve EC computing efficiency and reduce repeated IO; by decoupling the repair process and the source data acquisition process, the repair operation is simplified; by the operation-data binding method, read IO is aggregated, and write IO is aggregated through IO Aggregator, which can save EC Decode and IO overhead and improve overall performance.

[0133] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to implement the steps of any of the above method embodiments when running.

[0134] Specifically, in this embodiment, the above storage medium may be configured to store a computer program for performing the following steps:

[0135] Step S1: Unit repair operation and data source mapping:

[0136] First, bind the data source and data operation required to repair a specific Unit. The input is the Unit to be repaired, and the output is repair_mapping.

[0137] Step S2, Strip read operation mapping:

[0138] Read the data in repair_mapping, map it to the shard required for data recovery, and output it as strip_mapping;

[0139] Step S3: IO aggregation and data flushing:

[0140] According to the data and operation set mapped in step S1 and step S2, the original shard data is read, and then EC recovery and data flushing are performed.

[0141] Specifically, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0142] Compared with the prior art, the present invention adopts the operation mapping method to separate the repair operation and the data source, reduce coupling, and reduce complexity. On this basis, strip cache and shard cache are used to improve search efficiency, and IOAggregator is used to merge small IOs to improve write efficiency. Compared with the current technology, two mapping operations are performed to complete the mapping of the data to be repaired to the existing data, isolating the complex data distribution of each Shard encountered during the EC file repair process. It has clear logic and simple implementation, and can reduce the complexity of system implementation. At the same time, the mapping relationship between the target Unit and the logical Strip, and the mapping relationship between the logical Strip and the target Shard output in the two mapping operations facilitate the design of the Cache. When multiple Shards of a file need to be repaired, EC Decode and IO overhead can be saved, thereby improving overall performance.

[0143] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0144] Specifically, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0145] Specifically, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0146] Step S1: Unit repair operation and data source mapping:

[0147] First, bind the data source and data operation required to repair a specific Unit. The input is the Unit to be repaired, and the output is repair_mapping.

[0148] Step S2, Strip read operation mapping:

[0149] Read the data in repair_mapping, map it to the shard required for data recovery, and output it as strip_mapping;

[0150] Step S3: IO aggregation and data flushing:

[0151] According to the data and operation set mapped in step S1 and step S2, the original shard data is read, and then EC recovery and data flushing are performed.

[0152] Compared with the prior art, the present invention adopts the operation mapping method to separate the repair operation and the data source, reduce coupling, and reduce complexity. On this basis, strip cache and shard cache are used to improve search efficiency, and IOAggregator is used to merge small IOs to improve write efficiency. Compared with the current technology, two mapping operations are performed to complete the mapping of the data to be repaired to the existing data, isolating the complex data distribution of each Shard encountered during the EC file repair process. It has clear logic and simple implementation, and can reduce the complexity of system implementation. At the same time, the mapping relationship between the target Unit and the logical Strip, and the mapping relationship between the logical Strip and the target Shard output in the two mapping operations facilitate the design of the Cache. When multiple Shards of a file need to be repaired, EC Decode and IO overhead can be saved, thereby improving overall performance.

[0153] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0154] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0156] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present invention. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0159] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data recovery method for EC replica hybrid redundant storage, characterized in that: include: Step S1, block repair operation and data source mapping: First, bind the data source and data operation required to repair a specific block. The input is the block to be repaired, and the output is repair_mapping. The repair_mapping is a tuple containing strip_req and repair_op. The strip_req is the source stripe of the source data required to complete the repair operation. Repairing a block uses the data of one stripe. The repair_op is divided into writing original data and writing verification data. Step S2: Stripe read operation mapping: Read the data in repair_mapping, map it to the shards required for data recovery, and output it as strip_mapping, where strip_mapping is a tuple consisting of shard_req and strip_op; Step S3: IO aggregation and data flushing: According to the data and operation set mapped by step S1 and step S2, the original shard data is read, and then EC recovery and data flushing are performed.

2. A data recovery method for EC replica hybrid redundant storage according to claim 1, characterized in that: In the step S2, strip_req data in repair_mapping is read.

3. A data recovery method for EC replica hybrid redundant storage according to claim 2, characterized in that: The input in step S2 is strip_req.

4. The data recovery method for EC replica hybrid redundant storage according to claim 1, characterized in that: In step S2, the input is strip_req and the output is strip_mapping, where Shard_req refers to the shard data required to restore the stripe data. Restoring a stripe requires data from multiple shards. strip_op includes recovery from checksum and reading the original data.

5. A data recovery method for EC replica hybrid redundant storage according to claim 4, characterized in that: It also includes the steps for repairing Shard, as follows: Step a, determine multiple blocks to be repaired, bind operation data to these blocks respectively, and aggregate strip_req; Step b, get the stripe data from the stripe cache, if the cache does not hit, then execute the next step, otherwise jump directly to step g; Step c: Input the strip_req obtained in the previous step, obtain the shard operation, and aggregate the shard_req; Step d: Get shard data from the shard cache. If the cache is not hit, read the shard data and put it into the shard cache. Otherwise, jump to step f. Step e: Read the Shard data and put it into the shard cache; Step f, restore the Strip data and put it into the Strip cache; Step g: Perform a repair operation and write the data into an IO Aggregator, where the IO Aggregator is used to merge small IOs. Step h: If the shard has other shards that need to be repaired, jump to step a; otherwise, flush the data in the IO Aggregator.

6. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to implement the method described in any one of claims 1 to 5 when executed.

7. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the computer program to implement the method described in any one of claims 1 to 5.

Images