Data processing method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a data processing method and device, a storage medium and electronic equipment, and relates to the technical field of computers.The method comprises the steps that in response to a wrong disk recovery request of a storage system, a recovery method corresponding to a wrong disk is determined; the number of the staggered disks is at least one; performing data recovery on the wrong disk by utilizing a recovery method to obtain a data recovery result; each stripe in the storage system is divided into a first local check group, a second local check group and a global check group; the global check group comprises a global check disk, a first data disk part and a second data disk part; the first local check group comprises a first local check disk, a first data disk part and a global check disk; and the second local check group comprises a second local check disk, a second data disk part and a global check disk. Therefore, a grouping mode of the first local check group, the second local check group and the global check group is provided, at least one wrong disk can be recovered by using the recovery method corresponding to the wrong disk, and the fault-tolerant capability is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a data processing method, apparatus, storage medium, and electronic device. Background Art

[0002] With the rapid development of communication technologies and network technologies, digital information has grown exponentially, posing a huge challenge to data storage technologies. The reliability of data in storage systems and the energy consumption of storage systems have attracted increasing attention. In the face of a huge data scale, the reliability of data in a storage system is inversely proportional to the number of components in the storage system, that is, the more components in the storage system, the lower the reliability of the data in the storage system. Relevant investigations show that in an Internet data center composed of 600 disks, about 30 disks are damaged every month. In large-scale storage systems, the decrease in data reliability caused by disk failures is a quite serious problem. Therefore, relevant fault-tolerant technologies have become one of the research focuses.

[0003] RAID (Redundant Arrays of Independent Disks), that is, a disk array with redundancy capabilities, is a key technology for improving storage space. A disk array is formed by combining multiple independent disks to obtain a disk group with a large capacity. By adopting RAID storage technology, the storage capacity can be greatly increased, the request processing ability of the system input / output can be improved, and the data reliability can be improved through data distributed storage technology, parallel access means, and information redundancy technology.

[0004] RAID technology mainly uses data striping, data verification, and mirroring technologies to obtain strong performance, higher reliability, better fault tolerance, and strong scalability. According to different data application requirements, the strategies and architectures of these three technologies can be used alone or in combination. According to different strategies and architectures, RAID can be divided into different levels: RAID 0, RAID 1, RAID 5, RAID 6, RAID 10, etc.

[0005] RAID 0 adopts the data striping technology and does not provide redundancy or error repair capabilities. The implementation method is to connect N (N is greater than or equal to 2) identical hard disks in series in hardware through an intelligent disk controller or in software through a disk driver in the operating system to create a large volume set. In use, computer data is written to each hard disk in turn. Its advantage is that the hard disk capacity can be increased by an integer multiple. For example, if three 80GB hard disks are used to form a RAID 0 mode, the disk capacity is 240GB. Its disadvantage is that if any hard disk fails, the entire system will be damaged and the reliability is low.

[0006] RAID 1 is called disk mirroring. The principle is to mirror the data of one disk to another disk. That is, when data is written to one disk, a mirror file will be generated on another idle disk. Without affecting performance, it can ensure the reliability and repairability of the system to the greatest extent. As long as at least one disk in any pair of mirrored disks in the system can be used, the system can even run normally when half of the hard disks have problems. When one hard disk fails, the system will ignore this hard disk and instead use the remaining mirrored disks to read and write data, with good disk redundancy capabilities. The disadvantages are high cost and a disk utilization rate of only 50%. For example, for four 80GB-capacity hard disks, the available disk space is only 160GB. In addition, if a hard disk fails, the damaged hard disk needs to be replaced in time. Otherwise, if the remaining mirrored disks also have problems, the entire system will crash, and it will take a long time to resynchronize the mirror after replacing the new disk. Therefore, RAID 1 is mostly used in occasions where critical and important data needs to be saved. RAID 5 is an independent disk structure with distributed parity check. Its parity check code exists on all disks, and p0 among them represents the parity check value of the 0th stripe. RAID 5 has relatively high read performance and average write performance, and the block-based collective access efficiency is good. Since the parity check code is on different disks, the reliability is improved. However, it does not solve the parallelism of data transmission well, and the design of the controller is quite difficult. For RAID 5, most data transmissions only operate on one disk and can be parallelized. There is a "write penalty" in RAID 5, that is, for each write operation, four actual read / write operations will be generated, including two reads of the old data and parity information and two writes of the new data and parity information.

[0007] RAID 6 is an independent disk structure with two distributed storage parity check codes, which is an extension of RAID 5 and is mainly used in occasions where data must absolutely not be incorrect. Since a second parity check value is introduced, N + 2 disks are required, and at the same time, the design of the controller becomes very complex, further improving the data reliability of the disk array. More space is needed to store the check values, and there is a higher performance loss in write operations.

[0008] RAID technology has been widely used in today's distributed storage servers. Especially in RAID 5 and RAID 6, one or two error blocks can be recovered respectively. Through fault tolerance and data recovery technologies, the availability of data and the stability of the system can be effectively improved.

[0009] However, each data recovery is still limited by the speed limit when a large amount of data is read from each disk, and the fault tolerance ability still needs to be improved. Summary of the Invention

[0010] The present disclosure provides a data processing method, apparatus, storage medium, and electronic device to at least solve the above technical problems existing in the prior art.

[0011] The technical solution of the embodiments of the present disclosure is implemented as follows: In a first aspect, an embodiment of the present disclosure provides a data processing method, the method including: In response to a disk error recovery request of a storage system, determining a recovery method corresponding to the disk error; the number of disk errors is at least one; Using the recovery method to perform data recovery on the disk error to obtain a data recovery result; Wherein, each stripe in the storage system is divided into a first local parity group, a second local parity group, and a global parity group; The global parity group includes a global parity disk, a first data disk part, and a second data disk part; the first local parity group includes a first local parity disk, a first data disk part, and the global parity disk; the second local parity group includes a second local parity disk, a second data disk part, and the global parity disk.

[0012] In a second aspect, an embodiment of the present disclosure provides a data processing apparatus, the apparatus including: A first processing module, configured to determine a recovery method corresponding to a disk error in response to a disk error recovery request of a storage system; the number of disk errors is at least one; A second processing module, configured to perform data recovery on the disk error using the recovery method to obtain a data recovery result; Wherein, each stripe in the storage system is divided into a first local parity group, a second local parity group, and a global parity group; The global parity group includes a global parity disk, a first data disk part, and a second data disk part; the first local parity group includes a first local parity disk, a first data disk part, and the global parity disk; the second local parity group includes a second local parity disk, a second data disk part, and the global parity disk.

[0013] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the data processing methods.

[0014] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause a computer to execute the data processing method according to any one of the above.

[0015] The embodiments of the present disclosure have the following beneficial effects: Applying the data processing method, apparatus, storage medium and electronic device provided by the embodiments of the present disclosure to a storage system, the method includes: in response to a disk error recovery request of the storage system, determining a recovery method corresponding to the disk error; the number of the disks in error is at least one; using the recovery method to perform data recovery on the disks in error to obtain a data recovery result; wherein, each stripe in the storage system is divided into a first local parity group, a second local parity group and a global parity group; the global parity group includes a global parity disk, a first data disk part and a second data disk part; the first local parity group includes a first local parity disk, a first data disk part and the global parity disk; the second local parity group includes a second local parity disk, a second data disk part and the global parity disk. In this way, a grouping method of the first local parity group, the second local parity group and the global parity group is provided, and based on this grouping method, at least one disk in error can be recovered by using the recovery method corresponding to the disk in error, improving the fault tolerance of the storage system.

[0016] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of a RAID55 grouping provided by an embodiment of the present disclosure; Figure 2 It is a schematic flow chart of a data processing method provided by an embodiment of the present disclosure; Figure 3 It is a schematic diagram of grouping of data disks and parity disks provided by an embodiment of the present disclosure; Figure 4 It is a schematic diagram of one disk in error provided by an application embodiment of the present disclosure; Figure 5 It is a schematic diagram of two disks in error provided by an application embodiment of the present disclosure Figure 1 ; Figure 6 It is a schematic diagram of two disks in error provided by an application embodiment of the present disclosure Figure 2 ; Figure 7 It is a schematic diagram of two disks in error provided by an application embodiment of the present disclosure Figure 3 ; Figure 8 It is a schematic diagram of two disks in error provided by an application embodiment of the present disclosure Figure 4 ; Figure 9 It is a schematic diagram of three disks in error provided by an application embodiment of the present disclosure Figure 1 ; Figure 10 Schematic diagram of three misaligned disks provided by an application embodiment of the present disclosure Figure 2 ; Figure 11 Schematic diagram of three misaligned disks provided by an application embodiment of the present disclosure Figure 3 ; Figure 12 Schematic diagram of three misaligned disks provided by an application embodiment of the present disclosure Figure 4 ; Figure 13 Schematic diagram of three misaligned disks provided by an application embodiment of the present disclosure Figure 5 ; Figure 14 Schematic diagram of three misaligned disks provided by an application embodiment of the present disclosure Figure 6 ; Figure 15 Schematic diagram of three misaligned disks provided by an application embodiment of the present disclosure Figure 7 ; Figure 16 Schematic diagram of three misaligned disks provided by an application embodiment of the present disclosure Figure 8 ; Figure 17 Structural schematic diagram of a data processing device provided by an embodiment of the present disclosure; Figure 18 Structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0018] To make the objectives, features, and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of them. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present disclosure.

[0019] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0020] If similar descriptions such as "first / second" appear in the application documents, the following explanations are added. In the following description, the terms "first\second\third" only distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence when allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein are for the purpose of describing embodiments of this application only and are not intended to limit this application.

[0022] Before further elaborating on the embodiments of the present disclosure, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following explanations.

[0023] RAID55 is an enhanced version of RAID5, and its fault tolerance is stronger than that of RAID5. RAID55 can recover lost data through parity check when one or two disks fail. In addition, when three disks fail, RAID55 can recover some data, but not in all cases. The following are two situations where RAID 55 cannot recover data: The first situation: When all three failed disks (abbreviated as faulty disks) are in the same local RAID5 group, the data cannot be recovered. Because all parity check information is concentrated among these three disks, and losing these disks makes recovery impossible.

[0024] The second situation: When one of the three faulty disks is a physical disk failure and the other two are in the same local RAID5 group, recovery is also impossible because some parity check information is lost, resulting in recovery failure.

[0025] In addition, if RAID55 uses the traditional left - hand non - aligned disk dropping method, it can only recover any one or two faulty disks, and cannot recover any three disks, which also reduces the fault tolerance of the RAID55 algorithm.

[0026] The following provides an example to specifically illustrate RAID55 in the related art.

[0027] Suppose there are disks in total, including 2 local parity disks , , 1 global parity disk , data disks. Using RAID55, a maximum of 2 faulty disks can be recovered. Taking k = 13 as an example for illustration.

[0028] First, when k = 13, there are a total of 10 data disks. The 10 data disks and 1 global parity disk form a global RAID5, and its encoding method is: . represents the exclusive - OR operation, respectively represent each data disk (which can also be referred to as data, data block), represents the global parity disk (which can also be referred to as the global parity block, global parity data). The above-mentioned data disks and global parity disk are data belonging to different physical disks on the same stripe.

[0029] Then, divide the 10 data disks into two groups, with 5 data disks in each group; Take a total of 5 data disks and the local parity disk to form the first local RAID5 group, and the encoding method is: , where is the position information of the data disk , is the position information of the local parity disk . It changes according to the change of the stripe.

[0030] Take a total of 5 data disks and the local parity disk to form the second local RAID5 group, and the encoding method is: , where is the position information of the data disk , is the position information of the local parity disk .

[0031] To sum up, the encoding rule of RAID55 is shown in the following formula:

[0032] As Figure 1 shown, it is a schematic diagram of an encoding method of RAID55. Combining Figure 1 it can be known that there are the following situations in the decoding of RAID55: One disk is faulty: RAID55 can recover for any faulty disk; Two disks are faulty: RAID55 can recover for any two faulty disks; Three disks are faulty: There are the following two situations where data recovery cannot be achieved. The first is that all three faulty disks are in one local RAID5 group; the second is that one of the three faulty disks is the global parity disk and the other two are in the same local RAID5 group.

[0033] It should be noted that the above description is based on one stripe. In actual production, considering load balancing, common data is written to disks in a way that the left rotation is not aligned. As shown in Table 1, a schematic diagram of the RAID55 disk writing method is provided, where disk1, disk2... disk13 represent physical disks, and t1, t2... t13 represent stripes.

[0034] Table 1

[0035] In the table, the purpose of the disk writing method with left rotation not aligned is for load balancing. However, this disk writing method will cause any three failed disks in RAID55 not to be recovered. As shown in Table 2 below, when the 2nd, 3rd, and 8th physical disks (i.e., disk2, disk3, and disk8) have errors, the three failed disks in stripe 8 (i.e., k8) are all in the second local RAID5 group, and the data is not recoverable at this time. In fact, the left rotation not aligned disk writing method of RAID55 enables RAID55 to only recover any two failed disks, reducing the fault tolerance performance. Therefore, a method for improving the fault tolerance performance of RAID55 is needed.

[0036] Table 2

[0037] Based on this, the embodiments of the present disclosure provide a data processing method, apparatus, storage medium, and electronic device. The method includes: in response to a failed disk recovery request of a storage system, determining a recovery method corresponding to the failed disk; the number of the failed disks is at least one; using the recovery method to perform data recovery on the failed disk to obtain a data recovery result; wherein, each stripe in the storage system is divided into a first local parity group, a second local parity group, and a global parity group; the global parity group includes a global parity disk, a first data disk part, and a second data disk part; the first local parity group includes a first local parity disk, a first data disk part, and the global parity disk; the second local parity group includes a second local parity disk, a second data disk part, and the global parity disk. In this way, a grouping method of the first local parity group, the second local parity group, and the global parity group is provided. Based on this grouping method, at least one failed disk can be recovered by using the recovery method corresponding to the failed disk, improving the fault tolerance ability of the storage system.

[0038] Figure 2 As shown in the flowchart of a data processing method provided by the embodiments of the present disclosure, Figure 2 the method includes: Step 201, in response to a failed disk recovery request of a storage system, determining a recovery method corresponding to the failed disk; the number of the failed disks is at least one; Step 202: Use the recovery method to perform data recovery on the misaligned disk to obtain a data recovery result; Among them, each stripe in the storage system is divided into a first local parity group, a second local parity group, and a global parity group; The global parity group includes a global parity disk, a first data disk part, and a second data disk part; the first local parity group includes a first local parity disk, a first data disk part, and a global parity disk; the second local parity group includes a second local parity disk, a second data disk part, and a global parity disk.

[0039] Here, the method can be applied to a storage system. Each stripe in the storage system may include: a first local parity disk, a second local parity disk, a global parity disk, a first data disk part, and a second data disk part; among them, the first data disk part and the second data disk part have the same number of data disks.

[0040] A data disk can also be called a data block or a data unit, which is the actual data stored on a physical disk.

[0041] Parity disks such as local parity disks and global parity disks can also be called parity blocks or parity data units, which are parity data stored on a physical disk. In a storage system such as RAID, a physical disk is a physical entity in a disk array, referring to a real hard disk. Data disks and parity disks can be understood as logical concept blocks, which will be mapped to the storage locations of physical disks, that is, each data block and parity block will be written to a specific area of a physical disk.

[0042] Striping is a technology that disperses data storage across multiple disks. Specifically, striping divides data into individual data blocks and stores these data blocks sequentially on multiple physical disks. Correspondingly, when reading data, multiple data blocks can be obtained from multiple physical disks simultaneously, thereby improving the performance of the storage system.

[0043] For example, the total number of disks for the first local parity disk, the second local parity disk, the global parity disk, the first data disk part, and the second data disk part is k , that is, each stripe of the storage system has a total of k disks, including: 1 first local parity disk, 1 second local parity disk, 1 global parity disk, k - 3 data disks, k≥ 5. The first data disk part and the second data disk part each have ( k - 3) / 2 data disks.

[0044] The following takes one stripe as an example for illustration. In one example, assume k=If it is 13, there are a total of 10 data disks. 10 data disks and 1 global parity disk form a global parity group (which can also be called a global RAID5 group). The first data disk part includes , and the second data disk part includes .

[0045] The 5 data disks in the first data disk part, the first local parity disk (denoted as ), and the global parity disk (denoted as ) form the first local parity group (which can also be called the first local RAID5 group); The 5 data disks in the second data disk part, the second local parity disk (denoted as ), and the global parity disk (denoted as ) form the second local parity group (which can also be called the second local RAID5 group).

[0046] In another example, assume k= is 15. There are a total of 12 data disks. 10 data disks and 1 global parity disk form a global parity group (i.e., a global RAID5 group). The first data disk part includes , and the second data disk part includes .

[0047] Correspondingly, the 6 data disks in the first data disk part, the first local parity disk (denoted as ), and the global parity disk (denoted as ) form the first local parity group (i.e., the first local RAID5 group); The 6 data disks in the second data disk part, the second local parity disk (denoted as ), and the global parity disk (denoted as ) form the second local parity group (i.e., the second local RAID5 group).

[0048] It should be noted that the above only provides an example of the storage system when k= is 13, k= is 15. In actual application, the value of k is not limited.

[0049] In some embodiments, determining the recovery method corresponding to the faulty disk includes: Determining the faulty disk information in the faulty disk recovery request. The faulty disk information includes at least one of the following: the number of faulty disks, the type of faulty disks, and the parity group where the faulty disks are located; According to the faulty disk information, determining the recovery method, where the recovery method is a method for recovering the faulty disk data using the target encoding and decoding rules; The target encoding and decoding rules include at least one of the following: The first local encoding and decoding rule corresponding to the first local check group, where the first local encoding and decoding rule represents that the result of performing an exclusive OR operation after multiplying each disk in the first local check group by the position information of the disk is 0; The second local encoding and decoding rule corresponding to the second local check group, where the second local encoding and decoding rule represents that the result of performing an exclusive OR operation after multiplying each disk in the second local check group by the position information of the disk is 0; The first global encoding and decoding rule corresponding to the global check group, where the first global encoding and decoding rule represents that the result of performing an exclusive OR operation among the disks in the first local check group is 0; The first data disk part and the second data disk part have the same number of data disks.

[0050] Here, when a faulty disk is detected, the management unit of the storage system can be triggered to perform data recovery. That is, the management unit determines the corresponding recovery method according to the faulty disk recovery request, and the faulty disk recovery request carries the faulty disk information, and the specific recovery method can be determined based on the faulty disk information.

[0051] Among them, a faulty disk refers to a physical disk in the storage system that has a fault or anomaly, resulting in the inability to read or write data normally on its stored data disks and / or check disks, or data loss. Common faulty disk situations include hard disk damage, disk failure, disk unrecognizable or unable to be accessed normally, etc.

[0052] In a storage system, such as in RAID storage technology, data reliability can be ensured through redundant checks. That is, if a physical disk has a problem resulting in a faulty disk, the storage system can use the recovery method to perform data recovery on the faulty disk to obtain a data recovery result.

[0053] Here, each local check group and global check group corresponds to an encoding and decoding rule, so that when a faulty disk occurs, data can be recovered based on the encoding and decoding rule.

[0054] The data processing method can be a data processing method based on the RAID55 algorithm. The RAID55 algorithm uses striping with parity. Data disks are distributed and stored among multiple physical disks, and corresponding check disks (including global check disks and local check disks) are allocated for each data stripe, and each check disk is calculated through an exclusive OR operation (XOR).

[0055] The encoding and decoding rule can be used for encoding during data storage and decoding during data recovery.

[0056] Data Encoding: When data is written to a RAID 5 array, the data disks are calculated and stored, and a local parity disk and a global parity disk are generated for the data disks of each stripe. The encoding process generates parity information through exclusive-or (XOR) operations to ensure that data can be recovered through the parity disks even if some disks are damaged.

[0057] Data Decoding: When reading data, if a disk is damaged (i.e., a wrong disk appears), the storage system can use the remaining data disks and the corresponding parity disks to perform decoding using the XOR algorithm to recover the lost data. The XOR operation is reversible, so the original data block of the wrong disk can be recovered based on other data disks and parity disks on the stripe.

[0058] Specifically, the first global encoding and decoding rule corresponding to the global parity group indicates that the result of the exclusive-or operation among all disks (including data disks and the global parity disk p ) is 0, expressed as formula (1): (1) Where, are the data disks, n is the total number of data disks, represents the exclusive-or operation; The first local encoding and decoding rule corresponding to the first local parity group indicates that the result of the exclusive-or operation after multiplying each disk (including data disks, the global parity disk , the first local parity disk ) in the first local parity group by its position information is 0; expressed as formula (2): (2) Where, is the first local parity disk 's position information.

[0059] The second local encoding and decoding rule corresponding to the second local parity group indicates that the result of the exclusive-or operation after multiplying each disk (including data disks, the global parity disk , the second local parity disk ) in the second local parity group by its position information is 0; expressed as formula (3): (3) Where, is the second local parity disk 's position information.

[0060] In the above formulas (2) and (3), is the position information of the data disk , and the position information may include the disk's landing position in the specific physical disk where the disk is located.

[0061] Combine Figure 3 , taking the above k= 13 as an example, if n is 10, the following encoding and decoding rules can be obtained: The first global encoding and decoding rule, related to each data disk and the global parity disk, is expressed as: (4) The first local encoding and decoding rule is related to each data disk, a local parity disk and the global parity disk, and is expressed as: (5) The second local encoding and decoding rule is related to each data disk, another local parity disk and the global parity disk, and is expressed as: (6) In some embodiments, determining the recovery method according to the disk error information includes: If the number of disk errors is 1, determine the encoding and decoding rule corresponding to the parity group where the disk error is located as the target encoding and decoding rule; If the number of disk errors is 2 or 3, determine the target encoding and decoding rule according to the disk error type and the parity group where the disk error is located.

[0062] Here, the recovery method is a method for recovering the data of the disk error using the target encoding and decoding rule. The number of disk errors can be one or more, and the target encoding and decoding rule can also include one or more, that is, the recovery method can be to recover the data of one or more disk errors using one or more target encoding and decoding rules.

[0063] If the number of disk errors is 1, that is, no matter which disk is in error, the local encoding and decoding rule can be used for data recovery. For example, if the disk in error is a data disk, it can be decoded using the local encoding and decoding rule corresponding to its local parity group.

[0064] Taking the above k= 13 as an example, when a disk in error is one of the , it can be decoded using the above formula (5); when a disk in error is one of the

[0065] such as Figure 4 shown, a schematic diagram of a single disk in error is provided, Figure 4 and each row in it corresponds to a disk error situation.

[0066] Here, if the number of misaligned disks is 2 or 3, the target encoding and decoding rules can be determined according to the type of misaligned disks and the parity group where the misaligned disks are located. It should be noted that the parity group where the misaligned disks are located is actually the specific parity group in the stripe where the misaligned disks are located. Therefore, it is not difficult to understand that the misaligned disk information can also include the stripe where the misaligned disks are located, etc.

[0067] In some embodiments, if the number of misaligned disks is 2, determining the target encoding and decoding rules according to the type of misaligned disks and the parity group where the misaligned disks are located includes: If the misaligned disks meet the first condition, select the encoding and decoding rules of the first combination as the target encoding and decoding rules; Among them, the first condition is that the two misaligned disks include a global parity disk and a first disk, and the first disk is any data disk or a local parity disk; The encoding and decoding rules of the first combination include the local encoding and decoding rules that do not include the first disk and the local encoding and decoding rules that include the first disk.

[0068] Specifically, if the number of misaligned disks is 2, there are the following multiple situations. Among them, the first situation is that when one of the two misaligned disks (denoted as ) is a global parity disk , and the other is any disk (which can be a data disk or a local parity disk ), as shown in Figure 5 , each row corresponds to an example of two misaligned disks. In this case, the decoding can be performed according to the encoding and decoding rules of the first combination in the following two steps: The first step: Use the local encoding and decoding rules that do not include to first recover , and the second step: Use the local encoding and decoding rules that include to recover .

[0069] In some embodiments, if the number of misaligned disks is 2, determining the target encoding and decoding rules according to the type of misaligned disks and the parity group where the misaligned disks are located includes: If the misaligned disks meet the second condition, select the encoding and decoding rules of the second combination as the target encoding and decoding rules; Among them, the second condition is that the two misaligned disks belong to different local parity groups, and the two misaligned disks are data disks or local parity disks; The encoding and decoding rules of the second combination include the local encoding and decoding rules corresponding to each local parity group.

[0070] Specifically, if the number of misaligned disks is 2, there are the following multiple situations. Among them, the second situation is that when there is no global parity disk among the two misaligned disks , but , but Belong to two different local check groups, such as Figure 6 shown, each row corresponds to an example of a disk error. In this case, according to the encoding and decoding rules of the second combination, that is, two local encoding and decoding rules can be decoded simultaneously to restore the data.

[0071] In some embodiments, if the number of disk errors is 2, determining the target encoding and decoding rules according to the disk error type and the check group where the disk error is located includes: If the disk error meets the third condition, select the encoding and decoding rules of the third combination as the target encoding and decoding rules; Among them, the third condition is: the two disk errors include any data disk and local check disk belonging to the same local check group; The encoding and decoding rules of the third combination include: the first global encoding and decoding rule, and the local encoding and decoding rules corresponding to the any data disk and local check disk.

[0072] Specifically, if the number of disk errors is 2, there are the following multiple situations. Among them, the third situation is: when the two wrong disks do not have a global check disk , but the two wrong disks belong to the same local check group, but one is a data disk and one is the local check disk of the same group , such as Figure 7 shown. In this case, the data can be decoded according to the encoding and decoding rules of the third combination in the following two steps: The first step: use the first global encoding and decoding rule to restore the data disk , and the second step: solve and restore the local check disk through the local encoding and decoding rule including .

[0073] In some embodiments, if the number of disk errors is 2, determining the target encoding and decoding rules according to the disk error type and the check group where the disk error is located includes: If the disk error meets the fourth condition, select the encoding and decoding rules of the fourth combination as the target encoding and decoding rules; Among them, the fourth condition is: the two disk errors include two data disks belonging to the same local check group; The encoding and decoding rules of the fourth combination include: the first target decoding rule for the two data disks, and the first target decoding rule is generated based on the second global encoding and decoding rule, and the second global encoding and decoding rule is generated based on the merger of the first local encoding and decoding rule, the second local encoding and decoding rule, and the first global encoding and decoding rule.

[0074] Specifically, if the number of misaligned disks is 2, there are multiple cases as follows. The fourth case is: when the two misaligned disks do not include a global parity disk , and the two misaligned disks belong to the same local parity group and are both data disks. As shown in Figure 8 , in this case, the data is recovered according to the first local encoding and decoding rule, the second local encoding and decoding rule, and the first global encoding and decoding rule. Specifically, first, the second global encoding and decoding rule (denoted as global RAID6) is obtained by combining the first local encoding and decoding rule, the second local encoding and decoding rule, and the first global encoding and decoding rule. The second global encoding and decoding rule includes: The result of performing an exclusive OR operation between each disk (including data disks and global parity disks) is 0, expressed as formula (1): (1) The result of performing an exclusive OR operation after multiplying each disk (including data disks, the first local parity disk, and the second local parity disk) by its position information is 0; expressed as formula (7): (7) Based on formulas (1) and (7), the first target decoding rule for recovering the data disk can be obtained as follows:

[0075]

[0076] Taking the example of k= 13 above, the encoding and decoding rules are combined to construct the second global encoding and decoding rule as follows:

[0077] According to the above (8) and (9), by eliminating the data disk , we can solve :

[0078] We get:

[0079] Similarly, we can solve , by eliminating the data disk , we can solve :

[0080] In this way, according to the second global encoding and decoding rule, the data disks can be recovered respectively.

[0081] In some embodiments, if the number of misaligned disks is 3, determining a target encoding and decoding rule according to the misaligned disk type and the check group where the misaligned disk is located includes: If the misaligned disk meets the fifth condition, select the encoding and decoding rule of the fifth combination as the target encoding and decoding rule; Among them, the fifth condition is: the three misaligned disks include two second disks belonging to the same local check group, and a third disk belonging to another local check group; the second disk is a data disk, and the third disk is any disk; The encoding and decoding rule of the fifth combination includes: the local encoding and decoding rule corresponding to the third disk and the second target decoding rule for the two second disks, and the second target decoding rule is generated based on the third global encoding and decoding rule, and the third global encoding and decoding rule is generated by merging the local encoding and decoding rule corresponding to the second disk and the first global encoding and decoding rule.

[0082] In some embodiments, if the number of misaligned disks is 3, determining a target encoding and decoding rule according to the misaligned disk type and the check group where the misaligned disk is located includes: If the misaligned disk meets the sixth condition, select the encoding and decoding rule of the sixth combination as the target encoding and decoding rule; Among them, the sixth condition is: the three misaligned disks include a data disk and a local check disk belonging to the same local check group, and a fourth disk belonging to another local check group; the fourth disk is any disk; The encoding and decoding rule of the sixth combination includes: the local encoding and decoding rule corresponding to the fourth disk, the first global encoding and decoding rule, and the local encoding and decoding rules corresponding to the data disk and the local check disk.

[0083] Specifically, if the number of misaligned disks is 3, there are the following multiple situations. Among them, the first situation is: when the three misaligned disks are three disks in the same local check group (including data disks and / or local check disks , but not including global check disks ), at this time, it cannot be decoded. For example Figure 9 and Figure 10 the situations shown.

[0084] The second case is: the error pattern is 2 + 1, that is, when there are two error disks among the three error disks that are any disks in the same local parity group (which can be data disks and / or local parity disks, that is, the two second disks can be one data disk and the other local parity disk, or both are data disks), and the other error disk can be any disk in another local parity group, that is, the third disk can be a data disk, or a local parity disk, or a global parity disk. For the error disk types in the second case, it can be further divided into two sub-cases. For ease of understanding, as shown in Table 3 and Table 4, a correspondence table of error patterns and corresponding recovery methods is provided.

[0085] Table 3

[0086] Table 4

[0087] Error pattern A (equivalent to the error disks satisfying the fifth condition): Two error disks in the same local parity group are two data disks, and any disk in the second local parity group (which can be a data disk, a global parity disk, or a local parity disk). The corresponding recovery method adopts Method 1, including: The first step: Use the local encoding and decoding rule of the local parity group containing only one error disk (equivalent to a local encoding and decoding rule corresponding to a certain third disk) to recover the error disk. At this time, there are two remaining error disks; The second step: Determine the third global encoding and decoding rule, and perform decoding according to the third global encoding and decoding rule to recover the remaining two error disks, where the third global encoding and decoding rule is constructed based on another local encoding and decoding rule and the first global encoding and decoding rule.

[0088] In an example, taking the above k = 13 example as an example, assume the error disks include: in the first local parity group , and any data disk in the second local parity group (which can be a data disk, a global parity disk, or a local parity disk), as Figure 11 shown, the decoding method includes: The first step: Use the second local encoding and decoding rule (such as the above formula (6)) to solve (which can be a data disk, a global parity disk, or a local parity disk); The second step: Construct a global RAID6 (i.e., the third global encoding and decoding rule) according to the first local encoding and decoding rule (such as the above formula (5)) and the first global encoding and decoding rule (such as the above formula (4)), and perform decoding according to the third global encoding and decoding rule and , to obtain the second target decoding rule, as the following formula:

[0089] Through it can be solved :

[0090] Through it can be solved :

[0091] It is not difficult to understand that the above is only k = 13, the wrong disk includes and For other wrong disk situations, according to formulas (1) and (2), or formulas (1) and (3), the recovery formulas for the remaining two wrong disks can also be solved. For example, if two wrong disks in the same local parity group are two data disks, denoted as and , belonging to the first local parity group, then we get: ; .

[0092] If it belongs to the second local parity group, the corresponding formula can also be deduced, which will not be elaborated here.

[0093] Error mode B (equivalent to the wrong disk satisfying the sixth condition): Two wrong disks in the same local parity group are one data disk and one local parity disk , and any disk (which can be a data disk, a global parity disk, or a local parity disk) in another local parity group, as shown in Figure 12 . The corresponding recovery method adopts method two, including: The first step: Decode this wrong disk through the local encoding and decoding rule for one wrong disk (equivalent to a local encoding and decoding rule corresponding to a fourth disk). At this time, there are only two wrong disks left; The second step: Use the first global encoding and decoding rule to decode the data disk. At this time, there is only one wrong disk of the local parity disk left; The third step: Use another local encoding and decoding rule to decode the wrong disk of the local parity disk .

[0094] In an example, taking the above k = 13 example as an example, assume the wrong disks include: in the first local parity group, and any disk of the data disk (which can be a data disk, a global parity disk, or a local parity disk) in the second local parity group: The first step: Solve using the second local encoding and decoding rule (such as the above formula (6)) (which can be a data disk, a global check disk, or a local check disk); Step 2: Decode using the first global encoding and decoding rule (as shown in Equation (4) above); ; Step 3: Decode the faulty disk of the local check disk using the first local encoding and decoding rule (as shown in Equation (5) above). The faulty disk.

[0095] In this way, the three faulty disks can be recovered separately.

[0096] In some embodiments, if the number of faulty disks is 3, determining the target encoding and decoding rule according to the type of the faulty disk and the check group where the faulty disk is located includes: If the faulty disk meets the seventh condition, select the encoding and decoding rule of the seventh combination as the target encoding and decoding rule; Among them, the seventh condition is: the faulty disk includes a global check disk, and the fifth disk and the sixth disk from two local check groups respectively, the fifth disk is a local check disk or a data disk, and the sixth disk is a local check disk or a data disk; The encoding and decoding rule of the seventh combination includes: the third target decoding rule for the fifth disk, the fourth target decoding rule for the sixth disk, and the first global encoding and decoding rule, the first local encoding and decoding rule or the second local encoding and decoding rule; The third target decoding rule and the fourth target decoding rule are generated based on the first global encoding and decoding rule, the first local encoding and decoding rule and the second local encoding and decoding rule.

[0097] Specifically, if the number of faulty disks is 3, there are the following multiple situations. Among them, the third situation is: when one of the three faulty disks is a global check disk , and the other two disks are data disks or local check disks from two local check groups , that is, the error pattern is 1 + 1 + 1. For the faulty disk type in the third situation, it can be further divided into multiple sub - situations. For ease of understanding, as shown in Table 5 and Table 6, a correspondence table of error patterns and corresponding recovery methods is provided.

[0098] Table 5

[0099] Table 6

[0100] Error mode C1: The faulty disks include a local check disk in one local check group, a local check disk in another local check group, and a global check disk , as Figure 13 shown; The recovery method 3.1 includes: Step 1: Decode and restore the global check disk using the first global encoding and decoding rule , leaving only the faulty disks of two local check disks at this time; Step 2: Decode the faulty disks of the two local check disks simultaneously using two local encoding and decoding rules of the faulty disks.

[0101] Error mode C2: The faulty disks include a data disk in one local check group, a local check disk in another local check group, and the global check disk , as Figure 14 shown; The recovery method 3.2 includes: Derive the third target decoding rule for the data disk and the fourth target decoding rule for the local check disk according to the first global encoding and decoding rule, the first local encoding and decoding rule, and the second local encoding and decoding rule to restore the data disk and the local check disk respectively; Then restore the global check disk through the first global encoding and decoding rule.

[0102] In an example, taking the above k = 13 example as an example, assume that the faulty disks include: the data disk in the first local check group , and the local check disk in the second local check group , and the global check disk , as Figure 14 shown, the recovery method includes: Step 1, according to the first global encoding and decoding rule (as shown in the above formula (4)), the first local encoding and decoding rule (as shown in the above formula (5)), and the second local encoding and decoding rule (as shown in the above formula (6)), derive for the data disk and the local check disk the decoding rules, as follows: Solvable :

[0103] Obtain the third target decoding rule:

[0104] And, Solvable :

[0105] Obtain the fourth target decoding rule: .

[0106] It is not difficult to understand that the above is only k = 13, the faulty disks include and For the example of , and the partial parity disk in the second partial parity group is denoted as , then we get: ; .

[0107] If the data disk belongs to the second partial parity group and the partial parity disk belongs to the first partial parity group, the corresponding formula can also be deduced, which will not be elaborated here.

[0108] Step 2: Decode and recover the global parity disk through the first global encoding and decoding rule (such as formula (4) above), the first local encoding and decoding rule (such as formula (5) above), or the second local encoding and decoding rule (such as formula (6) above).

[0109] Error mode C3: The misaligned disks include the partial parity disk in one partial parity group , the data disk in another partial parity group, and the global parity disk , as shown in Figure 15 . The recovery method 3.3 is the same as the above recovery method 3.2 and will not be elaborated here.

[0110] Error mode C4: The misaligned disks include the data disk in one partial parity group, the data disk in another partial parity group, and the global parity disk , as shown in Figure 16 . The decoding method 3.4 includes: Deduce the third target decoding rule for the data disk and the fourth target decoding rule for the other data disk according to the first global encoding and decoding rule, the first local encoding and decoding rule, and the second local encoding and decoding rule to recover the two data disks respectively; Then recover the global parity disk through any local encoding and decoding rule.

[0111] In an example, taking the example of k = 13 above, assume the misaligned disks include: the data disk in the first partial parity group, and the data disk in the second partial parity group, and the global parity disk , as shown in Figure 16 . The recovery method 3.4 includes: Step 1: Deduce the third target decoding rule for the data disk and The decoding rules are as follows: 1. By eliminating , and can be solved:

[0112] 2. can be solved: :

[0113] 3. can be solved: :

[0114] It is not difficult to understand that the above is only k = 13. The wrong disk includes and as examples. For other wrong disk situations, according to formula (1) and (2), or formula (1) and (3), the recovery formulas for the remaining two wrong disks can also be solved. For example, if two wrong disks in different local check groups are two data disks, denoted as and respectively, then: ; .

[0115] In the second step, the global check disk is decoded and recovered through the first global encoding and decoding rule (such as formula (4) above), the first local encoding and decoding rule (such as formula (5) above), or the second local encoding and decoding rule (such as formula (6) above).

[0116] As described above, the method provided by the embodiments of the present disclosure can recover any one and two wrong disks and can recover some cases of three wrong disks (unsolvable only when all three wrong disks are in one local check group), and has stronger fault tolerance.

[0117] In some embodiments, the method further includes: Performing data disk dropping on the data disks, global check disk, first local check disk, and / or second local check disk of each stripe in a target disk dropping manner: The target disk dropping manner includes one of the following: left rotation alignment, right rotation alignment, left rotation misalignment.

[0118] In some embodiments, if the left rotation misalignment disk dropping manner is adopted, the performing data disk dropping on the data disks, global check disk, first local check disk, and / or second local check disk of each stripe includes: For the 1st to Nth stripes, taking the 1st to N-2nd physical disks, the first local parity physical disk, and the global parity physical disk as a group, they are placed on the disks in a non-aligned left-rotation manner; and, taking the N-1st to 2N-4th physical disks and the second local parity physical disk as a group, they are placed on the disks in a non-aligned left-rotation manner; For the (N+1)th to 2Nth stripes, taking the 1st to N-2nd physical disks and the first local parity physical disk as a group, they are placed on the disks in a non-aligned left-rotation manner; and, taking the global parity physical disk, the N-1st to 2N-4th physical disks, and the second local parity physical disk as a group, they are placed on the disks in a non-aligned left-rotation manner; where N = ( k +1) / 2, and the k represents the total number of disks in the storage system.

[0119] For example, as in the above example k =13, then there are a total of 2N = 13+1 stripes here. For the 1st to 7th stripes (denoted as t1, t2, t3, t4, t5, t6, t7), taking physical disks d1 - d5 and physical l0 disk and physical P disk as a group, they are placed on the disks in a non-aligned left-rotation manner; taking physical disks d6 - d10 and physical l1 disk as a group, they are placed on the disks in a non-aligned left-rotation manner; as shown in Table 7: Table 7

[0120] For the 8th to 14th stripes (denoted as t8, t9, t10, t11, t12, t13, t14), physical disks d1 - d5 and physical l0 disk as a group, they are placed on the disks in a non-aligned left-rotation manner; physical P disk, physical disks d6 - d10 and physical l1 disk as a group, they are placed on the disks in a non-aligned left-rotation manner; as shown in Table 8.

[0121] Table 8

[0122] The overall non-aligned left-rotation disk placement method is shown in Table 9.

[0123] Table 9

[0124] In the above Tables 7, 8, and 9, physical d1... physical d10, physical l0 disk, physical l1The disk and the physical P disk represent each physical disk, t1, t2... t14 represent each stripe, d1, d2... d10 in each stripe represent data disks, l0, l1 represent local parity disks, and p represents the global parity disk.

[0125] The above example is described based on one stripe. In actual production, considering load balancing, common data is written to disks in a left-rotated misaligned manner. Using the above left-rotated misaligned method provided by the embodiments of the present disclosure to write to disks is applicable to the encoding and decoding rules provided by the embodiments of the present disclosure, and can recover any one and two faulty disks, and can also solve the case of partially three faulty disks. Only the following two cases of three faulty disks are not recovered: The first case: All three faulty disks are in one local parity group (including physical data disks and physical l disks); The second case: One of the three faulty disks is a physical disk, and the other two are in one local parity group (including physical data disks and physical l disks).

[0126] Figure 17 It is a schematic structural diagram of a data processing device provided by an embodiment of the present disclosure; as Figure 17 shown, the device includes: A first processing module, configured to determine a recovery method corresponding to a faulty disk in response to a faulty disk recovery request of a storage system; the number of faulty disks is at least one; A second processing module, configured to perform data recovery on the faulty disk using the recovery method to obtain a data recovery result; Wherein, each stripe in the storage system is divided into a first local parity group, a second local parity group, and a global parity group; The global parity group includes a global parity disk, a first data disk part, and a second data disk part; the first local parity group includes a first local parity disk, a first data disk part, and a global parity disk; the second local parity group includes a second local parity disk, a second data disk part, and a global parity disk.

[0127] In some embodiments, the first processing module is configured to determine the faulty disk information in the faulty disk recovery request, and the faulty disk information includes at least one of the following: the number of faulty disks, the type of faulty disks, and the parity group where the faulty disks are located; According to the faulty disk information, determine the recovery method, and the recovery method is a method for recovering faulty disk data using a target encoding and decoding rule; The target encoding and decoding rule includes at least one of the following: The first local encoding and decoding rule corresponding to the first local parity group, and the first local encoding and decoding rule represents that the result of performing an exclusive OR operation after multiplying each disk in the first local parity group by the position information of the disk is 0; The second local encoding and decoding rule corresponding to the second local check group, where the second local encoding and decoding rule represents that the result of performing an exclusive OR operation after multiplying each disk in the second local check group by the position information of the disk is 0; The first global encoding and decoding rule corresponding to the global check group, where the first global encoding and decoding rule represents that the result of performing an exclusive OR operation among the disks in the first local check group is 0; The first data disk part and the second data disk part have the same number of data disks.

[0128] In some embodiments, the first processing module is configured to, if the number of faulty disks is 1, determine the encoding and decoding rule corresponding to the check group where the faulty disk is located as the target encoding and decoding rule; If the number of faulty disks is 2 or 3, determine the target encoding and decoding rule according to the type of the faulty disks and the check group where the faulty disks are located.

[0129] In some embodiments, if the number of faulty disks is 2, the first processing module is configured to, if the faulty disks meet the first condition, select the encoding and decoding rule of the first combination as the target encoding and decoding rule; Wherein, the first condition is that the two faulty disks include a global check disk and a first disk, and the first disk is any data disk or local check disk; The encoding and decoding rule of the first combination includes: the local encoding and decoding rule that does not include the first disk, and the local encoding and decoding rule that includes the first disk.

[0130] In some embodiments, if the number of faulty disks is 2, the first processing module is configured to, if the faulty disks meet the second condition, select the encoding and decoding rule of the second combination as the target encoding and decoding rule; Wherein, the second condition is that the two faulty disks belong to different local check groups, and the two faulty disks are data disks or local check disks; The encoding and decoding rule of the second combination includes: the local encoding and decoding rules respectively corresponding to each local check group.

[0131] In some embodiments, if the number of faulty disks is 2, the first processing module is configured to, if the faulty disks meet the third condition, select the encoding and decoding rule of the third combination as the target encoding and decoding rule; Wherein, the third condition is that the two faulty disks include any data disk and local check disk belonging to the same local check group; The encoding and decoding rule of the third combination includes: the first global encoding and decoding rule, and the local encoding and decoding rules corresponding to the any data disk and local check disk.

[0132] In some embodiments, if the number of misaligned disks is 2, the first processing module is configured to, if the misaligned disks meet the fourth condition, select the encoding and decoding rules of the fourth combination as the target encoding and decoding rules; Wherein, the fourth condition is that the two misaligned disks include two data disks belonging to the same local check group; The encoding and decoding rules of the fourth combination include: a first target decoding rule for the two data disks, the first target decoding rule being generated based on a second global encoding and decoding rule, and the second global encoding and decoding rule being generated based on the merger of the first local encoding and decoding rule, the second local encoding and decoding rule, and the first global encoding and decoding rule.

[0133] In some embodiments, if the number of misaligned disks is 3, the first processing module is configured to, if the misaligned disks meet the fifth condition, select the encoding and decoding rules of the fifth combination as the target encoding and decoding rules; Wherein, the fifth condition is that the three misaligned disks include two second disks belonging to the same local check group and a third disk belonging to another local check group; the second disk is a data disk and the third disk is any disk; The encoding and decoding rules of the fifth combination include: the local encoding and decoding rule corresponding to the third disk and a second target decoding rule for the two second disks, the second target decoding rule being generated based on a third global encoding and decoding rule, and the third global encoding and decoding rule being generated according to the merger of the local encoding and decoding rule corresponding to the second disk and the first global encoding and decoding rule.

[0134] In some embodiments, if the number of misaligned disks is 3, the first processing module is configured to, if the misaligned disks meet the sixth condition, select the encoding and decoding rules of the sixth combination as the target encoding and decoding rules; Wherein, the sixth condition is that the three misaligned disks include a data disk and a local check disk belonging to the same local check group and a fourth disk belonging to another local check group; the fourth disk is any disk; The encoding and decoding rules of the sixth combination include: the local encoding and decoding rule corresponding to the fourth disk, the first global encoding and decoding rule, and the local encoding and decoding rules corresponding to the data disk and the local check disk.

[0135] In some embodiments, if the number of misaligned disks is 3, the first processing module is configured to, if the misaligned disks meet the seventh condition, select the encoding and decoding rules of the seventh combination as the target encoding and decoding rules; Wherein, the seventh condition is that the three misaligned disks include a global check disk and a fifth disk and a sixth disk respectively from two local check groups, the fifth disk being a local check disk or a data disk, and the sixth disk being a local check disk or a data disk; The encoding and decoding rules of the seventh combination include: the third target decoding rule for the fifth disk, the fourth target decoding rule for the sixth disk, and the first global encoding and decoding rule, the first local encoding and decoding rule, or the second local encoding and decoding rule; The third target decoding rule and the fourth target decoding rule are generated based on the first global encoding and decoding rule, the first local encoding and decoding rule, and the second local encoding and decoding rule.

[0136] In some embodiments, the apparatus further includes: a disk dropping module, configured to perform data dropping on the data disks, global parity disks, first local parity disks, and / or second local parity disks of each stripe in a target disk dropping manner; The target disk dropping manner includes one of the following: left rotation alignment, right rotation alignment, and left rotation misalignment.

[0137] In some embodiments, if the left rotation misalignment disk dropping manner is adopted, the disk dropping module is configured to, for the first to the Nth stripes, group the first to the N - 2nd physical disks, the first local parity physical disk, and the global parity physical disk as a group, and perform disk dropping in a left rotation misalignment manner; and group the (N - 1)th to the (2N - 4)th physical disks and the second local parity physical disk as a group, and perform disk dropping in a left rotation misalignment manner; For the (N + 1)th to the 2Nth stripes, group the first to the N - 2nd physical disks and the first local parity physical disk as a group, and perform disk dropping in a left rotation misalignment manner; and group the global parity physical disk, the (N - 1)th to the (2N - 4)th physical disks, and the second local parity physical disk as a group, and perform disk dropping in a left rotation misalignment manner; where N = ( k + 1) / 2, k indicating the total number of disks in the storage system.

[0138] It can be understood that when the data processing apparatus provided in the above embodiments implements the corresponding data processing method, the above processing can be allocated to different program modules as needed to complete all or part of the above-described processing. In addition, the apparatus provided in the above embodiments and the embodiments of the corresponding method belong to the same concept, and the specific implementation process is detailed in the method embodiments and will not be elaborated here.

[0139] An embodiment of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method.

[0140] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, cause the processor to execute the data processing method provided by the embodiment of the present application.

[0141] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0142] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0143] As an example, the executable instructions may or may not correspond to a file in the file system, and may be stored as part of a file that stores other programs or data. For example, they may be stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program in question, or stored in multiple cooperating files (e.g., files storing one or more modules, subroutines, or code portions).

[0144] As an example, the executable instructions may be deployed to execute on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0145] Figure 18 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure; as Figure 18 shown, the electronic device 180 includes: a processor 1801 and a memory 1802 communicatively connected to the processor 1801; the memory 1802 stores instructions executable by the processor 1801. The instructions are executed by the processor 1801 to enable the processor 1801 to execute: In response to a disk error recovery request of a storage system, determine a recovery method corresponding to the disk error; the number of disks in error is at least one; Use the recovery method to perform data recovery on the disk in error to obtain a data recovery result; wherein, each stripe in the storage system is divided into a first local parity group, a second local parity group, and a global parity group; The global check group includes a global check disk, a first data disk part, and a second data disk part; the first local check group includes a first local check disk, a first data disk part, and the global check disk; the second local check group includes a second local check disk, a second data disk part, and the global check disk.

[0146] The electronic device provided in the above embodiment and the embodiment of the corresponding data processing method belong to the same concept. For the specific implementation process, refer to the method embodiment, which will not be elaborated here.

[0147] In practical applications, the electronic device 180 may further include: at least one network interface 1803. Each component in the electronic device 180 is coupled together through a bus system 1804. It can be understood that the bus system 1804 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1804 further includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 18 all kinds of buses are labeled as the bus system 1804. Among them, the number of the processors 1801 can be at least one, and the number of the memories 1802 can be at least one. The network interface 1803 is used for the communication between the electronic device 180 and other devices in a wired or wireless manner.

[0148] The memory 1802 in the embodiments of the present disclosure is used to store various types of data to support the operation of the electronic device 180.

[0149] The method disclosed in the above embodiments of the present disclosure can be applied to the processor 1801 or implemented by the processor 1801. The processor 1801 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 1801 or the instructions in the form of software. The above processor 1801 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1801 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor may be a microprocessor or any conventional processor, etc. Combining the steps of the method disclosed in the embodiments of the present disclosure, it can be directly embodied as being executed and completed by a hardware decoding processor, or by a combination of the hardware and software modules in the decoding processor. The software module may be located in a storage medium, and this storage medium is located in the memory 1802. The processor 1801 reads the information in the memory 1802 and combines its hardware to complete the steps of the foregoing data processing method.

[0150] In some embodiments, the electronic device 180 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontroller units (MCUs), microprocessors, or other electronic components for performing the foregoing methods.

[0151] It should be understood that the various forms of the processes shown above can be reordered, added to, or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. There is no limitation herein.

[0152] In the above description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0153] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the technical field to which this disclosure belongs. The terms used in this disclosure are only for the purpose of describing the embodiments of this disclosure and are not intended to limit this disclosure.

[0154] It should be understood that in the various embodiments of this disclosure, the magnitude of the sequence numbers of the various implementation processes does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this disclosure.

[0155] In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "a plurality" means two or more unless otherwise specifically defined.

[0156] As described above, this is only a specific implementation manner of the present disclosure. However, the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure can easily think of changes or substitutions, which should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims described above.

Claims

1. A data processing method, characterized in that, The method includes: In response to a disk error recovery request of a storage system, determining a recovery method corresponding to the disk error; the number of disks in error is at least one; Using the recovery method to perform data recovery on the disks in error to obtain a data recovery result; Wherein, each stripe in the storage system is divided into a first local parity group, a second local parity group, and a global parity group; The global parity group includes a global parity disk, a first data disk part, and a second data disk part; the first local parity group includes a first local parity disk, a first data disk part, and the global parity disk; the second local parity group includes a second local parity disk, a second data disk part, and the global parity disk.

2. The method according to claim 1, characterized in that The determining the recovery method corresponding to the disk in error includes: Determining the disk error information in the disk error recovery request, the disk error information including at least one of the following: the number of disks in error, the type of disk in error, and the parity group where the disk in error is located; According to the disk error information, determining the recovery method, the recovery method being a method for recovering disk error data using a target encoding and decoding rule; The target encoding and decoding rule includes at least one of the following: The first local encoding and decoding rule corresponding to the first local parity group, the first local encoding and decoding rule representing that the result of performing an exclusive OR operation after multiplying each disk in the first local parity group by the disk position information is 0; The second local encoding and decoding rule corresponding to the second local parity group, the second local encoding and decoding rule representing that the result of performing an exclusive OR operation after multiplying each disk in the second local parity group by the disk position information is 0; The first global encoding and decoding rule corresponding to the global parity group, the first global encoding and decoding rule representing that the result of performing an exclusive OR operation among the disks in the first local parity group is 0; The first data disk part and the second data disk part have the same number of data disks.

3. The method according to claim 2, characterized in that, The determining the recovery method according to the disk error information includes: If the number of disks in error is 1, determining the encoding and decoding rule corresponding to the parity group where the disk in error is located as the target encoding and decoding rule; If the number of disks in error is 2 or 3, determining the target encoding and decoding rule according to the type of disk in error and the parity group where the disk in error is located.

4. The method according to claim 3, wherein If the number of disks in error is 2, determining the target encoding and decoding rule according to the type of disk in error and the parity group where the disk in error is located includes: If the disk in error satisfies a first condition, selecting a first combination of encoding and decoding rules as the target encoding and decoding rule; Wherein, the first condition is: the two disks in error include the global parity disk and a first disk, and the first disk is any data disk or local parity disk; The first combination of encoding and decoding rules includes: the local encoding and decoding rule that does not include the first disk, and the local encoding and decoding rule that includes the first disk.

5. The method according to claim 3, wherein If the number of disks in error is 2, determining the target encoding and decoding rule according to the type of disk in error and the parity group where the disk in error is located includes: If the disk in error satisfies a second condition, selecting a second combination of encoding and decoding rules as the target encoding and decoding rule; Wherein, the second condition is: the two disks in error belong to different local parity groups, and the two disks are data disks or local parity disks; The second combination of encoding and decoding rules includes: the local encoding and decoding rules corresponding to each local parity group respectively.

6. The method according to claim 3, wherein If the number of misaligned disks is 2, determine the target encoding and decoding rules according to the misaligned disk type and the check group where the misaligned disk is located, including: If the misaligned disks meet the third condition, select the encoding and decoding rules of the third combination as the target encoding and decoding rules; Among them, the third condition is that the two misaligned disks include any data disk and local check disk belonging to the same local check group; The encoding and decoding rules of the third combination include: the first global encoding and decoding rule, and the local encoding and decoding rules corresponding to the any data disk and local check disk.

7. The method according to claim 3, characterized in that If the number of misaligned disks is 2, determine the target encoding and decoding rules according to the misaligned disk type and the check group where the misaligned disk is located, including: If the misaligned disks meet the fourth condition, select the encoding and decoding rules of the fourth combination as the target encoding and decoding rules; Among them, the fourth condition is that the two misaligned disks include two data disks belonging to the same local check group; The encoding and decoding rules of the fourth combination include: the first target decoding rule for the two data disks, and the first target decoding rule is generated based on the second global encoding and decoding rule, and the second global encoding and decoding rule is generated after merging the first local encoding and decoding rule, the second local encoding and decoding rule, and the first global encoding and decoding rule.

8. The method according to claim 3, wherein If the number of misaligned disks is 3, determine the target encoding and decoding rules according to the misaligned disk type and the check group where the misaligned disk is located, including: If the misaligned disks meet the fifth condition, select the encoding and decoding rules of the fifth combination as the target encoding and decoding rules; Among them, the fifth condition is that the three misaligned disks include two second disks belonging to the same local check group, and a third disk belonging to another local check group; the second disk is a data disk, and the third disk is any disk; The encoding and decoding rules of the fifth combination include: the local encoding and decoding rule corresponding to the third disk and the second target decoding rule for the two second disks, and the second target decoding rule is generated based on the third global encoding and decoding rule, and the third global encoding and decoding rule is generated after merging the local encoding and decoding rule corresponding to the second disk and the first global encoding and decoding rule.

9. The method according to claim 3, wherein If the number of misaligned disks is 3, determine the target encoding and decoding rules according to the misaligned disk type and the check group where the misaligned disk is located, including: If the misaligned disks meet the sixth condition, select the encoding and decoding rules of the sixth combination as the target encoding and decoding rules; [[ID= ​ 10. The method according to claim 3, wherein ​ ​ Among them, the seventh condition is that the three misaligned disks include the global parity disk, and the fifth disk and the sixth disk respectively from two local parity groups, where the fifth disk is a local parity disk or a data disk, and the sixth disk is a local parity disk or a data disk; The encoding and decoding rules of the seventh combination include: the third target decoding rule for the fifth disk, the fourth target decoding rule for the sixth disk, and the first global encoding and decoding rule, the first local encoding and decoding rule, or the second local encoding and decoding rule; The third target decoding rule and the fourth target decoding rule are generated based on the first global encoding and decoding rule, the first local encoding and decoding rule, and the second local encoding and decoding rule.

11. The method according to claim 1, wherein The method further includes: Using a target disk placement method to perform data disk placement on the data disks, global parity disks, first local parity disks, and / or second local parity disks of each stripe; The target disk placement method includes one of the following: left rotation alignment, right rotation alignment, and left rotation misalignment.

12. The method according to claim 11, wherein If the left rotation misalignment disk placement method is used, the performing data disk placement on the data disks, global parity disks, first local parity disks, and / or second local parity disks of each stripe includes: For the 1st to the Nth stripes, taking the 1st physical disk to the N-2nd physical disk, the first local parity physical disk, and the global parity physical disk as a group, and performing disk placement in the left rotation misalignment manner; and taking the N-1st physical disk to the 2N-4th physical disk and the second local parity physical disk as a group, and performing disk placement in the left rotation misalignment manner; For the (N + 1)th to the 2Nth stripes, taking the 1st physical disk to the N-2nd physical disk and the first local parity physical disk as a group, and performing disk placement in the left rotation misalignment manner; and taking the global parity physical disk, the N-1st physical disk to the 2N-4th physical disk, and the second local parity physical disk as a group, and performing disk placement in the left rotation misalignment manner; Where N = ( k + 1) / 2, k indicating that the storage system has a total number of disks.

13. A data processing device, characterized in that, The device includes: A first processing module, configured to determine a recovery method corresponding to a misaligned disk in response to a misaligned disk recovery request of a storage system; the number of misaligned disks is at least one; A second processing module, configured to perform data recovery on the misaligned disk by using the recovery method to obtain a data recovery result; Among them, each stripe in the storage system is divided into a first local parity group, a second local parity group, and a global parity group; The global parity group includes a global parity disk, a first data disk part, and a second data disk part; the first local parity group includes a first local parity disk, a first data disk part, and a global parity disk; the second local parity group includes a second local parity disk, a second data disk part, and a global parity disk.

14. An electronic device, characterized in that, Includes: At least one processor; And a memory communicatively connected to the at least one processor; where The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 12.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 12.

Images