A data processing method, apparatus, storage medium, and electronic device

Abstract

This disclosure provides a data processing method, apparatus, storage medium, and electronic device, relating to the field of computer technology. The method includes: responding to a disk recovery request from a storage system, determining a recovery method corresponding to the disk that is faulty; the number of faulty disks is at least one; using the recovery method to recover data from the faulty disks, obtaining a data recovery result; each stripe in the storage system is divided into a first local checksum group, a second local checksum group, and a global checksum group; the global checksum group includes a global checksum disk, a first data disk portion, and a second data disk portion; the first local checksum group includes a first local checksum disk, a first data disk portion, and a global checksum disk; the second local checksum group includes a second local checksum disk, a second data disk portion, and a global checksum disk. Thus, a grouping method for the first local checksum group, the second local checksum group, and the global checksum group is provided, and at least one faulty disk can be recovered using the recovery method corresponding to the faulty disk, improving fault tolerance.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a data processing method, apparatus, storage medium and electronic device. Background Technology

[0002] With the rapid development of communication and network technologies, digital information is growing exponentially, posing significant challenges to data storage technology. The reliability of data and the energy consumption of storage systems are receiving increasing attention. Faced with massive data volumes, the reliability of data in a storage system is inversely proportional to the number of components it contains; that is, the more components a storage system has, the lower its reliability. Related surveys show that in an internet data center consisting of 600 disks, approximately 30 disks fail each month. In large-scale storage systems, the decline in data reliability caused by disk failures is a serious problem; therefore, related fault-tolerant technologies have become a key research focus.

[0003] RAID (Redundant Arrays of Independent Disks) is a key technology for increasing storage capacity. A RAID array combines multiple independent disks into a single, high-capacity disk group. Using RAID storage technology can significantly increase storage capacity, improve system I / O processing capabilities, and enhance data reliability through distributed storage, parallel access, and information redundancy.

[0004] RAID technology primarily utilizes data striping, data parity, and mirroring to achieve superior performance, higher reliability, better fault tolerance, and greater scalability. Depending on the specific data application requirements, these three technologies can be used in combination or in different architectures. Based on these strategies and architectures, RAID can be categorized into different levels: RAID 0, RAID 1, RAID 5, RAID 6, RAID 10, etc.

[0005] RAID 0 uses data striping technology and does not provide redundancy or error correction capabilities. It is implemented by concatenating N (N greater than or equal to 2) identical hard drives together in hardware via an intelligent disk controller or in software via disk drivers in the operating system to create a large volume. During use, computer data is written sequentially to each hard drive. Its advantage is that it can double the hard drive capacity; for example, using three 80GB hard drives in RAID 0 mode will result in a total capacity of 240GB. Its disadvantage is that if any one hard drive fails, the entire system will be compromised, resulting in low reliability.

[0006] RAID 1, or RAID mirroring, works by mirroring data from one disk to another. Data is written to one disk while a mirror image is created on a spare disk. This maximizes system reliability and recoverability without impacting performance. As long as at least one disk in any mirror pair is usable, the system can continue operating even if half the hard drives fail. When a hard drive fails, the system ignores it and uses the remaining mirror disk for reading and writing data, providing excellent disk redundancy. The disadvantages are higher cost and a disk utilization rate of only 50%. For example, with four 80GB hard drives, only 160GB of disk space is usable. Furthermore, if a hard drive fails, it must be replaced promptly; otherwise, if the remaining mirror disk also fails, the entire system will crash. Replacing the drive with a new one also requires a lengthy process of resynchronizing the mirror image. Therefore, RAID 1 is primarily used for storing critical and important data.

[0007] RAID 5 is a distributed parity independent disk architecture, where the parity code exists on all disks, and p0 represents the parity value of the 0th stripe. RAID 5 offers high read performance and moderate write performance, with good efficiency for block-based collective access. Because the parity code is distributed across different disks, reliability is improved. However, it struggles with handling data transfer parallelism, and controller design is quite challenging. With RAID 5, most data transfers operate on only one disk, allowing for parallel operations. RAID 5 has a "write penalty," meaning that each write operation generates four actual read / write operations: two reads of old data and parity information, and two writes of new data and parity information.

[0008] RAID 6 is an extension of RAID 5, featuring two distributed parity values ​​stored on separate disks. It's primarily used in scenarios where data integrity is paramount. Due to the introduction of a second parity value, it requires N+2 disks, significantly increasing controller complexity and further enhancing data reliability. However, it demands more space for parity value storage and incurs a performance penalty during write operations.

[0009] RAID technology is widely used in today's distributed storage servers, especially in RAID 5 and RAID 6, which can recover one or two faulty blocks respectively. Through fault tolerance and data recovery technology, it can effectively improve data availability and system stability.

[0010] However, each data recovery is still limited by the speed of reading large amounts of data from each disk, and the fault tolerance still needs to be improved. Summary of the Invention

[0011] This disclosure provides a data processing method, apparatus, storage medium, and electronic device to at least solve the above-mentioned technical problems existing in the prior art.

[0012] The technical solution of this disclosure embodiment is implemented as follows:

[0013] In a first aspect, embodiments of this disclosure provide a data processing method, the method comprising:

[0014] In response to a disk recovery request from the storage system, a recovery method corresponding to the faulty disk is determined; the number of faulty disks is at least one.

[0015] The data recovery method described above is used to recover data from the faulty disk, and the data recovery result is obtained.

[0016] In the storage system, each stripe is divided into a first local check group, a second local check group, and a global check group;

[0017] The global verification group includes a global verification disk, a first data disk portion, and a second data disk portion; the first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk; the second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk.

[0018] Secondly, embodiments of this disclosure provide a data processing apparatus, the apparatus comprising:

[0019] The first processing module is used to respond to a disk recovery request from the storage system and determine the recovery method corresponding to the disk that is faulty; the number of faulty disks is at least one.

[0020] The second processing module is used to perform data recovery on the faulty disk using the recovery method, and obtain the data recovery result;

[0021] In the storage system, each stripe is divided into a first local check group, a second local check group, and a global check group;

[0022] The global verification group includes a global verification disk, a first data disk portion, and a second data disk portion; the first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk; the second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk.

[0023] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the data processing methods described above.

[0024] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the data processing method according to any one of the claims.

[0025] The embodiments disclosed herein have the following beneficial effects:

[0026] The data processing method, apparatus, storage medium, and electronic device provided in this disclosure are applied to a storage system. The method includes: responding to a disk recovery request from the storage system, determining a recovery method corresponding to the disk that is faulty; the number of faulty disks is at least one; and using the recovery method to recover data from the faulty disks to obtain a data recovery result. In the storage system, each stripe is divided into a first local verification group, a second local verification group, and a global verification group. The global verification group includes a global verification disk, a first data disk portion, and a second data disk portion. The first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk. The second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk. Thus, a grouping method for the first local verification group, the second local verification group, and the global verification group is provided. Based on this grouping method, at least one faulty disk can be recovered using the recovery method corresponding to the faulty disk, improving the fault tolerance capability of the storage system.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of a RAID55 grouping provided in an embodiment of the present disclosure;

[0029] Figure 2 A flowchart illustrating a data processing method provided in an embodiment of this disclosure;

[0030] Figure 3 A schematic diagram illustrating a grouping of data disks and verification disks provided in an embodiment of this disclosure;

[0031] Figure 4 A schematic diagram of a faulty disk provided as an application embodiment of this disclosure;

[0032] Figure 5 Illustrations of two misaligned disks provided for application embodiments of this disclosure Figure 1 ;

[0033] Figure 6 Illustrations of two misaligned disks provided for application embodiments of this disclosure Figure 2 ;

[0034] Figure 7 Illustrations of two misaligned disks provided for application embodiments of this disclosure Figure 3 ;

[0035] Figure 8 Illustrations of two misaligned disks provided for application embodiments of this disclosure Figure 4 ;

[0036] Figure 9 Illustrations of three misaligned disks provided for application embodiments of this disclosure Figure 1 ;

[0037] Figure 10 Illustrations of three misaligned disks provided for application embodiments of this disclosure Figure 2 ;

[0038] Figure 11 Illustrations of three misaligned disks provided for application embodiments of this disclosure Figure 3 ;

[0039] Figure 12 Illustrations of three misaligned disks provided for application embodiments of this disclosure Figure 4 ;

[0040] Figure 13 Illustrations of three misaligned disks provided for application embodiments of this disclosure Figure 5 ;

[0041] Figure 14 Illustrations of three misaligned disks provided for application embodiments of this disclosure Figure 6 ;

[0042] Figure 15 Illustrations of three misaligned disks provided for application embodiments of this disclosure Figure 7 ;

[0043] Figure 16 Illustrations of three misaligned disks provided for application embodiments of this disclosure Figure 8 ;

[0044] Figure 17 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure;

[0045] Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0046] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0047] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0048] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0050] Before providing a further detailed description of the embodiments of this disclosure, the nouns and terms involved in the embodiments of this disclosure will be explained, and the nouns and terms involved in the embodiments of this disclosure shall be interpreted as follows.

[0051] RAID 55 is an enhanced version of RAID 5, offering stronger fault tolerance. RAID 55 can recover lost data through parity checking when one or two disks fail. Furthermore, RAID 55 can recover some data when three disks fail, but not in all cases. Here are two situations where RAID 55 cannot recover data:

[0052] The first scenario: When all three failed disks (referred to as faulty disks) are in the same local RAID5 group, data recovery is impossible. This is because all parity information is concentrated among these three disks; losing these disks makes recovery impossible.

[0053] The second scenario: When one of the three faulty disks is a physical disk failure, and the other two are in the same local RAID5 group, recovery is also impossible. This is because some parity information is lost, leading to recovery failure.

[0054] In addition, if RAID55 uses the traditional left-hand misaligned disk disposal method, it can only recover any one or two misaligned disks, and cannot recover any three disks, which also reduces the fault tolerance of the RAID55 algorithm.

[0055] The following provides an example to illustrate RAID55 in related technologies.

[0056] Assuming there is a total There are 1 disk, including 2 partial check disks. , 1 global check disk , With one data disk and RAID 55, a maximum of two faulty disks can be recovered. An example using k=13 will be provided.

[0057] First, when k At time 13, there are a total of 10 data disks and 1 global parity disk. To form a global RAID5, the encoding method is as follows: . This represents the XOR operation. These represent the various data disks (also known as data or data blocks). This refers to the global check disk (also known as the global check block or global check data). The data disk and the global check disk mentioned above are data on the same strip but belong to different physical disks.

[0058] Then, divide the 10 data disks into two groups of 5 data disks each;

[0059] Will A total of 5 data disks and a partial check disk The first local RAID5 group is formed, and the encoding method is as follows: ,in, For data disk Location information, For local verification disk Location information. It changes according to the changes in the stripes.

[0060] Will A total of 5 data disks and a partial check disk The second local RAID5 group is formed, and the encoding method is as follows: ,in, For data disk Location information, For local verification disk Location information.

[0061] In summary, the RAID55 encoding rules are as follows:

[0062]

[0063] like Figure 1 The image shown is a schematic diagram of a RAID55 encoding method, combined with... Figure 1 It can be seen that RAID55 decoding has the following characteristics:

[0064] One faulty disk: RAID55 can recover from any faulty disk;

[0065] Two faulty disks: RAID 55 can recover from any two faulty disks;

[0066] If three disks are faulty, data recovery is impossible in either of the following two scenarios: First, all three faulty disks are in a single local RAID 5 group; second, one of the faulty disks is a global parity disk. The other two are in the same local RAID5 group.

[0067] It should be noted that the above description is based on a single stripe. In actual production, considering load balancing, the data is commonly written to disk in a left-hand non-aligned manner, as shown in Table 1, which provides a schematic diagram of a RAID55 disk writing method. In this diagram, disk1, disk2...disk13 represent each physical disk, and t1, t2...t13 represent each stripe.

[0068] Table 1

[0069]

[0070] The table shows that the purpose of using a left-hand misaligned disk placement method is for load balancing, but this method will cause any three faulty disks in the RAID55 array to fail to recover. As shown in Table 2, when the 2nd, 3rd, and 8th physical disks (disk2, disk3, and disk8) fail, the three faulty disks in stripe 8 (k8) will be unrecoverable. All data is contained in the second local RAID 5 group, making data recovery impossible. In fact, the left-hand misalignment of RAID 55 disks means it can only resolve any two misaligned disks, reducing its fault tolerance. Therefore, a method to improve the fault tolerance of RAID 55 is needed.

[0071] Table 2

[0072]

[0073] Based on this, embodiments of this disclosure provide a data processing method, apparatus, storage medium, and electronic device. The method includes: responding to a disk recovery request from a storage system, determining a recovery method corresponding to the disk that is faulty; the number of faulty disks is at least one; using the recovery method to recover data from the faulty disks, obtaining a data recovery result; wherein, each stripe in the storage system is divided into a first local verification group, a second local verification group, and a global verification group; the global verification group includes a global verification disk, a first data disk portion, and a second data disk portion; the first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk; the second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk. Thus, a grouping method for the first local verification group, the second local verification group, and the global verification group is provided. Based on this grouping method, at least one faulty disk can be recovered using the recovery method corresponding to the faulty disk, improving the fault tolerance capability of the storage system.

[0074] Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of the present disclosure, as shown below. Figure 2 As shown, the method includes:

[0075] Step 201: In response to the storage system's faulty disk recovery request, determine the recovery method corresponding to the faulty disk; the number of faulty disks is at least one.

[0076] Step 202: Use the recovery method to recover data from the faulty disk and obtain the data recovery result;

[0077] In the storage system, each stripe is divided into a first local check group, a second local check group, and a global check group;

[0078] The global verification group includes a global verification disk, a first data disk portion, and a second data disk portion; the first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk; the second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk.

[0079] Here, the method can be applied to a storage system. Each stripe in the storage system may include: a first local parity disk, a second local parity disk, a global parity disk, a first data disk portion, and a second data disk portion; wherein the first data disk portion and the second data disk portion have the same number of data disks.

[0080] A data disk, also known as a data block or data unit, is the actual data stored on a physical disk.

[0081] Local check disks, global check disks, and other check disks, also known as check blocks or check data units, are check data stored on physical disks.

[0082] In storage systems such as RAID, a physical disk is a physical entity within a disk array, referring to a real, existing hard drive. Data disks and parity disks can be understood as logical blocks that are mapped to storage locations on physical disks; that is, each data block and parity block is written to a specific area of ​​a physical disk.

[0083] Striping is a technique that distributes data across multiple disks. Specifically, striping divides data into blocks and stores these blocks sequentially across multiple physical disks. Consequently, when reading data, multiple data blocks can be retrieved from multiple physical disks simultaneously, thereby improving the performance of the storage system.

[0084] For example, the total number of disks in the first local check disk, the second local check disk, the global check disk, the first data disk portion, and the second data disk portion is: k That is, each stripe of the storage system has a total of k The disks include: one first local check disk, one second local check disk, and one global check disk. k - 3 data disks k≥ 5. The first data disk section and the second data disk section each have ( k -3) / 2 data disks.

[0085] The following example uses a strip as an example. In one example, suppose... k= 13, then there are a total of 10 data disks. These 10 data disks and 1 global parity disk form a global parity group (also known as a global RAID5 group). The first data disk part includes... The second data disk includes .

[0086] The first data disk section consists of five data disks and the first partial check disk (denoted as...). ), global check disk (denoted as) This forms the first local parity group (also known as the first local RAID5 group);

[0087] The second data disk section consists of five data disks and a second partial check disk (denoted as...). ), global check disk (denoted as) This forms the second local parity group (also known as the second local RAID5 group).

[0088] In another example, suppose k= 15, then there are a total of 12 data disks. Ten data disks and one global parity disk form a global parity group (i.e., a global RAID 5 group). The first data disk portion includes... The second data disk includes .

[0089] Correspondingly, the six data disks in the first data disk section and the first partial parity disk (denoted as...) ), global check disk (denoted as) This forms the first local parity group (i.e., the first local RAID5 group);

[0090] The second data disk section consists of six data disks and a second partial check disk (denoted as...). ), global check disk (denoted as) This forms the second local parity group (i.e., the second local RAID5 group).

[0091] It should be noted that the above is only one type. k= 13. k= A storage system example at 15 o'clock, for practical applications... k The value of is not limited.

[0092] In some embodiments, determining the recovery method corresponding to the faulty disk includes:

[0093] Determine the faulty disk information in the faulty disk recovery request, wherein the faulty disk information includes at least one of the following: number of faulty disks, type of faulty disks, and verification group to which the faulty disks belong;

[0094] Based on the faulty disk information, the recovery method is determined, which is a method for recovering faulty disk data using target encoding and decoding rules;

[0095] The target encoding / decoding rules include at least one of the following:

[0096] The first local encoding and decoding rule corresponding to the first local verification group, wherein the first local encoding and decoding rule indicates that the result of XOR operation after multiplying each disk in the first local verification group with the disk's position information is 0;

[0097] The second local encoding and decoding rule corresponding to the second local verification group, the second local encoding and decoding rule represents that the result of XOR operation after multiplying each disk in the second local verification group with the disk's position information is 0;

[0098] The first global encoding and decoding rule corresponding to the global verification group, the first global encoding and decoding rule indicates that the result of the XOR operation between each disk in the first local verification group is 0;

[0099] The first data disk portion and the second data disk portion have the same number of data disks.

[0100] Here, when a faulty disk is detected, the storage system's management unit can be triggered to perform data recovery. That is, the management unit determines the corresponding recovery method based on the faulty disk recovery request. The faulty disk recovery request carries faulty disk information, and the specific recovery method to be used can be determined based on the faulty disk information.

[0101] In this context, a disk error refers to a malfunction or anomaly in a storage system where a physical disk fails, preventing the data disk and / or parity disk from reading or writing data, or resulting in data loss. Common disk error scenarios include hard drive damage, disk failure, disk being unrecognized, or inaccessible.

[0102] In storage systems, such as RAID storage technology, redundancy checks can be used to ensure data reliability. That is, if a physical disk malfunctions, resulting in a faulty disk, the storage system can use the aforementioned recovery method to recover the data from the faulty disk and obtain the recovered data.

[0103] Here, each local verification group and global verification group has corresponding encoding and decoding rules, so that data can be recovered based on the encoding and decoding rules when a disk error occurs.

[0104] The data processing method can be a RAID55 algorithm-based method, which uses striping with parity. Data disks are distributed across multiple physical disks, and a corresponding parity disk (including a global parity disk and local parity disks) is assigned to each data stripe. The parity disks are calculated using an XOR operation.

[0105] Encoding and decoding rules can be used to encode data when it is stored, and to decode data when it is recovered.

[0106] Data Encoding: When data is written to a RAID 5 array, the data disks are calculated and stored, and local and global parity disks are generated for each stripe of data disks. The encoding process generates parity information through an XOR operation, ensuring that even if some disks fail, the data can be recovered from the parity disks.

[0107] Data Decoding: When reading data, if the disk is damaged (i.e., a faulty disk occurs), the storage system can use the XOR algorithm to decode and recover the lost data based on the remaining data disks and the corresponding parity disks. The XOR operation is reversible, therefore it can recover the original data blocks of the faulty disk based on other data disks and parity disks on the stripe.

[0108] Specifically, the first global encoding / decoding rule corresponding to the global verification group represents: each disk (including the data disk and the global verification disk) pThe result of an XOR operation between () and () is 0, as expressed in formula (1):

[0109] (1)

[0110] in, For each data disk, n is the total number of data disks. Indicates the XOR operation;

[0111] The first local encoding / decoding rule corresponding to the first local verification group is characterized by: each disk in the first local verification group (including data disk and global verification disk). First partial verification disk The result of multiplying the product with its location information and then performing an XOR operation is 0; this is represented by formula (2):

[0112] (2)

[0113] in, For the first local verification disk Location information.

[0114] The second local encoding / decoding rule representation corresponding to the second local verification group: each disk in the second local verification group (including data disk and global verification disk) Second local verification disk The result of multiplying the product with its location information and then performing an XOR operation is 0; this is represented by formula (3):

[0115] (3)

[0116] in, For the second local verification disk Location information.

[0117] In the above formulas (2) and (3), For data disk The location information may include the physical location of the disk within the disk itself.

[0118] Combine Figure 3 , with the above k= Taking 13 as an example, with n=10, the following encoding and decoding rules can be obtained:

[0119] The first global encoding / decoding rule, which is related to each data disk and the global parity disk, is expressed as follows: (4)

[0120] The first local encoding / decoding rule is to be consistent with each data disk and a local parity disk. Related to the global check disk, it is represented as: (5)

[0121] The second local encoding / decoding rule is to be consistent with each data disk and another local check disk. Related to the global check disk, it is represented as: (6)

[0122] In some embodiments, determining the recovery method based on the faulty disk information includes:

[0123] If the number of faulty disks is 1, determine the encoding / decoding rule corresponding to the verification group to which the faulty disk belongs, and use it as the target encoding / decoding rule;

[0124] If the number of faulty disks is 2 or 3, the target encoding / decoding rule is determined according to the faulty disk type and the verification group to which the faulty disk belongs.

[0125] Here, the recovery method is to recover the data of the faulty disk using the target encoding and decoding rules. The number of faulty disks can be one or more, and the target encoding and decoding rules can also include one or more. That is, the recovery method can be to recover the data of one or more faulty disks using one or more target encoding and decoding rules.

[0126] If the number of faulty disks is 1, meaning that data recovery can be performed using local encoding / decoding rules regardless of which disk is faulty. For example, if the faulty disk is a data disk, it can be decoded using the local encoding / decoding rules corresponding to its local check group.

[0127] Based on the above k= For example, in case 13, when a misaligned disk is... When one of them is true, the above formula (5) can be used for decoding; when one of the faulty disks is true, the above formula (5) can be used for decoding. When one of them is true, the above formula (6) can be used for decoding. When a faulty disk is true... When, formulas (5) and (6) can both be decoded.

[0128] like Figure 4 The diagram illustrates a misaligned disk. Figure 4 Each row in the table corresponds to one type of misalignment.

[0129] Here, if the number of faulty disks is 2 or 3, the target encoding / decoding rule can be determined based on the faulty disk type and the checksum group to which the faulty disk belongs. It should be noted that the checksum group to which the faulty disk belongs is actually the specific checksum group within the stripe to which the faulty disk belongs. Therefore, it is easy to understand that faulty disk information can also include the stripe to which the faulty disk belongs, etc.

[0130] In some embodiments, if the number of faulty disks is 2, the target encoding / decoding rule is determined based on the faulty disk type and the verification group to which the faulty disk belongs, including:

[0131] If the faulty disk meets the first condition, the encoding and decoding rules of the first combination are selected as the target encoding and decoding rules;

[0132] The first condition is: the two faulty disks include a global check disk and a first disk, where the first disk is any data disk or a local check disk;

[0133] The encoding and decoding rules of the first combination include: local encoding and decoding rules that do not include the first disk, and local encoding and decoding rules that include the first disk.

[0134] Specifically, if the number of misplaced disks is 2, there are several possible scenarios, the first of which is: when the two misplaced disks (denoted as ) One of them is a global check disk. The other is any disk. (Can be a data disk or a partial check disk) ),like Figure 5 As shown, each row corresponds to an example of two misaligned disks. In this case, decoding can be performed according to the encoding / decoding rules of the first combination, following these two steps:

[0135] Step 1: Use the one that does not contain The local encoding and decoding rules are restored first. Step 2: Use the included Local encoding / decoding rule recovery .

[0136] In some embodiments, if the number of faulty disks is 2, the target encoding / decoding rule is determined based on the faulty disk type and the verification group to which the faulty disk belongs, including:

[0137] If the faulty disk meets the second condition, the second combination of encoding and decoding rules is selected as the target encoding and decoding rule;

[0138] The second condition is that the two faulty disks belong to different local verification groups, and the two faulty disks are data disks or local verification disks.

[0139] The encoding and decoding rules of the second combination include: the local encoding and decoding rules corresponding to each local check group.

[0140] Specifically, if the number of misaligned disks is 2, there are several possible scenarios, the second of which is: when two disks are misaligned... There is no global check disk in China. ,but They belong to two different local check groups, such as Figure 6 As shown, each row corresponds to an example of a faulty disk. In this case, the data can be recovered by decoding according to the second combination of encoding and decoding rules, that is, by decoding the two local encoding and decoding rules simultaneously.

[0141] In some embodiments, if the number of faulty disks is 2, the target encoding / decoding rule is determined based on the faulty disk type and the verification group to which the faulty disk belongs, including:

[0142] If the faulty disk meets the third condition, the third combination of encoding and decoding rules is selected as the target encoding and decoding rule;

[0143] The third condition is that the two faulty disks include any data disk and a local check disk belonging to the same local check group;

[0144] The third set of encoding and decoding rules includes: a first global encoding and decoding rule, and local encoding and decoding rules corresponding to the arbitrary data disk and the local parity disk.

[0145] Specifically, if the number of misaligned disks is 2, there are several possible scenarios, among which the third scenario is: when two disks are misaligned... There is no global check disk in China. However, the two faulty disks belong to the same local check group, but one is a data disk. One is a local check disk in the same group. ,like Figure 7 As shown, in this case, decoding can be performed according to the following two steps based on the encoding and decoding rules of the third combination:

[0146] Step 1: Restore the data disk using the first global encoding / decoding rule Step 2: By including Local encoding / decoding rule solving to recover the local check disk .

[0147] In some embodiments, if the number of faulty disks is 2, the target encoding / decoding rule is determined based on the faulty disk type and the verification group to which the faulty disk belongs, including:

[0148] If the faulty disk meets the fourth condition, the encoding and decoding rule of the fourth combination is selected as the target encoding and decoding rule;

[0149] The fourth condition is: the two misaligned disks include two data disks belonging to the same local check group;

[0150] The fourth set of encoding and decoding rules includes: a first target decoding rule for two data disks, the first target decoding rule being generated based on a second global encoding and decoding rule, and the second global encoding and decoding rule being generated by merging the first local encoding and decoding rule, the second local encoding and decoding rule, and the first global encoding and decoding rule.

[0151] Specifically, if the number of misaligned disks is 2, there are several possible scenarios, among which the fourth scenario is: when two disks are misaligned... There is no global check disk in China. Two wrong disks They belong to the same local verification group and are all data disks, such as Figure 8 As shown, in this case, data is recovered according to the first local encoding / decoding rule, the second local encoding / decoding rule, and the first global encoding / decoding rule. Specifically, the first local encoding / decoding rule, the second local encoding / decoding rule, and the first global encoding / decoding rule are first merged to obtain the second global encoding / decoding rule (denoted as global RAID6). This second global encoding / decoding rule includes:

[0152] The result of the XOR operation between all disks (including the data disk and the global parity disk) is 0, as expressed in formula (1):

[0153] (1)

[0154] The result of multiplying each disk (including the data disk, the first local check disk, and the second local check disk) with its location information and then performing an XOR operation is 0; this is represented by formula (7):

[0155] (7)

[0156] The data disk recovery can be obtained according to formulas (1) and (7). The first target decoding rules are as follows:

[0157]

[0158]

[0159] Based on the above k= Taking example 13 as an example, the encoding and decoding rules are merged to construct the second global encoding and decoding rule as follows:

[0160]

[0161] Based on (8) and (9) above, through data disk Eliminate, can be solved :

[0162]

[0163] get:

[0164] Similarly, the solution can be found ,pass data disk Eliminate, can be solved :

[0165]

[0166] Thus, the data disks can be recovered separately according to the second global encoding / decoding rule. .

[0167] In some embodiments, if the number of faulty disks is 3, the target encoding / decoding rule is determined based on the faulty disk type and the verification group to which the faulty disk belongs, including:

[0168] If the faulty disk meets the fifth condition, the encoding / decoding rule of the fifth combination is selected as the target encoding / decoding rule;

[0169] The fifth condition is as follows: the three misaligned disks include two second disks belonging to the same local check group, and a third disk belonging to another local check group; the second disks are data disks, and the third disk is any disk;

[0170] The fifth set of encoding and decoding rules includes: local encoding and decoding rules corresponding to the third disk and second target decoding rules for the two second disks. The second target decoding rules are generated based on the third global encoding and decoding rules. The third global encoding and decoding rules are generated by merging the local encoding and decoding rules corresponding to the second disks and the first global encoding and decoding rules.

[0171] In some embodiments, if the number of faulty disks is 3, the target encoding / decoding rule is determined based on the faulty disk type and the verification group to which the faulty disk belongs, including:

[0172] If the faulty disk meets the sixth condition, the encoding / decoding rule of the sixth combination is selected as the target encoding / decoding rule;

[0173] The sixth condition is as follows: the three misaligned disks include a data disk and a local check disk belonging to the same local check group, and a fourth disk belonging to another local check group; the fourth disk can be any disk.

[0174] The encoding and decoding rules of the sixth combination include: the local encoding and decoding rules corresponding to the fourth disk, the first global encoding and decoding rules, and the local encoding and decoding rules corresponding to the data disk and the local check disk.

[0175] Specifically, if the number of faulty disks is 3, there are several possible scenarios. The first scenario is when the three faulty disks are three disks in the same local check group (including data disks and / or local check disks). However, this does not include the global check disk. At this point, decoding is not possible, for example... Figure 9 and Figure 10 The situation is shown.

[0176] The second scenario is the 2+1 error pattern, where two of the three faulty disks are any disks within the same local checksum group (this can be a data disk and / or a local checksum disk; one of the second disks can be a data disk and the other a local checksum disk, or both can be data disks). The third faulty disk can be any disk within another local checksum group, meaning the third disk can be a data disk, a local checksum disk, or a global checksum disk. This second scenario can be further divided into two sub-scenarios, as shown in Tables 3 and 4 for ease of understanding, providing a correspondence between error patterns and corresponding recovery methods.

[0177] Table 3

[0178]

[0179] Table 4

[0180]

[0181] Error pattern A (equivalent to a faulty disk meeting the fifth condition): two faulty disks in the same local check group are two data disks, and any disk in the second local check group. (This can be a data disk, a global check disk, or a local check disk). The corresponding recovery method is Method 1, which includes:

[0182] Step 1: Use the local encoding and decoding rules of a local check group containing only one faulty disk (equivalent to a local encoding and decoding rule corresponding to a third disk) to recover the faulty disk. At this point, there are two faulty disks remaining.

[0183] Step 2: Determine the third global encoding / decoding rule, and perform decoding according to the third global encoding / decoding rule to recover the remaining two faulty disks. The third global encoding / decoding rule is constructed based on another local encoding / decoding rule and the first global encoding / decoding rule.

[0184] In one example, as described above k Taking the example of =13, assume the faulty disk includes: the first local check group And any data disk in the second local verification group. (This can be a data disk, a global check disk, or a local check disk), such as Figure 11 As shown, the decoding methods include:

[0185] Step 1: Solve using the second local encoding / decoding rule (as shown in equation (6) above). (It can be a data disk, a global check disk, or a local check disk);

[0186] Step 2: Construct a global RAID6 (i.e., the third global encoding / decoding rule) based on the first local encoding / decoding rule (as shown in equation (5) above) and the first global encoding / decoding rule (as shown in equation (4) above), and perform decoding according to the third global encoding / decoding rule. and The second target decoding rule is obtained as follows:

[0187]

[0188] pass Solvable :

[0189] pass Solvable :

[0190] It's not hard to understand that the above is only... k =13. Misaligned discs include and For other disk misalignment cases, the recovery formulas for the remaining two misaligned disks can also be obtained using formulas (1) and (2), or formulas (1) and (3). For example, two misaligned disks in the same local check group are two data disks, denoted as... and If it belongs to the first local check group, then we get:

[0191] ;

[0192] .

[0193] If it belongs to the second local verification group, the corresponding formula can also be derived, which will not be elaborated here.

[0194] Error mode B (equivalent to the faulty disk meeting the sixth condition): Two faulty disks in the same local check group consist of one data disk and one local check disk. And any disk in another local check group. (This can be a data disk, a global check disk, or a local check disk), such as Figure 12 As shown. The corresponding recovery method uses method two, which includes:

[0195] Step 1: Decode the faulty disk using the local encoding and decoding rules for one faulty disk (equivalent to a local encoding and decoding rule for the fourth disk). At this point, only two faulty disks remain.

[0196] Step 2: Decode the data disk using the first global encoding / decoding rule. At this point, only a local parity disk remains. The wrong plate;

[0197] Step 3: Decode the local parity disk using a different local encoding / decoding rule. The wrong plate.

[0198] In one example, as described above k Taking the example of =13, assume the faulty disk includes: the first local check group And any data disk in the second local verification group. (This can be a data disk, a global check disk, or a local check disk):

[0199] Step 1: Solve using the second local encoding / decoding rule (as shown in equation (6) above). (It can be a data disk, a global check disk, or a local check disk);

[0200] Step 2: Decode using the first global encoding / decoding rule (as shown in equation (4) above). ;

[0201] Step 3: Decode the local parity disk using the first local encoding / decoding rule (as shown in equation (5) above). The wrong plate.

[0202] In this way, the three misaligned disks can be restored separately.

[0203] In some embodiments, if the number of faulty disks is 3, the target encoding / decoding rule is determined based on the faulty disk type and the verification group to which the faulty disk belongs, including:

[0204] If the faulty disk satisfies the seventh condition, the encoding / decoding rule of the seventh combination is selected as the target encoding / decoding rule;

[0205] The seventh condition is as follows: the faulty disk includes a global check disk, and a fifth disk and a sixth disk respectively from two local check groups, wherein the fifth disk is a local check disk or a data disk, and the sixth disk is a local check disk or a data disk;

[0206] The encoding and decoding rules of the seventh combination include: the third target decoding rule for the fifth disk, the fourth target decoding rule for the sixth disk, and the first global encoding and decoding rule, the first local encoding and decoding rule, or the second local encoding and decoding rule.

[0207] The third target decoding rule and the fourth target decoding rule are generated based on the first global encoding / decoding rule, the first local encoding / decoding rule, and the second local encoding / decoding rule.

[0208] Specifically, if the number of faulty disks is 3, there are several possible scenarios, the third of which is: when one of the three faulty disks is a global check disk. The other two disks come from data disks or local check disks in two local check groups. This refers to the error pattern 1+1+1. The third type of disk error can be further divided into several sub-cases. For ease of understanding, Tables 5 and 6 provide a correspondence between error patterns and corresponding recovery methods.

[0209] Table 5

[0210]

[0211] Table 6

[0212]

[0213] Error type C1: Error disks include local check disks in one local check group, local check disks in another local check group, and global check disks. ,like Figure 13 As shown; Recovery method 3.1 includes:

[0214] Step 1: Use the first global encoding / decoding rule to decode and restore the global verification disk. At this point, only two partially verified disks remain with incorrect disks;

[0215] Step 2: Decode the two local parity disks simultaneously using two local encoding / decoding rules. The wrong plate.

[0216] Error type C2: Faulty disks include data disks in one local check group, local check disks in another local check group, and global check disks. ,like Figure 14 As shown; Recovery method 3.2 includes:

[0217] Based on the first global encoding and decoding rule, the first local encoding and decoding rule, and the second local encoding and decoding rule, the third target decoding rule for the data disk and the fourth target decoding rule for the local parity disk are derived to recover the data disk and the local parity disk respectively.

[0218] Then restore the global verification disk using the first global encoding / decoding rule.

[0219] In one example, as described above k Taking the example of =13, assume the faulty disks include: the data disks in the first local check group. and the local check disk in the second local check group and global verification disk ,like Figure 14 As shown, the recovery methods include:

[0220] The first step is to deduce the data disk encoding rules based on the first global encoding / decoding rule (as shown in formula (4) above), the first local encoding / decoding rule (as shown in formula (5) above), and the second local encoding / decoding rule (as shown in formula (6) above). and local check disk The decoding rules are as follows:

[0221] Solvable :

[0222]

[0223] Obtain the third target decoding rule:

[0224] as well as, Solvable :

[0225]

[0226] The fourth target decoding rule is obtained: .

[0227] It's not hard to understand that the above is only... k =13. Misaligned discs include and For other disk misalignment cases, the recovery formulas for the two misaligned disks can also be obtained by applying formulas (1) and (2), or formulas (1) and (3). For example, the misaligned disks include the data disks in the first local check group, denoted as... And the local check disk in the second local check group is recorded as Then we get:

[0228] ;

[0229] .

[0230] If the data disk belongs to the second local check group and the local check disk belongs to the first local check group, the corresponding formula can also be derived, which will not be elaborated here.

[0231] The second step is to decode and restore the global verification disk using the first global encoding / decoding rule (as shown in formula (4) above), the first local encoding / decoding rule (as shown in formula (5) above), or the second local encoding / decoding rule (as shown in formula (6) above). .

[0232] Error type C3: Faulty disk includes a local check disk in a local check group. The data disk in another local check group, and the global check disk. ,like Figure 15As shown, recovery method 3.3 is the same as recovery method 3.2 above, and will not be repeated here.

[0233] Error type C4: The faulty disk includes a data disk in one local check group, a data disk in another local check group, and a global check disk. ,like Figure 16 As shown, decoding method 3.4 includes:

[0234] Based on the first global encoding and decoding rule, the first local encoding and decoding rule, and the second local encoding and decoding rule, the third target decoding rule for the data disk and the fourth target decoding rule for the other data disk are derived to recover the two data disks respectively.

[0235] Then restore the global verification disk using any local encoding / decoding rule.

[0236] In one example, as described above k Taking the example of =13, assume the faulty disks include: the data disks in the first local check group. and the data disk in the second local verification group. and global check disk ,like Figure 16 As shown, recovery method 3.4 includes:

[0237] The first step is to deduce the data disk encoding rules based on the first global encoding / decoding rule (as shown in equation (4) above), the first local encoding / decoding rule (as shown in equation (5) above), and the second local encoding / decoding rule (as shown in equation (6) above). and The decoding rules are as follows:

[0238] 1. Through eliminate , can be solved and :

[0239]

[0240] 2. Solvable :

[0241]

[0242] 3. Solvable :

[0243]

[0244] It's not hard to understand that the above is only... k =13. Misaligned discs include and For other disk misalignment cases, the recovery formulas for the remaining two misaligned disks can also be obtained using formulas (1) and (2), or formulas (1) and (3). For example, two misaligned disks in different local verification groups are two data disks, denoted as... and Then we get:

[0245] ;

[0246] .

[0247] The second step is to decode and restore the global verification disk using the first global encoding / decoding rule (as shown in formula (4) above), the first local encoding / decoding rule (as shown in formula (5) above), or the second local encoding / decoding rule (as shown in formula (6) above). .

[0248] As described above, the method provided in this disclosure can recover any one or two faulty disks, as well as partially recover three faulty disks (only when all three faulty disks are in a local check group is it unsolvable), and has stronger fault tolerance.

[0249] In some embodiments, the method further includes:

[0250] Data is written to the data disk, global parity disk, first local parity disk, and / or second local parity disk for each stripe using the target disk writing method:

[0251] The target placement method includes one of the following: left-hand alignment, right-hand alignment, and left-hand non-alignment.

[0252] In some embodiments, if the left-hand misaligned disk writing method is adopted, the data writing to the data disk, global parity disk, first local parity disk, and / or second local parity disk for each stripe includes:

[0253] For the 1st to the Nth stripe, the 1st to the N-2nd physical disk, the first local check physical disk, and the global check physical disk are grouped together and placed on the disk in a left-handed non-aligned manner; and the N-1st to the 2N-4th physical disk and the second local check physical disk are grouped together and placed on the disk in a left-handed non-aligned manner.

[0254] For the (N+1)th to the 2Nth stripe, the first physical disk to the (N-2)th physical disk and the first local check physical disk are grouped together and the stripe is placed on the disk in a left-handed, non-aligned manner; and the global check physical disk, the (N-1)th to the 2N-4th physical disk and the second local check physical disk are grouped together and the stripe is placed on the disk in a left-handed, non-aligned manner.

[0255] Where N = ( k+1) / 2, the k This indicates the total number of disks in the storage system.

[0256] For example, as shown in the example above k =13, so there are a total of 2N=13+1 stripes. For the 1st to 7th stripes (denoted as t1, t2, t3, t4, t5, t6, t7), let physics d1-physics d5 and physics... l0 The disks and the physical disk P are grouped together, and the disks are dropped in a left-handed, misaligned manner; starting with physical disks d6-d10 and physical disks P. l1 The discs are placed in a group, following a left-handed, misaligned arrangement; as shown in Table 7:

[0257] Table 7

[0258]

[0259] For strips 8-14 (denoted as t8, t9, t10, t11, t12, t13, t14), Physics d1-Physics d5 and Physics l0 The disks are grouped together and placed in a left-handed, misaligned manner; physical disk P, physical disk D6-physical disk D10, and physical disk D10. l1 The discs are placed in a group, and the discs are dropped in a left-handed, misaligned manner, as shown in Table 8.

[0260] Table 8

[0261]

[0262] The overall left-handed misaligned drop pattern is shown in Table 9.

[0263] Table 9

[0264]

[0265] The physics d1... physics d10 in Tables 7, 8, and 9 above, and physics... l0 disk, physical l1 Disk and physical disk P represent each physical disk, t1, t2...t14 represent each stripe, d1, d2...d10 in each stripe represent data disks, l0 and l1 represent local parity disks, and p represents global parity disk.

[0266] The above example is based on a single stripe. In actual production, considering load balancing, data is commonly written to disk in a left-handed, non-aligned manner. Using the left-handed, non-aligned method provided in this disclosure, the encoding and decoding rules provided in this disclosure are applicable. It can recover any one or two misaligned disks, and can resolve some cases of three misaligned disks. Only in the following two situations are three misaligned disks not recovered: First: All three misaligned disks are in a local check group (including physical data disk and physical...). l The first type: three misaligned disks, one of which is physical. The disk, along with two others, is in a local verification group (including the physical data disk and the physical disk). l plate).

[0267] Figure 17 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure; as shown below. Figure 17 As shown, the device includes:

[0268] The first processing module is used to respond to a disk recovery request from the storage system and determine the recovery method corresponding to the disk that is faulty; the number of faulty disks is at least one.

[0269] The second processing module is used to perform data recovery on the faulty disk using the recovery method, and obtain the data recovery result;

[0270] In the storage system, each stripe is divided into a first local check group, a second local check group, and a global check group;

[0271] The global verification group includes a global verification disk, a first data disk portion, and a second data disk portion; the first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk; the second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk.

[0272] In some embodiments, the first processing module is configured to determine the faulty disk information in the faulty disk recovery request, wherein the faulty disk information includes at least one of the following: number of faulty disks, type of faulty disk, and verification group to which the faulty disk is located;

[0273] Based on the faulty disk information, the recovery method is determined, which is a method for recovering faulty disk data using target encoding and decoding rules;

[0274] The target encoding / decoding rules include at least one of the following:

[0275] The first local encoding and decoding rule corresponding to the first local verification group, wherein the first local encoding and decoding rule indicates that the result of XOR operation after multiplying each disk in the first local verification group with the disk's position information is 0;

[0276] The second local encoding and decoding rule corresponding to the second local verification group, the second local encoding and decoding rule represents that the result of XOR operation after multiplying each disk in the second local verification group with the disk's position information is 0;

[0277] The first global encoding and decoding rule corresponding to the global verification group, the first global encoding and decoding rule indicates that the result of the XOR operation between each disk in the first local verification group is 0;

[0278] The first data disk portion and the second data disk portion have the same number of data disks.

[0279] In some embodiments, the first processing module is configured to, if the number of faulty disks is 1, determine the encoding and decoding rule corresponding to the verification group to which the faulty disk is located, and use it as the target encoding and decoding rule;

[0280] If the number of faulty disks is 2 or 3, the target encoding / decoding rule is determined according to the faulty disk type and the verification group to which the faulty disk belongs.

[0281] In some embodiments, if the number of faulty disks is 2, the first processing module is used to select the first combination of encoding and decoding rules as the target encoding and decoding rules if the faulty disks meet the first condition.

[0282] The first condition is: the two faulty disks include a global check disk and a first disk, where the first disk is any data disk or a local check disk;

[0283] The encoding and decoding rules of the first combination include: local encoding and decoding rules that do not include the first disk, and local encoding and decoding rules that include the first disk.

[0284] In some embodiments, if the number of faulty disks is 2, the first processing module is used to select the second combination of encoding and decoding rules as the target encoding and decoding rules if the faulty disks meet the second condition;

[0285] The second condition is that the two faulty disks belong to different local verification groups, and the two faulty disks are data disks or local verification disks.

[0286] The encoding and decoding rules of the second combination include: the local encoding and decoding rules corresponding to each local check group.

[0287] In some embodiments, if the number of faulty disks is 2, the first processing module is used to select a third combination of encoding and decoding rules as the target encoding and decoding rules if the faulty disks meet a third condition.

[0288] The third condition is that the two faulty disks include any data disk and a local check disk belonging to the same local check group;

[0289] The third set of encoding and decoding rules includes: a first global encoding and decoding rule, and local encoding and decoding rules corresponding to the arbitrary data disk and the local parity disk.

[0290] In some embodiments, if the number of faulty disks is 2, the first processing module is used to select the fourth combination of encoding and decoding rules as the target encoding and decoding rules if the faulty disks meet the fourth condition.

[0291] The fourth condition is: the two misaligned disks include two data disks belonging to the same local check group;

[0292] The fourth set of encoding and decoding rules includes: a first target decoding rule for two data disks, the first target decoding rule being generated based on a second global encoding and decoding rule, and the second global encoding and decoding rule being generated by merging the first local encoding and decoding rule, the second local encoding and decoding rule, and the first global encoding and decoding rule.

[0293] In some embodiments, if the number of faulty disks is 3, the first processing module is used to select the fifth combination of encoding and decoding rules as the target encoding and decoding rules if the faulty disks meet the fifth condition.

[0294] The fifth condition is as follows: the three misaligned disks include two second disks belonging to the same local check group, and a third disk belonging to another local check group; the second disks are data disks, and the third disk is any disk;

[0295] The fifth set of encoding and decoding rules includes: local encoding and decoding rules corresponding to the third disk and second target decoding rules for the two second disks. The second target decoding rules are generated based on the third global encoding and decoding rules. The third global encoding and decoding rules are generated by merging the local encoding and decoding rules corresponding to the second disks and the first global encoding and decoding rules.

[0296] In some embodiments, if the number of faulty disks is 3, the first processing module is used to select the sixth combination of encoding and decoding rules as the target encoding and decoding rules if the faulty disks meet the sixth condition.

[0297] The sixth condition is as follows: the three misaligned disks include a data disk and a local check disk belonging to the same local check group, and a fourth disk belonging to another local check group; the fourth disk can be any disk.

[0298] The encoding and decoding rules of the sixth combination include: the local encoding and decoding rules corresponding to the fourth disk, the first global encoding and decoding rules, and the local encoding and decoding rules corresponding to the data disk and the local check disk.

[0299] In some embodiments, if the number of faulty disks is 3, the first processing module is used to select the seventh combination of encoding and decoding rules as the target encoding and decoding rules if the faulty disks meet the seventh condition.

[0300] The seventh condition is: the three faulty disks include a global check disk, and the fifth disk and the sixth disk respectively from two local check groups, wherein the fifth disk is a local check disk or a data disk, and the sixth disk is a local check disk or a data disk;

[0301] The encoding and decoding rules of the seventh combination include: the third target decoding rule for the fifth disk, the fourth target decoding rule for the sixth disk, and the first global encoding and decoding rule, the first local encoding and decoding rule, or the second local encoding and decoding rule.

[0302] The third target decoding rule and the fourth target decoding rule are generated based on the first global encoding / decoding rule, the first local encoding / decoding rule, and the second local encoding / decoding rule.

[0303] In some embodiments, the apparatus further includes: a disk write module, configured to write data to the data disk, global check disk, first local check disk and / or second local check disk of each stripe using a target disk write method;

[0304] The target placement method includes one of the following: left-hand alignment, right-hand alignment, and left-hand non-alignment.

[0305] In some embodiments, if the left-hand misaligned disk placement method is adopted, the disk placement module is used to place the first to Nth stripes in a group of physical disks from the first physical disk to the (N-2)th physical disk, the first local check physical disk, and the global check physical disk in a left-hand misaligned manner; and to place the (N-1)th physical disk to the 2N-4th physical disk and the second local check physical disk in a group in a left-hand misaligned manner.

[0306] For the (N+1)th to the 2Nth stripe, the first physical disk to the (N-2)th physical disk and the first local check physical disk are grouped together and the stripe is placed on the disk in a left-handed, non-aligned manner; and the global check physical disk, the (N-1)th to the 2N-4th physical disk and the second local check physical disk are grouped together and the stripe is placed on the disk in a left-handed, non-aligned manner.

[0307] Where N = ( k +1) / 2, k This indicates the total number of disks in the storage system.

[0308] It is understood that, when implementing the corresponding data processing method, the data processing apparatus provided in the above embodiments can allocate the processing to different program modules as needed to complete all or part of the processing described above. Furthermore, the apparatus and the corresponding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0309] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a data processing method.

[0310] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are executed by a processor, causing the processor to execute the data processing method provided in this application.

[0311] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0312] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0313] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0314] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0315] Figure 18This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure; as shown below. Figure 18 As shown, the electronic device 180 includes: a processor 1801, and a memory 1802 communicatively connected to the processor 1801; the memory 1802 stores instructions executable by the processor 1801. The instructions are executed by the processor 1801 to enable the processor 1801 to perform:

[0316] In response to a disk recovery request from the storage system, a recovery method corresponding to the faulty disk is determined; the number of faulty disks is at least one.

[0317] The data recovery method described above is used to recover data from the faulty disk, and the data recovery result is obtained.

[0318] In the storage system, each stripe is divided into a first local check group, a second local check group, and a global check group;

[0319] The global verification group includes a global verification disk, a first data disk portion, and a second data disk portion; the first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk; the second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk.

[0320] The electronic devices and corresponding data processing methods provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0321] In practical applications, the electronic device 180 may further include at least one network interface 1803. The various components in the electronic device 180 are coupled together via a bus system 1804. It is understood that the bus system 1804 is used to implement communication between these components. In addition to a data bus, the bus system 1804 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 18 All buses are labeled as bus system 1804. The number of processors 1801 can be at least one, and the number of memories 1802 can be at least one. Network interface 1803 is used for wired or wireless communication between electronic device 180 and other devices.

[0322] The memory 1802 in this embodiment is used to store various types of data to support the operation of the electronic device 180.

[0323] The methods disclosed in the above embodiments of this disclosure can be applied to processor 1801, or implemented by processor 1801. Processor 1801 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 1801 or by instructions in the form of software. The processor 1801 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 1801 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 1802. Processor 1801 reads the information in memory 1802 and, in conjunction with its hardware, completes the steps of the aforementioned data processing method.

[0324] In some embodiments, the electronic device 180 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0325] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0326] In the above description, the term "some embodiments" refers to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0327] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in this disclosure is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0328] It should be understood that in the various embodiments of this disclosure, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0329] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0330] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: In response to a disk recovery request from the storage system, a recovery method corresponding to the faulty disk is determined; the number of faulty disks is at least one. The data recovery method described above is used to recover data from the faulty disk, and the data recovery result is obtained. In the storage system, each stripe is divided into a first local check group, a second local check group, and a global check group; The global verification group includes a global verification disk, a first data disk portion, and a second data disk portion; the first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk; the second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk. The recovery method for determining the faulty disk includes: Determine the faulty disk information in the faulty disk recovery request, wherein the faulty disk information includes at least one of the following: number of faulty disks, type of faulty disks, and verification group to which the faulty disks belong; Based on the faulty disk information, the recovery method is determined, which is a method for recovering faulty disk data using target encoding and decoding rules; The target encoding / decoding rules include at least one of the following: The first local encoding and decoding rule corresponding to the first local verification group, wherein the first local encoding and decoding rule indicates that the result of XOR operation after multiplying each disk in the first local verification group with the disk's position information is 0; The second local encoding and decoding rule corresponding to the second local verification group, the second local encoding and decoding rule represents that the result of XOR operation after multiplying each disk in the second local verification group with the disk's position information is 0; The first global encoding and decoding rule corresponding to the global verification group, the first global encoding and decoding rule indicates that the result of the XOR operation between each disk in the first local verification group is 0; The first data disk portion and the second data disk portion have the same number of data disks.

2. The method according to claim 1, characterized in that, The step of determining the recovery method based on the faulty disk information includes: If the number of faulty disks is 1, determine the encoding / decoding rule corresponding to the verification group to which the faulty disk belongs, and use it as the target encoding / decoding rule; If the number of faulty disks is 2 or 3, the target encoding / decoding rule is determined according to the faulty disk type and the verification group to which the faulty disk belongs.

3. The method according to claim 2, characterized in that, If the number of faulty disks is 2, the target encoding / decoding rules are determined based on the faulty disk type and the verification group to which the faulty disk belongs, including: If the faulty disk meets the first condition, the encoding and decoding rules of the first combination are selected as the target encoding and decoding rules; The first condition is: the two faulty disks include a global check disk and a first disk, where the first disk is any data disk or a local check disk; The encoding and decoding rules of the first combination include: local encoding and decoding rules that do not include the first disk, and local encoding and decoding rules that include the first disk.

4. The method according to claim 2, characterized in that, If the number of faulty disks is 2, the target encoding / decoding rules are determined based on the faulty disk type and the verification group to which the faulty disk belongs, including: If the faulty disk meets the second condition, the second combination of encoding and decoding rules is selected as the target encoding and decoding rule; The second condition is that the two faulty disks belong to different local verification groups, and the two faulty disks are data disks or local verification disks. The encoding and decoding rules of the second combination include: the local encoding and decoding rules corresponding to each local check group.

5. The method according to claim 2, characterized in that, If the number of faulty disks is 2, the target encoding / decoding rules are determined based on the faulty disk type and the verification group to which the faulty disk belongs, including: If the faulty disk meets the third condition, the third combination of encoding and decoding rules is selected as the target encoding and decoding rule; The third condition is that the two faulty disks include any data disk and a local check disk belonging to the same local check group; The third set of encoding and decoding rules includes: a first global encoding and decoding rule, and local encoding and decoding rules corresponding to the arbitrary data disk and the local parity disk.

6. The method according to claim 2, characterized in that, If the number of faulty disks is 2, the target encoding / decoding rules are determined based on the faulty disk type and the verification group to which the faulty disk belongs, including: If the faulty disk meets the fourth condition, the encoding and decoding rule of the fourth combination is selected as the target encoding and decoding rule; The fourth condition is: the two misaligned disks include two data disks belonging to the same local check group; The fourth set of encoding and decoding rules includes: a first target decoding rule for two data disks, the first target decoding rule being generated based on a second global encoding and decoding rule, and the second global encoding and decoding rule being generated by merging the first local encoding and decoding rule, the second local encoding and decoding rule, and the first global encoding and decoding rule.

7. The method according to claim 2, characterized in that, If the number of faulty disks is 3, the target encoding / decoding rules are determined based on the faulty disk type and the verification group to which the faulty disk belongs, including: If the faulty disk meets the fifth condition, the encoding / decoding rule of the fifth combination is selected as the target encoding / decoding rule; The fifth condition is as follows: the three misaligned disks include two second disks belonging to the same local check group, and a third disk belonging to another local check group; the second disks are data disks, and the third disk is any disk; The fifth set of encoding and decoding rules includes: local encoding and decoding rules corresponding to the third disk and second target decoding rules for the two second disks. The second target decoding rules are generated based on the third global encoding and decoding rules. The third global encoding and decoding rules are generated by merging the local encoding and decoding rules corresponding to the second disks and the first global encoding and decoding rules.

8. The method according to claim 2, characterized in that, If the number of faulty disks is 3, the target encoding / decoding rules are determined based on the faulty disk type and the verification group to which the faulty disk belongs, including: If the faulty disk meets the sixth condition, the encoding / decoding rule of the sixth combination is selected as the target encoding / decoding rule; The sixth condition is as follows: the three misaligned disks include a data disk and a local check disk belonging to the same local check group, and a fourth disk belonging to another local check group; the fourth disk can be any disk. The encoding and decoding rules of the sixth combination include: the local encoding and decoding rules corresponding to the fourth disk, the first global encoding and decoding rules, and the local encoding and decoding rules corresponding to the data disk and the local check disk.

9. The method according to claim 2, characterized in that, If the number of faulty disks is 3, the target encoding / decoding rules are determined based on the faulty disk type and the verification group to which the faulty disk belongs, including: If the faulty disk satisfies the seventh condition, the encoding / decoding rule of the seventh combination is selected as the target encoding / decoding rule; The seventh condition is: the three faulty disks include a global check disk, and the fifth disk and the sixth disk respectively from two local check groups, wherein the fifth disk is a local check disk or a data disk, and the sixth disk is a local check disk or a data disk; The encoding and decoding rules of the seventh combination include: the third target decoding rule for the fifth disk, the fourth target decoding rule for the sixth disk, and the first global encoding and decoding rule, the first local encoding and decoding rule, or the second local encoding and decoding rule. The third target decoding rule and the fourth target decoding rule are generated based on the first global encoding / decoding rule, the first local encoding / decoding rule, and the second local encoding / decoding rule.

10. The method according to claim 1, characterized in that, The method further includes: The data is written to the data disk, global parity disk, first local parity disk and / or second local parity disk of each stripe using the target disk writing method; The target placement method includes one of the following: left-hand alignment, right-hand alignment, and left-hand non-alignment.

11. The method according to claim 10, characterized in that, If the left-hand misaligned disk write method is adopted, the data write operation for each stripe's data disk, global parity disk, first local parity disk, and / or second local parity disk includes: For the 1st to the Nth stripe, the 1st to the N-2nd physical disk, the first local check physical disk, and the global check physical disk are grouped together and placed on the disk in a left-handed non-aligned manner; and the N-1st to the 2N-4th physical disk and the second local check physical disk are grouped together and placed on the disk in a left-handed non-aligned manner. For the (N+1)th to the 2Nth stripe, the first physical disk to the (N-2)th physical disk and the first local check physical disk are grouped together and the stripe is placed on the disk in a left-handed, non-aligned manner; and the global check physical disk, the (N-1)th to the 2N-4th physical disk and the second local check physical disk are grouped together and the stripe is placed on the disk in a left-handed, non-aligned manner. Where N = ( k +1) / 2, k This indicates the total number of disks in the storage system.

12. A data processing apparatus, characterized in that, The device includes: The first processing module is used to respond to a disk recovery request from the storage system and determine the recovery method corresponding to the disk that is faulty; the number of faulty disks is at least one. The second processing module is used to perform data recovery on the faulty disk using the recovery method, and obtain the data recovery result; In the storage system, each stripe is divided into a first local check group, a second local check group, and a global check group; The global verification group includes a global verification disk, a first data disk portion, and a second data disk portion; the first local verification group includes a first local verification disk, a first data disk portion, and a global verification disk; the second local verification group includes a second local verification disk, a second data disk portion, and a global verification disk. The first processing module is used to determine the faulty disk information in the faulty disk recovery request. The faulty disk information includes at least one of the following: number of faulty disks, type of faulty disk, and verification group of the faulty disk. Based on the faulty disk information, the recovery method is determined, which is a method for recovering faulty disk data using target encoding and decoding rules; The target encoding / decoding rules include at least one of the following: The first local encoding and decoding rule corresponding to the first local verification group, wherein the first local encoding and decoding rule indicates that the result of XOR operation after multiplying each disk in the first local verification group with the disk's position information is 0; The second local encoding and decoding rule corresponding to the second local verification group, the second local encoding and decoding rule represents that the result of XOR operation after multiplying each disk in the second local verification group with the disk's position information is 0; The first global encoding and decoding rule corresponding to the global verification group, the first global encoding and decoding rule indicates that the result of the XOR operation between each disk in the first local verification group is 0; The first data disk portion and the second data disk portion have the same number of data disks.

13. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 11.

Images