Data reconstruction method, electronic device and computer program product
By introducing a dynamic fencing mechanism with dynamic disk boundary window and RAID block span, the disk reconstruction algorithm is optimized, and the problems of unbalanced reconstruction efficiency and unbalanced load distribution in the storage system are solved, achieving efficient data reconstruction and performance improvement.
Patent Information
- Application Number
- CN202310601432.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-25
- Publication Date
- 2024-11-26
AI Technical Summary
When the existing storage system expands the resource pool, the fixed disk grouping mechanism leads to unbalanced reconstruction efficiency and unbalanced load distribution, and does not support expansion of less than the RAID width plus one disk, resulting in inefficient RAID performance and poor customer experience.
The dynamic disk boundary window and RAID block span are introduced, and the optimized disk reconstruction algorithm is adopted to support any number of disks through the dynamic fencing mechanism, and the reconstruction write domain is widened to improve data reconstruction efficiency.
It realizes efficient data reconstruction in mapped RAID, supports any number of disks, improves reconstruction efficiency, reduces the possibility of RAID data loss, and ensures load balancing and performance improvement.
Smart Images

Figure CN119025329A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computers, and more particularly, to a method, an electronic device, and a computer program product for data reconstruction. Background Art
[0002] With the development of data storage technology, various data storage systems have been able to provide users with higher and higher data storage capabilities. While improving data storage capabilities, users have also put forward higher and higher requirements for data reliability and scalability of storage systems. At present, a variety of data storage systems based on redundant array of independent disks (RAID) have been developed to improve data reliability. When one or more disks in a storage system fail, the data on the failed disk can be rebuilt from the data on other normally operating disks. Summary of the invention
[0003] Embodiments of the present disclosure provide a method, an electronic device, and a computer program product for data reconstruction.
[0004] According to a first aspect of the present disclosure, a method for data reconstruction is provided. The method includes determining a first disk set based on a predetermined range and a failed disk in response to a failure of a disk where a RAID block of a redundant array of independent disks RAID in a storage system is located, wherein the first disk set includes multiple disks associated with the RAID block within the predetermined range. The method also includes determining a second disk set for reconstructing data on the failed disk based on other disks in the first disk set other than the multiple disks associated with the RAID block. The method also includes determining multiple scores of multiple disks in the second disk set based on multiple disk correlations of the multiple disks in the second disk set, multiple RAID block spans, and multiple weights. The method also includes determining a target disk among the multiple disks in the second disk set based on the multiple scores. The method also includes reconstructing the data on the failed disk on the target disk.
[0005] According to a second aspect of the present disclosure, an electronic device is also provided. The electronic device includes a processor and a memory coupled to the processor, the memory having instructions stored therein, and the instructions cause the device to perform an action when executed by the processor. The action includes determining a first disk set based on a predetermined range and a failed disk in response to a failure of a disk where a RAID block of a redundant array of independent disks RAID in a storage system is located, wherein the first disk set includes multiple disks associated with the RAID block within the predetermined range. The action also includes determining a second disk set for rebuilding data on the failed disk based on other disks other than the multiple disks associated with the RAID block in the first disk set. The action also includes determining multiple scores of multiple disks in the second disk set based on multiple disk correlations of the multiple disks in the second disk set, multiple RAID block spans, and multiple weights. The action also includes determining a target disk among the multiple disks in the second disk set based on the multiple scores. The action also includes rebuilding the data on the failed disk on the target disk.
[0006] According to a third aspect of the present disclosure, there is provided a computer program product, which is tangibly stored on a computer-readable medium and comprises computer-executable instructions, which when executed by a device cause the device to perform the method according to the first aspect.
[0007] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:
[0009] Figure 1 A schematic diagram schematically illustrates an example environment in which embodiments of the present disclosure can be implemented;
[0010] Figure 2 A flowchart of a method for data reconstruction according to an exemplary implementation of the present disclosure is schematically shown;
[0011] Figure 3 A schematic diagram of a dynamic disk boundary window according to an exemplary implementation of the present disclosure is schematically shown;
[0012] Figure 4 A schematic diagram schematically illustrates a failure domain and a reconstruction write domain according to an exemplary implementation of the present disclosure;
[0013] Figure 5A block diagram schematically illustrates a process for determining a correlation between two disks according to an exemplary implementation of the present disclosure;
[0014] Figure 6 A schematic diagram schematically illustrates a RE span according to an exemplary implementation of the present disclosure;
[0015] Figure 7 Schematically illustrates the workflow of a high-level disk reconstruction algorithm according to an exemplary implementation of the present disclosure;
[0016] Figure 8 A schematic diagram schematically illustrates a process of determining a selectable disk set according to an exemplary implementation of the present disclosure; and
[0017] Fig. 9 A block diagram of a device for data reconstruction according to an exemplary implementation of the present disclosure is schematically shown.
[0018] Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. DETAILED DESCRIPTION
[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0020] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below. In addition, all specific numerical values in this document are examples, which are only for the purpose of helping understanding, and are not intended to limit the scope.
[0021] In conventional technology, a mapped redundant array of independent disks (mapped RAID) has been developed. In the mapped RAID, a disk is a logical concept and may include multiple disk extents (DE). Multiple blocks included in a logical disk may be distributed on different physical disks in a storage system. For multiple blocks in a stripe of the mapped RAID, the multiple blocks should be distributed on different physical storage devices so that when a physical disk where one of the multiple blocks is located fails, a reconstruction operation may be performed to recover data from the physical disks where other blocks are located.
[0022] In traditional technology, a storage system based on RAID can combine multiple disks into a disk array. By providing redundant disks, the reliability of the entire disk group can be much greater than that of a single disk. RAID can provide various advantages over a single disk, such as enhanced data integrity, enhanced fault tolerance, increased throughput or capacity, and so on. There are multiple RAID standards, such as RAID-1, RAID-2, RAID-3, RAID-4, RAID-5, RAID-6, RAID-10, RAID-50, and so on.
[0023] In the research, it was found that in order to facilitate the management of multiple disks in the storage system, it has been proposed to divide multiple disks into at least one group according to the grouping threshold (including the upper threshold and the lower threshold). For example, in a Drive Partnership Group (DPG), stripes in the user storage system can be created based on the disks in each group. Due to the limitation on the number of disks in a group, in some cases, each disk in the storage system cannot be divided into a corresponding group. Furthermore, there may be a situation where the disks in each group have a serious imbalance in usage load.
[0024] It will be understood that the number of disks included in a group has a threshold range (e.g., including an upper threshold and a lower threshold). Generally speaking, the upper threshold can be set to 64 or other values, and the lower threshold can be determined based on the stripe width of the storage system and the number of spare disks required. For example, in a 4D+1P storage system, the stripe width is 4+1=5, and the number of spare disks is 1, then the lower threshold can be determined based on the sum of the two: 5+1=6. At this time, each group can include 6 to 64 disks. When the number of disks included in the resource pool exceeds 64, there can be multiple groups.
[0025] The threshold range may impose many restrictions on the management of the storage system. For example, when the number of storage systems in a resource pool is expanded, the newly added storage system may not be assigned to a certain group. For another example, the disks in each group may have a serious imbalance in usage load, which will cause a serious imbalance in the available storage space and service life of each disk in the storage system, etc.
[0026] In some cases, DPGs within dynamic pools are used to limit the number of drives that other dynamic pool objects can span. This level of fencing can ensure that data is not spread across a large number of drives, which would increase the risk of experiencing multiple drive failures. This may degrade or destroy incremental rebuilds that fail PACOs (proactive copy) of RAID protection options (when a PACO fails due to a drive failure during the PACO, the rebuild can complete the remaining PACO portion instead of the entire drive contents).
[0027] For example, in some products, the number of disks in a DPG is fixed to 64, and in other products it is fixed to 25. For simplicity, this article takes DPG=64 as an example, but this example is not restrictive. In this way, it is necessary to keep the number of failure domains of disk(n) no more than 64. This fixed fence mechanism will have three limitations.
[0028] In the first restriction, if the resource pool has 64+6 disks, you can have 64 disks in one DPG, and if one disk fails, at most 63 disks are ready to rebuild. But for the second DPG with only 6 disks, if one disk fails, only 5 disks can be rebuilt. This means that the rebuild efficiency is not balanced between DPGs. Similarly, the load distribution of 64 disks and 6 disks is also unbalanced.
[0029] In the second restriction, at the DPG boundary, any extension less than the RAID width plus one disk is not supported. The following formula (1) represents the effective number of disks that can be selected for each disk type:
[0030] 64 × N + W + 1 <= Total number of disks <= 64 × N + 64 (1)
[0031] Where N>=0, N is an integer, N represents the number of disks; W represents the width of the RAID
[0032] In the third limitation, expansion is complex and time-consuming, especially when expanding across DPGs. This limitation leads to a poor customer experience and inefficient RAID performance.
[0033] In view of this, the method disclosed in the present invention provides a solution for data reconstruction. The solution proposes a dynamic fence mechanism based on dynamic disk boundaries. The dynamic fence mechanism introduces a "predetermined window (also known as a dynamic disk boundary window)" and a "RAID extent span" for each disk to represent the RE span of the relevant disk. The solution also proposes an optimized disk reconstruction algorithm. In this way, the dynamic fence mechanism not only supports the use of any number of disks in the mapped RAID, but also improves the data reconstruction efficiency by widening the reconstruction write domain.
[0034] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. The environment 100 may be referred to as a storage system 100. Figure 1 As shown, the storage system 100 may include a resource pool 110, and the resource pool 110 may include multiple disks. For example, disk 122, ..., disk 124, ..., and disk 130. A storage device may include one or more disks. However, in this article, for simplicity, disks and storage devices are sometimes used interchangeably.
[0035] The storage system 100 may be a RAID-based storage system. The storage system 100 may include multiple RAID extents (REs). One RE may be created across multiple disks according to a predetermined rule. One RE may include one or more stripes. However, in this article, for simplicity, a stripe may also be referred to as a RE. Therefore, in this article, stripes and REs may be used interchangeably.
[0036] Typically, a stripe can include multiple DEs. DE is the smallest disk resource unit in a mapped RAID. The capacity of a disk will be organized into a set of continuous non-overlapping ranges. All DEs on the same type of storage device should have the same fixed size. DE includes most of the space as user data space and a small portion of the space as internal metadata space.
[0037] As an example, a stripe can be composed of N DEs. N represents the RAID group width according to the RAID configuration policy (for example, 4+1RE contains 5 DEs). The mapped RAID group created on top of the resource pool applies some RAID algorithms to protect data integrity. A mapped RAID group is a collection of RAID ranges. The disks in a mapped RAID group are in the same performance tier. More specifically, they should be of the same type, such as NL-SAS, SAS, flash memory. In addition, in the storage system 100, user storage systems 140, ..., and 142 can be created based on the disks in the resource pool 110, respectively.
[0038] Figure 2 The flowchart of the method 200 for data reconstruction according to an exemplary implementation of the present disclosure is schematically shown. Figures 1 to 6 The method 200 is described below. At block 210, in response to a failure of a disk where a RAID block of a redundant array of independent disks RAID in a storage system is located, a first disk set is determined based on a predetermined range and the failed disk, wherein the first disk set includes a plurality of disks associated with the RAID block within the predetermined range.
[0039] As an example, assume that an RE involves disk 122, disk 124, disk 126, disk 128, and disk 130, where disk 126 fails. At this time, the data on disk 126 needs to be rebuilt on other disks. Then, the first disk set can be determined using the predetermined range and the failed disk 126. Figure 3 To describe the related concepts of the predetermined range.
[0040] Figure 3 A schematic diagram 300 of a dynamic disk boundary window according to an exemplary implementation of the present disclosure is schematically shown. In order to ensure the reconstruction performance when any disk in the pool fails, a new term "dynamic disk boundary window" is introduced herein. The dynamic disk boundary window is intended to provide a dynamic 64-disk fence boundary for each disk, rather than a physically fixed DPG boundary. The "64" is to ensure the reconstruction performance when any disk in the resource pool fails.
[0041] like Figure 3 As shown, a write domain that can be used to rebuild data can be selected based on a dynamic disk boundary window (e.g., dynamic disk boundary window 310 and dynamic disk boundary window 320). According to an exemplary implementation of the present disclosure, the concept of "grouping" in the prior art solution is abandoned, and multiple disks within the dynamic disk boundary window can be determined from the resource pool as the write domain.
[0042] In the following, only a 4D+1P storage system will be described as an example. The dynamic disk boundary window 310 and the dynamic disk boundary window 320 may have a predetermined width. The write domain used for rebuilding data may be determined based on other disks different from the target disk in the window dynamic disk boundary.
[0043] In this paper, for any disk i, the boundaries of its 64 disks can be {Disk((i+N-32)mod N),…,Disk i,…,Disk((i+31)mod N)}. All disks in the resource pool can be regarded as a continuous circle, and each disk has neighbors. For example, Disk 0 and Disk 159 can be regarded as connected end to end, so they are neighbors. Disk 0 and Disk N-1 are neighbors. So as long as N ≥ 64, each disk can have its own 32 left neighbors and 31 right neighbors.
[0044] like Figure 3 As shown, the dynamic pool consists of 160 disks {Disk 0, ..., Disk 159}. Considering Disk 80, its "dynamic disk boundary window" is {Disk 48, ..., Disk 111}. For Disk 96, its "dynamic disk boundary window" is {Disk 64, ..., Disk 127}.
[0045] It is easy to understand that if Disk 80 is related to Disk 96, then their related RE / DE must be located in the intersection 330 {Disk 64, ..., Disk 111}. For edge Disk 0, its "dynamic disk boundary window" is {Disk 128, ..., Disk 31}.
[0046] In the dynamic fence mapping RAID framework proposed in this paper, since there is no fixed DPG, the concept of failure domain has also changed. Each disk has its own failure domain. If a disk fails, the other disks in the failure domain will participate in the reconstruction. If another disk in the failure domain fails, a double disk failure will occur.
[0047] According to the "dynamic disk boundary window" rule defined for each disk in the dynamic fence mechanism, the grouping in the existing storage system is abandoned, which can uniformly manage the appropriate disks in the storage system in a more flexible way. In addition, the rule of the RAID block can be obtained: the serial number {DiskSN}_(right-most) of the rightmost disk of any RE and the serial number {DiskSN}_(left-most) of the leftmost disk (the leftmost and rightmost should be understood as: the storage pool is regarded as a circle, with the leftmost and rightmost in the clockwise direction) satisfy that the difference between the two ({DiskSN}_(right-most)-{DiskSN}_(left-most)+1) is not greater than 33. This reason will be combined with the following Figure 4 To explain.
[0048] Figure 4 A schematic diagram 400 of a failure domain and a reconstruction domain according to an exemplary implementation of the present disclosure is schematically shown. Figure 4 As shown in the figure, when the size of the dynamic disk boundary window is 64, the failure domain of disk(n) needs to be kept within 64. When disk(n) is the leftmost scene of RE, {DiskSN}_(left-most)=n, because the failure domain of disk(n) needs to be kept at {disk(n-32),…,disk(n+31)}, the rightmost disk of RE may be disk(n+31), {DiskSN}_(right-most)=n+31. So {DiskSN}_(right-most)-{DiskSN}_(left-most)+1=32.
[0049] Similarly, when disk(n) is the rightmost disk of RE, {DiskSN}_(right-most) = n, and the failure domain of disk(n) needs to be kept at {disk(n-32),…,disk(n+31)}. The leftmost disk of RE may be disk(n-32), {DiskSN}_(left-most) = n-32. So {DiskSN}_(right-most)-{DiskSN}_(left-most)+1 = 33.
[0050] Therefore, it can be concluded that if the storage system always maintains ({DiskSN}_(right-most)-{DiskSN}_(left-most)+1)<=32, the failure domain of 64 disks can be guaranteed. In other words, any RE that satisfies ({DiskSN}_(right-most)-{DiskSN}_(left-most)+1)<=32 will not violate the protection availability provided by the failure domain of 64 disks.
[0051] Return to Figure 2 After understanding the dynamic disk boundary window (also called the predetermined range), the dynamic disk boundary window for disk 122 can be determined for other disks in the RE, such as disk 122, except for the failed disk 126. Similarly, the dynamic disk boundary windows for disk 126, disk 128, and disk 130 can be determined. The intersection of these four dynamic disk boundary windows is obtained to obtain the dynamic disk boundary window intersection (also called the first disk set).
[0052] At block 220, a second disk set for rebuilding data on the failed disk is determined based on other disks in the first disk set other than the plurality of disks associated with the RAID block. As an example, disk 122, disk 124, disk 126, disk 128, and disk 130 may be removed from the dynamic disk boundary window intersection. The remaining disks in the dynamic disk boundary window intersection are determined as a selectable disk set (also referred to as a second disk set) for the failed disk 126.
[0053] At block 230, scores of the plurality of disks in the second disk set are determined based on the plurality of disk correlations, the plurality of RAID block spans, and the plurality of weights of the plurality of disks in the second disk set. As an example, assume that the plurality of disks in the selectable disk set include disk 120 and disk 134. Then, scores C may be determined for disk 120 and disk 132, respectively. Figure 5 and Figure 6 to describe how the score is determined.
[0054] First, the concept of correlation between two disks is introduced. Figure 5 A block diagram of a process 500 for determining the correlation between two disks according to an exemplary implementation of the present disclosure is schematically shown. Taking a 4D+1P RAID stripe as an example, it includes 5 disk extents (DEs), and the data in the 5 DEs are usually accessed in a correlated manner. This results in an increase in the amount of access to the 5 disks that each include 5 DEs. In general, for any two disks, the more stripes the two disks serve together, the greater the possibility that the two disks are accessed simultaneously, and the higher the workload of the two disks. Therefore, it should be avoided as much as possible that two disks serve too many stripes at the same time.
[0055] For the i-th disk and the j-th disk (i≠j) among multiple disks, the correlation between the two disks can be expressed as γ Disk i,Disk j According to an exemplary implementation of the present disclosure, the correlation of the disk itself can be set to 0, that is, γ Disk i,Disk i = 0. As the storage system is used, some stripes will be created in the storage system. Figure 5 Describes how to determine the correlation between disks.
[0056] Assume that the storage system includes N (N is a positive integer) disks D0 to DN-1. Three stripes 510, 520, and 530 have been created in the storage system. The correlation γ can be determined based on the number of stripes that commonly involve the i-th disk and the j-th disk. Disk i,Disk jIf we want to determine the correlation between disk D0 and disk D1 (i.e., i=0, j=1), we can find that only stripe 510 uses blocks from both disk D0 and disk D1, so γ Disk 0,Disk 1=1.
[0057] For another example, if we want to determine the correlation between disk D1 and disk D2 (i.e., i=1, j=2), we can find that stripe 510 uses blocks from both disk D1 and disk D2, and stripe 520 also uses blocks from both disk D1 and disk D2, so γ Disk 1,Disk 2=2. Similarly, the correlation between any two disks in the plurality of disks can be determined. The mapping relationship between the stripe and the blocks in the stripe can be conveniently obtained from the address mapping of the storage system. Therefore, the correlation γ between any two disks can be determined in a simple and effective manner using the exemplary implementation of the present disclosure. Disk 1,Disk 2.
[0058] According to an exemplary implementation of the present disclosure, the disk correlation of a specific disk among multiple disks can be determined based on the correlation between two disks. Specifically, the correlation between the specific disk and each other disk other than the specific disk can be determined respectively. Further, the disk correlation of the specific disk can be determined based on the determined correlation.
[0059] According to an exemplary implementation of the present disclosure, the disk correlation can be set to be proportional to the correlation between the disk and each other disk. For example, the sum of multiple correlations can be obtained, and the disk correlation can be determined based on the summation operation. Using the exemplary implementation of the present disclosure, the disk correlation of a specific disk can be determined based on the correlation between the specific disk and other disks based on simple mathematical operations. Assuming that it is desired to determine the disk correlation of the i-th disk, it can be based on the above reference Figure 5 The method described above determines the correlation between the i-th disk and the other j-th disk (0≤j≤N-1, and i≠j). According to an exemplary implementation of the present disclosure, the disk correlation γ of the i-th disk can be determined based on the following formula (2): Disk i :
[0060]
[0061] where γ Disk i , represents the disk correlation of the i-th disk, γ Disk i,Disk j represents the correlation between the i-th disk and the j-th disk, and N represents the number of disks in the resource pool.
[0062] Using the exemplary implementation of the present disclosure, for the i-th disk, the disk correlation γ of the device is Disk i Represents the sum of the correlations between the i-th disk and the other N-1 disks. At this point, the disk correlation can accurately measure the correlation between the i-th disk and other disks, and can reflect the workload of the disk. By selecting disks for creating stripes based on disk correlation, it is possible to accurately select appropriate disks that help achieve load balancing based on the workload of each disk.
[0063] It will be understood that the above formula (2) only schematically illustrates a specific example for determining the disk correlation. According to the exemplary implementation of the present disclosure, other formulas may also be used to determine the disk correlation. For example, the disk correlation may be determined based on the product of multiple correlations. The above has described how to determine the disk correlation γ of the i-th disk. Disk i , after determining the disk correlation γ Disk i After that, the disk correlation for RE (e.g., RE e) can be determined, which is obtained by formula (3):
[0064] γ REe =∑ Disk j∈REe γ Disk j (3)
[0065] where γ RE e Represents the disk correlation of RE e, which means the sum of the correlations of multiple disks where the DE constituting RE e is located.
[0066] Now return to Figure 2 In order to evaluate the distribution status of the disk, this paper introduces the score C, which combines the disk correlation and RE span. Figure 6 It is worth noting that the lower the disk correlation is, the larger the RE span is, the better C can be achieved, and the better the disk distribution system is. The score C can be calculated by the following formula (4):
[0067]
[0068] Among them, γ represents the correlation, S represents the RE span; ω γ Represents the weight of the correlation, ω S represents the weight of the RE span, and ω γ +ω S = 1. By adjusting the weights, a better overall balance can be achieved. Figure 6 To describe the newly introduced RE span.
[0069] Figure 6 A schematic diagram of RE span 600 according to an exemplary implementation of the present disclosure is schematically shown. For ease of understanding, a new term "RAID block span" (for simplicity, it can be called RE span) is introduced herein to represent the span of the associated disk in RE. It can be calculated as the disk count between the leftmost RE / DE associated disk and the rightmost RE / DE associated disk. The larger the RE span, the wider the reconstruction write domain, and the higher the system reconstruction efficiency.
[0070] like Figure 6 As shown, for this RE, the disk ID of the left boundary is 40, and the disk ID of the right boundary is 120. The RE span of this RE is the area between the dotted line 610 and the dotted line 620. In the illustrated case, the RE span S=120-40=80.
[0071] As a more general example, assume that there are N disks in the dynamic pool. For one RE, it consists of 5 DEs {DE0, DE1, DE2, DE3, DE4}, which reside on 5 disks respectively. First, the leftmost RE / DE related disk and the rightmost RE / DE related disk among the 5 related disks can be obtained. Then the RE span can be calculated by the following formula (5):
[0072] S = {DiskSN}_(rightmost end) - {DiskSN}_(leftmost end) + 1 (5) where {DiskSN}_n represents the serial number of disk(n).
[0073] For example, in a dynamic pool with 160 disks, if the given RE-associated disks are {disk 50, disk 60, disk 61, disk 62, disk 70}, its RE span S = 70-50 + 1 = 21. If the DE on disk 50 is relocated to disk 90, its RE-associated disks are {disk 60, disk 61, disk 62, disk 70, disk 90}, and its RE span becomes S = 90-60 + 1 = 31.
[0074] After understanding the RE span, we can consider two extreme scenarios of RE reconstruction write domain:
[0075] For RE{Disk(n-32),Disk(n-31),Disk(n-30),Disk(n-29),Disk(n)}, when DE(n) fails and needs to be replaced, it is necessary to follow ({DiskSN}_(right-most)-{DiskSN}_(left-most)+1)<=32. Therefore, DE(n-61) can be selected as the leftmost disk to expand the reconstruction write domain to replace DE(n) to form a new RE:{DE(n-61),DE(n-32),DE(n-31),DE(n-30),DE(n-29)}. This does not violate the RAID protection availability calculation.
[0076] Similarly, for RE{Disk(n), Disk(n+28), Disk(n+29), Disk(n+30), Disk(n+31)}, when DE(n) fails and needs to be replaced, the reconstruction write domain can be expanded by selecting DE(n+60) as the rightmost disk to replace DE(n) to form a new RE:{DE(n+28), DE(n+29), DE(n+30), DE(n+31), DE(n+60)}.
[0077] Therefore, the reconstruction write domain of disk(n) can be expanded to {Disk(n-61),…,Disk(n+60)}, with 61+61-1=121 disks. The traditional reconstruction domain method is 64-1=63 disks. Therefore, this new reconstruction method greatly improves the reconstruction efficiency by expanding the reconstruction write domain by 121 / 63=192%.
[0078] Now return to Figure 2 At block 240, a target disk is determined from among the plurality of disks in the second disk set based on the plurality of scores. As an example, assuming that in the second disk set including disk 120 and disk 130, the score for disk 120 is less than the score for disk 132, disk 120 may be selected as the target disk. At block 250, data on the failed disk is rebuilt on the target disk. For example, data on failed disk 126 is rebuilt on selected disk 120.
[0079] The data reconstruction mechanism of this solution can expand the reconstruction writing domain, reduce the reconstruction time, and reduce the possibility of RAID data loss. The dynamic protection mechanism based on dynamic disk boundaries can support new reconstruction mechanisms and expand the reconstruction writing domain.
[0080] Figure 7The workflow of a high-level disk reconstruction algorithm 700 according to an exemplary implementation of the present disclosure is schematically shown. Based on the dynamic fence framework proposed in the present disclosure, the present disclosure also proposes the following disk reconstruction algorithm. When any disk n fails, all DEs on the disk need to be rebuilt. Since the reconstruction domain of each RE is independent, they can be rebuilt concurrently. Here, the reconstruction algorithm of a single RE / DE is given, which can be called the target RE / DE.
[0081] At 710, the algorithm starts. At 720, for the remaining REs associated with the target DE, the disk set associated with the current RE is checked. For example, the remaining REs are RE a and RE e. One of the REs can be selected as the current RE, for example, RE e. The selectable disk set for the current RE e is determined by the following formula (6):
[0082]
[0083] Wherein, {the selectable disk set for RE e} represents the selectable disk set for RE e; express dynamic boundary window; {RE ecorrelated Disks} represents the disks related to RE e.
[0084] The following will combine Figure 8 To describe the process of determining a set of selectable disks. Figure 8 FIG. 8 schematically shows a process 800 for determining a selectable disk set according to an exemplary implementation of the present disclosure. Figure 8 As shown, disks associated with one RE include disk 810 (D80), disk 820 (D96), disk 830 (100), disk 840 (D70), and disk 850 (D90).
[0085] According to the size of the dynamic disk boundary window, the dynamic disk boundary window 812 for the disk 810 may be determined to be disks D48 to D111 (32 disks are selected from the left side of the disk 810, and 31 disks are selected from the right side of the disk 810).
[0086] The dynamic disk boundary window 822 for disk 820 is disks D64 to D127 (32 disks are selected from the left side of disk 820, and 31 disks are selected from the right side of disk 820). The dynamic disk boundary window 832 for disk 830 is disks D68 to D131 (32 disks are selected from the left side of disk 830, and 31 disks are selected from the right side of disk 830).
[0087] The dynamic disk boundary window 842 for disk 840 is disks D38 to D101 (32 disks are selected from the left side of disk 840, and 31 disks are selected from the right side of disk 840). The dynamic disk boundary window 852 for disk 850 is disks D58 to D121 (32 disks are selected from the left side of disk 850, and 31 disks are selected from the right side of disk 850).
[0088] At this time, it can be assumed that disk 810 associated with the RE fails, and the remaining disks are disk 820-disk 850. The intersection 860 (also referred to as the first disk set) of dynamic disk boundary windows 822, 832, 842, and 852 of disks 820-750 can be calculated. The calculated intersection is then used to exclude the disks associated with the RE (i.e., disk 810-disk 850), and a selectable disk set 870 (also referred to as the second disk set) can be obtained.
[0089] Now return to Figure 7 At 730, γ=γ is calculated Disk i,current RE +γ Disk i and score S, then calculate score C for each unselected disk i in the selectable disk set of the current RE e (using formula 4), where γ Disk i,current RE It can be calculated by the following formula (7):
[0090]
[0091] where γ Disk i,current RE Indicates the relevance of disk i to the current RE e, because at this time RE has already selected n-1 DEs. Here Disk Dj Represents the disk where the Dj is located.
[0092] At 740, the disk with the minimum score C is selected as the target disk for the target fault DE. If there are multiple disks with the same minimum score, a disk is randomly selected as the target disk. At 750, the data on the target fault DE is started to be rebuilt onto the target disk. At 760, the algorithm 700 ends.
[0093] In the disk reconstruction algorithm 700, the dynamic disk correlation and RE span can be always evaluated when determining where to rebuild the failed DE data, so that it is easier to balance the disk consumption of the entire mapping RAID and at the same time widen the write domain of the related RE as much as possible. In this way, the disk reconstruction algorithm 700 can greatly improve the reconstruction efficiency.
[0094] Fig. 91 shows a schematic block diagram of a device 900 that can be used to implement an embodiment of the present disclosure. The device 900 may be a device or apparatus described in an embodiment of the present disclosure. Fig. 9 As shown, the device 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 or computer program instructions loaded from a storage unit 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904. Although not shown in FIG. Fig. 9 As shown in FIG. 9 , device 900 may further include a co-processor.
[0095] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0096] The various methods or processes described above may be performed by the processing unit 901. For example, in some embodiments, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the CPU 901, one or more steps or actions in the methods or processes described above may be performed.
[0097] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.
[0098] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
[0099] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0100] The computer program instructions for performing the disclosed operation may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, programming languages including object-oriented programming languages, and conventional procedural programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In certain embodiments, by utilizing the state information of a computer-readable program instruction to customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.
[0101] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0102] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0103] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the frame can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous frames can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved. It should also be noted that each frame in the block diagram and / or flow chart, and the combination of frames in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0104] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A data reconstruction method, comprising: In response to a failure of a disk in which a RAID block of a redundant array of independent disks RAID in a storage system is located, determining a first disk set based on a predetermined range and the failed disk, wherein the first disk set includes a plurality of disks associated with the RAID block within the predetermined range; Determine a second disk set for rebuilding data on the failed disk based on other disks in the first disk set other than the plurality of disks associated with the RAID block; Determine a plurality of scores for the plurality of disks in the second disk set based on a plurality of disk correlations, a plurality of RAID block spans, and a plurality of weights of the plurality of disks in the second disk set; determining a target disk among the plurality of disks in the second set of disks based on the plurality of scores; as well as The data on the failed disk is rebuilt on the target disk.
2. The method according to claim 1, wherein the storage system comprises a plurality of disks, and the plurality of disks in the storage system are arranged in a ring manner, and the number of the plurality of disks in the storage system is greater than the number of the plurality of disks used to form the RAID.
3. The method according to claim 2, wherein determining the first disk set based on the predetermined range and the failed disk comprises: determining a size of a predetermined window, the size of the predetermined window indicating the number of disks within the predetermined range; respectively determining a plurality of disks within the predetermined window corresponding to disks among the plurality of disks associated with the RAID block; as well as An intersection of the plurality of disks respectively corresponding to disks among the plurality of disks is determined as the first disk set.
4. The method according to claim 3, wherein respectively determining a plurality of disks within the predetermined window corresponding to disks in the plurality of disks associated with the RAID block comprises: For a first disk among the plurality of disks associated with the RAID block, determine the rightmost and leftmost two disks in the clockwise direction and within the predetermined window among the plurality of disks in the storage system; determining a plurality of disks between the two disks as the plurality of disks for the first disk within the predetermined window; For a second disk among the plurality of disks associated with the RAID block, wherein the second disk is different from the first disk, determining two rightmost and leftmost disks in a clockwise direction and within the predetermined window among the plurality of disks in the storage system; as well as A plurality of disks between the two disks are determined as the plurality of disks for the second disk within the predetermined window. 5 . The method according to claim 1 , wherein the disk affinity indicates distribution of a plurality of RAID blocks that have been created in the storage system between a disk and other disks other than the disk.
6. The method of claim 2, wherein the RAID block span is determined based on: Among the plurality of disks in the second disk set and among the plurality of disks where the RAID block is located, determine two disk serial numbers of the rightmost and leftmost disks in the clockwise direction; and Based on the difference between the two disk serial numbers, the RAID block span is determined.
7. The method according to claim 1, wherein the plurality of weights comprises a first weight and a second weight, and based on a plurality of disk correlations of the plurality of disks in the second disk set, a plurality of RAID block spans, and a plurality of weights, determining a plurality of scores of the plurality of disks in the second disk set comprises: For a first disk among the plurality of disks associated with the RAID block, determining a first disk correlation and a first RAID block span corresponding to the first disk; determining a first product based on the first weight and the correlation with the first disk; determining a second product based on the first weight sum and the inverse of the first RAID block span; as well as A sum of the first product and the second product is determined as a first score for the first disk.
8. The method of claim 1, wherein determining a target disk among the plurality of disks in the second disk set based on the plurality of scores comprises: selecting a disk having a minimum score among the plurality of scores in the second disk set as the target disk; or In response to at least two disks having the same minimum score, one of the at least two disks is randomly selected as the target disk.
9. The method according to claim 1, wherein the failed disk is a first failed disk, the target disk is a first target disk, and the RAID block further includes a second failed disk, the method further comprising: In parallel with determining the first target disk corresponding to the first failed disk, determining a second target disk corresponding to the second failed disk for the second failed disk; as well as In parallel with rebuilding the data on the first failed disk on the first target disk, rebuilding the data on the second failed disk on the second target disk.
10. An electronic device comprising: processor; as well as a memory coupled to the processor, the memory having instructions stored therein, the instructions, when executed by the processor, causing the device to perform actions, the actions comprising: In response to a failure of a disk in which a RAID block of a redundant array of independent disks RAID in a storage system is located, determining a first disk set based on a predetermined range and the failed disk, wherein the first disk set includes a plurality of disks associated with the RAID block within the predetermined range; Determine a second disk set for rebuilding data on the failed disk based on other disks in the first disk set other than the plurality of disks associated with the RAID block; Determine a plurality of scores for the plurality of disks in the second disk set based on a plurality of disk correlations, a plurality of RAID block spans, and a plurality of weights of the plurality of disks in the second disk set; determining a target disk among the plurality of disks in the second set of disks based on the plurality of scores; and The data on the failed disk is rebuilt on the target disk.
11. The electronic device according to claim 10, wherein the storage system comprises a plurality of disks, and the plurality of disks in the storage system are arranged in a ring manner, and the number of the plurality of disks in the storage system is greater than the number of the plurality of disks used to form the RAID.
12. The electronic device of claim 11, wherein determining the first disk set based on the predetermined range and the failed disk comprises: determining a size of a predetermined window, the size of the predetermined window indicating the number of disks within the predetermined range; respectively determining a plurality of disks within the predetermined window corresponding to disks among the plurality of disks associated with the RAID block; as well as An intersection of the plurality of disks respectively corresponding to disks among the plurality of disks is determined as the first disk set.
13. The electronic device according to claim 12, wherein respectively determining a plurality of disks within the predetermined window corresponding to disks among the plurality of disks associated with the RAID block comprises: For a first disk among the plurality of disks associated with the RAID block, determine the rightmost and leftmost two disks in the clockwise direction and within the predetermined window among the plurality of disks in the storage system; determining a plurality of disks between the two disks as the plurality of disks for the first disk within the predetermined window; For a second disk among the plurality of disks associated with the RAID block, wherein the second disk is different from the first disk, determining two rightmost and leftmost disks in a clockwise direction and within the predetermined window among the plurality of disks in the storage system; as well as A plurality of disks between the two disks are determined as the plurality of disks for the second disk within the predetermined window.
14. The electronic device according to claim 10, wherein the disk affinity indicates distribution of a plurality of RAID blocks that have been created in the storage system between a disk and other disks other than the disk.
15. The electronic device of claim 11, wherein the RAID block span is determined based on the following actions: Among the plurality of disks in the second disk set and among the plurality of disks where the RAID block is located, determine two disk serial numbers of the rightmost and leftmost disks in the clockwise direction; and Based on the difference between the two disk serial numbers, the RAID block span is determined.
16. The electronic device according to claim 10, wherein the plurality of weights comprises a first weight and a second weight, and based on a plurality of disk correlations of the plurality of disks in the second disk set, a plurality of RAID block spans, and a plurality of weights, determining a plurality of scores of the plurality of disks in the second disk set comprises: For a first disk among the plurality of disks associated with the RAID block, determining a first disk correlation and a first RAID block span corresponding to the first disk; determining a first product based on the first weight and the correlation with the first disk; determining a second product based on the first weight sum and the inverse of the first RAID block span; as well as A sum of the first product and the second product is determined as a first score for the first disk.
17. The electronic device of claim 10, wherein based on the plurality of scores, determining a target disk among the plurality of disks in the second disk set comprises: selecting a disk having a minimum score among the plurality of scores in the second disk set as the target disk; or In response to at least two disks having the same minimum score, one of the at least two disks is randomly selected as the target disk.
18. The electronic device according to claim 10, wherein the failed disk is a first failed disk, the target disk is a first target disk, and the RAID block further includes a second failed disk, and the action further includes: In parallel with determining the first target disk corresponding to the first failed disk, determining a second target disk corresponding to the second failed disk for the second failed disk; as well as In parallel with rebuilding the data on the first failed disk on the first target disk, rebuilding the data on the second failed disk on the second target disk.
19. A computer program product tangibly stored on a computer readable medium and comprising computer executable instructions which, when executed by a device, cause the device to perform a method comprising: In response to a failure of a disk in which a RAID block of a redundant array of independent disks RAID in a storage system is located, determining a first disk set based on a predetermined range and the failed disk, wherein the first disk set includes a plurality of disks associated with the RAID block within the predetermined range; Determine a second disk set for rebuilding data on the failed disk based on other disks in the first disk set other than the plurality of disks associated with the RAID block; Determine a plurality of scores for the plurality of disks in the second disk set based on a plurality of disk correlations, a plurality of RAID block spans, and a plurality of weights of the plurality of disks in the second disk set; determining a target disk among the plurality of disks in the second set of disks based on the plurality of scores; as well as The data on the failed disk is rebuilt on the target disk.
20. The computer program product of claim 19, wherein the storage system comprises a plurality of disks, and the plurality of disks in the storage system are arranged in a ring manner, the number of the plurality of disks in the storage system being greater than the number of the plurality of disks used to form the RAID.
