Lrc code improvement method for cold and hot data, computer device and storage medium

Abstract

The application provides an LRC code improvement method for cold and hot data, a computer device and a storage medium, and the method comprises the following steps: generating a global check code according to RS encoding; judging whether there is hot data; when there is no hot data or a user does not distinguish cold and hot data, directly performing traditional (k, l, m) LRC encoding on the basis of (k, m) RS encoding, dividing k data disks into l groups, performing XOR encoding on the disks in each group to generate l local check codes; when there is hot data, distinguishing cold and hot data according to the actual working scene of the user and the definition of read / write frequency. The application combines LRC and the division of cold and hot data, and proposes an improvement method for LRC encoding when cold and hot data are divided according to the specific use scene of user data under a storage array, multiple hot area data recovery is accelerated, and the degradation read of multiple hot data has a certain acceleration effect when multiple hot data are damaged at the same time.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to an improved LRC code method for hot and cold data, a computer device, and a storage medium. Background Technology

[0002] To address the storage requirements of massive amounts of data and improve the reliability of the storage system, ensuring that data collection nodes can reconstruct the original files with a high probability, it is necessary to store an additional amount of redundancy on top of the original data. This allows the system to continue operating normally even if some nodes fail, and the data collection nodes can still decode and recover the original files. Simultaneously, to maintain system reliability, failed nodes need to be repaired promptly; therefore, designing a robust node repair mechanism is crucial.

[0003] Erasure coding is a forward error correction technique in coding theory, originally applied in the field of communications to address data loss and degradation during transmission. Due to its effectiveness in preventing data loss, erasure coding has been introduced into the storage field. It can effectively reduce storage overhead while maintaining the same reliability, and therefore is widely used in various storage systems and data centers, such as Microsoft Azure and Facebook's F4.

[0004] There are many types of erasure codes, among which Reed-Solomon (RS) codes, commonly used in distributed environments, are frequently found in practical storage systems. RS codes are related to two parameters, k and r. Given two positive integers k and r, an RS code encodes k data blocks into r additional parity blocks. The method of encoding these r parity blocks based on a Vandermonde matrix or a Cauchy matrix is ​​called an RS erasure code using a Vandermonde matrix or a Cauchy matrix. Specifically, the RS erasure code based on a Vandermonde matrix is ​​as follows:

[0005] The RS erasure code based on the Cauchy matrix is:

[0006] The upper k*k matrix corresponds to k original data blocks, and the lower r*k matrix corresponds to the encoding matrix. This is achieved by comparing the original data D1 to D... k Multiplying them together gives the newly added P1 to P. r These are the r checksums obtained from the encoding. If any at most r of these data are corrupted or lost during transmission, and error correction is needed, the original data blocks D1 to D2 are obtained by multiplying the inverse of the matrix corresponding to the remaining data with the original data. k(The derivation process will not be elaborated here).

[0007] From D1 to D r Taking data loss and decoding as an example, the RS erasure coding method for data recovery is as follows:

[0008] As we can see, the core concept of erasure coding is to construct an invertible coding matrix to generate check data, and its inverse matrix can be calculated to recover the original data. Common RS erasure codes use the Cauchy matrix or VanderMont matrix introduced above. The advantage of this is that the resulting matrix is ​​definitely invertible, and any of its submatrices are also invertible, and the size of the matrix is ​​simple to expand.

[0009] As can be seen above, decoding under RS ​​requires reading k data blocks for any error. Multiple errors can be decoded in parallel by multiple decoding modules, but k data blocks still need to be read. However, due to the limited read and write speed of current storage media (HDD, SSD, or any similar media), the recovery speed will be extremely slow when this speed k is large.

[0010] Therefore, Microsoft proposed the concept of Local Check Code (LRC), and the LRC method is as follows: Figure 1 As shown, an LRC implementation is shown in (6,2,2), where 6 refers to the presence of 6 user data blocks. Figure 1 These are D1-D6, where the first 2 refers to two additional local data blocks. Figure 1 The middle is and Its generation method is to directly perform XOR, that is: , and It is a global check block, generated by RS, as described in the RS introduction above.

[0011] In the case of LRC, when and If any data becomes corrupted, it can be directly recovered through a local check block, for example when When an error occurs, simply read By performing an XOR operation, we can obtain the result. However, when two errors occur simultaneously, such as When an error occurs, it is necessary to utilize and Recovery is performed via RS. Summary of the Invention

[0012] In view of this, the purpose of this invention is to propose an improved LRC code method for hot and cold data, a computer device, and a storage medium. Combining LRC and the division of hot and cold data, a scheme is proposed to accelerate the recovery (degraded read) of multiple hot data zones in a storage array when there is a hot and cold data division based on the specific usage scenario of user data. The scheme improves LRC encoding, and when multiple hot data zones are simultaneously damaged, it has a certain speed-up effect on the degraded read of multiple hot data zones.

[0013] To achieve the above objectives, in one respect, the present invention provides an improved method for LRC codes of hot and cold data, wherein the method includes the following steps: Step 1: Generate a global checksum according to RS encoding; Step 2: Determine if hot data exists. If hot data exists, proceed to Step 4; if no hot data exists or the user does not distinguish between hot and cold data, proceed to Step 3. Step 3: When there is no hot data or the user does not distinguish between hot and cold data, directly perform traditional (k,l,m)LRC encoding on the basis of (k,m)RS encoding, divide the k data disks into l groups, and XOR the disks in each group to generate l local check codes. Step 4: When hot data exists, distinguish between hot and cold data based on the user's actual work scenario and the definition of read / write frequency.

[0014] As a further aspect of the present invention, a global check code is generated according to the RS encoding. When the RS encoding is (6,2)RS encoding, the (6,2)RS erasure coding process based on the Vandermonde matrix is ​​as follows: .

[0015] As a further aspect of the present invention, when there is no hot data or the user does not distinguish between hot and cold data, traditional (6,2,2) LRC encoding is performed based on (6,2) RS encoding. The six data disks are divided into two groups, and XOR encoding is performed within each group to generate two local check codes. The traditional (6,2,2) LRC encoding process is as follows: .

[0016] As a further aspect of the present invention, when thermal data is available, When dividing data into hot and cold zones, cold zones are disks with relatively low read / write frequency thresholds, and hot zones are disks with relatively high read / write frequency thresholds. Hot zones are set to... The cold zone is set as .

[0017] As a further aspect of the present invention, when dividing hot and cold zones, if the number of disks containing hot data is less than or equal to the number of groups in the LRC, the hot zone is... The disks in the LRC are grouped by number, so that hot data disks are distributed in different groups; when the number of hot data disks is greater than the number of groups in the LRC, the hot zone is... The disks in the LRC are grouped so that each LRC group contains a hot data disk, and the remaining hot data disks are placed in the same LRC group as much as possible.

[0018] As a further aspect of the present invention, when the number of disks for hot data is less than or equal to the number of groups in LRC, the data block... The global verification block is and At this time, the hot zone is If the cold area is the remaining disk space, then:

[0019] Based on the (6,2)RS encoding, an improved (6,2,2)LRC encoding is implemented to address hot zones. The data is divided into two groups, and the improved (6,2,2) LRC code is: .

[0020] As a further aspect of the present invention, when the number of disks for hot data is less than or equal to the number of groups in LRC: For cold data, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code; For hot data, when only one disk needs to be repaired, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code.

[0021] As a further aspect of the present invention, if the hot zone disk Repair is needed; decoding should be performed based on the traditional (6,2,2) LRC code, and the data should be read. Six disks, decoded in the Galois domain according to Formula I: Ⅰ; If the improved (6,2,2) LRC code is decoded, the following can be read: Six disks, decoded according to Formula II: II.

[0022] As a further aspect of the present invention, when the number of disks for hot data is greater than the number of groups in LRC, the data blocks... The global verification block is and The hot zone is The cold zone is the remaining disk space: .

[0023] As a further aspect of the present invention, the hot zone The data is divided into two groups, and an improved (6,2,2) LRC encoding is applied based on RS encoding; where: For cold data, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code; For hot data, when only one disk needs to be repaired, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code.

[0024] As a further aspect of the present invention, if the hot zone disk is Repair is needed; decode using traditional (6,2,2) LRC code and read... Six disks are decoded in the Galois domain according to formulas III and IV: III; If the hot zone is It needs to be fixed by decoding using the improved (6,2,2) LRC code and reading... Six plates, according to get Decode according to Formula IV: IV.

[0025] In another aspect, the present invention provides a computer device including a memory and a processor, the memory storing a computer program which, when executed by the processor, performs any of the above-described methods for improving LRC codes for hot and cold data according to the present invention.

[0026] In another aspect, the present invention provides a computer-readable storage medium storing computer program instructions that, when executed, implement any of the above-described methods for improving LRC codes for hot and cold data according to the present invention.

[0027] The present invention has at least the following beneficial technical effects: This invention proposes an improved LRC code method for hot and cold data, a computer device, and a storage medium. Combining LRC and the division of hot and cold data, it proposes a scheme to accelerate the recovery (degraded read) of multiple hot data zones in a storage array when there is a hot and cold data division based on the specific usage scenario of user data. The scheme improves the LRC encoding, and when multiple hot data zones are simultaneously damaged, it has a certain speed-up effect on the degraded read of multiple hot data zones.

[0028] These or other aspects of this application will become more apparent from the following description of embodiments. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the application. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0030] In the diagram: Figure 1 This is a schematic diagram illustrating the LRC concept (6,2,2) proposed by Microsoft. Figure 2 This is a flowchart illustrating an embodiment of the LRC code improvement method for hot and cold data according to the present invention. Figure 3 This is a schematic diagram of the (6,2) RS erasure coding process based on the Vandermonde matrix in the LRC code improvement method for hot and cold data according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the traditional (6,2,2) LRC encoding in an improved LRC encoding method for hot and cold data according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the (6,2,2) LRC code improved based on RS encoding in the LRC code improvement method for cold and hot data according to an embodiment of the present invention. Figure 6 This is a schematic diagram illustrating the improvement of the (6,2,2) LRC code based on the (6,2) RS encoding in an LRC code improvement method for hot and cold data according to an embodiment of the present invention. Figure 7 This is a schematic diagram of the hardware structure of a computer device for implementing an LRC code improvement method for cold and hot data according to an embodiment of the present invention. Figure 8 This is a schematic diagram of an embodiment of a computer-readable storage medium for implementing an improved LRC code method for cold and hot data according to an embodiment of the present invention. Detailed Implementation

[0031] The present application will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0033] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two different entities or different parameters with the same name. Therefore, "first" and "second" are merely for convenience of expression and should not be construed as limiting the embodiments of the present invention. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as other steps or units inherent in a process, method, system, product, or device that includes a series of steps or units.

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0036] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0037] Because when and If any data becomes corrupted, it can be directly recovered through a local check block, for example when When an error occurs, simply read By performing an XOR operation, we can obtain the result. However, when two errors occur simultaneously, such as When an error occurs, it is necessary to utilize and Recovery is performed via RS.

[0038] When data first enters the system, it is marked as hot data by default. The system periodically checks the data. If a piece of hot data is accessed less than a certain number of times within a certain period, it may be marked as cold data; conversely, if a piece of cold data is accessed more than the set number of times within a certain period, it may be marked as hot data. In actual storage scenarios, there is a distinction between hot and cold data. For example, text files are often considered cold data, meaning their read / write frequency is not very high, while media files are often considered hot data, meaning their read / write frequency is high. The distinction between hot and cold data is based on the relative frequency of data read / write in the storage array. Accordingly, hot data, because of its higher read / write probability, has a higher probability of error, and because of its high read / write frequency, its recovery speed requirements are also higher when any error occurs. Cold data is the opposite.

[0039] Therefore, this invention proposes an improved LRC code method for hot and cold data, a computer device, and a storage medium. Combining LRC and the division of hot and cold data, it proposes a scheme to accelerate the recovery (degraded read) of multiple hot data zones in a storage array when there is a hot and cold data division based on the specific usage scenario of user data. The scheme improves the LRC encoding, and when multiple hot data zones are simultaneously damaged, it has a certain speed-up effect on the degraded read of multiple hot data zones.

[0040] See Figure 2 As shown, an embodiment of the present invention provides an improved method for LRC codes of hot and cold data, wherein the method includes the following steps S1-S4: Step 1: Generate a global checksum according to RS encoding.

[0041] See Figure 3 As shown, taking (6,2) RS coding as an example, the RS erasure coding based on the Vandermonde matrix is ​​as follows: .

[0042] Step 2: Determine if hot data exists. If hot data exists, proceed to Step 4; if no hot data exists or the user does not distinguish between hot and cold data, proceed to Step 3.

[0043] The specific distinction between hot and cold data depends on the user's actual work scenario and the definition of read / write frequency. However, hot data will definitely be accessed more frequently than cold data, so more attention should be paid to hot data to reduce its degradation read latency.

[0044] Step 3: When there is no hot data or the user does not distinguish between hot and cold data, directly perform traditional (k,l,m)LRC encoding on the basis of (k,m)RS encoding, divide the k data disks into l groups, and XOR the disks in each group to generate l local check codes.

[0045] like Figure 4 As shown, in Figure 4 Based on the (6,2)RS encoding, (6,2,2)LRC encoding is performed, dividing the 6 data disks into 2 groups. Within each group, XOR encoding is performed to generate 2 local check codes. During encoding, the traditional (6,2,2)LRC encoding is as follows: .

[0046] Step 4: When hot data exists, distinguish between hot and cold data based on the user's actual work scenario and the definition of read / write frequency.

[0047] In this embodiment, when When there is hot data, the specific distinction between hot and cold data depends on the user's actual work scenario and the definition of read and write frequency. Hot data has a higher probability of being read and written, so it has a higher probability of error. Also, because of its high read and write frequency, the recovery speed requirement is also higher when any error occurs. Cold data is the opposite.

[0048] In embodiments of the present invention, cold and hot zones are defined, where cold zones are disks (or chunks) with relatively low read / write frequencies, and hot zones are those with relatively high read / write frequencies. The specific division depends on the user's actual work scenario and the definition of read / write frequency. Here, we assume that the hot zone is... The relative cold zone is .

[0049] There are two scenarios at this point. First, when the number of disks containing hot data is less than or equal to the number of groups in the LRC, the hot zone... The first method involves grouping disks according to their number in the LRC, distributing hot data disks across different groups; the second method involves grouping hot data disks into different groups when the number of hot data disks exceeds the number of LRC groups. The disks in the LRC are grouped so that each LRC group contains a hot data disk, and the remaining hot data disks are placed in the same LRC group as much as possible.

[0050] In this embodiment, the first case is based on the above. Figure 4 In the middle, data block The global verification block is and For example, suppose the hot zone disk is at this time If the cold area is the remaining disk space, then:

[0051] like Figure 5 As shown, in Figure 5 The (6,2,2)LRC encoding is an improvement on the (6,2)RS encoding, which includes hot zones. The data is divided into two groups, and the improved (6,2,2) LRC code used for encoding is shown in the following expression (1): (1).

[0052] In this embodiment, for cold data, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code.

[0053] For hot data, when only one disk needs repair, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code. However, when multiple hot data disks fail simultaneously, the decoding method of the improved (6,2,2) LRC code is simpler and has less latency than that of the traditional (6,2,2) LRC code.

[0054] Assuming hot zone disk It needs to be fixed if used. Figure 4 Decoding the traditional (6,2,2) LRC code requires reading... Six disks, and then decoded according to the following expression (2). The whole process is carried out in the Galois domain, which is computationally complex and has a long delay.

[0055] (2) If using Figure 5 Decoding the improved (6,2,2) LRC code requires reading... Six disks, and then decode according to the following expression (3). The whole process only involves XOR operation, which is simple to calculate and has little delay.

[0056] (3) The second scenario, based on the above... Figure 4 In the middle, data block The global verification block is and For example, suppose the hot zone disk is at this time If the cold area is the remaining disk space, then: (4) like Figure 4 As shown, the hot zone The data is divided into two groups, in Figure 4 The (6,2,2)LRC encoding is an improvement on the (6,2)RS encoding, as shown in the following figure. Figure 5 As shown.

[0057] For cold data, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code.

[0058] For hot data, when only one disk needs repair, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code. However, when multiple hot data disks fail simultaneously, the decoding method of the improved (6,2,2) LRC code is simpler and has less latency than that of the traditional (6,2,2) LRC code.

[0059] Assuming the hot zone disk is It needs to be fixed if used. Figure 5 Decoding the traditional (6,2,2) LRC code requires reading... Six disks, then decoded according to the following expression, the whole process is carried out in the Galois domain.

[0060] (5) Assuming the hot zone disk is It needs to be fixed if used. Figure 6 Decoding the improved (6,2,2) LRC code requires reading... Six plates, then according to It can be obtained Then, it is decoded according to the following expression, which is simpler and has less delay than traditional LRC code decoding.

[0061] (6) This invention combines LRC (Local Recording) and hot / cold data partitioning to propose a scheme for accelerating the recovery (degraded read) of multiple hot data zones in a storage array when there is hot / cold data partitioning based on specific user data usage scenarios. The scheme improves LRC encoding, resulting in a certain speedup effect for degraded reads of multiple hot data zones when they are simultaneously corrupted.

[0062] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0063] It should be understood that although the above description follows a certain order, these steps are not necessarily executed in that order. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, some steps in this embodiment may include multiple steps or multiple stages, which are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least a portion of the steps or stages in other steps.

[0064] In a second aspect, a computer device 1000 is provided, including a memory 1001 and a processor 1002. The memory stores a computer program, which, when executed by the processor, implements the method of any of the above embodiments.

[0065] like Figure 7 The diagram shown is a hardware structure schematic of an embodiment of a computer device for implementing the LRC code improvement method for cold and hot data provided by the present invention. Figure 7 Taking the computer device 1000 shown as an example, this computer device includes a processor 1002 and a memory 1001, and may also include an input device 430 and an output device 440. The processor 1002, memory 1001, input device 430, and output device 440 can be connected via a bus or other means. Figure 7 Taking a bus connection as an example, input device 430 can receive input digital or character information, as well as generate signal inputs related to LRC code improvement of hot and cold data. Output device 440 may include display devices such as a display screen.

[0066] Memory 1001, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the LRC code improvement method for cold and hot data in the embodiments of this application. Memory 1001 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created by using the LRC code improvement method for cold and hot data, etc. In addition, memory 1001 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 1001 may optionally include memory remotely located relative to processor 1002, and these remote memories can be connected to the local module via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0067] In some embodiments, processor 1002 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 1002 is typically used to control the overall operation of computer device 1000. In this embodiment, processor 1002 is used to run program code stored in memory 1001 or process data. In this embodiment, the processors 1002 of multiple computer devices 1000 execute various server functions and data processing by running non-volatile software programs, instructions, and modules stored in memory 1001, thereby implementing the LRC code improvement method for cold and hot data in the above-described method embodiment, including the following steps: Step 1: Generate a global checksum according to RS encoding; Step 2: Determine if hot data exists. If hot data exists, proceed to Step 4; if no hot data exists or the user does not distinguish between hot and cold data, proceed to Step 3. Step 3: When there is no hot data or the user does not distinguish between hot and cold data, directly perform traditional (k,l,m)LRC encoding on the basis of (k,m)RS encoding, divide the k data disks into l groups, and XOR the disks in each group to generate l local check codes. Step 4: When hot data exists, distinguish between hot and cold data based on the user's actual work scenario and the definition of read / write frequency.

[0068] In this embodiment, a global check code is generated according to the RS encoding. When the RS encoding is (6,2)RS encoding, the (6,2)RS erasure coding process based on the Vandermonde matrix is ​​as follows: .

[0069] In this embodiment, when there is no hot data or the user does not distinguish between hot and cold data, traditional (6,2,2) LRC encoding is performed based on (6,2) RS encoding. The six data disks are divided into two groups, and XOR encoding is performed within each group to generate two local check codes. The traditional (6,2,2) LRC encoding process is as follows: .

[0070] In this embodiment, when thermal data exists, When dividing data into hot and cold zones, cold zones are disks with relatively low read / write frequency thresholds, and hot zones are disks with relatively high read / write frequency thresholds. Hot zones are set to... The cold zone is set as .

[0071] In this embodiment, when dividing hot and cold zones, if the number of disks containing hot data is less than or equal to the number of groups in the LRC, the hot zone is... The disks in the LRC are grouped by number, so that hot data disks are distributed in different groups; when the number of hot data disks is greater than the number of groups in the LRC, the hot zone is... The disks in the LRC are grouped so that each LRC group contains a hot data disk, and the remaining hot data disks are placed in the same LRC group as much as possible.

[0072] In this embodiment, when the number of disks for hot data is less than or equal to the number of LRC packets, the data block... The global verification block is and At this time, the hot zone is If the cold area is the remaining disk space, then:

[0073] Based on the (6,2)RS encoding, an improved (6,2,2)LRC encoding is implemented to address hot zones. The data is divided into two groups, and the improved (6,2,2) LRC code is: .

[0074] In this embodiment, when the number of disks for hot data is less than or equal to the number of LRC groups: For cold data, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code; For hot data, when only one disk needs to be repaired, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code.

[0075] In this embodiment, if the hot zone disk Repair is needed; decoding should be performed based on the traditional (6,2,2) LRC code, and the data should be read. Six disks, decoded in the Galois domain according to Formula I: Ⅰ; If the improved (6,2,2) LRC code is decoded, the following can be read: Six disks, decoded according to Formula II: II.

[0076] In this embodiment, when the number of disks containing hot data is greater than the number of LRC packets, the data blocks... The global verification block is and The hot zone is The cold zone is the remaining disk space: .

[0077] In this embodiment, the hot zone The data is divided into two groups, and an improved (6,2,2) LRC encoding is applied based on RS encoding; where: For cold data, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code; For hot data, when only one disk needs to be repaired, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code.

[0078] In this embodiment, if the hot zone disk is Repair is needed; decode using traditional (6,2,2) LRC code and read... Six disks are decoded in the Galois domain according to formulas III and IV: III; If the hot zone is It needs to be fixed by decoding using the improved (6,2,2) LRC code and reading... Six plates, according to get Decode according to Formula IV: IV.

[0079] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-compatible storage medium. When the program is executed, it can include the processes of the embodiments of the above methods, that is, the steps of implementing the LRC code improvement method for cold and hot data of the above method embodiments.

[0080] A fourth aspect of the present invention also provides a computer-readable storage medium. Figure 8 This is a schematic diagram of a computer-readable storage medium for an improved LRC code method for hot and cold data provided in an embodiment of the present invention. Figure 8 As shown, the computer-readable storage medium 2000 stores computer program instructions 2001, which can be executed by a processor. When the computer program instructions 2001 are executed, they implement the method of any of the above embodiments, that is, the LRC code improvement method for cold and hot data of the above method embodiments, including the following steps: Step 1: Generate a global checksum according to RS encoding; Step 2: Determine if hot data exists. If hot data exists, proceed to Step 4; if no hot data exists or the user does not distinguish between hot and cold data, proceed to Step 3. Step 3: When there is no hot data or the user does not distinguish between hot and cold data, directly perform traditional (k,l,m)LRC encoding on the basis of (k,m)RS encoding, divide the k data disks into l groups, and XOR the disks in each group to generate l local check codes. Step 4: When hot data exists, distinguish between hot and cold data based on the user's actual work scenario and the definition of read / write frequency.

[0081] In this embodiment, a global check code is generated according to the RS encoding. When the RS encoding is (6,2)RS encoding, the (6,2)RS erasure coding process based on the Vandermonde matrix is ​​as follows: .

[0082] In this embodiment, when there is no hot data or the user does not distinguish between hot and cold data, traditional (6,2,2) LRC encoding is performed based on (6,2) RS encoding. The six data disks are divided into two groups, and XOR encoding is performed within each group to generate two local check codes. The traditional (6,2,2) LRC encoding process is as follows: .

[0083] In this embodiment, when thermal data exists, When dividing data into hot and cold zones, cold zones are disks with relatively low read / write frequency thresholds, and hot zones are disks with relatively high read / write frequency thresholds. Hot zones are set to... The cold zone is set as .

[0084] In this embodiment, when dividing hot and cold zones, if the number of disks containing hot data is less than or equal to the number of groups in the LRC, the hot zone is... The disks in the LRC are grouped by number, so that hot data disks are distributed in different groups; when the number of hot data disks is greater than the number of groups in the LRC, the hot zone is... The disks in the LRC are grouped so that each LRC group contains a hot data disk, and the remaining hot data disks are placed in the same LRC group as much as possible.

[0085] In this embodiment, when the number of disks for hot data is less than or equal to the number of LRC packets, the data block... The global verification block is and At this time, the hot zone is If the cold area is the remaining disk space, then:

[0086] Based on the (6,2)RS encoding, an improved (6,2,2)LRC encoding is implemented to address hot zones. The data is divided into two groups, and the improved (6,2,2) LRC code is: .

[0087] In this embodiment, when the number of disks for hot data is less than or equal to the number of LRC groups: For cold data, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code; For hot data, when only one disk needs to be repaired, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code.

[0088] In this embodiment, if the hot zone disk Repair is needed; decoding should be performed based on the traditional (6,2,2) LRC code, and the data should be read. Six disks, decoded in the Galois domain according to Formula I: Ⅰ; If the improved (6,2,2) LRC code is decoded, the following can be read: Six disks, decoded according to Formula II: II.

[0089] In this embodiment, when the number of disks containing hot data is greater than the number of LRC packets, the data blocks... The global verification block is and The hot zone is The cold zone is the remaining disk space: .

[0090] In this embodiment, the hot zone The data is divided into two groups, and an improved (6,2,2) LRC encoding is applied based on RS encoding; where: For cold data, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code; For hot data, when only one disk needs to be repaired, the decoding method of the improved (6,2,2) LRC code is the same as that of the traditional (6,2,2) LRC code.

[0091] In this embodiment, if the hot zone disk is Repair is needed; decode using traditional (6,2,2) LRC code and read... Six disks are decoded in the Galois domain according to formulas III and IV: III; If the hot zone is It needs to be fixed by decoding using the improved (6,2,2) LRC code and reading... Six plates, according to get Decode according to Formula IV: IV.

[0092] It should be understood that, where there is no conflict, all the embodiments, features and advantages described above for the LRC code improvement method for cold and hot data according to the present invention are equally applicable to the LRC code improvement system and storage medium for cold and hot data according to the present invention.

[0093] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0094] Finally, it should be noted that the computer-readable storage medium (e.g., memory) described herein can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. By way of example, and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which can act as external cache memory. By way of example, and not limitation, RAM can be obtained in various forms, such as synchronous RAM (DRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The storage devices disclosed herein are intended to include, but are not limited to, these and other suitable types of memory.

[0095] The various exemplary logic blocks, modules, and circuits described herein can be implemented or performed using the following components designed to perform the functions herein: general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP, and / or any other such configuration.

[0096] In summary, this invention proposes an improved LRC code method for hot and cold data, a computer device, and a storage medium. Combining LRC and the division of hot and cold data, it proposes a scheme to accelerate the recovery (degraded read) of multiple hot data zones in a storage array when there is a hot and cold data division based on the specific usage scenario of user data. The scheme improves LRC encoding, and when multiple hot data zones are simultaneously damaged, it has a certain speed-up effect on the degraded read of multiple hot data zones.

[0097] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0098] It should be understood that, as used herein, the singular form "a" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the associatedly listed items. The embodiment numbers disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0099] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. An improved method for LRC codes of hot and cold data, characterized in that, Includes the following steps: Step 1: Generate a global checksum according to RS encoding; Step 2: Determine if hot data exists. If hot data exists, proceed to Step 4; if no hot data exists or the user does not distinguish between hot and cold data, proceed to Step 3. Step 3: When there is no hot data or the user does not distinguish between hot and cold data, directly perform traditional (k,l,m)LRC encoding on the basis of (k,m)RS encoding, divide the k data disks into l groups, and XOR the disks in each group to generate l local check codes. Step 4: When hot data exists, distinguish between hot and cold data based on the user's actual work scenario and the definition of read / write frequency; When dividing hot and cold zones, if the number of disks containing hot data is less than or equal to the number of groups in the LRC, the number of disks in the hot zone will be grouped so that the hot data disks are distributed in different groups of the LRC. When the number of hot data disks is greater than the number of LRC groups, the number of disks in the hot zone is grouped so that each LRC group contains one hot data disk, and then the remaining hot data disks are placed in the same LRC group as much as possible.

2. The method for improving LRC codes for hot and cold data according to claim 1, characterized in that, The global check code is generated according to the RS encoding. When the RS encoding is (6,2)RS encoding, the (6,2)RS erasure coding process based on the Vandermonde matrix is ​​as follows: 。 3. The method for improving LRC codes for hot and cold data according to claim 2, characterized in that, When there is no hot data or the user does not distinguish between hot and cold data, traditional (6,2,2) LRC encoding is performed based on (6,2) RS encoding. The six data disks are divided into two groups, and XOR encoding is performed within each group to generate two local check codes. The traditional (6,2,2) LRC encoding process is as follows: 。 4. The method for improving LRC codes for hot and cold data according to claim 3, characterized in that, When hot data exists When dividing data into hot and cold zones, cold zones are disks with relatively low read / write frequency thresholds, and hot zones are disks with relatively high read / write frequency thresholds. Hot zones are set to... The cold zone is set as .

5. The method for improving LRC codes for hot and cold data according to claim 1, characterized in that, When the number of disks for hot data is less than or equal to the number of groups in the LRC, the data blocks... The global verification block is and At this time, the hot zone is If the cold area is the remaining disk space, then: Based on the (6,2)RS encoding, an improved (6,2,2)LRC encoding is implemented to address hot zones. The data is divided into two groups, and the improved (6,2,2) LRC code is: 。 6. The method for improving LRC codes for hot and cold data according to claim 5, characterized in that, Hot Zone Repair is needed; decoding should be performed based on the traditional (6,2,2) LRC code, and the data should be read. Six disks, decoded in the Galois domain according to Formula I: Ⅰ； If the improved (6,2,2) LRC code is decoded, the following can be read: Six disks, decoded according to Formula II: Ⅱ。 7. The method for improving LRC codes for hot and cold data according to claim 6, characterized in that, When the number of disks containing hot data is greater than the number of LRC groups, data blocks The global verification block is and The hot zone is The cold zone is the remaining disk space: ； If the hot zone is Repair is needed; decode using traditional (6,2,2) LRC code and read... Six disks are decoded in the Galois domain according to formulas III and IV; Ⅲ； If the hot zone is It needs to be fixed by decoding using the improved (6,2,2) LRC code and reading... Six plates, according to get Decode according to Formula IV: Ⅳ。 8. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, which, when executed by the processor, performs the LRC code improvement method for hot and cold data as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The system stores computer program instructions, which, when executed, implement the LRC code improvement method for hot and cold data as described in any one of claims 1-7.

Images