Extensions for generic EVENODD encoding

Abstract

A computer-implemented method for an extension of a general EVENODD encoding includes encoding an array having (p‑1)×k data symbols into a p×(k+r) array. The method includes p being a prime number, r≥1, and k≤p. The method includes each column in the p×(k+r) array having even parity, and the symbol i in the r+jth column is the XOR of the symbols in a line with slope j modulo p in a ring topology in the k columns starting from the symbol i in column 0, where 0≤i≤p‑1 and 0≤j≤r‑1.

Description

Technical Field

[0001] The present invention relates to erasure correcting codes, and more particularly, the present invention relates to extensions for the general EVENODD code. Background Art

[0002] Reliable data transmission is an important aspect of data storage systems. Error detection and correction mechanisms are used to detect errors in data transmission and reconstruct the data when errors are detected. Error detection and correction mechanisms are particularly important for transmitting data over unreliable communication channels and / or channels that are susceptible to noise. Data redundancy mechanisms (such as parity calculations) can achieve reliable data transmission by adding redundant and / or additional data to the message using error detection and correction techniques. The redundant and / or additional data can be used to check the consistency of the message with the original message.

[0003] The classical Blaum-Roth (BR) code with r parity columns comprises a (p-1)×(k+r) (binary) array, where p is a prime number and k≤pr. Each line with slope i has even parity when the line is circular, where 0≤i≤r-1. BR codes are a family of array codes about GF(q) for correcting multi-phase burst errors and erasure. Each phased burst may correspond to an erroneous and / or erased column in the code array. An array is said to be erroneous if at least one of the multiple bits in a column is inverted. If the index of the erroneous column is known, erasure is performed on that column.

[0004] BR codes are geometrically characterized by parity constraints along certain diagonals in each code array. BR codes have an efficient decoding algorithm that avoids multiplication and / or division of the extension field. BR codes replace these operations with circular vector shifts about GF(q).

[0005] The EVENODD array code uses parity along multiple lines and provides independent parity when updating data symbols to minimize the number of updates in parity. Each storage node can be logically arranged as a single column. The data column can be k=p, where p is a prime number. The EVENODD code uses XOR operations for encoding and decoding to reduce processing time and cost compared to finite field operations.

[0006] Error detection and / or correction codes are limited in the number of bits that the code can detect and / or correct in a data stream. Typically, the costs associated with error detection and / or correction mechanisms are high, especially for error detection and / or correction mechanisms that provide relatively more bits of detection and / or correction capability. Summary of the invention

[0007] A computer-implemented method according to one approach includes encoding an array having (p - 1)×k data symbols into a p×(k + r) array. The method includes p being a prime number, r≥1 and k≤p. The method includes each column in the p×(k + r) array having even parity, and the symbol i in the r + j-th column being the XOR of the symbols in the line with slope j modulo p in k columns starting from the symbol i in column 0, where 0≤i≤p - 1 and 0≤j≤r - 1. The method provides the benefits of extended generalized independent parity codes and optimizes the number of XOR operations for encoding data.

[0008] The method may optionally include r = 2. The method provides the advantage of optimizing the number of XOR operations for encoding data, especially when k << p.

[0009] The method may optionally include decoding a p×(k + 2) array. Decoding the p×(k + 2) array includes recovering an erasure pattern by iteratively recovering at most one erased symbol in each column and at most one erased symbol in each group of k + 1 symbols. The method also includes setting the erased columns i and j to zero, where 0≤i<j≤k - 1. The method includes calculating the syndrome vector S 0 as the XOR of the k data columns and the first parity column, and calculating the syndrome vector S 1 as the XOR of the first data column, the second data column rotated once, the third data column rotated twice, up to the k-th data column rotated k - 1 times and the second parity column. The method also includes: calculating the vector V as the XOR of the syndrome vector S 0 and the syndrome vector S rotated p - i times 1 , and calculating the erased column C j as the solution of the recurrence (1⊕α j-i )C j =V. According to the method, (1⊕α j-i )C j represents the XOR of the vector C rotated j - i times j . The method includes calculating the erased column C i as the XOR of the vector C j and S 0 , and outputting the decoded p×(k + 2) array. The method provides the benefit of optimizing the number of XOR operations, which is less costly than finite field operations and faster in processing speed.

[0010] According to another approach, a computer program product for extending a generalized EVENODD code includes a computer-readable storage medium having program instructions. The program instructions are executable by a computer. The program instructions cause the computer to perform the前述 method.

[0011] According to another embodiment, the system includes a processor and logic integrated with the processor, executable by the processor, or integrated with the processor and executable by the processor. The logic is configured to execute the aforementioned method.

[0012] Other aspects and configurations of the present invention will become apparent from the following detailed description, which, taken in conjunction with the accompanying drawings, illustrates by way of example the principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A network architecture according to one configuration is shown.

[0014] Figure 2 It is shown that according to one method, Figure 1 A representative hardware environment associated with a server and / or client.

[0015] Figure 3 A hierarchical data storage system according to one approach is shown.

[0016] Figure 4 is an exemplary array according to one approach, where p=5, k=2, and r=3.

[0017] Figure 5 is an exemplary 4×5 array in a BR code according to one approach.

[0018] Figure 6 is an exemplary array of parities of slopes 0, 1, and 2 included in EIP(5,3) according to one approach.

[0019] Figure 7 are exemplary elements in EIP(5,3) according to one approach.

[0020] Figure 8 is a 4×5 array to be encoded according to one approach.

[0021] Fig. 9 is based on a method through Figure 8 The bits in each column are XORed to calculate the vertical parity of the data.

[0022] Fig.10 is a method of using the disclosed method according to a method Figure 8 The final encoded array.

[0023] Fig.11 is a table comparing the number of XOR operations of the optimized encoding algorithm for EBR(p,k,2) and the number of XOR operations of the encoding algorithm for EIP(p,k,r) according to one approach. DETAILED DESCRIPTION

[0024] The following description is for the purpose of illustrating the general principles of the present invention and is not intended to limit the inventive concept claimed herein. In addition, the specific features described herein can be used in combination with other described features in each of various possible combinations and permutations.

[0025] Unless otherwise explicitly defined herein, all terms are given the broadest interpretation, including the meanings implicit from the specification and the meanings understood by those skilled in the art and / or as defined in dictionaries, monographs, etc.

[0026] It should also be noted that, unless otherwise specified, as used in the specification and the appended claims, the singular forms "a", "an", and "the" include plural referents. Further, it will be understood that when the terms "comprises" and / or "comprising" are used in this specification, they specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0027] The following description discloses several preferred configurations of systems, methods, and computer program products for extending the general EVENODD code.

[0028] In a general manner, a computer-implemented method includes encoding an array having (p - 1)×k data symbols into a p×(k + r) array. The method includes that p is a prime number, r ≥ 1, and k ≤ p. The method includes that each column in the p×(k + r) array has even parity, and the symbol i in the (r + j)-th column is the XOR of the symbols in the line with slope j modulo p among k columns starting from the symbol i in column 0, where 0 ≤ i ≤ p - 1 and 0 ≤ j ≤ r - 1. The method provides the benefits of extending the general independent parity code and optimizes the number of XOR operations for encoding data.

[0029] In an alternative manner, the method may include r = 2. The method provides the benefit of optimizing the number of XOR operations, especially when k << p.

[0030] In another alternative manner, it includes decoding a p×(k + 2) array. Decoding the p×(k + 2) array includes recovering an erasure pattern by iteratively recovering at most one erased symbol in each column and at most one erased symbol in each group of k + 1 symbols. The method further includes setting the erased columns i and j to zero, where 0 ≤ i < j ≤ k - 1. The method includes calculating the syndrome vector S 0 as the XOR of the k data columns and the first parity column, and calculating the syndrome vector S 1The method further includes: calculating the vector V as the syndrome vector S 0 and the syndrome vector S rotated pi times 1 XOR of, and calculate the erased column C j is a recursive formula (1⊕α j-i )C j =V solution. According to this method, (1⊕α j-i )C j Represents the vector C rotated ji times j The method includes calculating the erasure column C i is the vector C j and S 0 The method provides the benefit of optimizing the number of XOR operations, which is cheaper and faster to process than finite field operations.

[0031] In another general manner, a computer program product for extending the general EVENODD code includes a computer-readable storage medium having program instructions. The program instructions can be executed by a computer. The program instructions cause the computer to perform the aforementioned method.

[0032] In another general approach, a system includes a processor and logic integrated with the processor, executable by the processor, or integrated with the processor and executable by the processor. The logic is configured to perform the aforementioned method.

[0033] Figure 1 An architecture 100 is shown according to one approach. Figure 1 As shown in FIG. 1 , a plurality of remote networks 102 are provided, including a first remote network 104 and a second remote network 106. A gateway 101 may be coupled between the remote network 102 and the neighboring network 108. In the context of the present architecture 100, the networks 104, 106 may each take any form, including but not limited to a local area network (LAN), a wide area network (WAN) such as the Internet, a public switched telephone network (PSTN), an internal telephone network, and the like.

[0034] In use, gateway 101 serves as an entry point from remote network 102 to adjacent network 108. As such, gateway 101 may function as both a router, which is capable of directing a given data packet arriving at gateway 101, and a switch, which provides the actual path in and out of gateway 101 for a given packet.

[0035] Also included is at least one data server 114, which is coupled to the adjacent network 108 and can be accessed from the remote network 102 via the gateway 101. It should be noted that (multiple) data servers 114 can include any type of computing device / component. Multiple user devices 116 are coupled to each data server 114. The user devices 116 can also be directly connected through one of the networks 104, 106, 108. Such user devices 116 can include desktop computers, laptop computers, handheld computers, printers, or any other type of logic. It should be noted that in one approach, the user devices 116 can also be directly coupled to any network.

[0036] A peripheral device 120 or a series of peripheral devices 120 (e.g., a fax machine, a printer, a networked and / or local storage unit or system, etc.) may be coupled to one or more of the networks 104, 106, 108. Note that the database and / or additional components may be used with or integrated into any type of network element coupled to the networks 104, 106, 108. In the context of this specification, a network element may refer to any network component.

[0037] According to some embodiments, the methods and systems described herein can be implemented with and / or on a virtual system and / or a system that simulates one or more other systems (such as a UNIX system that simulates an IBM z / OS environment, a UNIX system that virtually hosts a MICROSOFT WINDOWS environment, a MICROSOFT WINDOWS system that simulates an IBM z / OS environment, etc.) In some configurations, the virtualization and / or simulation can be enhanced by using VMWARE software.

[0038] In further embodiments, one or more networks 104, 106, 108 may represent a cluster of systems commonly referred to as a "cloud." In cloud computing, shared resources such as processing power, peripherals, software, data, servers, etc. are provided to any system in the cloud on an on-demand basis, allowing services to be accessed and distributed across many computing systems. Cloud computing typically involves Internet connections between systems operating in the cloud, but other technologies for connecting systems may also be used.

[0039] Figure 2 shows a method according to which Figure 1 1 and 12. The diagram shows a representative hardware environment associated with a user device 116 and / or server 114 of a workstation. The diagram shows a typical hardware configuration of a workstation having a central processing unit 210 (such as a microprocessor) and a number of other units interconnected via a system bus 212.

[0040] Figure 2The workstation shown in the figure includes a random access memory (RAM) 214, a read-only memory (ROM) 216, an input / output (I / O) adapter 218 for connecting peripheral devices such as a disk storage unit 220 to the bus 212, a user interface adapter 222 for connecting a keyboard 224, a mouse 226, a speaker 228, a microphone 232 and / or other user interface devices (such as a touch screen and a digital camera (not shown)) to the bus 212, a communications adapter 234 for connecting the workstation to a communications network 235 (e.g., a data processing network), and a display adapter 236 for connecting the bus 212 to a display device 238.

[0041] The workstation may have an operating system such as Microsoft Operating system (OS), MAC OS, UNIX OS, etc. It should be understood that the preferred method can also be implemented on platforms and operating systems other than the above. The preferred method can be written using extensible markup language (XML), C and / or C++ language or other programming languages ​​and object-oriented programming methods. Object-oriented programming (OOP) can be used, which has been increasingly used to develop complex applications.

[0042] Reference now Figure 3 , shows a storage system 300 according to one approach. Note that, according to various configurations, Figure 3 Some of the elements shown in the figure may be implemented as hardware and / or software. The storage system 300 may include a storage system manager 312 for communicating with multiple media and / or drives on at least one higher storage layer 302 and at least one lower storage layer 306. The higher storage layer 302 may preferably include: one or more random access and / or direct access media 304, such as a hard disk in a hard disk drive (HDD), a non-volatile memory (NVM), a solid state memory in a solid state drive (SSD), flash memory, an SSD array, a flash memory array, etc., and / or other media described herein or known in the art. The lower storage layer 306 may preferably include one or more lower performance storage media 308, including sequential access media, such as tapes and / or optical media in a tape drive, slower access HDDs, slower access SSDs, etc., and / or other media described herein or known in the art. One or more additional storage layers 316 may include any combination of storage memory media desired by the designer of the system 300. Furthermore, any upper storage tier 302 and / or lower storage tier 306 may include some combination of storage devices and / or storage media.

[0043] Storage system manager 312 can access the network 310 (such as Figure 3The storage system manager 312 may communicate with the drives and / or storage media 304, 308 on the upper storage tier 302 and the lower storage tier 306 using a storage area network (SAN) as shown in FIG. 1 or some other suitable network type. The storage system manager 312 may also communicate with one or more host systems (not shown) through a host interface 314, which may or may not be a part of the storage system manager 312. The storage system manager 312 and / or any other components of the storage system 300 may be implemented in hardware and / or software and may utilize a processor (not shown) such as a central processing unit (CPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc., for executing commands of a type known in the art. Of course, any arrangement of storage systems may be used, as will be apparent to those skilled in the art after reading this specification.

[0044] In more configurations, the storage system 300 may include any number of data storage layers, and may include the same or different storage media within each storage layer. For example, each data storage layer may include the same type of storage media, such as HDD, SSD, sequential access media (tapes in tape drives, optical disks in optical disk drives, etc.), direct access media (CD-ROM, DVD-ROM, etc.), or any combination of multiple media storage types. In one such configuration, the higher storage layer 302 may include most SSD storage media for storing data in a higher performance storage environment, and the other storage layers including the lower storage layer 306 and the additional storage layer 316 may include any combination of SSD, HDD, tape drives, etc., for storing data in a lower performance storage environment. In this way, more frequently accessed data, data with a higher priority, data that needs to be accessed more quickly, etc. can be stored in the higher storage layer 302, while data that does not have one of these attributes can be stored in the additional storage layer 316, including the lower storage layer 306. Of course, after reading this specification, those skilled in the art can design many other combinations of multiple storage media types according to the configuration proposed herein to implement different storage mechanisms.

[0045] According to some configurations, a storage system such as 300 may include logic configured to receive a request to open a data set, logic configured to determine whether the requested data set is stored in multiple associated parts in a lower storage tier 306 of the tiered data storage system 300, logic configured to move each associated part of the requested data set to a higher storage tier 302 of the tiered data storage system 300, and logic configured to assemble the requested data set from the associated parts on the higher storage tier 302 of the tiered data storage system 300.

[0046] Of course, according to various configurations, the logic may be implemented as a method or computer program product on any device and / or system.

[0047] Error detection and / or correction codes are limited in the number of bits that the code can detect and / or correct in a data stream. Typically, the costs associated with error detection and / or correction mechanisms are significant, especially for error detection and / or correction mechanisms that provide relatively more bit detection and / or correction capabilities.

[0048] In various configurations, array codes can be used in the cloud. In some approaches, each entry can correspond to an entire device. In some applications, each column represents a device, and each entry represents a sector and / or page in the device. Erasure codes involving local and global parity can be invoked.

[0049] Array codes with vertical parity can provide the benefits of intra-device parity. Extended independent parity (EIP) codes can use only XOR operations, which are low-cost and faster to process, compared to codes using finite field operations. EIP codes can be implemented into software-defined storage systems.

[0050] In various configurations, the EIP code array includes p information columns and r independent parity columns, where r ≥ 1. Each column may include p bits, where p is a prime number. The array may include k rows, where k ≤ p. The array may include any p value (e.g., a prime value) while the r parity columns remain independent of each other. A person of ordinary skill in the art will understand after reading this disclosure that any information columns, rows, parity columns, etc. referred to herein may be logically arranged as such columns, rows, etc.

[0051] In one approach, a classical BR code with r parity columns comprises a (p-1)×(k+r) (binary) array, where p is a prime number and k≤pr. Each line with slope i has parity when the line is looped, where 0≤i≤r-1. For a given set of bits, even parity includes counting the occurrences of bits with a value of 1. If the occurrence of a bit is odd, the parity bit value is set to 1, making the total count of the occurrences of 1 in the entire set even. If the count of 1 is even, the parity bit value is set to 0.

[0052] The various ways, examples, and / or lemmas disclosed herein may be implemented in combination with features from any other ways, examples, and / or lemmas listed herein, such as those described with reference to other figures. However, of course, such ways, examples, and / or lemmas proposed herein may be used in various applications and / or arrangements that may or may not be specifically described in the illustrative configurations listed herein. Furthermore, the various ways, examples, and / or lemmas proposed herein may be used in any desired context.

[0053] Figure 4 is an exemplary array where p=5, k=2 and r=3. The lines shown have slope 0 (e.g., horizontal line), slope 1 and slope 2, respectively. The fifth row of imaginary "0"s is added to facilitate the present description. Each of the lines has even parity (e.g., an even number of 1s) in the classic BR code.

[0054] Figure 5 is an exemplary 4×5 array in BR code. Each line of slope 0, slope 1, and slope 2 has even parity. It should be noted that the last row of 0 is a phantom row that is not written.

[0055] In one approach, an equivalent description of the BR code may include the following algebraic formula. Consider the module M p (x) = 1 + x + x 2+…+x p-1 In at least some embodiments, each column in the array can be viewed as an element in a polynomial ring. Let M p (α) = 0. The Reed-Solomon (RS) type matrix gives the parity check matrix of the [k+r,k] BR code, where k≤pr:

[0056]

[0057] [k+r,k]BR code is in module M p Maximum distance separable (MDS) codes over the polynomial ring of (x) (e.g., in a BR code, any r erasure columns in the array can be recovered). BR codes can be extended by adding additional vertical parities to r parities with slope i, where 0≤i≤r-1 (e.g., extended Blaum-Roth (EBR) codes).

[0058] BR codes and other array codes (including row diagonal parity (RDP) codes, EVENODD codes, etc.) and promotion of the aforementioned codes use parity along multiple lines and may have similar applications. Array codes may be used for architectures of the Redundant Array of Independent Disks (RAID) type (e.g., RAID 6 requiring two parity columns). In some ways, array codes may be used as a substitute for RS codes requiring finite field operations. Traditionally, RS codes are used to correct multiple errors and erasures. Erasures herein may be defined as one or more bits that are unreadable for any reason. RS codes are non-binary cyclic error correction codes that can detect and correct multiple random symbol errors. RS codes using finite field operations are usually complex and expensive to process. Array codes may be based on XOR operations and are usually not as complex as codes based on finite field operations. In various ways, RDP codes optimize the number of XORs used for encoding. In other ways, EVENODD codes provide independent parity to minimize the number of updates in parity when updating data symbols.

[0059] Definition of Extended Blaum-Roth (EBR) Code

[0060] Let p be a prime number, R p modulo 1+x p polynomial ring, and let Even-weighted modulo 1+x p The ideal value of the polynomial. If α p =1, then the extended Blaum-Roth code EBR(p,k,r) is R p (0) The [k+r,k] code on the [k+r,k] code has a parity check matrix given by (1). The EBR code adds vertical parity to the r parities with slope i, where 0≤i≤r-1. The vertical parity can be interpreted as a line with infinite slope. Each line of slope 0, slope 1, and slope 2 in the EBR code has even parity. The (p-1)×p array becomes a p×p array, where p is a prime number. The elements of the BR code can be modulo M p (x) is in the polynomial ring and M p (α) = 0. The elements of the EBR code can be p (0) and α p = 1. Geometrically, the elements of an EBR(p,r) code are p×p arrays such that each column has even parity and any line with slope j has even parity, where 0≤j≤r-1.

[0061] The EBR code may use only the XOR operation which is cheap and fast to process. The EBR code may be implemented into a software defined storage system.

[0062] An EBR code with r column parities can recover any r erasure columns and at most r erasure lines with slope j, where 0≤j≤r-1, and the column parities are protected by other parities. EBR codes can recover erasures without having to call data from the entire storage system and without referencing other columns. Traditionally, the farther the data is extracted from the network, the higher the cost of error correction and / or detection. For error correction and / or detection mechanisms, it is desirable to have a local recoverable property.

[0063] Definition of Extended Independent Parity (EIP) Code

[0064] For k ≤ p, in the modulo M with the independent parity symbols of the generalized EVENODD code (denoted by IP(p,k,r)) p The [k+r,k] code over the polynomial ring of (x) is defined by the following parity check matrix

[0065]

[0066] Let p be a prime number, R p modulo 1+x p polynomial ring, and let R p (0) Even-weighted modulo 1+x p The ideal value of the polynomial. If α p = 1, then the extended independent parity (EIP) code EIP(p,k,r) is about R p (0) The [k+r,k] code of , whose parity check matrix is ​​given by (2). The EIP code adds additional vertical parity to the array.

[0067] In one approach, the EIP code may encode (p-1)×k symbols into a p×(k+r) array with r parity columns, where p is a prime number, and k≤p and r≥1. Each symbol i in column r+j is the XOR of the symbols in the line with slope j modulo p in a ring topology in the k columns starting from symbol i in column 0, where 0≤i≤p-1 and 0≤j≤r-1. Each column in the p×(k+r) array includes even parity.

[0068] In various configurations described herein, for a given set of bits, even parity includes counting the occurrences of bits whose value is 1. If the occurrences of the bits are odd, then the parity bit value is set to 1, making the total count of the occurrences of 1 in the entire set even. If the count of 1 is even, then the parity bit value is set to 0.

[0069] In one approach, parity rows may be appended to a (p-1)×k array to form a p×k array. In one aspect, encoding the (p-1)×k array symbols into a p×(k+r) array includes computing r syndrome vectors of length p. The symbol of syndrome vector i is the XOR of the symbols in each line p of slope i in the p×k array, where 0≤i≤r-1. Encoding may include appending r parity columns to the p×k array, where the r syndrome vectors correspond to the r parity columns, and outputting a p×(k+r) encoded array. The p×(k+r) encoded array may be output in any manner known in the art.

[0070] Figure 6 is an exemplary array of parities with slopes of 0, 1, and 2 included in EIP (5, 3), where p=5 and r=3. Each vertical line has even parity. The array includes 5 logically arranged data columns and 3 logically arranged parity columns. The first parity column stores horizontal parity. The second parity column stores diagonal parity with a slope of 1. The third parity column stores diagonal parity with a slope of 2. Each parity column may be independent of the other parity columns. Independent parity columns may not interact with other parity columns. Independent parity columns may be a function of the data in the data column.

[0071] Figure 7 is an exemplary element in EIP (5,3), where each column includes an even number of 1s. For the first 5 data columns and the first parity column (e.g., independent of the remaining two parity columns), these 6 columns include horizontal parity. The additional parity columns are independent of the first parity column, and the additional parity columns are independent of each other.

[0072] It will be appreciated by those skilled in the art that the parameters of the array may be extended to more parameters than those provided in the various examples disclosed herein. For example, various variables (e.g., m, n, p, b, k, r, etc.) can be any suitable value. In one approach, the array can include any p value if the r parity columns remain independent of each other.

[0073] In various ways, a code IP(p,k,r) is valid if and only if it is in the module M p When the MDS is on the polynomial of (x), EIP(p,k,r) is R p ( 0 ). The code EIP(p,k,r) is an MDS where 1≤r≤3. For r>3, the code is an MDS responsive to a selected prime number. For an MDS array code with independent parity symbols, EIP(p,k,r) is an MDS and a list of prime numbers for which r≥4 is known in the art.

[0074] For any integer A unique integer m can be given by Represents an integer such that 0≤m≤p-1 and m≡l(mod p). You can use Interchange representation.

[0075] Encoding EIP(p,k,r)

[0076] Consider a (p-1)×k data array (a u,v ), 0≤u≤p-2 and 0≤v≤k-1, where p is a prime number and 1≤k≤p. For each j, 0≤j≤k-1, calculate

[0077]

[0078] Using C j represents the columns of the encoded p×(k+r) array, where 0≤j≤k+r-1. For 0≤j≤k-1, C can be obtained using (3) j In order to obtain C k+v , 0≤v≤r-1, let

[0079] Where 0≤v≤r–1(4)

[0080] The encoding is completed. The encoding algorithm optimizes the number of XOR operations. Any known counting algorithm can be used to determine the number of XORs as kp(r+1)-2k-rp.

[0081] Example 1

[0082] Consider the EIP(5,5,3) code which consists of a 5×8 array. Figure 8 is the 4×5 array to be encoded. The spaces indicate the positions to be encoded by the encoder.

[0083] According to the first step as described above, by performing an XOR operation on the bits in each column to obtain Fig. 9 The array shown in Figure 1 is used to calculate the vertical parity of the data. The result is an even number of 1s in each column.

[0084] According to the second step as described above, the first parity column is calculated to be the XOR of the first k=5 columns (e.g., columns 0 to 4); the second parity column is the XOR of the first data column, the second data column rotated once, the third data column rotated twice, the fourth data column rotated three times, and the fifth data column rotated four times; and the third parity column is the XOR of the first data column, the second data column rotated twice, the third data column rotated four times, the fourth data column rotated six times, and the fifth data column rotated eight times, as shown below:

[0085]

[0086]

[0087]

[0088] Fig.10 The above method is used Figure 8 The final encoded array of . In the case of k=p=5 and r=3, a total of 75 XOR operations are performed. In various ways, the total number of XOR operations using the aforementioned method is less than the total number of XOR operations using the conventional technique.

[0089] Lemma 1

[0090] If and only if the code IP(p,k,r) is about the polynomial modulo M p (x), the code EIP(p,k,r) is about R p (0) The MDS is shown below.

[0091] Let p be a prime number, r ≥ 1 and consider the p × (k + r) array Each column in R p (0) Consider the (p-1)×(k+r) array It is defined as

[0092] where 0≤i≤p–2 and 0≤j≤k+r–1 (5)

[0093] If and only if the array When in IP(p,k,r), the array Located in EIP(p,k,r).

[0094] Example 2

[0095] Consider the following 5×8 array in EIP(5,5,3)

[0096]

[0097] The corresponding array in IP(5,5,3) given by the previous lemma is given by

[0098]

[0099] Complemented The columns include odd parity. The remaining columns include even parity.

[0100] As will be understood by one of ordinary skill in the art upon reading this disclosure, when IP(p,k,r) is an MDS, at most r erasures in IP(p,k,r) may be decoded. arrive The transformation (5) provides a general decoding algorithm for EIP(p,k,r), as described below.

[0101] Decode EIP(p,k,r)

[0102] In various approaches, decoding the p×(k+r) array includes recovering at most one erased symbol in each column and at most one erased symbol in each group of k+1 symbols. A group of k+1 symbols includes k symbols in a line with slope j, where 0≤j≤r-1. In one approach, recovery starts with symbol i of the first column of the p×(k+r) array and symbol i of parity column r+j.

[0103] In various configurations, recovering at most one erased symbol in each column may indicate recovering the erased symbol in any column having exactly one erased symbol. Similarly, recovering at most one erased symbol in each group of k+1 symbols may indicate recovering one erased symbol in any group of k+1 symbols having exactly one erased symbol.

[0104] In another approach for decoding a p×(k+r) array, decoding includes assuming that the code IP(p,k,r) is MDS and that ρ≤r columns are erased. The remaining n+k-ρ columns each include at most one erasure. Each column having exactly one erased symbol can be corrected by obtaining the erased symbol as the XOR of the remaining symbols in the column. Assuming that the erased columns are initially set to zero, a new (p-1)×(k+r) array can be constructed by complementing the columns whose last symbol is equal to 1 and eliminating the last row of the resulting array. Decoding can also include correcting the ρ erased columns in IP(p,k,r) by applying the decoding algorithm in IP(p,k,r) to the p×(k+r) array, appending a row of 0 to the decoded array to obtain a p×(k+r) array, and complementing each column of odd weight in the p×(k+r) array. The p×(k+r) array can be output in any manner known in the art.

[0105] Consider the p×(k+r) array Assume that column 0≤j 0 <j 1 <... <j ρ-1 ≤k+r-1 is erasure, where ρ≤r and EIP(p,k,r) is MDS. Assume where 0≤u≤ρ-1 and according to (5) we obtain Decoding IP(p,k,r) The erasure position 0≤j 0 <j 1 <... <j ρ-1 ≤k+r-1. For each 0≤u≤ρ-1, we obtain

[0106]

[0107] Where 0≤i≤p-2. (7)

[0108] Output decoded array

[0109] Example 3

[0110] Consider the 5×8 array in EIP(5,5,3) in the MDS code

[0111]

[0112] The three blank columns (e.g., columns 1, 3, and 6) correspond to the locations to be erased. Perform the transformation given by (5) to obtain

[0113]

[0114] After reading this disclosure, the array in IP(5,3) is decoded using techniques known to one of ordinary skill in the art.

[0115]

[0116] The corresponding array in EIP(5,5,3) given by (6) and (7) is

[0117]

[0118] to complete the decoding.

[0119] Update the algorithm used for EIP(p,k,r)

[0120] Updating the code for EIP(p,k,r) involves relatively less processing power than conventional updating methods. Conventional methods for updating symbols require flipping most or all symbols in the code and / or recursion. The present updating algorithm uses significantly fewer operations. Specifically, each time a data symbol a is updated in EIP(p,k,r), i,j When 0≤i≤p-2, 0≤j≤k-1, update 2r+1 parity symbols (for example, vertical parity symbol a p-1,j and two symbols in each of the r parity columns).

[0121] Consider the p×(k+r) array Assume that by using b u,v Replace a u,v To update the symbol a u,v , where 0≤u≤p-2 and 0≤v≤k-1, for example a u,v ←b u,v .

[0122] Make the following updates:

[0123]

[0124] Where 0≤s≤r-1 (9)

[0125] Where 0≤s≤r-1. (10)

[0126] Example 4

[0127] Consider the previous 5×8 array

[0128]

[0129] In various embodiments, the symbol may be binary. In other embodiments, the symbol may be non-binary. In yet another embodiment, the symbol may be a combination of binary and non-binary. 2,3 =1 Update a 2,3 =0, where a 2,3 ⊕b 2,3 =1,

[0130] a 4,3 ←a 4,3 ⊕1=0

[0131] a 2,5 ←a 2,5 ⊕1=0

[0132] a 4,5 ←a 4,5 ⊕1=0

[0133] a 0,6 ←a 0,6 ⊕1=1

[0134] a 2,6 ←a 2,6 ⊕1=0

[0135] a 0,7 ←a 0,7 ⊕1=1

[0136] a 3,7 ←a 3,7⊕1=0.

[0137] Then, the updated array is

[0138]

[0139] Special cases for encoding and decoding EIP(p,k,2)

[0140] The encoding EIP(p,k,2) follows directly from the encoding of EIP(p,k,r), as detailed above. For the special case of k≤p-2 for a p×(k+2) array, the [k+2, k] code is the encoding of R p (0) When k≤p-2, the number of XOR operations used to encode EIP(p,k,2) is 3kp-2(k+p).

[0141] In various ways, encoding EIP(p,k,2) includes: calculating a first parity column, where the first parity column is the XOR of the k column; and calculating a second parity column, where the second parity column is the XOR of the first data column, the second data column rotated once, the third data column rotated twice, until the kth data column rotated k-1 times.

[0142] Fig.11 1 is a table 1100 comparing the number of XOR operations for the enhanced encoding algorithm for EBR(p,k,2) and the encoding algorithm for EIP(p,k,2). The enhanced encoding for EBR(p,k,2) includes 3kp-(k+2) XOR operations. In addition, encoding EIP(p,k,2) includes 3kp-2(k+p) XOR operations. As shown, for the same value of k, the encoding algorithm for EIP(p,k,2) is more efficient.

[0143] As shown in the figure, the encoding algorithm for EIP(p,k,2) contains fewer XOR operations than the enhanced encoding algorithm for EBR(p,k,2). The reduction of XOR operations is most obvious when k<<p.

[0144] The following lemma gives the recursion for decoding EIP(p,k,2).

[0145] Lemma 2

[0146] Let p be a prime number, α p =1, And 1≤j≤p-1. Recursive formula (1⊕α j ) z (α)= x (α) in R p (0) has a unique solution. Specifically, if but

[0147]

[0148] Where 1≤i≤p-1 (12)

[0149] Example 5

[0150] Let p = 7, x (α)=1⊕α 3 ⊕α 4 ⊕α 6 , for example, x 0 =1,x 1 =0,x 2 =0,x 3 =1,x 4 =1,x 5 =0 and x 6 =1. Solve the recursive equation (1⊕α 3 ) z (α)= x (α). According to (11) and (12),

[0151] z 0 =x 2 ⊕x 4 ⊕x 6 =0

[0152] z 3 =z 0 ⊕x 3 =1

[0153] z 6 =z 3 ⊕x 6 =0

[0154] z 2 =z 6 ⊕x 2 =0

[0155] z 5 =z 2 ⊕x 5 =0

[0156] z 1 =z 5 ⊕x 1 =0

[0157] z 4 =z 1 ⊕x 4 = 1, and

[0158] z (α)=α3 ⊕α 4 。

[0159] The recurrence includes XOR operations.

[0160] Decode EIP(p,k,2)

[0161] In various ways, decoding a p×(k + 2) array includes recovering an erasure pattern by iteratively recovering at most one erased symbol in each column and recovering at most one erased symbol in each group of k + 1 symbols. The erased columns i and j can be set to 0, where 0 ≤ i < j ≤ k - 1. Decoding can include computing a syndrome vector S 0 as the XOR sum of k data columns and the first parity column to compute the syndrome vector S 1 as the XOR of the first data column, the second data column rotated once, the third data column rotated twice, up to the kth data column rotated k - 1 times, and the second parity column. Decoding can include computing a vector V as the syndrome vector S 0 XORed with the syndrome vector S 1 rotated p - i times; computing the erased column C j as the solution to the recurrence (1⊕α j-i )C j = V, where (1⊕α j-i )C j represents the XOR of the vector C j rotated j - i times; and computing the erased column C i as the XOR of the vector C j and S 0 . The decoded p×(k + 2) array can be output in any manner known in the art.

[0162] In various configurations, recovering at most one erased symbol in each column can indicate recovering the erased symbol in any column having exactly one erased symbol. Similarly, recovering at most one erased symbol in each group of k + 1 symbols can indicate recovering the erased symbol in any group of k + 1 symbols having exactly one erased symbol.

[0163] Consider a p×(k + 2) array (a u,v ) ∈ EIP(p,k,2), 0 ≤ u ≤ p - 1 and 0 ≤ v ≤ k + 1, where p is a prime number and 1 ≤ k ≤ p.

[0164] Assume that columns i and j are erased, where 0 ≤ i < j ≤ k + 1. Assume that each of the remaining columns contains at most one erasure. In one method, a column with exactly one erasure can be corrected by XORing the p - 1 remaining symbols in the column. Each non - erased column has even parity. Assume that columns i and j are zero.

[0165] Represent the columns as C v , where 0 ≤ v ≤ k + 1 and compute the syndrome

[0166]

[0167]

[0168] Four different cases are described below:

[0169] 1) For i = k and j = k + 1 (e.g., two parity columns are erased): Then C k = S 0 and C k+1 = S 1 .

[0170] 2) For i < k and j = k + 1 (e.g., a data column and the second parity column are erased): Then C i = S 0 and C k+1 = α i C i ⊕ S 1 .

[0171] 3) For j = k (e.g., the first parity column is erased): Then C i = α -i S 1 and C k = C i ⊕ S 0 .

[0172] 4) For j < k (e.g., two data columns are erased), solve the linear system:

[0173] C i ⊕ C j = S 0

[0174] α i C i ⊕ α j C j = S 1 .

[0175] Solving the system gives

[0176] (1 ⊕ α j-i )C j=S 0 ⊕α -i S 1 (15)

[0177] C i =C j ⊕S 0 (16)

[0178] In calculating S 0 ⊕α -i S 1 Then, we use the recursive formula in Lemma 2 to obtain C in (15): j In calculating C j Then, use (16) to obtain C i .

[0179] Example 6

[0180] Consider the 5×7 array in EIP(5,2), where the empty cells correspond to erasures.

[0181]

[0182] Erasures in columns with exactly one erasure can be recovered by XORing the remaining symbols in the column. Each column includes an even number of 1s. Recovering erasures in columns with exactly one erasure by XORing the remaining symbols in the column results in the following array:

[0183]

[0184] Recover columns i=1 and j=4. Calculate syndrome

[0185]

[0186]

[0187] According to (15)

[0188]

[0189] And solve, since i = 1 and j = 4,

[0190]

[0191] Use (11) and (12). Then x 0 =1,x 1 =0,x 2 =0,x 3 =1 and x 4 =0,

[0192]

[0193]

[0194]

[0195]

[0196]

[0197] Use (16),

[0198]

[0199] The final decoded array is

[0200]

[0201] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0202] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0203] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0204] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0205] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0206] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0207] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0208] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0209] In addition, the system according to various embodiments may include a processor and logic integrated with the processor and / or executable by the processor, which is configured to perform one or more processing steps described herein. The processor may have any configuration described herein, such as a discrete processor or a processing circuit including many components (such as processing hardware, memory, I / O interface, etc.). Integrated with it, it means that the processor has logic embedded therein as hardware logic (such as an application-specific integrated circuit (ASIC), FPGA, etc.). Executable by the processor, it means that the logic is hardware logic, software logic (such as firmware, part of an operating system, part of an application), etc., or some combination of hardware and software logic that the processor can access and is configured to make the processor perform some functions when the processor executes. As known in the art, software logic can be stored in local and / or remote memory of any memory type. Any processor known in the art, such as a software processor module and / or a hardware processor, such as an ASIC, FPGA, a central processing unit (CPU), an integrated circuit (IC), a graphics processing unit (GPU), etc., can be used.

[0210] It will be apparent that the various features of the foregoing systems and / or methods may be combined in any manner, thereby creating multiple combinations from the description presented above.

[0211] It will further be appreciated that embodiments of the present invention may be provided as a service deployed on behalf of a client to provide services on demand.

[0212] Although various embodiments have been described above, it should be understood that they are exemplary only and not restrictive.Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

1. A computer-implemented method comprising: Encoding an array having (p-1)×k data symbols into a p×(k+r) array for extending an independent parity check code and optimizing an XOR operation for encoding data stored in a storage system, the storage system comprising a plurality of storage nodes, Among them, p is a prime number, Among them, r ≥ 1, Where k≤p, wherein each column in the p×(k+r) array has even parity, wherein each storage node is logically arranged as a single column, where symbol i in column r+j is the XOR of symbols in a line with slope j modulo p in a ring topology in k columns starting from symbol i in column 0, where 0≤i≤p-1 and 0≤j≤r-1; Outputting a p×(k+r) code array to at least some of the plurality of storage nodes, wherein the outputted p×(k+r) code array is configured to detect erasure-corrected data.

2. The computer-implemented method of claim 1 , wherein: Parity rows are appended to the (p-1)xk array to form a pxk array.

3. The computer-implemented method of claim 2, comprising: r syndrome vectors of length p are calculated, wherein the sign of the syndrome vector i is the XOR of the sign of each of the p lines with slope i in the p×k array, wherein 0≤i≤r-1.

4. The computer-implemented method of claim 3, comprising: appending r parity columns to the p×k array, wherein r syndrome vectors correspond to the r parity columns; and Output the p×(k+r) encoded array.

5. The computer-implemented method of claim 4, wherein: r＝2。 6. The computer-implemented method of claim 5, comprising: Calculating a first parity column, wherein the first parity column is an XOR of k data columns; and A second parity column is calculated, wherein the second parity column is an XOR of the first data column, the second data column rotated once, the third data column rotated twice, and the kth data column rotated k-1 times.

7. The computer-implemented method of claim 1 , comprising: Decoding the p×(k+r) array, wherein the decoding comprises: Recover at most one erased symbol in each column; and In each group of k+1 symbols, at most one erased symbol can be recovered. wherein the group of k+1 symbols comprises k symbols in a line with slope j, where 0≤j≤r-1, The recovery starts from the symbol i of the first column of the p×(k+r) array and the symbol i of the parity column r+j.

8. The computer-implemented method of claim 7, wherein: The decoding includes: Restore erasure mode by iteratively performing the following operations: Recover at most one erased symbol in each column; and At most one erased symbol is recovered in each group of k+1 symbols.

9. The computer-implemented method of claim 8, wherein: r＝2。 10. The computer-implemented method of claim 6, comprising: Decoding the p×(k+2) array, wherein the decoding comprises: Recover the erasure pattern by iteratively recovering at most one erased symbol in each column; Recover at most one erased symbol in each group of k+1 symbols; Set the erased columns i and j to zero, where 0≤i <j≤k-1; Calculate the syndrome vector S0 as the XOR of k data columns and the first parity column; The syndrome vector S1 is calculated as the XOR of the first data column, the second data column rotated once, the third data column rotated twice, and the kth data column rotated k-1 times and the second parity column. Calculate the vector V as the XOR of the syndrome vector S0 and the syndrome vector S1 rotated pi times; Calculate the erased column C j is a recursive formula (1⊕α j-i )C j =V solution, where (1⊕α j-i )C j Represents the vector C rotated ji times j XOR; Calculate the erased column C i is the vector C j XOR with S0; and Output the decoded p×(k+2) array.

11. The computer-implemented method of claim 1 , comprising: Decoding the p×(k+r) array, wherein the decoding comprises: Assume that code IP(p,k,r) is MDS and ρ≤r columns are erasure-corrected, where the remaining n+k-ρ columns each include at most one erasure; Correcting each column with only one erased symbol by obtaining the erased symbol as the XOR of the remaining symbols in the column; Assuming that the erased columns are initially set to zero, construct a new (p-1)×(k+r) array by complementing the columns whose last symbol is equal to 1; Eliminate the last row of the obtained array; Correcting the p erased columns in IP(p,k,r) by applying the decoding algorithm in IP(p,k,r) to the p×(k+r) array; append a row of zeros to the decoded array to obtain a p×(k+r) array; complementing each odd-weight column in the p×(k+r) array; and Output the p×(k+r) array.

12. A computer program product for extending a general EVENODD code, the computer program product comprising a computer-readable storage medium having program instructions embodied thereon, the program instructions being executable by a computer to cause the computer to perform the method of any one of claims 1-11.

13. A system comprising: processor; as well as A logic integrated with the processor and executable by the processor, the logic being configured to perform the method according to any one of claims 1-11.

14. A computer system comprising: processor; A computer-readable storage medium coupled to the processor, the computer-readable storage medium comprising instructions, which when executed by the processor implement the method of any one of claims 1-11.

15. A system, comprising modules respectively used to perform the steps of the method according to any one of claims 1 to 11.

Images