Data processing method and apparatus

Abstract

The present disclosure provides a data processing method and an apparatus. The method includes: obtaining first basic unit including first data where one or more errors occur; calculating first consistency check data of the first basic unit; determining, based on the first consistency check data and second consistency check data, first positions of at least part of the first data in the first basic unit, where the second consistency check data is consistency check data of second basic unit, and the second basic unit includes original data corresponding to the first basic unit; and decoding the first data by using the first positions.

Description

DATA PROCESSING METHOD AND APPARATUSTECHNICAL FIELD

[0001] The present disclosure relates to the field of information coding and encoding, and particularly to a data processing method and an apparatus.BACKGROUND

[0002] In a storage system, an error may occur in valid data due to some factors such as physical nature or processes behind read and write operation and also due to signal saturation and device wear off. In general, encoding and decoding technologies may be adopted to enhance reliability of data, in which data is encoded (e.g., by using error correction code (ECC) or erasure code (EC) encoding technologies) when the data is written and data is decoded (e.g., by using ECC or EC decoding technologies) when the data is read. In this way, when an error occurs in the valid data, decoding technologies may be used to correct and restore data.

[0003] When errors occurred in the valid data exceed the error correction capability of the decoding technologies, the valid data may not be restored, which may cause data loss.SUMMARY

[0004] In view of this, implementations of the present disclosure provide a data processing method and an apparatus, which may improve decoding capability of storage systems, thereby enhancing the reliability of storage systems.

[0005] In a first aspect, a data processing method is described. The method may be applied to a storage device, or a component in the storage device (e.g., a module, a circuit, or a chip in the storage device). The method includes: obtaining a first basic unit including first data where one or more errors occur, where the first basic unit is a unit for data encoding and data decoding; calculating first consistency check data of the first basic unit; determining, based on the first consistency check data and second consistency check data, first positions of at least part of the first data in the first basic unit, where the second consistency check data is consistency check data of second basic unit, and the second basic unit includes original data corresponding to the first basic unit; and decoding the first data by using the first positions.

[0006] By using the first positions of at least part of the first data in the first basic unit, the storage device may restore at least erroneous data of the first basic unit. In this way, decoding capability of the storage device may be improved, and restoration capability of erroneous data of the storage device may be improved, thereby increasing reliability of the storage system.

[0007] In a possible implementation, decoding the first data by using the first positions includes: performing bit flipping on data at the first positions in the first basic unit for correcting at least part of the first data.

[0008] In a possible implementation, the first positions include one or more positions of a portion of the first data, and the method further includes: decoding remaining data in the first data except for the portion of the first data, by using a first decoding means.

[0009] In a possible implementation, the first decoding means includes at least one of: decoding using anerasure code (EC), or decoding using an error correction code (ECC).

[0010] In a possible implementation, calculating the first consistency check data of the first basic unit includes: dividing data in the first basic unit into blocks to obtain a plurality of data blocks; and calculating, based on the plurality of data blocks, the first consistency check data using cyclic redundancy check (CRC) or locally recoverable code (LRC).

[0011] In a possible implementation, before the calculating the first consistency check data of the first basic unit, the method further includes: decoding the first basic unit by using a second decoding means, where the decoding the first basic unit by using a second decoding means fails.

[0012] In this way, in a case where the first basic unit is correctly decoded through the second decoding means, there is no need to perform subsequent steps. Therefore, decoding complexity of the storage device may be decreased, thereby increasing decoding efficiency of the storage device. In a case where decoding the first basic unit through the second decoding means fails, subsequent steps including calculating the first CCD are performed to decode the first basic unit, which may enhance the decoding capability of the storage device, thereby improving the reliability of the storage system.

[0001] In a possible implementation, the second decoding means includes at least one of: decoding using an erasure code (EC); or decoding using an error correction code (ECC).

[0002] In a possible implementation, decoding the first basic unit by using the second decoding means includes: decoding the first basic unit by using the ECC, where the decoding the first basic unit by using the ECC fails; performing intra-disk decoding on the first basic unit by using the EC, where the intra-disk decoding fails; and performing inter-disk decoding on the first basic unit by using the EC, where the interdisk decoding fails.

[0003] Due to the higher processing complexity of EC decoding compared to ECC decoding, and the higher processing complexity of inter-disk decoding compared to intra-disk decoding, in a case where the first basic unit can be correctly decoded through the ECC, there is no need to perform subsequent steps. Therefore, decoding complexity of the storage device may be decreased, thereby increasing decoding efficiency of the storage device. In a case where decoding the first basic unit through ECC fails, the storage device may intra- disk decoding the first basic unit by using the EC. In a case where intra-disk decoding fails, the storage device may perform inter-disk decoding on the first basic unit by using the EC, which may enhance the decoding capability of the storage device, thereby improving the reliability of the storage system.

[0004] In a possible implementation, determining the first positions of the at least part of the first data includes: determining first difference information between the first consistency check data and the second consistency check data; and determining, based on the first difference information and a first mapping, the first positions, where the first mapping indicates a correspondence between a plurality of difference information and a plurality of positions, where the plurality of difference information includes the first difference information.

[0005] The difference information may also be called CCD syndrome, and the first mapping relationship may also be called syndrome table. The first mapping includes multiple positions in a basic unit where erroneous data may occur and a CCD syndrome corresponding to each of the multiple positions.

[0006] In this way, the storage device may obtain the position corresponding to the current difference information by looking up the table, so as to determine the position of the erroneous data in the first basic unit. By looking up the table, the positions of the erroneous data in the first basic unit may be located more quickly, which may improve the efficiency of data decoding.

[0007] In a possible implementation, the plurality of positions are associated with a size of the first basic unit.

[0013] In a possible implementation, the plurality of positions are associated with a size of a data block in the first basic unit.

[0014] In a possible implementation, the method further includes: obtaining a third basic unit in which part of the first data has been corrected, in a case where decoding the first data by using the first positions fails, where the third basic unit includes remaining data of the first data where one or more errors occur; calculating third consistency check data of the third basic unit; determining second positions of the remaining data of the first data based on the third consistency check data and the second consistency check data; and decoding the remaining data of the first data by using the second positions.

[0008] After decoding the first basic unit by using the first positions fails, part of the erroneous data in the first basic unit may be restored. The corrected basic unit may be the third basic unit, so that it is possible to restore other erroneous data by redetermination of the second positions by using the third basic unit.

[0009] In a possible implementation, before obtaining the first basic unit, the method further includes: obtaining the second basic unit; encoding the second basic unit using a first encoding means to obtain an encoded basic unit; and calculating the second consistency check data of the encoded basic unit.

[0010] In a possible implementation, the first encoding means includes at least one of: encoding using an EC; or encoding using an ECC.

[0011] In a possible implementation, the first basic unit is a page.

[0012] In a second aspect, an apparatus is described. The apparatus has a function of implementing the first aspect. For example, the apparatus includes a corresponding module, unit, or means for performing operations in the first aspect. The module, unit, or means may be specifically implemented by using software, may be implemented by using hardware, or may be implemented by using software in combination with hardware.

[0013] In a third aspect, an apparatus is described, which includes: one or more processors; and a memory storing instructions which, when executed by the one or more processors, cause the apparatus to perform any implementation of the first aspect.

[0014] In a fourth aspect, a computer-readable storage medium is described, which has instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform the method in any possible implementation.

[0015] In a fifth aspect, a computer program product is described, which stores instructions which, when executed, cause an apparatus to perform the method in any possible implementation.

[0016] This disclosure encompasses various embodiments, including not only method embodiments, but also other embodiments such as apparatus embodiments and embodiments related to non- transitory computerreadable storage media. Embodiments may incorporate, individually or in combinations, the features disclosed herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG. 1 is a schematic diagram of an example redundant array of independent disks (RAID) storage system;

[0018] FIG. 2 is a schematic diagram illustrating data errors occurring in a stripe in a storage system;

[0019] FIG. 3 is a schematic flow diagram of an example process of joint encoding and decoding using EC and ECC;

[0020] FIG. 4 is a schematic flow diagram of an example joint encoding;

[0021] FIG. 5 is a schematic flow diagram of an example joint decoding;

[0022] FIG. 6 is a schematic diagram illustrating different cases of erroneous pages;

[0023] FIG. 7 is a flow diagram of a data processing method in accordance with some embodiments;

[0024] FIG. 8 is a flow diagram of an encoding process in accordance with some embodiments;

[0025] FIG. 9 is a schematic diagram of an encoding in accordance with some embodiments;

[0026] FIG. 10 is a schematic diagram of a decoding process in accordance with some embodiments;

[0027] FIG. 11 is a schematic diagram illustrating a partial decoding process in accordance with some embodiments;

[0028] FIG. 12 is a block diagram of an apparatus in accordance with some embodiments; and

[0029] FIG. 13 is a block diagram of an apparatus in accordance with some embodiments.DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure are described in details below in conjunction with the drawings.

[0031] To better understand the technical solutions provided by embodiments of the present disclosure, terms involved herein are briefly described below.

[0032] 1. Error Correction Code (ECC)

[0033] ECC is a technique for checking and correcting data errors by adding redundant information into the data. In cases of data transmission and data storage, an encoder may add some redundant information (e.g., parity bits) in original data. The redundant information is not simple duplicates of data, but is designed based on mathematical principles and algorithms, which may provide additional information to help a decoder to check and correct errors. After receiving data, the decoder may perform decoding process on the received data, in which the redundant information is used to check and correct data errors occurred in transmission or storage procedures. ECC is mainly used to correct random errors occurred in data, but it may not be possible to determine the exact position of an error in the data. Such an error may be called "Error".

[0034] For ECC encoding in storage, Hamming codes, Reed-Solomon (RS) codes, low density parity check (LDPC) codes, etc., are commonly used in industry.

[0035] RS codes are non-binary error correction codes or erasure codes. RS codes may operate on finite fields (i.e., Galois field) for calculation, which may correct errors in data. The error tolerance capability ofRS codes is optimal in theory under the same redundancy overhead. RS codes are mainly used as short codes, however operating over the non-binary finite fields (typically of size 256).

[0036] LDPC codes are linear block codes whose core is a sparse matrix (i.e., a matrix with many zero elements) for representing constraint conditions in encoding and decoding processes. LDPC codes are widely used in the field of mass storage as a coding technology with good error correction performance and low decoding complexity. LDPC codes use a sparse parity-check matrix (which may be called H matrix) to define codewords. A decoding process is usually performed using iterative algorithms, such as "belief- propagation". Additionally, LDPC codes allow effective usage of soft input information from channel to perform soft decoding. Although LDPC codes define codewords by their parity-check matric, they still essentially protect the data integrity when the data is decoded, by adding additional parity information.

[0037] 2. Erasure Code (EC)

[0038] EC is a data protection means that may split data into fragments, such as data pages, may expand and encode these data pages by adding redundant pages (also known as parity pages), and then may store them in different positions, such as disks, storage nodes, or other storage positions. In general, N1+M1 EC may be used to represent that Mi parity pages are generated based on Ni data pages, where M1 and N1 are both integers greater than one. This technique aims to improve reliability of data and error tolerance capability, ensuring that the original data may still be recovered if part of the data is lost or corrupted. EC may tolerate the loss or corruption of multiple data pages and may restore the lost or corrupted data pages to original data pages (also known as valid data pages) if the number of lost or corrupted data pages does not exceed the number of parity pages. However, the encoding and decoding processes of EC are relatively complex and require a lot of computation, which may increase the difficulty and cost of implementation when data protection is achieved. In general, EC is used to correct an erasure type of error in the data. Such errors may be called "Erasure". In a case where this type of error occurs, the position of the error in a data page where the error occurred may be obtained through EC. For EC encoding in storage, RS codes or LDPC codes are commonly used in the industry.

[0039] ECC and EC essentially provide additional parity information to improve decoding capabilities of a decoder by increasing storage overhead. ECC and EC may adopt the same encoding scheme, such as RS codes or LDPC codes, but ECC and EC are used to recover data at different granularities. ECC may perform bit-level check and recovery on data in a page to ensure data integrity, while EC may perform page-level check and recovery on data to provide global-level data protection, ensuring data integrity.

[0040] For example, ECC adopts RS codes or LDPC codes, and the encoder may generate M1-bit redundant parity for N1-bit original data in bytes using the RS codes or LDPC codes, where M1 is an integer greater than or equal to 1 and Ni is 8. N1-bit original data and M1-bit parity may constitute a codeword. In a case where the number of erroneous bits occurring in a codeword is less than or equal to Mi, the decoder may decode the codeword using the RS codes or LDPC codes to obtain the original data.

[0041] For example, in a case where EC adopts RS codes or LDPC codes, the encoder may encode N2 data pages using RS codes or LDPC codes to obtain redundant M2 parity pages, where N2 and M2 are all positive integers. In a case where the number of erroneous pages is less than or equal to M2, the decoder may use RScodes or LDPC codes to decode to recover the erroneous pages.

[0042] 3. Stripe

[0043] In striped storage, consecutive data may be divided into small pieces (each of which may be a data page), which may be stored in parallel across disks. In addition, the data page may be divided into data blocks with a same size, which may be dispersedly stored on hard disk drive to improve speed and efficiency of data transmission. A stripe may include multiple data pages. Such stripes may be distributed on different disks in a fixed sequence or algorithm, so that these stripes may be processed in parallel in a case where a storage device reads or writes data, thereby improving overall performance.

[0044] 4. Storage System

[0045] A storage system is a computer system or a set of storage devices for data storage, data management, data protection, and data access. The storage system is an important part of computer systems, and is used for ensuring data persistence, availability, and security. The storage system has the core functionality of storing data on physical or virtual storage media in some manner and effectively retrieving the data when needed.

[0046] Common storage media for storage systems may include hard disk drive (HDD) and solid state drive (SSD), and common storage architecture for storage systems may be, for example, storage area network (SAN), network attached storage (NAS), direct attached storage (DAS), redundant array of independent disks (RAID), etc.

[0047] SAN is a dedicated high-speed storage network that provides block-level storage services. SAN typically use fiber channels to connect servers and storage devices over a dedicated storage network.

[0048] NAS is a storage architecture that provides file-level storage services through standard network protocols. A NAS device is usually connected to an Ethernet network, allowing multiple clients to access shared storage over the network.

[0049] DAS is a storage architecture that connects storage devices directly to a single server or computer. A DAS device is usually connected directly to the host through interfaces. In the DAS architecture, storage devices are directly mounted on the server, making configuration and management relatively easy.

[0050] RAID is a common commercial architecture, which is a disk subsystem that includes independent disk drives. It aims to provide higher storage performance and data redundancy than a single disk. RAID achieves high data availability, high performance and data recovery through specific data distribution and redundancy technologies.

[0051] FIG. 1 illustrates an example RAID storage system. The RAID storage system works in L+P mode (e.g., using L+P EC encoding to perform data storage), where L and P are all positive integers. The RAID storage system includes 4 sets of disks, and each set of disks includes L data disks and P parity disks, where L and P are greater than or equal to 5 and 2, respectively. As shown in FIG. 1, the RAID storage system adopts striping to store data. Each stripe includes L data pages and P parity pages. Pages corresponding to a stripe are stored on a set of disks, in which L data pages are stored on L data disks and P parity pages are stored on P parity disks.

[0052] In the RAID storage system, an error may occur in valid data due to the physical nature or processesbehind read and write operation and also due to signal saturation and device wear off. For example, in the RAID storage system, when one disk in the RAID storage system is damaged, several uncorrectable data errors may occur on several pages on other disks in the RAID storage system at the same time in some stripes (e.g., additional erroneous pages in stripes).

[0053] Usually, encoding and decoding technologies may be used to enhance reliability of data, that is, the data is encoded (such as by using ECC or EC encoding) when the data is written, and the data is decoded (such as by using ECC or EC decoding) when the data is read, so that when an error occurs in valid data, the decoding technology may correct and recover the data. However, when errors occurred in the valid data exceed the error correction capability of the decoding technologies, the valid data may not be recovered, which may lead to data corruption or loss.

[0054] For example, the encoding technology is ECC encoding, and the decoding technology is ECC decoding. In a case where the number of erroneous bits in a codeword does not exceed the number of parity bits, a storage device having the RAID storage system deployed thereon may use ECC decoding to recover the erroneous bits. However, if the number of erroneous bits in a codeword exceeds the number of parity bits, the storage device may not recover the erroneous bits, which may lead to data loss. F or another example, the encoding technology is EC encoding, and the decoding technology is EC decoding. In a case where the number of lost or corrupted data pages does not exceed the number of parity pages, a storage device having the RAID storage system deployed thereon may use EC decoding to recover the lost or corrupted data pages. However, if the number of lost or corrupted data pages exceeds the number of parity bits, the storage device may not recover the corrupted bits, which may lead to data loss.

[0055] FIG. 2 illustrates data errors occurring in a stripe in a storage system. The stripe includes 6 pages, including 5 data pages and 1 parity page, which may be obtained by, for example, 5+1 EC encoding. In the 6 pages, there exist 2 pages where a data error occurs (which may be called erroneous pages), and 4 pages where no data error occurs (which may be called correct pages). According to the description of EC described above, for a stripe adopting 5+1 EC encoding, the storage device cannot recover erroneous pages based on EC decoding if the number of erroneous pages exceeds 1.

[0056] There may also be a joint encoding and decoding method, which may further enhance data decoding capability of storage devices having a storage system deployed thereon. Since encoding and decoding have a correspondence, the storage device uses a certain encoding mode when storing data, so that the storage device needs to adopt the decoding mode corresponding to the encoding mode when obtaining data from the disk. The joint encoding and decoding method is described in details below in conjunction with FIGS. 3 to 5.

[0057] FIG. 3 illustrates an example process of joint encoding and decoding using EC and ECC. As shown in FIG. 3, after obtaining original data, a storage device may encode the original data using EC encoding, then encode the data encoded by EC encoding using ECC encoding, and store the data after ECC encoding on a storage medium. When stored data is read, the storage device may decode the stored data using ECC decoding, then decode the data obtained by ECC decoding using EC decoding, thereby obtaining the original data.

[0058] It is noted that, the joint encoding and decoding shown in FIG. 3 is merely an example, and other types or sequences of joint encoding and decoding methods are also available, which may be selected by a person skilled in the art according to actual requirements of application. It is also noted that, ECC, EC, and a combination of both used in some embodiments of the present disclosure are merely examples, but not limitations for encoding and decoding.

[0059] In the process of data decoding in FIG. 3, when the storage device reads data (e.g., a page), whether there is an error in the read data may be detected. In a case where there is an error in the read data, the storage device, which uses EC decoding and ECC decoding to decode the data having the error, may obtain correct data.

[0060] FIG. 4 illustrates an example joint encoding. As shown in FIG. 4, after obtaining user data, a storage device may encode the user data using data integrity field (DIF) and joint encoding of EC (including intradisk EC and inter-disk EC) and ECC, and may store the encoded data to a storage medium.

[0061] It is noted that, the user data shown in FIG. 4 is processed using DIF first, then the intra-disk EC encoding, the inter-disk EC encoding, and finally the ECC encoding. However, the sequence of encoding modes and the specific modes of the joint encoding are merely examples, which may be selected by a person skilled in the art according to actual requirements of application.

[0062] FIG. 5 illustrates an example joint decoding. For a storage device using EC+ECC joint encoding, in a case where the storage device reads data in a disk, the storage device may decode the data in the disk using ECC and EC. If LDPC codes are adopted, the storage device may perform ECC hard decoding based on LDPC codes on a page having erroneous data (which may be called erroneous page) in a stripe. In a case where the ECC hard decoding based on LDPC code fails, the storage device may perform ECC soft decoding based on LDPC codes on the erroneous page (e.g., by using soft input information from channel). In a case where the ECC soft decoding based on LDPC codes also fails, the storage device may perform EC decoding based on LDPC codes on the erroneous page. In EC decoding based on LDPC code, the storage device may perform intra-disk EC decoding based on LDPC codes on pages in the stripe except for the erroneous page, where the intra-disk EC decoding based on LDPC codes includes intra-disk EC hard decoding based on LDPC codes and intra-disk EC soft decoding based on LDPC codes. Similar to the execution order of ECC hard decoding based on LDPC codes and ECC soft decoding based on LDPC codes, the storage device may perform the inter-disk EC hard decoding based on LDPC codes, and then perform the intra-disk EC soft decoding based on LDPC codes in a case where the intra-disk EC hard decoding based on LDPC codes fails. In a case where the intra-disk EC decoding based on LDPC codes fails, the storage device may perform inter-disk EC decoding based on LDPC codes. That is, pages stored on other disks in the storage system may be used to perform inter-disk EC hard decoding based on LDPC codes and inter-disk EC soft decoding based on LDPC codes. The storage device may perform the inter-disk EC hard decoding based on LDPC codes, and then perform the inter-disk EC soft decoding based on LDPC codes in a case where the inter-disk EC hard decoding based on LDPC codes fails. In a case where a page with data error is successfully decoded in one of the above decoding stages, the data in the page may be restored to the original data, and the storage device does not need to perform the remaining decoding stages after the stage. In a case where all thedecoding stages fail, data loss may occur.

[0063] It should be understood that when reading data, the storage device may verify the data firstly, for example, using Hash verification. If the verification passes, the storage device may determine that there is no erroneous data in the page. If the verification fails, the storage device may determine that there is erroneous data in the page.

[0064] FIG. 6 illustrates different cases of erroneous pages. In an example of a RAID storage system, in a case where the RAID storage system is working in A?+2 EC mode, each stripe may include K data pages and 2 parity pages, where K is a positive integer. In this example, in a case where the number of erroneous pages is 3, a possible distribution of the erroneous pages is as shown in FIG. 6, and may include the following 4 cases:

[0065] case 1, 3 erroneous pages all belong to disk failure (DF), and the percentage of such a case is 6.4%;

[0066] case 2, the first two erroneous pages of the 3 erroneous pages belong to DF, the other one belongs to uncorrectable error (UNC), and the percentage of such a case is 93.5%;

[0067] case 3, the first and third erroneous pages of the 3 erroneous pages belong to DF, and the second erroneous page belongs to UNC; and

[0068] case 4, the first erroneous page of the 3 erroneous pages belongs to DF and the other two erroneous pages belong to UNC. The percentage of each of cases 3 and 4 is 0.1%.

[0069] According to the above examples, the percentage of the case with 2 DF and 1 UNC is the majority.

[0070] The joint encoding and decoding methods may improve decoding capability of the storage device for erroneous data by further increasing parity information of the original data, which may solve the data restoration problem after data errors exceed error correction capability of a single ECC or EC, and reduce the probability of UNC to a certain extent. However, when there is a lot of erroneous data and parity information provided by a joint encoding is not enough to support restoration for erroneous data, data loss may still occur. For example, a joint encoding and decoding method may not solve the data restoration problem corresponding to the error case of 2 DF and 1 UNC shown in FIG. 6. Therefore, the joint encoding and decoding method may not improve the overall reliability of the storage system.

[0071] In view of this, a data processing method and an apparatus are provided in accordance with some embodiments of the present disclosure, which may determine positions of all or part of the erroneous data in a page by comparing consistency check data (CCD) of the data of an erroneous page with the CCD of the corresponding original data, and then decoding the data of the erroneous page by using the positions and a dedicated decoding mode, which may improve the decoding capability of the storage device, improve the recovery capability of the storage device for the error data, and improve the reliability of the storage system.

[0072] The data processing method of the embodiments of the present disclosure may be applied to various storage systems, which may be deployed in storage devices. The storage device may be deployed with different controllers, such as SSD controller, NAND controller, RAID storage controller, etc. The data processing method may be performed by the storage device, a controller in the storage device, or a chip or circuit in the storage device. For the convenience of description, the data processing method will be described below taking the data processing method performed by the storage device as an example.

[0073] FIG. 7 illustrates a data processing method in accordance with some embodiments. As shown in FIG. 7, the method 700 include steps 701 to 704.

[0074] In step 701, a storage device obtains a first basic unit including first data where one or more errors occur, where the first basic unit is a unit for data encoding and data decoding.

[0075] For example, the storage device may obtain the first basic unit by way of Hash verification.

[0076] It should be understood that a basic unit is a unit used for data encoding and data decoding. That is, the granularity of the data decoding is the same as the granularity of the data encoding. In some examples, a page or multiple bytes may be served as a basic unit for data encoding and data decoding.

[0077] In step 702, the storage device calculates first consistency check data (CCD) of the first basic unit.

[0078] CCD may refer to redundant check data for verification of integrity and correctness of pay load data.

[0079] In a possible implementation, the storage device may divide data of the first basic unit into blocks, so that a plurality of data blocks are obtained. The storage device may calculate, based on the plurality of data blocks, the first CCD using cyclic redundancy check (CRC) or locally recoverable code (LRC).

[0080] Because the first basic unit is a basic unit where errors occur, the first CCD obtained based on the first basic unit may reflect erroneous data of the first basic unit.

[0081] In an implementation, the CCD may be calculated based on a respective page, that is, a page corresponds to a respective CCD. In another implementation, the CCD may be calculated based on multiple pages (such as a strip), that is, multiple pages correspond to one CCD.

[0082] In step 703, the storage device determines first positions, in the first basic unit, of at least part of the first data, based on the first CCD and second CCD, where the second CCD is CCD of a second basic unit, and the second basic unit includes original data corresponding to the first basic unit.

[0083] Because the second basic unit includes the original data corresponding to the first basic unit, which means that the second basic unit includes correct data, the second basic unit may be correctly decoded, and the first CCD calculated based on the first basic unit is different from the second CCD. In this way, the storage device may determine the first positions in the first basic unit of at least part of the first data.

[0084] In the data processing method of the embodiments of the present disclosure, one bit may correspond to one position, and the first positions may be multiple positions corresponding to multiple bits. The multiple bits are at least part of the first data mentioned above.

[0085] In some examples, the second CCD here may be pre-calculated and stored, and the first CCD and the second CCD are calculated by a same algorithm. In this way, the accuracy of decoding may be guaranteed.

[0086] In step 704, the storage device decodes the first data using the first positions.

[0087] In a possible implementation, the storage device may perform bit flipping on data at the first positions in the first basic unit for correcting at least part of the first data.

[0088] The first decoding means may further include a data correction action, for example, bit flipping. The bit flipping refers to that " 1 " is changed to "0", or vice versa. Because data is represented in binary, a value of data may be "0" or " 1 ". Thus, in step 704, the storage device may perform the bit flipping on erroneous data at first positions for correcting the erroneous data, thereby facilitating correctly decoding the first basic unit.

[0089] In a possible implementation, the first positions include one or more positions of a portion of the first data, and the method further includes: the storage device decoding remaining data in the first data except for the portion of the first data, by using a first decoding means.

[0090] For example, the first decoding means may also include decoding using EC or ECC, or joint decoding using EC and ECC.

[0091] In the data processing method of the embodiments of the present disclosure, by using the first positions in the first basic unit of at least part of the first data, the storage device may restore at least erroneous data of the first basic unit. In this way, decoding capability of the storage device may be improved, restoration capability of erroneous data of the storage device may be improved, thereby increasing reliability of the storage system.

[0092] For better understanding the solutions of these embodiments, the encoding process of the embodiments is described in details below.

[0093] In a possible implementation, in a case where the original data (i.e., the second basic unit) is stored in the storage system, the storage device may encode the second basic unit using a first encoding means to obtain an encoded basic unit, and calculate and store the CCD (i.e., the second CCD) of the encoded basic unit.

[0094] The first encoding means may include encoding using EC or ECC, or joint encoding using EC and ECC.

[0095] Because a data error may occur in the original data in a case where the original data is stored in a disk, calculating the second CCD when storing data may conducive to the subsequent recovery of the original data.

[0096] The following embodiments are described using the first basic unit as a page as an example.

[0097] FIG. 8 illustrates an encoding process in accordance with some embodiments of the present disclosure. For example, the original data includes Q data pages with a certain size, where Q is a positive integer. As shown in FIG. 8, the storage device may perform Q+M EC encoding on the Q data pages to generate M parity pages, where the data pages and the parity pages have a same page size. After that, the storage device may perform ECC encoding on the Q+M pages encoded by EC encoding to further improve reliability. The storage device may calculate CCD of the Q+M pages encoded by EC encoding, and store the CCD for decoding in future.

[0098] In order to better understand the solutions of the embodiments, an example encoding means adopted is described below.

[0099] FIG. 9 illustrates an example encoding. As shown in FIG. 9, original data includes Page 1, Page 2 and Page 3, where Page 1 and Page 2 are data pages, and Page 3 is a parity page obtained by EC encoding based on Page 1 and Page 2. That is, the storage device uses 2+1 EC encoding. The data pages and parity page have a same data size (or page size), which is 4KB. Each page (data or parity page) has 512 data blocks of 8 Byte. After EC encoding, the storage device may perform ECC encoding and CCD calculation on these 3 pages, respectively. Through ECC encoding, each page will add extra ECC parity data, which may improve the decoding capability by adding extra storage overhead. After calculating CCD based on three pages, a i iCCD with a length of 512 bytes may be obtained. As shown in FIG. 9, since each page has 512 data blocks, an z-th element in the CCD may be generated according to z-th data blocks each in a respective one of the 3 pages, where the value range of i is [1, 512].

[0100] Then, the decoding process of embodiments of the present disclosure is described below, which corresponds to the encoding process described above.

[0101] In a possible implementation, before calculating first consistency check data of the first basic unit, the storage device may use second decoding means to decode the first basic unit. In a case where the decoding the first basic unit by using the second decoding means fails, the storage device may calculate the first CCD, determine the first positions in the first basic unit of at least part of the first data, and decode the first data using the first positions.

[0102] For example, the second decoding means may include decoding using EC or ECC, or joint decoding using EC and ECC.

[0103] In this way, in a case where the first basic unit is correctly decoded through the second decoding means, there is no need to perform subsequent steps. Therefore, decoding complexity of the storage device may be decreased, thereby increasing decoding efficiency of the storage device. In a case where decoding the first basic unit through the second decoding means fails, subsequent steps including calculating the first CCD are performed to decode the first data, which may enhance the decoding capability of the storage device, thereby improving the reliability of the storage system.

[0104] In a case where the second decoding means includes the joint decoding using EC and ECC, the decoding order of the first basic unit may be determined according to the encoding order of the second basic unit when storing the second basic unit.

[0105] In a possible implementation, the step in which the storage device decodes the first basic unit by using the second decoding means includes steps as follows. The storage device may decode the first basic unit by using ECC, where the decoding the first basic unit by using the ECC fails. Then, the storage device may perform intra-disk decoding on the first basic unit by using EC, where the intra-disk decoding fails. Then, the storage device may perform inter-disk decoding on the first basic unit by using EC, where the inter-disk fails.

[0106] Due to the higher processing complexity of EC decoding compared to ECC decoding, and the higher processing complexity of inter-disk decoding compared to intra-disk decoding, in a case where the first basic unit can be correctly decoded through the ECC, there may be no need to perform subsequent steps. Therefore, decoding complexity of the storage device may be decreased, thereby increasing decoding efficiency of the storage device. In a case where decoding the first basic unit through ECC fails, the storage device may perform intra-disk decoding on the first basic unit by using the EC. In a case where intra-disk decoding fails, the storage device may perform inter-disk decoding on the first basic unit by using the EC, which may enhance the decoding capability of the storage device, thereby improving the reliability of the storage system.

[0107] FIG. 10 illustrates a decoding process in accordance with some embodiments. The decoding process shown in FIG. 10 corresponds to the encoding process shown in FIG. 8.

[0108] As shown in FIG. 10, the storage device reads a first basic unit including the first data where one ormore errors occur. The storage device decodes the first basic unit using ECC. In a case where the decoding succeeds, the first basic unit is restored to the original basic unit (also called the second basic unit), and thus there is no need to perform subsequent steps. In a case where the decoding fails, the storage device performs intra-disk decoding on the first basic unit by using EC. In a case where the intra-disk decoding succeeds, there is no need to perform subsequent steps. In a case where the intra-disk decoding fails, the storage device performs inter-disk decoding on the first basic unit by using EC. In a case where the inter-disk decoding succeeds, the first basic unit is restored to the second basic unit, and there is no need to perform subsequent steps. In a case where the inter-disk decoding fails, the storage device calculates the first CCD, and determines the first positions of at least part of the first data in the first basic unit.

[0109] In other words, the storage device uses the first positions as feedback, and in a case where ECC decoding is used, the storage device may use the bit flipping to restore at least part of the first data. For example, if all the erroneous data of the first basic unit is recovered, the storage device may output the original basic unit, and finish the process. If a portion of the first data is recovered, the storage device may perform the ECC decoding on the remaining data in the first data except for the portion of the first data again.

[0110] For example, a portion of the first data is recovered, indicating that decoding the first data by using the first positions fails. In this case, the storage device may obtain a third basic unit in which part of the first data has been corrected, and the third basic unit includes remaining data of the first data where one or more errors occur. Then, the storage device may perform ECC decoding on the third basic unit. In a case where the decoding succeeds, the third basic unit is recovered, and the subsequent steps are not performed. In a case where the decoding fails, the storage device performs intra-disk decoding on the third basic unit by using EC. In a case where the intra-disk decoding succeeds, the third basic unit is recovered, and the subsequent steps are not performed. In a case where the intra-disk decoding fails, the storage device performs inter-disk decoding on the third basic unit by using EC. In a case where the inter-disk decoding succeeds, the third basic unit is recovered, and the subsequent steps are not performed. In a case where the inter-disk decoding fails, the storage device calculates CCD of the third basic unit (also called third CCD), and determines second positions of the remaining data of the first data in the third basic unit, based on the third CCD and the second CCD. The storage device may decode the remaining data of the first data in the third basic unit by using the second positions and the above decoding means.

[0111] For example, in a case where the third basic unit is not restored, the storage device may iterate through the preceding process. For example, in order to take into account the processing overhead and processing efficiency of the storage device, the storage device may perform the above process a preset number of times. In a case where the number of calculations of the CCDs exceeds the preset number of times, the decoding is determined as not successful, and the decoding is not continued.

[0112] FIG. 11 illustrates a partial decoding process in accordance with some embodiments.

[0113] Referring to FIG. 11, the storage device encodes data pages by using 5+1 EC encoding, so that Stripe 1 may include six pages (i.e., five data pages and one parity page). In a case when two pages in Stripe 1 have erroneous data (including Erasure and Error), the two pages with erroneous data cannot be decoded and restored by inter-disk decoding (inter-disk EC) because it exceeds the decoding capability of 5+1 ECencoding. In this case, the storage device may treat the pages with Erasure data as correctly decodable pages through hard mode, and then decode the pages with Error data through inter-disk decoding. Since a page with Erasure data is considered as a correctly decodable page in hard mode, which includes at least part of correct data, the storage device may restore the page with Error data to a page with Erasure data to partially restore the data of the page where erroneous data occurs. Then, the storage device calculates the first CCD based on the repaired Stripe 1 , and performs exclusive OR (XOR) operation on the first CCD and the second CCD to obtain CCD syndrome. Using the CCD syndrome, the storage device may determine the first positions of the erroneous data in the data page, and may use the EC decoding to flip bits of the erroneous data at the first positions to restore data. In a case where all the erroneous data is not recovered, the storage device may decode the remaining erroneous data again.

[0114] In a possible implementation, in step 703, determining first positions of the at least part of the first data includes: determining first difference information between the first CCD and the second CCD; and determining the first positions based on the first difference information and a first mapping, where the first mapping indicates a correspondence between multiple difference information and multiple positions, and the multiple difference information includes the first difference information.

[0115] The first difference information may also be called CCD syndrome, and the first mapping may also be called syndrome table. The first mapping includes multiple positions in a basic unit where erroneous data may occur and a CCD syndrome corresponding to each of the multiple positions. In other words, there is a one-to-one correspondence between multiple positions and multiple CCD syndromes. In this way, the storage device may obtain the first positions corresponding to the first difference information by looking up the table, so as to determine the first positions of the erroneous data in the first basic unit.

[0116] In a possible implementation, the multiple positions where erroneous data may occur are related to a size of the first basic unit. The multiple positions where erroneous data may occur are enumerated based on the size of the first basic unit.

[0117] In another possible implementation, in a case where the data in the first basic unit is divided into multiple blocks, the number of positions where erroneous data may occur is related to the data block size in the first basic unit. The multiple positions where erroneous data may occur are enumerated based on the data block size.

[0118] For better understanding, for example, only single-bit errors are considered. In some examples, in a case when a data block size in a page is 1 byte (i.e., 8 bits), there are a total of 8 possible error positions in a data block. In a case of data error occurs at a position, a CCD syndrome may be calculated. Therefore, eight records are included in the first mapping, which include eight positions and eight CCD syndromes.

[0119] In some examples, multi-bit errors are also possible. In this case, the first mapping may also include multiple positions and multiple CCD syndromes calculated in the case of multi-bit errors. The multiple positions corresponding to the multi-bit errors may be a permutation combination of any multiple positions, which is not limited in the embodiment of the present disclosure.

[0120] In some examples, the first mapping may be preconfigured, or may be precalculated by the storage device.

[0121] By looking up the table, the first positions of the erroneous data in the page may be located more quickly, which may improve the efficiency of data decoding.

[0122] How to calculate the CCD in a case where data of a page is divided into multiple blocks is described below in details in combination with two implementations.

[0123] In a possible implementation, the CCD is calculated using the CRC. CRC is a common error detection technique, and is widely applied to storage devices to ensure integrity of data.

[0124] For example, the CCD is calculated by the following formula:where C, represents an Z-th element in the CCD generated by using CRC8, Dy represents an Z-th block of an j-th page, where a value range of i is [1, R], and a value range ofj is [1, Z]; R is the number of blocks in a page, L is the number of pages in a stripe; Ptrepresents an Z-th block in P-parity page, and represents an Z-th block in g-parity page.where Aj j represents a coefficient corresponding to the Z-th block of the j-th page.

[0125] For the storage system in R+2 mode:

[0126] Di 1and Di 2are corrupted (Error) and Di 3has an Erasure which causes Di 3turns into Di'3= Di 3+ Bi. Then, Di,1and Di 2may be recovered as Di'1and Di'2by using the formula (4), because Di'3is used for restoration

[0127] According to formula (5), it may be obtained that, . Then, C / may be obtained withDij, P(, Qi by calculating CRC8 as formula (6), where C / is first CCD.

[0128] It is seen that,) , where ΔCirepresents the difference between first CCD and second CCD, namely the Z-th element in the CCD syndrome.

[0129] Because the x, and y. are known coefficients, the CCD syndrome ΔCiis only determined by thedifference part B, of Di'3and Di 3, that is, the CCD syndrome ΔCiis only associated with erroneous data of

[0130] For example, in FIG. 9, R = 512, L = 3, and the CCD calculation may refer to the following formula (7):

[0131] Where Ciis an i-th element in CCD, and since C, corresponds to the z-th data block of each page, i G { 1, 2, 3, 512}. bjkis a A-th bit of the j-th page, and since there are three pages in FIG. 9, j ∈ { 1, 2, 3}, k∈ { 1, 2, ..., 64}.

[0132] It is noted that, in these embodiments, the CCD calculation only involves three pages, but in other embodiments, the number of pages used to calculate CCD is not limited. For example, it may be all pages in a stripe in a RAID storage system.

[0133] Table 1 shows simulation results of raw bit error rate (RBER) corresponding to different standard deviations, in a case where the number of pages in a stripe is 128, 72, 68, 46, or 24, and the size of the data block in the page is 16, 32, or 64 (in bits), respectively. In Table 1, values in each value cell correspond to the number of stripes out of 100 on which data decoding can be performed to restore the data.

[0134] Table 1

[0135] It is seen that, in a case where the RBER does not exceed 0.017, data restoration may be performed for 100 stripes in the case of three granularities and five different strip sizes. In a case where the RBER is greater than or equal to 0.019, the data restoration capability corresponding to 100 stripes is negatively correlated with granularity. For example, in a case where the RBER is 0.023, at a granularity of 16, 100 stripes may be restored when the stripe size N is 128, 72, or 68, but when the stripe sizes N is 46 and 24, the corresponding number of restorable stripes is 93 and 91, respectively. In a case where the granularity is 32, for a stripe size N of 128, 72, 68, 46, or 24, the number of stripes that may be restored is 93, 95, 89, 95, or 89, respectively. In a case where the granularity is 64, there is no stripe that can be restored.

[0136] In another possible implementation, the storage device may use locally recoverable code (LRC) to calculate CCD. LRC is an error restoration code for data storage or transmission, intending to improve availability and restoration efficiency of data. It is different from conventional ECC, the LRC allows thatlost data is able to be restored using a small part of data in a case where a data block is corrupted. Such local restoration capability makes the LRC particularly suits for large-scale distributed storage system (e.g., cloud storage).

[0137] The number of zero collisions may be minimized in the case of a single-bit data error in a data block in each page.

[0138] In the embodiments, the ECC encoding may be the same as the embodiment shown in FIG. 9. K+2 RS codes may be adopted for EC,

[0139] In this embodiment, the granularity defines an input size for CCD calculation. The CCD calculation itself is changed to be a dot product over the Galois field used in the RS code, using a vector of coefficients chosen in a specific way, with a granularity of 64 bits in this case.

[0140] is stored data of z-th page, and the size of each page is 4KB, whereis the j-th data block of the z-th page, and the size of the data block is 8 Bytes, DK+1and DK+2respectively be the data stored at / ’-parity and ( / -parity pages, a value range of i is [1, K+2], and a value range of / is [1, 512], For each data block Dij, there is , where di tthe / -th byte of the z-th page.

[0141] The data of all the pages satisfy the following two formulations:

[0142]

[0143] which means that the / -th bytes dK+1 (of the / ’-parity page and dK+2 lof ( / -parity page can be computed bywhere {alta2, ... , a10, br, b2, ... , bK+2} are different elements of F256(a finite domain including 256 (i.e., 28) elements).

[0144] Then CCD is:

[0145] This gives it an algebraic attribute that there is no identical syndrome for the input first basic unit, and therefore there is no more than one-bit error in each data block. In addition, the CCD is also stored separately.

[0146] It is noted that, due to a linear nature and a characteristic of the encoding, the syndrome calculated according to the dot product does not depend on the input first basic unit, but only on error patterns of the first basic unit, which may not produce same CCD syndromes in a case where the error of a single data block does not exceed 1 bit, that is, zero collision may not be generated.

[0147] With respect to finite size and granularity of a stripe, matrix H is changed to common K+2 RS code mode from a form of Cauchy, and the calculation of Cj is appropriately updated. It is further noted that, the algebraic attribute of the generator matrix of this CCD matrix results in the minimum number of zerocollision in a case where erroneous bits in each data block do not exceed one bit and in the case of two medium errors and a single page error.

[0148] Table 2 shows the simulation results of the corresponding RBER under different standard deviations in a case where the strip size N = 72 (each strip includes 72 pages) and the granularity size is 16, 32 or 64 (the data block size in each page, expressed in bits). In these examples, simulation is performed for an allzero codeword source, and Table 2 shows the simulation results. In Table 2, values in each value cell correspond to the number of strips in 100 strips on which data decoding may be performed to restore data in the presence of 2 medium errors and 1 page error.

[0149] Table 2

[0150] It can be seen that, in a case where the RBER is lower than 0.019, the erroneous data may be completely restored for 100 strips at the three granularities. In a case where RBER is greater than 0.019, the data restoration capability corresponding to 100 stripes is negatively correlated with granularity. For example, when an RBER is 0.025, 98 out of 100 stripes may be restored with a granularity of 16. In a case where the granularity is 32, only 64 stripes may be restored. In a case where the granularity is 64, there is no stripe that may be restored.

[0151] In some embodiments of the present disclosure, an apparatus 1200 is provided as shown in FIG. 12.

[0152] The apparatus 1200 may have an obtaining module 1201 and a processing module 1202. In some embodiments, these module may be units. The obtaining module 1201 is configured to obtain a first basic unit including first data where one or more errors occur, where the first basic unit is a unit for data encoding and data decoding; the processing module 1202 is configured to calculate first consistency check data of the first basic unit; to determine, based on the first consistency check data and second consistency check data, first positions of at least part of the first data in the first basic unit, where the second consistency check data is consistency check data of second basic unit, and the second basic unit includes original data corresponding to the first basic unit; and decode the first data by using the first positions.

[0153] In a possible implementation, the processing module 1202 is configured to perform bit flipping on data at the first positions in the first basic unit for correcting at least part of the first data.

[0154] In a possible implementation, the first positions include one or more positions of a portion of the first data, and the processing module 1202 is configured to decode remaining data in the first data except for the portion of the first data, by using first decoding means.

[0155] In a possible implementation, the first decoding means includes at least one of: decoding using an EC, or decoding using an ECC.

[0156] In a possible implementation, the processing module 1202 is further configured to divide data in the first basic unit into blocks to obtain a plurality of data blocks; and calculating, based on the plurality of data blocks, the first consistency check data using CRC or LRC.

[0157] In a possible implementation, the processing module 1202 is further configured to decode the first basic unit by using second decoding means, where the decoding the first basic unit by using second decoding means fails.

[0158] In a possible implementation, the second decoding means includes at least one of: decoding using EC; or decoding using ECC.

[0159] In a possible implementation, the processing module 1202 is further configured to decode the first basic unit by using the ECC, where the decoding the first basic unit by using the ECC fails; perform intradisk decoding on the first basic unit by using the EC, where the intra-disk decoding fails; and perform interdisk decoding on the first basic unit by using the EC, where the inter-disk decoding fails.

[0160] In a possible implementation, the processing module 1202 is further configured to perform determining first difference information between the first consistency check data and the second consistency check data; and determining, based on the first difference information and a first mapping, the first positions, where the first mapping indicates a correspondence between a plurality of difference information and a plurality of positions, where the plurality of difference information includes the first difference information.

[0161] In a possible implementation, the plurality of positions are associated with a size of the first basic unit.

[0162] In a possible implementation, the plurality of positions are associated with a size of a data block in the first basic unit.

[0163] In a possible implementation, the obtaining module 1201 is configured to obtain a third basic unit in which part of the first data has been corrected, in a case where decoding the first data by using the first positions fails, where the third basic unit includes remaining data of the first data where one or more errors occur; the processing module 1202 is further configured to calculate third consistency check data of the third basic unit; determine second positions of the remaining data of the first data based on the third consistency check data and the second consistency check data; and decode the remaining data of the first data by using the second positions.

[0164] In a possible implementation, the obtaining module 1201 is configured to obtain the second basic unit; the processing module 1202 is further configured to encode the second basic unit using a first encoding means to obtain encoded basic unit; and calculate the second consistency check data of the encoded basic unit.

[0165] In a possible implementation, the first basic unit is a page.

[0166] The apparatus 1200 may be equivalent to the storage device mentioned above, and be used to perform the data processing method performed by the storage device. A more detailed description of the apparatus 1200 may be referred to the relevant description in the foregoing method embodiment shown in FIG. 7.

[0167] In some embodiments of the present disclosure, an apparatus 1300 is provided as shown in FIG. 13.

[0168] The apparatus 1300 includes a processor 1301 and an interface circuit 1302. The processor 1301 and the interface circuit 1302 are coupled to each other. It is understood that the interface circuit 1302 can be a transceiver or an input / output interface. In some embodiments, the apparatus 1300 may also include a memory 1303, and the memory 1303 is configured to store instructions executed by the processor 1301, the input data required by the processor 1301 to run the instructions or the data produced after the processor 1301 has run the instructions. The interface circuit 1302 may also be understood as part of the processor 1301, where the apparatus 1300 includes the processor 1301.

[0169] The apparatus 1300 may be equivalent to the storage device mentioned above, and be used to perform the data processing method performed by the storage device. A more detailed description of the apparatus 1300 may be referred to the relevant description in the foregoing method embodiment shown in FIG. 7.

[0170] Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium). The computer-readable storage medium has stored thereon program instructions that, when run on a network device / terminal device, cause the network device / terminal device to execute one or more steps of the method as described in any one of the above embodiments.

[0171] Some embodiments of the present disclosure further provide a computer program product. The computer program product includes program instructions carried on a non-transitory computer-readable storage medium. When executed on a network device / terminal device, the computer program instructions cause the network device / terminal device to perform one or more steps of the method as described in the above embodiments.

[0172] In the present disclosure, the terms “a”, “an” and “one” are defined to mean “at least one”, that is, these terms do not exclude a plural number of items, unless stated otherwise.

[0173] In the present disclosure, terms such as “substantially”, “generally” and “about”, which modify a value, condition or characteristic of a feature of an example embodiment, should be understood to mean that the value, condition or characteristic is defined within tolerances that are acceptable for the proper operation of this example embodiment for its intended application.

[0174] In the present disclosure, unless stated otherwise, the terms “connected” and “coupled”, and derivatives and variants thereof, refer herein to any structural or functional connection or coupling, either direct or indirect, between two or more elements. For example, the connection or coupling between the elements can be acoustical, mechanical, optical, electrical, thermal, logical, or any combinations thereof.

[0175] In the present disclosure, expressions such as “match”, “matching” and “matched”, including variants and derivatives thereof, are intended to refer herein to a condition in which two or more elements are either the same or within some predetermined tolerance of each other. That is, these terms are meant to encompass not only “exactly” or “identically” matching the two elements but also “substantially”, “approximately” or “subjectively” matching the two or more elements, as well as providing a higher or best match among a plurality of matching possibilities.

[0176] In the present disclosure, the expression “based on” is intended to mean “based at least partly on”,that is, this expression can mean “based solely on” or “based partially on”, and so should not be interpreted in a limited manner. More particularly, the expression “based on” could also be understood as meaning “depending on”, “representative of’, “indicative of’, “associated with” or similar expressions.

[0177] In the present disclosure, the terms "system" and "network" may be used interchangeably in embodiments of this disclosure. "At least one" means one or more, and "a plurality of means two or more. The term "and / or" describes an association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may indicate the following three cases: Only A exists, both A and B exist, and only B exists, where A and B may be singular or plural. The character " / " indicates an "or" relationship between associated objects. "At least one of the following items (pieces)" or a similar expression thereof indicates any combination of these items, including a single item (piece) or any combination of a plurality of items (pieces). For example, "at least one of A, B, or C" includes: only A; only B; only C; A and B; A and C; B and C; or A, B, and C, and "at least one of A, B, and C" may also be understood as including: only A; only B; only C; A and B; A and C; B and C; or A, B, and C. In addition, unless otherwise specified, ordinal numbers such as "first" and "second" in embodiments of this disclosure are used to distinguish between a plurality of objects, and are not used to limit a sequence, a time sequence, priorities, or importance of the plurality of objects.

[0178] A person skilled in the art should understand that embodiments of this disclosure may be provided as a method, an apparatus (or system), computer-readable storage medium, or a computer program product. Therefore, these embodiments may use a form of a hardware-only embodiment, a software-only embodiment, or an embodiment with a combination of software and hardware. Moreover, these embodiments may use a form of a computer program product that is implemented on one or more computer-usable storage media (including but not limited to a disk memory, an optical memory, and the like) that include computer-usable program code.

[0179] This disclosure is described with reference to the flowcharts and / or block diagrams of the method, the device (system), and the computer program product according to embodiments of this disclosure. It is understood that computer program instructions may be used to implement each process and / or each block in the flowcharts and / or the block diagrams and a combination of a process and / or a block in the flowcharts and / or the block diagrams. The computer program instructions may be provided for a general-purpose computer, a dedicated computer, an embedded processor, or a processor of another programmable data processing device to generate a machine, so that the instructions executed by the computer or the processor of the another programmable data processing device generate an apparatus for implementing a specific function in one or more procedures in the flowcharts and / or in one or more blocks in the block diagrams.

[0180] The computer program instructions may alternatively be stored in a computer-readable memory that can indicate a computer or another programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus implements a specific function in one or more procedures in the flowcharts and / or in one or more blocks in the block diagrams.

[0181] The computer program instructions may alternatively be loaded onto a computer or anotherprogrammable data processing device, so that a series of operations and steps are performed on the computer or another programmable device, so that computer- implemented processing is generated. Therefore, the instructions executed on the computer, or another programmable device provide steps for implementing a specific function in one or more procedures in the flowcharts and / or in one or more blocks in the block diagrams.

[0182] It is clear that a person skilled in the art can make various modifications and variations to the embodiments without departing from the scope of this disclosure. This disclosure is intended to cover these modifications and variations of this disclosure provided that they fall within the scope of protection defined by the following claims and their equivalent technologies.

[0183] The present disclosure encompasses various embodiments, including not only method embodiments, but also other embodiments such as apparatus embodiments and embodiments related to non-transitory computer readable storage media. Embodiments may incorporate, individually or in combinations, the features disclosed herein.

[0184] Although this disclosure refers to illustrative embodiments, this is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative embodiments, as well as other embodiments of the disclosure, will be apparent to persons skilled in the art upon reference to the description.

[0185] Features disclosed herein in the context of any particular embodiments may also or instead be implemented in other embodiments. Method embodiments, for example, may also or instead be implemented in apparatus, system, and / or computer program product embodiments. In addition, although embodiments are described primarily in the context of methods and apparatus, other implementations are also contemplated, as instructions stored on one or more non-transitory computer-readable media, for example. Such media could store programming or instructions to perform any of various methods consistent with the present disclosure.

Claims

CLAIMS1. A data processing method, comprising: obtaining a first basic unit comprising first data where one or more errors occur, wherein the first basic unit is a unit for data encoding and data decoding; calculating first consistency check data of the first basic unit; determining, based on the first consistency check data and second consistency check data, first positions of at least part of the first data in the first basic unit, wherein the second consistency check data is consistency check data of second basic unit, and the second basic unit comprises original data corresponding to the first basic unit; and decoding the first data by using the first positions.

2. The method of claim 1, wherein decoding the first data by using the first positions comprises: performing bit flipping on data at the first positions in the first basic unit for correcting the at least part of the first data.

3. The method of claim 2, wherein the first positions comprise one or more positions of a portion of the first data, and the method further comprises: decoding remaining data in the first data except for the portion of the first data, by using first decoding means.

4. The method of claim 3, wherein the first decoding means comprises at least one of: decoding using an EC, or decoding using an ECC.

5. The method of any one of claims 1 to 4, wherein calculating the first consistency check data of the first basic unit comprises: dividing data in the first basic unit into blocks to obtain a plurality of data blocks; and calculating, based on the plurality of data blocks, the first consistency check data using cyclic redundancy check (CRC) or locally recoverable code (LR.C).

6. The method of any one of claims 1 to 5, wherein before calculating the first consistency check data of the first basic unit, the method further comprises: decoding the first basic unit by using second decoding means, wherein the decoding the first basic unit by using second decoding means fails.

7. The method of claim 6, wherein decoding the first basic unit by using the second decoding means comprises: decoding the first basic unit by using the ECC, wherein the decoding the first basic unit by using the ECC fails; performing intra-disk decoding on the first basic unit by using the EC, wherein the intra-disk decoding fails; and performing inter-disk decoding on the first basic unit by using the EC, wherein the inter-disk decoding fails.

8. The method of any one of claims 1 to 7, wherein determining the first positions of the at least part of the first data comprises:determining first difference information between the first consistency check data and the second consistency check data; and determining, based on the first difference information and a first mapping, the first positions, wherein the first mapping indicates a correspondence between a plurality of difference information and a plurality of positions, wherein the plurality of difference information comprises the first difference information.

9. The method of claim 8, wherein the plurality of positions are associated with a size of the first basic unit.

10. The method of claim 8, wherein the plurality of positions are associated with a size of a data block in the first basic unit.1 1. The method of any one of claims 1 to 10, further comprising: obtaining a third basic unit in which part of the first data has been corrected, in a case where decoding the first data by using the first positions fails, wherein the third basic unit comprises remaining data of the first data where one or more errors occur; calculating third consistency check data of the third basic unit; determining, based on the third consistency check data and the second consistency check data, second positions of the remaining data of the first data; and decoding the remaining data of the first data by using the second positions.

12. The method of any one of claims 1 to 11, wherein before obtaining the first basic unit, the method further comprises: obtaining the second basic unit; encoding the second basic unit using a first encoding means to obtain encoded basic unit; calculating the second consistency check data of the encoded basic unit.

13. The method of any one of claims 1 to 12, wherein the first basic unit is a page.

14. An apparatus, configured to perform the method of any one of claims 1 to 13.

15. An apparatus, comprising: one or more processors; a memory storing instructions which, when executed by the one or more processors, cause the apparatus to perform the method of any one of claims 1 to 13.

16. A computer-readable storage medium having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 13.

17. A computer program product storing instructions which, when executed, cause an apparatus to perform the method of any one of claims 1 to 13.

Images