Hard disk data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a hard disk data processing method and device, electronic equipment and a storage medium. Relates to the technical field of computers. Based on the access frequency of the to-be-processed data, classifying the to-be-processed data to obtain a first type of data and a second type of data; the access frequency of the first type of data is greater than that of the second type of data; in response to the situation that a first storage unit storing the first type of data does not meet a first storage condition, performing data processing on the first type of data based on a first type of error correction code; in response to the situation that a second storage unit storing the second type of data does not meet a second storage condition, performing data processing on the second type of data based on a second type of error correction code; the error correction speed of the first type of error correction codes is higher than that of the second type of error correction codes. Through the application, the service life of the hard disk can be prolonged, and the data processing efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a hard disk data processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the explosive growth of data volume and the continuous rise of storage demand, the hard disk market has shown a significant growth trend. NAND Flash (NOT AND Flash) technology has evolved from storing 1 bit of data per storage cell in the early days to storing 3 / 4 bits of data per storage cell today, greatly improving storage density. However, this progress has also brought challenges such as increased inter-cell charge interference, decreased data retention capability, and rising read / write error rates, which have put a severe test on the ability of hard disks to ensure the reliability of Flash data.

[0003] Traditional error correction codes, such as BCH codes (a type of cyclic error correction code), gradually appear to be inadequate in terms of reliability when facing high error rate scenarios. Therefore, as a high-performance error correction scheme with strong error correction capability, Low-Density Parity-Check Code (LDPC) is introduced into the host chip of the hard disk and becomes the mainstream error correction method for modern high-density storage devices. LDPC is a probability-based ECC (Error Correction Code) error correction mechanism, and when the error rate of NAND Flash is high, the number of iterations needs to be increased, resulting in a prolonged LDPC decoding delay and affecting the Quality of Service (Qos).

[0004] For high-density NAND Flash, its error rate is usually high, and as the number of program / erase (P / E) increases, the error rate grows more significantly. Currently, the main strategy to improve LDPC decoding efficiency relies on optimizing the LDPC algorithm. However, given the characteristics of NAND Flash, its error rate increases with the number of P / E (write / erase), and there are differences in error rates between different blocks, so it is difficult to fully consider the hard disk life while improving data processing efficiency by relying solely on optimizing the LDPC algorithm. SUMMARY

[0005] The embodiments of the present application provide a hard disk data processing method and device, electronic equipment and storage medium.

[0006] The application provides a hard disk data processing method, which comprises the following steps: obtaining to-be-processed data in a hard disk; classifying the to-be-processed data based on the access frequency of the to-be-processed data to obtain first-class data and second-class data; the access frequency of the first-class data is greater than the access frequency of the second-class data; in response to a first storage unit storing the first-class data not satisfying a first storage condition, performing data processing on the first-class data based on a first error correction code; in response to a second storage unit storing the second-class data not satisfying a second storage condition, performing data processing on the second-class data based on a second error correction code; the error correction speed of the first error correction code is higher than the error correction speed of the second error correction code.

[0007] According to an embodiment of the application, the obtaining to-be-processed data in a hard disk comprises the following steps: obtaining initial data and a first error correction code in a system; the first error correction code is stored in the memory of the system; performing data processing on the initial data based on the first error correction code to obtain the to-be-processed data; and storing the to-be-processed data in a storage unit of the hard disk.

[0008] According to an embodiment of the application, the classifying the to-be-processed data based on the access frequency of the to-be-processed data to obtain first-class data and second-class data comprises the following steps: obtaining the access frequency of the to-be-processed data in the hard disk; determining to-be-processed data with an access frequency greater than a set frequency threshold as first-class data; and determining to-be-processed data with an access frequency less than or equal to the set frequency threshold as second-class data.

[0009] According to an embodiment of the application, the method further comprises the following steps: generating the first error correction code and the second error correction code based on a set base matrix; the number of rows and the number of columns of the base matrix are less than the number of rows and the number of columns of the first error correction code, and the number of rows and the number of columns of the base matrix are less than the number of rows and the number of columns of the second error correction code.

[0010] According to an embodiment of the application, the generating the first error correction code and the second error correction code based on a set base matrix comprises the following steps: expanding the base matrix based on a first expansion parameter and a first number of check rows to obtain the first error correction code; and expanding the base matrix based on a second expansion parameter and a second number of check rows to obtain the second error correction code; the first expansion parameter is less than the second expansion parameter, and the first number of check rows is less than the second number of check rows.

[0011] According to one embodiment of the present application, the extension parameter is associated with the size of a sub-matrix corresponding to each element in the base matrix; the sub-matrix includes a zero matrix and a cyclic shift matrix; the zero matrix is used to extend a zero element in the base matrix, and the cyclic shift matrix is used to extend a non-zero element in the base matrix; the number of check rows is associated with the number of rows of the base matrix.

[0012] According to one embodiment of the present application, the data processing of the first type of data based on the first type of error correction code in response to the first storage unit storing the first type of data not satisfying the first storage condition comprises: determining a first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, reading the first type of data from the first storage unit and decoding the first type of data based on the first type of error correction code to obtain decoded first type of data; updating an address mapping table of the hard disk, marking the first storage unit as a fourth storage unit, and determining a third storage unit; the error level of the third storage unit is lower than the first level threshold; encoding the decoded first type of data based on the first type of error correction code to obtain encoded first type of data; migrating the encoded first type of data to the third storage unit and clearing the data in the fourth storage unit.

[0013] According to one embodiment of the present application, the data processing of the second type of data based on the second type of error correction code in response to the second storage unit storing the second type of data not satisfying the second storage condition comprises: determining a second error level of the second storage unit; in response to the second error level being greater than a second level threshold included in the second storage condition, reading the second type of data from the second storage unit and decoding the first type of data based on the second type of error correction code to obtain decoded second type of data; updating an address mapping table of the hard disk, marking the second storage unit as an unusable unit, and determining the fourth storage unit; encoding the decoded second type of data based on the second type of error correction code to obtain encoded second type of data; migrating the encoded second type of data to the fourth storage unit and clearing the data in the unusable unit.

[0014] According to one embodiment of the present application, the first type of error correction code and the second type of error correction code are both low-density parity-check codes (LDPC).

[0015] The application further provides a hard disk data processing device, which comprises: an acquisition module, configured to acquire data to be processed in a hard disk; a classification module, configured to classify the data to be processed based on access frequency of the data to be processed, to obtain first data and second data; the access frequency of the first data is greater than the access frequency of the second data; a first processing module, configured to, in response to a first storage unit storing the first data not satisfying a first storage condition, perform data processing on the first data based on a first error correction code; and a second processing module, configured to, in response to a second storage unit storing the second data not satisfying a second storage condition, perform data processing on the second data based on a second error correction code; the error correction speed of the first error correction code is higher than the error correction speed of the second error correction code.

[0016] According to one embodiment of the application, the acquisition module is configured to: obtain initial data and a first error correction code in a system; the first error correction code is stored in a memory of the system; perform data processing on the initial data based on the first error correction code, to obtain the data to be processed; and store the data to be processed in a storage unit of the hard disk.

[0017] According to one embodiment of the application, the classification module is configured to: acquire access frequency of the data to be processed in the hard disk; determine data to be processed with access frequency greater than a set frequency threshold as first data; and determine data to be processed with access frequency less than or equal to the set frequency threshold as second data.

[0018] According to one embodiment of the application, the device further comprises a generation module, configured to: generate the first error correction code and the second error correction code based on a set base matrix; the number of rows and the number of columns of the base matrix are less than the number of rows and the number of columns of the first error correction code, and the number of rows and the number of columns of the base matrix are less than the number of rows and the number of columns of the second error correction code.

[0019] According to one embodiment of the application, the generation module is configured to: extend the base matrix based on a first extension parameter and a first number of check rows, to obtain the first error correction code; and extend the base matrix based on a second extension parameter and a second number of check rows, to obtain the second error correction code; the first extension parameter is less than the second extension parameter, and the first number of check rows is less than the second number of check rows.

[0020] According to one embodiment of the present application, the extension parameter is associated with the size of a sub-matrix corresponding to each element in the base matrix; the sub-matrix includes a zero matrix and a cyclic shift matrix; the zero matrix is used for extending a zero element in the base matrix, and the cyclic shift matrix is used for extending a non-zero element in the base matrix; the number of check rows is associated with the number of rows of the base matrix.

[0021] According to one embodiment of the present application, the first processing module is configured to: determine a first error level of the first storage unit; in response to the first error level being greater than a first threshold included in the first storage condition, read out the first type of data from the first storage unit based on a first error correction code and decode the first type of data to obtain decoded first type of data; update an address mapping table of the hard disk, mark the first storage unit as a fourth storage unit, and determine a third storage unit; the error level of the third storage unit is lower than the first threshold; encode the decoded first type of data based on the first error correction code to obtain encoded first type of data; and migrate the encoded first type of data to the third storage unit and clear the data in the fourth storage unit.

[0022] According to one embodiment of the present application, the second processing module is configured to: determine a second error level of the second storage unit; in response to the second error level being greater than a second threshold included in the second storage condition, read out the second type of data from the second storage unit based on a second error correction code and decode the first type of data to obtain decoded second type of data; update an address mapping table of the hard disk, mark the second storage unit as an unusable unit, and determine the fourth storage unit; encode the decoded second type of data based on the second error correction code to obtain encoded second type of data; and migrate the encoded second type of data to the fourth storage unit and clear the data in the unusable unit.

[0023] According to one embodiment of the present application, the first error correction code and the second error correction code are both low-density parity-check codes (LDPC).

[0024] The present application also provides an electronic device, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the above-mentioned embodiments.

[0025] The present application also provides a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method according to the above-mentioned embodiments.

[0026] The method provided by the embodiment of the present application comprises: obtaining to-be-processed data in a hard disk; classifying the to-be-processed data based on access frequencies of the to-be-processed data to obtain first-class data and second-class data; the access frequency of the first-class data is greater than the access frequency of the second-class data; in response to a first storage unit storing the first-class data not satisfying a first storage condition, performing data processing on the first-class data based on a first error correction code; in response to a second storage unit storing the second-class data not satisfying a second storage condition, performing data processing on the second-class data based on a second error correction code; and the error correction speed of the first error correction code is higher than the error correction speed of the second error correction code. Through the present application, the service life of the hard disk can be improved, and the data processing efficiency is improved.

[0027] It should be understood that the teachings of the present application do not require all the beneficial effects described above to be achieved, but a specific technical solution can achieve a specific technical effect, and other embodiments of the present application can also achieve beneficial effects not mentioned above. BRIEF DESCRIPTION OF DRAWINGS

[0028] The above and other objects, features and advantages of the exemplary embodiments of the present application will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: In the drawings, identical or corresponding reference signs refer to identical or corresponding parts.

[0029] Figure 1 A processing flow diagram of a hard disk data processing method provided by an embodiment of the present application is shown Figure 1 ; Figure 2 A processing flow diagram of a hard disk data processing method provided by an embodiment of the present application is shown Figure 2 ; Figure 3 An application scenario diagram of a hard disk data processing method provided by an embodiment of the present application is shown Figure 4 An optional schematic diagram of a hard disk data processing apparatus provided by an embodiment of the present application is shown Figure 5 An optional schematic diagram of an electronic device provided by an embodiment of the present application is shown DETAILED DESCRIPTION

[0030] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0031] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0034] The processing flow of the hard disk data processing method provided in the embodiment of the present application is described. Figure 1 , Figure 1 This is a schematic diagram of the processing flow of the hard disk data processing method provided in the embodiment of the present application. Figure 1 , will combine Figure 1 Steps S101-S104 are shown for explanation.

[0035] Step S101: Obtain data to be processed from the hard disk.

[0036] In some embodiments, the data to be processed may include data stored on a hard disk. The data to be processed can be accessed. The hard disk may include a solid-state drive (SSD) or a mechanical hard disk. The hard disk may also include other types of hard disks, and the embodiments of this application are not limited to a specific hard disk.

[0037] Step S102 , classifying the data to be processed based on the access frequency of the data to be processed to obtain first-category data and second-category data; the access frequency of the first-category data is greater than the access frequency of the second-category data.

[0038] In some embodiments, the access frequency can include the number of times the data to be processed is read or written in a preset time interval. The first type of data can include data with a high access frequency, i.e., hot data. Hot data has a high requirement for read performance. The second type of data can include data with a low access frequency, i.e., cold data. The requirement of cold data for read performance is lower than that of hot data.

[0039] In step S103, in response to the first storage unit storing the first type of data not satisfying the first storage condition, the first type of data is processed based on the first type of error correction code.

[0040] In some embodiments, the first storage unit can include a physical block in a hard disk for storing the first type of data. The first storage condition can represent the health status of the first storage unit, such as determining that the number of corresponding error bits is higher than a certain threshold when reading data from the first storage unit, and determining that the first storage unit does not satisfy the first storage condition. The first type of error correction code can include an error correction code with fast error correction speed but weak error correction capability. The first type of error correction code can be used to quickly process errors of the first type of data.

[0041] In step S104, in response to the second storage unit storing the second type of data not satisfying the second storage condition, the second type of data is processed based on the second type of error correction code; the error correction speed of the first type of error correction code is higher than the error correction speed of the second type of error correction code.

[0042] In some embodiments, the second storage unit can include a physical block in a hard disk for storing the second type of data. The second storage condition can represent the health status of the second storage unit, such as determining that the number of corresponding error bits is higher than a certain threshold when reading data from the second storage unit, and determining that the second storage unit does not satisfy the second storage condition. The second type of error correction code can include an error correction code with slow error correction speed but strong error correction capability. The second type of error correction code can be used to efficiently repair errors of the second type of data. The first type of error correction code and the second type of error correction code can both be LDPC.

[0043] As an example, data to be processed is read from a hard disk, which can specifically include files, database records, or logs, etc. Classification is performed based on the access frequency of the data, and data with a high access frequency is divided into the first type of data, and data with a low access frequency is divided into the second type of data. Then, it is determined whether the first storage unit storing the first type of data satisfies the first storage condition. If the first storage condition is not satisfied, the first type of data is decoded and encoded based on the first type of error correction code, and the encoded data is migrated to other storage units. It is determined whether the second storage unit storing the second type of data satisfies the second storage condition. If the condition is not satisfied, the data is decoded and encoded based on the second type of error correction code, and the encoded data is migrated to other storage units.

[0044] The method of the embodiment of the present application uses two sets of strong and weak error correction codes in the hard disk. The strong error correction code has a greater error correction capability than the weak error correction code, which can correct data when the number of error bits is large, but the strong error correction code takes longer to iterate once than the weak error correction code. In order to balance the performance and life of the hard disk, weak error correction codes are used when data is first written. As the number of erases and writes increases, when the number of error bits reaches a threshold and needs to be moved, hot data and cold data are distinguished. Weak error correction codes are still used for hot data, while strong error correction codes are used for cold data. In addition, to avoid the excessive number of iterations when decoding with weak error correction codes as the number of error bits increases, resulting in a decoding delay that is even higher than the strong error correction code delay, this method distinguishes between thresholds for hot and cold data. When the error level of the block storing hot data exceeds the threshold, the block is marked when the data needs to be moved. Later, when cold data needs to be moved, the data can be moved to this block, thereby increasing the service life of the hard disk. The error-correcting code can be dynamically adjusted, has simple rules, and is easy to implement in hardware. The same basic matrix can generate error-correcting codes with various code lengths and rates.

[0045] By using different error correction codes for hot and cold data, the hard drive's data processing efficiency can be effectively improved. Hot data is always stored in storage cells with less wear and tear, and weak error correction codes (which take less time per iteration) are used for encoding and decoding. Since the number of error bits is low, weak error correction codes can correct them, greatly improving data processing efficiency for hot data. Cold data is stored in storage cells with higher wear and tear, and strong error correction codes are used for encoding and decoding when data is moved or rewritten. This ensures maximum data processing efficiency for both hot and cold data while also separating them, optimizing data layout and extending the hard drive's lifespan.

[0046] In some embodiments, obtaining the data to be processed in the hard disk in step S101 may include: obtaining initial data and a first type of error correction code in the system; storing the first type of error correction code in the system's memory; performing data processing on the initial data based on the first type of error correction code to obtain the data to be processed; and storing the data to be processed in a storage unit of the hard disk.

[0047] In some embodiments, the initial data can include data first written into the hard disk in the system. The first type of error correction code can include LDPC stored in the system memory (DRAM). LDPC belongs to a type of ECC, which can be used to quickly detect and correct errors that occur during data transmission or storage. The base matrix of the LDPC can be loaded into the DRAM during system initialization. The data processing can include decoding and error correction coding operations on the initial data. The data to be processed can include data processed by the first error correction coding.

[0048] As an example, when a log file in the system is first stored to the hard disk, the system calls the LDPC error correction code in the memory to perform error correction coding processing on the log file to generate data to be processed. If errors are found in the log file during error correction coding processing, the LDPC is used to repair the existing errors. Finally, the data to be processed is written to the free storage unit of the hard disk, and the address mapping table of the hard disk is updated.

[0049] In some embodiments, the classification of the data to be processed based on the access frequency of the data to be processed in step S102 to obtain the first type of data and the second type of data can include: obtaining the access frequency of the data to be processed in the hard disk; determining the data to be processed with an access frequency greater than a set frequency threshold as the first type of data; and determining the data to be processed with an access frequency less than or equal to the set frequency threshold as the second type of data.

[0050] In some embodiments, the frequency threshold can include a set standard value, such as 10 times / hour, for distinguishing high-frequency and low-frequency data. The first type of data can include data to be processed in the hard disk with an access frequency greater than the frequency threshold. The second type of data can include data to be processed in the hard disk with an access frequency less than or equal to the frequency threshold. As an example, first collect the access frequency of each data to be processed in the hard disk, and count the number of times each data to be processed is read or written in the past one hour. The system sets the frequency threshold of the access frequency to 10 times / hour, determines the data to be processed with an access frequency greater than the frequency threshold as the first type of data, and determines the data to be processed with an access frequency less than or equal to the frequency threshold as the second type of data. The storage unit storing the first type of data is the first storage unit, and the storage unit storing the second type of data is the second storage unit.

[0051] In some embodiments, the processing flow of the hard disk data processing method is shown in Figure 2 As Figure 2 shown, the hard disk data processing method can further include: Step S201, obtaining data to be processed in the hard disk.

[0052] Step S202, classifying the to-be-processed data based on the access frequency of the to-be-processed data to obtain first type data and second type data.

[0053] The explanation and description of steps S201 and S202 are the same as those of steps S101 and S102, which will not be repeated here.

[0054] Step S203, generating first type error correction code and second type error correction code based on the set base matrix.

[0055] In some embodiments, the generating of the first type error correction code and the second type error correction code based on the set base matrix in step S203 comprises: extending the base matrix based on a first extension parameter and a first number of check rows to obtain the first type error correction code; and extending the base matrix based on a second extension parameter and a second number of check rows to obtain the second type error correction code.

[0056] In some embodiments, the number of rows and the number of columns of the base matrix are less than the number of rows and the number of columns of the first type error correction code, and the number of rows and the number of columns of the base matrix are less than the number of rows and the number of columns of the second type error correction code. The first extension parameter is less than the second extension parameter, and the first number of check rows is less than the second number of check rows. The extension parameter is associated with the size of a sub-matrix corresponding to each element in the base matrix; the sub-matrix includes a zero matrix and a cyclic shift matrix; the zero matrix is used to extend a zero element in the base matrix, and the cyclic shift matrix is used to extend a non-zero element in the base matrix; the number of check rows is associated with the number of rows of the base matrix. In the process of generating error correction code based on the base matrix, the sub-matrix is used to extend each element of the base matrix. For a zero element in the base matrix, the corresponding sub-matrix is a zero matrix; and for a non-zero element, the corresponding sub-matrix is a cyclic shift matrix, and the shift step of the cyclic shift matrix is determined by the value of the non-zero element. By adjusting the extension parameter and the number of check rows, different error correction codes can be generated.

[0057] As an example, first configure a base matrix of MxN (M rows and N columns), and the elements of the base matrix are 0, 1, 2, 3, etc. integers. Each non-zero element k (k >= 0) not only indicates that there is a connection at this position, but also implies the shift value of the subsequent cyclic shift matrix during extension. For each element in the base matrix, if the element value is 0, a ZxZ zero matrix is obtained in the extended matrix; if the element is a non-zero value, a ZxZ cyclic shift matrix is obtained in the extended matrix, and the cyclic shift matrix can be obtained by cyclically shifting a unit matrix by s, where s=k mod Z, for example, Z=3 and k=1. The original unit matrix is [0,0,1;0,1,0;1,0,0], and the matrix after cyclic shift is [1,0,0;0,0,1;0,1,0]. After extending the base matrix, a LDPC check matrix of size (MxZ)x(NxZ) can be obtained.

[0058] The first type of error-correcting code can be an LDPC code with a fast error correction speed but weak error correction capability, while the second type of error-correcting code can be an LDPC code with a slow error correction speed but strong error correction capability. Dynamically adjusting the corresponding extension parameters and number of check rows of the LDPC can change the error correction speed and capability of the LDPC. This embodiment includes two adjustment options. The first is to adjust the error correction speed and capability of the LDPC by dynamically adjusting the code length. If the basic matrix is ​​well designed, a longer code length generally results in stronger error correction capability for the LDPC. In this embodiment, this involves adjusting the size of the extended matrix Z×Z. The larger the extension parameter Z, the stronger the error correction capability of the LDPC. The second type of error-correcting code is to adjust the code rate. The more redundant check information the LDPC contains, the stronger the error correction capability. Specifically, the larger the number of check rows M in the basic matrix, the more check equations there are, the more check information generated, and thus the stronger the error correction capability of the corresponding LDPC. The first type of error-correcting code can be generated by expanding the basic matrix after pruning M rows. The second type of error-correcting code can be generated by expanding the complete basic matrix. The first type of error correction code and the second type of error correction code use the same set of codecs. During generation, only the extended parameters and the number of check rows need to be reconfigured. That is, the first type of error correction code is generated based on the first extended parameter and the first number of check rows, and the second type of error correction code is generated based on the second extended parameter and the second number of check rows. The first extended parameter is smaller than the second extended parameter, and the first number of check rows is smaller than the second number of check rows.

[0059] In some embodiments, in response to the first storage unit storing the first category data not meeting the first storage condition in step S103, data processing is performed on the first category data based on the first category error correction code, including: determining a first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, reading the first category data from the first storage unit and decoding the first category data based on the first category error correction code to obtain decoded first category data; updating the address mapping table of the hard disk, marking the first storage unit as the fourth storage unit, and determining the third storage unit; the error level of the third storage unit is lower than the first level threshold; encoding the decoded first category data based on the first category error correction code to obtain encoded first category data; migrating the encoded first category data to the third storage unit and clearing the data in the fourth storage unit.

[0060] In some embodiments, the first error level can be an index describing the health status of the first storage unit. Specifically, it can be the number of error bits returned by the LDPC module of the hard disk when reading data from the NAND Flash of the hard disk each time. The higher the number of error bits, the higher the first error level. The first level threshold can be a preset health status threshold value, when the first error level exceeds this value, the storage unit needs to be operated. The decoded first type data can be the data read from the first storage unit after the first type data is decoded by the FTL (Flash Translation Layer) through the first type error correction code. The address mapping table can include a mapping table recording the relationship between logical addresses and physical addresses in the hard disk system. Specifically, the address mapping table can be an FTL mapping table. The third storage unit can include a physical block in the hard disk with good health status. The third storage unit can be used to receive and store the decoded first type data, replacing the old first storage unit. The fourth storage unit is obtained by marking the old first storage unit. The marking can indicate that the system moves the second type data to the fourth storage unit.

[0061] As an example, the first type error correction code can be an LDPC with weak error correction capability. The first type data in the solid state disk is a group of user frequently used files, stored in the first storage unit. The error level of the first storage unit is determined to be 5, which exceeds the preset first level threshold 3. The FTL of the solid state disk reads the first type data from the first storage unit after decoding by the LDPC with weak error correction capability, and updates the FTL mapping table, marks the first storage unit as the fourth storage unit, and determines a third storage unit with good health status (error level 1, lower than the threshold 3). The FTL encodes the decoded first type data by the LDPC with weak error correction capability and writes it to the third storage unit, and clears the old data in the fourth storage unit.

[0062] In some embodiments, in response to the second storage unit storing the second type data not meeting the second storage condition, the data processing of the second type data based on the second type error correction code in step S104 includes: determining a second error level of the second storage unit; in response to the second error level being greater than a second level threshold included in the second storage condition, reading the second type data from the second storage unit and decoding the first type data based on the second type error correction code to obtain decoded second type data; updating the address mapping table of the hard disk, marking the second storage unit as an unusable unit, and determining a fourth storage unit; encoding the decoded second type data based on the second type error correction code to obtain encoded second type data; and migrating the encoded second type data to the fourth storage unit and clearing the data in the unusable unit.

[0063] In some embodiments, the second error level can be an index describing the health status of the second storage unit. Specifically, it can be the number of error bits returned by the LDPC module of the hard disk when reading data from the NAND Flash of the hard disk each time. The higher the number of error bits, the higher the second error level. The second level threshold can be a preset health status threshold value, when the second error level exceeds this value, the storage unit needs to be operated. The decoded second type of data can be the data read from the second storage unit after the second type of data is decoded by the FTL through the second type of error correction code. The unusable unit can include a storage unit whose error level exceeds the second level threshold. The unusable unit can be a bad block of the hard disk.

[0064] As an example, the second type of error correction code can be an LDPC with strong error correction capability. The second type of data in the solid state disk is a group of user backup files stored in the second storage unit. It is determined that the error level of the second storage unit is 8, which exceeds the preset second level threshold 6. The FTL of the solid state disk reads the second type of data from the second storage unit after decoding by the LDPC with strong error correction capability, and updates the FTL mapping table, marks the second storage unit as an unusable unit, and determines a fourth storage unit with good health status (error level 5, lower than threshold 6). The FTL encodes the decoded second type of data through the LDPC with strong error correction capability and writes it to the fourth storage unit, and clears the old data in the unusable unit.

[0065] Figure 3 An application scenario diagram of the hard disk data processing method provided by the embodiments of the present application is shown.

[0066] Reference Figure 3 An application scenario of the hard disk data processing method provided by the embodiments of the present application is applied to solid state disk data processing.

[0067] For data with access frequency greater than threshold T, the following description is hot data, and for data with access frequency less than threshold T, the following description is cold data. The threshold T can be a set frequency threshold.

[0068] When the system is initialized, the base matrix is loaded into the DRAM, and the LDPC with weak error correction capability is configured by default. When the data is initially written to the hard disk, it is not possible to determine whether the data is cold data or hot data, so all data is encoded using the LDPC with weak error correction capability, and is preferentially written to the block with lower wear level (smaller number of error bits returned by the LDPC module). When reading data from the NAND Flash of the solid state disk each time, the LDPC module returns a number of error bits, which is used to determine the error level. Based on the comparison of the error level with the preset threshold, it is determined whether to perform a data migration operation.

[0069] Hot data processing: If the error level of hot data is less than the threshold L1, no additional data migration is performed. If the error level of hot data exceeds the threshold L1, the block is considered to have a high error level and needs to be migrated to a more reliable block. ECC encoding with weak error correction capabilities is used during the migration, and after the operation is completed, the original storage block is marked as available for cold data storage.

[0070] Cold data processing: If the error level of cold data is less than the threshold L2, no additional data migration is performed. If the error level of cold data exceeds the threshold L2, the data is moved to a pre-marked block and the strong ECC encoding is switched during the move. When the wear level of a block approaches the limit of the strong ECC encoding's error correction capability, the block is marked as bad.

[0071] I understand. Figure 3 The application scenarios of the hard disk data processing method are only some exemplary implementations of the present application. The application scenarios of the hard disk data processing method in the present application include but are not limited to Figure 3 The application scenario of the hard disk data processing method shown.

[0072] The following continues to describe the exemplary structure of the software modules included in the hard disk data processing device 90 provided in the embodiment of the present application. In some embodiments, such as Figure 4 As shown, the hard disk data processing device 90 may include: an acquisition module 901 for acquiring data to be processed in the hard disk; A classification module 902 is configured to classify the data to be processed based on the access frequency of the data to be processed, thereby obtaining first-category data and second-category data; the access frequency of the first-category data is greater than the access frequency of the second-category data; A first processing module 903 is configured to, in response to a first storage unit storing the first type of data not satisfying a first storage condition, perform data processing on the first type of data based on a first type of error correction code; The second processing module 904 is configured to process the second type of data based on the second type of error correction code in response to the second storage unit storing the second type of data not satisfying the second storage condition; the error correction speed of the first type of error correction code is higher than the error correction speed of the second type of error correction code.

[0073] In some embodiments, the acquisition module 901 may be used to: Obtaining initial data and a first-type error correction code in the system; the first-type error correction code is stored in a memory of the system; Performing data processing on the initial data based on the first type of error correction code to obtain data to be processed; The data to be processed is stored in the storage unit of the hard disk.

[0074] In some embodiments, the classification module 902 can be configured to: acquire the access frequency of the to-be-processed data in the hard disk; determine the to-be-processed data with the access frequency greater than the set frequency threshold as the first type of data; determine the to-be-processed data with the access frequency less than or equal to the set frequency threshold as the second type of data.

[0075] In some embodiments, the hard disk data processing apparatus 90 further comprises a generation module configured to: generate the first type of error correction code and the second type of error correction code based on a set base matrix; the number of rows and columns of the base matrix is less than the number of rows and columns of the first type of error correction code, and the number of rows and columns of the base matrix is less than the number of rows and columns of the second type of error correction code.

[0076] In some embodiments, the generation module can be configured to: extend the base matrix based on a first extension parameter and a first number of check rows to obtain the first type of error correction code; extend the base matrix based on a second extension parameter and a second number of check rows to obtain the second type of error correction code; the first extension parameter is less than the second extension parameter, and the first number of check rows is less than the second number of check rows.

[0077] In some embodiments, the extension parameter is associated with the size of a sub-matrix corresponding to each element in the base matrix; the sub-matrix includes a zero matrix and a cyclic shift matrix; the zero matrix is used to extend the zero element in the base matrix, and the cyclic shift matrix is used to extend the non-zero element in the base matrix; the number of check rows is associated with the number of rows of the base matrix.

[0078] In some embodiments, the first processing module 903 can be configured to: determine the first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, read out the first type of data from the first storage unit based on the first type of error correction code and decode the first type of data to obtain decoded first type of data; update the address mapping table of the hard disk, mark the first storage unit as a fourth storage unit, and determine a third storage unit; the error level of the third storage unit is lower than the first level threshold; encode the decoded first type of data based on the first type of error correction code to obtain encoded first type of data; migrate the encoded first type of data to the third storage unit and clear the data in the fourth storage unit.

[0079] In some embodiments, the second processing module 904 can be configured to: determine a second error level of the second storage unit; in response to the second error level being greater than a second threshold included in the second storage condition, read the second type of data from the second storage unit based on the second error correction code and decode the first type of data to obtain decoded second type of data; update the address mapping table of the hard disk, mark the second storage unit as an unusable unit, and determine a fourth storage unit; encode the decoded second type of data based on the second error correction code to obtain encoded second type of data; migrate the encoded second type of data to the fourth storage unit and clear the data in the unusable unit.

[0080] In some embodiments, the first error correction code and the second error correction code are both low density parity check codes (LDPC).

[0081] It should be noted that the description of the device embodiments of the present application is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments, and thus will not be described here. For technical details not described in the device embodiments of the present application, they can be understood according to the description of any one of the accompanying drawings. Figures 1 to 3

[0082] According to the embodiments of the present application, the present application further provides an electronic device and a non-transitory computer readable storage medium.

[0083] Figure 5 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0084] As Figure 5 ​As shown, the electronic device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in the ROM 802 or a computer program loaded into the RAM 803 from the storage unit 808. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An I / O interface 805 is also connected to the bus 804.

[0085] A plurality of components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806 such as a keyboard, a mouse, and the like, an output unit 807 such as various types of displays, a speaker, and the like, a storage unit 808 such as a hard disk, an optical disk, and the like, and a communication unit 809 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0086] The computing unit 801 can be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 801 performs various methods and processes described above, such as the hard disk data processing method. For example, in some embodiments, the hard disk data processing method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the hard disk data processing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the hard disk data processing method by any other appropriate means, such as by means of firmware.

[0087] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0088] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or server.

[0089] In the context of the present application, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0091] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0092] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is generally established by computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0093] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without departing from the desired results of the technology disclosed herein, which are not limited herein.

[0094] In addition, the terms "first", "second", etc., are used herein only to describe different instances, and do not imply or suggest relative importance or imply the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0095] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A hard disk data processing method, characterized by, The method comprises: acquiring data to be processed in a hard disk; classifying the data to be processed based on access frequency of the data to be processed to obtain first data and second data; the access frequency of the first data is greater than the access frequency of the second data; in response to a first storage unit storing the first data not satisfying a first storage condition, performing data processing on the first data based on a first error correction code; in response to a second storage unit storing the second data not satisfying a second storage condition, performing data processing on the second data based on a second error correction code; the error correction speed of the first error correction code is higher than the error correction speed of the second error correction code.

2. The method of claim 1, wherein, The acquisition of the data to be processed in the hard disk comprises: obtaining initial data and a first error correction code in a system; the first error correction code is stored in a memory of the system; performing data processing on the initial data based on the first error correction code to obtain the data to be processed; storing the data to be processed in a storage unit of the hard disk.

3. The method according to claim 1, characterized in that The classification of the data to be processed based on the access frequency of the data to be processed to obtain the first data and the second data comprises: acquiring the access frequency of the data to be processed in the hard disk; determining the data to be processed with an access frequency greater than a set frequency threshold as the first data; determining the data to be processed with an access frequency less than or equal to the set frequency threshold as the second data.

4. The method of claim 1, wherein, The method further comprises: generating the first error correction code and the second error correction code based on a set base matrix; the number of rows and columns of the base matrix is less than the number of rows and columns of the first error correction code, and the number of rows and columns of the base matrix is less than the number of rows and columns of the second error correction code.

5. The method according to claim 4, characterized in that The generation of the first error correction code and the second error correction code based on the set base matrix comprises: extending the base matrix based on a first extension parameter and a first number of check rows to obtain the first error correction code; extending the base matrix based on a second extension parameter and a second number of check rows to obtain the second error correction code; the first extension parameter is less than the second extension parameter, and the first number of check rows is less than the second number of check rows.

6. The method of claim 5, wherein: an extension parameter is associated with a size of a sub-matrix corresponding to each element in the base matrix; the sub-matrix comprises a zero matrix and a cyclic shift matrix; the zero matrix is used for extending a zero element in the base matrix, and the cyclic shift matrix is used for extending a non-zero element in the base matrix; a number of check rows is associated with a number of rows of the base matrix.

7. The method of claim 1, wherein, The data processing on the first data based on the first error correction code in response to the first storage unit storing the first data not satisfying the first storage condition comprises: determining a first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, reading the first type of data from the first storage unit and decoding the first type of data based on a first type of error correction code to obtain decoded first type of data; updating an address mapping table of the hard disk to mark the first storage unit as a fourth storage unit and determining a third storage unit; the third storage unit has an error level lower than the first level threshold; encoding the decoded first type of data based on the first type of error correction code to obtain encoded first type of data; migrating the encoded first type of data to the third storage unit and clearing data in the fourth storage unit.

8. The method of claim 7, wherein, The response to the second storage unit storing the second type of data not satisfying the second storage condition includes: determining a second error level of the second storage unit; in response to the second error level being greater than a second level threshold included in the second storage condition, reading the second type of data from the second storage unit and decoding the first type of data based on a second type of error correction code to obtain decoded second type of data; updating an address mapping table of the hard disk to mark the second storage unit as an unusable unit and determining the fourth storage unit; encoding the decoded second type of data based on the second type of error correction code to obtain encoded second type of data; migrating the encoded second type of data to the fourth storage unit and clearing data in the unusable unit.

9. The method of claim 1, wherein, The first type of error correction code and the second type of error correction code are both low-density parity-check codes (LDPC).

10. A hard disk data processing apparatus, characterized by comprising: The apparatus includes: an acquisition module configured to acquire to-be-processed data in a hard disk; a classification module configured to classify the to-be-processed data based on access frequencies of the to-be-processed data to obtain first type of data and second type of data; the first type of data has a higher access frequency than the second type of data; a first processing module configured to, in response to a first storage unit storing the first type of data not satisfying a first storage condition, perform data processing on the first type of data based on a first type of error correction code; a second processing module configured to, in response to a second storage unit storing the second type of data not satisfying a second storage condition, perform data processing on the second type of data based on a second type of error correction code; the first type of error correction code has a higher error correction speed than the second type of error correction code.

11. The apparatus of claim 10, wherein, The acquisition module is configured to: obtain initial data and a first type of error correction code in a system; the first type of error correction code is stored in a memory of the system; perform data processing on the initial data based on the first type of error correction code to obtain the to-be-processed data; store the to-be-processed data in a storage unit of the hard disk.

12. The apparatus of claim 10, wherein, The classification module is configured to: obtain access frequencies of the to-be-processed data in the hard disk; determine to-be-processed data with an access frequency greater than a set frequency threshold as first type of data; determine to-be-processed data with an access frequency less than or equal to the set frequency threshold as second type of data.

13. The apparatus of claim 10, wherein, The apparatus further comprises a generating module, configured to: generate the first error correction code and the second error correction code based on a set base matrix; a number of rows and a number of columns of the base matrix are less than a number of rows and a number of columns of the first error correction code, and the number of rows and the number of columns of the base matrix are less than a number of rows and a number of columns of the second error correction code.

14. The apparatus of claim 13, wherein, The generating module is configured to: extend the base matrix based on a first extension parameter and a first number of check rows to obtain the first error correction code; extend the base matrix based on a second extension parameter and a second number of check rows to obtain the second error correction code; the first extension parameter is less than the second extension parameter, and the first number of check rows is less than the second number of check rows.

15. The apparatus of claim 14, wherein: an extension parameter is associated with a size of a sub-matrix corresponding to each element in the base matrix; the sub-matrix comprises a zero matrix and a cyclic shift matrix; the zero matrix is used to extend a zero element in the base matrix, and the cyclic shift matrix is used to extend a non-zero element in the base matrix; a number of check rows is associated with a number of rows of the base matrix.

16. The apparatus of claim 10, wherein, The first processing module is configured to: determine a first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, read out the first type of data from the first storage unit based on the first error correction code and decode the first type of data to obtain decoded first type of data; update an address mapping table of the hard disk to mark the first storage unit as a fourth storage unit and determine a third storage unit; the third storage unit has an error level lower than the first level threshold; encode the decoded first type of data based on the first error correction code to obtain encoded first type of data; migrate the encoded first type of data to the third storage unit and clear data in the fourth storage unit.

17. The device according to claim 16, characterized in that The second processing module is configured to: determine a second error level of the second storage unit; in response to the second error level being greater than a second level threshold included in the second storage condition, read out the second type of data from the second storage unit based on the second error correction code and decode the first type of data to obtain decoded second type of data; update an address mapping table of the hard disk to mark the second storage unit as an unusable unit and determine the fourth storage unit; encode the decoded second type of data based on the second error correction code to obtain encoded second type of data; migrate the encoded second type of data to the fourth storage unit and clear data in the unusable unit.

18. The apparatus of claim 10, wherein, The first error correction code and the second error correction code are both low density parity check codes (LDPC).

19. An electronic device, comprising: comprise: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

20. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are for causing a computer to perform the method of any one of claims 1-9.

Images