A hard disk data processing method and device, electronic equipment and storage medium

Abstract

The application provides a hard disk data processing method and device, electronic equipment and storage medium; it relates to the technical field of computers, and the method comprises the following steps: obtaining to-be-processed data in a hard disk; based on the access frequency of the to-be-processed data, the to-be-processed data is classified to obtain first-class data and second-class data; the access frequency of the first-class data is greater than the access frequency of the second-class data; in response to a first storage unit storing the first-class data not meeting a first storage condition, the first-class data is subjected to data processing based on a first error correction code; in response to a second storage unit storing the second-class data not meeting a second storage condition, the second-class data is subjected to data processing based on a second error correction code; the error correction speed of the first error correction code is higher than the error correction speed of the second error correction code. Through the application, the service life of the hard disk can be improved, and the data processing efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a hard disk data processing method, apparatus, electronic device and storage medium. Background Technology

[0002] With the surge in data volume and the continuous increase in storage demand, the hard drive market has shown significant growth. NAND Flash (NOT AND Flash) technology has evolved from storing 1 bit of data per cell to storing 3 / 4 bits of data per cell, resulting in a substantial increase in storage density. However, this advancement has also brought challenges such as increased inter-cell charge interference, decreased data retention capacity, and higher read / write error rates, posing a severe test to the hard drive's ability to ensure the reliability of Flash data.

[0003] Traditional error correction codes, such as BCH codes (a type of cyclic error correction code), are increasingly proving unreliable in high-error-rate scenarios. Therefore, Low-Density Parity-Check (LDPC) codes, as a high-performance error correction scheme with strong error correction capabilities, have been introduced into hard drive controller chips and have become the mainstream error correction method in modern high-density storage devices. LDPC is a probabilistic ECC (Error Correction Code) error correction mechanism. When the NAND Flash bit error rate is high, the number of iterations needs to be increased, leading to prolonged LDPC decoding latency and impacting Quality of Service (QoS).

[0004] For high-density NAND Flash, the bit error rate (BER) is typically high, and this increase is even more pronounced with each program / erase (P / E) cycle. Currently, the main strategy for improving LDPC decoding efficiency relies on optimizing the LDPC algorithm. However, given the characteristics of NAND Flash, its BER rises with the number of P / E cycles, and there are differences in BER between different blocks. Therefore, simply optimizing the LDPC algorithm is insufficient to comprehensively improve data processing efficiency while simultaneously considering the lifespan of the hard drive. Summary of the Invention

[0005] This application provides a hard disk data processing method, apparatus, electronic device, and storage medium.

[0006] This application provides a hard disk data processing method, the method comprising: acquiring data to be processed from the hard disk; classifying the data to be processed based on the access frequency of the data to be processed to obtain a first type of data and a second type of data; wherein the access frequency of the first type of data is greater than the access frequency of the second type of data; in response to a first storage unit storing the first type of data not meeting a first storage condition, processing the first type of data based on a first type of error correction code; in response to a second storage unit storing the second type of data not meeting a second storage condition, processing the second type of data based on a second type of error correction code; wherein the error correction speed of the first type of error correction code is higher than the error correction speed of the second type of error correction code.

[0007] According to one embodiment of this application, obtaining the data to be processed from the hard disk includes: obtaining initial data and a first type of error correction code in the system; storing the first type of error correction code in the memory of the system; performing data processing on the initial data based on the first type of error correction code to obtain the data to be processed; and storing the data to be processed in the storage unit of the hard disk.

[0008] According to one embodiment of this application, classifying the data to be processed based on the access frequency of the data to be processed to obtain a first type of data and a second type of data includes: obtaining the access frequency of the data to be processed in the hard disk; determining the data to be processed with an access frequency greater than a set frequency threshold as the first type of data; and determining the data to be processed with an access frequency less than or equal to the set frequency threshold as the second type of data.

[0009] According to one embodiment of this application, the method further includes: generating a first type of error correction code and a second type of error correction code based on a set base matrix; wherein the number of rows and columns of the base matrix is ​​less than the number of rows and columns of the first type of error correction code, and the number of rows and columns of the base matrix is ​​less than the number of rows and columns of the second type of error correction code.

[0010] According to one embodiment of this application, generating the first type of error correction code and the second type of error correction code based on a set base matrix includes: expanding the base matrix based on a first extension parameter and a first number of check rows to obtain the first type of error correction code; expanding the base matrix based on a second extension parameter and a second number of check rows to obtain the second type of error correction code; wherein the first extension parameter is less than the second extension parameter, and the first number of check rows is less than the second number of check rows.

[0011] According to one embodiment of this application, the expansion parameter is associated with the size of the submatrix corresponding to each element in the base matrix; the submatrix includes a zero matrix and a cyclic shift matrix; the zero matrix is ​​used to expand the zero elements in the base matrix, and the cyclic shift matrix is ​​used to expand the non-zero elements in the base matrix; the number of check rows is associated with the number of rows in the base matrix.

[0012] According to one embodiment of this application, in response to the first storage unit storing the first type of data not meeting the first storage condition, data processing of the first type of data based on a first type of error correction code includes: determining a first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, reading the first type of data from the first storage unit and decoding the first type of data based on the first type of error correction code to obtain decoded first type of data; updating the address mapping table of the hard disk, marking the first storage unit as a fourth storage unit, and determining a third storage unit; the error level of the third storage unit being lower than the first level threshold; encoding the decoded first type of data based on the first type of error correction code to obtain encoded first type of data; migrating the encoded first type of data to the third storage unit and clearing the data in the fourth storage unit.

[0013] According to one embodiment of this application, in response to the second storage unit storing the second type of data not meeting the second storage condition, data processing of the second type of data based on the second type of error correction code includes: determining a second error level of the second storage unit; in response to the second error level being greater than the second level threshold included in the second storage condition, reading the second type of data from the second storage unit and decoding the first type of data based on the second type of error correction code to obtain decoded second type of data; updating the address mapping table of the hard disk, marking the second storage unit as an unavailable unit, and determining the fourth storage unit; encoding the decoded second type of data based on the second type of error correction code to obtain encoded second type of data; migrating the encoded second type of data to the fourth storage unit and clearing the data in the unavailable unit.

[0014] According to one embodiment of this application, both the first type of error correction code and the second type of error correction code are low-density parity-check codes (LDPC).

[0015] This application also provides a hard disk data processing apparatus, the apparatus comprising: an acquisition module for acquiring data to be processed in the hard disk; a classification module for classifying the data to be processed based on the access frequency of the data to be processed to obtain a first type of data and a second type of data; wherein the access frequency of the first type of data is greater than the access frequency of the second type of data; a first processing module for processing the first type of data based on a first type of error correction code in response to a first storage unit storing the first type of data not meeting a first storage condition; and a second processing module for processing the second type of data based on a second type of error correction code in response to a second storage unit storing the second type of data not meeting a second storage condition; wherein the error correction speed of the first type of error correction code is higher than the error correction speed of the second type of error correction code.

[0016] According to one embodiment of this application, the acquisition module is configured to: acquire initial data and a first type of error correction code in the system; store the first type of error correction code in the memory of the system; perform data processing on the initial data based on the first type of error correction code to obtain the data to be processed; and store the data to be processed in the storage unit of the hard disk.

[0017] According to one embodiment of this application, the classification module is used to: obtain the access frequency of the data to be processed in the hard disk; determine the data to be processed with an access frequency greater than a set frequency threshold as a first type of data; and determine the data to be processed with an access frequency less than or equal to the set frequency threshold as a second type of data.

[0018] According to one embodiment of this application, the apparatus further includes a generation module, which is configured to: generate a first type of error correction code and a second type of error correction code based on a set base matrix; wherein the number of rows and columns of the base matrix is ​​less than the number of rows and columns of the first type of error correction code, and the number of rows and columns of the base matrix is ​​less than the number of rows and columns of the second type of error correction code.

[0019] According to one embodiment of this application, the generation module is configured to: expand the base matrix based on a first expansion parameter and a first number of check rows to obtain a first type of error correction code; expand the base matrix based on a second expansion parameter and a second number of check rows to obtain a second type of error correction code; wherein the first expansion parameter is less than the second expansion parameter, and the first number of check rows is less than the second number of check rows.

[0020] According to one embodiment of this application, the expansion parameter is associated with the size of the submatrix corresponding to each element in the base matrix; the submatrix includes a zero matrix and a cyclic shift matrix; the zero matrix is ​​used to expand the zero elements in the base matrix, and the cyclic shift matrix is ​​used to expand the non-zero elements in the base matrix; the number of check rows is associated with the number of rows in the base matrix.

[0021] According to one embodiment of this application, the first processing module is configured to: determine a first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, read the first type of data from the first storage unit based on a first type of error correction code and decode the first type of data to obtain decoded first type of data; update the address mapping table of the hard disk, mark the first storage unit as a fourth storage unit, and determine a third storage unit; the error level of the third storage unit is lower than the first level threshold; encode the decoded first type of data based on the first type of error correction code to obtain encoded first type of data; migrate the encoded first type of data to the third storage unit and clear the data in the fourth storage unit.

[0022] According to one embodiment of this application, the second processing module is configured to: determine a second error level of the second storage unit; in response to the second error level being greater than a second level threshold included in the second storage condition, read the second type of data from the second storage unit based on a second type of error correction code and decode the first type of data to obtain decoded second type of data; update the address mapping table of the hard disk, mark the second storage unit as an unavailable unit, and determine the fourth storage unit; encode the decoded second type of data based on the second type of error correction code to obtain encoded second type of data; migrate the encoded second type of data to the fourth storage unit and clear the data in the unavailable unit.

[0023] According to one embodiment of this application, both the first type of error correction code and the second type of error correction code are low-density parity-check codes (LDPC).

[0024] This application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of the above-described embodiments.

[0025] This application also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method according to the above embodiments.

[0026] The method of this application embodiment acquires data to be processed from a hard disk; classifies the data to be processed based on its access frequency to obtain a first type of data and a second type of data; the access frequency of the first type of data is greater than that of the second type of data; in response to a first storage unit storing the first type of data not meeting a first storage condition, data processing is performed on the first type of data based on a first type of error correction code; in response to a second storage unit storing the second type of data not meeting a second storage condition, data processing is performed on the second type of data based on a second type of error correction code; the error correction speed of the first type of error correction code is higher than that of the second type of error correction code. This application can improve the lifespan of the hard disk and increase data processing efficiency.

[0027] It should be understood that the teachings of this application are not required to achieve all the beneficial effects described above, but rather that a specific technical solution can achieve a specific technical effect, and other embodiments of this application can also achieve beneficial effects not mentioned above. Attached Figure Description

[0028] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which:

[0029] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0030] Figure 1 This paper illustrates a schematic diagram of the hard disk data processing method provided in an embodiment of this application. Figure 1 ;

[0031] Figure 2 This paper illustrates a schematic diagram of the hard disk data processing method provided in an embodiment of this application. Figure 2 ;

[0032] Figure 3 This illustration shows an application scenario of the hard disk data processing method provided in an embodiment of this application.

[0033] Figure 4 This illustration shows an optional schematic diagram of a hard disk data processing apparatus provided in an embodiment of this application;

[0034] Figure 5 An optional schematic diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0035] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0037] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0039] The processing flow of the hard disk data processing method provided in the embodiments of this application will be described. See [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic diagram of the hard disk data processing method provided in the embodiments of this application. Figure 1 , will combine Figure 1 Steps S101-S104 are explained below.

[0040] Step S101: Obtain the data to be processed from the hard disk.

[0041] In some embodiments, the data to be processed may include data stored on a hard disk. The data to be processed can be accessed. The hard disk may include a solid-state drive (SSD) and a hard disk drive (HDD). The hard disk may also include other types of hard disks, and this application embodiment does not limit the specific hard disk.

[0042] Step S102: Based on the access frequency of the data to be processed, classify the data to be processed to obtain a first category of data and a second category of data; the access frequency of the first category of data is greater than that of the second category of data.

[0043] In some embodiments, access frequency may include the number of times the data to be processed is read or written within a preset time interval. The first type of data may include data with a high access frequency, i.e., hot data. Hot data has higher requirements for read performance. The second type of data may include data with a low access frequency, i.e., cold data. Cold data has lower requirements for read performance than hot data.

[0044] Step S103: In response to the first storage unit storing the first type of data not meeting the first storage condition, data processing is performed on the first type of data based on the first type of error correction code.

[0045] In some embodiments, the first storage unit may include a physical block on a hard disk for storing a first type of data. A first storage condition can characterize the health status of the first storage unit; for example, if the number of error bits is determined to be higher than a certain threshold when reading data from the first storage unit, then the first storage unit is determined not to meet the first storage condition. A first type of error correction code may include an error correction code that has fast error correction speed but weak error correction capability. The first type of error correction code can be used to quickly process errors in the first type of data.

[0046] Step S104: In response to the second storage unit storing the second type of data not meeting the second storage condition, data processing is performed on the second type of data based on the second type of error correction code; the error correction speed of the first type of error correction code is higher than that of the second type of error correction code.

[0047] In some embodiments, the second storage unit may include physical blocks on a hard disk for storing second-type data. The second storage condition can characterize the health status of the second storage unit; for example, if the number of error bits is determined to be higher than a certain threshold when reading data from the second storage unit, then the second storage unit is determined not to meet the second storage condition. The second type of error correction code may include error correction codes that are slow to correct errors but have strong error correction capabilities. The second type of error correction code can be used to efficiently repair errors in the second-type data. Both the first and second type of error correction codes can be LDPC.

[0048] As an example, data to be processed is read from the hard drive. This data can specifically include files, database records, or logs. Data is categorized based on its access frequency: frequently accessed data is classified into Category 1, and infrequently accessed data into Category 2. Then, it is determined whether the first storage unit storing Category 1 data meets a first storage condition. If not, the Category 1 data is decoded and encoded using a first type of error correction code, and the encoded data is migrated to other storage units. Similarly, it is determined whether the second storage unit storing Category 2 data meets a second storage condition. If not, the data is decoded and encoded using a second type of error correction code, and the encoded data is migrated to other storage units.

[0049] The method in this application uses two sets of error correction codes, strong and weak, in the hard drive. The strong error correction code has a greater error correction capability than the weak error correction code and can correct data when the number of error bits is large. However, the iteration time of the strong error correction code is longer than that of the weak error correction code. To balance hard drive performance and lifespan, the weak error correction code is used during the initial data write. As the number of write cycles increases, when the number of error bits reaches a threshold and needs to be moved, hot and cold data are distinguished. The weak error correction code is still used for hot data, while the strong error correction code is used for cold data. Furthermore, to avoid the decoding latency being higher than that of the strong error correction code due to excessive iterations when using the weak error correction code for decoding as the number of error bits increases, this method distinguishes between hot and cold data thresholds. When the error level of the block storing hot data exceeds the threshold, and the data needs to be moved, the block is marked. Subsequently, when cold data needs to be moved, the data can be moved to this block, thereby improving the lifespan of the hard drive. Error correction codes can be dynamically adjusted, have simple rules, are easy to implement in hardware, and the same set of basic matrices can generate error correction codes with various code lengths and code rates.

[0050] By using different error correction codes for hot and cold data, the data processing efficiency of hard drives can be effectively improved. Hot data is always stored in storage cells with low wear, and weak error correction codes (which take less time for each iteration) are consistently used for encoding and decoding. Because the number of error bits is low, weak error correction codes can correct the errors, greatly improving the data processing efficiency for hot data. Cold data, on the other hand, is stored in storage cells with higher wear, and strong error correction codes are used for encoding and decoding during data movement and rewriting. This approach maximizes the data processing efficiency of hot and cold data while also separating them, optimizing data layout and extending the lifespan of the hard drive.

[0051] In some embodiments, obtaining the data to be processed from the hard disk in step S101 may include: obtaining initial data and a first type of error correction code in the system; storing the first type of error correction code in the system's memory; performing data processing on the initial data based on the first type of error correction code to obtain the data to be processed; and storing the data to be processed in the storage unit of the hard disk.

[0052] In some embodiments, the initial data may include: data written to the hard disk for the first time in the system. The first type of error correction code may include LDPC stored in system memory (DRAM). LDPC is a type of ECC and can be used to quickly detect and correct errors occurring during data transmission or storage. The fundamental matrix of the LDPC can be loaded into DRAM during system initialization. Data processing may include operations such as decoding and error correction encoding of the initial data. The data to be processed may include: data after the first error correction encoding process.

[0053] As an example, when the system's log file is first stored on the hard drive, the system uses LDPC error correction coding in memory to perform error correction encoding on the log file, generating data to be processed. If errors are found in the log file during the error correction encoding process, LDPC is used to repair the errors. Finally, the data to be processed is written to an available storage unit on the hard drive, and the hard drive's address mapping table is updated.

[0054] In some embodiments, the step S102 of classifying the data to be processed based on the access frequency of the data to be processed to obtain a first type of data and a second type of data may include: obtaining the access frequency of the data to be processed in the hard disk; determining the data to be processed with an access frequency greater than a set frequency threshold as the first type of data; and determining the data to be processed with an access frequency less than or equal to the set frequency threshold as the second type of data.

[0055] In some embodiments, the frequency threshold may include a set standard value for distinguishing between high-frequency and low-frequency data, such as 10 times / hour. The first type of data may include unprocessed data on the hard drive with an access frequency greater than the frequency threshold. The second type of data may include unprocessed data on the hard drive with an access frequency less than or equal to the frequency threshold.

[0056] As an example, the system first collects the access frequency of each piece of data to be processed on the hard drive, counting the number of times each piece of data was read or written in the past hour. The system sets a frequency threshold of 10 times per hour. Data with an access frequency greater than the threshold is classified as Category 1 data, and data with an access frequency less than or equal to the threshold is classified as Category 2 data. The storage unit for Category 1 data is designated as the first storage unit, and the storage unit for Category 2 data is designated as the second storage unit.

[0057] In some embodiments, the processing flow of the hard disk data processing method is illustrated. Figure 2 ,like Figure 2 As shown, hard disk data processing methods may also include:

[0058] Step S201: Obtain the data to be processed from the hard disk.

[0059] Step S202: Based on the access frequency of the data to be processed, classify the data to be processed to obtain the first type of data and the second type of data.

[0060] The explanations for steps S201 and S202 are the same as those for steps S101 and S102, and will not be repeated here.

[0061] Step S203: Based on the set basic matrix, generate the first type of error correction code and the second type of error correction code.

[0062] In some embodiments, step S203, generating a first type of error correction code and a second type of error correction code based on a set base matrix, includes: expanding the base matrix based on a first extension parameter and a first number of check rows to obtain a first type of error correction code; and expanding the base matrix based on a second extension parameter and a second number of check rows to obtain a second type of error correction code.

[0063] In some embodiments, the number of rows and columns of the base matrix is ​​less than that of the first type of error correction code, and the number of rows and columns of the base matrix is ​​less than that of the second type of error correction code. The first expansion parameter is less than the second expansion parameter, and the first number of check rows is less than the second number of check rows. The expansion parameter is associated with the size of the submatrix corresponding to each element in the base matrix; the submatrix includes a zero matrix and a cyclic shift matrix; the zero matrix is ​​used to expand the zero elements in the base matrix, and the cyclic shift matrix is ​​used to expand the non-zero elements in the base matrix; the number of check rows is associated with the number of rows in the base matrix. During the generation of error correction codes based on the base matrix, the submatrix is ​​used to expand each element of the base matrix. For zero elements in the base matrix, the corresponding submatrix is ​​the zero matrix; while for non-zero elements, the corresponding submatrix is ​​the cyclic shift matrix, and the shift step size of the cyclic shift matrix is ​​determined by the value of the non-zero element. By adjusting the expansion parameter and the number of check rows, different error correction codes can be generated.

[0064] As an example, we first configure an M×N (M rows, N columns) base matrix. The elements of the base matrix are integers 0, 1, 2, 3, etc. Each non-zero element k (k>= 0) not only indicates a connection at that position but also implicitly contains the shift value of the cyclic shift matrix for subsequent expansion. For each element in the base matrix, if the element value is 0, it corresponds to a Z×Z zero matrix in the expanded matrix; if the element is a non-zero value, it corresponds to a Z×Z cyclic shift matrix in the expanded matrix. The cyclic shift matrix can be obtained by cyclically shifting the identity matrix by s, where s=k mod Z. For example, if Z=3 and k=1, the original identity matrix is ​​[0,0,1;0,1,0;1,0,0], and the cyclically shifted matrix is ​​[1,0,0;0,0,1;0,1,0]. After expanding the base matrix, we can obtain an LDPC parity check matrix of size (M×Z)×(N×Z).

[0065] Type I error-correcting codes can be LDPCs with fast error correction speed but weak error correction capability, while Type II error-correcting codes can be LDPCs with slow error correction speed but strong error correction capability. Dynamically adjusting the extension parameters and the number of parity rows corresponding to the LDPC can change its error correction speed and capability. This embodiment includes two adjustment directions. First, by dynamically adjusting the code length, the error correction speed and capability of the LDPC can be adjusted. With a well-designed base matrix, a longer code length generally results in a stronger error correction capability. In this embodiment, this corresponds to adjusting the size of the extension matrix Z×Z; the larger the extension parameter Z, the stronger the error correction capability of the LDPC. Second, by adjusting the code rate, the more redundant parity information contained in the LDPC, the stronger its error correction capability. That is, the larger the number of parity rows M in the base matrix, the more parity equations are generated, and thus the stronger the error correction capability of the corresponding LDPC. Type I error-correcting codes can be generated by expanding the base matrix after reducing the number of rows M, while Type II error-correcting codes can be generated by expanding the complete base matrix. Type I and Type II error correction codes use the same codec. During generation, only the extension parameters and the number of check rows need to be reconfigured. That is, Type I error correction codes are generated based on the first extension parameters and the first number of check rows, and Type II error correction codes are generated based on the second extension parameters and the second number of check rows. The first extension parameter is less than the second extension parameter, and the first number of check rows is less than the second number of check rows.

[0066] In some embodiments, step S103, in response to the first storage unit storing the first type of data not meeting the first storage condition, performs data processing on the first type of data based on the first type of error correction code, including: determining the first error level of the first storage unit; in response to the first error level being greater than the first level threshold included in the first storage condition, reading the first type of data from the first storage unit and decoding the first type of data based on the first type of error correction code to obtain decoded first type of data; updating the address mapping table of the hard disk, marking the first storage unit as the fourth storage unit, and determining the third storage unit; the error level of the third storage unit being lower than the first level threshold; encoding the decoded first type of data based on the first type of error correction code to obtain encoded first type of data; migrating the encoded first type of data to the third storage unit and clearing the data in the fourth storage unit.

[0067] In some embodiments, the first error level can be an indicator describing the health status of the first storage unit. Specifically, it can be the number of error bits returned by the LDPC module of the hard disk each time data is read from the NAND Flash of the hard disk for LDPC correction. The higher the number of error bits, the higher the first error level. The first level threshold can be a preset health status threshold; when the first error level exceeds this value, the storage unit needs to be operated. The decoded first type of data can be the data read from the first storage unit after the FTL (Flash Translation Layer) decodes the first type of data using the first type of error correction code. The address mapping table can include a mapping table recording the relationship between logical addresses and physical addresses in the hard disk system; specifically, the address mapping table can be an FTL mapping table. The third storage unit can include: a healthy physical block in the hard disk. The third storage unit can be used to receive and store the decoded first type of data, replacing the old first storage unit. The fourth storage unit is obtained by marking the old first storage unit; the marking can instruct the system to move the second type of data to the fourth storage unit.

[0068] As an example, the first type of error correction code can be LDPC with weak error correction capability. The first type of data in the solid-state drive (SSD) is a group of frequently used user files stored in the first storage unit. The error level of the first storage unit is determined to be 5, exceeding the preset first-level threshold of 3. The SSD's FTL (Fault Tolerance) reads the first type of data from the first storage unit after decoding with the weak LDPC and updates the FTL mapping table, marking the first storage unit as the fourth storage unit. It also identifies a healthy third storage unit (error level 1, below threshold 3). The FTL then writes the decoded first type of data to the third storage unit after encoding with the weak LDPC and clears the old data in the fourth storage unit.

[0069] In some embodiments, in step S104, in response to the second storage unit storing the second type of data not meeting the second storage condition, data processing is performed on the second type of data based on the second type of error correction code, including: determining the second error level of the second storage unit; in response to the second error level being greater than the second level threshold included in the second storage condition, reading the second type of data from the second storage unit and decoding the first type of data based on the second type of error correction code to obtain the decoded second type of data; updating the address mapping table of the hard disk, marking the second storage unit as an unavailable unit, and determining the fourth storage unit; encoding the decoded second type of data based on the second type of error correction code to obtain the encoded second type of data; migrating the encoded second type of data to the fourth storage unit and clearing the data in the unavailable unit.

[0070] In some embodiments, the second error level can be an indicator describing the health status of the second storage unit. Specifically, it can be the number of error bits returned by the LDPC correction module of the hard disk each time data is read from the NAND Flash of the hard disk. The higher the number of error bits, the higher the second error level. The second level threshold can be a preset health status threshold; when the second error level exceeds this value, the storage unit needs to be operated on. The decoded second type of data can be the data read from the second storage unit after the second type of data has been decoded by the FTL using the second type of error correction code. Unusable units can include storage units whose error level exceeds the second level threshold. Unusable units can specifically be bad blocks on the hard disk.

[0071] As an example, the second type of error correction code can be LDPC with strong error correction capabilities. The second type of data in the solid-state drive (SSD) is a set of user backup files stored in the second storage unit. The error level of the second storage unit is determined to be 8, exceeding the preset second-level threshold of 6. The SSD's FTL (Fault Tolerance) reads the second type of data from the second storage unit after decoding with LDPC and updates the FTL mapping table, marking the second storage unit as unavailable. A healthy fourth storage unit (error level 5, below threshold 6) is identified. The FTL then writes the decoded second type of data to the fourth storage unit after encoding with LDPC and clears the old data in the unavailable unit.

[0072] Figure 3 This illustration shows an application scenario of the hard disk data processing method provided in an embodiment of this application.

[0073] refer to Figure 3 This application provides an example of a hard disk data processing method for solid-state hard disk data processing.

[0074] Data accessed more frequently than a threshold T is described as "hot data," and data accessed less frequently than a threshold T is described as "cold data." The threshold T can be a set frequency threshold.

[0075] During system initialization, the basic matrix is ​​loaded into DRAM, and LDPC with weak error correction capability is configured by default. When data is initially written to the hard drive, it is impossible to determine whether the data is cold or hot; therefore, all data is encoded using LDPC with weak error correction capability, prioritizing the writing of blocks with lower wear (those with fewer error bits returned by the LDPC module). Each time data is read from the NAND Flash of the solid-state drive, the LDPC module returns an error bit count, which is used to determine the error level. Based on the comparison between the error level and a preset threshold, it is determined whether to perform a data migration operation.

[0076] Hot data processing: If the error level of hot data is less than the threshold L1, no additional data migration operation is performed. If the error level of hot data exceeds the threshold L1, the block is considered to have a high error level and the data needs to be migrated to a more reliable block. ECC encoding with weak error correction capabilities is used during migration, and the original storage block is marked as usable for cold data storage after the operation is completed.

[0077] Cold data processing: If the error level of cold data is less than the threshold L2, no additional data migration operation is performed. If the error level of cold data exceeds the threshold L2, the data is migrated to a pre-marked block, and ECC encoding with strong error correction capability is switched during migration. When the wear level of a block approaches the limit of the error correction capability of the strong error correction capability ECC encoding, the block is marked as a bad block.

[0078] Understandable. Figure 3 The application scenarios of the hard disk data processing method in this application are only some exemplary implementations in the embodiments of this application. The application scenarios of the hard disk data processing method in the embodiments of this application include, but are not limited to, those of other methods. Figure 3 The application scenarios of the hard disk data processing method shown are illustrated.

[0079] The exemplary structure of the software modules included in the hard disk data processing apparatus 90 provided in this application embodiment will be further described below. In some embodiments, such as Figure 4 As shown, the hard disk data processing device 90 may include: an acquisition module 901, used to acquire data to be processed in the hard disk;

[0080] The classification module 902 is used to classify the data to be processed based on the access frequency of the data to be processed, and obtain a first category of data and a second category of data; the access frequency of the first category of data is greater than the access frequency of the second category of data.

[0081] The first processing module 903 is used to process the first type of data based on the first type of error correction code in response to the first storage unit storing the first type of data not meeting the first storage condition.

[0082] The second processing module 904 is used to process the second type of data based on the second type of error correction code in response to the second storage unit storing the second type of data not meeting the second storage condition; the error correction speed of the first type of error correction code is higher than that of the second type of error correction code.

[0083] In some embodiments, the acquisition module 901 can be used for:

[0084] Obtain the initial data and Type I error correction code from the system; the Type I error correction code is stored in the system's memory;

[0085] The initial data is processed based on the first type of error correction code to obtain the data to be processed.

[0086] The data to be processed is stored in the storage units of the hard disk.

[0087] In some embodiments, the classification module 902 can be used for:

[0088] Obtain the access frequency of the data to be processed on the hard drive;

[0089] Data whose access frequency exceeds the set frequency threshold is identified as the first type of data.

[0090] Data to be processed with an access frequency less than or equal to a set frequency threshold is classified as the second type of data.

[0091] In some embodiments, the hard disk data processing apparatus 90 further includes a generation module, which is configured to:

[0092] Based on the defined fundamental matrix, generate first-type and second-type error-correcting codes;

[0093] The number of rows and columns in the basic matrix is ​​less than that in the first type of error correction code, and the number of rows and columns in the basic matrix is ​​less than that in the second type of error correction code.

[0094] In some embodiments, the generation module can be used for:

[0095] Based on the first extension parameter and the first number of check rows, the basic matrix is ​​extended to obtain the first type of error correction code;

[0096] Based on the second extension parameter and the second check row number, the basic matrix is ​​extended to obtain the second type of error correction code;

[0097] The first extended parameter is less than the second extended parameter, and the first number of check rows is less than the second number of check rows.

[0098] In some embodiments, the expansion parameter is associated with the size of a submatrix corresponding to each element in the base matrix; the submatrix includes a zero matrix and a cyclic shift matrix; the zero matrix is ​​used to expand the zero elements in the base matrix, and the cyclic shift matrix is ​​used to expand the non-zero elements in the base matrix.

[0099] The number of rows in the validation matrix is ​​related to the number of rows in the underlying matrix.

[0100] In some embodiments, the first processing module 903 may be used for:

[0101] Determine the first error level of the first storage unit;

[0102] In response to a first error level being greater than a first level threshold included in the first storage condition, the first type of data is read from the first storage unit and decoded based on the first type of error correction code to obtain the decoded first type of data;

[0103] Update the hard drive's address mapping table, mark the first storage unit as the fourth storage unit, and determine the third storage unit; the error level of the third storage unit is lower than the first level threshold;

[0104] Based on the first type of error correction code, the decoded first type of data is encoded to obtain the encoded first type of data;

[0105] The encoded first type of data is migrated to the third storage unit and the data in the fourth storage unit is cleared.

[0106] In some embodiments, the second processing module 904 may be used for:

[0107] Determine the second error level for the second storage unit;

[0108] In response to a second error level being greater than the second level threshold included in the second storage condition, based on the second type of error correction code, the second type of data is read from the second storage unit and the first type of data is decoded to obtain the decoded second type of data;

[0109] Update the hard drive's address mapping table, mark the second storage unit as an unavailable unit, and determine the fourth storage unit;

[0110] Based on the second type of error correction code, the decoded second type of data is encoded to obtain the encoded second type of data;

[0111] The encoded second type of data is migrated to the fourth storage unit, and the data in the unusable units is cleared.

[0112] In some embodiments, both the first type of error correction code and the second type of error correction code are low-density parity-check codes (LDPC).

[0113] It should be noted that the description of the apparatus in this application is similar to the description of the method embodiments described above, and has similar beneficial effects as the method embodiments, therefore, it will not be repeated. For any technical details not covered in the hard disk data processing apparatus provided in this application, please refer to... Figures 1 to 3 The meaning is understood in accordance with the description of any of the accompanying drawings.

[0114] According to embodiments of this application, this application also provides an electronic device and a non-transitory computer-readable storage medium.

[0115] Figure 5A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0116] like Figure 5 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in ROM 802 or a computer program loaded into RAM 803 from storage unit 808. RAM 803 can also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. I / O interface 805 is also connected to bus 804.

[0117] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as hard disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0118] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the hard disk data processing method. For example, in some embodiments, the hard disk data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the hard disk data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the hard disk data processing method by any other suitable means (e.g., by means of firmware).

[0119] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0121] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0123] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0124] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0125] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0126] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0127] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A hard disk data processing method, characterized in that, The method includes: Retrieve the data to be processed from the hard drive; Based on the access frequency of the data to be processed, the data to be processed is classified into a first category of data and a second category of data; the access frequency of the first category of data is greater than the access frequency of the second category of data. In response to the first storage unit storing the first type of data not meeting the first storage condition, data processing is performed on the first type of data based on the first type of error correction code; The step of processing the first type of data based on a first type of error correction code in response to the first storage unit not meeting a first storage condition includes: determining a first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, reading the first type of data from the first storage unit and decoding the first type of data based on the first type of error correction code to obtain decoded first type of data; updating the address mapping table of the hard disk, marking the first storage unit as a fourth storage unit, and determining a third storage unit; wherein the error level of the third storage unit is lower than the first level threshold; encoding the decoded first type of data based on the first type of error correction code to obtain encoded first type of data; and migrating the encoded first type of data to the third storage unit and clearing the data in the fourth storage unit. In response to the second storage unit storing the second type of data not meeting the second storage condition, data processing is performed on the second type of data based on the second type of error correction code; the error correction speed of the first type of error correction code is higher than that of the second type of error correction code.

2. The method according to claim 1, characterized in that, The process of acquiring the data to be processed from the hard drive includes: The system obtains initial data and a first type of error correction code; the first type of error correction code is stored in the system's memory. The initial data is processed based on the first type of error correction code to obtain the data to be processed. The data to be processed is stored in the storage unit of the hard disk.

3. The method according to claim 1, characterized in that, The process of classifying the data to be processed based on its access frequency yields a first category and a second category, including: Obtain the access frequency of the data to be processed in the hard disk; Data whose access frequency exceeds the set frequency threshold is identified as the first type of data. Data to be processed with an access frequency less than or equal to a set frequency threshold is classified as the second type of data.

4. The method according to claim 1, characterized in that, The method further includes: Based on the defined fundamental matrix, the first type of error correction code and the second type of error correction code are generated; The number of rows and columns of the basic matrix is ​​less than the number of rows and columns of the first type of error correction code, and the number of rows and columns of the basic matrix is ​​less than the number of rows and columns of the second type of error correction code.

5. The method according to claim 4, characterized in that, The generation of the first type of error correction code and the second type of error correction code based on the set fundamental matrix includes: Based on the first extended parameter and the first number of check rows, the basic matrix is ​​extended to obtain the first type of error correction code; Based on the second extension parameter and the second check row number, the basic matrix is ​​extended to obtain the second type of error correction code; The first extended parameter is less than the second extended parameter, and the first number of verification rows is less than the second number of verification rows.

6. The method according to claim 5, characterized in that, The expansion parameter is associated with the size of the submatrix corresponding to each element in the base matrix; the submatrix includes a zero matrix and a cyclic shift matrix; the zero matrix is ​​used to expand the zero elements in the base matrix, and the cyclic shift matrix is ​​used to expand the non-zero elements in the base matrix; The number of rows verified is related to the number of rows in the underlying matrix.

7. The method according to claim 1, characterized in that, In response to the second storage unit storing the second type of data not meeting the second storage condition, the second type of data is processed based on the second type of error correction code, including: Determine the second error level of the second storage cell; In response to the second error level being greater than the second level threshold included in the second storage condition, the second type of data is read from the second storage unit based on the second type of error correction code, and the first type of data is decoded to obtain the decoded second type of data; Update the address mapping table of the hard disk, mark the second storage unit as an unavailable unit, and determine the fourth storage unit; Based on the second type of error correction code, the decoded second type of data is encoded to obtain the encoded second type of data; The encoded second type of data is migrated to the fourth storage unit and the data in the unusable unit is cleared.

8. The method according to claim 1, characterized in that, Both the first type of error correction code and the second type of error correction code are low-density parity-check codes (LDPC).

9. A hard disk data processing device, characterized in that, The device includes: The acquisition module is used to acquire data to be processed from the hard drive. The classification module is used to classify the data to be processed based on the access frequency of the data to be processed, to obtain a first type of data and a second type of data; the access frequency of the first type of data is greater than the access frequency of the second type of data. The first processing module is configured to process the first type of data based on a first type of error correction code in response to the first storage unit storing the first type of data not meeting the first storage condition. The step of processing the first type of data based on a first type of error correction code in response to the first storage unit not meeting a first storage condition includes: determining a first error level of the first storage unit; in response to the first error level being greater than a first level threshold included in the first storage condition, reading the first type of data from the first storage unit and decoding the first type of data based on the first type of error correction code to obtain decoded first type of data; updating the address mapping table of the hard disk, marking the first storage unit as a fourth storage unit, and determining a third storage unit; wherein the error level of the third storage unit is lower than the first level threshold; encoding the decoded first type of data based on the first type of error correction code to obtain encoded first type of data; and migrating the encoded first type of data to the third storage unit and clearing the data in the fourth storage unit. The second processing module is used to process the second type of data based on the second type of error correction code in response to the second storage unit storing the second type of data not meeting the second storage condition; the error correction speed of the first type of error correction code is higher than that of the second type of error correction code.

10. The apparatus according to claim 9, characterized in that, The acquisition module is used for: The system obtains initial data and a first type of error correction code; the first type of error correction code is stored in the system's memory. The initial data is processed based on the first type of error correction code to obtain the data to be processed. The data to be processed is stored in the storage unit of the hard disk.

11. The apparatus according to claim 9, characterized in that, The classification module is used for: Obtain the access frequency of the data to be processed in the hard disk; Data whose access frequency exceeds the set frequency threshold is identified as the first type of data. Data to be processed with an access frequency less than or equal to a set frequency threshold is classified as the second type of data.

12. The apparatus according to claim 9, characterized in that, The apparatus further includes a generation module, the generation module being configured to: Based on the defined fundamental matrix, the first type of error correction code and the second type of error correction code are generated; The number of rows and columns of the basic matrix is ​​less than the number of rows and columns of the first type of error correction code, and the number of rows and columns of the basic matrix is ​​less than the number of rows and columns of the second type of error correction code.

13. The apparatus according to claim 12, characterized in that, The generation module is used for: Based on the first extended parameter and the first number of check rows, the basic matrix is ​​extended to obtain the first type of error correction code; Based on the second extension parameter and the second check row number, the basic matrix is ​​extended to obtain the second type of error correction code; The first extended parameter is less than the second extended parameter, and the first number of verification rows is less than the second number of verification rows.

14. The apparatus according to claim 13, characterized in that, The expansion parameter is associated with the size of the submatrix corresponding to each element in the base matrix; the submatrix includes a zero matrix and a cyclic shift matrix; the zero matrix is ​​used to expand the zero elements in the base matrix, and the cyclic shift matrix is ​​used to expand the non-zero elements in the base matrix; The number of rows verified is related to the number of rows in the underlying matrix.

15. The apparatus according to claim 9, characterized in that, The second processing module is used for: Determine the second error level of the second storage cell; In response to the second error level being greater than the second level threshold included in the second storage condition, the second type of data is read from the second storage unit based on the second type of error correction code, and the first type of data is decoded to obtain the decoded second type of data; Update the address mapping table of the hard disk, mark the second storage unit as an unavailable unit, and determine the fourth storage unit; Based on the second type of error correction code, the decoded second type of data is encoded to obtain the encoded second type of data; The encoded second type of data is migrated to the fourth storage unit and the data in the unusable unit is cleared.

16. The apparatus according to claim 9, characterized in that, Both the first type of error correction code and the second type of error correction code are low-density parity-check codes (LDPC).

17. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

Images