Data storage method and related device

Abstract

The invention discloses a data storage method and a related device, and the method comprises the steps: firstly obtaining factory data which needs to be stored through a TLC NAND technology, then compressing the factory data to obtain compressed data, dividing the compressed data into a plurality of strips, then generating a corresponding XOR check code for each strip, and storing the XOR check code in a database; and finally, the compressed data and the XOR check code corresponding to each strip are written into different physical units, so that the factory data are compressed and then put into a disk, the written data volume can be reduced, the abrasion of the embedded storage equipment can be relieved, the service life can be prolonged, and the storage efficiency can be improved by utilizing the XOR check code. The XOR check code and the survivor data are stored in different physical units in a dispersed manner, so that data recovery and reconstruction can be performed through the XOR check code and the survivor data when part of data in the strip is lost, the data integrity requirement of the embedded storage device for sudden power failure is met, the persistent retention capability of data storage is further improved, and the data storage efficiency is improved. And the reliability of data storage is improved.

Description

Technical Field

[0001] This application relates to the field of data storage technology, specifically to a data storage method and related apparatus. Background Technology

[0002] With the continuous growth of storage demands in mobile devices, embedded storage devices such as UFS and eMMC based on TLC NAND technology have become mainstream mobile storage solutions. These storage devices are widely used in mobile phones, tablets, and other terminal devices, responsible for storing operating systems, applications, and user data.

[0003] Traditional embedded storage devices primarily ensure data integrity through basic data protection mechanisms such as ECC error correction codes and flash memory conversion layers.

[0004] However, the above methods do not provide a high level of data protection for embedded storage devices. Summary of the Invention

[0005] In view of this, this application provides a data storage method and related apparatus, which effectively improves the security of data storage by partitioning and storing data through a multi-layered protection mechanism.

[0006] To solve the above problems, the technical solution provided in this application is as follows:

[0007] On the one hand, this application provides a data processing method, the method comprising:

[0008] Obtain factory data, which refers to data that needs to be stored using TLC NAND technology;

[0009] The factory data is compressed to obtain compressed data, and the compressed data is divided into multiple stripes.

[0010] Based on the compressed data, an XOR checksum corresponding to each of the multiple stripes is generated;

[0011] The compressed data and the XOR checksum corresponding to each stripe are written into different physical units.

[0012] In one possible implementation, writing the compressed data corresponding to each stripe and the XOR checksum to different physical units includes:

[0013] The compressed data and XOR checksum corresponding to each stripe are written into different channels respectively;

[0014] And / or,

[0015] The compressed data and XOR checksum corresponding to each stripe are written into different planes within the same channel.

[0016] In one possible implementation, compressing the factory data to obtain compressed data includes:

[0017] Different compression methods are preset for different data types;

[0018] Based on the different data types in the factory data, and using the corresponding compression method, compressed data is obtained.

[0019] In one possible implementation, the data types of the compressed data include metadata, hot data, warm data, and cold data. The hot data refers to data whose access frequency is greater than a first preset threshold. The warm data refers to data whose access frequency is between the first preset threshold and a second preset threshold. The cold data refers to data whose access frequency is less than the second preset threshold. The first preset threshold is greater than the second preset threshold.

[0020] The compressed data obtained by compressing the factory data includes:

[0021] The metadata is compressed using a fast lossless compression method, the hot data is compressed using a lightweight compression algorithm, the warm data is compressed using a balanced compression algorithm, and the cold data is compressed using a high ratio compression algorithm to obtain compressed data.

[0022] In one possible implementation, generating the XOR checksum corresponding to each of the plurality of stripes based on the compressed data includes:

[0023] For multiple data blocks within each of the multiple stripes, they are sequentially divided into k intervals, and each data block in the i-th interval is XORed byte by byte to generate the i-th XOR checksum, resulting in K corresponding XOR checksums, where K is dynamically adjusted according to their importance.

[0024] The step of writing the compressed data and the XOR checksum corresponding to each stripe to different physical units includes: for each stripe, storing the compressed data in a first group of physical units, and storing the K XOR checksums in K groups of physical units different from the first group of physical units.

[0025] In another aspect, this application provides a data processing apparatus, which includes an acquisition unit, a compression unit, a generation unit, and a writing unit:

[0026] The acquisition unit is used to acquire factory data, which refers to data that needs to be stored using TLC NAND technology.

[0027] The compression unit is used to compress the factory data to obtain compressed data, and divide the compressed data into multiple strips;

[0028] The generation unit is used to generate XOR checksums for each of the multiple stripes based on the compressed data.

[0029] The writing unit is used to write the compressed data corresponding to each stripe and the XOR checksum to different physical units.

[0030] In one possible implementation, the writing unit is used for:

[0031] The compressed data and XOR checksum corresponding to each stripe are written into different channels respectively;

[0032] And / or,

[0033] The compressed data and XOR checksum corresponding to each stripe are written into different planes within the same channel.

[0034] In one possible implementation, the compression unit is used for:

[0035] Different compression methods are preset for different data types;

[0036] Based on the different data types in the factory data, and using the corresponding compression method, compressed data is obtained.

[0037] In one possible implementation, the data types of the compressed data include metadata, hot data, warm data, and cold data. The hot data refers to data whose access frequency is greater than a first preset threshold. The warm data refers to data whose access frequency is between the first preset threshold and a second preset threshold. The cold data refers to data whose access frequency is less than the second preset threshold. The first preset threshold is greater than the second preset threshold.

[0038] The compression unit is used for:

[0039] The metadata is compressed using a fast lossless compression method, the hot data is compressed using a lightweight compression algorithm, the warm data is compressed using a balanced compression algorithm, and the cold data is compressed using a high ratio compression algorithm to obtain compressed data.

[0040] In one possible implementation, the generation unit is configured to: sequentially divide multiple data blocks within each of the multiple stripes into k intervals, and perform an XOR operation byte-by-byte on the data blocks in the i-th interval to generate the i-th XOR checksum, thereby obtaining the corresponding K XOR checksums, wherein K is dynamically adjusted according to the importance.

[0041] The writing unit is configured to: store the compressed data into a first group of physical units for each stripe, and store the K XOR checksums into K groups of physical units that are different from the first group of physical units.

[0042] In another aspect, this application provides a computer device, which includes a processor and a memory:

[0043] The memory is used to store computer programs;

[0044] The processor is configured to execute the method described in any of the above-described embodiments according to the computer program.

[0045] In another aspect, this application provides a computer-readable storage medium for storing a computer program that, when executed by a computer device, implements the method described in any of the above-mentioned embodiments.

[0046] In another aspect, this application provides a computer program product including a computer program, which, when run on a computer device, causes the computer device to perform any of the methods described above.

[0047] As can be seen from the above technical solution, this solution first obtains the factory data, which refers to the data to be stored using TLC NAND technology. Then, the factory data is compressed to obtain compressed data, which is then divided into multiple stripes. Next, a corresponding XOR checksum is generated for each stripe. Finally, the compressed data and XOR checksum corresponding to each stripe are written to different physical units. Thus, by compressing the factory data before writing it to disk, the amount of data written can be reduced, which helps alleviate wear and tear on embedded storage devices such as UFS and eMMC based on TLC NAND technology, improving their lifespan. Furthermore, by utilizing the XOR checksum and distributing it and the compressed data across different physical units, data recovery and reconstruction can be performed when some data within a stripe is lost, using the XOR checksum and surviving data. This meets the data integrity requirements of embedded storage devices in the event of a sudden power outage. Compared to related technologies, this solution further improves the persistence and reliability of data storage. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a data storage method provided in an embodiment of this application;

[0050] Figure 2 A schematic diagram of a dual-layer RAID protection mechanism provided in an embodiment of this application;

[0051] Figure 3 A flowchart illustrating a differentiated compression strategy provided in an embodiment of this application;

[0052] Figure 4 An application architecture diagram of a data storage method provided in this application embodiment;

[0053] Figure 5 A flowchart illustrating another data storage method provided in an embodiment of this application;

[0054] Figure 6 A flowchart illustrating data reading and recovery is provided as an embodiment of this application;

[0055] Figure 7 This is a schematic diagram of a data storage device provided in an embodiment of this application. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0057] As described in the background section, traditional embedded storage devices mainly employ the following technical solutions:

[0058] (1) Single data storage: Factory data is usually stored in NAND flash memory in the form of a single copy, relying on ECC error correction code to ensure basic data integrity.

[0059] (2) Uniform wear management: Block-level wear leveling is achieved through flash conversion layer to avoid overwriting of specific storage cells.

[0060] (3) Regular refresh: Some high-end devices will periodically read and rewrite data that has not been accessed for a long time to alleviate the problem of declining data retention capacity.

[0061] The related technology adopts an automotive-grade design compliant with the AEC-Q100 standard, supports wide temperature operation (-40℃ to 105℃), and integrates basic data protection mechanisms such as ECC error correction and wear leveling, which can further improve the security of data storage. However, the following major drawbacks still exist:

[0062] (1) Limited data retention capability: Each storage cell of TLC NAND stores 3 bits of data. The charge level difference is small. After long-term storage, charge leakage leads to a significant increase in the data error rate.

[0063] (2) Insufficient redundancy mechanism: Existing RAID solutions are mainly designed for multi-channel architecture of enterprise-level SSDs and are not suitable for embedded storage devices with limited channel resources.

[0064] (3) Low efficiency of refresh mechanism: Regular full disk refresh will greatly increase flash memory wear, shorten device life, and cannot effectively identify high-risk data that really needs to be refreshed.

[0065] (4) Poor adaptability to extreme environments: In harsh environments such as high temperature and high humidity, the data retention time is greatly shortened, and traditional error correction mechanisms are unable to cope with the rapidly increasing error rate.

[0066] These defects lead to data corruption after the equipment has been idle for a long time, which in turn causes serious problems such as system failure to start and loss of factory data.

[0067] This application provides a data storage method that compresses factory data before writing it to disk, reducing the amount of data written. This helps alleviate wear and tear on embedded devices such as UFS and EMMC based on TLC NAND technology, extending their lifespan. Furthermore, by utilizing XOR checksums and distributing them and the compressed data across different physical units, data recovery and reconstruction can be performed when some data within a stripe is lost, using the XOR checksums and surviving data. This meets the data integrity requirements of embedded storage devices in the event of a sudden power outage. Compared to related technologies, this method further improves the persistent storage capability of embedded devices and enhances the reliability of data storage.

[0068] The solutions provided in this application relate to the field of data storage technology, and are specifically illustrated through the following embodiments.

[0069] See Figure 1 The diagram shown is a flowchart illustrating a data storage method provided in an embodiment of this application, including steps S101-S104. The following description uses embedded devices such as UFS and EMMC based on TLC NAND technology as examples.

[0070] S101: Obtain factory data.

[0071] Factory data refers to data that needs to be stored using TLC NAND technology. In other words, this factory data needs to be stored in TLC NAND flash memory.

[0072] S102: Compress the factory data to obtain compressed data, and divide the compressed data into multiple stripes.

[0073] Compressing data can effectively reduce the amount of data, thereby alleviating wear and tear on embedded devices and extending their lifespan when stored there, which in turn helps improve the persistent storage capability of the data.

[0074] Each strip includes multiple data blocks. In this embodiment, the factory data is divided into multiple strips, which means that continuous factory data is horizontally cut into several data blocks (usually one page or several pages) according to a fixed length.

[0075] For example, a compressed data stream can be divided into multiple stripes with a fixed step size S = D × PageSize (D is usually 4, 8, or 16), where each strip contains D data blocks of equal length. , ,..., (4 kB per page).

[0076] S103: Based on the compressed data, generate the XOR checksum corresponding to each stripe in the multiple stripes.

[0077] Perform bitwise XOR operations in parallel within each stripe: = … The result P is the same length as the data block (4 kB), which is the XOR checksum corresponding to this stripe. If the redundancy depth K=3, then the calculation is repeated for even / odd sequence blocks separately. , A total of K independent check codes were obtained.

[0078] S104: Write the compressed data and XOR checksum corresponding to each stripe into different physical units.

[0079] For each stripe, the generated checksum and D-block data blocks are split and sent to different physical pages of Channel / lane / Plane / Die to complete the distributed writing.

[0080] Therefore, compressing factory data before writing it to disk reduces the amount of data written, which helps alleviate wear and tear on embedded devices such as UFS and eMMC based on TLC NAND technology, and improves their lifespan. Furthermore, by using XOR checksums and distributing them and the compressed data across different physical units, data recovery and reconstruction can be performed when some data within a stripe is lost, using XOR checksums and surviving data. This meets the data integrity requirements of embedded storage devices in the event of a sudden power outage. Compared to related technologies, this further improves the persistent storage capability of embedded devices and enhances the reliability of data storage.

[0081] In one possible implementation, S104 includes:

[0082] The compressed data and XOR checksum corresponding to each stripe are written into different channels respectively;

[0083] And / or,

[0084] The compressed data and XOR checksum corresponding to each stripe are written into different planes within the same channel.

[0085] refer to Figure 2 The diagram illustrates a dual-layer RAID protection mechanism provided in this application embodiment. A RAID redundancy mechanism is established between different Plans on the same channel, and a RAID redundancy relationship is established between different channels. Compressed data is split into four data blocks, two in channel 1 and two in channel 2. The two data blocks in channel 1 and channel 2 are then assigned to different Plans. An XOR checksum is generated based on the data blocks in channel 1 and stored in channel 2. If a portion of the data blocks in channel 1, such as Die1, fails, the XOR checksum value from channel 1 can be obtained from channel 2. Then, based on the XOR checksum value of Plan1 stored in Plan 2 in channel 1, as well as Die2, Die3, and Die4, Die1 is reconstructed.

[0086] Therefore, dual-layer RAID protection is introduced into embedded devices through the above method. The RAID relationship between different Planes within the same channel provides fault tolerance for the failure of a single Plan, and this method has low overhead and limited impact on performance. The RAID relationship between different channels provides protection for the failure of the entire channel, effectively enhancing data reliability and access parallelism, thereby providing hardware-level fault tolerance.

[0087] In one possible implementation, the factory data is compressed in step S102 to obtain compressed data including:

[0088] A1: Different compression methods are preset for different data types;

[0089] A2: Based on the different data types in the factory data, and using the corresponding compression methods, compressed data is obtained.

[0090] For example, executable files are marked as the first data type and compressed using a lossless compression method by default; fonts and image resources are marked as the second data type and compressed using a standard compression algorithm by default; logs are marked as the third data type and compressed using a high-ratio compression algorithm by default. Then, for each data type, the factory data is compressed using the corresponding compression method to obtain compressed data.

[0091] Therefore, based on the preset compression method corresponding to the data type, the appropriate compression method can be adaptively selected to perform differentiated compression on the factory data, thereby maximizing storage space utilization and extending the life of embedded devices while ensuring system performance.

[0092] In one possible implementation, the data types of compressed data include metadata, hot data, warm data, and cold data;

[0093] S102 includes:

[0094] The metadata is compressed using a fast and lossless compression method, the hot data is compressed using a lightweight compression algorithm, the warm data is compressed using a balanced compression algorithm, and the cold data is compressed using a high-ratio compression algorithm, resulting in compressed data.

[0095] Hot data refers to data whose access frequency is greater than the first preset threshold, warm data refers to data whose access frequency is between the first preset threshold and the second preset threshold, and cold data refers to data whose access frequency is less than the second preset threshold, wherein the first preset threshold is greater than the second preset threshold.

[0096] refer to Figure 3 The flowchart shown is a flowchart of a differentiated compression strategy provided in an embodiment of this application. First, the data type and access pattern (access frequency) are analyzed. For metadata, a fast lossless compression method is used for compression, and 3+ backup copies of the metadata are stored in physical space. For frequently read and written hot data, a lightweight compression algorithm is used to maintain high access speed. For warm data with medium access frequency, a balanced compression algorithm is used to optimize the balance between space and performance. For cold data with infrequent access, a high-ratio compression is used to maximize space saving. Then, compression operations are performed on different data types of the factory data, and the compression effect is evaluated. If the compression ratio reaches a preset standard, the data is stored; otherwise, it is recompressed and then stored.

[0097] Therefore, using fast and lossless compression methods such as LZ4 for metadata can ensure fast access to critical system data; using lightweight compression algorithms for hot data can balance performance and space efficiency; and using high-ratio compression algorithms (such as ZSTD) for cold data can minimize storage space usage.

[0098] In one possible implementation, S103 includes:

[0099] For multiple data blocks within each stripe in multiple stripes, divide them into k intervals in sequence, and perform an XOR operation on each byte of the data block in the i-th interval to generate the i-th XOR check code, thus obtaining the corresponding K XOR check codes.

[0100] S104 includes:

[0101] For each stripe, the compressed data is stored in the first group of physical units, and the K XOR check codes are stored in K groups of physical units that are different from the first group of physical units.

[0102] Here, K represents the verification depth, which can be dynamically adjusted according to the importance of the data. For example, a larger value can be set for data types that are of higher importance.

[0103] As an example, K=3, the number of data blocks D within a stripe is 6, and the data block numbers are as follows: , ,..., Then, it is divided into 3 intervals in a cyclical manner, the first interval including , The second interval includes , The third interval includes , Then, a cyclic XOR operation is performed to generate three XOR checksums. , , ,in Then Write to CH1-Plane0, Write to CH2-Plane1, Write to CH3-Plane0.

[0104] Therefore, by dynamically adjusting the verification depth according to the importance of the data, more XOR copies can be generated for important data to recover more failed data, which can further improve the security and reliability of the data.

[0105] To more clearly describe this data storage method, the following explanation will be provided in conjunction with a specific implementation scenario. (Reference) Figure 4The diagram shown is an application architecture diagram of a data storage method provided in an embodiment of this application. First, data is stored in a storage device: the factory data is compressed in layers by a data preprocessing module, and then the compressed data is redundantly encoded by a redundancy coding engine. A cross-plan and channel RAID redundancy mechanism is implemented by generating XOR checksums, and the redundancy level can be adaptive. For example, higher-level RAID, such as RAID5 or RAID6 technology, is used for data of high importance. Then, the encoded data blocks are stored through distributed storage management, physical address mapping is generated, and the data blocks and corresponding XOR checksums are stored in different physical units in the NAND flash array.

[0106] Then, when the host system's overall file system issues an I / O request, it can read the data stream from the NAND flash array and perform data recovery and reconstruction through the storage controller.

[0107] refer to Figure 5 The diagram shows a flowchart of another data storage method provided in this application embodiment. First, a write request for factory data is received. Then, the data is divided into blocks of fixed size. Next, XOR check information is generated for each data block. Then, the data and XOR check information are distributed and stored in different physical locations. Then, the metadata mapping table (i.e., physical address mapping) is updated. Finally, the data recoverability is verified.

[0108] See Figure 6 The diagram shows a flowchart of a data reading and recovery process provided in an embodiment of this application. First, the host system attempts to read the target data block. If the data block is completely readable, the system returns the data; otherwise, it verifies whether there is available XOR check information.

[0109] If the XOR checksum is present, locate the data and reconstruct it. Otherwise, determine if there is RAID redundant data. If so, use the RAID redundancy to recover the data. Otherwise, try an advanced error correction algorithm to recover the data. If the recovery is successful, return the data; otherwise, return a data read failure message.

[0110] Therefore, the technical solution provided by the present invention has the following significant beneficial effects:

[0111] (1) Data retention period is greatly extended: Through multi-level protection mechanism, the data retention time of TLC NAND is extended from the typical 1 year to more than 5 years, meeting the needs of long-term idle equipment.

[0112] (2) Significantly improved startup reliability: The critical data of the system is protected to ensure that the equipment can still start normally after long-term storage.

[0113] (3) Cost-effectiveness optimization: Achieve maximum reliability improvement with limited hardware resources and avoid cost increase caused by over-design.

[0114] (4) Effective power consumption control: Intelligent compression reduces the actual amount of writing, lowers power consumption, and extends device life.

[0115] Based on the above embodiments, this application provides a data storage device, with reference to... Figure 7 The diagram shown is a schematic of a data storage device provided in an embodiment of this application. The device 300 includes an acquisition unit 301, a compression unit 302, a generation unit 303, and a writing unit 304.

[0116] The acquisition unit is used to acquire factory data, which refers to data that needs to be stored using TLC NAND technology.

[0117] The compression unit is used to compress the factory data to obtain compressed data, and divide the compressed data into multiple strips;

[0118] The generation unit is used to generate XOR checksums for each of the multiple stripes based on the compressed data.

[0119] The writing unit is used to write the compressed data corresponding to each stripe and the XOR checksum to different physical units.

[0120] Therefore, compressing factory data before writing it to disk reduces the amount of data written, which helps alleviate wear and tear on embedded devices such as UFS and eMMC based on TLC NAND technology, and improves their lifespan. Furthermore, by using XOR checksums and distributing them and the compressed data across different physical units, data recovery and reconstruction can be performed when some data within a stripe is lost, using XOR checksums and surviving data. This meets the data integrity requirements of embedded storage devices in the event of a sudden power outage. Compared to related technologies, this further improves the persistent storage capability of embedded devices and enhances the reliability of data storage.

[0121] In one possible implementation, the writing unit is used for:

[0122] The compressed data and XOR checksum corresponding to each stripe are written into different channels respectively;

[0123] And / or,

[0124] The compressed data and XOR checksum corresponding to each stripe are written into different planes within the same channel.

[0125] Therefore, dual-layer RAID protection is introduced into embedded devices through the above method. The RAID relationship between different Planes within the same channel provides fault tolerance for the failure of a single Plan, and this method has low overhead and limited impact on performance. The RAID relationship between different channels provides protection for the failure of the entire channel, effectively enhancing data reliability and access parallelism, thereby providing hardware-level fault tolerance.

[0126] In one possible implementation, the compression unit is used for:

[0127] Different compression methods are preset for different data types;

[0128] Based on the different data types in the factory data, and using the corresponding compression method, compressed data is obtained.

[0129] Therefore, based on the preset compression method corresponding to the data type, the appropriate compression method can be adaptively selected to perform differentiated compression on the factory data, thereby maximizing storage space utilization and extending the life of embedded devices while ensuring system performance.

[0130] In one possible implementation, the data types of the compressed data include metadata, hot data, warm data, and cold data. The hot data refers to data whose access frequency is greater than a first preset threshold. The warm data refers to data whose access frequency is between the first preset threshold and a second preset threshold. The cold data refers to data whose access frequency is less than the second preset threshold. The first preset threshold is greater than the second preset threshold.

[0131] The compression unit is used for:

[0132] The metadata is compressed using a fast lossless compression method, the hot data is compressed using a lightweight compression algorithm, the warm data is compressed using a balanced compression algorithm, and the cold data is compressed using a high ratio compression algorithm to obtain compressed data.

[0133] Therefore, using fast and lossless compression methods such as LZ4 for metadata can ensure fast access to critical system data; using lightweight compression algorithms for hot data can balance performance and space efficiency; and using high-ratio compression algorithms (such as ZSTD) for cold data can minimize storage space usage.

[0134] In one possible implementation, the generation unit is configured to: sequentially divide multiple data blocks within each of the multiple stripes into k intervals, and perform an XOR operation byte-by-byte on the data blocks in the i-th interval to generate the i-th XOR checksum, thereby obtaining the corresponding K XOR checksums, wherein K is dynamically adjusted according to the importance.

[0135] The writing unit is configured to: store the compressed data into a first group of physical units for each stripe, and store the K XOR checksums into K groups of physical units that are different from the first group of physical units.

[0136] Therefore, by dynamically adjusting the verification depth according to the importance of the data, more XOR copies can be generated for important data to recover from the failure of more random pages, which can further improve the security and reliability of the data.

[0137] Based on the above embodiments, this application provides a computer device, which includes a processor and a memory:

[0138] The memory is used to store computer programs;

[0139] The processor is used to execute the above data storage method according to the computer program.

[0140] Based on the above embodiments, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a computer device, implements the above-described data storage method.

[0141] Based on the above embodiments, this application provides a computer program product including a computer program, which, when run on a computer device, causes the computer device to execute the above data storage method.

[0142] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0143] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data storage method, characterized in that, The method includes: Obtain factory data, which refers to data that needs to be stored using TLC NAND technology; The factory data is compressed to obtain compressed data, and the compressed data is divided into multiple stripes. Based on the compressed data, an XOR checksum corresponding to each of the multiple stripes is generated; The compressed data and the XOR checksum corresponding to each stripe are written into different physical units.

2. The method according to claim 1, characterized in that, The step of writing the compressed data corresponding to each stripe and the XOR checksum to different physical units includes: The compressed data and XOR checksum corresponding to each stripe are written into different channels respectively; And / or, The compressed data and XOR checksum corresponding to each stripe are written into different planes within the same channel.

3. The method according to claim 1, characterized in that, The process of compressing the factory data to obtain compressed data includes: Different compression methods are preset for different data types; Based on the different data types in the factory data, and using the corresponding compression method, compressed data is obtained.

4. The method according to any one of claims 1-3, characterized in that, The data types of the compressed data include metadata, hot data, warm data, and cold data. Hot data refers to data whose access frequency is greater than a first preset threshold. Warm data refers to data whose access frequency is between the first preset threshold and a second preset threshold. Cold data refers to data whose access frequency is less than the second preset threshold. The first preset threshold is greater than the second preset threshold. The compressed data obtained by compressing the factory data includes: The metadata is compressed using a fast lossless compression method, the hot data is compressed using a lightweight compression algorithm, the warm data is compressed using a balanced compression algorithm, and the cold data is compressed using a high ratio compression algorithm to obtain compressed data.

5. The method according to claim 1, characterized in that, The step of generating the XOR checksum corresponding to each of the multiple stripes based on the compressed data includes: For multiple data blocks within each of the multiple stripes, they are sequentially divided into k intervals, and each data block in the i-th interval is XORed byte by byte to generate the i-th XOR checksum, resulting in K corresponding XOR checksums, where K is dynamically adjusted according to their importance. The step of writing the compressed data and the XOR checksum corresponding to each stripe to different physical units includes: for each stripe, storing the compressed data in a first group of physical units, and storing the K XOR checksums in K groups of physical units different from the first group of physical units.

6. A data processing apparatus, characterized in that, The device includes an acquisition unit, a compression unit, a generation unit, and a writing unit: The acquisition unit is used to acquire factory data, which refers to data that needs to be stored using TLC NAND technology. The compression unit is used to compress the factory data to obtain compressed data, and divide the compressed data into multiple strips; The generation unit is used to generate XOR checksums for each of the multiple stripes based on the compressed data. The writing unit is used to write the compressed data corresponding to each stripe and the XOR checksum to different physical units.

7. The apparatus according to claim 6, characterized in that, The writing unit is used for: The compressed data and XOR checksum corresponding to each stripe are written into different channels respectively; And / or, The compressed data and XOR checksum corresponding to each stripe are written into different planes within the same channel.

8. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store computer programs; The processor is configured to perform the method according to any one of claims 1-5 according to the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when executed by a computer device, performs the method described in any one of claims 1-5.

10. A computer program product comprising a computer program, characterized in that, When it is run on a computer device, it causes the computer device to perform the method described in any one of claims 1-5.

Images