Data protection method, storage array controller, and storage device

Abstract

The embodiment of the application relates to the technical field of storage arrays, and discloses a data protection method, a storage array controller and a storage device, the method compresses at least two data blocks corresponding to user data and a check block respectively, adds protection data to the compressed data blocks and the check block respectively, and stores the data blocks and the check block after adding the protection data into corresponding memories, so that the data protection strength of the storage array can be increased, and the data security of the storage array is improved.

Description

Technical Field

[0001] This application relates to the field of storage array technology, and in particular to a data protection method, a storage array controller, and a storage device. Background Technology

[0002] Redundant Arrays of Independent Disks (RAID) storage technology uses multiple storage devices grouped together to form a storage array, enabling multiple storage devices to work simultaneously to improve data storage performance and provide data redundancy protection.

[0003] Currently, common RAID storage technology involves calculating parity data through the RAID controller, dividing the data into multiple data blocks, and writing these blocks along with the parity data into the storage array's memory. When one data block becomes corrupted, the corrupted block is recovered by reading the parity data and calculating against other undamaged blocks. However, this approach has a problem: when multiple data blocks and the parity data become corrupted simultaneously, it increases the complexity of data recovery and reduces the strength of data protection. Summary of the Invention

[0004] This application provides a data protection method, a storage array controller, and a storage device, which solves the technical problem of low data protection strength, increases data protection strength, and thus increases data reliability.

[0005] To address the aforementioned technical problems, this application provides the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a data protection method applied to a storage array, the storage array including at least two memories, the method comprising:

[0007] Acquire user data, wherein the user data is divided into at least two first data blocks;

[0008] Calculate the check data corresponding to at least two first data blocks to generate the first check block;

[0009] The first data block is compressed to obtain the second data block corresponding to the first data block, and the first check block is compressed to obtain the second check block corresponding to the first check block.

[0010] Add protection data to the second data block to generate the third data block; and add protection data to the second check block to generate the third check block.

[0011] Store the third data block to the corresponding memory, and store the third check block to the corresponding memory.

[0012] In some embodiments, the method further includes:

[0013] Get the data length of the second data block, and get the data length of the second check block;

[0014] Add protection data to the second data block to generate the third data block, including:

[0015] The data protection type corresponding to the second data block is determined based on the data length of the second data block;

[0016] Based on the data protection type corresponding to the second data block, protection data is added to the second data block to generate the third data block;

[0017] Add protective data to the second check block to generate the third check block, which includes:

[0018] The data protection type corresponding to the second check block is determined based on the data length of the second check block;

[0019] Based on the data protection type corresponding to the second check block, add protection data to the second check block to generate the third check block.

[0020] In some embodiments, the data protection type includes a first data protection type and a second data protection type;

[0021] Based on the data length of the second data block, determine the corresponding data protection type for the second data block, including:

[0022] If the data length of the second data block is greater than the preset length threshold, then the data protection type corresponding to the second data block is determined to be the first data protection type.

[0023] If the data length of the second data block is less than or equal to the preset length threshold, then the data protection type corresponding to the second data block is determined to be the second data protection type.

[0024] Based on the data length of the second check block, determine the data protection type corresponding to the second check block, including:

[0025] If the data length of the second verification block is greater than the preset length threshold, then the data protection type corresponding to the second verification block is determined to be the first data protection type.

[0026] If the data length of the second verification block is less than or equal to the preset length threshold, then the data protection type corresponding to the second verification block is determined to be the second data protection type.

[0027] In some embodiments, the data length of the first protected data corresponding to the first data protection type is less than the data length of the second protected data corresponding to the second data protection type;

[0028] Based on the data protection type corresponding to the second data block, add protection data to the second data block to generate the third data block, including:

[0029] Based on the first data protection type or the second data protection type, add the first protection data or the second protection data to the second data block to generate the third data block;

[0030] Based on the data protection type corresponding to the second check block, protection data is added to the second check block to generate the third check block, including:

[0031] Based on the first data protection type or the second data protection type, add the first protection data or the second protection data to the second verification block to generate the third verification block.

[0032] In some embodiments,

[0033] The first data protection type corresponds to the first preset algorithm, and the method also includes:

[0034] The first protected data is calculated based on the first preset algorithm and the data length of the second data block;

[0035] The second data protection type corresponds to the second preset algorithm, and the method also includes:

[0036] The second protection data is calculated based on the second preset algorithm and the data length of the second verification block.

[0037] In some embodiments, the method further includes:

[0038] Read user data from memory;

[0039] Based on the data protection type corresponding to the user data, the user data is decoded to obtain the decoded user data;

[0040] If the decoded user data contains errors, determine whether the data protection type corresponding to the user data is the second data protection type.

[0041] If so, the decoded user data is corrected to obtain the corrected user data, and the corrected user data is forwarded to the host.

[0042] If not, the decoded user data will be forwarded to the host.

[0043] In some embodiments, after correcting errors in the decoded user data, the method further includes:

[0044] Determine whether the error correction was successful;

[0045] If the error correction is successful, the corrected user data is forwarded to the host and the corrected user data is written back to the memory.

[0046] If error correction fails, the system reads other data blocks from the stripe corresponding to the user data from the storage array to perform RAID calculations in order to recover the user data.

[0047] Secondly, embodiments of this application provide a storage array controller, which includes: an algorithm engine, a compression engine, and a data protection engine;

[0048] The algorithm engine, connected to the compression engine, is used to obtain at least two first data blocks corresponding to user data, calculate the verification data corresponding to at least two first data blocks to generate a first verification block, and send the at least two first data blocks and the first verification block to the compression engine.

[0049] The compression engine connects the algorithm engine and the data protection engine. It is used to compress the first data block to obtain the second data block corresponding to the first data block, and to compress the first verification block to obtain the second verification block corresponding to the first verification block. The second data block and the second verification block are then sent to the data protection engine.

[0050] The data protection engine, connected to the compression engine, is used to add protection data to the second data block and generate the third data block, and to add protection data to the second check block and generate the third check block.

[0051] In some embodiments, the data protection engine includes a data preprocessing engine;

[0052] A data preprocessing engine is used to obtain the data length of the second data block and the data length of the second check block. Based on the data length of the second data block, the engine corresponding to the second data block is determined, and based on the data length of the second check block, the engine corresponding to the second check block is determined.

[0053] In some embodiments, the data protection engine includes a first protection engine and a second protection engine, wherein the first protection engine corresponds to a first preset algorithm and the second protection engine corresponds to a second preset algorithm.

[0054] The data preprocessing engine is also used to determine the protection engine corresponding to the second data block as the first protection engine when the data length of the second data block is greater than the preset length threshold, and to determine the protection engine corresponding to the second data block as the second protection engine when the data length of the second data block is less than or equal to the preset length threshold.

[0055] The data preprocessing engine is also used to determine the protection engine corresponding to the second verification block as the first protection engine when the data length of the second verification block is greater than the preset length threshold, and to determine the protection engine corresponding to the second verification block as the second protection engine when the data length of the second verification block is less than or equal to the preset length threshold.

[0056] The data protection engine is also used to add first protection data or second protection data to the second data block according to the first protection engine or the second protection engine to generate a third data block, and to add first protection data or second protection data to the second verification block according to the first protection engine or the second protection engine to generate a third verification block.

[0057] In some embodiments, the algorithm engine includes a RAID algorithm engine;

[0058] The first protection engine includes the DIF / DIX engine;

[0059] The second protection engine includes the DPECC engine.

[0060] In some embodiments,

[0061] The data protection engine is also used to read user data from the memory, decode the user data according to the data protection type corresponding to the user data, and obtain the decoded user data. If the decoded user data contains errors and the data protection type corresponding to the user data is the second data protection type, the data protection engine corrects the errors in the decoded user data to obtain the corrected user data, and forwards the corrected user data to the compression engine.

[0062] The compression engine is also used to decompress the error-corrected user data and forward the decompressed user data to the algorithm engine, so that the algorithm engine can forward the decompressed user data to the host interface.

[0063] In some embodiments,

[0064] The algorithm engine is also used to perform RAID calculations to recover user data when the data protection type corresponding to the user data is the first data protection type;

[0065] The algorithm engine is also used to perform RAID calculations to recover user data after the data protection engine fails to correct errors in the decoded user data by reading other data blocks of the stripe corresponding to the user data from the storage array.

[0066] Thirdly, embodiments of this application provide a storage device, including:

[0067] Such as the storage array controller in the second aspect, wherein the storage array controller includes a host interface and a storage device interface, the host interface being used to connect to a host;

[0068] The storage array includes at least two memory devices, each of which is communicatively connected to the storage array controller via a storage device interface.

[0069] The beneficial effects of the embodiments of this application are as follows: Unlike the prior art, the data protection method provided in this application embodiment is applied to a storage array. The method includes: acquiring user data, wherein the user data is divided into at least two first data blocks; calculating verification data corresponding to at least two first data blocks to generate a first verification block; compressing the first data blocks to obtain second data blocks corresponding to the first data blocks, and compressing the first verification blocks to obtain second verification blocks corresponding to the first verification blocks; adding protection data to the second data blocks to generate a third data block, and adding protection data to the second verification blocks to generate a third verification block; storing the third data block in a corresponding memory, and storing the third verification block in a corresponding memory.

[0070] By compressing at least two data blocks and a check block corresponding to user data, and adding protection data to the compressed data blocks and check blocks respectively, and storing the data blocks and check blocks with added protection data to the corresponding memory, this application can increase the protection of data in the storage array and improve the security of data in the storage array. Attached Figure Description

[0071] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0072] Figure 1 This is an example schematic diagram of a striped storage provided in an embodiment of this application;

[0073] Figure 2 This is a schematic diagram illustrating the connection relationship between a storage device and a host according to an embodiment of this application;

[0074] Figure 3 This is a schematic diagram of the structure of a storage device provided in an embodiment of this application;

[0075] Figure 4A This is a schematic diagram of the structure of a storage array controller provided in an embodiment of this application;

[0076] Figure 4B This is a schematic diagram of another storage array controller provided in an embodiment of this application;

[0077] Figure 5 This is a flowchart illustrating a data protection method provided in an embodiment of this application;

[0078] Figure 6 This is an example schematic diagram of obtaining a second data block by compressing a first data block, as provided in an embodiment of this application;

[0079] Figure 7 yes Figure 5 A detailed flowchart of step S504;

[0080] Figure 8 yes Figure 7 Detailed flowchart of step S5041;

[0081] Figure 9 yes Figure 7 Detailed flowchart of step S5042;

[0082] Figure 10 yes Figure 5 A detailed flowchart of step S504;

[0083] Figure 11 yes Figure 10 Detailed flowchart of step S5043;

[0084] Figure 12 yes Figure 10 Detailed flowchart of step S5044;

[0085] Figure 13 This is a schematic diagram of a process for calculating first protection data provided in an embodiment of this application;

[0086] Figure 14 This is a schematic diagram of a process for calculating second protection data provided in an embodiment of this application;

[0087] Figure 15 This is a schematic diagram illustrating an example of data block and check block compression and decompression provided in an embodiment of this application.

[0088] Figure 16 This is an example schematic diagram of storing the third data block and the third parity block in a storage array according to an embodiment of this application;

[0089] Figure 17 This is a schematic diagram of the structure of a storage array controller provided in an embodiment of this application;

[0090] Figure 18 This is a schematic diagram of the structure of a data protection engine provided in an embodiment of this application;

[0091] Figure 19This is a schematic diagram of the overall structure of a storage device provided in an embodiment of this application;

[0092] Figure 20 This is a schematic diagram illustrating reading user data from a memory according to an embodiment of this application;

[0093] Figure 21 This is a schematic diagram illustrating another method for reading user data from a memory according to an embodiment of this application;

[0094] Figure 22 This is a schematic diagram illustrating another method of reading user data from a memory provided in an embodiment of this application;

[0095] Figure 23 This is a schematic diagram of a process for reading user data from a memory according to an embodiment of this application;

[0096] Figure 24 This is a schematic diagram of another process for reading user data from a memory, provided in an embodiment of this application.

[0097] Explanation of reference numerals in the attached figures:

[0098] 110 storage array 1233 Second protection engine 120 Storage array controller 122 NVMe controller 121 Data Path 123 Storage device interface 1201 Algorithm Engine 200 host 1202 compression engine 201 Processor system 1203 Data Protection Engine 202 Dynamic Random Access Memory 1231 Data preprocessing engine 203 Storage interface 124 Host Interface Detailed Implementation

[0099] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0100] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0101] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0102] Before introducing the embodiments of this application, a brief introduction to the RAID storage technology known to the inventors of this application will be given to facilitate the understanding of the embodiments of this application.

[0103] RAID is a commonly used storage technology that uses multiple storage devices grouped together to form a storage array, working simultaneously to improve performance or provide data redundancy protection.

[0104] Please see Figure 1 , Figure 1 This is an example schematic diagram of a striped storage provided in an embodiment of this application;

[0105] like Figure 1 As shown, A1-A3 form a stripe, Ap is the parity data of A1-A3, B1-B3 form a stripe, Bp is the parity data of B1-B3, C1-C3 form a stripe, Cp is the parity data of C1-C3, D1-D3 form a stripe, Dp is the parity data of D1-D3. The storage array includes multiple memories, and each data block and parity data in each stripe are stored on different memories.

[0106] RAID storage technology primarily achieves data security and improved storage performance through three operations: data striping, data verification, and mirroring. Specifically:

[0107] Data striping: This involves dividing continuous data into segments and storing them on different hard drives. In other words, data is distributed across different hard drives in the form of stripes, thereby improving system throughput. The size of each stripe depends on the number and type of hard drives in the storage array.

[0108] Data Parity: This function performs calculations on the data in the storage array to calculate the corresponding parity data. When reading data, the parity data is used to determine if the data is correct. If an error occurs on one of the hard drives, the data can be recovered using the parity data and data from other hard drives, thus improving data reliability.

[0109] Mirroring: Writing data to multiple hard drives simultaneously to achieve a complete backup of the data, which can improve data reliability and increase storage redundancy.

[0110] The technical solution of this application will be described in detail below with reference to the accompanying drawings:

[0111] Please see Figure 2 This is a schematic diagram illustrating the connection relationship between a storage device and a host according to an embodiment of this application;

[0112] like Figure 2As shown, the storage device 100 includes a storage array 110 and a storage array controller 120. The storage device 100 is connected to the host 200 via a wired interface to realize data interaction. The wired interface includes a PCIe bus interface.

[0113] Storage array 110 is communicatively connected to storage array controller 120. Storage array 110 stores host data and verification data sent by storage array controller 120. The verification data is used to detect and correct errors when host data errors occur. Storage array 110 includes multiple storage devices, including serial ATA (SATA) hard drives. Multiple SATA hard drives form a redundant array of independent disks (RAID). Storage array 110 can be constructed from RAID arrays. It is understood that the storage devices also include hard disk drives (HDDs), hard drives based on SCSI (Small Computer System Interface), and other storage devices used for data storage.

[0114] Storage array controller 120 is communicatively connected to storage array 110 and host 200. Storage array controller 120 processes data, adds protection measures to host data and stores it in storage array 110, provides redundancy protection for host data to improve data security in the storage array, and converts data in storage array 110 into a data format corresponding to host 200 for transmission to host 200. (See also...) Figure 3 , Figure 3 This is a schematic diagram of the structure of a storage device provided in an embodiment of this application;

[0115] like Figure 3 As shown, the storage device 100 includes a storage array 110 and a storage array controller 120.

[0116] Storage array 110 is communicatively connected to storage array controller 120. Storage array 110 is used to store data, and includes multiple storage devices such as magnetic tape and solid-state drives. Storage array controller 120 includes a host interface for connecting to a host computer.

[0117] The storage array controller 120, connected to the storage array 110, processes data to generate corresponding verification data. This verification data is used to detect data errors and for data recovery. The storage array controller includes a RAID controller, which can be a hardware device comprising chips such as those from Intel, AMD, or NVIDIA. The RAID controller is mounted on a slot in the host computer, such as a PCI or PCIe slot. Data is transmitted from the host computer to the RAID controller via a bus, such as the System PCIe Bus, to manage data storage and redundancy. The RAID controller also includes multiple storage device interfaces. When the storage devices are SATA hard drives, the RAID controller communicates with multiple SATA hard drives through the SATA interfaces to store data. Alternatively, the RAID controller can be a software program running on the host computer's operating system, communicating with the external storage array through specific hardware interfaces, such as USB or PCIe interfaces.

[0118] The host 200 includes a processor system 201, a dynamic random access memory 202, and a storage interface 203.

[0119] The processor system 201 is connected to the dynamic random access memory 202 and the storage interface 203. The processor system 201 is used for data inside the host 200, as well as for controlling and coordinating the operation between the components inside the host 200.

[0120] Dynamic Random Access Memory (DRAM) 202 communicates with processor system 201 and storage interface 203. DRAM 202 is used to store data of host 200, which can accelerate data access speed and improve the performance of host 200. DRAM 202 may include DDR memory, such as: Double Data Rate Synchronous Dynamic Random-Access Memory (DDR SDRAM), Generation IV Double Data Rate Synchronous Dynamic Random-Access Memory (DDR4 SDRAM), Low Power Double Data Rate Version 4 SDRAM (LPDDR4 SDRAM), Graphics Double Data Rate SDRAM (GDDR SDRAM), Low Power Double Data Rate SDRAM (LPDDR SDRAM), Rambus Dynamic Random Access Memory (RDRAM), etc. DDR memory includes DDR chips, and the DDR chips are provided with DDR space for caching host data.

[0121] Storage interface 203 communicates with storage array controller 120, processor system 201, and dynamic random access memory 202, and is used to realize data transmission between storage device 100 and host 200.

[0122] Please refer to the following: Figure 4A , Figure 4A This is a schematic diagram of the structure of a storage array controller provided in an embodiment of this application;

[0123] like Figure 4A As shown, the storage array controller 120 includes a data path 121, which includes an algorithm engine 1201. The algorithm engine 1201 is used to generate verification data for the data of the host 200. For example, when the algorithm engine 1201 receives data from the host 200, it generates verification data corresponding to the data, adds verification data to the data of the host 200, and saves it to the storage array 110 to improve the security of the data of the host 200. When the host 200 retrieves data from the storage array, it transmits the data in the storage array 110 to the algorithm engine 1201 to check whether the data retrieved from the storage array 110 has errors. If no errors have occurred, the data is sent to the host 200. If the data has errors, the data with errors is recovered.

[0124] Please refer to the following: Figure 4B , Figure 4B This is a schematic diagram of another storage array controller provided in an embodiment of this application;

[0125] In this embodiment of the application, since the data of the host 200 is stored in the storage array 110 in stripe form, the above-mentioned Figure 4A The current method only generates verification data for the data of host 200, without adding protection measures for each data block and verification data in the corresponding stripe of host 200 data. When a data block is corrupted, simply using the verification data to recover the data cannot effectively protect the data. It is necessary to add corresponding protection measures for each data block to reduce the risk of data corruption. At the same time, corresponding protection measures should be added to the verification data to avoid risks such as leakage of verification data.

[0126] like Figure 4B As shown, the Figure 4B based on Figure 4A Improvements have been made. The storage array controller 120 includes a data path 121, which includes an algorithm engine 1201, a compression engine 1202, and a data protection engine 1203.

[0127] Algorithm engine 1201, connected to compression engine 1202, is used to obtain user data divided into at least two first data blocks, calculate the verification data corresponding to at least two first data blocks to generate a first verification block, and send the at least two first data blocks and the first verification block to compression engine 1202.

[0128] Compression engine 1202, connected to algorithm engine 1201 and data protection engine 1203, is used to add protection data to each first data block and first check block, compress them to generate a second data block corresponding to each first data block, and generate a second check block corresponding to each first check block, and send the second data block and the second check block to data protection engine 1203.

[0129] The data protection engine 1203 and the connection algorithm engine 1201 are used to add protection data to the second data block and the second check block, generate the third data block and the third check block, and save the third data block and the third check block to the storage array 110.

[0130] In this embodiment of the application, by adding protection measures to user data and verification data through algorithm engine 1201, compression engine 1202 and data protection engine 1203, the protection of data in the storage array can be increased.

[0131] In the application embodiment, the algorithm engine includes a RAID algorithm engine. The RAID algorithm engine can be configured with different RAID levels (such as RAID0, RAID1, RAID5, etc.) as needed. When the RAID algorithm engine is configured with RAID0 or RAID1, it is not necessary to generate verification data for user data. Instead, the data blocks in the stripe corresponding to the user data are directly compressed by the compression engine 1202, and further protection measures are added to the data blocks by the data protection engine 1203.

[0132] It's important to note that RAID 0 is a method of distributing data across multiple storage devices; RAID 0 only includes striping operations. RAID 1 is a method of completely replicating data across multiple storage devices, including mirroring and striping operations. RAID 5 is a method of distributing both data and parity data across multiple storage devices. Understandably, RAID 0 and RAID 1 do not require generating parity data; therefore, when processing user data using RAID 0 or RAID 1, only compression and additional protection measures are needed.

[0133] Please see Figure 5 , Figure 5 This is a flowchart illustrating a data protection method provided in an embodiment of this application;

[0134] This data protection method is applied to a storage array, which includes at least two memories.

[0135] like Figure 5 As shown, the data protection method includes:

[0136] Step S501: Obtain user data, wherein the user data is divided into at least two first data blocks;

[0137] Specifically, the system receives user data sent by the host to obtain user data. This user data may also come from other systems or applications, such as the Central Processing Unit (CPU) system, or from Dynamic Random Access Memory (DRAM). The system retrieves user data cached in DRAM.

[0138] Specifically, the acquired user data is striped to divide the user data into at least two first data blocks, which together form a strip.

[0139] In this embodiment of the application, user data is divided into multiple data blocks, and multiple data blocks are saved on different storage devices, which can realize redundant backup of data and ensure the reliability and availability of data.

[0140] Step S502: Calculate the check data corresponding to at least two first data blocks to generate a first check block;

[0141] Specifically, a verification algorithm is used to calculate verification data for dividing user data into at least two first data blocks to generate a first verification block. The verification algorithm includes, but is not limited to, Exclusive OR (XOR), Cyclic Redundancy Check (CRC), Message-Digest Algorithm 5 (MD5), hash algorithms (including SM2, SM3, SM4, and SM9), and Secure Hash Algorithm (SHA). It should be noted that the aforementioned verification algorithms are existing technologies, and detailed explanations of the verification data calculation are not provided here; please refer to existing technologies for details.

[0142] In the embodiments of this application, the first verification block and at least two first data blocks are simultaneously stored in different memories.

[0143] In this embodiment, the first verification block is used to check whether the user data has an error. An error means that the first data block has been modified or lost. When the user data is divided into at least two data blocks, and one of the first data blocks has an error, the erroneous first data block can be recovered by calculation using other undamaged first data blocks and the first verification block.

[0144] Step S503: Compress the first data block to obtain the second data block corresponding to the first data block, and compress the first check block to obtain the second check block corresponding to the first check block.

[0145] Specifically, the first data block is compressed using a compression algorithm to obtain a second data block that corresponds one-to-one with the first data block. Similarly, the first checksum block is compressed using the same algorithm to obtain a second checksum block that corresponds one-to-one with the first checksum block. The second data block occupies less storage space than the first data block, and the second checksum block occupies less storage space than the first checksum block. It should be noted that the compression algorithm used is an existing compression algorithm, such as transparent compression algorithms like Zlib, Gzip, Bzip2, Deflate, Lz4, Lzo, and Snappy.

[0146] In this embodiment, the first data block and the first check block are compressed using a compression algorithm. The first data block and the first check block are compressed in units of granularity. Granularity is the smallest data unit. The smaller the granularity, the smaller the unit of granularity, and the smaller the space occupied by the unit of data. The type of granularity is the unit of granularity used by the compression algorithm. For example, if the type of granularity is bytes, the 4K first data block is compressed to 64B using a byte-based compression algorithm, that is, the 4K first data block is subdivided into 64B.

[0147] It should be noted that different compression algorithms correspond to different types of granularity, and the specific type of granularity is adjusted according to the specific algorithm.

[0148] In this embodiment of the application, by compressing the first data block and the first check block, the reserved space corresponding to the second data block and the reserved space corresponding to the second check block can be obtained. The reserved space corresponding to the second data block is the reserved space obtained by subtracting the storage space occupied by the second data block from the storage space occupied by the first data block. The reserved space corresponding to the second check block is the reserved space obtained by subtracting the storage space occupied by the second check block from the storage space occupied by the first check block. The reserved space is set to not be used to store user data. The reserved space is used to store data used to protect the second data block and the second check block, so as to further protect the data security of each second data block and the second check block and improve data security.

[0149] Please refer to 6 again. Figure 6 This is an example schematic diagram of obtaining a second data block by compressing a first data block, as provided in an embodiment of this application;

[0150] like Figure 6 As shown, after compressing the first data block, a second data block is obtained, and the reserved space corresponding to the second data block is obtained. The reserved space corresponding to the second data block is obtained by subtracting the storage space occupied by the second data block from the storage space occupied by the first data block.

[0151] Step S504: Add protection data to the second data block to generate the third data block, and add protection data to the second check block to generate the third check block;

[0152] In this embodiment of the application, before adding protection data to the second data block and before adding protection data to the second check block, it is necessary to obtain the data lengths of the second data block and the second check block.

[0153] In this embodiment of the application, while compressing the first data block and the first check block to obtain the second data block and the second check block, compression information (CI) is generated. The compression information includes the type of granularity and the number of granularities. The number of granularities is the number of units corresponding to the type of granularity. For example, if the first data block of size 4K is compressed to 64B, then the type of granularity is bytes and the number of granularities is 64. Furthermore, by multiplying the length corresponding to the type of granularity by the number of granularities, the data length of the second data block and the second check block can be obtained.

[0154] Specifically, the compression information corresponding to the second data block is obtained. The compression information corresponding to the second data block includes the type of granularity and the number of granularities corresponding to the second data block. The length corresponding to the type of granularity of the second data block is multiplied by the number of granularities corresponding to the second data block to obtain the data length corresponding to the second data block. Based on the data length corresponding to the second data block, protection data is added to the second data block to generate the third data block. The data length of the third data block is equal to the data length of the first data block.

[0155] Specifically, the compression information corresponding to the second verification block is obtained. The compression information corresponding to the second verification block includes the type of granularity and the number of granularities corresponding to the second verification block. The length corresponding to the type of granularity of the second verification block is multiplied by the number of granularities corresponding to the second verification block to obtain the data length corresponding to the second verification block. Based on the data length corresponding to the second verification block, protection data is added to the second verification block to generate the third verification block. The data length of the third verification block is equal to the data length of the first verification block.

[0156] In this embodiment of the application, the protection data is used to ensure the integrity and accuracy of the second data block and the second check block during transmission.

[0157] Please refer to the following: Figure 7 , Figure 7 yes Figure 5 A detailed flowchart of step S504;

[0158] like Figure 7 As shown, step S504 includes:

[0159] Step S5041: Determine the data protection type corresponding to the second data block based on the data length of the second data block;

[0160] In this embodiment, the data protection types include a first data protection type and a second data protection type. These types are used to add protection data to the second data block and the second check block, thereby providing protection measures to prevent tampering during transmission. The first data protection type corresponds to a first preset algorithm, and the second data protection type corresponds to a second preset algorithm. Both algorithms generate checksums, which are used to detect errors generated during data transmission. The length of the first protected data corresponding to the first data protection type is shorter than the length of the second protected data corresponding to the second data protection type; that is, the length of the checksum generated by the first preset algorithm is shorter than the length of the checksum generated by the second preset algorithm, meaning the checksum generated by the second preset algorithm requires more storage space.

[0161] Specifically, based on the data length of the second data block, the size of the remaining space after the first data block is compressed is determined, thereby determining the data protection type corresponding to the second data block.

[0162] Please refer to the following: Figure 8 , Figure 8 yes Figure 7 Detailed flowchart of step S5041;

[0163] like Figure 8 As shown, step S5041 includes:

[0164] Step S5411: Determine whether the data length of the second data block is greater than a preset length threshold;

[0165] Specifically, determine whether the data length of the second data block is greater than a preset length threshold. If the data length of the second data block is greater than the preset length threshold, proceed to step S5412. If the data length of the second data block is less than or equal to the preset length threshold, proceed to step S5412.

[0166] Step S5412: Determine that the data protection type corresponding to the second data block is the first data protection type;

[0167] Specifically, if the data length of the second data block is greater than the preset length threshold, that is, the reserved space corresponding to the second data block is small, meaning that the reserved space can only store short-length check codes, then the data protection type corresponding to the second data block is determined to be the first data protection type, so that the check code corresponding to the second data block can be generated through the second data protection type.

[0168] Step S5413: Determine that the data protection type corresponding to the second data block is the second data protection type;

[0169] Specifically, if the data length of the second data block is less than or equal to a preset length threshold, that is, the reserved space corresponding to the second data block is large, meaning that the reserved space can store a longer checksum, then the data protection type corresponding to the second data block is determined to be the second data protection type, so that the checksum corresponding to the second data block can be generated through the second data protection type.

[0170] Step S5042: Add protection data to the second data block according to the data protection type corresponding to the second data block, and generate the third data block;

[0171] Specifically, based on the data protection type corresponding to the second data block, protection data is added to the second data block. If the data length of the second data block is determined to be greater than a preset length threshold, then the data protection type corresponding to the second data block is determined to be the first data protection type. Protection data is generated using the first data protection type and added after the second data block to generate the third data block. If the data length of the second data block is determined to be less than or equal to the preset length threshold, then the data protection type corresponding to the second data block is determined to be the second data protection type. Protection data is generated using the second data protection type and added after the second data block to generate the third data block. The protection data includes metadata (METAData), which includes a checksum and an identifier (Flag). The Flag includes integrity protection information, key information, and algorithms, etc.

[0172] In this embodiment of the application, by adding protective data to each data block, the data protection strength of each data block can be increased, the data protection effectiveness can be improved, and the risk of data errors can be reduced.

[0173] Please refer to 9 again. Figure 9 yes Figure 7 Detailed flowchart of step S5042;

[0174] like Figure 9 As shown, step S5042 includes:

[0175] Step S5421: According to the first data protection type or the second data protection type, add the first protection data or the second protection data to the second data block to generate the third data block;

[0176] Specifically, if the data protection type corresponding to the second data block is the first data protection type, then according to the first preset algorithm corresponding to the first data protection type, the first protection data of the second data block is calculated, and the first protection data is added to the second data block to generate the third data block; if the data protection type corresponding to the second data block is the second data protection type, then according to the second preset algorithm corresponding to the second data protection type, the second protection data of the second data block is calculated, and the second protection data is added to the second data block to generate the third data block, wherein the data length of the first protection data is less than the data length of the second protection data. The first preset algorithm includes the DIF / DIX algorithm, and the second preset algorithm includes Differential Private Exponential Constant Compounding Code (DPECC). DPECC includes the ErrorCorrectionCode (ECC) algorithm based on Reed-Solomon (RS) and the BCH algorithm (Bose-Chaudhuri-Hocquenghem, BCH). 。

[0177] Please refer to 10 as well. Figure 10 yes Figure 5 A detailed flowchart of step S504;

[0178] like Figure 10 As shown, step S504 includes:

[0179] Step S5043: Determine the data protection type corresponding to the second check block based on the data length of the second check block;

[0180] Specifically, based on the data length of the second check block, the size of the remaining space after the first check block is compressed is determined, thereby determining the data protection type corresponding to the second check block.

[0181] Please refer to 11 again. Figure 11 yes Figure 10 Detailed flowchart of step S5043;

[0182] like Figure 11 As shown, step S5043 includes:

[0183] Step S5431: Determine whether the data length of the second verification block is greater than the preset length threshold;

[0184] Specifically, it is determined whether the data length of the second verification block is greater than the preset length threshold. If the data length of the second verification block is greater than the preset length threshold, the process jumps to step S5432. If the data length of the second verification block is less than or equal to the preset length threshold, the process jumps to step S5432.

[0185] Step S5432: Determine that the data protection type corresponding to the second check block is the first data protection type;

[0186] Specifically, if the data length of the second verification block is greater than the preset length threshold, that is, the reserved space corresponding to the second verification block is small, meaning that the reserved space can only store short-length verification codes, then the data protection type corresponding to the second verification block is determined to be the first data protection type, so as to generate the verification code corresponding to the second verification block through the second data protection type.

[0187] Step S5433: Determine that the data protection type corresponding to the second check block is the second data protection type;

[0188] Specifically, if the data length of the second verification block is less than or equal to the preset length threshold, that is, the reserved space corresponding to the second verification block is large, that is, the reserved space can store a longer verification code, then the data protection type corresponding to the second verification block is determined to be the second data protection type, so as to generate the verification code corresponding to the second verification block through the second data protection type.

[0189] Step S5044: Based on the data protection type corresponding to the second check block, add protection data to the second check block to generate the third check block;

[0190] Specifically, based on the data protection type corresponding to the second verification block, protection data is added to the second verification block. That is, if it is determined that the data length of the second verification block is greater than a preset length threshold, then the data protection type corresponding to the second verification block is determined to be the first data protection type, protection data is generated using the first data protection type, and this protection data is added after the second verification block to generate the third verification block. If it is determined that the data length of the second verification block is less than or equal to the preset length threshold, then the data protection type corresponding to the second verification block is determined to be the second data protection type, protection data is generated using the second data protection type, and this protection data is added after the second verification block to generate the third verification block.

[0191] In this embodiment, the third data block includes a second data block and protection data. When the sum of the data length of the second data block and the data length of the protection data is less than the data length of the first data block, that is, the protection data in the third data block does not completely fill the reserved space corresponding to the second data block, virtual data is added to the reserved space corresponding to the second data block to fill the reserved space corresponding to the second data block, so that the data length of the third data block is equal to the length of the first data block. The data length of the third data block = the data length of the second data block + the data length of the protection data + the data length of the virtual data.

[0192] In the embodiments of this application, virtual data is used to process missing values, outliers, or data that cannot be directly observed, thereby improving the integrity and accuracy of the data.

[0193] In this embodiment of the application, by adding protective data to the second verification block, the security of the data can be further improved to prevent the second verification block from being tampered with.

[0194] Please refer to the following: Figure 12 , Figure 12 yes Figure 10 Detailed flowchart of step S5044;

[0195] like Figure 12 As shown, step S5044 includes:

[0196] Step S5441: According to the first data protection type or the second data protection type, add the first protection data or the second protection data to the second verification block to generate the third verification block;

[0197] Specifically, if the data protection type corresponding to the second verification block is the first data protection type, then according to the first preset algorithm corresponding to the first data protection type, the first protection data of the second verification block is calculated and encoded, and the first protection data is added to the second verification block to generate the third verification block; if the data protection type corresponding to the second data block is the second data protection type, then according to the second preset algorithm corresponding to the second data protection type, the second protection data of the second verification block is calculated and encoded, and the second protection data is added to the second verification block to generate the third verification block, wherein the data length of the first protection data is less than the data length of the second protection data.

[0198] Please refer to the following: Figure 13 , Figure 13 This application provides a schematic flowchart for calculating first protection data;

[0199] like Figure 13 As shown, the first protection data is calculated, including:

[0200] Step S1301: Calculate the first protection data according to the first preset algorithm and the data length of the second data block;

[0201] In this embodiment of the application, the DIF / DIX algorithm is used as an example. The DIF / DIX algorithm adopts an end-to-end protection protocol. The end-to-end protection protocol is a protocol that ensures that data can be fully protected and encrypted during transmission. The end-to-end protection protocol includes SSL / TLS protocol (Secure Sockets Layer, SSL; Transport Layer Security, TLS), IPsec protocol, and Zero Touch Encryption (ZTE).

[0202] Specifically, based on the data length of the second data block, the difference between the data length of the first data block and the data length of the second data block is calculated. Based on this difference, the check code length corresponding to the first preset algorithm is preset. The preset check code length corresponding to the first preset algorithm is used as the parameter of the DIF / DIX algorithm. The first protection data is generated by encoding through the DIF / DIX algorithm. The length of the first protection data is the same as the preset check code length corresponding to the first preset algorithm.

[0203] For example, if the data length of the first data block is 64B and the data length of the second data block is 32B, then the difference between the data length of the first data block and the data length of the second data block is 32B, and the check code length can be preset to 32B.

[0204] In the embodiments of this application, when the first data protection type is used to generate the first protection data for the second verification block, the principle is the same as that for calculating the first protection data of the second data block, and will not be described in detail here.

[0205] In this embodiment, given a preset checksum length, when the length of the first data block is not an integer multiple of the length of the second data block, the first protection data is added to the second data block. The resulting third data block does not have the same length as the first data block. Therefore, the difference between the lengths of the first and second data blocks is subtracted from the preset checksum length to obtain a virtual data length. This virtual data length is then padded with virtual data, and the first protection data is added after the virtual data. For example, if the length of the first data block is 64 bytes and the length of the second data block is 28 bytes, the difference between them is 36 bytes. The preset checksum length is 32 bytes, so the virtual data length is 36 bytes - 32 bytes = 4 bytes. This virtual data is then padded to fill the virtual data length. Virtual data is simulated or tested data used to handle missing values, outliers, or data that cannot be directly observed, thus improving data integrity and accuracy. Virtual data includes dummy data.

[0206] Please refer to the following: Figure 14, Figure 14 This is a schematic diagram of a process for calculating second protection data provided in an embodiment of this application;

[0207] like Figure 14 As shown, the calculation of the second protection data includes:

[0208] Step S1401: Calculate the second protection data according to the second preset algorithm and the data length of the second verification block;

[0209] In this embodiment of the application, the ErrorCorrectionCode (ECC) algorithm based on Reed-Solomon (RS) is used as an example to calculate the second protection data.

[0210] Specifically, based on the data length of the second data block, the difference between the data length of the first check block and the data length of the second check block is calculated. Based on this difference, the check code length corresponding to the second preset algorithm is preset. The preset check code length corresponding to the second preset algorithm is used as the parameter of the ErrorCorrectionCode (ECC) algorithm based on Reed-Solomon (RS). The second protection data is generated by encoding the ErrorCorrectionCode (ECC) algorithm based on Reed-Solomon (RS). The length of the second protection data is the same as the preset check code length corresponding to the second preset algorithm.

[0211] In the embodiments of this application, when the second data protection type is used to generate the first protection data for the second data block, the principle is the same as that for calculating the second protection data of the second check block, and will not be described in detail here.

[0212] In this embodiment of the application, when the data length of the first verification block is not an integer multiple of the data length of the second verification block, virtual data is used to fill the gap, so that the data length of the third verification block is equal to the data length of the first verification block. The third verification block consists of a second data block, virtual data, and second protection data.

[0213] In the embodiments of this application, by adding protection data to each second data block and second check block, the data security and reliability of each data block and check block can be improved.

[0214] Please refer to the following: Figure 15 , Figure 15 This is a schematic diagram illustrating an example of data block and check block compression and decompression provided in an embodiment of this application.

[0215] like Figure 15As shown, the algorithm engine, corresponding to the verification algorithm, divides the user data into three first data blocks. The verification algorithm calculates the first verification block corresponding to the three first data blocks and sends the three first data blocks and the first verification block to the compression engine. The compression engine compresses each first data block and the first verification block using its corresponding compression algorithm to obtain a second data block and a second verification block. Multiple second data blocks and second verification blocks are then sent to the data protection engine. The data protection engine adds protection data to the second data blocks and the second verification block using its corresponding protection algorithm, and adds virtual data to complete the second data blocks and the second verification block to obtain a third data block and a third verification block. The third data block consists of the second data block, virtual data, and first protection data / second protection data. The third verification block consists of the second verification block, virtual data, and first protection data / second protection data.

[0216] Step S505: Store the third data block in the corresponding memory, and store the third parity block in the corresponding memory;

[0217] Specifically, the third data block with added protection data is stored in the memory, and the third verification block with added protection data is also stored in the memory.

[0218] Please refer to the following: Figure 16 , Figure 16 This is an example schematic diagram of storing the third data block and the third parity block in a storage array according to an embodiment of this application;

[0219] like Figure 16 As shown, A1, A2, A3, B1, B2, B3, C1, C2, C3, D1, D2, and D3 are all first data blocks. A1-A3, B1-B3, C1-C3, and D1-D3 each form a stripe. Ap is the second parity block after compressing the first parity block corresponding to A1-A3, Bp is the second parity block after compressing the first parity block corresponding to B1-B3, Cp is the second parity block after compressing the first parity block corresponding to C1-C3, and Dp is the second parity block after compressing the first parity block corresponding to D1-D3. The second check block is a compressed version of the first check block. DP includes virtual data and protection data. A11-A31 are second data blocks, B11-B31 are second data blocks, C11-C31 are second data blocks, and D11-D31 are second data blocks. The second check block and DP together form the third check block. The storage array 110 includes memory 1, memory 2, memory 3, and memory 4. Multiple third data blocks are randomly stored in the memory of the storage array 110. Multiple third data blocks form a stripe.

[0220] It should be noted that, Figure 16This example only uses the division of each user's data into 3 data blocks and the addition of protection data to each data block. The specific division of user data into multiple data blocks can be determined according to specific needs. In addition, the number of memories included in the storage array 110 can be determined according to specific needs and is not limited here.

[0221] In this embodiment of the application, by adding protection data to each data block in each stripe corresponding to each user data, errors in user data during transmission can be avoided, thereby improving the protection of the data.

[0222] In this embodiment of the application, when retrieving user data from the storage array, it is necessary to decode the data in the storage array, specifically including: decoding the third data block, specifically including:

[0223] The third data block corresponding to the user data is retrieved from the storage array. Using the algorithm corresponding to the protection data of the third data block, it is decoded to determine if there are any errors. If no errors are found, the protection data in the third data block is deleted, resulting in the second data block. This second data block is then decompressed using a compression algorithm. If errors are found in the third data block, it indicates that an error occurred during transmission. The erroneous third data block is then recovered, specifically including:

[0224] The data protection type of the third data block is determined. If it is the second data protection type, error correction is directly performed on the third data block. If the error correction is successful, the recovery of the third data block is completed. The protected data of the third data block is stripped to restore the second data block. The second data block is then decompressed to obtain the first data block, thus completing the decoding of the user data.

[0225] If the data protection type of the third data block is the first data protection type, or if the error correction fails when performing the error correction under the second data protection type, the RAID recovery process will be initiated. Specifically, this process includes: reading other data blocks of the stripe corresponding to the user data from the storage array to perform RAID calculations in order to recover the user data. These other data blocks include parity blocks.

[0226] Retrieve other lossless third data blocks and third parity blocks from the same band in the storage array, and recover the third data block that currently has a bit error.

[0227] Among them, the algorithms corresponding to the protection data of the third data block include a first preset algorithm and a second preset algorithm, both of which are verification algorithms.

[0228] Please refer to the following: Figure 20 , Figure 20 This is a schematic diagram illustrating reading user data from a memory according to an embodiment of this application;

[0229] like Figure 20 As shown, user data A11 is read from memory 1. The data protection engine decodes user data A11 and determines its correctness. If user data A11 is normal, that is, there are no errors in user data A11, then data A1 is sent to the compression engine so that the compression engine can decompress data A1 to obtain decompressed data A1. Then, the compression engine forwards the decompressed data A1 to the algorithm engine. The algorithm engine then passes the decompressed data A1 through the host interface for transmission to the host.

[0230] The above-mentioned situation assumes that user data A11 does not contain any errors. If user data A11 contains errors, then it is necessary to correct user data A11 through the data protection engine.

[0231] Please refer to the following: Figure 21 , Figure 21 This is a schematic diagram illustrating another method for reading user data from a memory according to an embodiment of this application;

[0232] like Figure 21 As shown, when the data protection engine decodes user data A11, if errors occur in user data A11, the data protection engine corrects the errors to obtain corrected user data, and then sends the corrected user data to the compression engine. After receiving the corrected user data A1, the compression engine decompresses the corrected user data A1 to obtain decompressed user data A1, and then sends the decompressed user data A1 to the algorithm engine. The algorithm engine then transparently transmits the decompressed data A1 to the host interface for transmission to the host.

[0233] Understandably, the scenarios mentioned above represent situations where the data protection engine successfully corrects user data, and the data protection engine transmits the successfully corrected user data to the compression engine. If the data protection engine fails to correct the data, then the algorithm engine needs to use data blocks from other storage devices to recover the data.

[0234] For details, please refer to [link / reference]. Figure 22 , Figure 22 This is a schematic diagram illustrating another method of reading user data from a memory provided in an embodiment of this application;

[0235] like Figure 22As shown, when user data A11 in memory 1 has a bit error, and the data protection engine is unable to successfully correct the error in user data A11, for example, if the data protection algorithm is a non-ECC protection algorithm, or if ECC recovery fails, then the algorithm engine needs to use data blocks from other memories to perform the recovery operation. For example, data recovery can be performed using user data A21 in memory 2, user data A31 in memory 3, and check data Ap in memory 4. It can be understood that user data A11, A21, and A31 constitute a stripe, and check data Ap is the check data of the stripe.

[0236] Please refer to the following: Figure 23 , Figure 23 This is a schematic diagram of a process for reading user data from a memory according to an embodiment of this application;

[0237] like Figure 23 As shown, the process of reading user data from memory includes:

[0238] Step S2301: Read the user data of the storage array through the storage device interface and send it to the data protection engine;

[0239] Specifically, the storage array controller reads user data from the storage array through the storage device interface. This user data includes a third data block, which is then received by the data protection engine.

[0240] Step S2302: The data protection engine receives user data and decodes it according to the data protection type;

[0241] Specifically, the data protection engine receives user data, i.e., the third data block, and decodes the third data block according to the data protection type corresponding to the third data block. For example, if the data protection type corresponding to the third data block is the first data protection type, the third data block is decoded by the first preset algorithm to obtain the decoded third data block; if the data protection type corresponding to the third data block is the second data protection type, the third data block is decoded by the second preset algorithm to obtain the decoded third data block.

[0242] Step S2303: Are there any bit errors?

[0243] Specifically, it determines whether there are any errors in the decoded third data block, that is, whether there are any errors in the decoded user data;

[0244] If the decoded user data contains errors, proceed to step S2307;

[0245] If the decoded user data does not contain any errors, proceed to step S2304.

[0246] Step S2304: The data protection engine forwards the data to the compression engine;

[0247] Specifically, if the decoded user data does not contain any errors, the data protection engine will delete the protection data in the third data block, obtain the second data block corresponding to the third data block, and forward the second data block to the compression engine.

[0248] Step S2305: The compression engine receives the data, decompresses the data, and forwards it to the algorithm engine;

[0249] Specifically, the compression engine receives the second data block sent by the data protection engine, decompresses the second data block to obtain the first data block corresponding to the second data block, and forwards the first data block to the algorithm engine.

[0250] Step S2306: The algorithm engine receives the data and forwards it directly to the host interface;

[0251] Specifically, the algorithm engine receives the first data block sent by the compression engine and forwards it to the host through the host interface, so that the host can receive the first data block, which is the decompressed user data.

[0252] Step S2307: Is it a second data protection type?

[0253] Specifically, it determines whether the third data block corresponds to the second data protection type, where the second data protection type corresponds to an error-correcting algorithm, such as ECC algorithm or BCH algorithm. In this case, the data protection engine can correct the data using the error-correcting algorithm.

[0254] If the third data block corresponds to the second data protection type, proceed to step S2308;

[0255] If the third data block does not correspond to the second data protection type, for example, if the third data block corresponds to the first data protection type, then proceed to step S2310.

[0256] Step S2308: The data protection engine performs error correction;

[0257] Specifically, if the third data block corresponds to the second data protection type, the data protection engine performs data error correction on the third data block. Specifically, the data protection engine performs data error correction on the third data block using error correction algorithms, such as ECC algorithm and BCH algorithm.

[0258] Step S2309: Was the error correction successful?

[0259] Specifically, determine whether the data protection engine has successfully corrected the third data block. If so, proceed to steps S2304 and S2311.

[0260] Specifically, if the data protection engine successfully corrects the error in the third data block and obtains the corrected user data, the data protection engine forwards the corrected user data to the compression engine. The compression engine receives the corrected user data, decompresses the data, and forwards the decompressed user data to the algorithm engine. The algorithm engine then sends the decompressed user data back to the host through the host interface.

[0261] Step S2310: Enter the RAID recovery process;

[0262] Specifically, if the data protection engine fails to correct the error in the third data block, it enters the RAID recovery process. This RAID recovery process includes: reading other data blocks of the stripe corresponding to the user data from the storage array, performing RAID calculations to recover the user data, where other data blocks include parity blocks, for example: [reference needed]. Figure 22 The second data block A1 is recovered using data blocks A21 and A31 in memory 2 and memory 3, respectively, and the parity block Ap in memory 4. The specific process includes:

[0263] (1) Obtain the third data block A21 in memory 2, the third data block A31 in memory 3 and the third parity block Ap in memory 4, wherein the third data block A11 in memory 1, the third data block A21 in memory 2 and the third data block A31 in memory 3 form a stripe, wherein the third parity block Ap in memory 4 is the parity block corresponding to the third data block A11 in memory 1, the third data block A21 in memory 2 and the third data block A31 in memory 3;

[0264] (2) The data protection engine decodes the third data block A21 in memory 2, the third data block A31 in memory 3, and the third parity block Ap in memory 4 to determine whether there are any errors. If no errors are found, the protection data DP in the third data block A21 in memory 2, the third data block A31 in memory 3, and the third parity block Ap in memory 4 are deleted respectively to obtain the second data block A2 corresponding to the third data block A21, the second data block A3 corresponding to the third data block A31, and the second parity block Ap corresponding to the third parity block Ap.

[0265] Understandably, if any of the third data block A21 in memory 2, the third data block A31 in memory 3, or the third parity block Ap in memory 4 contains a bit error, the RAID recovery will fail, and the algorithm engine will return a data error to the host.

[0266] (3) The compression engine decompresses the second data block A2 corresponding to the third data block A21, the second data block A3 corresponding to the third data block A31, and the second check block Ap corresponding to the third check block Ap, respectively, to obtain the first data block A'2 corresponding to the second data block A2, the first data block A'3 corresponding to the second data block A3, and the first check block A'p corresponding to the second check block Ap.

[0267] (4) The algorithm engine recovers the first data block A'1 corresponding to the third data block A11 based on the first data block A'2, the first data block A'3 and the first check block A'p.

[0268] (5) The algorithm engine sends the first data block A'1 to the host.

[0269] Specifically, the algorithm engine sends the first data block A'1 to the host interface through the NVMe controller so that the host can obtain the first data block A'1.

[0270] Step S2311: Write the correct data back directly to the original storage array;

[0271] Specifically, if the data protection engine successfully corrects the error in the third data block, it writes the corrected user data back to the corresponding memory in the storage array, ensuring that the memory stores the correct user data. It's understandable that the write-back process is a rewrite, consistent with the write process, and will not be elaborated further here.

[0272] Please refer to the following: Figure 24 , Figure 24 This is a schematic diagram of another process for reading user data from a memory according to an embodiment of this application;

[0273] like Figure 24 As shown, the process of reading user data from memory includes:

[0274] Step S2401: Read user data from the storage array through the storage interface and forward it to the data protection engine;

[0275] Step S2402: The data protection engine receives user data and decodes it according to the data protection type;

[0276] Step S2403: Are there any errors?

[0277] Step S2404: The data protection engine forwards the data to the compression engine;

[0278] Step S2405: The compression engine receives the data, decompresses the data, and forwards it to the algorithm engine;

[0279] Step S2406: The algorithm engine receives data;

[0280] Step S2407: Should all other stripe data be received?

[0281] Specifically, the algorithm engine determines whether all other stripe data has been received. If not, it returns to step S2401 to continue acquiring other stripe data until all other stripe data is acquired. It is understandable that all other stripe data, including the third data block and the third check block on the same stripe, needs to be acquired in order to recover the erroneous third data block.

[0282] Step S2408: The algorithm engine performs RAID calculations, recovers the target data, and forwards it to the host interface;

[0283] The target data refers to the original user data in the storage array, that is, the uncompressed data corresponding to the third data block where the error occurred, i.e., the first data block.

[0284] Step S2409: Is it a second data protection type?

[0285] Step S2410: The data protection engine performs error correction;

[0286] Step S2411: Was the error correction successful?

[0287] Step S2412: RAID recovery process failed, reporting a data error to the host.

[0288] Specifically, if the RAID recovery process fails, or if the third data block does not correspond to the second data protection type, data recovery cannot be performed. For example, if a read error is encountered, and DPECC cannot correct the error or it is of type DIX, the entire recovery process will fail. In this case, the algorithm engine sends a data error message to the host.

[0289] By using a data protection engine and / or RAID recovery process to recover data that has encountered errors, this application can better achieve data reading.

[0290] In this embodiment of the application, user data is acquired, wherein the user data is divided into at least two first data blocks; verification data corresponding to at least two first data blocks is calculated to generate a first verification block; the first data blocks are compressed to obtain second data blocks corresponding to the first data blocks, and the first verification blocks are compressed to obtain second verification blocks corresponding to the first verification blocks; protection data is added to the second data blocks to generate a third data block, and protection data is added to the second verification blocks to generate a third verification block; the third data blocks are stored in the corresponding memory, and the third verification blocks are stored in the corresponding memory.

[0291] By compressing at least two data blocks and a check block corresponding to the user data respectively, and adding protection data to the compressed data blocks and check blocks respectively, and storing the data blocks and check blocks with added protection data to the corresponding memory, this application can increase the protection of the data in the storage array and improve the security of the data in the storage array.

[0292] Please see Figure 17 , Figure 17 This is a schematic diagram of the structure of a storage array controller provided in an embodiment of this application;

[0293] like Figure 17 As shown, the storage array controller 120 includes: an algorithm engine 1201, a compression engine 1202, and a data protection engine 1203.

[0294] Algorithm engine 1201, connected to compression engine 1202, is used to acquire at least two first data blocks corresponding to user data, calculate the check data corresponding to the at least two first data blocks, and send the at least two first data blocks and the check data to compression engine 1202. Algorithm engine 1201 includes a RAID algorithm engine, which can be configured with different check algorithms corresponding to different RAID levels as needed, such as RAID 0, RAID 1, and RAID 5.

[0295] Compression engine 1202, connected to algorithm engine 1201 and data protection engine 1203, is used to compress the first data block to obtain the second data block corresponding to the first data block, and to compress the first check block to obtain the second check block corresponding to the first check block, and to send the second data block and the second check block to data protection engine 1203.

[0296] The data protection engine 1203 is connected to the compression engine 1202, and is used to add protection data to the second data block, generate the third data block, add protection data to the second check block, generate the third check block, and store the third data block to the storage array 110 through the storage device interface 123.

[0297] Please see Figure 18 , Figure 18 This is a schematic diagram of the structure of a data protection engine provided in an embodiment of this application;

[0298] like Figure 18 As shown, the data protection engine 1203 includes a data preprocessing engine 1231, a first protection engine 1232, and a second protection engine 1233.

[0299] The data preprocessing engine 1231 is used to obtain the data length of the second data block and the data length of the second check block. Further, based on the data length of the second data block, it determines the protection engine corresponding to the second data block and the protection engine corresponding to the second check block.

[0300] The data preprocessing engine 1231 is also used to determine the protection engine corresponding to the second data block as the first protection engine when the data length of the second data block is greater than the preset length threshold, and to determine the protection engine corresponding to the second data block as the second protection engine when the data length of the second data block is less than or equal to the preset length threshold.

[0301] The data preprocessing engine 1231 is also used to determine the protection engine corresponding to the second verification block as the first protection engine when the data length of the second verification block is greater than the preset length threshold, and to determine the protection engine corresponding to the second verification block as the second protection engine when the data length of the second verification block is less than or equal to the preset length threshold.

[0302] The first protection engine 1232 is used to calculate the first protection data according to the first preset algorithm and the data length of the second data block. The first protection engine includes a DIF / DX engine, which corresponds to the DIF / DIX mechanism. The DIF / DIX engine is used to generate the first protection data for the second data block, and the first protection data can detect whether the second data block has errors.

[0303] The second protection engine 1233 is used to calculate the second protection data according to the second preset algorithm and the data length of the second check block. The second protection engine 1233 includes a DPECC engine, which corresponds to the second preset algorithm. The second preset algorithm includes various verification algorithms, such as the ECC algorithm. The DPECC engine is used to generate the second protection data for the second check block. The second protection data can detect whether the second data block is erroneous and correct damaged data.

[0304] In this embodiment of the application, before sending user data to the data path 121, the algorithm engine 1201 needs to be initialized first. For example, the number of data blocks into which each user data is divided is determined, such as determining that each user data is divided into 3 data blocks, so as to calculate the 3 data blocks corresponding to each user data to generate a check block. At the same time, the buffer is initialized to 0 to ensure that the buffer is not used by other programs to improve the calculation efficiency. The buffer is used to buffer user data.

[0305] Furthermore, the compression engine 1202 is initialized to determine the granularity, that is, to determine the smallest compressed data unit, so that user data is compressed based on the smallest compressed data unit.

[0306] Furthermore, initialize the data protection engine 1203, set the protection engine corresponding to the protected data, that is, determine the algorithm corresponding to different data lengths.

[0307] In this embodiment of the application, the processing efficiency of user data can be improved by initializing the data path 121.

[0308] In this application embodiment, by compressing at least two data blocks and a check block corresponding to user data respectively, and adding protection data to the compressed data blocks and check blocks respectively, and storing the data blocks and check blocks with added protection data to the corresponding memory, this application can increase the protection of data in the storage array and improve the security of data in the storage array.

[0309] Please see Figure 19 , Figure 19 This is a schematic diagram of the overall structure of a storage device provided in an embodiment of this application;

[0310] like Figure 19 As shown, the storage device 100 includes a storage array 110 and a storage array controller 120. The storage array controller 120 includes a data path 121, an NVMe controller 122, a storage device interface 123, and a host interface 124. The host 200 includes a processor system 201, a dynamic random access memory 202, and a storage interface 203.

[0311] Data path 121 connects NVMe controller 122 and storage device interface 123, and is used to process data from host 200, add protection data to the data, and store the protected data in storage array 110. Data path 121 includes algorithm engine 1201, compression engine 1202, and data protection engine 1203.

[0312] Algorithm engine 1201, connected to compression engine 1202, is used to obtain at least two first data blocks corresponding to user data, wherein the user data comes from host 200, and calculate the verification data corresponding to at least two first data blocks, and send at least two first data blocks and verification data to compression engine 1202.

[0313] Specifically, the algorithm engine 1201 is also used to read other data blocks of the stripe corresponding to the user data through the storage array to perform RAID calculations in order to recover the user data when the data protection type corresponding to the user data is the first data protection type;

[0314] Algorithm engine 1201 is also used to perform RAID calculations to recover user data after the data protection engine fails to correct errors in the decoded user data by reading other data blocks of the stripe corresponding to the user data from the storage array.

[0315] Compression engine 1202 is connected to RAID algorithm engine 1201 and data protection engine 1203. It is used to compress the first data block to obtain the second data block corresponding to the first data block, and to compress the first parity block to obtain the second parity block corresponding to the first parity block. The second data block and the second parity block are then sent to data protection engine 1203.

[0316] Specifically, the compression engine 1202 is also used to decompress the error-corrected user data and forward the decompressed user data to the algorithm engine 1201, so that the algorithm engine 1201 forwards the decompressed user data to the host interface.

[0317] The data protection engine 1203 is connected to the compression engine 1202, and is used to add protection data to the second data block, generate the third data block, add protection data to the second check block, generate the third check block, and store the third data block to the storage array 110 through the storage device interface 123.

[0318] Specifically, the data protection engine 1203 is also used to read user data from the memory, decode the user data according to the data protection type corresponding to the user data, and obtain the decoded user data. If the decoded user data contains errors and the data protection type corresponding to the user data is the second data protection type, the data protection engine corrects the decoded user data to obtain the corrected user data and forwards the corrected user data to the compression engine.

[0319] The NVMe controller 122 communicates with the storage interface 203 and the data path 121 to realize data exchange and communication between the host 200 and the storage device 100. For example, it receives the data stream sent by the host 200 from the storage interface 203. The data stream is user data in the host 200. It converts the format of the user data and transmits it to the data path 121 to add protection data to the user data through the data path 121. The user data with added protection data is then stored in the storage array 110. The NVMe controller 122 adopts application-level communication protocols, such as Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Simple Mail Transfer Protocol (SMTP), and Message Queuing Telemetry Transport (MQTT).

[0320] Storage device interface 123 communicates with storage array 110 and data path 121. Storage device interface 123 is used to realize data exchange between storage device controller 120 and storage array 110. For example, it receives data streams sent from storage array 110, which are data in storage array 110, and performs format conversion on the data in storage array 110. When storage array 110 includes multiple SATA hard drives, storage device interface 123 can use the SATA interface protocol to communicate with multiple SATA hard drives through the SATA interface protocol.

[0321] The host interface 124 is used to connect to the host 200 to enable data and / or command interaction between the storage array controller 120 and the host 200. For example, the host interface 124 is connected to the storage interface 203 of the host 200 to enable data and / or command interaction between the storage array controller 120 and the host 200. In this embodiment, the host interface 124 includes, but is not limited to, interfaces such as USB and PCIe.

[0322] Specifically, the host 200 includes a processor system 201, a dynamic random access memory 202, and a storage interface 203.

[0323] The processor system 201 is connected to the dynamic random access memory 202 and the storage interface 203. The processor system 201 is used to process data inside the host 200 and to control and coordinate the operation between the components inside the host 200.

[0324] Dynamic random access memory 202 is connected to processor system 201 and storage interface 203 and is used to cache data to improve the data processing speed of host 200.

[0325] Storage interface 203 communicates with processor system 201 and dynamic random access memory 202, and is used to realize data transmission between storage device 100 and host 200. Storage interface 123 includes PCIe controller and physical layer (Port Physical Layer, PHY). PCIe controller adopts PCIe interface protocol.

[0326] It should be noted that the aforementioned algorithm engine 1201, compression engine 1202, and data protection engine 1203 are also used to perform other related steps. For details, please refer to the relevant content mentioned in the above embodiments, which will not be repeated here. In this embodiment, user data is acquired, wherein the user data is divided into at least two first data blocks; verification data corresponding to at least two first data blocks is calculated to generate a first verification block; the first data blocks are compressed to obtain second data blocks corresponding to the first data blocks, and the first verification blocks are compressed to obtain second verification blocks corresponding to the first verification blocks; protection data is added to the second data blocks to generate a third data block, and protection data is added to the second verification blocks to generate a third verification block; the third data block is stored in a corresponding memory, and the third verification block is stored in a corresponding memory.

[0327] By compressing at least two data blocks and a check block corresponding to the user data respectively, and adding protection data to the compressed data blocks and check blocks respectively, and storing the data blocks and check blocks with added protection data to the corresponding memory, this application can increase the protection of the data in the storage array and improve the security of the data in the storage array.

[0328] The apparatus or device embodiments described above are merely illustrative. The unit modules described as separate components may or may not be physically separate, and the components shown as module units may or may not be physical units; that is, they may be located in one place or distributed across multiple network module units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0329] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions for a computer device (which may be a personal computer, server, or network device, etc.) to execute the various embodiments or some parts of the embodiments.

[0330] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above. For the sake of brevity, they are not provided in detail; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A data protection method, characterized in that, Applied to a storage array, the storage array comprising at least two memories, the method includes: Acquire user data, wherein the user data is divided into at least two first data blocks; Calculate at least two check data corresponding to the first data blocks to generate the first check block; The first data block is compressed to obtain a second data block corresponding to the first data block, and the first verification block is compressed to obtain a second verification block corresponding to the first verification block. Add protection data to the second data block to generate a third data block, and add protection data to the second verification block to generate a third verification block; The third data block is stored in the corresponding memory, and the third verification block is stored in the corresponding memory; The method further includes: Obtain the data length of the second data block, and obtain the data length of the second check block; The step of adding protection data to the second data block to generate the third data block includes: The data protection type corresponding to the second data block is determined based on the data length of the second data block; Based on the data protection type corresponding to the second data block, protection data is added to the second data block to generate the third data block; Add protection data to the second check block to generate a third check block, including: The data protection type corresponding to the second check block is determined based on the data length of the second check block; Based on the data protection type corresponding to the second verification block, protection data is added to the second verification block to generate the third verification block.

2. The method according to claim 1, characterized in that, The data protection types include a first data protection type and a second data protection type; The step of determining the data protection type corresponding to the second data block based on the data length of the second data block includes: If the data length of the second data block is greater than a preset length threshold, then the data protection type corresponding to the second data block is determined to be the first data protection type. If the data length of the second data block is less than or equal to a preset length threshold, then the data protection type corresponding to the second data block is determined to be the second data protection type. The step of determining the data protection type corresponding to the second check block based on the data length of the second check block includes: If the data length of the second verification block is greater than the preset length threshold, then the data protection type corresponding to the second verification block is determined to be the first data protection type. If the data length of the second verification block is less than or equal to a preset length threshold, then the data protection type corresponding to the second verification block is determined to be the second data protection type.

3. The method according to claim 2, characterized in that, The data length of the first protected data corresponding to the first data protection type is less than the data length of the second protected data corresponding to the second data protection type. The step of adding protection data to the second data block according to the data protection type corresponding to the second data block to generate the third data block includes: Based on the first data protection type or the second data protection type, add the first protection data or the second protection data to the second data block to generate a third data block; The step of adding protection data to the second verification block according to the data protection type corresponding to the second verification block to generate the third verification block includes: Based on the first data protection type or the second data protection type, the first protection data or the second protection data is added to the second verification block to generate the third verification block.

4. The method according to claim 3, characterized in that, The first data protection type corresponds to the first preset algorithm, and the method further includes: The first protected data is calculated based on the first preset algorithm and the data length of the second data block; The second data protection type corresponds to the second preset algorithm, and the method further includes: The second protection data is calculated based on the second preset algorithm and the data length of the second verification block.

5. The method according to claim 1, characterized in that, The method further includes: Read user data from the memory; According to the data protection type corresponding to the user data, the user data is decoded to obtain the decoded user data; If the decoded user data contains errors, determine whether the data protection type corresponding to the user data is the second data protection type; If so, the decoded user data is corrected to obtain corrected user data, and the corrected user data is forwarded to the host. If not, the decoded user data will be forwarded to the host.

6. The method according to claim 5, characterized in that, After correcting the decoded user data, the method further includes: Determine whether the error correction was successful; If the error correction is successful, the corrected user data is forwarded to the host and the corrected user data is written back to the memory. If error correction fails, the user data is recovered by reading other data blocks of the stripe corresponding to the user data from the storage array and performing RAID calculations.

7. A storage array controller, characterized in that, Applied to a storage array, the storage array including at least two memories, the storage array controller including: an algorithm engine, a compression engine, and a data protection engine; The algorithm engine, connected to the compression engine, is used to obtain at least two first data blocks corresponding to user data, and calculate the verification data corresponding to at least two first data blocks to generate a first verification block, and send at least two first data blocks and the first verification block to the compression engine. The compression engine, connected to the algorithm engine and the data protection engine, is used to compress the first data block to obtain a second data block corresponding to the first data block, and to compress the first verification block to obtain a second verification block corresponding to the first verification block, and to send the second data block and the second verification block to the data protection engine. The data protection engine, connected to the compression engine, is used to add protection data to the second data block to generate a third data block, and to add protection data to the second verification block to generate a third verification block. The data protection engine includes a data preprocessing engine; The data preprocessing engine is used to obtain the data length of the second data block, and to obtain the data length of the second verification block, to determine the protection engine corresponding to the second data block based on the data length of the second data block, and to determine the protection engine corresponding to the second verification block based on the data length of the second verification block.

8. The storage array controller according to claim 7, characterized in that, The data protection engine includes a first protection engine and a second protection engine. The first protection engine corresponds to a first preset algorithm, and the second protection engine corresponds to a second preset algorithm. The data preprocessing engine is further configured to determine the protection engine corresponding to the second data block as the first protection engine when the data length of the second data block is greater than a preset length threshold, and to determine the protection engine corresponding to the second data block as the second protection engine when the data length of the second data block is less than or equal to the preset length threshold. The data preprocessing engine is further configured to determine the protection engine corresponding to the second verification block as the first protection engine when the data length of the second verification block is greater than a preset length threshold, and to determine the protection engine corresponding to the second verification block as the second protection engine when the data length of the second verification block is less than or equal to the preset length threshold. The first protection engine is used to calculate the first protection data based on the first preset algorithm and the data length of the second data block; The second protection engine is used to calculate the second protection data based on the second preset algorithm and the data length of the second verification block.

9. The storage array controller according to claim 7, characterized in that, The data protection engine is further configured to read user data from the memory, decode the user data according to the data protection type corresponding to the user data, and obtain decoded user data. If the decoded user data contains errors and the data protection type corresponding to the user data is the second data protection type, the data protection engine corrects the decoded user data to obtain corrected user data and forwards the corrected user data to the compression engine. The compression engine is also used to decompress the error-corrected user data and forward the decompressed user data to the algorithm engine, so that the algorithm engine forwards the decompressed user data to the host interface.

10. The storage array controller according to claim 9, characterized in that, The algorithm engine is also used to read other data blocks of the stripe corresponding to the user data through the storage array to perform RAID calculation in order to recover the user data when the data protection type corresponding to the user data is the first data protection type. The algorithm engine is also used to perform RAID calculations by reading other data blocks of the stripe corresponding to the user data through the storage array after the data protection engine fails to correct the decoded user data.

11. A storage device, characterized in that, include: The storage array controller according to any one of claims 7-10, wherein the storage array controller includes a host interface and a storage device interface, the host interface being used to connect to a host; The storage array includes at least two memories, each of which is communicatively connected to the storage array controller via the storage device interface.

Images