Storage array control method, controller and storage device

Abstract

The invention discloses a storage array control method, a controller and a storage device, a storage array physical space is divided into a plurality of storage areas, sequential writing is executed in each storage area, and array random writing is converted into sequential writing in the storage areas. When a write request is processed, new data is additionally written into the current active storage area, the mapping relation is updated, old data is marked to be invalid, the old data and a check block are prevented from being read back, and write amplification and read amplification are eliminated. The random writing performance is remarkably improved, and the service life is prolonged.

Description

Technical Field

[0001] This application relates to the field of storage systems, specifically to a storage array control method, controller, and storage device. Background Technology

[0002] In modern distributed storage systems, erasure coding (EC) or RAID technology is commonly used to ensure data reliability. These technologies divide data into multiple blocks and calculate several parity blocks, then distribute these blocks across different storage nodes or hard drives to achieve fault tolerance. Traditional EC / RAID storage arrays have significant performance limitations when facing data updates, especially random write operations. When modifying an existing data block, existing solutions cannot directly overwrite it. They must first read back all other unmodified data blocks within the same stripe, along with the original parity blocks, combine the new data with the read-back old data, recalculate the new parity blocks for the entire stripe, and finally write the new data block and the new parity blocks together. This process leads to severe "read amplification" and "write amplification" problems, meaning a small data update operation is amplified into multiple physical read / write operations.

[0003] DRAM-less SSDs offer significant advantages over DRAM-based SSDs in terms of cost and power consumption. However, their lifespan is significantly shortened due to write amplification, severely limiting their application in high-reliability storage arrays. Therefore, there is an urgent need in the field for a storage array control solution that can effectively eliminate or significantly reduce read / write amplification, improve random write performance, and extend SSD lifespan. Summary of the Invention

[0004] This specification describes a storage array control method, controller, and storage device through several embodiments, and the specific technical solutions adopted are as follows:

[0005] In a first aspect, embodiments of this specification provide a storage array control method applied to a storage array controller, the storage array controller being electrically connected to a storage array and communicatively connected to a host, comprising: dividing the storage array into multiple storage areas, configuring multiple stripes in each storage area, wherein sequential writes are performed within the storage area; receiving a first write instruction from the host for a first target address, querying a mapping table to determine a first entity address in a first stripe within the corresponding first storage area; writing the first write data of the first write instruction to a second entity address in a second stripe within a second storage area; and updating the mapping address of the first target address to the second entity address in the mapping table.

[0006] Furthermore, the storage array controller also establishes and maintains a storage status table and a valid data count table. After updating the mapping address of the first target address to the second entity address in the mapping table, the controller further includes: marking the first entity address as invalid in the storage status table; and updating the valid data count of the first storage area in the valid data count table.

[0007] Furthermore, writing the first write data to the second address in the second stripe pointed to by the write pointer includes: when the data block of the second stripe is full, calculating the check block of the second stripe based on the first write data and the remaining write data in the second stripe; and writing the second stripe to the storage array.

[0008] Furthermore, writing the first write data of the first write instruction to the second entity address in the second stripe within the second storage area includes: writing the first write data to the second address in the second stripe pointed to by the write pointer within the second storage area, and updating the position of the write pointer according to the length of the first write data.

[0009] Furthermore, it also includes: in response to the fact that the proportion of valid data in the third storage area is lower than a preset threshold, performing garbage collection on the third storage area; wherein the entity address marked as valid is determined according to the storage status table of the third storage area; the valid data in the valid entity address is read; and a second write request is generated to write the valid data into the fourth storage area.

[0010] Furthermore, after generating a second write request to write the valid data to the fourth storage area, the mapping relationship of the valid data is updated in the mapping table; the valid count of the third storage area is cleared to zero in the valid count table; and the storage status table of the third storage area is reset.

[0011] Secondly, embodiments of this specification provide a storage array controller electrically connected to a storage array and communicatively connected to a host. The storage array controller includes an NVMe-oF controller, a storage area manager, a redundancy management unit, and a PCIe interface control unit. The storage area manager is configured to: divide the storage array into multiple storage areas, configure multiple stripes within each storage area, and perform sequential writes within each storage area; receive a first write instruction from the host for a first target address, query a mapping table to determine a first entity address in a first stripe within the corresponding first storage area; write the first write data of the first write instruction to a second entity address in a second stripe within a second storage area; and update the mapping address of the first target address to the second entity address in the mapping table.

[0012] Furthermore, the storage area manager includes: a sequential write conversion module for converting random write commands from the host into sequential write commands within the storage; an address translation and mapping module for establishing and managing a mapping table, a storage status table, and a validity count table; wherein the mapping table records the mapping relationship between the host's target address and the physical address of the storage array; the storage status table records the validity status of stored data; the validity count table records the amount of currently valid data in each storage area; and a garbage collection module for implementing storage area-based garbage collection in the storage array.

[0013] Furthermore, the redundancy manager is used to: calculate the check block of the stripe based on the data blocks when any stripe is full; and to perform error correction based on the remaining data blocks and check blocks when any data block in the stripe is read incorrectly.

[0014] Thirdly, embodiments of this specification provide a storage device communicatively connected to a host, including a storage array controller and a storage array; wherein the storage array controller is used to implement the storage array control method as described in the first aspect on the storage array.

[0015] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0016] In several embodiments of this specification, the provided distributed storage array control method divides the physical storage space of the distributed storage array into multiple storage areas of a preset size. Each storage area is configured as multiple redundant stripes, and random write commands are converted into sequential append writes within the storage area. During data updates, it avoids reading back old data blocks and old parity blocks by dynamically maintaining the mapping relationship between logical addresses and physical addresses, directly accessing the physical location of the latest data during reads. By implementing address mapping, sequential writes, and garbage collection at the controller level, it broadens the application scenarios of DRAM-less SSDs in high-performance storage systems.

[0017] Other features and advantages of various embodiments of this specification will be further revealed in the following detailed description and accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a storage device provided for an embodiment of this specification.

[0020] Figure 2 This is a schematic diagram of a storage area of ​​a storage array provided in an embodiment of this specification.

[0021] Figure 3 This is a data writing flowchart provided for an embodiment of this specification.

[0022] Figure 4 This is a schematic diagram illustrating data writing provided in the embodiments of this specification.

[0023] Figure 5 This is a schematic diagram of a mapping table update provided in an embodiment of the present invention.

[0024] Figure 6 This is a schematic diagram of the storage state table bitmap provided in an embodiment of the present invention.

[0025] Figure 7 This is a schematic diagram of data writing according to an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of this specification will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of this specification and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of this specification.

[0027] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0028] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to facilitate the description of the embodiments and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.

[0029] All data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0030] Please see Figure 1 The diagram shown is a schematic representation of a storage device provided in an embodiment of this specification.

[0031] like Figure 1 As shown in the embodiments of this specification, a storage device 20 is communicatively connected to a host system 10. The host system 10 is, for example, a personal computer, a laptop computer, or a server. The storage device 20 includes a storage array controller 210 and a storage array 220.

[0032] The storage array controller 210 includes an NVME-oF controller 211, a storage area manager 212, a redundancy management unit 213, and a PCIe interface control unit 214.

[0033] The NVME-oF controller 211 is used to implement the protocol interface control and related extensions of NVMe-oF (NVMe over Fabrics). NVM Express (NVMe) is a standard host controller interface for PCIe-based solid-state drives (SSDs). The NVME-oF controller 211 can communicate with the host via RDMA (Remote Direct Memory Access), Fibre Channel, and other interconnection methods.

[0034] The memory area manager 212 is used to convert received random write commands into sequential writes within the memory area. The memory area manager 211 includes a sequential write conversion module, an address translation and mapping module, and a garbage collection module.

[0035] The sequential write conversion module is used to convert random write commands from the host into sequential write commands within the storage area. It receives arbitrary random write commands (random address, random data amount) from host 10 and converts them into sequential append writes within pre-divided storage areas (ZONEs). Write data sent to storage array 220 is always written to the write pointer position of the current ZONE, thus achieving random writes at the ZONE granularity and sequential writes within the ZONE.

[0036] The address translation and mapping module is used to establish and manage the mapping table, storage status table, and validity count table. The mapping table records the mapping relationship between the target address of host 10 and the physical address of storage array 220, dynamically mapping the target address (logical block address) of host 10 from NVMe-oF controller 211 to the physical address (physical block address) of storage array 220. The storage status table records the validity status of stored data. The validity count table records the number of currently valid data entries in each storage area.

[0037] The garbage collection module is used to implement storage area-based garbage collection in storage array 220. When the effective data ratio of a storage area falls below a certain threshold, garbage collection for that storage area is triggered. All still valid data blocks (excluding parity blocks) in that storage area are read and rewritten sequentially to a new free storage area. Subsequently, the address mapping table is updated and the original storage area is cleared, making it free and reusable.

[0038] Redundancy management unit 213 is used to implement EC (erasure coding) or Redundant Array of Independent Disks (RAID) error correction code architecture on storage array 220. Data written to storage array 220 is encoded to generate parity data (also known as RAID parity data) that can simultaneously protect multiple physical units. Subsequently, when storage array controller 210 reads data from multiple physical units in storage array 220, it simultaneously reads the corresponding RAID parity data and performs decoding and / or error checking and correction operations on the read data based on this RAID parity data.

[0039] Storage array 220 is used to store user data sent by host 10. Storage array 220 may include multiple rewritable non-volatile memory modules, such as solid state disks (SSDs).

[0040] In one possible implementation, such as Figure 2 As shown, the storage array includes multiple SSDs, and the physical space of each SSD is divided into multiple fixed-size storage zones. Redundancy is configured within each zone, and the zone is divided into multiple stripes.

[0041] The following describes a storage array control method provided by an embodiment of this specification.

[0042] The memory controller divides the memory array into multiple memory regions, and configures multiple stripes within each memory region, where sequential writes are performed.

[0043] Upon receiving the host’s first write instruction for the first target address, the mapping table is queried to determine the first entity address in the first stripe within the first memory area;

[0044] Write the first write data of the first write instruction to the second entity address in the second stripe of the second memory area;

[0045] Write the first piece of data to the second address in the second stripe pointed to by the write pointer in the second memory area, and update the position of the write pointer according to the length of the first piece of data.

[0046] Furthermore, when the data blocks of the second strip are full, the check block of the second strip is calculated based on the first written data and the remaining written data in the second strip, and the second strip is written to the storage array.

[0047] Update the mapping address of the first target address to the second entity address in the mapping table.

[0048] Furthermore, the first entity address is marked as invalid in the storage status table, and the valid data count of the first storage area is updated in the valid data count table.

[0049] In one possible implementation, in response to the proportion of valid data in the third storage area being lower than a preset threshold, garbage collection is performed on the third storage area; wherein, based on the storage status table of the third storage area, a valid entity address is determined, valid data in the valid entity address is read, and a second write request is generated to write the valid data to the fourth storage area.

[0050] Update the mapping relationship of valid data in the mapping table, clear the valid count of the third storage area to zero in the valid count table, and reset the storage status table of the third storage area.

[0051] Specifically, a storage zone manager is built between the NVME-oF controller and the redundancy management unit. The storage zone manager maintains a mapping table, a storage status table (implemented using bitmaps), and a valid data count table. The mapping table records the mapping relationship between the target address and the physical address, the storage status table records the valid status of the data stored in the physical address, and the valid data count table records the number of valid data stored in the storage zone.

[0052] For details, please refer to Figure 3 The illustrated embodiment of this specification provides a data writing flowchart.

[0053] Receive write command; query mapping table to determine the entity address of old data; sequentially write new data to the latest free address of the current ZONE; update mapping table; mark the entity address of old data as invalid; update the old ZONE valid count table; transfer new data to the redundancy management unit and encode it into a new stripe.

[0054] The following describes a possible implementation of the data writing process in a storage array control method provided in this application.

[0055] The following example illustrates the data update process:

[0056] like Figure 4 As shown, the NVMe-oF controller receives a write IO request from the host, with the target address being OF-LBA1 and the data being Data_new.

[0057] Instead of overwriting the old data, Data_new is written sequentially to a free physical location in the currently active ZONE (e.g., SSD-LBA8).

[0058] like Figure 5 As shown, OF-LBA1 is pointed to the new SSD-LBA8.

[0059] Mark the old address (assuming it originally pointed to SSD-LBA1) as invalid in the storage status table bitmap, and decrement the valid data count of the original ZONE in the valid data count table. The storage status table bitmap is as follows: Figure 6 As shown.

[0060] like Figure 7 As shown, the new write request (which has been converted to a sequential write) and the updated mapping information are passed to the redundancy management unit.

[0061] The redundancy management unit treats the new data, Data_new, as part of the new stripe, and only needs to compute and write the new check block. There is absolutely no need to read the old data and old check blocks from the old stripe.

[0062] Compared to existing solutions, adding a storage area manager before the redundancy management unit eliminates the need for all data block modifications. Instead, data at the original physical address is directly marked as invalid, and the address mapping table is refreshed to the new physical address. The operations of reading back data and parity blocks disappear, and the write amplification and read amplification introduced by rewriting old data are eliminated, significantly improving disk persistence performance and SSD lifespan.

[0063] At the same time, parity blocks and data blocks are processed in the same way (append write), which means that parity blocks and data blocks have the same wear. Therefore, there is no need to force parity blocks to be distributed to specific SSDs, achieving balanced wear across all SSDs.

[0064] The following describes a possible implementation of the data writing process in a storage array control method provided in this application.

[0065] The system continuously monitors the valid data count table and triggers garbage collection when the percentage of valid data in a zone falls below a preset threshold.

[0066] Scan the bitmap (i.e., storage status table) of the ZONE to be reclaimed to find all valid data that is still marked as Valid.

[0067] This valid data is read out, and a new write command is generated and submitted to the storage array controller.

[0068] Write the valid data sequentially into a new free zone.

[0069] After the write is complete, update the mapping relationship corresponding to these valid data OF-LBAs in the mapping table, and point the mapping relationship to their new entity address in the new ZONE.

[0070] The valid count of the original zone is cleared to zero, and a zone Reset command is sent to the storage array to clear the entire zone and make it idle, ready for subsequent writing.

[0071] Efficient garbage collection is achieved at the array level, thus minimizing the garbage collection requirements of the back-end SSDs. Consequently, the DRAM capacity requirements of the SSDs are reduced, allowing the use of DRAM-less SSDs without internal garbage collection without compromising performance.

[0072] In summary, the storage array control method, storage array controller, and storage device provided in this application convert random writes into sequential writes to the SSD, avoiding read-back operations for updating data, thereby greatly improving write performance and significantly extending the lifespan of the SSD. Furthermore, through array-level garbage collection, the DRAM capacity requirements of the SSD are reduced, lowering the cost of the storage array.

[0073] Where there is no conflict, the technical features in this embodiment and implementation scheme can be combined arbitrarily.

[0074] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes multiple computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated.

[0075] When implemented through hardware or firmware, the aforementioned method flow is programmed into the hardware circuit to obtain the corresponding hardware circuit structure and achieve the corresponding function. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit, whose logic function is determined by the user programming the device. Designers can program a digital system onto a PLD themselves, eliminating the need for chip manufacturers to design and fabricate dedicated integrated circuit chips. Furthermore, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, similar to the software compiler used in program development. The original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There is not just one HDL, but many. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of the aforementioned hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logic method flow can be easily obtained.

[0076] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. A storage array control method, applied to a storage array controller, wherein the storage array controller is electrically connected to a storage array and communicatively connected to a host, characterized in that, include: The storage array is divided into multiple storage areas, and multiple stripes are configured in each storage area, wherein sequential writes are performed in the storage area; Upon receiving the host’s first write instruction for the first target address, the mapping table is queried to determine the first entity address in the first stripe within the first memory area; Write the first write data of the first write instruction to the second entity address in the second stripe within the second memory area; Update the mapping address of the first target address to the second entity address in the mapping table.

2. The storage array control method as described in claim 1, characterized in that, The storage array controller also establishes and maintains a storage status table and a valid data technology table. After updating the mapping address of the first target address to the second entity address in the mapping table, the method further includes: Mark the first entity address as invalid in the storage status table; Update the valid data count of the first storage area in the valid data count table.

3. The storage array control method as described in claim 1, characterized in that, Writing the first data to the second address in the second stripe pointed to by the write pointer includes: When the data block of the second stripe is full, the check block of the second stripe is calculated based on the first written data and the remaining written data in the second stripe; Write the second stripe into the storage array.

4. The storage array control method as described in claim 1, characterized in that, Writing the first write data of the first write instruction to the second entity address in the second stripe within the second memory area includes: The first write data is written to the second address in the second stripe pointed to by the write pointer in the second storage area, and the position of the write pointer is updated according to the length of the first write data.

5. The storage array control method as described in claim 2, characterized in that, The method further includes: If the percentage of valid data in the third storage area is lower than a preset threshold, garbage collection is performed in the third storage area. Waste recycling includes: The entity addresses marked as valid are determined based on the storage status table of the third storage area; Read valid data from the valid entity address; A second write request is generated to write the valid data to the fourth storage area.

6. The storage array control method as described in claim 5, characterized in that, After generating a second write request to write the valid data to the fourth storage area, the method further includes: Update the mapping relationship of the valid data in the mapping table; The valid count of the third storage area is cleared to zero in the valid count table. Reset the storage status table of the third storage area.

7. A storage array controller, electrically connected to a storage array and communicatively connected to a host, characterized in that, The storage array controller includes: NVME-oF controller, memory area manager, redundancy management unit, and PCIe interface control unit; wherein the memory area manager is configured to: The storage array is divided into multiple storage areas, and multiple stripes are configured in each storage area, wherein sequential writes are performed in the storage area; Upon receiving the host’s first write instruction for the first target address, the mapping table is queried to determine the first entity address in the first stripe within the first memory area; Write the first write data of the first write instruction to the second entity address in the second stripe within the second memory area; Update the mapping address of the first target address to the second entity address in the mapping table.

8. The storage array controller of claim 7, wherein the storage area manager comprises: The sequential write conversion module is used to convert random write commands from the host into sequential write commands within the storage area; The address translation and mapping module is used to create and manage mapping tables, store status tables and valid count tables; in: The mapping table is used to record the mapping relationship between the target address of the host and the physical address of the storage array; The storage status table is used to record the valid status of the stored data; The valid count table is used to record the number of currently valid data in each storage area; The garbage collection module is used to implement storage area-based garbage collection in the storage array.

9. The storage array controller of claim 8, wherein the redundancy manager is used for: Once the data blocks of any strip are full, calculate the check block for that strip based on the data blocks; as well as, If any data block in the strip is read incorrectly, error correction is performed based on the remaining data blocks and the check block.

10. A storage device communicatively connected to a host computer, characterized in that, Includes the storage array controller and the storage array; The storage array controller is used to implement the storage array control method as described in any one of claims 1-6 on the storage array.

Images