Storage system and data storage method

Abstract

Disclosed in embodiments of the present application are a storage system and a data storage method, which are used for reducing metadata management complexity. The storage system comprises: a plurality of flash disks, wherein at least one of the plurality of flash disks is a hybrid flash disk, a storage space of the hybrid flash disk comprises a first space and a second space, the first space comprises a plurality of erase blocks, and the second space comprises a plurality of zones; a virtualization operation is performed on the plurality of flash disks to form a storage pool, the storage pool comprising a plurality of logic blocks; a redundant array of independent disks (RAID) group is created on the basis of the storage pool, the RAID group comprising the plurality of logic blocks, the capacity of all the logic blocks being the same, wherein the storage space of a first logic block among the plurality of logic blocks at least comes from the first space and the second space, the first space provides a storage space for the first logic block at a page-level granularity, and the second space provides a storage space for the first logic block at a zone-level granularity.

Description

A storage system and a data storage method

[0001] This application claims priority to Chinese Patent Application No. 202410650048.0, filed on May 23, 2024, entitled "A Storage System and a Data Storage Method", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of storage, and more particularly to a storage system and a data storage method. Background Technology

[0003] Solid-state drives (SSDs) typically use expensive dynamic random access memory (DRAM) to cache flash translation layer (FTL) mapping tables and require a significant amount of storage space for garbage collection (GC). To reduce these overheads, SSDs with a zoned namespace (ZNS) interface have emerged. Referring to Figure 1, in a ZNS interface SSD (ZNS SSD), the storage space is divided into multiple zones, and each zone contains one or more blocks. Therefore, its mapping granularity is much larger than that of a page.

[0004] In storage systems (refer to Figure 2), when using ZNS SSDs to form a Redundant Array of Independent Disks (RAID), storage space is typically provided to the RAID group at the zone level. Since the granularity of a zone is larger than that of the original page level, if the RAID group contains multiple types of ZNS SSDs, and the zone sizes contained in different types of zone SSDs are inconsistent, it is difficult to ensure that the capacity of different logical blocks within the RAID group is consistent.

[0005] When the capacity of logical blocks within a RAID group is inconsistent, it is necessary to record and manage the capacity of multiple logical blocks simultaneously, which leads to complex metadata management. Summary of the Invention

[0006] This application provides a storage system and a data storage method to reduce the complexity of metadata management.

[0007] Firstly, this application provides a storage system including multiple flash drives, at least one of which is a hybrid flash drive. It may also include other types of flash drives, such as block-level or partition-level flash drives. The multiple flash drives are virtualized to form a storage pool, which includes several logical blocks. Each logical block comes from partitions and erase blocks provided by different flash drives. A RAID group is constructed based on the storage pool, and the RAID group includes multiple logical blocks. The storage system includes a hybrid flash drive 1, which comprises a first space and a second space. The first space provides storage space to the logical blocks at the page level, while the second space provides storage space to the logical blocks at the partition level. That is, the granularity of the storage space provided by the first and second spaces to the logical blocks is different. In this case, logical block 1 can be constructed from the first and second spaces of the hybrid flash drive 1, making the capacity of logical block 1 the same as the capacity of the other logical blocks. This same capacity is not absolute; as long as the difference is within an acceptable range, it is acceptable. Thus, for this RAID group, only one logical block capacity needs to be recorded, thereby simplifying the complexity of metadata management. Furthermore, when the logical blocks have the same size, the data stripes based on the logical blocks include the same number of data stripe units. This means only one data stripe type needs to be recorded, further simplifying metadata management complexity. Simultaneously, since the data stripes include the same number of data stripe units, wasted storage space can be avoided.

[0008] In one possible implementation, when multiple hybrid flash drives are all hybrid flash drives, the partition sizes of the second space included in some of the hybrid flash drives are different. This allows logical blocks to be constructed jointly by the first and second spaces included in the hybrid flash drives, ensuring that the logical block capacity in a RAID group is the same, thereby simplifying metadata management. It should be noted that the different partition sizes of the second space included in each of the multiple hybrid flash drives can mean that all the partition sizes of the second space included in the hybrid flash drives are different, or that some of the hybrid flash drives include partition sizes of different second spaces, while others include partition sizes of the same second space.

[0009] In one possible implementation, the storage space of logical block 2 in multiple logical blocks comes from hybrid flash drive 2, and the granularity of the storage space provided by hybrid flash drive 2 to logical block 2 is partitioning. The storage space of logical block 3 in multiple logical blocks comes from hybrid flash drive 3, and the granularity of the storage space provided by hybrid flash drive 3 to logical block 3 is page.

[0010] In one possible implementation, space 1 is configured with a portion of the flash memory chips included in the hybrid flash drive, and space 2 is configured with a portion of the flash memory chips included in flash drive 1. This allows the capacity of spaces 1 and 2 to be flexibly and dynamically adjusted according to actual needs. In other implementations, the storage medium of the hybrid flash drive can include two types, meaning the storage medium in space 1 is different from the storage medium included in space 2.

[0011] In one possible implementation, the hybrid flash drive includes interface 1, interface 2, controller 1, and controller 2. Interface 1 is connected to controller 1 and is used to receive read / write requests / instructions to read and write data from space 1. Interface 2 is connected to controller 2 and is used to receive read / write requests / instructions to read and write data from space 2. In this implementation, space 1 and space 2 are equivalent to two types of disks; space 1 corresponds to a Block SSD, and space 2 corresponds to a ZNS SSD.

[0012] In one possible implementation, the hybrid flash drive includes an interface 3 and a controller 3. The interface 3 is connected to the controller 3 and is used to receive read / write requests / instructions to read and write data from space 3. In this implementation, the controller of the hybrid flash drive determines which space (space 1 or space 2) is used to read or write data. Both of the above implementations are acceptable and are not specifically limited in this application.

[0013] In one possible implementation, the storage space of each logical block in the multiple logical blocks comes from different flash drives. This ensures that if one or two flash drives used to build the RAID group fail, the data can be reconstructed using the data stored on the other flash drives, thereby improving the reliability of the storage system.

[0014] In one possible implementation, the RAID group is located in the disk enclosure, which also includes a control unit. The control unit constructs the RAID group and, after constructing the RAID group, establishes a mapping relationship between logical block 0 and hybrid flash disk 1, as well as between space 1 and space 2, for subsequent use when writing data to and reading data from hybrid flash disk 1.

[0015] In one possible implementation, the control unit is also used to receive write requests and, based on the mapping relationship established after the RAID group is constructed, determine the hybrid flash drive 1 and space 1 and space 2 corresponding to the logical address carrying data in the write request, and then store the data in the pages of space 1 including the erase block and the partitions included in space 2.

[0016] In one possible implementation, the control unit is further configured to send a write instruction to the hybrid flash drive 1 before storing the data to the hybrid flash drive 1. The write instruction carries a logical block address 1 and the length of data 1, so that the hybrid flash drive determines, based on the length of data 1, to store data 1 in a page of space 1 including an erase block and in a partition of space 2.

[0017] In one possible implementation, the control unit is further configured to send a write instruction 1 to the hybrid flash drive 1 before storing the data, the write instruction 1 carrying a logical block address 1 and the length of data 1, so that the hybrid flash drive 1 determines the erase block that can be written to based on the length of data 1, writes data 1 into the pages included in the determined erase block, and establishes a mapping relationship between the pages and the logical block address 1; and sends a write instruction 2 to the hybrid flash drive 1, the write instruction 2 carrying a logical block address 2 and the length of data 2, so that the flash drive 2 determines the partition that can be written to based on the length of data 2, and establishes a mapping relationship between the partition identifier, the offset within the partition, and the logical block address 2 after writing data 2 into the partition.

[0018] Secondly, this application also provides a data storage method applied to a storage system, the storage system including a Redundant Array of Independent Disks (RAID) group, the RAID group including multiple logical blocks, the multiple logical blocks having the same capacity, the method including: obtaining a write request, the write request carrying data; storing the data into space 1 and space 2 included in logical block 0 of the multiple logical blocks, space 1 providing storage space for logical block 0 at the page level, and space 2 providing storage space for logical block 0 at the partition level.

[0019] In one possible implementation, the data has a logical block address, which is the address of logical block 0. Storing the data into spaces 1 and 2 of the hybrid flash drive 1 corresponding to logical block 0 in multiple logical blocks includes: determining the hybrid flash drive 1, space 1, and space 2 corresponding to the logical block address based on the mapping relationship between the logical block address and the hybrid flash drive 1, as well as space 1 and space 2; and storing the data into partition 1 of space 2 and pages included in erase block 1 of space 1.

[0020] In one possible implementation, the method further includes sending a write instruction to the hybrid flash drive 1 before storing the data in partition 1 of space 2 and the pages included in erase block 1 of space 1, respectively. The write instruction carries a logical block address and the length of the data.

[0021] In one possible implementation, the method further includes: before storing the data into partition 1 of space 2 and the pages included in erase block 1 of space 1, respectively, sending a write instruction 1 to the hybrid flash drive 1, the write instruction 1 carrying a logical block address 1 and the length of data 1; sending a write instruction 2 to the hybrid flash drive 1, the write instruction 2 carrying a logical block address 2 and the length of data 2; wherein the data includes data 1 and data 2, and the logical block address includes logical block address 1 and logical block address 2.

[0022] Thirdly, this application also provides a hybrid flash drive, comprising: a first space and a second space; the first space is configured with a first portion of flash memory chips included in the hybrid flash drive, and the first space includes a plurality of erase blocks; the second space is configured with a second portion of flash memory chips included in the hybrid flash drive, and the second space includes a plurality of partitions.

[0023] In one possible implementation, the hybrid flash drive includes a first interface, a second interface, a first controller, and a second controller; the first interface is connected to the first controller and serves as an input / output interface for the first space; the second interface is connected to the second controller and serves as an input / output interface for the second space.

[0024] In one possible implementation, the hybrid flash drive includes a third interface and a third controller, the third interface and the third controller being connected, the third interface being a shared port for the first space and the second space.

[0025] Fourthly, this application also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the data storage method as described in the second aspect above.

[0026] Fifthly, this application also provides a computer program product that, when run on a computer, causes the computer to perform the data storage method as described in the second aspect above. Attached Figure Description

[0027] Figure 1 is a schematic diagram of the presentation of ZNS SSD on the host side in the prior art;

[0028] Figure 2 is a schematic diagram of building a RAID group using different types of ZNS SSDs;

[0029] Figure 3 is a schematic diagram of the presentation of an HDD supporting the Block interface protocol on the host side in the prior art;

[0030] Figure 4 is a schematic diagram of the presentation of an SSD supporting the Block interface protocol on the host side in the prior art;

[0031] Figure 5 is a schematic diagram of data striping of the RAID group constructed based on Figure 2;

[0032] Figure 6 illustrates the storage space wastage that occurs when data striping is performed on the RAID group constructed based on Figure 2.

[0033] Figure 7 is a schematic diagram of the structure of a storage system provided in this application;

[0034] Figures 8A and 8B are schematic diagrams of the hardware structure of an SSD provided in this application;

[0035] Figure 9 is a schematic diagram of the internal storage structure of a flash memory chip included in an SSD;

[0036] Figure 10 is a schematic diagram of constructing a RAID group provided in this application;

[0037] Figure 11 is a schematic diagram of another storage system provided in this application;

[0038] Figure 12 is a schematic diagram of another storage system provided in this application;

[0039] Figure 13 is a schematic diagram of another storage system provided in this application;

[0040] Figure 14 is a schematic diagram of another storage system provided in this application;

[0041] Figure 15 is a schematic diagram of another storage system provided in this application;

[0042] Figure 16 is a schematic diagram of another storage system provided in this application;

[0043] Figure 17 is a schematic diagram of another storage system provided in this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the specific embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0045] To better understand the storage system and the RAID group constructed by the storage system provided in the embodiments of this application, these interface protocols will be further described below.

[0046] Figure 3 is an abstract connection diagram between a hard disk drive (HDD) supporting the Block interface protocol and the host device. On the host side, an HDD supporting the Block interface protocol appears as a one-dimensional array of the entire LBA (Logical Block Address) within a namespace. This allows the host to read, write, and overwrite data in any order without considering the underlying physical implementation, thus simplifying storage management in the host software. For HDDs supporting the Block interface protocol, there is a static mapping between physical block addresses and logical block addresses.

[0047] For SSDs, read and write operations are typically performed at the page level, while erasure is performed at the block level. The granularity of read / write and erase operations within the disk is not consistent. When data needs to be rewritten, a blank page needs to be found, meaning the updated data is written to a blank page located elsewhere, while the data in the original page is marked as garbage. In other words, SSDs employ a remote data update strategy, which means the mapping between logical block addresses seen by the host and physical block addresses within the disk is not static but dynamically changing. See Figure 4. To be compatible with the Block interface protocol, an FTL layer is added to the SSD. To support the FTL layer, the SSD needs to add expensive DRAM to cache the FTL mapping table. The larger the SSD capacity, the larger the mapping table, and the larger the required DRAM capacity. Furthermore, SSDs typically also need to include over-provisioning (OP) space for garbage collection. OP space refers to the capacity that users cannot operate on; its size is the actual capacity of the SSD minus the user-available capacity. Therefore, the above operations not only cause significant performance fluctuations and write amplification in SSDs, but also require large-capacity DRAM caches and OPs, which will significantly increase hardware costs.

[0048] To reduce the high overhead associated with Block interface protocol compatibility, the LBA seen by the host in ZNS SSDs is divided into multiple partitions. Within each partition, writes are generally sequential; random writes and in-place updates are not allowed. If data updates are needed, the entire partition must be reset before writing can begin again from the beginning. In this case, the address mapping table maintained by this storage device is at the partition level, while the address mapping table maintained by storage devices supporting the Block interface protocol is at the page level. The partition granularity is much larger than the page granularity, resulting in less metadata and reduced DRAM usage. Furthermore, as shown in Figure 2, the storage space of the ZNS SSD is divided into multiple physical partitions, while the LBA seen by the host is divided into multiple logical partitions. The ZNS SSD aligns the granular boundaries of physical and logical partitions, achieving a static one-to-one mapping between them. This allows the ZNS SSD to transfer all data management responsibility to the host, with garbage collection entirely handled by the host. In other words, when reclaiming a logical partition, the host side is responsible for moving the valid data in the logical partition to another blank logical partition. Since the granularity of the logical partition and the physical partition is the same, the SSD side only needs to directly erase the physical partition that needs to be reclaimed, without needing to perform garbage collection within the SSD. Therefore, there is no need to reserve capacity space, and the computing performance requirements of the processor within the SSD will also be reduced.

[0049] While ZNS SSDs overcome some of the drawbacks of Block SSDs—such as requiring only a coarse-grained address mapping table for each partition, resulting in minimal DRAM requirements—and the fact that partition resets invalidate all blocks within a partition, eliminating the need for garbage collection and thus reducing performance fluctuations and excess provisioning space, they still have shortcomings in other applications. For instance, certain issues may arise when building RAID arrays using different types of ZNS SSDs.

[0050] When different types of ZNS SSDs are used to build a RAID group, meaning the partition space sizes of different ZNS SSDs are different, the logical block capacities included in the constructed RAID group may be inconsistent. For example, some logical blocks in a RAID group may have large capacities, while others may have small capacities. It's important to note that as SSDs evolve towards larger drives (SSDs with storage capacities greater than 30TB are generally referred to as large SSDs), SSD capacity increases typically occur in two ways: either increasing the number of pages within a single block, or keeping the number of pages within a single block the same but increasing the capacity of the pages themselves. In this evolutionary process, two different types of SSDs may exist: ordinary drives and large drives. The storage capacity of blocks in ordinary drives and large drives will differ. This difference in block storage capacity further leads to differences in the zone space size based on the block, resulting in the aforementioned different types of ZNS SSDs.

[0051] Figure 2 illustrates the case where logical block capacities are inconsistent. When logical block capacities are inconsistent, a RAID group needs to record the capacities of multiple logical blocks. Further, based on Figure 2 and referring to Figure 5, when a RAID group is divided into multiple data stripes, there are two types of data stripes. The first type uses four data stripes (D1, D2, D3, D4) to calculate the parity shards P and Q (4+2 mode). The second type uses two data stripes (D10, D11) to calculate the parity shards P and Q (2+2 mode). Compared to a single RAID mode, when a RAID group includes more than one type of data stripe, it is necessary to record and manage the lengths of multiple data stripes. Furthermore, different data stripe lengths result in different storage locations for the parity shards, further increasing the complexity of metadata management. The more ZNS SSD types and data stripe types within a RAID group, the greater the management difficulty.

[0052] On the other hand, when there are many types of ZNS SSDs, unusable storage space can easily occur, resulting in wasted storage space. For example, as shown in Figure 6, if there is a ZNS SSD with a partition granularity much larger than other member disks in the flash drive used to build the RAID group, it cannot support the minimum 2+2 mode. That is, if there are no other ZNS SSDs with the same partition granularity to form a data stripe with it, then the logical block built by the partition using that ZNS SSD may have wasted space.

[0053] To address the aforementioned technical problems, please refer to Figure 7. This application embodiment provides a storage system 700, which includes multiple flash drives 701. At least one of the flash drives 701 is a hybrid flash drive. The storage space of the hybrid flash drive includes spaces 7011 and 7012. Space 7011 includes multiple erase blocks 7012, and space 7012 includes multiple partitions. In a specific implementation, the multiple flash drives 701 can be virtualized to form a storage pool. The storage pool includes several logical blocks, and the storage space of these logical blocks comes from zones and erase blocks provided by the multiple flash drives. After constructing the storage pool, a RAID group 702 can be built based on the storage pool. The RAID group 702 includes multiple logical blocks, and the storage space of logical block 7021 among these logical blocks comes at least from spaces 7011 and 7012. Space 7011 provides storage space to logical block 7021 at a first granularity, and space 7012 provides storage space to logical block 7021 at a second granularity. In this embodiment, a first logical block is constructed using two storage spaces with different granularities, so that the capacity of the constructed first logical block is the same as the capacity of other logical blocks in the multiple logical blocks. Thus, only the capacity of one type of logical block needs to be recorded in such a RAID group, thereby reducing the complexity of metadata management.

[0054] In this embodiment, the hardware structure of the hybrid flash drive includes two forms, which are described below. Referring to Figure 8A, the SSD includes a main controller 801, a flash array 802 composed of multiple flash memory chips, and a cache 803. The main controller 801 is connected to the host, the flash array 802, and the cache 803. The main controller 801 is responsible for complex tasks such as managing data storage and maintaining SSD performance and lifespan. For example, the main controller 801 receives access commands from the host, parses the commands, converts them into commands that allow direct access to the flash array 802, sends them to the flash array 802, obtains the access results, and returns the results to the host. The main controller 801 is typically presented as an application-specific integrated circuit (ASIC), but can also be implemented based on a field-programmable gate array (FPGA) or a central processing unit (CPU). In practical applications, considering factors such as cost, performance, and power consumption, the controller is usually made into an ASIC chip. The cache 803 is optional in the SSD and is typically implemented using dynamic random access memory (DRAM). It is used to store various data generated during operation, which helps improve the controller's response speed to inference device commands. The flash memory array 802 includes some flash memory chips configured as space 7011, and some flash memory chips configured as space 7012. The main controller 801 also includes an interface 804 and several channel controllers. The interface is used for communication with the host. Through the several channel controllers, the main controller 801 can operate multiple flash memory chips in parallel, thereby improving the underlying bandwidth. In this implementation, there is only one interface 804, meaning that interface 804 and the main controller 801 are shared by spaces 7011 and 7012. In other words, access to spaces 7011 and 7012 is all through interface 804 and the main controller 801.

[0055] In some possible implementations, as shown in Figure 8B, there are two main controllers 801, represented as main controller 801A and main controller 801B. There are also two interfaces 804, represented as interface 804A and interface 804B. Interface 804A is connected to main controller 801A and serves as the input / output interface for space 7011. Interface 804B is connected to main controller 801B and serves as the input / output space for space 7012. The first interface can be any of the following: Serial Advanced Technology Attachment (SATA) interface, Serial Attached SCSI (SAS) interface, or Peripheral Component Interconnect Express (PCIe) interface. The second interface can be a Non-Volatile Memory Express (NVMe) interface, or other interfaces that support the ZNS instruction set. The ZNS instruction set is a set of storage device-specific instructions used to manage the region namespace in a ZNS SSD.

[0056] In the above description, SSDs supporting the Block interface protocol are referred to as Block SSDs, and SSDs supporting the ZNS interface protocol are referred to as ZNS SSDs. In this application, flash drives that support both the Block and ZNS interface protocols are referred to as hybrid flash drives, denoted as ZB SSDs. The storage system 700 shown in Figure 7 may include not only hybrid flash drives but also other types of flash drives, such as the aforementioned Block SSDs or ZNS SSDs.

[0057] Flash array 802 is used to store various types of data. Specifically, flash array 802 may include one or more flash memory dies, each of which is typically presented as a chip. The specific type of flash memory is NAND flash memory. It should be noted that although flash memory is used as an example here, other types of non-volatile memory can also be used, such as phase change memory (PCM) and resistive random access memory (RRAM), without affecting the technical solution of this application. The distribution of the storage space of a flash memory die can be seen in Figure 9. A flash memory die is divided into two regions (Plane). A plane contains multiple blocks, and a block consists of several pages. For example, every 128 pages make up a block, and every 2048 blocks make up a plane. The address of a page is called the Physical Block Address (PBA). Depending on the manufacturer and manufacturing process, typical page values ​​are 4 kilobytes (KB), 8 KB, etc., and typical block values ​​are 8 megabytes (MB), 16 MB, etc. As the demand for large-capacity storage increases, the storage capacity of storage devices continues to grow, and the value of a block can be hundreds of megabytes.

[0058] The granularity at which spaces 7011 and 7012 provide storage space to logical block 7021 is illustrated below with reference to Figure 9. Space 7011 comprises multiple erase blocks. Taking the Block shown in Figure 9 as an example, a Block is 8MB in size, contains 1000 pages, and a page is 8KB in size. Space 7011 provides storage space to logical block 7021 at the page level. Space 7012 of the hybrid flash drive is divided into several partitions. Each partition contains one or more erase blocks. Taking the Block shown in Figure 9 as an example, a Block is 8MB in size, and a partition contains 10 Blocks, with a total size of 80MB. The first space provides storage space to logical block 7021 at the partition level. In some possible implementations, space 7012 is divided into several PLOGs (Persistence Layer LOGs), and each PLOG contains one or more erase blocks.

[0059] It should be noted here that since space 7011 provides storage space to logical block 7021 at the page level, space 7011 should also include reserved space OP for garbage collection during subsequent use.

[0060] After introducing the granular concept of how space 7011 and space 7012 of a hybrid flash drive provide storage space to logical block 7021, we will further introduce the concept of logical blocks included in a RAID group in conjunction with the RAID group construction process. Assuming the required RAID type is RAID6, the EC redundancy ratio is 4+2, and the size of the logical block to be built is 5MB, then based on the above information, we know that six logical blocks with a capacity of 5MB each need to be built. Here, these six logical blocks are represented as logical block 0 to logical block 5, and logical block 0 is logical block 7021. Please refer to Figure 10. The storage system 700 includes hybrid flash drives, specifically ZB SSD1-ZB SSD6. The granularity of the storage space 7011 provided to logical blocks 0-5 in ZB SSD1-ZB SSD6 differs. This difference could be that the granularity of the storage space 7012 provided to the logical blocks in ZB SSD1-ZB SSD6 is different for all of them. The granularity of the storage space 7012 provided to the logical blocks in ZB SSD1-ZB SSD6 can be partially the same and partially different. For example, the granularity of the storage space 7012 provided to the logical blocks in ZB SSD1-ZB SSD3 is the same (80MB), and the granularity of the storage space 7012 provided to the logical blocks in ZB SSD3-ZB SSD6 is the same (120MB), but different from the 80MB granularity of the storage space 7012 provided to the logical blocks in ZB SSD1-ZB SSD3. The granularity of the storage space 7012 provided to the logical blocks in ZB SSD1-ZB SSD6 is different for all of them. The granularity of the storage space provided by the first space 7011 of SSD6 to the logical block can be the same or different. Among them, when the granularity of the storage space provided by the first space 7011 to the logical block of ZB SSD1-ZB SSD6 is different, it can be partially the same and partially different.

[0061] Before building the RAID group, the storage space of each ZB SSD is first partitioned. The space 7012 within each ZB SSD is divided into several partitions. These partitions within the ZB SSD's space 7012 are called physical partitions. The physical partitions within a ZB SSD's space 7012 are all of the same size. For example, the physical partition size of ZB SSD1 is 4MB, ZB SSD2 is 2MB, ZB SSD3 is 1.5MB, ZB SSD4 is 1MB, ZB SSD5 is 1MB, and ZB SSD6 is 1MB. The first space of ZB SSD1 through ZB SSD6 includes multiple erase blocks, called physical erase blocks. Each physical erase block contains multiple pages.

[0062] After determining the physical partitions of ZB SSD1-ZB SSD6, these physical partitions are then mapped to logical partitions, and their physical erase blocks are mapped to logical erase blocks. The logical partitions and logical erase blocks of ZB SSD1-ZB SSD6 then constitute a storage pool, which provides storage space upwards. In the specific implementation, logical blocks can be constructed based on the storage pool. Therefore, in this embodiment, the logical blocks are constructed from the mapped logical partitions and / or logical erase blocks. Taking logical block 0 as an example, firstly, logical partition 01 is retrieved from the storage pool. Logical partition 01 is 4MB in size, which cannot provide 5MB of storage space. If another logical partition of the same size is retrieved, the total size of the two logical partitions would be 8MB, exceeding 5MB. In this case, 128 pages from logical erase block 01 can be retrieved from the storage pool. Each page is 8KB in size. Thus, the 128 pages from logical erase block 01 can be combined with logical partition 01 to form a 5MB logical block 0. Here, logical partition 01 corresponds to physical partition 01 in ZB SSD1, and logical erase block 01 corresponds to ZB... In SSD1, there is a physical erase block 01. For logical block 1, logical partitions 11 and 12 can be taken from the storage pool. Logical partitions 11 and 12 are both 2MB in size, totaling 4MB. If another logical partition of the same size is taken, the total would be 6MB, exceeding 5MB. In this case, 128 pages from logical erase block 11 can be taken from the storage pool. Each page is 8KB. Thus, logical erase block 11, logical partitions 11 and 12 can construct a 5MB logical block 1. Logical partitions 11 and 12 correspond to physical partitions 11 and 12 in ZB SSD2, respectively. Logical erase block 11 corresponds to ZB... One physical erase block in SSD2; for logical block 2, logical partitions 21, 22, and 23, each 1.5MB in size, can be taken from the storage pool, totaling 4.5MB. Additionally, 64 pages (8KB each) need to be taken from logical erase block 21. Thus, logical erase block 21, along with logical partitions 21, 22, and 23, can form a 5MB logical block. The remaining logical blocks can be constructed in this way, ensuring that all logical blocks have the same size. "Significant size" means the logical blocks are roughly the same size, with acceptable margins of error, not necessarily absolutely identical.

[0063] After construction is complete, each logical block in the logical block group is assigned a set of logical block addresses. This set of logical block addresses constitutes the logical block addresses of the completed logical block group. As can be seen from the logical block construction process described above, the mapping relationship between logical blocks and physical partitions and / or physical erase blocks is not a simple one. Taking logical block 0 as an example, the storage space of logical block 0 consists of physical partitions and physical erase blocks on ZB SSD1. That is, logical block 0 corresponds to physical partition 01 and physical erase block 01 on ZB SSD1. Therefore, it is necessary to record the mapping relationship between the logical block addresses of logical block 0 and ZB SSD1, as well as spaces 7011 and 7012, for subsequent data reading. Similarly, the mapping relationship between the logical block addresses of other logical blocks and other ZB SSDs, as well as spaces 7011 and 7012 on other ZB SSDs, also needs to be recorded for subsequent data reading. This mapping relationship can be found in Table 1 below. It should be noted that, in Figure 10, for illustrative purposes, there is a one-to-one correspondence between logical partitions and physical partitions, and between logical erase blocks and physical erase blocks. However, when actually writing data to a ZB SSD, it is not necessary to write data to the physical erase block corresponding to the logical partition according to the mapping relationship shown in the figure. As long as there is a physical partition on the space 7012 corresponding to the logical partition that can be written to, it is acceptable. The same principle applies to the first space of the ZB SSD.

[0064] Table 1

[0065] Regarding Table 1 above, it should be noted that when the hybrid flash drive includes two interfaces and two controllers, it is necessary to establish a mapping relationship between the logical block address and the hybrid flash drive, as well as between space 7011 and space 7012. However, when the hybrid flash drive includes one interface and one controller, it is sufficient to establish a mapping relationship between the logical block address and the hybrid flash drive.

[0066] The above example of constructing a RAID group is merely an illustration. In actual implementation, other methods can be used to construct it, as long as the capacity of the constructed logical blocks is consistent. This application embodiment does not impose any limitations. The RAID group constructed above is the smallest allocation unit of the storage pool. When the storage service layer requests storage space from the storage pool, the storage pool can provide one or more logical block groups to the storage service layer. Simultaneously, the RAID group includes multiple data stripes, and each data stripe includes multiple data stripe units. The storage space of a data stripe unit in one of the multiple data stripes comes from at least two different types of storage space in the hybrid flash drive, namely, space 7011 and space 7012 mentioned above. The storage space of data stripe units in other data stripes can come from one type of flash memory space in the hybrid flash drive, such as space 7011 or space 7012 of the hybrid flash drive.

[0067] In practice, within the same RAID group, a flash drive typically participates in the construction of only one logical block. This means that the storage space in each logical block comes from different flash drives. For example, referring to the RAID group construction process above, the storage space for logical block 0 comes from ZB SSD1, logical block 1 from ZB SSD2, logical block 2 from ZB SSD3, logical block 3 from ZB SSD4, logical block 4 from ZB SSD5, and logical block 5 from ZB SSD6. Each logical block's storage space comes from a different hybrid flash drive. On the other hand, in the embodiments of this application, the storage space of different logical blocks can be from a single source or from multiple sources. In other words, the storage space of logical block 0 comes from spaces 7011 and 7012 of the ZB SSD, while the storage space of logical blocks 3, 4, and 5 comes from space 7012 of the ZB SSD. In some possible implementations, the storage space of the logical blocks can also come from space 7011 of the ZB SSD.

[0068] In this embodiment, logical blocks 7021 are constructed by using two different granularities of storage space in a hybrid flash drive. This ensures that the capacity of the constructed logical block 7021 is the same as that of other logical blocks within the multiple logical blocks. Therefore, only one type of logical block capacity needs to be recorded in this RAID group, reducing metadata management complexity. Furthermore, the RAID group can include multiple data stripes. Since the multiple logical blocks in the RAID group have the same capacity, each data stripe in the RAID group contains the same number of data stripe units. Thus, only one type of data stripe length and one type of parity stripe storage location need to be recorded in this RAID group, further simplifying metadata management.

[0069] Furthermore, if the logical blocks in a RAID group have the same capacity, then logical blocks of the same capacity can ensure that the calculation mode is the same when calculating parity data, thus facilitating acceleration using software or hardware. Also, with logical blocks of the same capacity in a RAID group, each logical block can participate in data striping, thereby effectively utilizing the storage space of the logical blocks in the RAID group and avoiding space waste.

[0070] The process of building a RAID group has been described above. After the RAID group is built, the following section will introduce the specific usage process of the RAID group.

[0071] Please refer to Figure 11. The RAID group is located in disk enclosure 1100, which includes a hybrid flash drive 1101. The disk enclosure also includes:

[0072] Control unit 1102 is communicatively connected to hybrid flash drive 1101. Control unit 1102 is used to execute the RAID group construction shown in Figure 10. After constructing the RAID group, it is also necessary to establish the mapping relationship between logical block 0 and hybrid flash drive 1101, as well as spaces 7011 and 7012, as detailed in Table 1. In the specific implementation process, control unit 1101 determines the source of storage space for each logical block to be constructed based on the information obtained about the logical blocks to be constructed, such as the size of the logical blocks to be constructed being 5M, the number of logical blocks to be constructed being 6, and its own record of the available storage space owned by each hybrid flash drive 1101. To ensure that there is always sufficient available space in the storage system 700 for creating the RAID group, control unit 1102 can monitor the available storage space of each hybrid flash drive in real time, thereby obtaining the available storage space of the entire storage system 700.

[0073] The control unit 1102 can have various forms:

[0074] Format 1: Typically, the control unit 1102 includes a central processing unit (CPU) and memory. The CPU is used to perform address translation and read / write data operations. The memory is used to temporarily store data to be written to the hybrid flash drive, or data to be read from the hybrid flash drive and sent to the host.

[0075] In this context, RAM refers to internal memory that directly exchanges data with the processor. It can read and write data at any time at high speed, serving as temporary data storage for the operating system or other running programs. RAM includes at least two types of memory, such as random access memory (RAM) or read-only memory (ROM). For example, RAM can be Dynamic Random Access Memory (DRAM) or Storage Class Memory (SCM). DRAM is a semiconductor memory, and like most RAM, it is a volatile memory device. SCM is a hybrid storage technology that combines the characteristics of traditional storage devices and RAM. SCM offers faster read and write speeds than hard drives, but slower access speeds than DRAM, and is also cheaper. However, DRAM and SCM are merely illustrative examples in this embodiment; RAM can also include other types of RAM, such as Static Random Access Memory (SRAM). For read-only memory, examples include programmable read-only memory (PROM) and erasable programmable read-only memory (EPROM). Additionally, the memory can also be a dual in-line memory module (DIMM), i.e., a module composed of dynamic random access memory (DRAM), or a solid-state drive (SSD). In practical applications, the control unit can be configured with multiple memory modules of different types. This embodiment does not limit the number or type of memory. Furthermore, the memory can be configured to have a power-saving function. A power-saving function means that when the system experiences a power outage and is then powered on again, the data stored in the memory will not be lost. Memory with a power-saving function is called non-volatile memory.

[0076] Type 2: The control unit 1102 is a programmable electronic component, such as a Data Processing Unit (DPU). The DPU possesses the versatility and programmability of a CPU, but is more specialized, capable of efficiently operating on network packets, storage requests, or analysis requests. The DPU differs from the CPU through a high degree of parallelism (handling a large number of requests). In some possible implementations, the DPU can also be replaced by a Graphics Processing Unit (GPU), an embedded Neural-Network Processing Unit (NPU), or other processing chips. Typically, the number of control units 1102 can be one, two, or more. When the storage system 700 contains at least two control units 1102, the two control units 1102 can serve as backups for each other, preventing the hybrid flash drives under that control unit 1102 from becoming unusable if one control unit 1102 fails. When the number of control units 1102 is two or more, there can be a hierarchical relationship between the hybrid flash drives and the control units 1102, meaning each control unit 1102 can only access the hybrid flash drives belonging to it.

[0077] Format 3: The functions of the control unit 1102 can be offloaded to the network interface card (NIC). In other words, in this implementation, the storage system 700 does not have a control unit; instead, the NIC performs data reading / writing, address translation, and other computational functions. In this case, the NIC is a smart NIC. It can include a CPU and memory. In some applications, the NIC may also have persistent memory media, such as persistent memory (PM), non-volatile random access memory (NVRAM), or phase-change memory (PCM). The CPU performs address translation and data reading / writing operations. There is no hierarchical relationship between the NIC and the hybrid flash drives in the storage system; the NIC can access any one of the multiple hybrid flash drives.

[0078] In some possible implementations, the storage device that provides storage space to the logical block at the first granularity can be a hybrid flash drive that supports the block interface protocol as described above. Of course, its storage medium can also be a storage class memory (SCM), a magnetic random access memory (MRAM), or an HDD, or any other hard drive that can support the block interface protocol. There are no restrictions here.

[0079] Furthermore, the control unit 1102 is also used to receive a write request, the request carrying data, the data having a first logical block address.

[0080] RAID group 701 may contain one or more stripes. The data fragments and parity fragments contained within a stripe can both be referred to as stripe units. In this embodiment, a stripe unit size of 1MB is used as an example, but it is not limited to 1MB. Continuing with the above example, the created logical block group includes logical blocks 0-5, where logical blocks 0, 1, 2, and 3 are data block groups, and logical blocks 4 and 5 are parity block groups. In the specific implementation process, if the received data cannot fill a data strip, the received data can be temporarily stored in memory. When the data stored in memory reaches a certain value, such as 8MB, the data is divided into two groups of data fragments. Each group includes four data fragments (group 1 includes data fragments 00, 01, 02, and 03; group 2 includes data fragments 10, 11, 12, and 13). The size of each data fragment is 1MB. Then, the check fragments for each group of data fragments are calculated. Two check fragments are calculated for each group (the check fragments for group 1 are P00 and Q00; the check fragments for group 2 are P10 and Q10). The size of each check fragment is also 1MB. The data in data fragments 00 and 10 are the data carried in the receiving request.

[0081] Before sending data to the hybrid flash drive, the control unit 1102 needs to determine whether there is an allocated logical block group. If so, and the logical block group still has enough space to accommodate the data, the control unit 1102 can instruct the hybrid flash drive to write the data into the allocated logical block group. In a specific implementation, taking data fragment 00 and data fragment 10 as examples, since the data carries the address of the first logical block, the control unit 1102 first determines the first logical block corresponding to the address based on the first logical block address (e.g., LBA200-LBA209). Based on the mapping relationship in Table 1 above, the control unit 1102 can determine the hybrid flash drive corresponding to logical block 0, as well as the first and second spaces of the hybrid flash drive. Then, data fragment 00 and data fragment 10 are stored respectively in the pages included in the first physical erase block of the first space of the hybrid flash drive and the first physical partition included in the second space. Here, the first physical erase block can be any physical erase block in the first space of the hybrid flash drive that can be written to, and the first physical partition can be any partition in the second space of the hybrid flash drive that can be written to.

[0082] Before the control unit 1102 writes data fragment 00 and data fragment 10 to the first physical erase block and the first physical partition respectively, the control unit 1102 also sends a write command to the hybrid flash drive 703. Depending on the interface settings of the hybrid flash drive, the way the command is sent may include, but is not limited to, the following two methods, which will be described below.

[0083] As mentioned in the description above, only sequential writes are supported in the write operation within the partition. The LBAs within a single partition are distributed continuously, and the write pointer (WP) always points to the next LAB position for sequential writing. In order to remember where this "sequential write" has been written to, if repeated writing is required within a single partition, a partition reset operation is required first.

[0084] Each partition has a series of states, which are used for decision-making. All partitions are in an Empty state before use. If writing is required, the partition must be changed from the Empty state to the Open state. Each partition has a capacity limit. Once the write volume reaches the partition's capacity limit, the partition will be in a Full state. If the ZNS SSD has a maximum number of partitions, and the number of Open partitions reaches the limit, and you want to open a new partition, one of these limited partitions needs to be switched to the Closed state. A Closed partition can still be written to, but it must first enter the Open state.

[0085] In Method 1, the control unit sends a write command to the hybrid flash drive. This command carries logical block addresses LBA200-LBA209 and the data length. Upon receiving this command, the hybrid flash drive's main controller, aware of its storage space partitioning, divides the data based on its length. This means determining whether all data is stored in the first space, all in the second space, or a combination of both. For example, assuming the data length is 1MB and a physical partition of the hybrid flash drive is 1MB, the controller can determine the remaining capacity of the physical partition based on the current write pointer position and the partition's size. If only 0.5MB of the physical partition remains writable, then 0.5MB of the data is stored in the second space partition, while the remaining 0.5MB is stored in a page within a physical erase block in the first space.

[0086] Method 2

[0087] The control unit sends a first write command to the hybrid flash drive through the first interface. The first write command carries logical block addresses LBA200-LBA204 (first sub-logical block addresses), data fragment 00, and the length of data fragment 00 (i.e., the first data). After ZNS SSD1 receives the first write command, ZNS SSD1 determines the first physical partition where data can be written based on the logical block addresses LBA200-LBA204, data fragment 00, and the data length of data fragment 00 carried in the first write command, and then writes data fragment 00 into the first physical partition.

[0088] After the first data is written, a mapping relationship is established between the logical block addresses of the logical partition and the physical block addresses of the physical partition. This mapping relationship is the correspondence between the logical block address and the physical partition identifier, as well as the offset within the physical partition. Taking logical block 0 as an example, a mapping relationship is established between the logical block addresses LBA200-LBA204 of logical block 0 (LBA200-LBA209) and the first physical partition 01 and the offset within the first physical partition 01. For details, please refer to Table 2 below.

[0089] Table 2

[0090] In this embodiment, the control unit 1102 also sends a second write command to the hybrid flash drive. The second write command carries logical block addresses LBA205-LBA209 (second sub-logical block addresses), data fragment 10 (second data), and the length of data fragment 10. Upon receiving the second write command, the hybrid flash drive determines the first physical erase block capable of data writing based on the length of data fragment 10 carried in the second write command. Then, it stores data fragment 10 in the pages included in the first physical erase block. After the second data writing is complete, the Block SSD1 needs to establish a mapping relationship between the physical block address of the page containing data fragment 10 and the logical block addresses LBA205-LBA209, as detailed in Table 3 below. This mapping relationship is added to (first write) or modified (overwrite write) the FTL. With such a mapping, whenever data is read, the SSD first looks up the PBA corresponding to the LBA of that data in the FTL, and then reads the corresponding data based on the PBA.

[0091] Table 3

[0092] The above describes the process of writing data to the flash drive; the following describes the process of reading data from the flash drive. After receiving a read request, the control unit 1102, based on the logical block addresses LBA200-LBA209 carried in the read request and the correspondence in Table 1 above, determines the second space of the hybrid flash drive corresponding to logical block addresses LBA200-LBA204, and sends a first read instruction to the hybrid flash drive. The first read instruction carries logical block addresses LBA200-LBA204. After receiving the first read instruction, the hybrid flash drive, based on the mapping relationship in Table 2 above, determines the physical partition identifier corresponding to logical block addresses LBA200-LBA204, and the start and end physical block addresses within the physical partition. Based on the physical partition identifier and the start and end physical addresses, it reads data with an offset corresponding to the length from the physical partition corresponding to the physical partition identifier and sends it to the host.

[0093] Correspondingly, the control unit 1102 also determines the first space of the hybrid flash drive corresponding to the logical block address LBA205-LBA209 based on the logical block address LBA205-LBA209, and then sends a second read instruction to the hybrid flash drive. The second read instruction carries the logical block address LBA205-LBA209. After receiving the second read instruction, the hybrid flash drive maps the logical block address LBA205-LBA209 carried in the second read instruction to a physical block address based on the mapping relationship in Table 3 above, and then reads the corresponding data based on the physical block address and sends it to the host.

[0094] Continuing with the example above, during the data reading process, if the data in data shard 00, data shard 01, data shard 03, and check shards 00 and 01 can all be read normally, but the flash drive containing the data in data shard 02 malfunctions and cannot be read normally, in this case, data shards 00, 01, 02, and 03 can be reconstructed from the data in data shards 00, 01, 03, and check shards 00 and 01. This ensures that the damaged data shards can be reconstructed, thereby significantly improving data reliability.

[0095] Users access data through applications, and the computers running these applications are typically referred to as "application servers" or "hosts." Therefore, as shown in Figure 12, the storage system illustrated in Figure 11 further includes hosts; in practice, there can be one or more hosts. Hosts can be physical machines or virtual machines. Hosts include, but are not limited to, desktop computers, servers, laptops, and mobile devices. Hosts access the storage system to retrieve data via fiber optic switches. However, the switch is only an optional device; application servers can also communicate directly with the storage system via a network. Here, "network" can refer to a Local Area Network (LAN), which can be implemented using various structures, devices, and protocols; for example, LAN structures can include Ethernet, wireless, etc. Data communication protocols used in a LAN can include Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Internet Protocol (IP), Hypertext Transfer Protocol (HTTP), Wireless Access Protocol (WAP), Handheld Device Transport Protocol (HDTP), Session Initiation Protocol (SIP), etc.; alternatively, fiber optic switches can be replaced with Ethernet switches, InfiniBand switches, RoCE (RDMA over Converged Ethernet) switches, etc. In the storage system shown in Figure 12, the RAID group construction process shown in Figure 10 can be completed by the host.

[0096] In some possible implementations, the RAID group is located in the disk enclosure, and the storage system also includes:

[0097] The controller communicates with the disk enclosure and is used to execute the construction of the RAID group. After the RAID group is constructed, the controller establishes the mapping relationship between the first logical block and the hybrid flash drive, as well as the first space and the second space.

[0098] Referring to Figure 13, the controller can be a component included in the engine. Taking an engine with two controllers as an example, controller 0 and controller 1 have a mirror channel. When controller 0 writes data to its memory, it can send a copy of the data to controller 1 via the mirror channel. Controller 1 then stores the copy in its local memory. Thus, controller 0 and controller 1 act as backups for each other. When controller 0 fails, controller 1 can take over its operations, and vice versa, preventing hardware failures from causing the entire storage system to become unavailable. When four controllers are deployed in the engine, any two controllers have a mirror channel, thus any two controllers act as backups for each other.

[0099] In terms of hardware, the controller includes at least a processor and memory. The processor is a CPU used to process data access requests from outside the storage system, as well as requests generated within the storage system. For example, when the processor receives write data requests from the host through the front-end port, it temporarily stores the data in these requests in memory. When the total amount of data in memory reaches a certain threshold, the processor sends the data stored in memory to the flash drive for persistent storage through the back-end port.

[0100] The engine also includes front-end and back-end interfaces. The front-end interface communicates with the host to provide storage services. The back-end interface communicates with the flash drives. In practice, these multiple flash drives can be in the form of hard drive enclosures, communicating with the engine through the back-end ports. The back-end interfaces exist within the engine as adapter cards, and an engine can simultaneously use two or more back-end interfaces to connect multiple hard drive enclosures. Alternatively, the adapter card can be integrated onto the motherboard, in which case it can communicate with the processor via the PCIe bus.

[0101] Depending on the communication protocol between the engine and the disk enclosure, the disk enclosure may be a Serial Attached SCSI (SAS) disk enclosure, a Non-Volatile Memory Express (NVMe) disk enclosure, an Internet Protocol (IP) disk enclosure, or other types of disk enclosures. SAS disk enclosures use the SAS 3.0 protocol, and each enclosure supports 25 SAS disks. The application server connects to the disk enclosure via an onboard SAS interface or a SAS interface module. NVMe disk enclosures are more like a complete computer system, with NVMe disks plugged into the NVMe disk enclosure. The NVMe disk enclosure then connects to the application server via a RAMA port.

[0102] In some possible implementations, the engine can have hard drive slots, and flash drives can be deployed directly on the engine. In this case, the engine can also have a back-end port, through which a hard drive enclosure can be connected when the flash drive space is insufficient, in order to achieve the purpose of expansion.

[0103] Furthermore, please refer to Figure 14. The storage system described in Figure 13 also includes a host, which is communicatively connected to the front-end port of the engine. In the storage system shown in Figure 14, the RAID group construction process shown in Figure 10 can be executed by the host.

[0104] In some possible implementations, based on the storage system shown in Figure 7, the storage system further includes one or more hosts. The hosts are described in Figure 12 and will not be repeated here. In this scenario, the communication connection methods between multiple flash drives (which can be block-level flash drives, partition-level flash drives, or hybrid flash drives as described above) and the host include, but are not limited to, the following methods, which are described below.

[0105] Method 1: Please refer to Figure 15. Multiple flash drives can be directly plugged into the slots of the host computer. The interface type of the slot can be Serial ATA (Serial Advanced Technology Attachment, SATA), SAS, PCIe, or NVMe, etc. Among them, the SAS interface adds SCSI technology to the SATA interface. This technology is mainly used to improve the stability and security of data transmission.

[0106] In some possible implementations, the flash drive can be placed in a server rack containing a hard drive tray. The tray can have a plastic or metal frame and include multiple hard drive slots—specifically, 4, 8, 16, or 32, or more. The tray includes an interface for communication with the host, such as via a fiber optic switch, Ethernet switch, InfiniBand switch, or RoCE (RDMA over Converged Ethernet) switch. Alternatively, communication with the host can be via a network, specifically a Local Area Network (LAN). LANs can be implemented using various architectures, devices, and protocols, such as Ethernet and wireless. Data communication protocols used in LANs can include Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Internet Protocol (IP), Hypertext Transfer Protocol (HTTP), Wireless Access Protocol (WAP), Handheld Device Transport Protocol (HDTP), and Session Initiation Protocol (SIP), among others. In practice, several hard drives can be located in the same rack or in different racks, distributed across various locations, and remotely connected to the host via a gateway or router.

[0107] In this implementation, the RAID group construction process can be performed by the host, or it can be offloaded to another dedicated computer device. This dedicated computer can be a device designed or programmed solely to perform the RAID group construction function, such as a DPU, NPU, or smart network interface card, etc. No specific limitations are imposed in this embodiment. The host's RAID group construction process has already been described in detail above and will not be repeated here. It should be further noted that when constructing a RAID group, a single flash drive can only participate in the construction of one logical block within the same RAID group. This ensures that if one or two flash drives fail, the original data can be recovered from the data stored on the remaining flash drives, thus preventing data loss.

[0108] After the RAID group is built, only one logical block in the RAID group needs to have its storage space composed of ZNS SSD storage space and Block SSD storage space. It is not required that the storage space of all logical blocks be multi-source, that is, all of them need to be composed of ZNS SSD storage space and Block SSD storage space.

[0109] When the second hard drive group communicates with the host via a disk enclosure, the disk enclosure may include one or more control units. A description of these control units can be found in the description of control unit 1102 shown in Figure 11, and will not be repeated here. It should be noted that in this scenario, the user primarily uses ZNS SSDs to build the RAID group, while Block SSDs are only used to fill in the logical block capacity and do not require particularly large capacities. To save costs and ensure the independence between hard drives, small-capacity, pluggable storage blocks can be used instead of ordinary Block SSDs.

[0110] In some possible implementations, the storage system shown in Figure 11 can also be applied to distributed scenarios. Referring to Figure 16, this scenario includes a cluster of compute nodes and a cluster of storage nodes. The compute node cluster includes one or more compute nodes that can communicate with each other. A compute node is a computing device, such as a server, desktop computer, or the controller of a storage array. In hardware, a compute node includes at least a processor, memory, and a network interface card (NIC). The processor is a central processing unit (CPU) used to process data access requests from outside the compute node or requests generated internally within the compute node. For example, when the processor receives a write data request from a user, it temporarily stores the data in the write data request in memory. When the total amount of data in memory reaches a certain threshold, the processor sends the data stored in memory to the storage node for persistent storage. In addition, the processor is also used for data computation or processing, such as metadata management, deduplication, data compression, virtualization of storage space, and address translation. In practical applications, there are often multiple CPUs, and each CPU has one or more CPU cores. This application does not limit the number of CPUs or the number of CPU cores. For an introduction to memory, please refer to the memory description shown in Figure 9, which will not be described in detail here.

[0111] Any compute node can access any storage node in the storage node cluster via the network. The storage node cluster consists of multiple storage nodes, which can be the storage system shown in Figure 11. In a distributed scenario, the flash drives used to build a RAID group can come from different flash drives within a single storage node, or from flash drives within different storage nodes. When the flash drives in a RAID group come from different storage nodes, if one storage node fails, data can still be recovered using data stored on the flash drives of the remaining healthy storage nodes, thereby improving the reliability of the storage system.

[0112] Furthermore, in this embodiment, different types of flash drives can be used to construct RAID groups. In other words, the storage system provided in this embodiment is compatible with different types of flash drives. Thus, if a flash drive in the storage system fails, any type of flash drive can be used as a replacement. For example, if a ZB SSD in the storage system fails, it can be replaced by a ZB SSD, a Block SSD, or a ZNS SSD without affecting the construction of logical blocks, thereby improving the system's compatibility.

[0113] In some possible implementations, the storage system shown in Figure 11 can also be applied to in-memory computing scenarios in distributed environments. Referring to Figure 17, in an in-memory computing scenario, the control unit 1102 shown in Figure 11 can be a server or a processor included in a desktop computer. A server is a device with computing capabilities; for example, an Advanced Reduced Instruction Set Machine (ARM) server or an x86 server can serve as the server. In terms of hardware, in addition to the processor, the server includes other components such as memory, network interface card (NIC), and hard disk. The processor, memory, NIC, and hard disk are connected via a bus. The processor and memory provide computing resources. Specifically, the processor is a central processing unit used to handle data access requests from outside the server and also to handle requests generated internally by the server. For example, when the processor receives a write data request, it temporarily stores the data in the write data request in memory. When the total amount of data in memory reaches a certain threshold, the processor sends the data stored in memory to the hard disk for persistent storage. Here, "persistent storage" refers to the ability of the flash drive to retain the recorded data after power failure. In addition to its primary function, the processor is also used for data computation and processing, such as metadata management, deduplication, data compression, data verification, virtualization of storage space, and address translation. In practical applications, there can often be multiple CPUs, with each CPU having one or more CPU cores. This application does not limit the number of CPUs or the number of CPU cores. For a description of memory, please refer to the memory description included in Figure 1; it will not be repeated here.

[0114] In this embodiment, the server in this application scenario can be installed in a server rack, which can have multiple slots. The number of slots can be 4, 8, 16, 32, or other suitable numbers, with each slot accommodating one server. In specific implementations, the storage system is scalable. In some possible implementations, servers can be inserted into or removed from a server rack, depending on the actual situation. The storage capacity of each server can be any integer multiple of 4TB, such as 8TB, 12TB, 16TB, 32TB, etc.

[0115] Secondly, this application also provides a data storage method, which can be applied to the storage system shown in Figure 7 of the first aspect. Specifically, this data storage method refers to the process in the first aspect where the control unit 1102 receives a write request and writes data to the flash drive; to avoid redundancy, it will not be described further here. It should be noted that, depending on the specific architecture of the storage system, the executing entity of this data storage method may vary slightly. In specific implementation, the executing entity of this data storage method can be the control unit shown in Figure 12 of the first aspect, the host shown in Figure 12, or the engine shown in Figure 13.

[0116] Thirdly, embodiments of this application also provide a computer-readable medium including instructions that, when executed on a computer, cause the computer to perform the data storage method as described in the second aspect above.

[0117] Fourthly, embodiments of this application also provide a computer program product that, when run on a computer, causes the computer to execute the data storage method as described in the second aspect above.

[0118] The methods provided in this application can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium, etc.

[0119] After introducing a storage system provided by the embodiments of this application, the application scenarios of the storage system provided by the embodiments of this application will be introduced, including but not limited to the following:

[0120] The first type: The storage system provided in this application embodiment can support artificial intelligence applications, machine learning applications, big data analytics applications, and many other types of applications. The rapid growth of such applications is driven by three technologies: deep learning (DL), graphics processing units (GPUs), and big data. Deep learning is a computational model that utilizes massively parallel neural networks inspired by the human brain. GPUs are modern processors with thousands of cores, well-suited for running algorithms that roughly match the parallelism of the human brain.

[0121] The second approach: The storage system provided in this application embodiment can be used in a neuromorphic computing environment. Neuromorphic computing is a form of computing that mimics brain cells. To support neuromorphic computing, the architecture of interconnected "neurons" replaces the traditional computing model with low-power signals transmitted directly between neurons, achieving more efficient computation.

[0122] Thirdly, the storage system provided in this application embodiment can also be configured to support the storage or use of blockchain. Such a blockchain can be represented as a continuously growing list of records, called blocks, which are linked and protected using cryptography. Each block in the blockchain may contain a hash pointer as a link to the previous block, a timestamp, transaction data, etc. This structure makes data modification and tampering extremely difficult. With the continuous development of technology, blockchain has been widely applied in finance, logistics, healthcare, public services, and other fields, providing new solutions for data security and trustworthiness.

[0123] Of course, the storage system provided in this application can also be applied to other application scenarios, such as big data analysis, edge computing, etc., which are not specifically limited in the embodiments of this application.

[0124] In summary, the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A storage system, characterized in that, include: Multiple flash drives, wherein at least one of the multiple flash drives is a hybrid flash drive, the storage space of the hybrid flash drive includes a first space and a second space, the first space includes multiple erase blocks; the second space includes multiple partition zones; The plurality of flash drives are virtualized to form a storage pool, the storage pool comprising several logical blocks, the storage space of which comes from the zones and erase blocks provided by the plurality of flash drives; Based on the storage pool, an independent disk redundant array (RAID) group is created. The RAID group contains multiple logical blocks, and different logical blocks come from different flash drives. Each logical block has the same capacity. The storage space of the first logical block among the multiple logical blocks comes from at least the first space and the second space. The first space provides storage space for the first logical block at the page level, and the second space provides storage space for the first logical block at the zone level.

2. The storage system according to claim 1, characterized in that, All of the aforementioned flash drives are hybrid flash drives; Some of the hybrid flash drives have different partition sizes in the first space.

3. The storage system according to claim 1, characterized in that, The first space is configured with a first portion of flash memory chips included in the hybrid flash drive; the second space is configured with a second portion of flash memory chips included in the hybrid flash drive.

4. The storage system according to any one of claims 1-3, characterized in that, The hybrid flash drive includes a first interface, a second interface, a first controller, and a second controller; The first interface is connected to the first controller, and the first interface is the input / output interface of the first space; The second interface is connected to the second controller, and the second interface is the input / output interface of the second space.

5. The storage system according to any one of claims 1-3, characterized in that, The hybrid flash drive includes a third interface and a third controller, the third interface and the third controller are connected, and the third interface is a shared interface between the first space and the second space.

6. The storage system according to any one of claims 1-3, characterized in that, The RAID group is located in a disk enclosure, which also includes: Control unit, used to create the RAID group based on the storage pool; and Establish a mapping relationship between the first logical block and the hybrid flash drive, as well as between the first space and the second space.

7. The storage system according to claim 6, characterized in that, The control unit is also used for: Receive a write request, the write request carrying data and the address of the first logical block pointing to the first logical block; Based on the mapping relationship, the hybrid flash drive corresponding to the first logical block address, as well as the first space and the second space, are determined; The data is stored in the pages included in the first erase block of the first space and the first partition of the second space, respectively.

8. The storage system according to claim 7, characterized in that, The control unit is also used for: Before storing the data into the pages included in the first erase block of the first space and the first partition of the second space, respectively, a write instruction is sent to the hybrid flash drive, the write instruction carrying the address of the first logical block and the length of the data.

9. The storage system according to claim 5, characterized in that, The hybrid flash drive includes a first interface and a second interface, wherein the first interface is an input / output interface for the first space, and the second interface is an input / output interface for the second space. The control unit is further configured to: Before storing the data into the pages included in the first erase block of the first space and the first partition of the second space respectively, a first write instruction is sent to the hybrid flash drive through the first interface. The first write instruction carries the address of the first sub-logical block and the length of the first data. A second write instruction is sent to the hybrid flash drive through the second interface. The second write instruction carries the address of the second sub-logical block and the length of the second data. The first logical block address includes the first sub-logical block address and the second sub-logical block address, and the data includes the first data and the second data.

10. A data storage method, characterized in that, Applied to a storage system, the storage system including a Redundant Array of Independent Disks (RAID) group, the RAID group comprising multiple logical blocks of equal capacity, the method comprising: Obtain a write request, which carries data; The data is stored in the first space and the second space of the hybrid flash drive corresponding to the first logical block among the plurality of logical blocks. The first space provides storage space for the first logical block at the page level, and the second space provides storage space for the first logical block at the zone level. The first space includes a plurality of erase blocks, and the second space includes a plurality of partitions.

11. The method according to claim 10, characterized in that, The write request also includes a first logical block address pointing to the first logical block, and storing the data into the first space and the second space of the hybrid flash drive corresponding to the first logical block among the plurality of logical blocks, including: Based on the mapping relationship between the first logical block and the hybrid flash drive, as well as the first space and the second space, the first space and the second space of the hybrid flash drive corresponding to the address of the first logical block are determined; The data is stored in the pages included in the first erase block of the first space and the first partition included in the second space, respectively.

12. The method according to claim 11, characterized in that, The method further includes: Before storing the data into the pages included in the first erase block of the first space and the first partition included in the second space, a write instruction is sent to the hybrid flash drive, the write instruction carrying the address of the first logical block and the length of the data.

13. The method according to claim 11, characterized in that, The hybrid flash drive includes a first interface and a second interface, wherein the first interface is an input / output interface for the first space, and the second interface is an input / output interface for the second space. The method further includes: Before storing the data into the pages included in the first erase block of the first space and the second partition included in the second space, a first write instruction is sent to the hybrid flash drive through the first interface. The first write instruction carries the address of the first sub-logical block and the length of the first data. A second write instruction is sent to the hybrid flash drive through the second interface. The second write instruction carries the address of a second sub-logical block and the length of second data. The data includes the first data and the second data, and the first logical block address includes the address of the first sub-logical block and the address of the second sub-logical block.

14. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 10-13 above.

15. A computer program product, characterized in that, When it is run on a computer, it causes the computer to perform the method as described in any one of claims 10-13 above.

Images