Partition hints for partitioned namespace storage devices

Abstract

Partition hints for use with a zoned namespace (ZNS) storage device. The partition hints include one or more of a first hint indicating that the partition is part of a group of multiple partitions, a second hint indicating that the partition is to be quickly filled, or a third hint indicating that the partition is associated with a background operation. The first hint is structured to instruct the ZNS storage device to allocate a first storage resource to the partition that is physically adjacent to a second storage resource reserved for other partitions in the multiple partitions. The second hint is structured to instruct the ZNS storage device to bypass a staging area when writing to the partition. The third hint is structured to instruct the ZNS storage device to deprioritize at least one operation of writing to the partition or to bypass the staging area when writing to the partition.

Description

Technical Field

[0001] This disclosure relates to systems, methods, and apparatus for improving the performance and utilization of storage devices by increasing the collaboration between software executing on a computer system and the storage device. Background Technology

[0002] Historically, non-volatile storage devices used by computers have primarily been block-based. That is, the storage device presents available storage as a sequential series of independent physical storage blocks, which are directly mapped to corresponding sequential physical storage areas on the underlying physical storage medium, such as the areas of magnetic disks in a hard disk drive (HDD), or areas of a magnetic tape device. Accordingly, the interfaces traditionally used to access non-volatile storage devices have also been block-based, and the operating system and other software have therefore interacted with non-volatile storage devices in a block-based manner, understanding how the sequential blocks presented by the storage device are internally mapped to sequential physical areas on the device's underlying physical storage medium.

[0003] For compatibility and interoperability with existing hardware and software interfaces, newer non-volatile storage devices use these traditional storage interfaces, although internally they may not actually map sequential external storage blocks to sequential physical storage areas. For example, solid-state drives (SSDs) store data in integrated circuit components that include semiconductor cells such as NAND flash memory, phase-change memory (PCM), etc. Due to the physical nature of the underlying semiconductor cells, the "sequential" blocks presented externally to the SSD are intelligently distributed and redistributed within the SSD's memory cells for purposes such as caching, performance optimization, wear leveling, and garbage collection, rather than being sequentially distributed internally. Even modern conventional HDDs lack a sequential mapping between externally visible blocks and the underlying storage medium used, due to technologies such as shingled magnetic recording (SMR).

[0004] This disconnect between the hardware and software interfaces and the internal structure of the storage medium can lead to suboptimal use of the internal storage medium, resulting in wasted storage, reduced performance, increased wear and tear on the physical media, and increased power consumption. To address these drawbacks, recent standards, such as the Partitioned Namespace (ZNS) extension to the Non-Volatile Memory Fast (NVMe) specification, have sought to more closely align the software's view of the storage device with the device's internal storage characteristics. Specifically, ZNS storage devices are block-based storage devices that allow software to request the creation of namespace "partitions," each consisting of multiple blocks. Internally, ZNS storage devices treat each partition explicitly and separately, attempting to allocate storage resources to a given partition in a way that promotes efficient use of available storage, improves performance, reduces wear and tear, and lowers power consumption. According to the ZNS specification, each partition can only be written sequentially, starting from the beginning of the partition. Furthermore, data within a given partition cannot be arbitrarily overwritten; instead, a partition can be completely overwritten by first (at least logically) erasing it and then rewriting it from the beginning. Reading data from a partition is largely unrestricted, so data can be read in the same way as traditional storage devices. Summary of the Invention

[0005] While facilitating tighter interaction between software and the internal components of storage devices, ZNS storage devices can present significant challenges for many software use cases. For example, modern SSDs typically consist of multiple NAND flash memory dies, each containing multiple memory cells, each capable of electronically storing one or more bits of data (typically one to four bits per cell). Many NAND dies arrange these memory cells into pages (the smallest structure that can be read or written), and then blocks (the smallest structure that can be erased). Modern NAND dies comprise multiple physical planes (e.g., vertically arranged) and are capable of forming a given block from multiple pages on a single plane (i.e., a single-plane block), or distributing a given block across two or more planes (i.e., a multi-plane block formed using pages from two or more planes). Due to the efficiency gained when accessing adjacent planes in parallel (e.g., in terms of throughput and energy usage), most ZNS storage devices only allow the allocation of partitions that are multiples of the size of a single plane of the die (thus forming multi-plane blocks spanning these planes). This means that, for example, the minimum partition size that software can request from a ZNS storage device may be on the order of 256MB or larger, which is not optimal for many software use cases.

[0006] Furthermore, many ZNS storage devices only allow a given partition to be "open" for writing for a limited period of time. This may be due to one or more practical issues (e.g., the complexity of managing a large number of open partitions at once), or physical issues (e.g., limitations imposed by the physical constraints of NAND memory cells, which prevent a given NAND flash page from being kept open for writing).

[0007] At least some of the embodiments described herein enable a computer system to provide one or more partition-related "hints" to a ZNS storage device to facilitate improvements in one or more of the following: (i) the allocation of resources to a partition by the storage device, or (ii) subsequent use of the partition by the storage device. At least some of the embodiments described herein also enable a ZNS storage device to operate in response to one or more partition-related "hints" to improve one or more of the following: (i) the allocation of resources to a partition by the storage device, or (ii) subsequent use of the partition by the storage device. Therefore, at least some of the embodiments described herein describe related products comprising a computer system (e.g., based on software executing thereon) that operates by using one or more hints to instruct a ZNS storage device, and a ZNS storage device that operates in response to these hints.

[0008] In embodiments, one or more prompts include partition group prompts, which enable the computer system to indicate that a particular partition is part of a group of partitions that the computer system will program together as a group (e.g., when writing to a partition group, the computer system will divide the write operation across all partitions within the group). In embodiments, partition group prompts enable the ZNS storage device to allocate storage resources to a particular partition that is physically adjacent to storage resources already (or to be) allocated to other partitions in the group. In embodiments, when partition group prompts are supported, the ZNS storage device allows the creation of partition sizes that result in additional unnatural (or suboptimal) allocations of internal storage resources (e.g., by using single-plane blocks on multi-plane NAND dies, by using dual-plane blocks on quad-plane NAND dies, etc.), because it is recognized that the anticipated inefficiencies (e.g., in performance, power consumption, etc.) caused by such unnatural allocations will be eliminated (or at least mitigated) by the way the computer system accesses the partitions (i.e., by accessing the partitions in parallel with physically related partitions). Therefore, partition group prompts enable the computer system to request partition sizes more consistent with the computer system's use of these partitions while retaining the benefits of using larger partition sizes that are more suitable for the physical characteristics of the ZNS storage device.

[0009] In one embodiment, one or more prompts include a fast-fill prompt, which enables the computer system to instruct a specific partition to be "fast-filled" when it is written to (i.e., programmed). In this embodiment, the fast-fill prompt communicates with the ZNS storage device, ensuring that all data to be written is readily available upon request when a write partition request is made. Because all data to be written is readily available, the ZNS storage device is ensured that it will not need to keep the partition "open" for writing once the data it is waiting to be written becomes fully available. This allows the ZNS storage device to bypass data staging areas (e.g., caches or dedicated areas of the underlying storage medium) and write data directly to the partition. Therefore, the fast-fill prompt improves write performance by reducing the number of write operations required to perform a write operation (i.e., by bypassing staging areas), reduces the use of staging area storage resources, and, if the staging area storage has limited endurance (e.g., in the case of NAND flash memory), increases the operational lifespan of the staging area storage.

[0010] In one embodiment, one or more prompts include background operation prompts, which enable the computer system to indicate that I / O operations on a specific partition are associated with background activity. In another embodiment, the background operation prompt communicates with the ZNS storage device, allowing I / O operations to be prioritized relative to other I / O operations when they are issued to a partition. In an example of prioritizing I / O operations, the ZNS storage device blocks one or more I / O operations to a first partition assigned a background operation prompt until one or more I / O operations to a second partition not assigned a background operation prompt are completed. Therefore, background operation prompts improve the overall performance of the device by enabling the ZNS storage device to focus on higher-priority I / O operations. In some embodiments, prompts may include background operation prompts but not (explicit or implicit) fast-fill prompts. In some embodiments, prompts may include both background operation prompts and explicit fast-fill prompts. In some embodiments, background operation prompts implicitly imply fast-fill prompts. In one embodiment, one or more I / O operations sent to a first partition that is assigned both a background operation prompt and an explicit or implicit fast fill prompt are blocked until one or more I / O operations sent to a second partition that is assigned a fast fill prompt but not a background operation prompt are completed.

[0011] In embodiments, methods, systems, and computer program products operate to utilize prompts for partitions when interacting with a ZNS storage device. In these embodiments, a computer system determines to define a partition on the ZNS storage device for storing at least a portion of a dataset. The computer system sends one or more messages to the ZNS storage device. The messages(s) instruct the ZNS storage device to create the partition and provide the ZNS storage device with one or more prompts for the partition. The prompts(s) for the partition(s) include at least one of the following:

[0012] (i) a first prompt indicating that the partition is part of a group of partitions, (ii) a second prompt indicating that the partition will be quickly filled, or (iii) a third prompt indicating that the partition is associated with a background operation. When the first prompt is included, it is configured to instruct the ZNS storage device to allocate a first portion of storage resources to the partition, the first portion of which is physically adjacent to a second portion of second storage resources reserved for other partitions in the group. When the second prompt is included, it is configured to instruct the ZNS storage device to bypass a staging area when writing to the partition. When the third prompt is included, it is configured to instruct the ZNS storage device to perform at least one of the following: (i) reduce the priority of at least one operation writing to the partition, or (ii) bypass a staging area when writing to the partition.

[0013] In related embodiments, the ZNS storage device operates in response to prompts(s) provided by a computer system. In these related embodiments, the ZNS storage device includes physical data storage resources and logic configuring the ZNS storage device to operate in response to prompts for a partition. The logic creates a record for the partition based at least on instructions received from the computer system that identify the partition. Based at least on one or more prompts(s) for the partition received from the computer system that identify the partition, the logic performs at least one of the following: (i) when the prompts(s) for the partition include a first prompt indicating that the partition is part of a group of partitions, allocating a first portion of physical data storage resources to the partition, the first portion of which is physically adjacent to a second portion of physical storage resources reserved for other partitions in the group; (ii) when the prompts(s) for the partition include a second prompt indicating that the partition will be quickly filled, the second prompt bypasses a staging area when writing to the partition; or (iii) when the prompts(s) for the partition include a third prompt indicating that the partition is associated with a background operation, performing at least one of the following: (i) reducing the priority of at least one operation writing to the partition, or (ii) bypassing a staging area when writing to the partition.

[0014] This summary is provided to present a simplified version of the selected concepts, which will be further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. Attached Figure Description

[0015] To illustrate how the above and other advantages and features of the invention can be obtained, the invention, which has been briefly described above, will be described in more detail with reference to specific embodiments of the invention shown in the accompanying drawings. It is understood that these drawings depict only exemplary embodiments of the invention and are therefore not intended to limit its scope. The invention will be described and explained with additional specificity and detail using the drawings, wherein:

[0016] Figure 1 An example computer architecture is shown that helps to utilize partitioning hints for ZNS storage devices;

[0017] Figure 2A An example of the allocation of natural-size storage partitions within a NAND flash memory medium is shown;

[0018] Figure 2B An example of the allocation of a set of non-natural-sized storage partitions is shown, which together reach the natural partition size of the NAND flash memory medium;

[0019] Figure 3 An example of a metadata format is shown, which can be used to transmit one or more partition-specific hints; and

[0020] Figure 4 A flowchart is shown as an example method for utilizing partition hints for ZNS storage devices. Detailed Implementation

[0021] Figure 1An example computer architecture 100 is illustrated, which facilitates the use of partitioning hints for ZNS storage devices. Computer architecture 100 includes a computer system 101 and a ZNS storage device 111. Computer system 101 includes one or more processors 102, system memory 103, one or more communication controllers 104, and persistent storage 105, all communicatively coupled via bus 106. ZNS storage device 111 is shown using one (or more) communication controllers 104 to communicate with computer system 101 via communication channel 118. Although not explicitly shown, ZNS storage device 111 may be directly electrically coupled to bus 106 or electrically coupled to communication controller 104. The specific type of communication controller 104 can be varied to place ZNS storage device 111 near computer system 101 (or even as part of computer system 101) or at a remote location within computer system 101.

[0022] ZNS storage device 111 is shown as including controller 112 (including prompt processor 113 and partition manager 114) and storage resources 115 (including storage area 117, and possibly including scratch area 116). The specific form of storage resources 115 may vary, and in at least some embodiments, storage resources 115 include one or more of integrated circuit-based storage media (e.g., NAND flash memory, PCM, etc.) or magnetic storage media (e.g., one or more magnetic hard disk drive modes).

[0023] Persistent storage device 105 is shown as including partition manager 107. Typically, partition manager 107 operates to create and manage storage partitions at ZNS storage device 111 based on dataset(s) to be stored by computer system 101 at ZNS storage device 111. To facilitate this function, partition manager 107 is shown as including dataset analyzer 108, partition generator 109, and prompt generator 110.

[0024] Dataset analyzer 108 identifies and analyzes datasets to be stored at ZNS storage device 111 to determine one or more attributes of these datasets. In an embodiment, these attributes include one or more of the following: dataset size, dataset type, dataset source, dataset availability, type of operation associated with the creation of the dataset, identity of one or more related datasets, etc. Dataset analyzer 108 uses this analysis to determine whether to create one or more partitions to store one or more datasets.

[0025] Partition generator 109 generates requests for creating partitions at ZNS storage device 111, such as partitions for storing one or more datasets identified and analyzed by dataset analyzer 108. In an embodiment, partition generator 109 generates requests for creating partitions based on one or more dataset attributes (e.g., dataset size and dataset availability) determined by dataset analyzer 108. For example, partition generator 109 may use dataset size to determine at least one of the number or size of partitions required to store the dataset. In another example, partition generator 109 may use dataset size and the identity of one or more related datasets, together with one or more related datasets, to determine at least one of the number or size of partitions required to store the dataset. In yet another example, partition generator 109 may use dataset availability to determine at least one of the number or size of partitions required to store the available portion of the dataset.

[0026] Partition manager 107 uses one or more messages sent via communication channel 118 to transmit one or more partition creation requests generated by partition generator 109 to controller 112 at ZNS storage device 111. Partition manager 114 at ZNS storage device 111 then creates the requested partitions (including allocating storage resources in storage area 117 to each partition). In some embodiments, partition manager 114 returns an acknowledgment / success message to partition manager 107 upon successful partition creation, or a failure / error message to partition manager 107 if partition creation fails (e.g., due to a lack of available storage resources within storage area 117).

[0027] Hint generator 110 generates one or more hints for one or more partitions generated by partition generator 109. These hints are configured to instruct ZNS storage device 111 to operate relative to the partition(s)(s) to which the hint is applied in a particular manner. In an embodiment, hint generator 110 generates hints based on one or more dataset attributes determined by dataset analyzer 108.

[0028] Partition manager 107 uses one or more messages (which may be the same messages discussed earlier or separate messages) sent via communication channel 118 to transmit one or more prompts generated by prompt generator 110 to controller 112 at ZNS storage device 111. Prompt processor 113 at ZNS storage device 111 then causes ZNS storage device 111 to at least attempt to operate according to these prompts. In some embodiments, prompt processor 113 treats prompts as suggestions and makes a “best effort” to act according to the prompts. In these embodiments, prompt processor 113 is free to ignore (or even reject) prompts that it should choose (e.g., if the prompt would negatively impact the performance of ZNS storage device 111, if ZNS storage device 111 lacks the ability or resources to fulfill the prompt, etc.). In other embodiments, prompt processor 113 treats prompts as binding instructions and makes every reasonable effort to act according to the prompts. In these other embodiments, if prompt processor 113 cannot fulfill a prompt associated with an operation, prompt processor 113 may reject the operation. In some embodiments, the prompting processor 113 may return a message such as confirmation, success, error, or failure to the partition manager 107 as appropriate, although in other embodiments the prompting processor 113 may not send a return message to the partition manager 107 (i.e., it remains silent on whether to honor the prompt).

[0029] In an embodiment, the prompt generator 110 generates a partition group prompt indicating that the partition to which the prompt is applied is part of a partition group (i.e., comprising two or more partitions). In some embodiments, when a partition has a size smaller than the natural (e.g., optimal or preferred) partition size of the ZNS storage device 111 (i.e., based on the physical characteristics of the storage resource 115), the prompt generator 110 generates a partition group prompt for that partition, and thus the partition manager 107 has grouped the partition with one or more other partitions such that the size of the group matches the natural partition size of the ZNS storage device 111. In an embodiment, the partition group stores related datasets, which are written / programmed together by the computer system 101 as a group. For example, when programming a partition group, the computer system 101 partitions write operations on all partitions within the group. In an embodiment, the partition group prompt is an operable instruction (i.e., by the prompt processor 113) to cause the ZNS storage device 111 to allocate physically adjacent storage resources to the partitions in the group to obtain performance benefits and power efficiency when the computer system 101 partitions I / O operations on all partitions within the group.

[0030] Figure 2A and Figure 2B This concept was demonstrated. Specifically, Figure 2AExample 200a shows the allocation of natural-size storage partitions within a NAND flash memory medium. Figure 2A Multiple NAND flash memory planes (i.e., "Plane 0" to "Plane 3") are shown, for example, planes arranged vertically on the same NAND flash memory die. Each of these planes comprises multiple pages (P), and each page (P) consists of multiple NAND memory cells. Figure 2A In this diagram, the physical storage resources (i.e., pages) within these planes are shown as being allocated to a single partition (i.e., Z0) comprising multiple blocks (i.e., B0-Bn). As shown, the first block (i.e., B0) of the partition is allocated to use the first page from each of the four planes, the second block (i.e., B1) is allocated to use the second page from each of the four planes, the third block (i.e., B2) is allocated to use the third page from each of the four planes, and so on. Here, the partition has the “natural” size of the NAND flash memory medium, as its blocks comprise pages arranged vertically across all the dies. Due to the physical characteristics of the NAND flash memory medium, in this configuration, the partition size and allocation are designed for efficient (and perhaps even optimal) performance and power consumption when programming the partition, since each of the four planes is accessed together during partition programming. However, in some cases, Figure 2A The size of the partition poses a challenge to the software running on computer system 101, which operates on datasets smaller than the partition size, especially since the partition is programmed in an append-only manner.

[0031] on the other hand, Figure 2B Example 200b shows the allocation of a set of non-natural-sized storage partitions that together reach the natural partition size of the NAND flash memory medium. Figure 2B Multiple identical vertically arranged NAND flash memory planes (i.e., "Plane 0" to "Plane 3") are shown, each NAND flash memory plane comprising multiple pages (P) composed of NAND memory cells. However, in Figure 2BIn this configuration, the physical storage resources (i.e., pages) within these planes are shown as being allocated to two partitions. One partition (Z0) comprises blocks B0-Bn allocated from pages in planes 0 and 1, and the other partition (Z1) comprises blocks B0-Bn allocated from pages in planes 2 and 3. In this embodiment, the allocation is based on prompting processor 113 operating according to one or more prompts generated by prompt generator 110, which designates partitions Z0 and Z1 as part of a partition group. Here, each of partitions Z0 and Z1 has a “non-natural” size of NAND flash memory medium because the pages comprised of their blocks span less than the vertically arranged plane of all the dies. Due to the physical characteristics of the NAND flash memory medium, in this configuration, the size and allocation of each of partitions Z0 and Z1 are designed for less efficient power consumption when programming partitions individually (i.e., because only two of the four planes are accessed together during programming partitions). Figure 2A (Compared to the configuration). However, since partitions Z0 and Z1 are allocated from physically adjacent planes, if the computer system 101 divides write operations between partitions Z0 and Z1, it is possible to achieve [something] during partition programming. Figure 2A The configuration achieves effective (and perhaps even optimal) performance and power consumption (i.e., because each of the four planes is programmed together).

[0032] In some cases, Figure 2B The size of the partitions will be better aligned with the size of the dataset used by the software executing at computer system 101. Therefore, the partition group hints generated by hint generator 110 enable partition generator 109 to create non-natural-sized partitions for ZNS storage device 111 (which are suitable for the size of the dataset used by the software), while enabling ZNS storage device 111 to operate on groups of natural-sized partitions and achieve the associated performance and power efficiency.

[0033] In additional or alternative embodiments, prompt generator 110 generates a quick-fill prompt indicating that a write operation for the partition to which it is applied will be able to provide all the data required to complete the write in a timely manner (e.g., not exceeding any constraints that ZNS storage device 111 may have, such as the amount of time a partition can be "open" for writing). In some embodiments, prompt generator 110 generates a quick-fill prompt when all the data required to complete the write will be available at the time of writing, or when the availability of the data required to complete the write can be guaranteed when the write begins. In embodiments, the quick-fill prompt is an instruction operable (i.e., by prompt processor 113) to cause ZNS storage device 111 to bypass staging areas (e.g., caches) when writing to a partition. For example, Figure 1ZNS storage device 111 is shown to include a scratch area 116. In some ZNS storage devices, scratch area 116 utilizes a common physical storage medium as storage area 117 (e.g., both use NAND flash memory, both employ magnetic storage devices, etc.). In other ZNS storage devices, scratch area 116 utilizes a different physical storage medium than storage area 117 (e.g., one uses NAND flash memory, the other uses magnetic storage devices; one uses random access memory, the other uses NAND flash memory, etc.). By using fast fill hints to bypass the scratch area, ZNS storage device 111 reduces the number of operations required to perform write operations, resulting in increased write performance, reduced power consumption, reduced use of available storage space in scratch area 116, and reduced wear on the underlying physical storage medium supporting scratch area 116.

[0034] In additional or alternative embodiments, prompt generator 110 generates a background operation prompt indicating that I / O operations for the partition to which the prompt is applied are related to background activity of computer system 101. In some embodiments, prompt generator 110 generates a background operation prompt when a background operation performed by computer system 101 generates data to be programmed into a partition. In some embodiments (e.g., when the background operation performed by computer system 101 is a low-priority operation), the background operation prompt is operable (i.e., by prompt processor 113) to cause ZNS storage device 111 to reduce the priority of one or more I / O operations received from computer system 101 (compared to one or more other I / O operations processed by ZNS storage device 111). In an example of reducing the priority of I / O operations, ZNS storage device blocks one or more I / O operations to a first partition to which a background operation prompt has been assigned until one or more I / O operations to a second partition to which no background operation prompt has been assigned are completed. In this way, background operation prompts improve the overall performance of the ZNS storage device 111 by enabling it to focus on higher-priority I / O operations.

[0035] In additional or alternative embodiments (e.g., when background operations performed by computer system 101 can generate the data required for writing in a timely manner), background operation prompts also include, or are interpreted as operable (i.e., by prompt processor 113), to cause ZNS storage device 111 to bypass fast-fill prompts for scratch area 116 when writing to the partition to which the prompt is applied. Thus, in these embodiments, background operation prompts also result in increased write performance, reduced power usage, reduced use of available storage space in scratch area 116, and reduced wear on the underlying physical storage medium supporting scratch area 116. In some embodiments, prompts may include background operation prompts but not (explicit or implicit) fast-fill prompts. In some embodiments, prompts may include both background operation prompts and explicit fast-fill prompts. In embodiments, one or more first I / O operations issued to a first partition assigned both background operation prompts and explicit or implicit fast-fill prompts are blocked until one or more I / O operations issued to a second partition assigned fast-fill prompts but not background operation prompts are completed.

[0036] As discussed, a partition group prompt indicates that multiple partitions are part of a partition group. In an embodiment, the partition group prompt assigns a partition group identifier to the partition group. In some embodiments, this partition group identifier applies only to the partition group prompt. However, in other embodiments, the partition group identifier also applies to one or both of the quick fill prompt or the background operation prompt. Therefore, in an embodiment, assigning a quick fill prompt or the background operation prompt applies the prompt to each of the multiple partitions associated with the partition group identifier.

[0037] In some embodiments, partition manager 107 communicates any and multiple prompts to ZNS storage device 111 via one or more commands, or as parameters to one or more commands. In one example, partition manager 107 transmits any and multiple prompts to ZNS storage device 111 as parameters to a partition creation command. In another example, partition manager 107 uses commands specific to communicate prompts to transmit any and multiple prompts to ZNS storage device 111.

[0038] In additional or alternative embodiments, partition manager 107 transmits any and multiple hints to ZNS storage device 111 by setting one or more fields in partition-specific metadata. For example, Figure 3 Example 300 shows a non-restricted metadata format that can be used to transmit one or more partition-specific hints. For example... Figure 3As shown, the example metadata format includes a partition group identifier field. In an embodiment, the partition group identifier field is used to define the partition group identifier of the partition to which the metadata is applied. In an embodiment, this same group identifier is typically used for each partition in a set of partitions. Therefore, for example, partition manager 107 sets the partition group identifier field to a common value in the partition-specific metadata for each partition in the partition group. Figure 3 As shown, the example metadata format also includes a hint field. In this embodiment, the hint field is used to assign one or both of the background operation hints or quick-fill hints to the partition to which the metadata is applied. Figure 3 As shown, the example metadata format also includes a partition group hint field. In an embodiment, the partition group hint field is used to specify that a partition is not part of a partition group (e.g., a value of 0), or to specify the identifier of a partition within a partition group (e.g., 1 to 255 in the example).

[0039] As shown in Example 300, in some embodiments, partition manager 107 indicates to ZNS storage device 111 which partitions will be part of a group by transmitting a common partition group identifier for each of these partitions (e.g., using a partition group identifier metadata field associated with each partition). In these embodiments, prompting processor 113 identifies which partitions are part of a given partition group by determining that the common partition group identifier has been assigned to multiple partitions. In alternative embodiments, partition manager 107 indicates to ZNS storage device 111 which partitions will be part of a group by directly transmitting to ZNS storage device 111 a list or range of partitions (possibly with partition group identifiers) that will be part of the group. In these embodiments, prompting processor 113 identifies which partitions are part of a given partition group based on the directly transmitted list or range of partitions.

[0040] The following discussion now concerns methods and method actions. Although method actions may be discussed in a specific order, or may be illustrated in a flowchart as occurring in a specific order, a specific order is not required unless specifically stated otherwise, or because an action depends on the need for another action to be performed before that action is performed.

[0041] Figure 4A flowchart of an example method 400 for utilizing partition hints for a ZNS storage device is shown. Method 400 will be described with reference to the components and data of computer architecture 100. As shown, method 400 includes actions (i.e., 401-404) as part of sub-method 400a, executed by computer system 101, for utilizing hints for partitions when interacting with ZNS storage device 111. Method 400 also includes actions (i.e., 405-409) as part of sub-method 400b, executed by ZNS storage device 111, for operating in response to hints for partitions. In an embodiment, ZNS storage device 111 includes physical data storage resources (e.g., storage resource 115, such as integrated circuit components or magnetic storage media) and logic (e.g., controller 112) for implementing actions 405-409. In some embodiments, method 400 is a standalone integrated method executed by computer architecture 100 as a whole, while in other embodiments method 400 includes related methods 400a / 400b executed separately by computer system 101 and ZNS storage device 111, respectively.

[0042] Initially, method 400a includes the action 401 of determining to define a partition on the ZNS storage device for storing a dataset. In some embodiments, action 401 includes determining that a partition is to be defined on the ZNS storage device for storing at least a portion of the dataset. In the example, based on analysis of the dataset by the dataset analyzer 108, the dataset analyzer 108 determines that at least one partition should be created for storing at least a portion of the dataset.

[0043] Method 400a also includes action 402 of sending one or more messages to the ZNS storage device. As shown, action 402 includes action 403 of instructing the ZNS storage device to create a partition and action 404 of providing prompts(s) for the partition.

[0044] In action 403, one or more messages instruct the ZNS storage device to create a partition. In the example, partition generator 109 generates one or more messages (e.g., commands) and initiates a request to create a partition at ZNS storage device 111 by sending these messages(s) to the ZNS storage device(s) via communication channel 118 by partition manager 107.

[0045] As shown in the figure, based on the receipt of the messages(s) described in action 403, method 400b includes action 405 of creating a record for the partition. In some embodiments, action 405 includes creating a record for the partition based at least on an instruction received from the computer system identifying the partition. In the example, partition manager 114 creates the record for the partition, which includes initiating the allocation of storage resources 115 from storage area 117 to the partition.

[0046] In action 404, one or more messages provide one or more hints for the partition (i.e., to ZNS storage device 111). In the example, in action 403, hint generator 110 generates one or more hints for the partition requested by partition generator 109 and initiates the transmission of messages(s) ...

[0047] As discussed, in some embodiments, partition manager 107 transmits prompts(s) to ZNS storage device 111 using direct commands (or command parameters). Therefore, in some embodiments of action 404, one or more messages provide one or more prompts to the ZNS storage device for that partition, based at least on transmitting commands about the partition. Also, as discussed, in some embodiments, partition manager 107 uses metadata fields (such as...) Figure 3 These (as shown) transmit multiple hints to ZNS storage device 111. Therefore, in some embodiments of action 404, at least based on setting one or more hint-specific fields within the metadata associated with the partition, one or more messages provide one or more hints for that partition to the ZNS storage device.

[0048] As explained, in some embodiments, the prompt generator 110 determines which prompts(s) to send for a particular partition(s) based on attributes of the data to be stored in the partition (e.g., data size, data availability, data origin, etc.) determined by the dataset analyzer 108. Thus, in some embodiments, method 400a includes determining one or more prompts(s) for a partition based on one or more attributes that identify the dataset.

[0049] like Figure 4 As shown, based on the reception of the messages(s) described in action 404, method 400b includes action 406 of receiving partition(s) hints(s) from a computer system. In some embodiments, action 406 includes identifying one or more hints for a partition received from the computer system. In the example, hint processor 113 receives the hints(s) generated by hint generator 110 in action 404.

[0050] It is worth noting that no specific order is shown between actions 403 and 404, indicating that these actions may be performed serially or in parallel, depending on the implementation. Similarly, no specific order is shown between actions 405 and 406, indicating that these actions may be performed serially or in parallel, depending on the implementation. Furthermore, it should be noted that, depending on the implementation, the messages sent in actions 403 and 404 may be the same message or may be completely different messages.

[0051] As shown in action 404, in some embodiments, the prompts include a first prompt indicating that the partition is part of a partition group. Therefore, in some embodiments, action 404 includes one or more prompts including a first prompt indicating that the partition is part of a group of partitions, the first prompt being configured to instruct the ZNS storage device to allocate a first set of storage resources to the partition, the first set of storage resources being physically adjacent to a second set of second storage resources reserved for other partitions in the group. In the example, prompt generator 110 generates a partition group prompt for the partition and sends the prompt to ZNS storage device 111 via communication channel 118.

[0052] As previously mentioned, in some embodiments, partition manager 107 indicates to ZNS storage device 111 which partitions will become part of a group by transmitting a common partition group identifier for each of these partitions to ZNS storage device 111. Therefore, in some embodiments of action 404, the first indication includes a partition group identifier, and each of the plurality of partitions in the group is associated with a partition group identifier. Also as previously mentioned, in some embodiments, partition manager 107 indicates to ZNS storage device 111 which partitions will become part of a group by directly transmitting to ZNS storage device 111 a list or range of partitions (possibly with partition group identifiers) that will become part of the group. Therefore, in other embodiments of action 404, the first indication includes the identity of each of the plurality of partitions in the group.

[0053] When one or more prompts include a first (i.e., partition group) prompt, method 400b includes action 407, based on the first prompt, allocating storage resources to the partition that are physically adjacent to storage resources reserved for other partitions in the group. In some embodiments, action 407 includes, when one or more prompts for the partition include a first prompt indicating that the partition is part of a group of partitions, allocating a first portion of physical data storage resources to the partition that is physically adjacent to a second portion of physical data storage resources reserved for other partitions in the group. In the example, as part of the allocation of storage resources 115 from storage region 117 to partition initiated in action 405, prompt processor 113 ensures that physically adjacent storage resources are allocated to partitions in the same partition group. Figure 2B This example is described, in which physically adjacent NAND flash memory plane pairs (i.e., the first pair including planes 0 and 1 and the second pair including planes 2 and 3) are used for resource allocation of partitions (i.e., Z0 and Z1) in the same partition group.

[0054] As shown in action 404, in some embodiments, the prompts include a second prompt indicating that the partition will be quickly filled. Therefore, in some embodiments, action 404 includes one or more prompts that include a second prompt indicating that the partition will be quickly filled, the second prompt being configured to instruct the ZNS storage device to bypass a staging area when writing to the partition. In one example, prompt generator 110 generates a quick-fill prompt for the partition and sends the prompt to ZNS storage device 111 via communication channel 118.

[0055] When one or more prompts include a second (i.e., fast fill) prompt, method 400b includes action 408, based on the second prompt, bypassing the staging area when writing to the partition. In some embodiments, action 408 includes bypassing the staging area when writing to the partition when one or more prompts for the partition include a second prompt indicating that the partition will be fast filled. In an example, based on having received a second prompt for a particular partition, when controller 112 subsequently processes a programming operation for that partition, prompt processor 113 ensures that the programming operation bypasses staging area 116 and is applied directly to storage area 117.

[0056] As shown in action 404, in some embodiments, the prompts include a third prompt indicating that the partition is associated with a background operation. Therefore, in some embodiments, action 404 includes one or more prompts including a third prompt indicating that the partition is associated with a background operation, the third prompt being configured to instruct the ZNS storage device to perform at least one of the following: (i) reduce the priority of at least one operation written to the partition, or (ii) bypass a staging area when writing to the partition. In the example, prompt generator 110 generates a background operation prompt for the partition and sends the prompt to ZNS storage device 111 via communication channel 118.

[0057] When one or more prompts include a third (i.e., background operation) prompt, method 400b includes action 409 of reducing the priority of an operation writing to the partition based on the third prompt, or bypassing a staging area when writing to the partition. In some embodiments, action 409 includes performing at least one of the following when one or more prompts for a partition include a third prompt indicating that the partition is associated with a background operation: (i) reducing the priority of at least one operation writing to the partition, or (ii) bypassing a staging area when writing to the partition. In one example, based on having received a third prompt for a particular partition, when controller 112 subsequently processes an I / O operation for that partition, prompt processor 113 ensures that the I / O is processed with a lower priority than at least one other operation. In another example, based on having received a third prompt for a particular partition, when controller 112 subsequently processes a programming operation for that partition, prompt processor 113 ensures that the programming operation bypasses staging area 116 and is applied directly to storage area 117.

[0058] As discussed, in embodiments, a partition group identifier applies to one or both of the quickfill prompt or background operation prompt, and assigning a quickfill prompt or background operation prompt applies that prompt to all partitions associated with the partition group identifier. Therefore, in embodiments, providing a second prompt indicating that a partition will be quickfilled includes applying the second prompt to each partition in a group of multiple partitions (thus associating each of those partitions with the quickfill prompt). Similarly, in embodiments, providing a third prompt indicating that a partition is associated with background operation includes applying the third prompt to each partition in a group of multiple partitions (thus associating each of those partitions with the background operation prompt).

[0059] It is worth noting that, in the embodiments, one software use case that benefits from the disclosed partition hints is data storage that utilizes "error coding" techniques to provide data persistence and efficiently use storage resources. Some conventional error coding implementations initially redundantly store the incoming dataset, for example by storing three copies of the dataset separately in different file system data blocks, which are then "sealed" as the data block is filled. Once a data block is sealed, it becomes read-only and undergoes erasure coding. Under erasure coding, a data block is divided into multiple data fragments of the same size (therefore, the data fragments are smaller than the data block), along with mathematical derivation code fragments that can be used to reproduce any data fragment (if later lost). The data block that initially stored the dataset can then be garbage collected, and further reading of the dataset can be provided via the data fragments.

[0060] In some error-coding implementations, fragment sizes are preferably 128MB or less, and fragments are mapped to ZNS partitions when using ZNS storage devices, thus constructing data blocks from multiple partitions. However, the natural partition size of many ZNS storage devices may be much larger than 128MB, such as 256MB or larger. Using the disclosed group hints, embodiments of this paper enable fragments to be mapped to partitions smaller than the natural partition size of the ZNS storage device (e.g., 128MB instead of 256MB), but maintain the performance of the “natural” ZNS storage device by accessing these fragments as groups. Furthermore, using the disclosed fast fill hints, partitions corresponding to fragments can be filled quickly because the data used to fill those partitions is already available (i.e., because the fragments come from already filled / sealed data blocks). Finally, some embodiments consider erasure coding for background operations, and in these embodiments, the disclosed background operation hints enable the ZNS storage device to reduce the priority of I / O operations associated with those background operations as needed to maintain the overall performance of the ZNS storage device.

[0061] Therefore, at least some of the embodiments described herein enable a computer system to provide one or more partition-related "hints" to a ZNS storage device, and enable the ZNS storage device to operate on one or more partition-related "hints" to improve one or more of the following: (i) resource allocation of the storage device to the partition, or (ii) subsequent use of the partition by the storage device. In embodiments, these hints include partition group hints, which enable the computer system to request partition sizes consistent with the computer system's use of these partitions while retaining the benefit of using larger partition sizes more suitable for the physical characteristics of the ZNS storage device. In embodiments, these hints include fast-fill hints, which enable the ZNS storage device to bypass data staging partitions when directly programming partitions, thereby improving write performance, reducing staging area storage resource usage, and increasing the operational lifespan of the staging area storage device. In embodiments, these hints include background hints, which enable the ZNS storage device to reduce the priority of I / O operations on the partition (improving the overall performance of the ZNS storage device), or to treat the partition as fast-fill.

[0062] Although the subject matter has been described in language specific to structural features and / or methodological behavior, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above, or the order of such actions. Rather, the described features and actions are disclosed as exemplary forms for implementing the claims.

[0063] As discussed in more detail below, embodiments of the present invention may include or utilize dedicated or general-purpose computer systems comprising computer hardware, such as one or more processors and system memory. Embodiments within the scope of the invention also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available medium accessible by a general-purpose or dedicated computer system. A computer-readable medium storing computer-executable instructions and / or data structures is a computer storage medium. A computer-readable medium carrying computer-executable instructions and / or data structures is a transmission medium. Therefore, by way of example and not limitation, embodiments of the present invention may include at least two distinctly different computer-readable media: computer storage media and transmission media.

[0064] Computer storage media are physical storage media that store computer-executable instructions and / or data structures. Physical storage media include computer hardware such as RAM, ROM, EEPROM, solid-state drives (“SSDs”), flash memory, phase-change memory (“PCM”), optical disc storage, magnetic disk storage, or other magnetic storage devices, or any other hardware storage device that can be used to store program code in the form of computer-executable instructions or data structures, which can be accessed and executed by general-purpose or special-purpose computer systems to achieve the functions disclosed in this invention.

[0065] Transmission media may include networks and / or data links, which may be used to carry program code in the form of computer-executable instructions or data structures and may be accessed by general-purpose or special-purpose computer systems. A “network” is defined as one or more data links used to transmit electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computer system via a network or another communication connection (hardwired, wireless, or a combination of hardwired and wireless), the computer system may consider that connection as a transmission medium. Combinations of the above should also be included within the scope of computer-readable media.

[0066] Furthermore, upon arrival at various computer system components, program code in the form of computer-executable instructions or data structures can be automatically transferred from the transmission medium to the computer storage medium (and vice versa). For example, computer-executable instructions or data structures received via a network or data link can be buffered in RAM within a network interface module (e.g., a "NIC") and then ultimately transferred to the computer system RAM and / or non-volatile computer storage media at the computer system. Therefore, it should be understood that computer storage media can be included in computer system components that also (or even primarily) utilize the transmission medium.

[0067] Computer-executable instructions include, for example, instructions and data that, when executed at one or more processors, cause a general-purpose computer system, a special-purpose computer system, or a special-purpose processing device to perform a specific function or set of functions. Computer-executable instructions can be, for example, binary files, intermediate format instructions such as assembly language, or even source code.

[0068] Those skilled in the art will understand that this invention can be practiced in network computing environments with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablet computers, pagers, routers, switches, etc. This invention can also be practiced in distributed system environments, where both local and remote computer systems perform tasks, and the local and remote systems are linked via a network (via a hardwired data link, a wireless data link, or a combination of hardwired and wireless data links). Thus, in a distributed system environment, the computer system can include multiple constituent computer systems. In a distributed system environment, program modules can reside in local and remote memory storage devices.

[0069] Those skilled in the art will also understand that the present invention can be practiced in a cloud computing environment. A cloud computing environment can be distributed, although this is not required. When distributed, a cloud computing environment can be internationally distributed within an organization and / or have components owned across multiple organizations. In this specification and the following claims, “cloud computing” is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services). The definition of “cloud computing” is not limited to any other numerous advantages that can be obtained from such a model when properly deployed.

[0070] Cloud computing models can be composed of various features, such as on-demand self-service, broad network access, resource pooling, rapid elasticity, and measurable services. Cloud computing models can also appear in various service models, such as Software as a Service (“SaaS”), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). Different deployment models can also be used to deploy cloud computing models, such as private clouds, community clouds, public clouds, and hybrid clouds.

[0071] Some embodiments, such as cloud computing environments, may include a system comprising one or more hosts, each capable of running one or more virtual machines. During operation, the virtual machines emulate the operating computing system, while supporting an operating system and possibly one or more other applications. In some embodiments, each host includes a hypervisor that emulates virtual resources used for the virtual machines using physical resources abstracted from a view of the virtual machines. The hypervisor also provides appropriate isolation between virtual machines. Thus, from the perspective of any given virtual machine, the hypervisor provides the illusion that the virtual machine interacts with physical resources, even if the virtual machine only interacts with the appearance of physical resources (e.g., virtual resources). Examples of physical resources include processing capacity, memory, disk space, network bandwidth, media drives, etc.

[0072] The invention may be practiced in other specific forms without departing from the spirit or essential characteristics thereof. The described embodiments are to be considered exemplary and not restrictive in all respects only. Therefore, the scope of the invention is indicated by the appended claims rather than by the foregoing description. All modifications within the meaning and equivalence of the claims should be included within their scope. When elements are introduced in the appended claims, the articles “a,” “an,” “the,” and “the” are intended to indicate the presence of one or more elements. The terms “comprising,” “including,” and “having” are intended to be inclusive, meaning that other elements besides those listed may exist. Unless otherwise specified, the terms “set” and “subset” are indented to exclude an empty set, so a “set” is defined as a non-empty set and a “subset” as a non-empty subset. Furthermore, unless otherwise specified, the term “subset” does not include its entire superset (i.e., a superset contains at least one item not included in the subset).

Claims

1. A method (400a) implemented at a computer system (101) including a processor (102) for utilizing hints about a partition when interacting with a namespace storage device (111) of a partition, the method comprising: Determine (108) that the partition will be defined on the namespace storage device of the partition, the partition being used to store at least a portion of the dataset; as well as Send one or more messages (107) to the namespace storage device of the partition, the one or more messages being: Instructing (109) the namespace storage device of the partition to create the partition; and (110) Provide one or more prompts for the partition to the namespace storage device of the partition, the one or more prompts for the partition including at least one of the following: A first indication indicates that the partition is part of a group of partitions. The first indication is configured to indicate that the namespace storage device of the partition allocates a first portion of storage resources (117) to the partition, the first portion being physically adjacent to a second portion of second storage resources reserved for other partitions in the group of partitions. The second prompt indicates that the partition will be quickly filled, and the second prompt is configured to instruct the namespace storage device of the partition to bypass the staging area (116) when writing to the partition; or A third prompt indicates that the partition is associated with a background operation, and the third prompt is configured to instruct the namespace storage device of the partition to reduce the priority of at least one operation that writes to the partition.

2. The method of claim 1, wherein the method includes providing the first prompt to the namespace storage device of the partition.

3. The method according to claim 2, wherein, As a result of receiving the first prompt, the namespace storage device of the partition allocates a first portion of storage resources to the partition, the first portion being physically adjacent to a second portion of second storage resources allocated to other partitions among the plurality of partitions.

4. The method according to any one of claims 1-3, wherein the method includes providing the second prompt to the namespace storage device of the partition.

5. The method according to claim 4, wherein, As a result of receiving the second prompt, the namespace storage device of the partition bypasses the temporary storage area when writing to the partition.

6. The method according to any one of claims 1-3 and 5, wherein the method includes providing the second prompt to the namespace storage device of the partition.

7. The method according to claim 6, wherein, As a result of receiving the third prompt, the namespace storage device of the partition reduces the priority of the at least one operation written to the partition.

8. The method according to any one of claims 1-3, 5 and 7, wherein, The one or more messages provide the one or more prompts for the partition to the namespace storage device, based at least on setting one or more prompt-specific fields within the command transmitted about the partition or the metadata associated with the partition.

9. The method according to any one of claims 1-3, 5 and 7, wherein the first prompt includes a partition group identifier, and each of the plurality of partitions in the group is associated with the partition group identifier.

10. The method according to any one of claims 1-3, 5 and 7, wherein the first prompt includes the identity of each of the plurality of partitions in the group.

11. The method according to any one of claims 1-3, 5 and 7, further comprising: The one or more hints for the partition are determined based on one or more attributes that identify the dataset.

12. A partitioned namespace storage device (111) that operates in response to a prompt for a partition, comprising: Physical data storage resources (117); as well as Logic (112), which configures the namespace storage device of the partition to perform at least the following: At least based on instructions received from the computer system (101) identifying the partition, a record of the partition described in (114) is created; and Based at least on one or more prompts received from the computer system regarding the partition, perform at least one of the following: (113) When the one or more prompts for the partition include a first prompt indicating that the partition is part of a group of partitions, a first portion of the physical data storage resources is allocated to the partition, the first portion being physically adjacent to a second portion of the physical data storage source reserved for other partitions in the group of partitions; When the one or more prompts for the partition include a second prompt, the second prompt indicates that the partition will be filled quickly, bypassing the staging area (116) when writing to the partition; or When the one or more prompts for the partition include a third prompt indicating that the partition is associated with a background operation, the priority of at least one operation that writes to the partition is reduced.

13. The namespace storage device for a partition according to claim 12, wherein the one or more prompts for the partition include the first prompt, wherein the namespace storage device for the partition allocates a first portion of the physical data storage resources to the partition, the first portion being physically adjacent to a second portion of the physical data storage resources reserved for other partitions among the plurality of partitions.

14. The namespace storage device for a partition according to any one of claims 12 and 13, wherein the one or more prompts for the partition include the second prompt, wherein the namespace storage device for the partition bypasses the staging area when writing to the partition.

15. The namespace storage device for a partition according to any one of claims 12 and 13, wherein the one or more prompts for the partition include the third prompt, wherein the namespace storage device for the partition lowers the priority of at least one operation written to the partition.

Images