Controlled system management based on storage device thermal load
The zoned namespace architecture in storage systems balances thermal and workload distribution by analyzing die temperature profiles, improving performance and reducing failure risks in enterprise data storage devices.
Patent Information
- Application Number
- JP2024525717
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-04-06
- Filing Date
- 2023-04-04
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2043-04-04
AI Technical Summary
Enterprise data storage devices experience uneven thermal loads and workloads across dies due to manufacturing variations and airflow, leading to performance degradation and potential failure from thermal throttling.
A storage system with a zoned namespace architecture that collects and analyzes thermal statistics to allocate zones within data storage devices, balancing thermal and workload distribution by selecting dies based on temperature profiles and airflow, using a controller to manage thermal statistics and rebalance access queues.
Improves system performance by optimizing thermal and workload balance, reducing the risk of performance degradation and failure, and enhancing overall efficiency.
Smart Images

Figure 0007587087000001 
Figure 0007587087000002 
Figure 0007587087000003
Abstract
Description
[Technical field]
[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Non-provisional Application No. 17 / 714,863, entitled "Controlled System Management Based On Storage Device Thermal Load," filed April 6, 2022, the entire contents of which are incorporated by reference herein for all purposes.
[0002] FIELD OF THEINVENTION Embodiments of the present disclosure relate generally to storage systems having data storage devices such as solid state drives (SSDs) that use a zone namespace architecture, and more specifically to programming data based on thermal statistics and load balancing of the storage systems and data storage devices. [Background technology]
[0003] Enterprise data storage devices typically have large capacity, which requires multiple dies with large form factors. Because enterprise performance expectations tend to be line speed, enterprise data storage devices also require a lot of power, which can lead to high heat loads. The heat load can be distributed among the dies of the multiple dies. Because the workload may not be spread evenly across the dies of the multiple dies, certain parts of the enterprise data storage device may be hotter than other parts of the enterprise data storage device based on the workload and usage. For example, a die that experiences more usage or workload may have a higher temperature than a die that experiences less usage or workload.
[0004] Additionally, due to die manufacturing variations, the power consumption of each die may vary depending on which die is targeted in the workload. Furthermore, the physical proximity of each die to the airflow source may cause temperature differences across the dies. Similarly, the location of each die may affect adjacent dies. Thus, the heat load and potential throughput may be affected by the thermal profile of the enterprise data storage device. Thermal throttling may be used to reduce the temperature of a data storage device that has reached or exceeded a threshold temperature. However, thermal throttling may reduce the performance of the data storage device, which may cause the data storage device to not reach performance expectations.
[0005] Therefore, there is a need in the art to improve the balance of workloads and thermal loads within storage systems and data storage devices. Summary of the Invention
[0006] The present disclosure relates generally to a storage system having data storage devices such as solid-state drives (SSDs) using a zoned namespace architecture, and more specifically to programming data based on thermal statistics and load balancing of the storage system and the data storage devices. The storage system includes two or more data storage devices and a controller coupled to the two or more data storage devices. Each data storage device of the two or more data storage devices includes a zoned namespace (ZNS) architecture. The controller is configured to collect thermal statistics for each data storage device of the two or more data storage devices, analyze the collected thermal statistics, and designate a zone by selecting one or more dies in at least one data storage device of the two or more data storage devices based on the analyzed collected thermal statistics. The data storage device includes a memory device having a plurality of dies and a controller coupled to the memory device. The controller is configured to collect thermal statistics for each die of the plurality of dies, analyze the collected statistics, and allocate the one or more dies to form a zone.
[0007] In one embodiment, a storage system includes two or more data storage devices, each data storage device of the two or more data storage devices including a Zone Name Space (ZNS) architecture, and a controller coupled to the two or more data storage devices, wherein the controller is configured to collect thermal statistics for each data storage device of the two or more data storage devices, analyze the collected thermal statistics, and designate a zone by selecting one or more dies in at least one data storage device of the two or more data storage devices based on the analyzed collected thermal statistics.
[0008] In another embodiment, a data storage device includes a memory device having a Zone Name Space (ZNS) architecture, the memory device including a plurality of dies, and a controller coupled to the memory device, the controller configured to collect thermal statistics for each die of the plurality of dies, analyze the collected statistics, and allocate one or more dies of the plurality of dies to form zones.
[0009] In another embodiment, a storage system includes two or more memory means and a controller coupled to the two or more memory means, the controller configured to allocate zones formed from one or more portions of the two or more memory means based on thermal statistics of each memory means of the two or more memory means. [Brief description of the drawings]
[0010] So that the above-mentioned features of the present disclosure can be understood in detail, a more particular description of the present disclosure, briefly summarized above, can be made by reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical embodiments of the present disclosure and therefore should not be considered as limiting its scope, as the present disclosure may admit of other equally effective embodiments. [Figure 1] 1 is a schematic block diagram illustrating a storage system in which a data storage device can function as a storage device for a host device, according to a particular embodiment. [Diagram 2] 1 is an illustrative diagram of a storage system having multiple data storage devices, in accordance with certain embodiments. [Figure 3A] 1 is an exemplary diagram of a zone namespace utilized within a storage device, according to certain embodiments. [Figure 3B] FIG. 3B is an example diagram of a state diagram of a zone namespace of the storage device of FIG. 3A in accordance with a particular embodiment. [Figure 4]1 is an illustrative diagram of forming zones in a memory device in accordance with a particular embodiment. [Diagram 5] FIG. 1 is a flow diagram illustrating a method for providing a zone to a host device, according to certain embodiments. [Figure 6] 1 is a schematic block diagram illustrating a storage system including a controller coupled to a plurality of data storage devices, the controller including a load balancer and a thermal statistics accumulator / analyzer module, according to a particular embodiment. [Figure 7] 1 is a flow diagram illustrating a method for rebalancing access queues of a data storage device, according to certain embodiments. [Figure 8] FIG. 2 is a flow diagram illustrating a method for processing write commands in a storage system with load balancing based on thermal statistics and future workload, according to a particular embodiment.
[0011] To facilitate understanding, the same reference numbers have been used, whenever possible, to designate identical elements common to the figures. It is contemplated that elements disclosed in one embodiment may be beneficially utilized on other embodiments without specific mention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] In the following, reference will be made to embodiments of the present disclosure. However, it should be understood that the present disclosure is not limited to the specifically described embodiments. Instead, any combination of the following features and elements, whether associated with different embodiments or not, is contemplated to implement and practice the present disclosure. Furthermore, although embodiments of the present disclosure may achieve advantages over other possible solutions and / or over the prior art, whether or not a particular advantage is achieved by a given embodiment does not limit the present disclosure. Thus, the following aspects, features, embodiments, and advantages are merely illustrative and are not considered elements or limitations of the appended claims unless expressly recited in the claims. Similarly, references to "the present disclosure" are not to be construed as a generalization of any inventive subject matter disclosed herein, and should not be considered elements or limitations of the appended claims unless expressly recited in the claims.
[0013] The present disclosure relates generally to a storage system having data storage devices such as solid state drives (SSDs) using a zone namespace architecture, and more specifically to programming data based on thermal statistics and load balancing of the storage system and the data storage devices. The storage system includes two or more data storage devices and a controller coupled to the two or more data storage devices. Each data storage device of the two or more data storage devices includes a zone namespace (ZNS) architecture. The controller is configured to collect thermal statistics for each data storage device of the two or more data storage devices, analyze the collected thermal statistics, and designate a zone by selecting one or more dies in at least one data storage device of the two or more data storage devices based on the analyzed collected thermal statistics. The data storage device includes a memory device having a plurality of dies and a controller coupled to the memory device. The controller is configured to collect thermal statistics for each die of the plurality of dies, analyze the collected statistics, and allocate the one or more dies to form a zone.
[0014] FIG. 1 is a schematic block diagram illustrating a storage system 100 having a data storage device 106 that may function as a storage device for a host device 104, according to a particular embodiment. For example, the host device 104 may utilize a non-volatile memory (NVM) 110 included in the data storage device 106 to store and retrieve data. The host device 104 includes a host DRAM 138. In some implementations, the storage system 100 may include multiple storage devices, such as the data storage device 106, that may operate as a storage array. For example, the storage system 100 may include multiple data storage devices 106 configured as a redundant array of inexpensive / independent disks (RAID) that collectively function as a mass storage device for the host device 104. The storage system 100 may be an enterprise storage system or a client storage system. Both the enterprise storage system and the client storage system include one or more data storage devices, with the enterprise storage system including a threshold number or more data storage devices and the client storage system including less than a threshold number of data storage devices.
[0015] The host device 104 may store and / or retrieve data to and / or from one or more storage devices, such as the data storage device 106. As illustrated in FIG. 1, the host device 104 may communicate with the data storage device 106 via an interface 114. The host device 104 may comprise any of a wide range of devices, including a computer server, a network-attached storage (NAS) unit, a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a telephone such as a so-called "smart" phone, a so-called "smart" pad, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, or other device capable of transmitting or receiving data from a data storage device, etc.
[0016] Data storage device 106 includes controller 108, NVM 110, power supply 111, volatile memory 112, interface 114, and write buffer 116. In some embodiments, data storage device 106 may include additional components not shown in FIG. 1 for clarity. For example, data storage device 106 may include a printed circuit board (PCB) to which components such as data storage device 106 are mechanically attached and which includes conductive traces that electrically interconnect the components of data storage device 106. In some embodiments, the physical dimensions and connector configuration of data storage device 106 may conform to one or more standard form factors. Some exemplary standard form factors include, but are not limited to, a 3.5" data storage device (e.g., HDD or SSD), a 2.5" data storage device, a 1.8" data storage device, peripheral component interconnect (PCI), PCI-extended (PCI-X), PCI Express (PCIe) (e.g., PCIe x1, x4, x8, x16, PCIe mini card, mini PCI, etc.). In some embodiments, the data storage device 106 may be directly coupled to the motherboard of the host device 104 (e.g., soldered or plugged directly into a connector).
[0017] The interface 114 may include one or both of a data bus for exchanging data with the host device 104 and a control bus for exchanging commands with the host device 104. The interface 114 may operate according to any suitable protocol. For example, the interface 114 may operate according to one or more of the following protocols: advanced technology attachment (ATA) (e.g., serial-ATA (SATA) and parallel-ATA (PATA)), Fibre Channel Protocol (FCP), serially attached SCSI (SAS), PCI and PCIe, small computer system interface (SCSI), non-volatile memory express (NVMe), OpenCAPI, GenZ, Cache Coherent Interface Accelerator (CCIX), Open Channel SSD (OCSSD), etc. The interface 114 (e.g., a data bus, a control bus, or both) is electrically connected to the controller 108 and provides an electrical connection between the host device 104 and the controller 108, allowing data to be exchanged between the host device 104 and the controller 108. In some embodiments, the electrical connection of the interface 114 may also allow the data storage device 106 to receive power from the host device 104. For example, as illustrated in FIG. 1, the power supply 111 may receive power from the host device 104 via the interface 114.
[0018] NVM 110 may include multiple memory devices or memory units. NVM 110 may be configured to store and / or retrieve data. For example, a memory unit of NVM 110 may receive data and a message from controller 108 instructing the memory unit to store the data. Similarly, a memory unit may receive a message from controller 108 instructing the memory unit to retrieve data. In some embodiments, each of the memory units may be referred to as a die. In some embodiments, NVM 110 may include multiple dies (i.e., multiple memory units). In some embodiments, each memory unit may be configured to store a relatively large amount of data (e.g., 128MB, 256MB, 512MB, 1GB, 2GB, 4GB, 8GB, 16GB, 32GB, 64GB, 128GB, 256GB, 512GB, 1TB, etc.).
[0019] In some embodiments, each memory unit may include any type of non-volatile memory device, such as a flash memory device, a phase-change memory (PCM) device, a resistive random-access memory (ReRAM) device, a magneto-resistive random-access memory (MRAM) device, a ferroelectric random-access memory (F-RAM), a holographic memory device, and any other type of non-volatile memory device.
[0020] The NVM 110 may include multiple flash memory devices or memory units. The NVM flash memory devices may include NAND or NOR based flash memory devices and may store data based on the charge contained in the floating gate of the transistor of each flash memory cell. In an NVM flash memory device, the flash memory device may be divided into multiple dies, each of which includes multiple physical or logical blocks, which may be further divided into multiple pages. Each of the multiple blocks in a particular memory device may include multiple NVM cells. Rows of NVM cells may be electrically connected using word lines to define each of the multiple pages. Each cell in each of the multiple pages may be electrically connected to a respective bit line. Furthermore, the NVM flash memory device may be a 2D or 3D device and may be a single level cell (SLC), multi-level cell (MLC), triple level cell (TLC), or quad level cell (QLC). The controller 108 may write data to and read data from the NVM flash memory device at the page level, and may erase data from the NVM flash memory device at the block level.
[0021] The power source 111 may provide power to one or more components of the data storage device 106. When operating in a standard mode, the power source 111 may provide power to one or more components using power provided by an external device, such as the host device 104. For example, the power source 111 may provide power to one or more components using power received from the host device 104 via the interface 114. In some embodiments, the power source 111 may include one or more power storage components configured to provide power to one or more components when operating in a shutdown mode, such as when the power source 111 stops receiving power from an external device. In this manner, the power source 111 may function as an on-board power source. Some examples of the one or more power storage components include, but are not limited to, capacitors, supercapacitors, batteries, and the like. In some embodiments, the amount of power that may be stored by the one or more power storage components may be a function of the cost and / or size (e.g., area / volume) of the one or more power storage components. In other words, as the amount of power stored by the one or more power storage components increases, the cost and / or size of the one or more power storage components also increases.
[0022] The volatile memory 112 may be used by the controller 108 to store information. The volatile memory 112 may include one or more volatile memory devices. In some embodiments, the controller 108 may use the volatile memory 112 as a cache. For example, the controller 108 may store cached information in the volatile memory 112 until the cached information is written to the NVM 110. As illustrated in FIG. 1, the volatile memory 112 may consume power received from the power source 111. Examples of the volatile memory 112 include, but are not limited to, random-access memory (RAM), dynamic random access memory (DRAM), static RAM (SRAM), and synchronous dynamic RAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, etc.)).
[0023] The controller 108 may manage one or more operations of the data storage device 106. For example, the controller 108 may manage reading data from and / or writing data to the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 may initiate the data storage command, store the data in the NVM 110, and monitor the progress of the data storage command. The controller 108 may determine at least one operating characteristic of the storage system 100 and store the at least one operating characteristic in the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 temporarily stores data associated with the write command in an internal memory or write buffer 116 before sending the data to the NVM 110.
[0024] The controller includes a thermal statistics accumulation module 150. The thermal statistics accumulation module 150 is coupled to the NVM 110. The thermal statistics accumulation module 150 may be configured to retrieve thermal statistics of the NVM 110, where the thermal statistics may be a temperature of each of a plurality of regions of the NVM 110. In one embodiment, each region may be a different memory device of the NVM 110, where the different memory devices have different memory architectures (e.g., SLC memory, MLC memory, TLC memory, QLC memory, PLC memory, etc.). In another embodiment, each region may include two or more adjacent dies of the plurality of dies of the NVM 110. In yet another embodiment, each region includes one die of the plurality of dies of the NVM 110. The thermal statistics accumulation module 150 may be configured to generate a temperature profile or heat map of the NVM 110.
[0025] FIG. 2 is an exemplary diagram of a storage system 200 having multiple data storage devices, according to a particular embodiment, where each data storage device of the multiple data storage devices may be data storage device 106 of FIG. 1. The multiple data storage devices are represented as points in a coordinate system, where each point, such as (1,1), represents a data storage device in storage system 200. Storage system 200 may include any number of data storage devices, such as "x" by "y" data storage devices. Data storage devices in storage system 200 may be arranged in rows and columns. It should be understood that the illustrated configuration is not intended to be limiting and is provided only as an exemplary embodiment. For example, the storage system may have different configuration patterns other than rows and columns. Although a planar arrangement of data storage devices is shown, it should be further understood that storage system 200 may have a stacked configuration. In other words, there may be data storage devices above and / or below the illustrated exemplary view of storage system 200.
[0026] Each data storage device is separated from adjacent data storage devices to allow for airflow from airflow source 202, physical components (e.g., wiring), etc. For example, a data storage device located at (1,1) is at a distance "a" from an adjacent data storage device located at (1,2), at a distance "b" from an adjacent data storage device located at (2,1), and at a distance "c" from a diagonally adjacent data storage device located at (2,2). It should be understood that "a", "b", and "c" may be the same or different. Similarly, in embodiments in which data storage devices are stacked, the distance between adjacently stacked data storage devices is considered. In storage system 200, there is a temperature profile between each of the data storage devices. The temperature profile may depend on several factors, such as, for example, the distance between adjacent data storage devices (e.g., "a", "b", and "c"), the proximity of the data storage devices to the center of storage system 200, the workload of the data storage devices, and the distance of the data storage devices from airflow source 202. For example, data storage devices in the center of storage system 200 may dissipate heat more slowly than data storage devices toward the edge of storage system 200. Similarly, data storage devices closer to airflow source 202 may be cooler than data storage devices further closer to airflow source 202. In another embodiment, data storage devices having a higher workload may be hotter than data storage devices having a lower workload. The thermal relationship between adjacent data storage devices may be predetermined or derived using a fixed benchmark that is run at each data storage device operation. In one embodiment, analyzing the collected thermal statistics is performed during an offline storage system management period, such as when storage system 200 is idle.
[0027] FIG. 3A is an exemplary diagram of a zone namespace (ZNS) 302 view utilized in a data storage device 300, according to a particular embodiment. The data storage device 300 can present a ZNS 302 view to a host device, such as the host device 104 of FIG. 1. The data storage device 300 can be a data storage device 106 of the storage system 100 of FIG. 1. The data storage device 300 can have one or more ZNSs 302, and each ZNS 302 can be a different size. The data storage device 300 can further comprise one or more traditional namespaces in addition to the one or more zone namespaces 302. Moreover, the ZNS 302 can be a zoned block command (ZBC) for SAS and / or a zoned-device ATA command set (ZAC) for SATA. Host-side zone activity can be more directly related to media activity in a zoned drive due to the possible relationship between logical activity and physical activity.
[0028] In the data storage device 300, the ZNS 302 is a quantity of NVM that can be formatted into logical blocks such that the capacity is divided into multiple zones 306a-306n (collectively referred to as zones 306). The NVM can be the storage unit or NVM 110 of FIG. 1. Each of the zones 306 comprises multiple physical blocks or erase blocks (not shown) of the memory unit or NVM 304, and each erase block is associated with multiple logical blocks (not shown). Each of the zones 306 can have a size aligned with the capacity of one or more erase blocks of the NVM or NAND device. When the controller 308 receives a command, such as from a host device (not shown) or a submission queue of the host device, the controller 308 can read data from and write data to multiple logical blocks associated with the multiple erase blocks (EB) of the ZNS 302. Each logical block is associated with a unique LBA or sector.
[0029] In one embodiment, the NVM 304 is a NAND device. The NAND device comprises one or more dies. Each of the one or more dies includes one or more planes. Each of the one or more planes comprises one or more erase blocks. Each of the one or more erase blocks comprises one or more word lines (e.g., 256 word lines). Each of the one or more word lines can be addressed in one or more pages. For example, an MLC NAND die can use an upper page and a lower page to reach two bits in each cell of a word line (e.g., 16 KiB per page). Furthermore, each page can be accessed at a granularity of less than or equal to a full page. The controller can frequently access the NAND at a user data granularity logical block address (LBA) size of 512 bytes. Thus, as mentioned in the following description, the location of the NAND corresponds to a granularity of 512 bytes. Thus, a 512 byte LBA size and a 16 KiB page size for two pages of MLC NAND results in 32 LBAs per word line. However, the NAND location sizes are not intended to be limiting and are used merely as examples.
[0030] As data is written to an erase block, one or more logical blocks are correspondingly updated within the zone 306 to track where the data is located within the NVM 304. Data may be written to one zone 306 at a time until the zone 306 is full, or to multiple zones 306 so that the zones 306 are partially full. Similarly, when writing data to a particular zone 306, data may be written to multiple erase blocks one block at a time, page by page, or wordline by wordline, in sequential order of NAND locations, until moving to an adjacent block (i.e., writing to a first erase block until it is full before moving to a second erase block), or may be written to multiple blocks at a time, page by page, or wordline by wordline, in sequential order of NAND locations, to partially fill each block in parallel (i.e., writing to the first NAND location or page of each erase block before writing to the second NAND location or page of each erase block). This sequential programming of all NAND locations is a typical, non-limiting requirement of many NAND EBs.
[0031] When the controller 308 selects an erase block to store data for each zone, the controller 308 can select the erase block at the time of zone opening or when the first word line of that particular erase block needs to be filled. This can be more differentiated when the above-mentioned method of completely filling one erase block before starting the next erase block is utilized. The controller 308 can use the time difference to select a more optimal erase block on a just-in-time basis. The decision of which erase block is allocated and assigned to each zone and its adjacent LBAs can be made within the controller 308 for zero or more simultaneous zones at any one time.
[0032] Each zone 306 is associated with a zone starting logical block address (ZSLBA) or zone starting sector. The ZSLBA is the first available LBA within the zone 306. For example, the first zone 306a is ZSLBA 1. a The second zone 306b is associated with the SLBA. b The third zone 306c is associated with the SLBA. c The fourth zone 306d is associated with the SLBA. d The nth zone 306n (i.e., the last zone) is associated with the SLBA, n Each zone 306 is identified by its ZSLBA and is configured to receive sequential writes (i.e., writing data to the NVM 110 in the order in which the write commands are received).
[0033] As data is written to the zone 306, the write pointer 310 is advanced or updated to point or indicate the next available block in the zone 306 for writing data to keep track of the next write starting point (i.e., the completion point of the previous write is equal to the starting point of the subsequent write). Thus, the write pointer 310 indicates where the subsequent write to the zone 306 will begin. Subsequent write commands are "zone add" commands, where data associated with the subsequent write command is added to the zone 306 at the location the write pointer 310 indicates as the next starting point. An ordered list of LBAs in the zone 306 may be stored for write ordering. Each zone 306 may have its own write pointer 310. Thus, when a write command is received, the zone is identified by its ZSLBA, and the write pointer 310 determines where in the identified zone the writing of the data will begin.
[0034] FIG. 3B is an example diagram of a state diagram 350 of the ZNS 302 of the data storage device 300 of FIG. 3A, according to a particular embodiment. In the state diagram 350, each zone can be in different states, such as empty, active, full, or offline. When a zone is empty, the zone has no data (i.e., none of the erase blocks in the zone currently store data) and the write pointer is at ZSLBA (i.e., WP=0). An empty zone switches to an open and active zone when a write is scheduled to the zone or when a zone open command is issued by the host. Zone management (ZM) commands can be used to move a zone between a zone open state and a zone closed state, both of which are active states. When a zone is active, it has open blocks that can be written to and the host can be provided with a description of the recommended time in the active state. The controller 308 includes the ZM. Zone metadata can be stored in the ZM and / or the controller 308.
[0035] The term "written to" includes programming user data to zero or more NAND locations in an erase block and / or partially filled NAND locations in an erase block when the user data does not fill all available NAND locations. The term "written to" may also include moving a zone full due to internal drive processing needs (open block data retention concerns as bits of error accumulate more quickly on open erase blocks), the data storage device 300 closing or filling a zone due to resource constraints such as too many open zones to keep track of or defect conditions discovered, among others, or the host device closing a zone due to concerns such as no more data to send to the drive, computer shutdown, error processing on the host, limited host resources for tracking, among others.
[0036] An active zone may be open or closed. An open zone is an empty or partially filled zone that is ready to be written to and has resources currently allocated to it. Data received from a host device with a write or add zone command may be programmed into an open erase block that is not currently filled with previous data. A closed zone is an empty or partially filled zone that is not currently receiving a write from a host in progress. Moving a zone from an open state to a closed state allows the controller 308 to reallocate resources to other tasks. These tasks may include, but are not limited to, other zones that are open, other traditional non-zone areas, or other controller needs.
[0037] In both open and closed zones, the write pointer points to a location within the zone somewhere between the ZSLBA and the end of the last LBA of the zone (i.e., WP>0). An active zone can switch between open and closed states as specified by the ZM or when a write is scheduled to the zone. Alternatively, the ZM can reset an active zone, clearing or erasing the data stored in the zone and switching the zone back to an empty zone. When an active zone is filled, it switches to a full state. A full zone is a zone that is completely filled with data and has no more sectors or LBAs available to write data to (i.e., WP=zone capacity, ZCAP). In a full zone, the write pointer points to the end of the writable capacity of the zone. Read commands for data stored in a full zone can still be executed.
[0038] A zone may have any total capacity, such as 256 MiB, 512 MiB, or 2 GiB. However, a small portion of each zone, such as a portion of each zone that stores parity data and one or more set aside erase blocks, may be inaccessible for writing data, but may still be read. For example, if the total capacity of zone 206 is 512 MiB, the ZCAP may be 470 MiB, which is the capacity available for writing data, but 42 MiB is unavailable for writing data. In another embodiment, if the total capacity of zone 306 is 2 GiB, the ZCAP may be 1077 MiB. The writable capacity (ZCAP) of a zone is less than or equal to the total zone storage capacity. Data storage device 300 may determine the ZCAP of each zone upon zone reset. For example, controller 308 or ZM may determine the ZCAP of each zone. Data storage device 300 may determine the ZCAP of a zone when the zone is reset. It should be understood that the total capacitance and ZCAP values listed above are not intended to be limiting, but rather to provide examples of possible embodiments.
[0039] The ZM can reset a full zone and schedule the erasure of data stored in the zone so that the zone reverts to an empty zone. When a full zone is reset, the zone may be marked as an empty zone ready to be written to, but the zone may not be immediately cleared of data. However, a reset zone must be erased before switching to an open and active zone. A zone may be erased any time between a ZM reset and a ZM open. Upon resetting a zone, the data storage device 300 may determine a new ZCAP for the reset zone and update the writeable ZCAP attribute in the zone metadata. An offline zone is a zone that is unavailable for writing data. An offline zone may be in a full, empty, or in an inactive partially full state.
[0040] Resetting a zone clears or schedules the erasure of all data stored in the zone, thus eliminating the need for garbage collection of individual erase blocks and improving the overall garbage collection process of the data storage device 300. The data storage device 300 may mark one or more erase blocks for erasure. When a new zone is to be formed and the data storage device 300 anticipates a ZM open, one or more erase blocks marked for erasure may be erased. The data storage device 300 may further determine and create the physical backing of the zone upon erasure of the erase blocks. Thus, when a new zone is opened and erase blocks are selected to form the zone, the erase blocks are erased. Moreover, each time a zone is reset, a new order of LBAs and write pointer 310 for the zone 306 may be selected, allowing the zone 306 to tolerate receiving commands out of sequential order. The write pointer 310 may optionally be turned off to allow the command to be written to whatever the starting LBA indicated for the command.
[0041] 3A , when the host device 104 sends a write command to write data to the zone 306, the controller 308 retrieves the write command and identifies the write command as a write to the newly opened zone 306. The controller 308 selects a set of EBs to store the data associated with the write command in the newly opened zone 306, and the newly opened zone 306 switches to the active zone 306. The write command may be a command to write new data or a command to move valid data to another zone for garbage collection purposes. The controller 308 is configured to DMA read the new command from the submission queue populated by the host device.
[0042] In the empty zone 306 that has just been switched to the active zone 306, the write pointer 310 points to the logical block associated with ZSLBA as the first available logical block, so that data is allocated to the zone 306 and the associated set of contiguous LBAs of the zone 306 starting from ZSLBA. The data may be written to one or more erase blocks or NAND locations allocated to the physical location of the zone 306. After the data associated with the write command is written to the zone 306, the write pointer 310 is updated to point to the next LBA available for a host write (i.e., the completion point of the first write). The write data from this host write command is programmed contiguously to the next available NAND locations within the erase block selected for the physical backing of the zone.
[0043] For example, the controller 308 may receive a first write command to the third zone 306c or a first add zone command. The host device 104 sequentially identifies which logical block of the zone 306 to write the data associated with the first command to. The data associated with the first command is then written to the first or next available LBA in the third zone 306c as indicated by the write pointer 310, and the write pointer 310 is advanced or updated to point to the next available LBA available for host write (i.e., WP>0). If the controller 308 receives a second write command to the third zone 306c or a second add zone command, the data associated with the second write command is written to the next available LBA in the third zone 306c identified by the write pointer 310. Once the data associated with the second command is written to the third zone 306c, the write pointer 310 is again advanced or updated to point to the next available LBA available for host write. Resetting the third zone 306c causes the write pointer 310 to cThe state is returned to SLBA (ie, WP=0) and the third zone 306c switches to an empty zone.
[0044] For simplicity, the term "erase block" may be referred to as "block" in this specification.
[0045] FIG. 4 is an exemplary diagram of forming zones, such as the first zone 306a of FIG. 3A, in a memory device 400, according to a particular embodiment. The memory device 400 includes a plurality of dies 402a-402n, collectively referred to as dies 402, where each die of the plurality of dies 402a-402n includes a first plane 404a and a second plane 404b, collectively referred to as planes 404. It should be understood that each die includes three or more planes (e.g., four planes, eight planes, etc.). It should be understood that the embodiments herein may be applicable to any die architecture having one or more planes. Each of the planes 404 includes a plurality of blocks 406a-406n, collectively referred to as blocks 406. Although 32 dies 402 are shown in the memory device 400, any number of dies may be included.
[0046] Each die may be separated from adjacent dies to allow for airflow from the airflow source 408, physical components (e.g., wiring), etc. For example, die 410a is at a distance "d" from adjacent die 410b, at a distance "e" from adjacent die 410c, and at a distance "f" from diagonally adjacent die 410d. It should be understood that "d", "e", and "f" may be the same or different. Similarly, in embodiments where dies are stacked, the distance between adjacently stacked dies is considered. In memory device 400, a temperature profile exists between each of the multiple dies 402a-402n. The temperature profile may depend on several factors, such as, for example, the distance between adjacent dies (e.g., "d", "e", and "f"), the proximity of the die to the center of memory device 400, the workload of the die, the distance of the die from airflow source 408, etc. For example, dies in the center of memory device 400 may dissipate heat slower than dies toward the edges of memory device 400. Similarly, a die closer to the airflow source 408 may be cooler than a die further closer to the airflow source 408. In another embodiment, a die having a higher workload may be hotter than a die having a lower workload. The thermal relationship between adjacent die may be predetermined or derived using a fixed benchmark that analyzes the temperature and usage of each die. In one embodiment, analyzing the collected thermal statistics occurs during an offline storage system management period, such as when the memory device 400 is idle.
[0047] 3A includes blocks 406 from each plane 404 of each die 402. In some implementations, a zone may include one or more blocks 406 from each plane 404 of each die 402. Additionally, in some embodiments, one or more of the dies 402 of the memory device 400 may be provisioned to store XOR or parity data. The zones may be formed based on the physical temperature profile of the die 402 such that selected blocks from the die are based on reducing the overall temperature rise of the memory device 400.
[0048] Further, data is written contiguously from block to block in the first zone such that data is written to B0 406a before data is written to B1 406b. Data is also written contiguously from zone to zone such that data is written from the first zone before data is written to the second zone. A zone may have any writeable capacity (ZCAP), such as 256 MiB or 512 MiB, as described above. Each zone of the multiple zones may have the same zone capacity. Data is erased in zone capacity sizes when a data storage device, such as data storage device 106 of FIG. 1, receives a zone reset request (or potentially generates a zone reset request as part of a data management operation, such as garbage collection). In other words, individual blocks cannot be erased unless the entire zone is erased or moved to a zone empty state (i.e., zone empty), as described in FIG. 3B. However, if the data storage device 106 includes non-volatile memory with partial ZNS capabilities, data is erased from the data storage device 106 in zone capacity sizes in the portion of the non-volatile memory that has ZNS capabilities. Data may be erased in block sizes from non-ZNS enabled non-volatile storage units.
[0049] Further, the location of data stored in a ZNS-enabled portion of an NVM, such as NVM 110 of FIG. 1, is recorded in a first logical to physical (L2P) table as an LBA in a volatile memory unit, such as volatile memory 112. The location of data stored in a non-ZNS-enabled portion of an NVM, such as NVM 110 of FIG. 1, is recorded in a second L2P table as an LBA in a volatile memory unit, such as volatile memory 112. Volatile memory 112 may be a DRAM unit. Further, NVM 110 may include a first L2P table that matches the first L2P table of volatile memory 112 and a second L2P table that matches the second L2P table of volatile memory 112. The L2P table in NVM 110 is updated to match the L2P table of volatile memory 112.
[0050] The L2P table contains pointers to each physical location of data in NVM 110. The physical locations of data are mapped to a logical array such that the pointer address array contains locations mapped from the die to NAND locations. In a block, the total number of pointers is calculated as follows: 256WL x 3 pages / WL x 4 slots / page x 1 pointer / slot = 3072 pointers. In a first zone of capacity containing 62 blocks, there may be 190,464 pointers (i.e., 3,072 pointers / block x 62 blocks = 190,464 pointers). Each pointer contains a certain amount of data that utilizes the available storage of volatile memory 112 and / or NVM 110. Although TLC memory is illustrated, the embodiments herein are not limited to TLC memory and the embodiments described herein may be applicable to other memory densities such as, for example, SLC memory, MLC memory, QLC memory, PLC memory, etc.
[0051] FIG. 5 is a flow diagram illustrating a method 500 of providing a zone, such as the first zone 306a of FIG. 3A, to a host device, such as the host device 104 of FIG. 1, according to a particular embodiment. The method 500 can be implemented by a controller, such as the controller 108 of FIG. 1. For illustrative purposes, aspects of the memory device 400 of FIG. 4 and the storage system 100 of FIG. 1 can be referenced herein. At block 502, the controller 108 receives a zone open command from the host device 104. The zone open command can be received explicitly (e.g., specified in the command) or implicitly (e.g., the command does not include a zone open command, but the command is directed to a zone that has not yet been opened). At block 504, the controller 108 analyzes die thermal statistics. For example, the controller 108 can utilize the thermal statistics accumulation module 150 to accumulate thermal statistics for each die and analyze each die, thereby generating a thermal profile for the die. At block 506, the controller 108 determines which group of dies to assign a zone to. In block 508 , the controller 108 provides the newly opened zone to the host device 104 .
[0052] In another embodiment, the physical location of the conventional namespace may be selected based on the die temperature profile. The conventional namespace is the non-ZNS namespace in a ZNS device. The conventional namespace may be a very small area where random writes are allowed. Conventional namespaces may be located on groups of die, much like zones, and may be moved from die group to die group based on the thermal statistics of the die.
[0053] FIG. 6 is a schematic block diagram illustrating a storage system 600 including a controller 606 coupled to a plurality of data storage devices 614a-614n, according to a particular embodiment, the controller 606 including a load balancer 608 and a thermal statistics accumulator / analyzer module 610 (which may be separate components). The controller 606 further includes a flash interface module (FIM) 612. In one embodiment, the thermal statistics accumulator / analyzer module 610 may be located within the FIM 612. The storage system 600 may be the storage system 200 of FIG. 2, where the controller 606 manages input / output traffic from one or more host devices to the plurality of data storage devices 614a-614n. Each of the data storage devices 614a-614n may be the data storage device 106 of FIG. 1. It should be understood that the one or more host devices may not be directly coupled, but may be linked via a network structure.
[0054] The controller 606 is coupled to the host device 602, where the host device 602 includes a command queue 604. The host device 602 generates commands and stores the generated commands in the command queue 604. In some embodiments, the host device 602 may send instructions to the controller 606 to fetch the generated commands stored in the command queue 604. In other embodiments, the host device 602 may send the generated commands from the command queue 604 to the controller 606.
[0055] The load balancer 608 may be configured to balance the future workload and thermal load of each of the data storage devices 614a-614n. Additionally, the thermal statistics accumulator / analyzer module 610 may be configured to collect and analyze thermal statistics from each of the plurality of data storage devices 614a-614n. For example, the analysis may also include determining the thermal impact of allocating commands from the command queue 604 to the data storage device and how allocating commands from the command queue 604 to the data storage device may affect the temperature profile of adjacent data storage devices. Based on the results of the thermal statistics accumulator / analyzer module 610, the load balancer 608 transmits the received commands from the command queue 604 to the FIM 612, where the FIM 612 passes the commands to the target data storage device.
[0056] In one embodiment, each data storage device of the plurality of data storage devices 614a-614n may also include a thermal statistics accumulator / analyzer module such that the thermal statistics accumulator / analyzer module of each data storage device sends feedback to the host device 602. The host device 602 may modify the command queue 604 based on the feedback by adding or removing jobs or commands based on the thermal load of each data storage device.
[0057] FIG. 7 is a flow diagram illustrating a method 700 for rebalancing access queues of a data storage device, such as one of the plurality of data storage devices 614a-614n of FIG. 6, according to a particular embodiment. The method 700 may be implemented by the controller 606 of FIG. 6. For illustrative purposes, aspects of the storage system 600 of FIG. 6 may be referenced herein. At block 702, the controller 606 receives a future workload from the host device 602. At block 704, the controller 606 receives current thermal statistics from the thermal statistics accumulator / analyzer module 610. At block 706, the controller 606 rebalances the access queues of the plurality of data storage devices 614a-614n according to the workload and the current thermal statistics.
[0058] Rebalancing the access queues of the multiple data storage devices 614a-614n may be done using different approaches based on optimization techniques. For example, the optimization may follow a greedy schedule where the thermal requirements of each data storage device are satisfied one by one. If the thermal requirements of a data storage device are not satisfied in the current schedule, the data storage device is pushed to the next slot in the access queue that satisfies the thermal requirements of the data storage device. The controller 606 may also check that by pushing a data storage device to the access queue, other previously satisfied thermal requirements of the data storage device are no longer satisfied. The controller 606 may also take into account latency priorities, such that a relatively "hot" data storage may not be the last data storage device to be accessed. For example, latency priorities may be implemented using a token-based system, whereby each data storage device waiting in the access queue receives a token that increases its priority in the next time slot.
[0059] 8 is a flow diagram illustrating a method 800 for processing write commands in a storage system, such as storage system 600 of FIG. 6, with load balancing based on thermal statistics and future workload, according to a particular embodiment. Aspects of storage system 600 of FIG. 6 and storage system 100 of FIG. 1 may be referenced herein for illustrative purposes. For example, method 800 may be implemented by controller 606.
[0060] At block 802, the controller 606 receives a write command from the host device 602 to write data to a zone of one of the plurality of data storage devices 614a-614n. At block 804, the controller 606 accumulates thermal statistics for each of the plurality of data storage devices 614a-614n. In some embodiments, the accumulation / collection of thermal statistics occurs continuously. At block 806, the controller 606 determines a current workload for each of the plurality of data storage devices 614a-614n. At block 808, the controller 606 selects one of the plurality of data storage devices 614a-614n based on the thermal statistics and the workload of each data storage device. The selected data storage device may be based on a thermal requirements optimization scheme, such as the rebalancing scheme described above.
[0061] In block 810, the controller 606 determines whether the selected data storage device has an open zone. If the selected data storage device has an open zone in block 810, then the controller 606 sends a write command to the controller of the data storage device, and in block 818, the controller of the data storage device programs the data of the write command into the open zone. However, if the selected data storage device does not have an open zone in block 810, then the controller 606 sends a zone open command to the selected data storage device in block 812. In block 814, the controller of the selected data storage device accumulates thermal statistics of the multiple dies. In block 816, the controller of the selected data storage device opens a new zone based on the thermal statistics of the multiple dies. In block 818, the controller 606 sends a write command to the controller of the data storage device, and the controller of the data storage device programs the data of the write command into the open zone.
[0062] System performance of a storage system can be improved by balancing workloads and opening new zones based on thermal statistics of each data storage device in a storage system including multiple data storage devices and multiple dies within each memory device of each data storage device.
[0063] In one embodiment, a storage system includes two or more data storage devices, each data storage device of the two or more data storage devices including a Zone Name Space (ZNS) architecture, and a controller coupled to the two or more data storage devices, wherein the controller is configured to collect thermal statistics for each data storage device of the two or more data storage devices, analyze the collected thermal statistics, and designate a zone by selecting one or more dies in at least one data storage device of the two or more data storage devices based on the analyzed collected thermal statistics.
[0064] The controller is further configured to receive a zone open command and program the host data into the specified zone. Receiving the zone open command occurs prior to analyzing the collected thermal statistics. Collecting the thermal statistics occurs continuously. Analyzing the collected thermal statistics occurs during an offline storage system management period. The selected one or more dies have a first thermal statistic and the remaining one or more dies have a second thermal statistic. The first thermal statistic is less than the second thermal statistic. The first thermal statistic and the second thermal statistic are a temperature profile. The controller is further configured to select the one or more dies based on a predicted workload. The two or more data storage devices are arranged in rows and columns to allow airflow therebetween. A first data storage of the two or more data storage devices has a first thermal statistic and a second data storage of the two or more data storage devices has a second thermal statistic. The first thermal statistic is different from the second thermal statistic. The first data storage device is located a first distance from an airflow source and the second data storage device is located a second distance from the airflow source. The first distance and the second distance are different. Analyzing the collected thermal statistics includes analyzing thermal statistics of the first data storage device and at least one of thermal statistics of the second data storage device and thermal statistics of the third data storage device. The second data storage device is disposed directly adjacent to the first data storage device and the third data storage device is disposed diagonally adjacent to the first data storage device. The second data storage device is a plurality of data storage devices and the third data storage device is a plurality of data storage devices. Each data storage device of the two or more data storage devices includes a plurality of dies. The controller is configured to select one or more of the plurality of dies to form the zone based on the collected thermal statistics of the plurality of dies.
[0065] In another embodiment, a data storage device includes a memory device having a Zone Name Space (ZNS) architecture, the memory device including a plurality of dies, and a controller coupled to the memory device, the controller configured to collect thermal statistics for each die of the plurality of dies, analyze the collected statistics, and allocate one or more dies of the plurality of dies to form zones.
[0066] The thermal statistics for each die of the plurality of dies are based at least on a usage of the die, a proximity of the die to an airflow source of the data storage device, and a temperature of a die adjacent to the die. The controller is further configured to provide the collected thermal statistics to the host device, receive a workload based on the provided collected thermal statistics, and rebalance an access queue of the data storage device based on the received workload and the analyzed collected thermal statistics. The controller is further configured to receive a workload type indication from the host device, receive thermal statistics from a thermal statistics accumulation module disposed within the controller, and rebalance an access queue of the data storage device based on the received workload type indication and the analyzed collected thermal statistics.
[0067] In another embodiment, a storage system includes two or more memory means and a controller coupled to the two or more memory means, the controller configured to allocate zones formed from one or more portions of the two or more memory means based on thermal statistics of each memory means of the two or more memory means.
[0068] The storage system is an enterprise storage system. The storage system is a client storage system.
[0069] While the above is directed to embodiments of the present disclosure, other and further embodiments of the present disclosure may be devised without departing from the basic scope thereof, which scope is determined by the following claims.
Claims
1. 1. A storage system comprising: two or more data storage devices, each data storage device of the two or more data storage devices comprising a Zone Name Space (ZNS) architecture, the two or more data storage devices being arranged in rows and columns to enable airflow therebetween; a controller coupled to the two or more data storage devices, the controller comprising: collecting thermal statistics for each data storage device of the two or more data storage devices; analyzing the collected thermal statistics, where analyzing the collected thermal statistics includes determining a thermal impact that an allocation of commands to a data storage device has on thermal statistics of data storage devices adjacent to the data storage device; and a storage system configured to designate a zone by selecting one or more dies within at least one data storage device of the two or more data storage devices based on the analyzed collected thermal statistics.
2. The controller: Receives a zone open command 2. The storage system of claim 1, further configured to program host data into the designated zone.
3. The storage system of claim 2 , wherein receiving the zone open command occurs prior to analyzing the collected thermal statistics.
4. The storage system of claim 1 , wherein said collecting thermal statistics occurs continuously.
5. 2. The storage system of claim 1, wherein analyzing the collected thermal statistics occurs during an offline storage system administration period.
6. 2. The storage system of claim 1, wherein the selected one or more dies have first thermal statistics and the remaining one or more dies have second thermal statistics, the first thermal statistics being less than the second thermal statistics, and the first thermal statistics and the second thermal statistics are a temperature profile.
7. The storage system of claim 1 , wherein the controller is further configured to select the one or more dies based on a predicted workload.
8. a first data storage device of the two or more data storage devices having a first thermal statistic and a second data storage device of the two or more data storage devices having a second thermal statistic; 2. The storage system of claim 1, wherein the first thermal statistics are different from the second thermal statistics.
9. the first data storage device is located a first distance from an air flow source and the second data storage device is located a second distance from the air flow source; 9. The storage system of claim 8, wherein the first distance and the second distance are different.
10. 2. The storage system of claim 1, wherein analyzing the collected thermal statistics comprises analyzing at least one of thermal statistics of a first data storage device, thermal statistics of a second data storage device, and thermal statistics of a third data storage device, wherein the second data storage device is located immediately adjacent to the first data storage device and the third data storage device is located diagonally adjacent to the first data storage device.
11. 11. The storage system of claim 10, wherein the second data storage device is a plurality of data storage devices and the third data storage device is a plurality of data storage devices.
12. 2. The storage system of claim 1, wherein each data storage device of the two or more data storage devices includes a plurality of dies, and the controller is configured to select one or more of the plurality of dies to form the zones based on collected thermal statistics of the plurality of dies.
13. 1. A data storage device, comprising: A memory device having a Zone Name Space (ZNS) architecture, the memory device including a plurality of dies arranged in rows and columns to allow airflow therebetween; a controller coupled to the memory device, the controller comprising: collecting thermal statistics for each die of the plurality of dies; analyzing the collected thermal statistics, where analyzing the collected thermal statistics includes determining a thermal impact that an allocation of commands to a data storage device has on thermal statistics of data storage devices adjacent to the data storage device; A data storage device configured to allocate one or more of the plurality of dies to form zones.
14. 14. The data storage device of claim 13, wherein the thermal statistics for each die of the plurality of dies are based at least on a usage of the die, a proximity of the die to an airflow source of the data storage device, and a temperature of a die adjacent to the die.
15. The controller: providing the collected thermal statistics to a host device; receiving a workload based on the collected thermal statistics provided; The data storage device of claim 13 , further configured to rebalance an access queue of the data storage device based on the received workload and the collected thermal statistics.
16. The controller: receiving a workload type indication from a host device; receiving the thermal statistics from a thermal statistics accumulation module located within the controller; 14. The data storage device of claim 13, further configured to rebalance an access queue of the data storage device based on the received workload type indication and the collected thermal statistics.
17. 1. A storage system comprising: two or more memory means arranged in rows and columns to permit air flow therebetween; a controller coupled to the two or more memory means, the controller configured to allocate zones formed from portions of one or more of the two or more memory means based on an analysis of collected thermal statistics of each memory means of the two or more memory means, the analysis of the collected thermal statistics including determining a thermal impact that an allocation of commands to one memory means of the two or more memory means has on thermal statistics of memory means adjacent to the one memory means of the two or more memory means.
18. 20. The storage system of claim 17, wherein the storage system is an enterprise storage system.
19. 20. The storage system of claim 17, wherein the storage system is a client storage system.
Citation Information
Patent Citations
Information processing unit, control method thereof and program
JP2014106917A
Semiconductor device
JP2016212745A
MEMORY ALLOCATION AND MEMORY WRITE REDIRECTION IN A CLOUD COMPUTING SYSTEM BASED ON THE TEMPERATURE OF A MEMORY MODULE
JP2023533445A
Storage device
US20170003889A1
Zoned Namespaces in Solid-State Drives
US20210089217A1
