Disable dynamic program suspension for random write SSD workloads.

Abstract

The present invention provides an electronic device, an electronic storage system, and a method for improving read operation latency when accessing NAND-based storage media. [Solution] The storage control method includes: a controller controlling access to a NAND-based storage medium including multiple NAND devices; disabling program suspension operations only for a specific NAND device if the current workload for a particular NAND device among the multiple NAND devices is a random write workload; and enabling program suspension operations only for the specific NAND device if a host read is pending for the specific NAND device.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims the benefit of U.S. Patent Application No. 17 / 241,976, filed on April 27, 2021, which is hereby incorporated by reference in its entirety.

Background Art

[0002] In certain NAND media devices, it may be possible to access the NAND device or a NAND array within the NAND device at once with either a single read or a single program operation command (i.e., write command). Here, the NAND array may include a series of NAND strings, and the NAND strings may include a plurality of NAND cells. The program operation may be interrupted temporarily to accommodate a pending read operation, and after the read operation is completed, the program operation can be resumed. This is called program operation interruption - resumption. The latency of the program operation may be one order of magnitude greater than that of the read operation. To improve the latency of the read operation, in a NAND solid - state drive (SSD), the program operation suspension - resumption (PSR) function of the NAND device is utilized to interrupt (pause) an ongoing program operation on the NAND array, advance one or more queued read operations, and after these are completed, resume the interrupted program operation. In a program operation on a NAND device or a NAND array within the NAND device, a write operation is performed on the NAND media. A program operation may be performed to program (i.e., write) to NAND such as multi - level cell (MLC) NAND. The program operation may also be performed on other types of NAND (e.g., quad - level cell (QLC), triple - level cell (TLC), single - level cell (SLC), etc.) and other types of non - volatile memory.

Brief Description of the Drawings

[0003] The subject matter described herein is illustrated as an example and is not intended as an limitation in the accompanying drawings. For the sake of brevity and clarity, the elements depicted in the drawings are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Furthermore, where appropriate, reference numerals are repeated throughout the drawings to indicate corresponding or similar elements. The drawings are as follows:

[0004] [Figure 1] This is a block diagram relating to an example of an electronic storage system according to one embodiment.

[0005] [Figure 2] This is a block diagram relating to an example of an electronic device according to one embodiment.

[0006] [Figure 3A] This is a flowchart illustrating an example of a storage control method according to one embodiment. [Figure 3B] This is a flowchart illustrating an example of a storage control method according to one embodiment.

[0007] [Figure 4] This is a flowchart illustrating another example of a storage control method according to one embodiment.

[0008] [Figure 5] This is a flowchart illustrating another example of a storage control method according to one embodiment.

[0009] [Figure 6] This is an explanatory diagram comparing the timing of various NAND operations according to one embodiment with a baseline.

[0010] [Figure 7] This is a block diagram relating to another example of a computing system according to one embodiment.

[0011] [Figure 8] This is a block diagram relating to an example of a solid-state drive (SSD) device according to one embodiment. [Modes for carrying out the invention]

[0012] Referring to the accompanying drawings, one or more embodiments or implementations are described herein. Specific configurations and arrangements are described, but it should be understood that these are for illustrative purposes only. Those skilled in the art will recognize that other configurations and arrangements may be used without departing from the spirit and scope of this specification. Those skilled in the art will also see that the methods and / or arrangements described herein may be used in a variety of other systems and applications not described herein.

[0013] The following description includes various implementation examples that may be explicitly shown in architectures such as a system-on-a-chip (SoC) architecture, but the implementation of the methods and / or arrangements described herein is not limited to any particular architecture and / or computing system, and may be implemented in any architecture and / or computing system for similar purposes. For example, the methods and / or arrangements described herein may be implemented in various architectures using multiple integrated circuit (IC) chips and / or packages, and / or in various computing devices and / or consumer electronic (CE) devices such as set-top boxes and smartphones. Furthermore, the following description may include numerous specific details, such as examples of logic implementations, types and interrelationships of system components, and choices of logic partitioning / integration, but the claimed subject matter can be implemented without such specific details. In other examples, some subject matter, such as control structures and complete software instruction sequences, may not be shown in detail in order to avoid obscuring the subject matter disclosed herein.

[0014] The subject matter disclosed herein may be implemented in hardware, firmware, software, or any combination thereof. The subject matter disclosed herein may also be implemented as instructions stored in a machine-readable medium that can be read and executed by one or more processors. A machine-readable medium may include any medium and / or mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device). For example, machine-readable mediums may include read-only memory (ROM); random-access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; propagating signals in electrical, optical, acoustic, or other forms (e.g., carrier waves, infrared signals, digital signals, etc.), and others.

[0015] References in this specification such as “one implementation example,” “one implementation example,” and “an exemplary implementation example” indicate that the implementation example described may include certain features, structures, or characteristics, but not all embodiments necessarily include such specific features, structures, or characteristics. Furthermore, such language does not necessarily refer to the same implementation example. Moreover, if a certain feature, structure, or characteristic is described in relation to one embodiment, it is stated that such features, structures, or characteristics may be obtained in relation to other implementation examples, whether or not they are explicitly described herein, as being within the knowledge of those skilled in the art.

[0016] Various embodiments described herein may include memory components and / or interfaces to memory components. Such memory components may include volatile and / or non-volatile (NV) memory. Volatile memory may be a storage medium that requires power to maintain the state of data stored in it. Non-limiting examples of volatile memory may include various types of RAM, such as dynamic RAM (DRAM) or static RAM (SRAM). Certain types of DRAM may be used in memory modules, such as synchronous dynamic RAM (SDRAM). NV memory (NVM) may be a storage medium that does not require power to maintain the state of data stored in it. In one embodiment, a memory device may include a block-addressable memory device, for example, one based on NAND technology. In one embodiment, a memory device may include, or may not include, a memory device using NAND flash memory or other memory with multiple threshold levels. A memory device may refer to the die itself and / or a packaged memory product.

[0017] Referring to Figure 1, one embodiment of the electronic storage system 10 may include a NAND-based storage medium 12 containing a plurality of NAND devices, and a controller 11 communicatively connected to the NAND-based storage medium 12. For example, the NAND devices may correspond to a single NAND die. The controller 11 may include a circuit 13 that determines whether the current workload for a particular NAND device among the plurality of NAND devices is a random write workload, and if so, disables the program suspension operation for that particular NAND device only. In some embodiments, the circuit 13 may further determine whether a host read is pending for a particular NAND device, and if so, enable the program suspension operation for that particular NAND device only.

[0018] For example, circuit 13 may be configured to maintain a command queue (e.g., a NAND device queue (NDQ)) for each of a plurality of NAND devices, maintain a counter value associated with each command queue, and count each host read for each command queue. In some embodiments, circuit 13 may further be configured to increment a counter value associated with the command queue of a particular NAND device when a command from a host read is placed in the command queue of that particular NAND device, and to decrement a counter value associated with the command queue of a particular NAND device when a command from a host read is cleared from the command queue of that particular NAND device. For example, circuit 13 may be configured to determine that the current workload for a particular NAND device among a plurality of NAND devices is a random write workload if the counter value associated with the command queue of a particular NAND device is zero, and to determine that a host read is pending for that particular NAND device if the counter value associated with the command queue of that particular NAND device is non-zero. In any of the embodiments herein, the controller 11 and the NAND-based storage medium 12 may be incorporated into a solid-state drive (SSD).

[0019] Each embodiment of the controller 11, the NAND-based storage medium 12, the circuit 13, and other system components may be implemented in hardware, software, or any preferred combination thereof. For example, hardware implementations may include configurable logic such as programmable logic arrays (PLAs), field-programmable gate arrays (FPGAs), and composite programmable logic devices (CPLDs), or fixed-function logic hardware using circuit technologies such as application-specific integrated circuits (ASICs), complementary metal-oxide-semiconductor (CMOS) or transistor-transistor logic (TTL) technology, or any combination thereof. Embodiments of the controller 11 may include general-purpose controllers, special-purpose controllers, memory controllers, storage controllers, microcontrollers, general-purpose processors, special-purpose processors, central processor units (CPUs), execution units, and the like. In some embodiments, the NAND-based storage medium 12, the circuit 13, and / or other system memory may be located within various components, including the controller 11, or in the same location (e.g., on the same die).

[0020] Alternatively, or additionally, all or part of these components may be implemented in one or more modules as a set of logic instructions stored in a machine-readable or computer-readable storage medium (e.g., RAM, ROM, programmable ROM (PROM), firmware, flash memory, etc.) executed by a processor or computing device. For example, computer program code for performing the operations of the components may be written in any combination of one or more programming languages ​​applicable to the operating system (OS), including object-oriented programming languages ​​such as Python®, Perl, Java®, Smalltalk®, C++, C#, and traditional procedural programming languages ​​such as the C programming language or similar programming languages. For example, a NAND-based storage medium 12, another NAND-based storage medium, or other system memory may, when executed by the controller 11, store a set of instructions that cause the system 10 to implement one or more components, features, or aspects of the system 10 (e.g., circuit 13 determining whether the current workload for a particular NAND device is a random write workload, or disabling program suspension operations only for a particular NAND device).

[0021] Referring now to FIG. 2, one embodiment of the electronic device 14 may include one or more substrates 15, and a controller 16 coupled to the one or more substrates 15. The controller 16 may control access to a NAND-based storage medium including a plurality of NAND devices, determine whether a current workload for a particular NAND device among the plurality of NAND devices is a random write workload, and if so determined, may include a circuit 17 that disables the program suspend operation only for the particular NAND device. In some embodiments, the circuit 17 may further determine whether a host read for a particular NAND device is pending, and if so determined, may be configured to enable the program suspend operation only for the particular NAND device.

[0022] For example, the circuit 17 may be configured to maintain a command queue for each of the plurality of NAND devices, maintain a counter value associated with each of the command queues to count each respective host read for each of the command queues. In some embodiments, the circuit 17 may further be configured to increment a counter value associated with the command queue of a particular NAND device when a command from a host read is placed in the command queue of the particular NAND device, and decrement a counter value associated with the command queue of the particular NAND device when a command from a host read is erased from the command queue of the particular NAND device. For example, the circuit 17 may be configured to determine that the current workload for a particular NAND device among the plurality of NAND devices is a random write workload when a counter value associated with the command queue of the particular NAND device is zero, and determine that a host read for the particular NAND device is pending when a counter value associated with the command queue of the particular NAND device is non-zero. In any of the embodiments herein, the controller and the NAND-based storage medium may be incorporated into a solid state drive (SSD).

[0023] Embodiments of circuit 17 may be implemented in a system, apparatus, computer, device, etc. (e.g., those described herein). More specifically, examples of hardware implementations of circuit 17 may include configurable logic such as PLA, FPGA, CPLD, or fixed function logic hardware using circuit technologies such as ASIC, CMOS, or TTL technology, or any combination thereof. Alternatively, or additionally, circuit 17 may be implemented as a set of logic instructions stored in a machine-readable or computer-readable storage medium (e.g., RAM, ROM, PROM, firmware, flash memory, etc.) executed by a processor or computing device, in one or more modules. For example, the computer program code for executing the operations of the components may be described in any combination of one or more programming languages applicable / appropriate to the OS, including object-oriented programming languages such as Python®, Perl, Java®, Smalltalk®, C++, C#, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages.

[0024] For example, circuit 17 may be implemented in a semiconductor device, which may include one or more substrates 15, to which circuit 17 is coupled. In some embodiments, circuit 17 may be at least partially implemented in one or more of configurable logic and fixed function hardware logic on a semiconductor substrate (e.g., silicon, sapphire, gallium arsenide, etc.). For example, circuit 17 may include a transistor array and / or other integrated circuit components coupled to a substrate 15 having a channel region of transistors disposed within substrate 15. The interface between circuit 17 and substrate 15 may not be a step junction. Circuit 17 may be considered to include an epitaxial layer grown on the initial wafer of substrate 15.

[0025] Referring now to Figures 3A and 3B, one embodiment of the storage control method 20 may include, in block 21, a step of controlling access to a NAND-based storage medium including a plurality of NAND devices; in block 22, a step of determining whether the current workload for a particular NAND device among the plurality of NAND devices is a random write workload; and, if so, a step of disabling program suspension operations for that particular NAND device only in block 23. Some embodiments of the method 20 may further include, in block 24, a step of determining whether a host read is pending for a particular NAND device; and, if so, a step of enabling program suspension operations for that particular NAND device only in block 25.

[0026] For example, Method 20 may also include the steps of maintaining a command queue for each of several NAND devices in block 26, and maintaining a counter value associated with each command queue in block 27 to count each host read for each command queue. Some embodiments of Method 20 may further include the steps of incrementing a counter value associated with a command queue for a particular NAND device in block 28 when a command from a host read is placed in the command queue for that particular NAND device, and decrementing a counter value associated with a command queue for a particular NAND device in block 29 when a command from a host read is cleared from the command queue for that particular NAND device. For example, Method 20 may also include the steps of determining in block 30 that the current workload for a particular NAND device among several NAND devices is a random write workload if the counter value associated with the command queue for that particular NAND device is zero, and / or determining in block 31 that a host read is pending for that particular NAND device if the counter value associated with the command queue for that particular NAND device is non-zero. In any of the embodiments described herein, the NAND-based storage medium may be incorporated into the SSD in block 32.

[0027] Embodiments of Method 20 may be implemented, for example, in systems, apparatus, computers, devices, etc., as described herein. More specifically, hardware implementation examples of Method 20 may include configurable logic such as PLA, FPGA, CPLD, or coarse-grained reconfigurable fabric (CGRA), or fixed-function logic hardware using circuit technology such as ASIC, CMOS, or TTL technology, or any combination thereof. Alternatively or additionally, Method 20 may be implemented in one or more modules as a set of logic instructions stored in a machine-readable or computer-readable storage medium (e.g., RAM, ROM, PROM, firmware, flash memory, etc.) executed by a processor or computing device. For example, computer program code for performing the operation of the components may be written in any combination of one or more OS-applicable / appropriate programming languages, including object-oriented programming languages ​​such as Python®, Perl, Java®, Smalltalk®, C++, C#, and conventional procedural programming languages ​​such as the C programming language or similar programming languages.

[0028] For example, Method 20 may be implemented in a computer-readable medium as described in relation to Examples 22-28 below. Embodiments or parts of Method 20 may be implemented in firmware, applications (e.g., through an application programming interface (API)), or driver software running on an operating system (OS). Furthermore, logic instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, state setting data, configuration data for integrated circuits, and state information that personalizes other structural components specific to electronic circuits and / or hardware (e.g., host processor, central processing unit / CPU, microcontroller, etc.).

[0029] Advantageously, in some embodiments, techniques can be provided to optimize the random write performance of an SSD. In some NAND media systems, it is possible to access the NAND array at once with either a program command (i.e., a write) or a read command. Since program latency can be an order of magnitude larger than read latency, some NAND-based SSDs may leverage the program pause-resume (PSR) function of the NAND-based SSD to improve host read latency (e.g., an important metric for SSD users), which allows an ongoing program operation to be paused (hidden), and then resumed after one or more queued host reads have completed.

[0030] One obstacle to random write workload throughput in SSDs is NAND programming time (tProg). During a random write workload, the SSD internally reads invalid data from the NAND array, programs it to a new location, and frees up new unwritten area. The problem is that the interruption of program operations caused by internal reads during a random write workload increases the time required to complete the program command, thus contributing to overhead that hinders random write workload throughput. Fortunately, some embodiments provide techniques to improve the random write performance of SSDs with little to no impact on host read latency.

[0031] In some embodiments, techniques are provided to erase or disable PSR commands during random write workloads (as opposed to any other workloads). Advantageously, in some embodiments, it becomes possible to reduce or minimize the effective tProg, thereby increasing the utilization of SSD controller bandwidth available for NAND device operations. Some embodiments can be effectively utilized for random write workloads because, since there are no host reads during random write workloads, the SSD can improve or optimize the workload for host throughput during random write workloads while ensuring no impact on host read latency for other workloads. Advantageously, some embodiments can be implemented using relatively low-complexity techniques that provide significant benefits to random write throughput without negatively impacting key SSD metrics (such as host read latency in random read and any mixed read / write workloads).

[0032] In random write workloads on NAND-type SSDs, internal reads by the SSD controller occur to move data from one NAND location to a new NAND location for the purpose of garbage collection of NAND that was written to by the host but is no longer valid. This allows the invalid NAND to be reused for new writes. In the garbage collection process, the SSD controller typically reads the remaining valid host write data from the target eviction NAND device to the SSD controller, so that the host write data can be corrected with an error correction code (ECC) algorithm and then scheduled to be written to the new NAND location. Advantageously, in some embodiments, internal reads targeting NAND devices are distinguished from host reads targeting NAND devices, and PSR operations are enabled only if host reads exist in the target NDQ.

[0033] The SSD controller channel may refer to a dedicated direct memory access (DMA) engine that moves data between the SSD controller and the NAND device. Channel overhead may refer to any additional, non-essential DMA or status command traffic that consumes channel bandwidth.

[0034] In some embodiments, PSR operations are disabled only for random write workloads, which is advantageous in that it improves random write throughput by reducing or minimizing channel overhead and increases or maximizes throughput by reducing effective tProg. However, disabling PSR for other workloads, especially mixed workloads, causes increased read latency because the program prior to the read must complete before sending out the read. In some embodiments, PSR operations are dynamically enabled or disabled so that host read latency is not affected by mixed read / write workloads. Advantageously, to improve the throughput of random write workloads, the PSR overhead is also reduced or eliminated from the random write workload.

[0035] In some embodiments, dynamic PSR disabling is performed locally at the NAND device level rather than at the overall SSD level. Advantageously, such localized dynamic PSR disabling can scale to any random write workload that may be localized to only a subset of NAND devices within the SSD. Non-limiting examples of cases where a random write workload may be localized to only a subset of NAND devices include when the SSD implements an NVM Express (NVMe) set, NVMe Endurance Groups exposing NAND partitions to the host, etc.; when the SSD implements internal NAND partitioning in the form of media banking; when the host workload is bursty in its random writes; or when the SSD implements any kind of NVMe host specification that allows any kind of accurate or inaccurate host write data placement that would expose some but not all NAND devices to internal reads for garbage collection (for example, or even for media policy reasons).

[0036] In some embodiments, techniques may be provided to dynamically disable PSR on a per-NAND device basis, but only for localized random write workloads. Similarly (for example, additionally or alternatively), in some embodiments, techniques may be provided to enable PSR only when a NAND device is subject to a host read and / or has pending host reads.

[0037] SSDs may incorporate a technique that provides a software queue that records NAND commands sent to NAND devices while the NAND devices are busy with ongoing commands. In some implementations, the software queue may be called the NAND device queue (NDQ). In some embodiments, a counter (e.g., implemented in the SSD controller) is provided that is maintained to track the number of host reads in the NDQ. Hereafter, this counter will be called the host read counter (HRC). In some embodiments, the HRC may be the most important mechanism used to determine whether NDQ management enables or disables PSR operations for NAND devices associated with the NDQ. Each NDQ has its own counter instance, so that each NAND device has a localized workload targeting the NAND device. For example, if an SSD has 10 NAND devices attached to the SSD controller, there will be 10 NDQs and therefore 10 HRCs, with one HRC per NDQ. Furthermore, depending on the SSD architecture and host workload, each NDQ may experience different workloads. In that sense, in the remainder of this specification relating to embodiments, workload refers to a workload localized to each NAND device.

[0038] Referring to Figure 4, one embodiment of the storage control method 40 may be called NANDCmdEnQueuetoNDQ. When the SSD controller determines the target NAND device for a new NAND command, the SSD controller is ready to place the command into the target NAND device queue. Incoming new NAND commands are added to the NDQ in the square frame 41. In the square frame 42, if the new NAND command being placed into the NDQ is a NAND command generated from a host read command, the HRC in the square frame 43 is incremented for the associated NDQ. If the NAND read command is generated internally by the SSD controller, for example, for garbage collection, the counter is not incremented. Later, when a NAND read command generated from a host read is cleared from the NDQ, the HRC is decremented (see Figure 5).

[0039] Referring to Figure 5, one embodiment of the storage control method 50 may be called NANDCmdDispatchtoNDQ. When any NAND read request (e.g., internal or host) reaches the beginning of the NDQ in box 51 (e.g., when a read request is about to be sent to the SSD controller channel and therefore to the target NAND device), the HRC of that NDQ is evaluated in box 52. If the HRC in box 52 is zero (e.g., meaning the current workload is a random write workload), the PSR in box 53 is invalid for this command. In this case, the NAND read command should be an internal SSD controller read and not a host read, so if there is any pending program on the target NAND device before the (internal) read can be sent to the target NAND device, it will be forced to wait until the ongoing program is completed.

[0040] If the HRC in box 52 is not zero (for example, meaning that the current workload includes a NAND command generated from a host read command), then a PSR is enabled for this command in box 54. If a program is continuing on the target device associated with the NDQ, the PSR operation is enabled because the NDQ may be a mixture of both host and internal read commands, and a program suspend operation may be issued for each PSR policy that may allow the continuing program to be paused and then the NAND read command to be sent after the program has been suspended. Method 50 can then proceed to the step of determining in box 55 whether the NAND command was sent from a host read, and if so, to decrement the HRC in box 56.

[0041] Disabling PSR when there are no host reads in the NDQ and enabling PSR when there are host reads in the NDQ dynamically ensures that the read latency benefits of host read commands in pure read and mixed read / write workloads are maintained, while also ensuring that PSR operations and associated channel overhead are eliminated from random write workloads. Advantageously, in some embodiments, significant improvements are observed in input / output operations per second (IOPS), effective tProg, and random write workload performance compared to a similarly configured baseline without using localized dynamic PSR disabling.

[0042] Referring to Figure 6, a timing diagram 60 for illustrative purposes compares one embodiment using localized dynamic PSR disabling technique for a single NAND device with a baseline. As shown in Figure 6, the embodiment using localized dynamic PSR disabling technique has virtually eliminated PSR overhead, while the baseline takes more time for host writes compared to the embodiment using localized dynamic PSR disabling technique. The improvement in tProg may be due to the reduction in PSR overhead, which cannot interrupt the progress of a program while a read is in progress.

[0043] The technologies described herein may be provided to a variety of computing systems, including non-portable computing devices such as desktops, workstations, servers, and rack systems; portable computing devices such as smartphones, tablets, ultra-portable personal computers (UMPCs), laptop computers, Ultrabook computing devices, smartwatches, smart glasses, and smart bracelets; and / or client / edge devices such as Internet of Things (IoT) devices (e.g., sensors, cameras, etc.).

[0044] Referring here to Figure 7, one embodiment of the computing system 100 may include one or more processors 102-1 to 102-N (hereinafter generally referred to as “multiple processors 102” or “processor 102”). The multiple processors 102 can communicate with each other via an interconnection or bus 104. Each processor 102 may include various components, and for clarity only some of them will be described with reference to processor 102-1. Thus, each of the remaining processors 102-2 to 102-N may include the same or similar components as described with reference to processor 102-1.

[0045] In some embodiments, the processor 102-1 may include one or more processor cores 106-1 to 106-M (referred to here as “multiple cores 106” or more generally “core 106”), a cache 108 (which may be a shared cache or a private cache in various embodiments), and / or a router 110. The processor cores 106 may be implemented on a single integrated circuit (IC) chip. The chip may further include one or more shared caches and / or private caches (such as cache 108), a bus or interconnect (such as bus or interconnect 112), a memory controller, or other components.

[0046] In some embodiments, routers 110 may be used to communicate between various components of the processor 102-1 and / or system 100. Furthermore, the processor 102-1 may include more than one router 110. In addition, multiple routers 110 may communicate to enable data routing between various components inside or outside the processor 102-1.

[0047] Cache 108 may store data (e.g., instructions) used by one or more components of processor 102-1 (such as core 106). For example, cache 108 may locally cache data stored in memory 114 for faster access by components of processor 102. As shown in Figure 7, memory 114 can communicate with processor 102 via interconnect 104. In some embodiments, cache 108 (which may be shared) may have various levels; for example, cache 108 may be an intermediate-level cache and / or a last-level cache (LLC). Also, each of core 106 may include a level 1 (L1) cache (116-1) (commonly referred to here as "L1 cache 116"). Various components of processor 102-1 can communicate directly with cache 108 through a bus (e.g., bus 112) and / or a memory controller or hub.

[0048] As shown in Figure 7, the memory 114 may be connected to other components of the system 100 through the memory controller 120. The memory 114 may include volatile memory and may be referred to without distinction as main memory or system memory. Although the memory controller 120 is shown as being connected between the interconnect 104 and the memory 114, the memory controller 120 may be located elsewhere in the system 100. For example, in some embodiments, the memory controller 120 or a part of it may be located on one of the multiple processors 102.

[0049] System 100 can communicate with other devices / systems / networks via a network interface 128 (for example, communicating with a computer network and / or cloud 129 via a wired or wireless interface). For example, the network interface 128 may include an antenna (not shown) for wirelessly communicating with the network / cloud 129 (for example, via an IEEE 802.11 interface (including IEEE 802.11a / b / g / n / ac, etc.), a cellular interface, 3G, 4G, LTE, Bluetooth®, etc.).

[0050] System 100 may also include storage devices such as SSDs 130 connected to interconnect 104 via SSD controller logic 125. Thus, logic 125 can control access to SSDs 130 by various components of System 100. Furthermore, even though Figure 7 shows logic 125 directly connected to interconnect 104, logic 125 can alternatively communicate with one or more other components of System 100 via storage buses / interconnections (e.g., SATA (Serial Advanced Technology Attachment) bus, Peripheral Component Interconnect (PCI) (or PCI Express (PCIe) interface), NVM Express (NVMe), etc.) (for example, if the storage bus is connected to interconnect 104 via some other logic such as a bus bridge, chipset, etc.). Furthermore, the logic 125 may be incorporated into memory controller logic (for example, as described with reference to Figure 8), or it may be provided on the same integrated circuit (IC) device (for example, on the same circuit board device as the SSD 130, or in the same enclosure as the SSD 130) in various embodiments.

[0051] Furthermore, the logic 125 and / or SSD 130 may be coupled to one or more sensors (not shown) to receive information (e.g., in the form of one or more bits or signals) indicating the status or value of a value detected by one or more sensors. These sensors may be located in close proximity to components of system 100 (or other computing systems described herein), including core 106, interconnect 104 or 112, components outside the processor 102, SSD 130, SSD bus, SATA bus, logic 125, circuit 160, etc., to detect fluctuations in various factors affecting the power / thermal behavior of the system / platform (e.g., temperature, operating frequency, operating voltage, power consumption, and / or inter-core communication activity).

[0052] Figure 8 illustrates a block diagram of various components of the SSD 130 according to one embodiment. As shown in Figure 8, the circuit 160 may be located in various positions, such as inside the SSD 130 or the controller 382, ​​and may include similar techniques described in relation to Figure 7. The SSD 130 includes a controller 382 (including one or more processor cores or processors 384 and memory controller logic 386), a cache 138, RAM 388, firmware storage 390, and one or more NAND devices 392-1 to 392-N (collectively referred to as NAND media 392). The NAND media 392 is coupled to the memory controller logic 386 via one or more memory channels or buses. The SSD 130 also communicates with logic 125 via an interface (such as SATA, SAS, PCIe, NVMe, etc.). The processor 384 and / or controller 382 can compress / decompress data written to or read from NAND devices 392-1 to 392-N.

[0053] As illustrated in Figures 7 and 8, the SSD 130 may include circuit 160, which may be in the same enclosure as the SSD 130 and / or fully integrated on the printed circuit board (PCB) of the SSD 130. One or more of the features / aspects / operations described with reference to Figures 1 to 6 may be performed by one or more of the components in Figures 7 and / or 8. Also, one or more of the features / aspects / operations in Figures 1 to 6 may be programmed into firmware 390. Furthermore, the SSD controller logic 125 may also include circuit 160. Advantageously, circuit 160 may include techniques for implementing one or more aspects of any of the systems 10 (Figure 1), apparatus 14 (Figure 2), methods 20 (Figures 3A to 3B), methods 40 (Figure 4), methods 50 (Figure 5), timing diagrams 60 (Figure 6), and / or features described herein.

[0054] For example, circuit 160 may be configured to determine whether the current workload for a particular NAND device among one or more NAND devices 392-1 to 392-N is a random write workload, and if so, disable the program suspension operation (or, for example, the PSR operation) for that particular NAND device only. Advantageously, disabling the PSR improves tProg, resulting in improved I / O or random write performance per second. In some embodiments, circuit 160 may further determine whether a host read is pending for a particular NAND device, and if so, enable the program suspension operation for that particular NAND device only.

[0055] For example, circuit 160 may be configured to maintain an NDQ (e.g., a command queue) for each of one or more NAND devices 392-1 to 392-N, and to maintain an HRC (e.g., a counter value) associated with each NDQ, thereby counting each host read for each NDQ. In some embodiments, circuit 160 may further be configured to increment the HRC associated with the NDQ of a particular NAND device when a command from a host read is placed in the NDQ of that particular NAND device, and to decrement the HRC associated with the NDQ of a particular NAND device when a command from a host read is cleared from the NDQ of that particular NAND device. For example, circuit 160 may be configured to determine that the current workload for a particular NAND device among one or more NAND devices 392-1 to 392-N is a random write workload if the HRC associated with the NDQ of a particular NAND device is zero, and to determine that there is a pending host read for that particular NAND device if the HRC associated with the NDQ of that particular NAND device is non-zero.

[0056] In other embodiments, the SSD 130 may be replaced with any suitable storage / memory / technology / medium. In some embodiments, the circuit 160 may be coupled to one or more substrates (e.g., silicon, sapphire, gallium arsenide, printed circuit board (PCB), etc.) and may include channel areas of transistors located on one or more substrates. In other embodiments, the SSD 130 may include two or more types of storage media. For example, the majority of the storage may be NAND, and may further include some faster, finer-grained (e.g., byte-addressable) NVM. The SSD 130 may optionally or additionally include persistent volatile memory (e.g., DRAM or SRAM with power backed up by a battery or capacitor). For example, the SSD 130 may include power loss protection (PLI) technology with energy storage capacitors. The energy storage capacitors can supply enough energy (power) to complete any ongoing commands and ensure that any data in the DRAM / SRAM is recorded to the non-volatile NAND medium. The capacitor can function as a backup battery for persistent volatile memory. As shown in Figures 7 and 8, features or aspects of circuit 160 may be distributed throughout system 100 and / or located in the same place as / integrated with various components of system 100.

[0057] [Additional points to note and examples]

[0058] Embodiment 1 includes an electronic device which includes one or more circuit boards and a controller connected to one or more circuit boards, the controller which controls access to a NAND-based storage medium which includes multiple NAND devices, determines whether the current workload for a particular NAND device among the multiple NAND devices is a random write workload, and if so, includes circuitry which disables program suspension operations only for that particular NAND device.

[0059] Example 2 includes the apparatus described in Example 1, and the circuit further determines whether a host read is pending for a particular NAND device, and if so, enables a program suspension operation only for that particular NAND device.

[0060] Example 3 includes the apparatus described in Example 2, and the circuit further maintains a command queue for each of the multiple NAND devices, maintains a counter value associated with each command queue, and counts each host read for each command queue.

[0061] Embodiment 4 includes the apparatus described in Embodiment 3, wherein the circuit further increments a counter value associated with the command queue of a particular NAND device when a command from a host read is placed in the command queue of that particular NAND device, and decrements the counter value associated with the command queue of that particular NAND device when a command from a host read is cleared from the command queue of that particular NAND device.

[0062] Example 5 includes the apparatus described in Example 4, and the circuit further determines that the current workload for a particular NAND device among several NAND devices is a random write workload if the counter value associated with the command queue of that particular NAND device is zero.

[0063] Example 6 includes the apparatus described in any of Examples 4 to 5, and the circuit further determines that a host read is pending for a particular NAND device if the counter value associated with the command queue of that particular NAND device is non-zero.

[0064] Example 7 includes the apparatus described in any of Examples 1 to 6, wherein the controller and NAND-based storage medium are incorporated into a solid-state drive.

[0065] Embodiment 8 includes an electronic storage system, which includes a NAND-based storage medium containing a plurality of NAND devices, and a controller communicatively connected to the NAND-based storage medium, the controller including a circuit that determines whether the current workload for a particular NAND device among the plurality of NAND devices is a random write workload, and if so, disables program suspension operations for that particular NAND device only.

[0066] Example 9 includes the system described in Example 8, and the circuit further determines whether a host read is pending for a particular NAND device, and if so, enables a program suspension operation only for that particular NAND device.

[0067] Example 10 includes the system described in Example 9, and the circuit further maintains a command queue for each of the multiple NAND devices, maintains a counter value associated with each command queue, and counts each host read for each command queue.

[0068] Example 11 includes the system described in Example 10, wherein the circuit further increments a counter value associated with the command queue of a particular NAND device when a command from a host read is placed in the command queue of that particular NAND device, and decrements the counter value associated with the command queue of that particular NAND device when a command from a host read is cleared from the command queue of that particular NAND device.

[0069] Example 12 includes the system described in Example 11, and the circuit further determines that the current workload for a particular NAND device among several NAND devices is a random write workload if the counter value associated with the command queue of that particular NAND device is zero.

[0070] Example 13 includes the system described in any of Examples 11 to 12, and the circuit further determines that a host read is pending for a particular NAND device if the counter value associated with the command queue of that particular NAND device is non-zero.

[0071] Example 14 includes the system described in any of Examples 8 to 13, wherein the controller and NAND-based storage medium are incorporated into a solid-state drive.

[0072] Embodiment 15 includes a method for controlling storage, the method comprising the steps of controlling access to a NAND-based storage medium including multiple NAND devices, determining whether the current workload for a particular NAND device among the multiple NAND devices is a random write workload, and, if determined to be so, disabling program suspension operations for that particular NAND device only.

[0073] Example 16 includes the method described in Example 15, and further comprises the steps of determining whether a host read is pending for a particular NAND device, and, if so, enabling a program suspension operation only for that particular NAND device.

[0074] Example 17 includes the method described in Example 16, further comprising the steps of maintaining a command queue for each of a plurality of NAND devices, and maintaining a counter value associated with each of the command queues to count each host read for each command queue.

[0075] Embodiment 18 includes the method described in Embodiment 17, and further comprises the steps of: incrementing a counter value associated with the command queue of a particular NAND device when a command from a host read is placed in the command queue of that particular NAND device; and decrementing a counter value associated with the command queue of a particular NAND device when a command from a host read is cleared from the command queue of that particular NAND device.

[0076] Example 19 includes the method described in Example 18, and further comprises the step of determining that the current workload for a particular NAND device among multiple NAND devices is a random write workload if the counter value associated with the command queue of that particular NAND device is zero.

[0077] Example 20 includes the method described in any of Examples 18 to 19, and further comprises the step of determining that a host read is pending for a particular NAND device if the counter value associated with the command queue of that particular NAND device is non-zero.

[0078] Example 21 includes the method described in any of Examples 15 to 20, wherein the NAND-based storage medium is incorporated into a solid-state drive.

[0079] Embodiment 22 includes at least one non-temporary machine-readable medium containing a plurality of instructions, which, in response to execution on the computing device, cause the computing device to control access to a NAND-based storage medium containing a plurality of NAND devices, determine whether the current workload on a particular NAND device among the plurality of NAND devices is a random write workload, and if so, disable a program suspension operation for that particular NAND device only.

[0080] Example 23 includes at least one non-temporary machine-readable medium as described in Example 22, the medium including a plurality of further instructions, which, in response to execution on the computing device, cause the computing device to determine whether a host read is pending for a particular NAND device, and if so, enable a program suspension operation for that particular NAND device only.

[0081] Example 24 includes at least one non-temporary machine-readable medium as described in Example 23, the medium including a plurality of further instructions, which, in response to execution on the computing device, cause the computing device to maintain a command queue for each of a plurality of NAND devices, maintain a counter value associated with each of the command queues, and count each host read for each of the command queues.

[0082] Example 25 includes at least one non-temporary machine-readable medium as described in Example 24, the medium including a plurality of further instructions, which, in response to execution on the computing device, cause the computing device to increment a counter value associated with the command queue of a particular NAND device when a command from a host read is placed in the command queue of that particular NAND device, and to decrement a counter value associated with the command queue of a particular NAND device when a command from a host read is cleared from the command queue of that particular NAND device.

[0083] Example 26 includes at least one non-temporary machine-readable medium as described in Example 25, the medium including a plurality of further instructions, which, in response to execution on the computing device, cause the computing device to determine that the current workload for a particular NAND device among a plurality of NAND devices is a random write workload if the counter value associated with the command queue of that particular NAND device is zero.

[0084] Example 27 includes at least one non-temporary machine-readable medium described in any of Examples 25 to 26, the medium including a plurality of further instructions, which, in response to execution on the computing device, cause the computing device to determine that a host read is pending for a particular NAND device if a counter value associated with a command queue for that particular NAND device is non-zero.

[0085] Example 28 includes at least one non-temporary machine-readable medium described in any of Examples 22 to 27, wherein the NAND-based storage medium is incorporated into a solid-state drive.

[0086] Embodiment 29 includes a storage controller device, which comprises means for controlling access to a NAND-based storage medium including a plurality of NAND devices, means for determining whether the current workload for a particular NAND device among the plurality of NAND devices is a random write workload, and, if it is determined to be so, means for disabling program suspension operations only for that particular NAND device.

[0087] Example 30 includes the apparatus described in Example 29, and further comprises means for determining whether a host read is pending for a particular NAND device, and, if so, means for enabling a program suspension operation only for that particular NAND device.

[0088] Example 31 includes the apparatus described in Example 30, and further comprises means for maintaining command queues for each of a plurality of NAND devices, and means for maintaining counter values ​​associated with each command queue and for counting each host read for each command queue.

[0089] Embodiment 32 includes the apparatus described in Embodiment 31, and further comprises means for incrementing a counter value associated with the command queue of a specific NAND device when a command from a host read is placed in the command queue of that NAND device, and means for decrementing a counter value associated with the command queue of a specific NAND device when a command from a host read is cleared from the command queue of that NAND device.

[0090] Example 33 includes the apparatus described in Example 32, and further comprises means for determining that the current workload for a particular NAND device among multiple NAND devices is a random write workload when the counter value associated with the command queue of that particular NAND device is zero.

[0091] Example 34 includes the apparatus described in any of Examples 32 to 33, and further comprises means for determining that a host read is pending for a particular NAND device if the counter value associated with the command queue of that particular NAND device is non-zero.

[0092] Example 35 includes the apparatus described in any of Examples 29 to 34, wherein the NAND-based storage medium is incorporated into a solid-state drive.

[0093] The term “connected” may be used herein to refer to any kind of direct or indirect relationship between the components in question, and may apply to electrical, mechanical, fluid, optical, electromagnetic, electromechanical, or other connections. Furthermore, terms such as “first,” “second,” etc., may be used herein solely for the sake of clarity and, unless otherwise indicated, do not have any specific temporal or chronological meaning.

[0094] In this application and claims, a series of items leading to the term “one or more of ~” may mean any combination of the terms described. For example, the phrase “one or more of A, B, and C” and the phrase “one or more of A, B, or C” may both mean A; B; C; A and B; A and C; B and C; or A, B, and C. Various components of the system described herein may be implemented by software, firmware, and / or hardware, and / or any combination thereof. For example, various components of the system or device described herein may be provided at least in part by hardware such as a computing SoC, which may be found in a computing system such as a smartphone. A person skilled in the art will recognize that the system described herein may include additional components not shown in the corresponding figures. For example, the system described herein may include additional components such as a bitstream multiplexer or demultiplexer module, which are not shown for clarity.

[0095] The implementation of the exemplary processes described herein may include performing all operations shown in the illustrated order, but the disclosure is not limited thereto. In various examples, the implementation of the exemplary processes herein may include only a subset of the operations shown, operations performed in a different order than illustrated, or additional operations.

[0096] Furthermore, one or more of the operations described herein may be performed in response to instructions provided by one or more computer program products. Such program products may include a signal-carrying medium that, when executed by a processor, provides instructions that can produce the functions described herein. The computer program products may be provided in one or more machine-readable media of any form. Thus, for example, a processor including one or more image processing devices or processor cores may execute one or more of the exemplary process blocks described herein in response to program code and / or instructions or instruction sets sent to the processor by one or more machine-readable media. Generally, the machine-readable media may carry software in the form of program code and / or instructions or instruction sets that can implement at least some of the operations described herein and / or any part of the devices, systems, or any module or component described herein in any of the devices, systems, or any modules or components described herein.

[0097] In any implementation described herein, the term “module” means any combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide the functionality described herein. Software may be embodied as software packages, code, and / or instruction sets or instructions, and in any implementation described herein, “hardware” may include, for example, hardwired circuitry, programmable circuitry, state machine circuitry, fixed-function circuitry, execution unit circuitry, and / or firmware storing instructions executed by programmable circuitry, either alone or in any combination. These modules may be embodied together or individually as circuitry that forms part of a larger system (e.g., integrated circuits (ICs) and system-on-a-chip (SoCs)).

[0098] Various embodiments may be implemented using hardware elements, software elements, or a combination of both. Examples of hardware elements may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, and inductors), integrated circuits, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), logic gates, registers, semiconductor devices, chips, microchips, and chipsets. Examples of software may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application programming interfaces (APIs), instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. The determination of whether an embodiment is implemented using hardware and / or software elements may depend on any number of factors, such as the desired computation rate, power level, thermal tolerance, processing cycle budget, input data rate, output data rate, memory resources, data bus speed, and other design or performance constraints.

[0099] One or more aspects of at least one embodiment may be implemented by representative instructions, stored in a machine-readable medium, that represent various logics within a processor, which, when read by a machine, cause the machine to create logic for performing the techniques described herein. Such representations are known as IP cores and may be stored in a tangible machine-readable medium and supplied to various customers or manufacturing facilities for loading into manufacturing machines that actually produce the logic or processor.

[0100] While certain features described herein have been explained with reference to various implementation examples, this explanation is not intended to be constrained. Therefore, various modifications of the implementation examples described herein and other implementation examples, though obvious to those skilled in the art to whom this disclosure relates, are considered to be within the spirit and scope of this disclosure.

[0101] These embodiments are not limited to those described herein, but it will be recognized that they may be implemented with modifications and alterations without departing from the scope of the attached claims. For example, the embodiments may include certain combinations of features. However, the embodiments are not limited in this respect, and in various implementations, the embodiments may include the implementation of only a subset of such features, the implementation of such features in different orders, the implementation of such features in different combinations, and / or the implementation of additional features beyond those explicitly described. Accordingly, the scope of these embodiments should be determined by referring to the attached claims together with the entire scope of the equivalents to which such claims are entitled.

Claims

1. One or more substrates; and A controller connected to the aforementioned one or more substrates, the controller includes, Control access to NAND-based storage media containing multiple NAND devices, Determine whether the current workload on a specific NAND device among the aforementioned multiple NAND devices is a random write workload, and if so, Disable program suspension operation only for the aforementioned specific NAND devices. Includes a circuit An electronic device equipped with the following features.

2. The aforementioned circuit further, Determine whether a host read is pending for the aforementioned specific NAND device; if so, The electronic device according to claim 1, wherein the program suspension operation is enabled only for the specified NAND device.

3. The aforementioned circuit further, Maintain the command queues of each of the aforementioned multiple NAND devices; The electronic device according to claim 2, which maintains a counter value associated with each of the command queues and counts each host read for each of the command queues.

4. The aforementioned circuit further, When a command from a host read is placed in the command queue of the particular NAND device, the counter value associated with the command queue of the particular NAND device is incremented; The electronic device according to claim 3, wherein when the command from the host read is cleared from the command queue of the particular NAND device, the counter value associated with the command queue of the particular NAND device is decremented.

5. The aforementioned circuit further, The electronic device according to claim 4, wherein if the counter value associated with the command queue of the particular NAND device is zero, it is determined that the current workload for the particular NAND device among the plurality of NAND devices is the random write workload.

6. The aforementioned circuit further, The electronic device according to claim 4, wherein if the counter value associated with the command queue of the specific NAND device is non-zero, it is determined that the host read is pending for the specific NAND device.

7. The electronic device according to any one of claims 1 to 6, wherein the controller and the NAND-based storage medium are incorporated into a solid-state drive.

8. NAND-based storage media containing multiple NAND devices; and A controller is connected to the NAND-based storage medium in a communication manner, and the controller has: Determine whether the current workload on a specific NAND device among the aforementioned multiple NAND devices is a random write workload, and if so, Disable program suspension operation only for the aforementioned specific NAND device. Includes a circuit An electronic storage system equipped with the following features.

9. The aforementioned circuit further, Determine whether a host read is pending for the aforementioned specific NAND device; if so, The electronic storage system according to claim 8, wherein the program suspension operation is enabled only for the specified NAND device.

10. The aforementioned circuit further, Maintain the command queues of each of the aforementioned multiple NAND devices; The electronic storage system according to claim 9, which maintains a counter value associated with each of the command queues and counts each host read for each of the command queues.

11. The aforementioned circuit further, When a command from a host read is placed in the command queue of the particular NAND device, the counter value associated with the command queue of the particular NAND device is incremented; The electronic storage system according to claim 10, wherein when the command from the host read is cleared from the command queue of the particular NAND device, the counter value associated with the command queue of the particular NAND device is decremented.

12. The aforementioned circuit further, The electronic storage system according to claim 11, wherein if the counter value associated with the command queue of the particular NAND device is zero, it is determined that the current workload for the particular NAND device among the plurality of NAND devices is the random write workload.

13. The aforementioned circuit further, The electronic storage system according to claim 11, wherein if the counter value associated with the command queue of the specific NAND device is non-zero, it is determined that the host read is pending for the specific NAND device.

14. The electronic storage system according to any one of claims 8 to 13, wherein the controller and the NAND-based storage medium are incorporated into a solid-state drive.

15. A storage control method, A step in controlling access to a NAND-based storage medium containing multiple NAND devices; A step of determining whether the current workload for a particular NAND device among the plurality of NAND devices is a random write workload; and if it is determined to be so, The step of disabling the program suspension operation only for the aforementioned specific NAND device. A method that includes [a certain feature].

16. A step of determining whether a host read is pending for the aforementioned specific NAND device; and if so, Steps to enable the program suspension operation only for the specified NAND device. The method according to claim 15, further comprising:

17. The step of maintaining the command queues for each of the multiple NAND devices; and A step in which a counter value associated with each of the command queues is maintained, and each of the host reads is counted for each of the command queues. The method according to claim 16, further comprising:

18. When a command from a host read is placed in the command queue of the particular NAND device, the step of incrementing the counter value associated with the command queue of the particular NAND device; and When the command from the host read is cleared from the command queue of the specific NAND device, the counter value associated with the command queue of the specific NAND device is decremented. The method according to claim 17, further comprising:

19. The method according to claim 18, further comprising the step of determining that the current workload for the particular NAND device among the plurality of NAND devices is the random write workload if the counter value associated with the command queue of the particular NAND device is zero.

20. The method according to claim 18, further comprising the step of determining that a host read is pending for the particular NAND device if the counter value associated with the command queue for the particular NAND device is non-zero.

Images