SYSTEM AND METHOD FOR ADAPTIVE COMMAND REQUEST AGGREGATION

Abstract

Procedures, exhibiting: Receiving an indication from a host device (300; 400) of one or more commands available for retrieval from at least one transmission queue (304; 410), wherein the host device (300; 400) has caused the one or more commands to be stored on the at least one transmission queue (304; 410) so that a storage device (100; 420) can process them; and in response to receiving the display, by the storage device (100; 420): accessing a retrieval threshold, wherein the retrieval threshold indicates a number of commands or a waiting time; Determine whether the retrieval threshold has been reached; in response to the determination that the retrieval threshold has been reached, retrieving one or more commands from the at least one transmission queue (304; 410); and in response to the determination that the retrieval threshold has not been reached, wait to retrieve one or more commands from the at least one transmission queue (304; 410).

Description

REFERENCE TO RELATED REGISTRATIONS

[0001] This application claims priority over US patent application no. 15 / 468,620, filed on March 24, 2017, and US patent application no. 15 / 613,795, filed on June 5, 2017. BACKGROUND

[0002] NVM Express (NVMe) is a standard for accessing non-volatile storage media connected via a PCI Express (PCIe) bus. NVMe can be used with a variety of non-volatile storage media, such as solid-state drives (SSDs). A key focus of NVMe is the I / O communication between a host device (which can access and / or write to the non-volatile storage medium) and a storage device (which includes the non-volatile storage medium). In this context, NVMe implements a paired send and exit queue mechanism, where host software on the host device places commands into the send queue and then notifies the storage device of the commands entered into the send queue. In response to the notification, the storage device retrieves the command from the send queue.

[0003] Furthermore, the disclosures in DE 10 2015 005 744 A1, US 2002 / 0 108 005 A1 and US 6 065 089 A may be helpful for understanding the present invention.

[0004] German patent DE 10 2015 005 744 A1 discloses techniques for peer-to-peer Peripheral Component Interconnect Express (PCIe) memory transfers. In some embodiments, the techniques are implementable as a method for providing peer-to-peer memory transfers between PCIe devices, providing a queue for data communicated between a first PCIe device and a target PCIe device in a memory element of the first PCIe device, receiving queue memory element allocation information transmitted by a host device that is communicatively coupled to the first PCIe device and the target PCIe device, at the first PCIe device, and generating a memory instruction using a computer processor of the first PCIe device.

[0005] US 2002 / 0108005A1 describes a PCI (Peripheral Component Interconnect) network adapter that manages read / write requests by establishing dynamic queues. The PCI network adapter establishes a unique queue for each destination node, allowing requests for each node to be processed separately. The PCI network adapter determines whether a remote read / write request should be added to the linked list for the destination node or rejected. If the number of pending requests for the destination node is below a predefined threshold and the total buffer is not full, the request is added to the linked list for the destination node. Otherwise, the request is rejected. For write requests, all pending read requests for that node are canceled when the request is added to the linked list for the destination node.

[0006] US 6,065,089 A relates to a method and device for generating an interrupt signal. A counter value is decremented each time a task is completed by a slave processor. The counter value is incremented each time a task is read from the slave processor. A delay value is set using the counter value. After a time interval defined by the delay value, an interrupt is generated. The counter value is compared to a threshold value. The interrupt is not generated after the time interval has elapsed, but rather when a condition is detected in which the counter value is below the threshold or the completion queue is full. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings, which are included in and form part of this description, illustrate various aspects of the invention and, together with the description, serve to explain its principles. Wherever appropriate, the same reference numerals are used in all drawings to denote identical or similar elements. Fig. Figure 1A is a block diagram of an example non-volatile storage system. Fig. Figure 1B is a block diagram of a memory module that includes a variety of non-volatile memory systems and a host. Fig. 1C is a block diagram of a hierarchical storage system. Fig. 2A is a block diagram of exemplary components of a non-volatile memory system controller. Fig. 1A. Fig. 2B is a block diagram of exemplary components of a non-volatile memory chip of the non-volatile memory system from Fig. 1A. Fig. Figure 3A is a block diagram of the host device and NVMe control, illustrating a sequence for a host device and a storage device to request and process an NVMe command. Fig. 3B is an illustration of the transmission queue and the termination queue. Fig. Figure 4 is a block diagram with further exemplary components of a host system and a storage device. Fig. Figure 5 is a block diagram illustrating the transmission queue and retrieval coalescence. Fig. Figure 6 is a flowchart of a first exemplary retrieval coalescence procedure. Fig. Figure 7 is a flowchart of a second exemplary retrieval coalescence procedure. Fig. Figure 8 is a flowchart of an exemplary procedure for iteratively selecting a threshold for retrieval coalescence. DETAILED DESCRIPTION OF THE EXECUTION FORMS Overview

[0008] As explained in more detail below, the host device causes commands to be sent to the transmission queue (for example, by direct transmission for a transmission queue located on the host device or by indirect transmission for a transmission queue located on the storage device). The host device then notifies the storage device of the commands in the transmission queue. In response to the notification, the storage device retrieves the commands from the transmission queue and processes them. In this respect, the host device possesses the intelligence up to the point at which the storage device retrieves the command. Specifically, the point at which the storage device retrieves the command from the transmission queue is directly dependent on the point at which the host sends the notification.

[0009] In one implementation, the storage device includes the intelligence, separate from or in addition to the notification provided by the host device, of when to retrieve commands from the delivery queue. Specifically, the storage device may use one or more retrieval criteria or one or more retrieval thresholds, which are further explained below, to determine when to retrieve commands in response to a notification from the host device about commands in the delivery queue. As explained below, the retrieval criteria or thresholds may be pre-programmed in the storage device and / or dynamically determined by the storage device. For example, the retrieval criteria or threshold may be directed to a specific number of commands to retrieve.In particular, upon receiving a notification from the host device, the storage device can compare the number of commands to be retrieved against the retrieval threshold. If the number of commands to be retrieved is greater than the retrieval threshold, the storage device retrieves the commands from the delivery queue(s). If the number of commands to be retrieved is less than the retrieval threshold, the storage device can wait (for example, a predetermined amount of time or until the host device sends another notification about the commands to be retrieved) before retrieving the commands from the delivery queue(s). As another example, the retrieval criteria or the retrieval threshold can be time-based.In particular, in response to receiving a notification from the host device, the storage device can calculate a time (such as the time since the storage device last retrieved instructions from a delivery queue, or the time since the storage device last received notification from the host device about instructions to retrieve from the delivery queue) and compare the calculated time to the retrieval threshold. If the calculated time is greater than the retrieval threshold, the storage device retrieves the instructions from the delivery queue(s).If the calculated time is less than the retrieval threshold, the storage device can wait (for example, wait until the retrieval threshold time is reached or wait until the host device sends another notification of commands to be retrieved) before retrieving the commands from the delivery queue(s). As explained below, the retrieval criteria or threshold can apply to a single delivery queue (for example, a retrieval threshold for delivery queue A to determine whether the number of commands to be retrieved from delivery queue A is greater than the retrieval threshold), to a group of delivery queues (for example, all delivery queues classified as delivery queues), or to all delivery queues.

[0010] In a first specific implementation, the storage device can consider one or more factors dependent on the current state of the host device or the storage device when determining the retrieval criteria or retrieval threshold. For example, the storage device can consider one or more aspects of the transmit queue and / or the termination queue (e.g., the number of currently processed commands, the number of commands in the termination queue, etc.). As another example, the storage device can consider the state of the communication interface between the storage device and the host device (e.g., the level of activity on the input bus to the storage device). As yet another example, the storage device can consider the state of the storage device or the host device.In particular, the storage device can determine its processing capacity (e.g., whether the storage device has excess or insufficient processing capacity). In response to this determination, the storage device can determine whether the number of instructions required to retrieve the one or more instructions from the transmission queue should be greater than or equal to the retrieval threshold. Thus, in response to the determination that the storage device has sufficient processing capacity, the storage device can determine that the number of instructions required to retrieve the one or more instructions from the transmission queue does not need to be greater than or equal to the retrieval threshold.Or, in response to the determination that the storage device has a lower processing capacity, the storage device can determine that the number of instructions to be retrieved must be greater than or equal to the retrieval threshold before it retrieves the one or more instructions from the delivery queue. Similarly, the storage device can determine a state of the host device and specify whether the retrieval criteria or threshold(s) should be implemented accordingly. In particular, the storage device can determine the processing capacity of the host device based on the host device's response time to an interruption (as in step 6 in [reference]). Fig. 3A). In this way, the storage device can dynamically determine whether the retrieval criteria or the threshold(s) should be implemented.

[0011] In a second specific implementation, the storage device can consider one or more factors when determining the retrieval criteria or retrieval threshold, based on an analysis of a previous state of the host device or the storage device. For example, the storage device can analyze one or more patterns of previous notifications from the host device regarding the delivery queue, such as the timing between previous host device notifications, the number of commands typically retrieved with previous notifications, and so on. In this context, one of the retrieval criteria can be a retrieval wait time, as indicated by the timing between previous host device notifications.For example, the retrieval wait time can include the time for the storage device to wait after receiving a notification that a command is available to be retrieved from one or more delivery queues.

[0012] In a third specific implementation, the storage device can consider one or more factors independently of an analysis of the current or previous state of the host device or the storage device. For example, the storage device can access one or more predefined retrieval parameters, such as waiting a specified time after notifying the host device or waiting until a specified number of instructions are to be retrieved from the delivery queue. Thus, one of the retrieval criteria can include a retrieval threshold regarding the specified number of instructions to be retrieved from the delivery queue.In this respect, adding intelligence to the storage device (separate from or in addition to the intelligence on the host device) regarding when to retrieve commands can implement the process of retrieving commands from the transmission queue more efficiently. Designs

[0013] The following embodiments describe non-volatile memory devices and associated methods for processing instructions. Before referring to these and other embodiments, the following paragraphs provide a discussion of exemplary non-volatile memory devices and memory modules that may be used with these embodiments. Of course, these are only examples, and other suitable types of non-volatile memory devices and / or memory modules may be used.

[0014] Fig. Figure 1A is a block diagram illustrating a non-volatile memory device 100. The non-volatile memory device 100 may include a controller 102 and non-volatile memory, which may consist of one or more non-volatile memory chips 104. As used here, the term element refers to the set of non-volatile memory cells and associated switching logic for managing the physical operation of these non-volatile memory cells, which are formed on a single semiconductor substrate. The controller 102 may interface with a host device or host system and transmit command sequences for read, program, and erase operations to the non-volatile memory chip(s) 104. As explained below, the commands may include logical and / or physical addresses.

[0015] The Controller 102 (which may be a flash memory controller) can take the form of a processing circuit, a microprocessor or processor, and a computer-readable medium that stores computer-readable program code (e.g., software or firmware) that can be executed by the (micro)processor, logic gates, switches, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. The Controller 102 can be configured with hardware and / or firmware to perform the various functions described below and shown in the flowcharts. Furthermore, some of the components shown as being inside the controller may also be stored outside of the controller, and other components may be used externally.Additionally, the phrase “operationally in communication with” could mean directly in communication with, or indirectly (wired or wirelessly) in communication with, one or more components that may or may not be shown or described here.

[0016] As used herein, a flash memory controller is a device that manages data stored on flash memory and communicates with a host, such as a computer or other electronic device. A flash memory controller can have various functions in addition to the specific functionality described here. For example, the flash memory controller can format the flash memory to ensure that the memory is functioning properly, to weed out bad flash memory cells, and to allocate spare cells to replace future faulty cells. Some portions of the spare cells can be used to contain firmware to operate the flash memory controller and implement other features. One example of firmware is a flash translation layer.When a host device needs to read data from or write data to flash memory during operation, the host communicates with the flash memory controller. If the host provides a logical address to which data should be read / written, the flash memory controller can translate the logical address received from the host into a physical address within the flash memory controller. The flash memory controller can also perform various memory management functions, such as, but not limited to, wear balancing (distributing write operations to avoid wear on specific memory blocks that would otherwise be repeatedly written to) and garbage collection (once a block is full, moving only the valid data pages to a new block so that the full block can be erased and reused).

[0017] The interface between the controller 102 and the non-volatile memory chip 104 can be any suitable flash interface, such as toggle mode 200, 400, or 800. In one embodiment, the storage device 100 can be a card-based system, such as a secure digital (SD) or a secure digital micro card (microSD). In an alternative embodiment, the non-volatile storage device 100 can be part of an embedded storage device.

[0018] Although the non-volatile storage device 100 in the in Fig. Although the illustrated example in Figure 1A may include a single channel between the controller 102 and the non-volatile memory chip(s) 104, the subject matter described herein is not limited to a single memory channel. For example, in some NAND memory device architectures, 2, 4, 8, or more NAND channels may exist between the controller and the NAND memory chip(s) 104, depending on the capabilities of the controller. In each of the embodiments described herein, more than a single channel may exist between the controller and the memory chip(s) 104, even if a single channel is shown in the drawings.

[0019] Fig. Figure 1B illustrates a memory module 200 that includes multiple non-volatile storage devices 100. As such, the memory module 200 can include a memory controller 202 connected to a host 220 and to a storage system 204, which includes a plurality of non-volatile storage devices 100. The interface between the memory controller 202 and the non-volatile storage devices 100 can be a bus interface, such as a SATA (Serial Advanced Technology Attachment) interface, a PCIe (Peripheral Component Interface Express) interface, an eMMC (embedded MultiMediaCard) interface, an SD interface, or a USB (Universal Serial Bus) interface. In one embodiment, the storage system 200 can be a solid-state drive (SSD), such as those found in portable computing devices like laptops, tablets, and mobile phones.

[0020] Fig. Figure 1C is a block diagram of a hierarchical storage system 250. The hierarchical storage system 250 can include a variety of storage controllers 202, each of which controls a corresponding storage system 204. Host systems 252 can access storage within the hierarchical storage system 250 via a bus interface. Example bus interfaces can include, for example, an NVMe (Non-Volatile Memory Express) interface, an FCoE (Fiber Channel over Ethernet) interface, an SD interface, a USB interface, a SATA interface, a PCIe interface, or an eMMC interface. In one embodiment, the Fig. The hierarchical storage system 250 shown in Figure 1C is a frame-mountable mass storage system accessible from multiple host computers, such as those found in a data center or other location where mass storage is required. In one embodiment, the host systems 252 can include the functionality described in Host 220.

[0021] Fig. Figure 2A is a block diagram illustrating exemplary components of the controller 102 in more detail. The controller 102 can include a front-end module 108 coupled to a host, a back-end module 110 coupled to the non-volatile memory chip(s) 104, and various other modules that perform different functions of the non-volatile memory device 100. In general, a module can be hardware or a combination of hardware and software. For example, each module can include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a circuit, a digital logic circuit, an analog circuit, a combination of discrete circuits, gates, or any other type of hardware or combination thereof.Additionally or alternatively, each module can include memory hardware containing instructions executable by a processor or processor circuitry to implement one or more of the module's features. If a module includes the portion of memory containing instructions executable by the processor, the module may or may not contain the processor. In some examples, each module may simply be the portion of memory containing instructions executable by the processor to implement the features of the corresponding module, without the module containing any other hardware. Because each module contains at least some hardware, even if the included hardware includes software, each module can be interchangeably referred to as a hardware module.

[0022] The controller 102 can include a buffer manager / bus control module 114, which manages buffers in a random access memory (RAM) 116 and controls the internal bus allocation for communication on an internal communication bus 117 of the controller 102. A read-only memory (ROM) 118 can store and / or access the system's boot code. Although they are in Fig. In 2A, where the controller 102 is shown as being arranged separately, in other embodiments either the RAM 116 or the ROM 118, or both, may be located within the controller 102. In still other embodiments, parts of the RAM 116 and the ROM 118 may be located both within and outside the controller 102. Furthermore, in some implementations, the controller 102, the RAM 116, and the ROM 118 may be arranged on separate semiconductor devices. As explained below, in one implementation, the transmission queues and the termination queues may be stored in the controller memory buffer, which may be located within the RAM 116.

[0023] Additionally, the front-end module 108 can have a host interface 120 and a physical layer (PHY) interface 122, which provide the electrical interface with the host or the next-level storage controller. The choice of host interface 120 type can depend on the type of storage used. Example host interface 120 types can include, but are not limited to, SATA, SATA Express, SAS, Fibre Channel, USB, PCIe, and NVMe. The host interface 120 can typically allow the transmission of data, control signals, and timing signals.

[0024] The backend module 110 can contain an ECC (Error Correction Controller) engine 124, which encodes the data bytes received from the host and decodes the data bytes read from the non-volatile memory chip(s) 104, correcting errors. As explained in more detail below, the ECC engine can be tunable to generate different amounts of ECC data based on the mode (e.g., generating normal-mode ECC data in normal programming mode and burst-mode ECC data in burst programming mode, with the burst-mode ECC data being larger than the normal-mode ECC data). The backend module 110 can also contain an instruction sequencer 126, which generates instruction sequences, such as program, read, and erase instruction sequences, to be transferred to the non-volatile memory die(s) 104.Additionally, the backend module 110 can include a RAID (Redundant Array of Independent Drives) module 128, which manages the generation of RAID parity and the recovery of failed data. RAID parity can be used as an additional integrity protection measure for the data written to the non-volatile storage device 100. In some cases, the RAID module 128 can be part of the ECC engine 124. A memory interface 130 delivers the command sequences to the non-volatile memory die(s) 104 and receives status information from the non-volatile memory die(s) 104. In addition to the command sequences and status information, data to be programmed into and read from the non-volatile memory chip(s) 104 can be communicated via the memory interface 130.In one embodiment, the memory interface 130 can be a DDR (Double Data Rate) interface and / or a Toggle Mode 200, 400, or 800 interface. A flash control layer 132 can control the overall operation of the backend module 110.

[0025] Thus, the controller 102 can include one or more management tables for managing operations of the storage system 100. One type of management table contains a logical-to-physical address mapping table. The size of the logical-to-physical mapping table can grow with the storage size. In this context, the logical-to-physical address mapping table for high-performance storage devices (e.g., larger than 32G) may be too large to be stored in SRAM; it can be stored in non-volatile memory 104 along with user and host data. Therefore, accesses to the non-volatile memory 104 may first require reading the logical-to-physical address mapping table from the non-volatile memory 104.

[0026] Additional modules of the in Fig. The illustrated storage device 100 in Figure 2A can include a media management layer 138 for wear compensation of memory cells of the non-volatile memory chip 104. The non-volatile storage device 100 can also include other discrete components 140, such as external electrical interfaces, external RAM, resistors, capacitors, or other components that may be connected to the controller 102. In alternative embodiments, one or more of the components RAID module 128, media management layer 138, and buffer management / bus controller 114 are optional components that may not be necessary in the controller 102.

[0027] Another module of the in Fig. The non-volatile storage device 100 shown in Figure 2A can contain the retrieval coalescing module 112. As explained in more detail below, the storage device can determine whether retrievals should be merged into the transmission queue, using the retrieval coalescing module 112 to perform this determination.

[0028] Fig. Figure 2B is a block diagram that shows exemplary components of a non-volatile memory chip 104 in more detail. The non-volatile memory chip 104 can include a non-volatile memory array 142. The non-volatile memory array 142 can include a plurality of non-volatile memory elements or cells, each configured to store one or more data bits. The non-volatile memory elements or cells can be any suitable non-volatile memory cells, including NAND flash memory cells and / or NOR flash memory cells in a two-dimensional and / or three-dimensional configuration. The memory cells can take the form of solid-state (e.g., flash) memory cells and can be programmable once, multiple times, or multiply.Furthermore, the memory elements or cells can be configured as single-level cells (SLCs), which store a single data bit per cell, multi-level cells (MLCs), which store multiple data bits per cell, or combinations thereof. For some example configurations, the multi-level cells (MLCs) can include triple-level cells (TLCs), which store three data bits per cell.

[0029] Additionally, a flash memory cell in the array 142 can include a floating-gate transistor (FGT) that has a floating gate and a control gate. The floating gate is surrounded by an insulator or insulating material that helps to retain charge within the floating gate. The presence or absence of charge within the floating gate can cause a shift in the FGT's threshold voltage, which is used to distinguish logic levels. That is, the FGT threshold voltage can indicate the data stored in the memory cell. Hereinafter, FGT, memory element, and memory cell can be used interchangeably to refer to the same physical unit.

[0030] The memory cells in the memory array 142 can be arranged in a matrix-like structure of rows and columns. At the intersection of a row and a column is a control gate (or memory cell). A column of control gates can be called a string. Control gates in a string or column can be electrically connected in series. A row of control gates can be called a page. Control gates of control gates in a page or row can be electrically connected to each other.

[0031] The memory array 142 can also include word lines and bit lines connected to the FGTs. Each page of FGTs is connected to a word line. In particular, each word line can be coupled to the control gates of FGTs in a page. Additionally, each string of FGTs can be coupled to a bit line. Furthermore, a single string can span multiple word lines, and the number of FGTs in a string can be equal to the number of pages in a block.

[0032] The non-volatile memory chip 104 can further include a side buffer or data cache 144, which temporarily stores data to be sampled from and / or programmed to the memory array 142. The non-volatile memory chip 104 can also include a row address decoder 146 and a column address decoder 148. The row address decoder 146 can decode a row address and select a specific word line in the memory array 142 when data is written to or read from the memory cells of the memory array 142. The column address decoder 148 can decode a column address to select a specific group of bit lines in the memory array 142 for electrical coupling with the data cache 144.

[0033] Furthermore, the non-volatile memory chip 104 can include a peripheral circuit 150. The peripheral circuit 150 can include a state machine 151, which provides status information to the controller 102. Other functionalities of the state machine 151 are described in more detail below.

[0034] Fig. Figure 3A shows a sequence of steps for executing a command using the NVMe standard. The NVMe standard reveals a scalable host control interface tailored to the requirements of enterprise and client systems that use PCI Express-based solid-state drives. Such systems rely on a mechanism for paired submit and exit queues.

[0035] As shown, the host device 300 includes a host storage 302, and the storage device includes a controller, such as an NVMe controller 310. In one implementation, the host storage 302 includes a transmit queue 304 and a terminate queue 306. Alternatively, the transmit queue and the terminate queue reside in the storage device, for example, in a controller buffer. Furthermore, in one implementation, the transmit queue and the terminate queue may have a 1:1 correlation. Alternatively, the transmit queue and the terminate queue may not have a 1:1 correlation.

[0036] In practice, during the initialization phase, the host device 300 creates one or more transmission queues and one or more corresponding termination queues. Specifically, the host device 300 can inform the storage device about the transmission queue(s) and the termination queue(s) by sending information such as the base address for each queue to the storage device. In this respect, each transmission queue has a corresponding termination queue. When the transmission queue and the termination queue reside within the host device, the host device sends information to the storage device so that the storage device can determine the locations of the transmission queue and the termination queue within the host device.In one specific implementation, the host device sends a command indicating the creation of the transmit queue and the exit queue. The command may include a PRP1 pointer, which is a pointer to a list on the host device specifying the memory locations of the specific transmit queue or exit queue. In practice, the storage device sends a TLP read request using the PRP1 to retrieve the PRP list and stores the PRP list on the storage device to determine the memory locations within the host device to be used in future commands for reading from the specific transmit queue or writing to the specific exit queue.Alternatively, the host device 300 can instruct the storage device to create the transmit queue(s) and the corresponding termination queue(s) in a memory located in the storage device, such as a control buffer.

[0037] The transmit queue 304 and the terminate queue 306 can be based on a ring buffer with a start pointer and an end pointer. In one implementation, the transmit queue and the terminate queue are recirculating buffers with fixed slot sizes, such as 64 bytes for transmit queues and 16 bytes for terminate queues. For example, a host device can have up to 64 KB of pending instructions on a transmit queue.

[0038] After creating the transmission queue(s) and notifying the storage device of the created transmission queue(s), the host device 300 can write one or more commands to the transmission queue. This is in Fig. 3A is specified as step 1, labeled "queue command". In particular, it illustrates Fig. 3A, that four instructions were written to the delivery queue. In one implementation, the storage device is unaware that the host device 300 updated the delivery queue 304 with four instructions because the host device 300 updated its own host memory 302. In another implementation (e.g., when the delivery queue(s) and the termination queue(s) reside in the control memory buffer), the storage device can monitor a communication interface between the host device 300 and the storage device for certain communications, such as writing to the delivery queue(s) present on the storage device.For example, the storage device can monitor the transport layer packets (TLPs) on the PCI Express bus to determine whether the host device 300 has sent a TLP that will result in an update to the delivery queue in the control memory buffer. In this respect, the storage device can identify one or more entries to be written to the delivery queue(s).

[0039] In step 2, the host device 300 writes to an end-doorbell register of the transmission queue 312 in the storage device. This write to the end-doorbell register of the transmission queue 312 indicates to the storage device that the host has sent one or more instructions to this specific transmission queue 304 (e.g., 4 instructions, as in Fig. (3A illustrated). In particular, writing to the end-doorbell register of the transmission queue 312 is a form of notification from the host device to the storage device that the host device has caused one or more instructions to be placed in the transmission queue. As explained in more detail below, this notification from the host device can trigger an analysis by the storage device to determine whether the instructions should be collected in batches.

[0040] Writing to the end doorbell register of the delivery queue 312 can take one of several forms. On the one hand, the host device 300 indicates a new end for the delivery queue 304, thus indicating the number of instructions written to the delivery queue 304. Since the storage device therefore knows the base address for the delivery queue 304, it only needs to know the end address to indicate the number of new instructions written to the delivery queue 304. As explained in more detail below, the storage device can consider the number of new instructions to fetch as a factor in determining whether the fetching of the instructions should be aggregated.

[0041] After an instruction (or set of instructions) has been processed, the storage device then sets the new beginning of the transmission queue 304 accordingly. This allows the end pointer to represent an "offset" from the beginning pointer. On the other hand, the host device 300 indicates the number of instructions that have been written to the transmission queue 304. In practice, each transmission queue 304 has a corresponding end doorbell register in the storage device, so that when the host device 300 updates a particular doorbell register (correlated with a specific transmission queue 304), the storage device can determine, based on the doorbell register, which specific transmission queue 304 has been updated.

[0042] After step 2 (where the storage device is informed about one or more commands on the transmission queue 304) and before step 3 (where the storage device retrieves the command(s)), the storage device is aware that one or more commands are pending in transmission queue 304. Generally, there can be multiple transmission queues (with potentially many pending commands in the various queues). Thus, before executing step 3, the storage device controller can decide between the different transmission queues to select the specific queue from which to retrieve the command(s).In this respect, the storage device can consider the number of new instructions to be retrieved from a single transmission queue, the number of new instructions to be retrieved from multiple transmission queues, or the number of new instructions to be retrieved from all available transmission queues to determine whether the retrieval of instructions from the transmission queue(s) should be aggregated. In this respect, the storage device can consider a single transmission queue alone to decide whether instructions should be retrieved from that single transmission queue. Alternatively, the storage device can consider multiple transmission queues in combination to decide whether instructions should be retrieved from the multiple transmission queues.

[0043] In response to the determination of which specific transmission queue 304 the instruction(s) to retrieve, the storage device retrieves the instruction(s) from specific transmission queue 304 in step 3. In practice, the storage device can access the base address of specific transmission queue 304 and the pointer to the current start pointer implemented in the host device 300.

[0044] As explained above, a memory area can be allocated to the transmit queue or the termination queue (such as in the host device or in the control buffer in the storage device). The transmit queue and the termination queue can contain multiple entries, each associated with a specific instruction. The size of each entry can be a predetermined size, such as 64 KB. In this context, entries within the transmit queue can be determined using the base address for the transmit queue and by multiplying the base address by the number of entries by the size of each entry (e.g., 64 KB).

[0045] As explained above, the storage device is aware of the end pointer it was notified about in step 2. Therefore, the storage device can receive all new commands from the 304 delivery queue. In NVMe, the storage device can send a TLP request to receive the command(s) from the 304 delivery queue. In response to receiving the TLP request, the host device 300 sends a TLP termination message containing the commands to the 304 delivery queue. Thus, at the end of step 3, the storage device receives the command(s) from the 304 delivery queue.

[0046] In step 4, the storage device processes the instruction. In one implementation, the storage device analyzes the instructions and determines the steps for executing them (e.g., read / write / etc.). For example, the instruction might include a read instruction. In response to receiving the read instruction, the storage device analyzes it, performs address translation, and accesses the flash memory to receive the data. After receiving the data, the storage device arranges for the data to be written to the host device based on information contained in the instruction (e.g., the PRP-1 discussed below). As another example, the instruction might include a write instruction.In response to receiving the write command, the storage device analyzes the write command, determines the location of the data on the host device being written, reads the data from that location on the host device, and writes the data to the flash memory.

[0047] In particular, the storage device can receive a read or write command with a PRP1 pointer. For example, a read command where the host device requests the storage device to read from the flash memory includes a PRP1 pointer pointing to a PRP list. The storage device receives the PRP list to determine the memory locations within the host device from which to write the data read from the flash memory. Similarly, a write command where the host device requests the storage device to write data to the flash memory includes a PRP1 pointer pointing to a PRP list. The storage device receives the PRP list to determine the memory locations within the host device from which to read the data from the flash memory (and subsequently to write the read data to the flash memory).

[0048] Each entry in the PRP list can be associated with a specific section in the host device memory and can have a predetermined size, such as 4 KB. Thus, in a 1 Mb transfer, there can be 250 references in the PRP list, each 4 KB in size. In practice, the storage device may retrieve data out of sequence. This can occur because the data to be retrieved resides on multiple flash chips, with the chips being available for data retrieval at different times. For example, the storage device might retrieve the data corresponding to 100-200 KB of the 1 Mb transfer before retrieving the data corresponding to 0-100 KB of the 1 Mb transfer.Nevertheless, since the storage device has the PRP list (and therefore knows the storage locations where the host device expects the 100-200 Kb to be stored), it can transfer the data corresponding to 100-200 Kb of the 1 Mb transfer without first having retrieved the data corresponding to 0-100 Kb of the 1 Mb transfer.

[0049] In NVMe, multiple PCI Express TLPs can be used to transfer data from the storage device to the host device 300. Typically, the transferred data is stored in the host device 302's memory based on a hint in the command (e.g., the command includes an address to store the requested data).

[0050] After the data transfer is complete, in step 5, the storage device controller sends a termination message to the relevant termination queue 306. As mentioned above, during the initialization phase, the host device 300 maps the transmit queues to the termination queues. This allows the host device 300 to know which commands in the transmit queue have terminated, based on which termination queue the storage device is writing to. The termination message can contain information regarding the processing of the command(s), such as whether the command completed successfully or if an error occurred during its execution.

[0051] After step 5, host device 300 is unaware that the storage device has entered the termination queue 306. This is because the storage device initiates data being written to termination queue 306. In this context, the storage device notifies host device 300 in step 6 that an update to termination queue 306 has occurred. Specifically, the storage device sends an interrupt to host device 300 (for example, in NVMe, host device 300 might use an MSIe interrupt). As explained in more detail below, the storage device can aggregate interrupts based on one or more factors.

[0052] In response to receiving the interrupt, host device 300 determines that one or more termination entries are pending in termination queue 306 for host device 300. In step 7, host device 300 then processes the entries in termination queue 306.

[0053] After the host has processed the entries from termination queue 306, in step 8, host device 300 notifies the storage device of the entries that host device 300 has processed from termination queue 306. This can be done by updating an initial doorbell register 314 of the termination queue, which indicates to the storage device that host device 300 has processed one or more entries from termination queue 306.

[0054] In response to the update of the start doorbell register 314 of the termination queue, the storage device updates the start of the termination queue 306. With the new start, the storage device knows which entries in the termination queue 306 have already been processed by the host device 300 and can be overwritten.

[0055] Fig. Figure 3B illustrates the transmit queue and the exit queue with their respective head and end pointers. The head and end pointers associated with each MVMe transmit queue and exit queue can be held internally in the storage device. In practice, the head pointer identifies the next request to be read from the respective queue. The end pointer identifies where the next open request should be transmitted. These pointers in the storage device can be referred to as doorbell registers, as explained above. The transmitter can increment the end pointer after transmitting a new entry to the identified open entry. If the incremented end pointer exceeds the queue size, the end pointer can roll to zero.

[0056] For example, the host device in the transmission queue can add an instruction, advance the end pointer, and inform the storage device controller of the new end pointer by writing to the transmission queue's end doorbell register. Furthermore, the storage device controller in the transmission queue can remove an instruction from the transmission queue (for example, by fetching the instruction), advance the head pointer, and inform the host device of a new head pointer, as in Fig. 3B is shown. Conversely, the storage device controller can add a termination entry to the termination queue (and invert the phase marker as a message to the host device), advance the termination pointer, and inform the host device of the new termination pointer by sending a break (step 6 in Fig. 3A). Furthermore, the host device in the termination queue can remove the termination entry from the termination queue, advance the head pointer, and inform the host device of a new head pointer by writing to the termination queue's start doorbell register, as described in Fig. 3B is shown. The submitter can continue to write entries to the queue as long as the queue is not full.

[0057] The consumption of entries from a queue can be performed either by the storage device (for example, when retrieving new entries from a submission queue, see below) or by the host software (for example, when reading termination entries from a termination queue, see below). The consumer of entries in a queue can use the current header pointer to identify the next entry to be retrieved from the queue. Specifically, the consumer can increment the header pointer after retrieving the next entry from the queue. If the incremented header pointer exceeds the queue size, the header pointer can roll to zero. The consumer can continue to remove entries from the queue as long as the empty queue condition is not met.

[0058] For NVMe-based devices, the storage device can be considered the master on the PCIe bus. Device-host communication can be scheduled via the queue management system. For example, communication can be scheduled by scheduling command transmission between the host device and the storage device using doorbell registers. The host device can signal new commands to the storage device by writing a new number to a doorbell register in the storage device (see step 2 of Fig. 3A). The storage device can use this value to calculate the number of instructions that need to be retrieved. Then, the storage device can send an instruction read request to the host device (specifying the number of instructions required), and only then would the host device send the instruction information to the storage device.

[0059] In PCIe buses, data is sent in packets. The size of the data sent in each packet is determined by the storage device, while the maximum size is determined by the host device. Since each packet may contain much more than just the data, an efficient way for the storage device to utilize the PCIe bus is to send as much data as possible in each packet.

[0060] In certain operating systems (or intermittently in certain system benchmarks), the host device signals only one new instruction at a time to the storage device, as opposed to aggregating Doorbell Write commands to be multiple instructions at once. This results in increased activity on the PCIe bus, as the bus does not send the maximum possible data size, as well as higher power consumption and reduced system performance. In this regard, the storage device can utilize the instruction fetch aggregation 434 (see below), which is not present (or used) on the host device, as additional aggregation intelligence on the storage device.

[0061] Fig. Figure 4 is a block diagram showing further example components of a host system 400 and a storage device 420. The host system 400 includes one or more processors 402 and the host memory 404. The host memory 404 can include physical region pages (PRPs) 406, data buffers 408, one or more transmit queues 410, one or more terminate queues 412, and other memory 414.

[0062] Fig. Figure 4 further illustrates a communication interface between the host device 400 and the storage device 420. In an initial implementation (in Fig. (4 not shown) the communication interface between the host device and the storage device is implemented as a simplex interface, with communications to and from the storage device on the same path. In a second implementation (in Fig. As illustrated in Figure 4, the communication interface between the host device 400 and the storage device 420 is implemented as a duplex interface, with a separate input path and a separate output path. From the perspective of the storage device 420, the input path contains incoming requests from the host device 400 to the storage device 420. Conversely, from the perspective of the storage device 420, the output path contains outgoing requests from the storage device 420 to the host device 400.

[0063] The incoming requests (requests from the host device 400 to the storage device 420) can be segmented in various ways, such as incoming read requests and incoming write requests. For example, the host device 400 can send a read request to read a section of memory in the storage device 420 or a write request to write to a section of memory in the storage device 420 via the input path. Similarly, the storage device 420 can send a read request to a section of memory in the host device 400 or a write request to write to a section of memory in the host device 400 via the output path.

[0064] In practice, when using MVMe, there can be a series of read requests (a request from the host device to read data resident on the storage device, and vice versa) and a series of write requests (a request from the host device to write data to a location resident on the storage device, and vice versa). Specifically, in NVMe, the storage device and the host device communicate with each other via TLP (Transaction Layer Packet) requests, such as TLP read requests to perform a read on the other device, or TLP write requests to perform a write on the other device.In an example (with the transmit queue and exit queue residing on the host device), the storage device—in response to a TLP write request (sent via the input path) from the host device to the Doorbell register on the storage device (where the write to the Doorbell register indicates that a command is present on the transmit queue)—uses a TLP read request (sent via the output path) to retrieve the write command from the transmit queue (which resides on the host device). Thus, the write command is a request for the storage device to write data to non-volatile memory. The storage device then parses the write command for information such as a PRP pointer reference (e.g., PRP1) to a PRP list.The PRP list is a set of information, such as pointers or addresses, that indicates the location of data on the host device. The storage device then uses another TLP read request to read data from the pointers or addresses in the PRP list. Afterward, the storage device performs the write operation by storing the data in non-volatile memory (such as flash memory) on the storage device. Once the data is stored, the storage device uses a TLP write request to add an entry to the exit queue (indicating that the write operation has finished). Finally, the storage device uses a TLP write request to generate an interrupt for the host device, signaling to the host device that an entry exists in the exit queue.In response to the interrupt, the host device reads the entry in the termination queue and then issues a TLP write request to the CQ doorbell write register, indicating that the host device has checked the entry in the termination queue.

[0065] As another example (again with the transmit queue and exit queue residing on the host device), the storage device—in response to a TLP write request from the host to the Doorbell register on the storage device (where the write to the Doorbell register indicates that a command is present on the transmit queue)—uses a TLP read request to retrieve the read command from the transmit queue (which resides on the host device). Thus, the read command is a request for the storage device to read data from the non-volatile memory and send the read data to the host device. The storage device then reads the non-volatile memory (e.g., flash memory) to retrieve the data. The storage device can perform a number of operations on the data, such as error correction, encryption / decryption, and so on.The process involves inserting memory buffers between each serial operation. The storage device can then parse the read command for information, such as a PRP pointer's reference (e.g., PRP1) to a PRP list. The PRP list consists of a set of information, such as pointers or addresses, that specifies the location in the host device where the data read from non-volatile memory will be stored (and optionally error-corrected, encrypted, etc.). The storage device then uses a TLP read request to read data from the pointers or addresses in the PRP list. Afterward, the storage device uses a TLP write request to write the data read from non-volatile memory.After writing the data to the host device, the storage device uses a TLP write request to write an entry to the exit queue (indicating that the read instruction has completed). Finally, the storage device uses another TLP write request to generate an interrupt for the host device, signaling to the host device that an entry exists in the exit queue. In response to the interrupt, the host device reads the entry in the exit queue and then issues a TLP write request to the CQ doorbell write register, indicating that the host device has checked the entry in the exit queue.

[0066] The storage device 420 includes the storage device controller 422 and the storage arrays 450. The storage array 450 can be segmented in various ways, such as into 10 sections, as shown in Fig. Figure 4 illustrates this. The storage device controller 422 can include one or more processors 424 and contain one or all of a PCIe, MAC, and PHY interface 432. The flash interface module 430 is configured to control and access the storage arrays 450. Fig. The Flash Interface Module 430 also includes the Low Density Parity Check (LDPC) 442, which is a linear error correction code. Other error correction methods are being considered.

[0067] Command retriever 426 is configured to retrieve commands from the transmission queues 410 on the host system 400 and internally send them to the storage device 420. Command executor 428 is configured to evaluate and execute the commands retrieved from the transmission queues 410.

[0068] In one implementation, the command taker receives the doorbell update directly from the register access controller 438 (represented as line 460). This allows the command taker 426 to retrieve the command from the host device 400 immediately upon notification.

[0069] In an alternative implementation, the doorbell update is sent to Command Fetch Aggregation 434 and Doorbell Register 440. Command Fetch Aggregation 434 is configured to aggregate command fetches until one or more criteria are met. The criteria can include one or more aspects of the system, one or more aspects of the commands, or the like, as explained above. The storage device can determine the one or more criteria based on analyzing patterns associated with notifications from the host device via the delivery queue. As one example, the criterion can include the number of commands received, as further explained below. As another example, the criterion can include a temporal aspect (e.g., waiting a specified amount of time). As yet another example, the criterion can include a system aspect (e.g.,The criteria may include an indication of bus activity; or a status of the respective transmission queue (e.g., the number of commands to be retrieved from the respective transmission queue); or the number of pending host command requests (e.g., the queue depth). As explained in more detail below, the Command Fetch Aggregation 434 can use a static value for the criterion (e.g., a predefined, unchanging value). Alternatively, the Command Fetch Aggregation 434 can analyze the system to dynamically select the value for the criterion (e.g., iteratively selecting different criteria and analyzing the system response to determine an optimal or improved criterion).

[0070] In an implementation, one or more criteria can be used for retrieving commands from all delivery queues. As explained above, the host device 400 can have multiple delivery queues. In this respect, the criterion can be analyzed across all delivery queues. For example, the command retrieval aggregation 434, with respect to timing, can receive a doorbell update from an initial delivery queue and wait a predetermined amount of time before retrieving available commands from all delivery queues. Alternatively, one or more criteria can also apply to retrieving commands from a single, specific delivery queue.For example, the command retrieval aggregation 434, with respect to timing, can receive a doorbell update from a first transmission queue and wait a predetermined amount of time before retrieving available commands only from the first transmission queue. In yet another implementation, one or more of the criteria for retrieving commands can apply to some, but not all, of the transmission queues. The transmission queues can be grouped in various ways, such as based on the type of transmission queue, the priority of the transmission queue, or the like. The priority with respect to transmission queues is explained in U.S. Application No. 15 / 585,717, which is incorporated herein by reference in its entirety.For example, the Command Fetch Aggregation 434 can receive a doorbell update from a first delivery queue and wait a predetermined amount of time before fetching available commands from delivery queues of the same priority. As another example, the Command Fetch Aggregation 434 can assign a predetermined number (e.g., a predetermined amount of time or a predetermined number of commands to fetch) to each delivery queue based on the priority of that queue. In a first specific example, a higher-priority queue can be assigned a lower predetermined number (e.g., a shorter predetermined amount of time to wait before deciding to fetch, or a lower threshold for the number of commands in the queue before deciding to fetch).Furthermore, a higher predetermined number can be assigned to a lower-priority queue (e.g., a longer predetermined waiting time before deciding to retrieve, or a higher threshold for the number of commands in the queue before retrieval). In a second specific example, a lower-priority queue can be assigned a lower predetermined number (e.g., a shorter predetermined waiting time before deciding to retrieve, or a lower threshold for the number of commands in the queue before retrieval).

[0071] In one specific implementation, the instruction retrieval aggregation 434 aggregates the instruction retrieval request in the storage device 420 until the number of instructions to be retrieved exceeds a certain threshold. When the number of instructions to be retrieved exceeds the certain threshold, the instruction retrieval aggregation 434 can send an instruction to the instruction retriever 426 to retrieve the instructions. In this respect, the counter 436 can be used to count the number of instructions (or to count the elapsed time) so that the instruction retrieval aggregation 434 can compare the counted number of instructions (the counted elapsed time) with the certain threshold. In one implementation, the threshold is fixed, as explained above. In an alternative implementation, the threshold is dynamic, based on one, any combination, or all of the following: bus activity (e.g.,Bus overload; the time between doorbell rings; the number of commands requested with each doorbell ring; and the number of pending host requests. For example, the Command Fetch Aggregation 434 can select an initial threshold, analyze one or more aspects of the system (such as the activity of the input and / or output bus), and select an updated threshold. In this respect, the Command Fetch Aggregation 434 can have an intelligent feedback analysis mechanism.

[0072] In a second specific implementation, the instruction call aggregation 434 aggregates the instruction call request in the memory device 420 based on time. The instruction call aggregation 434 can use a clock (not in Fig. (Figure 4 is shown). For example, the command retrieval aggregation 434 can receive a doorbell update and wait a predetermined amount of time before instructing the command retriever 426 to retrieve the commands. The wait time can be predetermined and static or selected based on a dynamic analysis (e.g., based on an iterative analysis, as explained above).

[0073] In a third specific implementation, the command call aggregation 434 aggregates the command call request in the storage device 420 based on a system aspect. As an example, in response to receiving the doorbell update, the command call aggregation 434 can determine whether the system aspect is present (e.g., a specification of a predetermined bus operationality; etc.). If the command call aggregation 434 determines that the system aspect is not present, it can wait until the system aspect is present. Alternatively, it can wait a predetermined amount of time until the system aspect is present. If the system aspect is not present, the command call aggregation 434 can instruct the command caller 426 to retrieve commands.

[0074] Fig. Figure 5 is a block diagram illustrating the transmission queue and retrieval coalescing. In particular, it illustrates Fig. 5. The relationship between a delivery queue and the fetch coalescing algorithm. In an implementation, one or more watermark levels can be defined for each delivery queue. A watermark level can contain a number of instructions at a particular stage or processing stage, such as a number of instructions available for fetching. As explained above, the storage device can fetch an available instruction immediately after the host device is notified of the available instruction in the delivery queue. In an implementation that includes in Fig. As illustrated in Figure 5, two watermark levels are defined as different fill levels of the transmission queue. The low watermark level can represent the nearly empty state (indicating that the storage device controller 422 has additional capacity), while the high watermark level represents the nearly full state (indicating that the storage device controller 422 may be at or above capacity). In this respect, the low watermark level has a number that is smaller than the high watermark level. Although two watermark levels are illustrated, one watermark level or three or more watermark levels are considered.

[0075] One or more of the watermark layers (e.g., one or both of those in Fig. The 5 illustrated low and high watermark levels can be dynamically adjusted or changed based on one or more aspects of the system. As explained above, various aspects of the system include, but are not limited to, bus activity, queue depth, etc. In this respect, the storage device can analyze these aspects by dynamically determining the watermark level(s).

[0076] Fig. Figure 6 is a flowchart (600) of a first exemplary fetch coalescing procedure. At 602, the storage device determines whether the fetch will be coalesced. If not, the instruction is fetched at 608 immediately upon receiving the doorbell update. If so, one or more aspects are analyzed, such as: the activity of the communication interface (e.g., the PCIe bus); timing (e.g., specifying the time between receiving interrupts); the number of instructions to be fetched; and the number of pending requests from the host device. At 606, based on the analysis, the storage device determines whether the fetch should be coalesced. If so, at 610, the fetch is at least partially coalesced (e.g., partially or completely coalescing the fetch). If not, the instruction is fetched at 608 immediately.

[0077] Fig. Figure 7 is a flowchart (700) of a second exemplary fetch coalescing procedure. In one implementation, after each doorbell update, the storage device can compare the number of aggregated instructions with the instruction fetch threshold. At 704, the storage device determines whether the instruction fetch threshold has been exceeded. If so, at 708, the storage device sends the instruction fetch to the host device. If not, at 706, the storage device updates the instruction aggregation value and sets the timeout timer. At 710, the storage device determines whether the timeout (as determined by the timer) has been reached. If so, flowchart 700 passes to 708 so that the storage device sends the instruction fetch to the host device. If not, at 712, the storage device determines whether another doorbell update has been received. If so, the flowchart returns to 704.If not, the flowchart changes from 700 to 710 to determine if the timeout has occurred.

[0078] Fig. Figure 8 is a flowchart (800) of an exemplary procedure for iteratively selecting a threshold for retrieval coalescence. At 802, the storage device selects a threshold for retrieval aggregation. At 804, the storage device analyzes its response to the selected threshold. For example, the storage device might analyze bus activity in response to the chosen threshold. At 806, based on the analysis, the storage device can determine whether to change the threshold. If so, at 808, the storage device changes the selected threshold and returns to 804.

[0079] Finally, as mentioned above, any suitable type of memory can be used. Semiconductor memory devices include volatile memory devices, such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory), non-volatile memory such as ReRAM (Resistive Random Access Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory (which can also be considered a subset of EEPROM), FRAM (Ferroelectric Random Access Memory), and MRAM (Magnetoresistive Random Access Memory), and other semiconductor elements capable of storing information. Each type of memory device can have different configurations. For example, flash memory devices can be configured in either a NAND or NOR array.

[0080] The memory devices can be composed of passive and / or active elements in any combination. As a non-limiting example, passive semiconductor memory elements include ReRAM device elements, which in some embodiments contain a resistive switching element, such as an anti-fuse, phase-change material, etc., and optionally a control element, such as a diode, etc. Furthermore, as a non-limiting example, active semiconductor memory elements include EEPROM and flash memory device elements, which in some embodiments contain elements with a charge storage region, such as a floating gate, conductive nanoparticles, or a charge-storing dielectric material.

[0081] Multiple memory elements can be configured to be cascaded or to be individually accessible. As a non-restrictive example, flash memory devices in NAND configuration (NAND memory) typically contain cascaded memory elements. A NAND memory array can be configured to consist of multiple memory chains, in which each chain comprises multiple memory elements sharing a single bit line and being accessed as a group. Alternatively, memory elements can be configured to be individually accessible, such as a NOR memory array. NAND and NOR memory configurations are examples, and memory elements can be configured in other ways.

[0082] The semiconductor memory elements, which are arranged within and / or above a substrate, can be arranged in two or three dimensions, such as a two-dimensional memory structure or a three-dimensional memory structure.

[0083] In a two-dimensional memory structure, the semiconductor memory elements are arranged in a single plane or memory device plane. Typically, in a two-dimensional memory structure, memory elements are arranged in a plane (e.g., in an xz-direction plane) that extends substantially parallel to a major face of a substrate supporting the memory elements. The substrate may be a wafer over or within which the layer of memory elements is formed, or it may be a support substrate that is attached to the memory elements after formation. As a non-restrictive example, the substrate may include a semiconductor such as silicon.

[0084] The memory elements can be arranged in an ordered array at the individual storage device level, such as in a multitude of rows and / or columns. However, the memory elements can also be arranged in irregular or non-orthogonal configurations. Each memory element can have two or more electrodes or contact lines, such as bit lines and word lines.

[0085] A three-dimensional storage array is arranged such that storage elements occupy multiple levels or multiple storage device levels, forming a structure in three dimensions (i.e. in the x, y, and z directions, where the y direction is essentially perpendicular and the x and z directions are essentially parallel to the main surface of the substrate).

[0086] As a non-restrictive example, a three-dimensional memory structure can be arranged vertically as a stack of multiple two-dimensional memory device levels. As another non-restrictive example, a three-dimensional memory array can be arranged as multiple vertical columns (e.g., columns extending substantially perpendicular to the main surface of the substrate, i.e., in the y-direction), with each column containing multiple memory elements. The columns can be arranged in a two-dimensional configuration, e.g., in an xz-plane, resulting in a three-dimensional array of memory elements with elements on multiple vertically stacked memory levels. Other configurations of memory elements in three dimensions can also form a three-dimensional memory array.

[0087] As a non-restrictive example, in a three-dimensional NAND memory array, the memory elements can be coupled to form a NAND chain within a single horizontal (e.g., xy) memory device layer. Alternatively, the memory elements can be coupled to form a vertical NAND chain spanning multiple horizontal memory device layers. Other three-dimensional configurations can be considered, with some NAND chains containing memory elements within a single memory layer, while other chains containing memory elements spanning multiple memory layers. Three-dimensional memory arrays can also be designed as a NOR configuration and as a ReRAM configuration.

[0088] Typically, in a monolithic three-dimensional storage array, one or more storage device layers are formed on top of a single substrate. Optionally, the monolithic three-dimensional storage array may also have one or more storage layers at least partially within the single substrate. As a non-restrictive example, the substrate may include a semiconductor such as silicon. In a monolithic three-dimensional array, the layers that form each storage device layer of the array are usually formed on top of the layers of the underlying storage device layers of the array. However, layers of adjacent storage device layers in a monolithic three-dimensional storage array may be shared, or there may be intermediate layers between storage device layers.

[0089] Two-dimensional arrays can then be formed separately and subsequently packed together to create a non-monolithic, multi-layered memory device. For example, non-monolithic stacked memories can be constructed by forming memory layers on separate substrates and then stacking them on top of each other. The substrates can be thinned or removed from the memory device layers prior to stacking, but because the memory device layers are initially formed on separate substrates, the resulting memory arrays are not monolithic three-dimensional memory arrays. Furthermore, multiple two-dimensional or three-dimensional memory arrays (monolithic or non-monolithic) can be formed on separate chips and then packed together to create a stacked-chip memory device.

[0090] Associated circuitry is typically required for the operation and communication of the memory elements. As non-restrictive examples, memory devices may include circuitry used to control and drive memory elements to perform functions such as programming and reading. This associated circuitry may reside on the same substrate as the memory elements and / or on a separate substrate. For example, a controller for memory read / write operations may be located on a separate controller chip and / or on the same substrate as the memory elements.

[0091] The foregoing detailed description is intended to be understood as an illustration of selected forms that the invention may take, and not as a definition of the invention. Only the following claims, including all equivalents, are intended to define the scope of the claimed invention. Finally, it should be noted that each aspect of each of the preferred embodiments described herein may be used alone or in combination with one another.

Claims

[1] Method, comprising: Receiving an indication from a host device (300; 400) of one or more commands available for retrieval from at least one transmission queue (304; 410), wherein the host device (300; 400) has caused the one or more commands to be stored on the at least one transmission queue (304; 410) so that a storage device (100; 420) can process them; and in response to receiving the display, by the storage device (100; 420): accessing a retrieval threshold, wherein the retrieval threshold indicates a number of commands or a waiting time; Determine whether the retrieval threshold has been reached; in response to the determination that the retrieval threshold has been reached, retrieving one or more commands from the at least one transmission queue (304; 410); and in response to the determination that the retrieval threshold has not been reached, wait to retrieve one or more commands from the at least one transmission queue (304; 410). [2] Method according to claim 1, wherein: The retrieval threshold indicates a number of commands; and where determining whether the retrieval threshold has been reached includes determining whether the number of retrieval instructions is greater than or equal to the retrieval threshold. [3] Method according to claim 1, wherein: the retrieval threshold indicates a waiting time; and where determining whether the retrieval threshold has been reached includes determining whether a time since a last retrieval for instructions is greater than or equal to the retrieval threshold. [4] Method according to claim 1, further comprising analyzing at least one aspect of a communication interface (432) between the host device (400) and the storage device (420) to dynamically determine the retrieval threshold. [5] Method according to claim 4, wherein analyzing at least one aspect of the communication interface (432) comprises analyzing an operation of the communication interface (432). [6] Method according to claim 1, further comprising: Analyzing patterns between different displays of the host device (300; 400); and Selecting the retrieval threshold based on the analysis. [7] Method according to claim 1, further comprising: Determining the processing capacity of the storage device (100; 420); and Based on the specified processing capacity, determine whether a number of retrieval commands must be greater than or equal to the retrieval threshold before the one or more commands are retrieved from the at least one transmission queue (304; 410). [8] Method according to claim 7, wherein: in response to the determination that the storage device (100; 420) has a higher processing capacity, the determination that the number of instructions to be retrieved need not be greater than or equal to the retrieval threshold before the one or more instructions are retrieved from the at least one transmission queue (304; 410); and In response to the determination that the storage device (100; 420) has a lower processing capacity, the determination that the number of instructions to be retrieved must be greater than or equal to the retrieval threshold before the one or more instructions are retrieved from the at least one transmission queue (304; 410). [9] Method according to claim 1, further comprising dynamically determining the retrieval threshold by the storage device (100; 420). [10] Method according to claim 9, wherein the storage device (100; 420) dynamically determines the retrieval threshold based on an analysis by the storage device (100; 420) of a number of commands requested for retrieval in previously received displays from the host device (300; 400). [11] Method according to claim 9, wherein the storage device (100; 420) dynamically determines the retrieval threshold based on a number of outstanding requests from the host device (300; 400). [12] Non-volatile storage device (420), comprising: a non-volatile memory (450); a communication interface (432) for communicating with a host device (400); and a controller (422) in communication with the non-volatile memory (450) and the communication interface (432), wherein the controller (422) is configured to: Analyzing one or more aspects relating to the at least one transmission queue (410), wherein the host device (400) has caused the instructions to be stored on the at least one transmission queue (410) so that the storage device (420) can process them; based on the analysis of one or more aspects, determining one or more retrieval criteria, wherein the retrieval criteria indicate when the storage device (420) should retrieve instructions from the at least one transmission queue (410); Receiving an indication from the host device (400) about one or more commands available for retrieval from the one or more transmission queue (410); and in response to receiving the notification: Determine whether one or more of the retrieval criteria are met; in response to the determination that one or more of the retrieval criteria are met, retrieving the one or more commands from the at least one transmission queue (410); and in response to the determination that one or more retrieval criteria are not met, wait to retrieve one or more commands from the at least one transmission queue (410). [13] Storage device (420) according to claim 12, wherein: the controller (422) is set up to analyze one or more aspects relating to the at least one transmission queue (410) by analyzing patterns between different displays of the host device (400); that one or more retrieval criteria include a waiting time; and In response to receiving the display, the controller (422) is set up to wait based on the waiting time before retrieving the one or more commands from the at least one transmission queue (410). [14] Storage device (420) according to claim 12, wherein: the controller (422) is configured to analyze one or more aspects relating to the at least one transmission queue (410) by analyzing the bus activity between the host device (400) and the storage device (420); and that one or more retrieval criteria are based on bus activity. [15] Storage device (420) according to claim 12, wherein: the controller (422) is configured to analyze one or more aspects relating to the at least one transmission queue (410) by determining a number of commands from the at least one transmission queue (410) that are currently being processed by the storage device (420); and that one or more retrieval criteria are based on the number of commands from the at least one transmission queue (410) that is currently being processed by the storage device (420). [16] Storage device (420) according to claim 12, wherein the control (422) is configured to analyze at least one aspect of the host device (400) in order to determine the one or more retrieval criteria. [17] Storage device (420) according to claim 12, wherein the control (422) is configured as follows: in response to receiving the display from the host device (400), analyzing one or more aspects and determining one or more retrieval criteria. [18] Storage device (420) according to claim 12, wherein the control (422) is configured as follows: prior to receiving the display from the host device (400), analyzing one or more aspects and determining one or more retrieval criteria. [19] Non-volatile storage device (420), comprising: Means (432, 438) for receiving a notification from a host device (400) about one or more commands available for retrieval from at least one transmission queue (410), wherein the host device (400) has caused the one or more commands to be stored on the at least one transmission queue (410) so that the storage device (420) can process them; and in response to receiving the notification: Means (434) for accessing a call threshold, wherein the call threshold specifies a number of commands or a waiting time; Means (434, 436) to determine whether the retrieval threshold has been reached; in response to the determination that the retrieval threshold has been reached, means (426) to retrieve the one or more commands from the at least one transmission queue (410); and in response to the determination that a number of retrieval commands is greater than or equal to the retrieval threshold, means to wait for the retrieval of one or more commands from the at least one transmission queue (410). [20] Storage device (420) according to claim 19, wherein: The retrieval threshold indicates a number of commands; and wherein the means (434, 436) for determining whether the retrieval threshold has been reached include means for determining whether a number of retrieval instructions is greater than or equal to the retrieval threshold. [21] Storage device (420) according to claim 19, wherein: the retrieval threshold indicates a waiting time; and the means (434) for determining whether the retrieval threshold has been reached includes means for determining whether a time since a last retrieval of instructions is greater than or equal to the retrieval threshold.

Images