Memory system and method of controlling non-volatile memory

Abstract

An embodiment of the present invention intends to provide a memory system and a method of controlling a non-volatile memory that can flexibly control access to a non-volatile memory. A memory system includes a non-volatile memory and a control circuit electrically connected to the non-volatile memory. The control circuit is configured to, upon reception of a first read request issued from a host during a current write operation, determine a first ratio of a first duration to a second duration, the first duration being an accumulated time of one or more read operations executed subsequent to completion of a most recently completed write operation and prior to completion of the current write operation, the second duration being a required time for the current write operation; and determine whether to suspend the current write operation based on the first ratio.

Description

Technical Field

[0001] The embodiments described in this article mainly relate to a memory system and a method for controlling non-volatile memory. Prior Technology

[0002] In recent years, memory systems, including those using non-volatile memory, have been widely used. Among them, solid-state drives (SSDs) that use NAND flash memory are a typical example.

[0003] SSDs have been used as storage devices in various host computer systems, such as servers in data centers.

[0004] For storage devices like SSDs, a new technology is needed that can flexibly control access to non-volatile memory. Summary of the Invention

[0005] The embodiments of the present invention aim to provide a memory system and a method for controlling non-volatile memory, which can flexibly control access to non-volatile memory.

[0006] Generally, according to one embodiment, the memory system includes a non-volatile memory and control circuitry electrically connected to the non-volatile memory. This control circuitry is configured to: upon receiving a first read request from the host during a current write operation, determine a first ratio between a first duration and a second duration, where the first duration refers to the cumulative time of one or more read operations performed from the completion of the most recent completed write operation to the completion of the current write operation, and the second duration refers to the time required for the current write operation; and determine whether to pause the current write operation based on the first ratio. Simple Explanation of the Diagram

[0007] [Figure 1] is a block diagram of an example memory system according to one embodiment. [Figure 2] is a block diagram of an example configuration for an example coordinator according to this embodiment. [Figure 3] is a diagram illustrating an example of how a coordinator coordinates read and write performance according to this embodiment. [Figure 4] is an example diagram of another coordination method used for an example coordinator according to this embodiment. [Figure 5] is a flowchart of an example procedure for determining the write time of the die according to this embodiment. [Figure 6] is a flowchart of an example procedure for determining the read-die time for an example coordinator according to this embodiment. [Figure 7] is a flowchart illustrating an example of a procedure for determining whether the coordinator according to this embodiment has enabled the write pause function. [Figure 8] is a flowchart of an example of a procedure for executing a write pause in an example coordinator according to this embodiment. Implementation

[0008] An embodiment is described below with reference to the accompanying drawings.

[0009] The following description is intended to exemplify the apparatus and methods for implementing the technical concept of this embodiment. The technical concept of this embodiment is not limited to the structure, shape, arrangement, material, etc., of the various configuration elements described below. Various modifications that are obvious to those skilled in the art are within the scope of protection of this disclosure. For ease of understanding, the dimensions, thickness, planar dimensions, shapes, etc., of each element in the drawings may differ from actual elements and are only schematic representations. The drawings may include elements with different size ratios or different proportional relationships. Corresponding elements in the drawings may use the same element symbol, and repeated descriptions are omitted. Some elements may have multiple names. These names are merely exemplary examples and do not preclude the possibility that the element may have other names. Elements not given multiple names may also have other names. Connections refer not only to direct connections but also to connections through other elements. Unless explicitly stated that the number of elements is plural, the element may be a single element or multiple elements.

[0010] Figure 1 is a block diagram illustrating an example of the memory system 4 according to this embodiment. The memory system 4 is electrically connected to the host 2. The host 2 can be a storage server storing a large amount of data of various types stored in the memory system 4, or it can be a server or a personal computer.

[0011] Memory system 4 is a semiconductor storage device used to write data to and read data from non-volatile memory. An example of memory system 4 is a solid-state drive (SSD). An example of non-volatile memory is NAND flash memory. Memory system 4 can function as storage in host 2. Memory system 4 can be built into host 2 or connected to host 2 via cable or network.

[0012] The memory system 4 includes non-volatile memory 12, volatile memory 14, and controller 16.

[0013] An example of non-volatile memory 12 is NAND flash memory. In this invention, non-volatile memory 12 is also referred to as NAND flash memory 12. NAND flash memory 12 includes a memory cell array comprising a plurality of memory cells arranged in a matrix. Each memory cell stores data in a non-volatile manner based on a threshold voltage. NAND flash memory 12 can be a flash memory with a two-dimensional structure or a flash memory with a three-dimensional structure.

[0014] The NAND flash memory 12's memory cell array comprises multiple blocks. Each block comprises multiple pages. A block is the smallest unit of a data erase operation. A data erase operation refers to erasing (or initializing) the data stored in a memory cell by setting the threshold voltage of the memory cell in the block to be erased to a predetermined voltage or lower. Blocks are sometimes also called erase blocks or physical blocks. Each page comprises multiple memory cells, which are connected together to a single word line. Pages are the unit for write and read operations. Word lines can also serve as the unit for write and read operations.

[0015] An example of volatile memory 14 is dynamic random access memory (DRAM) or static random access memory (SRAM), or both. In this invention, volatile memory 14 is also referred to as DRAM 14. DRAM 14 may include a firmware area 14a, a logic / physical address translation table area 14b, and a buffer 14c. Buffer 14c is an area used for temporarily storing data.

[0016] Firmware is a control program that includes a set of instructions for the CPU 28 to perform various processes. The CPU 28 controls the operation of the controller 16 by executing the firmware. When the memory system 4 starts up, the firmware can be read from the NAND flash memory 12 or (not shown) read-only memory (ROM) and stored in the firmware area 14a.

[0017] The logical / physical address translation table includes a lookup table (LUT). The LUT manages the mapping between data identifiers used to identify data to be accessed and the physical addresses of the NAND flash memory 12. An example of a data identifier is a logical address used by host 2 to specify the address of a storage area in memory system 4. An example of a logical address is a logical block address (LBA). A physical address is represented by a block address indicating the destination block to be written to and an offset within the block indicating the location within the destination block to be written to. A block address is a block identifier used to identify a specific block included in the blocks of NAND flash memory 12.

[0018] When memory system 4 starts up, the logical-to-physical address translation table can be read from NAND flash memory 12 and stored in area 14b. The logical-to-physical address translation table stored in area 14b can be written to NAND flash memory 12 at appropriate times during the operation of memory system 4.

[0019] CPU 28 can use a logical / physical address translation table to identify the physical storage location of data in NAND flash memory 12.

[0020] The buffer 14c includes: a write buffer for temporarily storing data to be written to the NAND flash memory 12; and a read buffer for temporarily storing data read from the NAND flash memory 12.

[0021] The controller 16 acts as a memory controller for controlling the NAND flash memory 12. The controller 16 is configured, for example, as a system-on-a-chip (SoC).

[0022] A page of NAND flash memory 12 can only be written to once per program / erase cycle. Therefore, controller 16 writes the updated data corresponding to a specific logical address to a physical storage location other than the physical storage location where the old data corresponding to that logical address is stored. Controller 16 updates the logical / physical address translation table to associate that logical address with the other physical storage location. In this way, controller 16 invalidates the old data. The data referenced by the logical / physical address translation table, i.e., associated with a logical address, is called valid data. Data not associated with any logical address is called invalid data. Valid data refers to data that host 2 may request to read later. Invalid data refers to data that host 2 may not request to read.

[0023] The controller 16 includes a host interface circuit (HIT) 22, a non-volatile memory interface circuit (NVI) 24, a volatile memory interface circuit (VVI) 26, a CPU 28, an error correction code encoder / decoder (ECC encoder / decoder) 30, and a direct memory access controller (DMAC) 32. These components (HIT 22, NVI 24, VVI 26, CPU 28, ECC encoder / decoder 30, and DMAC 46) are interconnected via bus 34.

[0024] Host I / F 22 is used as circuitry to receive various requests and data items from host 2. For example, host I / F 22 may be a PCIe controller. In a configuration where memory system 4 is connected to host 2 via Ethernet, host I / F 22 may be an NVMe over Fabrics™ (NVMeOF™) controller.

[0025] Host I / F 22 receives various requests and data items from Host 2. Then, Host I / F 22 transfers the received requests and data items to one of Non-volatile Memory I / F 24, Volatile Memory I / F 26, or CPU 28.

[0026] An example of a request is a write request and a read request. A write request is a request to write data to NAND flash memory 12. A write request specifies, for example, the LBA corresponding to the data to be written, the length of the data to be written, and the address of a buffer indicating the location in the host 2's memory where the written data is stored. A read request is a request to read data from NAND flash memory 12. A read request specifies, for example, the LBA corresponding to the data to be read, the length of the data to be read, and the address of a buffer indicating the location in the host 2's memory where the read data will be transferred.

[0027] Host I / F 22 also serves as circuitry for sending responses and data corresponding to requests to Host 2. An example of a response is a write completion response or a read completion response. Host I / F 22 receives the responses and data to be sent from one of the non-volatile memory I / F 24, volatile memory I / F 26, or CPU 28.

[0028] The non-volatile memory I / F 24 electrically connects the controller 16 and the NAND flash memory 12. The non-volatile memory I / F 24 conforms to interface standards such as Toggle DDR and Open NAND Flash Interface (ONFI).

[0029] The non-volatile memory I / F 24 serves as a NAND control circuit, used to control the NAND flash memory 12.

[0030] Volatile memory I / F 26 serves as a DRAM control circuit, used to control access to DRAM 14.

[0031] CPU 28 is a processor used to control host I / F 22, non-volatile memory I / F 24, volatile memory I / F 26, and ECC encoder / decoder 30. CPU 28 performs various processes by executing firmware loaded into DRAM 14. CPU 28 can perform command processing, etc., to handle various requests from host 2. The operation of CPU 28 is controlled by the firmware it executes.

[0032] When CPU 28 receives a request from host 2, it generates a command to be sent to NAND flash memory 12 to fulfill the request. CPU 28 sends this command to NAND flash memory 12 via non-volatile memory I / F 24. NAND flash memory 12 executes the operation according to the command and sends a completion response to CPU 28 via non-volatile memory I / F 24 after completing the command.

[0033] CPU 28 can also function as coordinator 28a, which, for example, selectively suspends write operations based on the cumulative execution time (or "die time") of previous read / write operations to coordinate the read / write performance of memory system 4. Coordinator 28a can be implemented through firmware executed by CPU 28. Detailed information about coordinator 28a will be described later.

[0034] When data is written to NAND flash memory 12, ECC encoder / decoder 30 encodes the data and adds error correction codes (ECC) as redundancy codes. When data is read from NAND flash memory 12, ECC encoder / decoder 30 uses the added ECC codes in the read data to perform error correction on the data.

[0035] DMAC 32 is a circuit that performs direct memory access (DMA). Under the control of CPU 28, DMAC 32 performs data transfer between the host 2's memory (not shown) and DRAM 14.

[0036] The functions of each component in controller 16 can be implemented through dedicated hardware in controller 16, or through firmware executed by CPU 28, or a combination of both.

[0037] NAND flash memory 12 may include multiple NAND flash memory chips D0, D1, D2…Dm-1. In Figure 1, these NAND flash memory chips are simply referred to as flash memory chips D0, D1, D2…Dm-1. In this invention, sometimes one of these NAND flash memory chips that is not specifically designated is simply referred to as NAND flash memory chip D. Each individual NAND flash memory chip D can operate independently. Each NAND flash memory chip D can also function as a parallel operating unit. It should be noted that the NAND flash memory chip D can have a multi-plane configuration. In this case, each plane of the NAND flash memory chip D can also function as a parallel operating unit.

[0038] When the NAND flash memory 12 includes multiple NAND flash memory dies D, the block address indicating the destination block to be written is represented by a combination of a die identifier and a block identifier. When the NAND flash memory die D includes multiple planes, the block address is represented by a combination of a die identifier, a plane identifier, and a block identifier.

[0039] The processing of coordinator 28a is described.

[0040] Clients of memory systems (such as mainframe users in data centers) often have complex workloads. Some memory systems have write pause functionality to coordinate read and write performance.

[0041] Read performance is expressed as the read operation time of the memory system. Write performance is expressed as the write operation time of the memory system. Read operations include command generation, sensing, and data output operations. Sensing operations refer to the operation of the NAND flash memory 12 sensing data in the memory cell. Data output operations refer to the operation of transferring the sensed data from the NAND flash memory 12 to the controller 16. Commands associated with read operations include sensing commands and data output commands. Write operations include command generation, data input, and programming operations. Data input operations refer to the operation of transferring write data from the controller 16 to the NAND flash memory 12. Programming operations refer to the operation of the NAND flash memory 12 programming the write data into the memory cell. Commands associated with write operations include data input commands and programming commands.

[0042] The read and write performance of a memory system is coordinated by adjusting the ratio between read and write operation times. This ratio will be referred to as the W / R ratio below.

[0043] The write operation time of a single write request can sometimes be several times or even tens of times longer than the read operation time of a single read request. Therefore, when a memory system receives a read request while performing a write operation, the read operation must wait for the write operation to complete before it can begin, resulting in a prolonged time from receiving the read request to completing the read operation. In other words, the memory system's response efficiency to read requests decreases significantly.

[0044] The write pause function suspends an ongoing write operation performed by the memory system upon receiving a read request, performs a read operation during the pause, and then resumes the paused write operation after the read operation is completed. This improves the memory system's responsiveness to read requests. The write pause function is sometimes referred to as the read pause function.

[0045] However, when the host sends unthrottled jobs to the memory system, the read and write performance of the memory system varies depending on the workload and the internal state of the memory system. In this case, a simple write pause function may not meet the customer's requirements for the read and write performance of the memory system. For example, even a memory system with write pause functionality may, in some cases, only process write requests or only process read requests.

[0046] A throttled job is a job for which the host throttles the response time to the speed that the memory system can handle. An unthrottled job is a job for which the host does not throttle the response time. An unthrottled job is a job for which the host directly places the request into the submission queue without considering the read / write performance of the memory system.

[0047] One example of a workload that impacts memory system read / write performance is the size of data contained in a submitted queue of configurations or requests.

[0048] Examples of internal states of a memory system that affect read / write performance include pause conditions for write operations, the configuration of read and write buffers, and the over-provisioning ratio of NAND flash memory.

[0049] Memory systems can perform compaction (garbage collection) operations. Compaction refers to the operation of reducing the number of active blocks (containing a mixture of valid and invalid data) and increasing the number of free blocks. An active block is a block that contains at least one valid data item. A free block is a block that contains no valid data items and only invalid data items. After an erase operation, free blocks can be used as destination blocks for writing new data. Compaction operations include reading valid data from active blocks (compact read operation) and writing valid data to another block (compact write operation). Compaction operations convert active blocks into free blocks.

[0050] NAND flash memory read and write operations include not only responses to host read and write requests, but also operations resulting from compression. Under low overprovisioning conditions, compression operations are performed more frequently. In this situation, operations resulting from host requests and operations resulting from compression are prone to conflict. In other words, overprovisioning can negatively impact the read and write performance of the memory system.

[0051] Customers sometimes wish to specify the W / R ratio of the memory system. To meet this demand, the concept of die time fairness can be understood to be adopted. Grain time fairness means that the W / R ratio of each flash memory chip D in the NAND flash memory has a desired value by adjusting the time of each operation. In the present invention, the total cumulative read operation time for each flash memory chip D is referred to as the read chip time. The total cumulative write operation time of each flash memory chip D is called the write chip time.

[0052] When both read requests and write requests are stored in the commit queue, the extraction operations of read requests and write requests can be coordinated from the commit queue, thereby allowing adjustment of the W / R ratio to meet customer needs. However, if only one of the read request or write request is stored in the commit queue, only operations related to the stored request can be performed. Therefore, the W / R ratio cannot be adjusted by adjusting the extraction operation. In order to make the W / R ratio adjustable, it is necessary to wait for the read request and the write request to be stored in the commit queue at the same time before performing the action related to the request (especially the extract operation). However, doing so reduces the latency of the memory system and is therefore not ideal.

[0053] To make the W / R ratio adjustable, a larger buffer memory needs to be provided in the memory system to adjust the timing of sending commands to NAND flash memory when read requests and write requests are stored in commit queues with deeper queue depths than the read and write performance of NAND flash memory. However, providing a larger buffer memory is not ideal.

[0054] The memory system 4 according to this embodiment determines: (1) the timing of performing the time addition of the read operation, and the timing of the time of the write operation time, (2) the timing of determining which of the longer read the grain time or the write grain time, (3) the conditions for suspending the write operation, (4) the conditions for resuming the write operation, etc., thereby adjusting the read and write operations of the memory system 4 to meet customer demand.

[0055] 2 is a diagram of a sample configuration for example coordinator 28a. The coordinator 28a includes a selector 54 , a judger 56 , a buffer allocator 58 , a buffer allocator 64 , a layout allocator 68 , a selector 62 , and a command scheduler 70 . The functionality of these components can be achieved through CPU 28 execution firmware, specialized hardware in controller 16, or a combination of both.

[0056] Host 2 writes a request to commit queue 52. This request is, for example, an unthrottled operation. The output of commit queue 52 is provided as input to coordinator 28a. As shown in the example of Figure 2, the commit queues include commit queue 52a for handling read requests and commit queue 52b for handling write requests. Alternatively, read and write requests can be mixed and written into a single queue.

[0057] Read requests from submission queue 52a and write requests from submission queue 52b are input to the decision 56 via a selector 54, which has two inputs and one output. It should be noted that the requests input to the decision 56 may include requests generated in the memory system 4, such as requests for compressed read operations (compressed read requests). In this case, the selector 54 is configured as a selector with three inputs and one output, and requests generated in the memory system 4 are also input to the selector 54.

[0058] Determiner 56 determines whether the request input from selector 54 is a read request or a write request.

[0059] A read request is input from the decision 56 to the buffer allocator 58. The buffer allocator 58 responds to the read request by allocating a read buffer in buffer 14c. Based on the LBA contained in the read request, the target entity address is output from the logical / entity address translation table 60 stored in the logical / entity address translation table area 14b. Read command generation information is generated, containing at least the read request, identification information on the read buffer, and the target entity address. This entity address includes at least a die identifier. This read command generation information is input to a selector 62, which has two inputs and one output.

[0060] A write request is input from the determiner 56 to the buffer allocator 64. The buffer allocator 64 responds to the write request by allocating a write buffer in buffer 14c. Subsequently, the controller 16 retrieves the write data from the host 2 and writes it to the write buffer. The layout allocator 68 determines the write destination entity address. The layout allocator 68 updates the logical / entity address translation table 60 to associate the determined entity address with the LBA contained in the write request. Write command generation information is generated, containing at least the write request, identification information on the write buffer, and the write destination entity address. This entity address includes at least a die identifier. The write command generation information is input to the selector 62.

[0061] Selector 62 transmits read command generation information or write command generation information to command scheduler 70 based on the following factors: (1) the timing of adding read operation times and the timing of adding write operation times, (2) the timing of determining which is longer, the read die time or the write die time, (3) the conditions for pausing write operations, and (4) the conditions for resuming write operations. Therefore, the ratio between the number of read operation-related commands and the number of write operation-related commands generated by command scheduler 70 can be adjusted to meet customer needs.

[0062] Command scheduler 70 generates information based on read commands, producing sensing commands and data output commands for each flash memory chip D. Command scheduler 70 transmits the generated commands to NAND flash memory 12 via non-volatile memory I / F 24. Command scheduler 70 generates information based on write commands, producing data input commands and program commands for each flash memory chip D. Command scheduler 70 transmits the generated commands to NAND flash memory 12 via non-volatile memory I / F 24.

[0063] NAND flash memory 12 reads data from the target memory cell according to a sensing command. NAND flash memory 12 transmits data to controller 16 according to a data output command. NAND flash memory 12 retrieves write data from controller 16 according to a data input command. NAND flash memory 12 writes data to the target memory cell according to a program command.

[0064] When a read request is received during a write operation (such as a data input operation or a program operation), the command scheduler 70 can generate a pause command, and subsequently generate a sense command and a data output command. The command scheduler 70 transmits the generated commands to the NAND flash memory 12 via the non-volatile memory I / F 24. The NAND flash memory 12 pauses the write operation according to the pause command. The NAND flash memory 12 performs a sense operation according to the sense command and a data output operation according to the data output command.

[0065] After the data output operation is completed, the command scheduler 70 generates a recovery command. The command scheduler 70 transmits the generated command to the NAND flash memory 12 through the non-volatile memory I / F 24. The NAND flash memory 12 resumes the write operation according to this recovery command.

[0066] Figure 3 is a diagram illustrating an example of how coordinator 28a coordinates read and write performance. Coordinator 28a has a write pause function based on die time fairness. In Figure 3, W represents the write operation time. Specifically, W represents the operation time of a write operation (host write operation) performed in response to a write request (host write request) received from host 2, or the operation time of a write operation (compressed write operation) performed due to a compression operation. The write operation time includes the command generation time of data input commands and program commands, the data input time, and the program time.

[0067] In Figure 3, R represents the host read operation time. The host read operation is performed in response to receiving a read request (host read request) from host 2. The host read operation time includes the command generation time of the sensing command and the data output command, the sensing time, and the data output time.

[0068] Each inverted triangle in Figure 3 represents a pause point in the write operation.

[0069] As shown in Figure 3, the coordination period refers to the duration from the completion of the last (e.g., the last) write operation (e.g., the most recently completed write operation) to the completion of the current write operation. The controller 16 can identify whether a program operation (i.e., a write operation) has completed, for example, by sending a status read command to the NAND flash memory 12. Alternatively, the ready / busy signal output by the NAND flash memory 12 indicates a ready state, which allows the controller 16 to identify whether a program operation has completed. The coordinator 28a coordinates read and write performance within the coordination period.

[0070] Write die time (DTw) includes the estimated write operation time at the start of a write operation. The following example of a procedure for calculating write die time (DTw) is described with reference to Figure 5.

[0071] The read time DTr is the cumulative total time (actual measured value) of host read operations. The time taken for each completed host read operation is added to the read time DTr. An example of the procedure for calculating the read time DTr will be described later with reference to Figure 6.

[0072] When host I / F 22 receives a host read request from host 2 during a write operation on NAND flash memory 12, coordinator 28a makes a determination related to the die time. That is, coordinator 28a compares the ratio Rrw (=DTr / DTw) between the read die time DTr and the write die time DTw within the coordination cycle with the first value requested by the client.

[0073] This first value is set by, for example, host 2. For example, with a first value of 1.0, coordinator 28a will pause write operations and perform read operations, making the processing time for read requests equal to the processing time for write requests. With a first value of 2.0, coordinator 28a will pause write operations and perform read operations, making the processing time for read requests twice that of write requests. With a first value of 0.5, coordinator 28a will pause write operations and perform read operations, making the processing time for read requests half that of write requests.

[0074] If the ratio Rrw is less than a first value, meaning the write die time DTw is relatively long, the coordinator 28a sends a pause command to the NAND flash memory 12, causing the NAND flash memory 12 to pause write operations. The coordinator 28a then causes the NAND flash memory 12 to perform sensing operations and data output operations corresponding to the received host read request. If the write operation is paused and the aforementioned read operation is performed, the read die time DTr will increase, and the ratio Rrw will approach the first value. It should be noted that the paused write operation may include command generation operations. However, the time required for command generation operations is very short; therefore, command generation operations are generally not targeted for pausing.

[0075] If the ratio Rrw reaches or exceeds a first value (i.e., the read die time DTr is relatively long), the coordinator 28a sends a non-pause command to the NAND flash memory 12. During this coordination cycle, no write operations are paused, and no host read operations are performed. Therefore, the read die time DTr does not increase, and the ratio Rrw does not change.

[0076] Figure 3 illustrates an example: After the previous write operation was completed and before the current write request was received, the coordinator 28a received three host read requests. The memory system 4 executed three host read operations R1, R2, and R3 corresponding to these three host read requests, respectively. The coordinator 28a calculated the cumulative total time of the host read operations R1, R2, and R3 as the read chip time DTr.

[0077] Subsequently, memory system 4 receives a write request and begins the write operation. Coordinator 28a uses the estimated time required to perform the write operation as the write die time DTw. If, during the write operation, a host read request is received, and the ratio Rrw (=DTr / DTw) of the read die time DTr to the write die time DTw is less than a first value, memory system 4 pauses the write operation and performs a host read operation. In the example of Figure 3, if a fourth host read request is received during the write operation of the coordination cycle, memory system 4 pauses the write operation and performs a host read operation R4. The write operation performed at this time is represented as sub-write operation W1. Coordinator 28a adds the time of host read operation R4 to the read die time DTr. After host read operation R4 is completed, memory system 4 resumes the write operation. Subsequent resumed write operations are represented as sub-write operation W2.

[0078] Upon receiving a fifth host read request during sub-write operation W2 in the coordination cycle, memory system 4 suspends sub-write operation W2 and executes host read operation R5. Coordinator 28a adds the time of host read operation R5 to the read die time DTr. After host read operation R5 is completed, memory system 4 resumes write operations. This resumption of write operations is referred to as sub-write operation W3.

[0079] Upon receiving a sixth host read request during sub-write operation W3 of the coordination cycle, memory system 4 suspends sub-write operation W3 and executes host read operation R6. Coordinator 28a adds the time of host read operation R6 to the read die time DTr. After host read operation R6 is completed, memory system 4 resumes write operations. This resumption of write operations is referred to as sub-write operation W4.

[0080] Upon receiving a seventh host read request during sub-write operation W4 in the coordination cycle, memory system 4 suspends sub-write operation W4 and executes host read operation R7. Coordinator 28a adds the time of host read operation R7 to the read die time DTr. After host read operation R7 is completed, memory system 4 resumes write operations. This resumption of write operations is referred to as sub-write operation W5.

[0081] Upon receiving an eighth host read request during sub-write operation W5 of the coordination cycle, memory system 4 suspends sub-write operation W5 and executes host read operation R8. Coordinator 28a adds the time of host read operation R8 to the read die time DTr. After host read operation R8 is completed, memory system 4 resumes write operations. This resumption of write operations is referred to as sub-write operation W6.

[0082] Assuming that the ratio Rrw reaches or exceeds the first value after the time of the host read operation R8, then even if memory system 4 receives a host read request during a write operation, memory system 4 will not pause the write operation again. That is, memory system 4 will continue executing sub-write operation W6 until the write operation is complete.

[0083] Figure 4 is another example diagram used to illustrate the read / write performance of the example coordinator 28a. Similar to Figure 3, W in Figure 4 represents the write operation time. Similarly to Figure 3, R in Figure 4 represents the host read operation time. Furthermore, r in Figure 4 represents the compressed read operation time.

[0084] As described above, the read requests input to coordinator 28a include not only host read requests but also compressed read requests generated by CPU 28. The write requests input to coordinator 28a include host write requests sent from host 2 and compressed write requests generated by CPU 28.

[0085] Compressed read operations and compressed write operations work together. Therefore, as shown in Figure 4, the coordinator 28a does not accumulate the time of the compressed read operation r as the read die time DTr, but instead adds it to the write die time DTw. That is, the write die time DTw is the sum of the accumulated time (actual measured value) of all compressed read operations, plus the estimated write operation time (host write operation time or compressed write operation time) when a write operation (host write operation or compressed write operation) begins.

[0086] Similarly, as shown in the example in Figure 4, and similar to the example in Figure 3, by adjusting the number of pauses during a write operation according to the ratio Rrw, read performance and write performance can be coordinated to meet customer needs.

[0087] Figure 4 illustrates an example where, after the previous write operation, coordinator 28a receives four host read requests and four compressed read requests before receiving the current write request. Memory system 4 executes four host read operations R1, R2, R3, and R4, and four compressed read operations r1, r2, r3, and r4, corresponding to the four host read requests and the four compressed read requests, respectively. Coordinator 28a calculates the cumulative total time of host read operations R1, R2, R3, and R4 as the read die time DTr. An example of the procedure for calculating the read die time DTr will be explained later with reference to Figure 6. Coordinator 28a calculates the cumulative total time of compressed read operations r1, r2, r3, and r4 as the write die time DTw. An example of the procedure for calculating the write die time DTw will be explained later with reference to Figure 5.

[0088] Subsequently, memory system 4 receives a write request and begins the write operation. Coordinator 28a adds an estimate of the time required to perform the write operation to the write die time DTw. If, during the write operation, a host read request is received, and the ratio Rrw is less than a first value, memory system 4 pauses the write operation and performs a host read operation. As shown in the example in Figure 4, if a fifth host read request is received during the coordination period of the write operation, memory system 4 pauses the write operation (sub-write operation W1) and performs a host read operation R5. Coordinator 28a adds the time of host read operation R5 to the read die time DTr. After host read operation R5 is completed, memory system 4 resumes the write operation (sub-write operation W2). Similarly, memory system 4 performs sub-write operations W2 to W8 and host read operations R6 to R12. Coordinator 28a adds the time of host read operations R6 to R12 to the read die time DTr.

[0089] Assuming that the ratio Rrw reaches or exceeds the first value during the time of the host read operation R12, then even if memory system 4 receives a host read request during the write operation, memory system 4 will not pause the write operation. That is, memory system 4 will continue executing sub-write operation W9 until the write operation is complete.

[0090] It is important to note that in the example shown in Figure 4, no compressed read requests are generated after the write operation at the start of the coordination cycle. However, no restrictions are imposed on this. If a compressed read request has been generated, the coordinator 28a can pause the write operation and perform a compressed read operation. In this case, the coordinator 28a can add the time of the executed compressed read operation to the write die time DTw.

[0091] Figures 5, 6, 7, and 8 are flowcharts illustrating an example procedure for coordinator 28a to coordinate read and write performance. Before coordinator 28a performs the coordination operation, host 2 informs memory system 4 of a first value Ref to be compared with the ratio Rrw. This first value Ref is the ratio between the read operation time and the write operation time of each flash memory die in memory system 4 that host 2 expects.

[0092] Figure 5 is a flowchart illustrating an example procedure for coordinator 28a to determine the write die time DTw. Coordinator 28a determines whether a write operation is being performed (step S12). This determination can be performed by sending a status read command to NAND flash memory 12 or by checking the ready / busy signal output by NAND flash memory 12. As shown in Figure 3 or Figure 4, if a write operation includes multiple sub-write operations, the write operation is not determined to have been performed until the last sub-write operation is completed. If a write operation is being performed (step S12 is "Yes"), coordinator 28a re-executes step S12.

[0093] If there is no write operation in progress (step S12 is "No"), then coordinator 28a resets the write die time DTw (step S14). That is, when a write operation is completed, the write die time DTw will be reset.

[0094] Coordinator 28a determines whether the compressed read operation is complete (step S16). This determination can be based on whether a completion response for the compressed read operation is received from NAND flash memory 12. If the compressed read operation is complete (step S16 is "Yes"), coordinator 28a adds the compressed read operation time to the write die time DTw (step S18). That is, if a compressed read operation is completed, the time of the completed operation is added to the write die time DTw, as shown in Figure 4. If the compressed read operation is not completed (step S16 is "No"), coordinator 28a does not execute step S18.

[0095] After step S18, or if the determination result of step S16 is "No", coordinator 28a determines whether a write operation has started (step S20). This determination can be based on whether a write request has been received. If the write operation has not started (step S20 is "No"), coordinator 28a re-executes step S16. If the write operation has started (step S20 is "Yes"), coordinator 28a estimates the write operation time. As shown in Figure 3 or Figure 4, coordinator 28a adds the estimated write operation time to the write die time DTw (step S22).

[0096] Subsequently, the coordinator 28a determines whether the compressed read operation is complete (step S32). If the compressed read operation is complete (step S32 is "Yes"), the coordinator 28a adds the compressed read operation time to the die write time DTw (step S34). If the compressed read operation is not complete (step S32 is "No"), the coordinator 28a does not execute step S34.

[0097] After executing step S34, or if step S32 is "No", the coordinator 28a determines whether a write operation is being performed (step S36). If the write operation is being performed (step S36 is "Yes"), the coordinator 28a executes step S32 again.

[0098] If the write operation is not in the execution state (step S36 is "No"), then the coordinator 28a completes the process of determining the write die time DTw. As shown in Figure 3, if no compressed read operation is performed in a coordination cycle, the write die time DTw is the estimated write operation time when the write operation begins. As shown in Figure 4, if one or more compressed read operations are performed in a coordination cycle, the write die time DTw is the sum of the estimated write operation time when the write operation begins and the accumulated compressed read times.

[0099] It should be noted that controller 16 may not perform any compressed read operations during the write operation. In this case, coordinator 28a can omit the determination steps in steps S32 and S34.

[0100] Figure 6 is a flowchart illustrating an example of the procedure by which coordinator 28a determines the read die time DTr. Coordinator 28a determines whether a write operation is in progress (step S42). The determination process for step S42 can be performed in a similar manner to step S12. If a write operation is in progress (step S42 is "Yes"), coordinator 28a will execute step S42 again. Similarly, as shown in Figure 3 or Figure 4, if a write operation includes multiple sub-write operations, the write operation is not determined to have been executed until the last sub-write operation is completed. If the write operation is not in progress (step S42 is "No"), coordinator 28a will reset the read die time DTr (step S44). That is, when a write operation is completed, the read die time DTr is reset.

[0101] Coordinator 28a determines whether the host read operation is complete (step S46). This determination may be based on whether a completion response for the host read operation is received from NAND flash memory 12. If the host read operation is complete (step S46 is "Yes"), coordinator 28a adds the host read operation time to the read die time DTr (step S48). If the host read operation is not complete (step S46 is "No"), coordinator 28a does not execute step S48.

[0102] After step S48, or if the determination result of step S46 is "No", coordinator 28a determines whether a write operation has started (step S50). The determination process of step S50 can be performed in a similar manner to step S20. If the write operation has not started (step S50 is "No"), coordinator 28a re-executes step S46.

[0103] After the write operation begins (step S50 is "Yes"), the coordinator 28a determines whether the host read operation is complete (step S52). This host read operation can be, for example, a pause in the read. The determination process in step S52 can be performed in a similar manner to step S46. If the host read operation is complete (step S52 is "Yes"), the coordinator 28a adds the host read operation time to the read die time DTr (step S54). If the host read operation is not complete (step S52 is "No"), the coordinator 28a does not execute step S54.

[0104] After step S54, or if the determination result of step S52 is "no", the coordinator 28a determines whether the write operation started in step S50 is being performed (step S56). The determination process of step S56 can be performed in a similar manner to step S12.

[0105] If a write operation is in progress (step S56 is "Yes"), coordinator 28a re-executes step S52. If no write operation is currently in progress (step S56 is "No"), coordinator 28a ends the process of determining the read die time DTr.

[0106] Figure 7 is a flowchart illustrating an example of how the coordinator 28a determines whether to apply the write pause function based on the ratio Rrw. If many read operations were performed between the completion of the previous write operation and the start of the current write operation, resulting in a longer read die time DTr, the current write request can be considered a throttling task, for example. In this case, the controller 16 will not apply the write pause function based on the ratio Rrw, but rather based on a preset rule.

[0107] Coordinator 28a determines whether a write operation is currently in progress (step S62). The determination process for step S62 can be performed in a similar manner to step S12. If a write operation is currently in progress (step S62 is "yes"), coordinator 28a re-executes step S62.

[0108] If a write operation is not in progress (step S62 is "No"), coordinator 28a determines whether to start the write operation (step S64). The determination process for step S64 can be performed in a similar manner to step S20. If the write operation has not started (step S64 is "No"), coordinator 28a re-executes step S64. If the write operation has started (step S64 is "Yes"), coordinator 28a determines whether the read die time DTr is less than the determination threshold Th_DTr (step S66). The determination threshold Th_DTr is used to determine whether to apply the write pause function based on the ratio Rrw.

[0109] If the read die time DTr is less than the determined threshold Th_DTr (yes in step S66), the coordinator 28a determines to apply the write pause function based on the ratio Rrw (step S70). If the read die time DTr is greater than or equal to the determined threshold Th_DTr (no in step S66), the coordinator 28a determines to apply the write pause function based on a preset rule (step S72).

[0110] An example of a write pause function based on preset rules is as follows: if the number of write pauses is less than the upper limit set in memory system 4, the write operation is paused; and if the number of write pauses reaches or exceeds the upper limit, the write operation continues without pausing. Another example of a write pause function based on preset rules is as follows: if the accumulated total write pause time is less than the upper limit time, the write operation is paused; and if the write pause time reaches or exceeds the upper limit time, the write operation continues without pausing. After executing step S70 or S72, the process ends.

[0111] Figure 8 is a flowchart illustrating an example of the procedure for executing a write pause function if the coordinator 28a determines, based on the ratio Rrw in step S70 illustrated in Figure 7, to apply the write pause function. The coordinator 28a determines whether a read request is received during the write operation (step S82). The read request is either a host read request or a compressed read request. If no read request is received during the write operation (step S82 is "No"), the coordinator 28a will repeat the determination in step S82. If a read request is received during the write operation (step S82 is "Yes"), the coordinator 28a will determine whether to allow the write operation to be paused based on a preset rule (step S84). Even if the write pause function is applied based on the ratio Rrw, the preset rule will still be applied. It should be noted that the determination in step S84 is the same as the determination in step S66 described with reference to Figure 7.

[0112] Based on the determination result of step S84 (step S86), coordinator 28a determines whether to allow write pauses. If the number of write pauses reaches or exceeds a preset upper limit, and / or the accumulated total write pause time reaches or exceeds a preset upper limit, then coordinator 28a determines that write pauses are not allowed.

[0113] If write pause is not allowed (step S86 is "No"), coordinator 28a does not pause the write operation and continues to execute the write operation (step S88). The read request received in step S82 will be executed after the write operation is completed.

[0114] If write pause is allowed (step S86 is "Yes"), coordinator 28a determines whether the read die time DTr is less than the write die time DTw multiplied by a first value Ref (step S90). If the read die time DTr is less than the write die time DTw multiplied by the first value Ref, it means that the ratio Rrw (=DTr / DTw) is less than the first value Ref. This first value Ref is the ratio between the read time and write time of each flash memory die as expected by the customer.

[0115] If the read die time DTr is greater than or equal to the write die time DTw multiplied by the first value Ref ("No" in step S90), then the coordinator 28a does not pause the write operation and continues to execute the write operation (step S88). If the read die time DTr is less than the write die time DTw multiplied by the first value Ref ("Yes" in step S90), then the coordinator 28a pauses the write operation (step S92). That is, the coordinator 28a sends a pause command to the NAND flash memory 12. After receiving the pause command, the NAND flash memory 12 pauses the ongoing write operation.

[0116] Coordinator 28a executes a pause read (step S94). That is, coordinator 28a sends a read command to NAND flash memory 12. After receiving the read command, NAND flash memory 12 performs a read operation.

[0117] Coordinator 28a determines whether there are any unexecuted read requests stored (step S96). Coordinator 28a may determine whether there are any read requests in the commit queue. Alternatively, coordinator 28a may determine, for example, whether there are any read requests retrieved from the commit queue but not yet executed stored in volatile memory 14.

[0118] If there are any unexecuted read requests (marked "Yes" in step S96), then the coordinator 28a determines whether to allow the read to be paused based on preset rules (step S98). The determination in step S98 is the same as the determination in step S84.

[0119] Based on the determination result of step S98, coordinator 28a determines whether to allow read pause (step S100). If the number of write pauses reaches or exceeds a preset upper limit, and / or the accumulated total write pause time reaches or exceeds a preset upper limit, then coordinator 28a determines that read pause is not allowed.

[0120] If pausing the read is allowed (step S100 is "Yes"), the coordinator 28a determines whether the read die time DTr is less than the write die time DTw multiplied by the first value Ref (step S102).

[0121] If the read die time DTr is less than the write die time DTw multiplied by the first value Ref (if step S102 is "Yes"), then the coordinator 28a performs a pause read (step S94). That is, the coordinator 28a sends a read command to the NAND flash memory 12 to execute the unexecuted read request identified in step S96.

[0122] If no unexecuted read request is stored (step S96 is "No"), or reading is not allowed to be paused (step S100 is "No"), or the read die time DTr is greater than or equal to the write die time DTw multiplied by the first value Ref (step S102 is "No"), the coordinator 28a resumes the paused write operation (step S104). That is, the coordinator 28a sends a recovery command to the NAND flash memory 12. After receiving the recovery command, the NAND flash memory 12 resumes the execution of the paused write operation.

[0123] After step S88 or S104, the process ends.

[0124] If a read request occurs during a write operation, the memory system 4 in this embodiment will pause the write operation and perform a read operation. Therefore, a read operation can be performed without waiting for the write operation to complete, and the ratio Rrw between the read die time DTr and the write die time DTw can be adjusted according to the user's expectations.

[0125] If the time interval between the completion of the last write operation and the start of the current write operation is long, and a large number of read operations have already been performed when the write request occurs (step S66 in Figure 7 is "No"), then the memory system 4 will not coordinate read performance and write performance based on the ratio Rrw (see step S72 in Figure 7). In this case, the write operation is paused so that the number of write pauses does not exceed a preset upper limit, and / or the accumulated total write pause time does not exceed a preset upper limit (see steps S84, S86, S98, and S100 in Figure 8).

[0126] The memory system 4 in this embodiment adjusts the read and write operation times based on the read and write times during the coordination period from the completion of the previous write operation to the completion of the current write operation. Therefore, the performance of read requests executed between the completion of the previous write operation and the occurrence of the current write request can be adjusted.

[0127] The coordinator 28a coordinates performance based on the read and write times within a coordination cycle. Therefore, there is no need to use a large buffer memory for temporary requests.

[0128] If the pauseable conditions are met (steps S86 and S90 in Figure 8), the coordinator 28a pauses the write operation and executes the read request as shown in step S92. Thus, the latency of the memory system 4 is not reduced.

[0129] It is important to note that this assumes the coordination period is from the completion of a previous (e.g., the last) write operation (e.g., the most recently completed write operation) to the completion of the current write operation. However, it can also be extended to the completion of a write operation several times prior to the completion of the current write operation.

[0130] If the number of read requests in the first coordination period is large, and the read die time DTr in the first coordination period does not meet the pauseable condition (step S90 in Figure 8), the pauseable condition in the second coordination period (immediately following the first coordination period) can be changed. For example, if the read die time DTr in the first coordination period is longer than the write die time DTw multiplied by the first value Ref by a difference Δr, thus failing to meet the pauseable condition, then the pauseable condition in the second coordination period can be: the read die time DTr is less than the write die time DTw multiplied by the first value Ref plus the difference Δr, i.e., DTr < (DTw × Ref) + Δr. In other words, the determination condition in step S90 of the second coordination period can be DTr - Δr < (DTw × Ref). That is, the overrun time of the read operation in the first coordination period can be subtracted from the read die time DTr in the second coordination period, making it easier to meet the pauseable condition in the second coordination period. Therefore, if the readout time measured in the first coordination cycle exceeds the target readout time by an excess amount, this excess amount can be subtracted from the readout time measured in the subsequent second coordination cycle, thereby increasing the likelihood that the pause condition will be met in the second coordination cycle.

[0131] The requests received by coordinator 28a may include erase requests. Similar to write operations, erase operations also require a certain amount of time. Furthermore, clients sometimes wish to set the ratio between read operation time and erase time. In this case, "write" in the above description can be replaced with "erase," thereby allowing read performance and erase performance to be coordinated as described above.

[0132] For example, when adjusting the ratio between read operation time and erase operation time, "write operation" in Figures 5, 6, 7 and 8 is changed to "erase operation", "write operation time" is changed to "erase operation time", "write die time" is changed to "erase die time", and "write operation" is changed to "erase operation".

[0133] In addition, the memory system 4 can adjust the W / R ratio and the ratio between read operation time and erase operation time to achieve the desired value.

[0134] While several embodiments have been described, these embodiments are merely illustrative and not intended to limit the scope of the invention. In fact, the novel apparatus and methods described herein can be implemented in various other forms; furthermore, various omissions, substitutions, and modifications can be made to the embodiments described herein without departing from the spirit of the invention. The appended claims and their equivalents are intended to cover all forms or modifications falling within the scope and spirit of the invention.

[0135] 2: Host 4: Memory System 12: Non-volatile memory 14: Volatile memory 14a: Firmware Region 14b: Logical / Entity Address Translation Table Area 14c: Buffer 16: Controller 22: Host I / F 24: Non-volatile memory I / F 26: Volatile memory I / F 28: CPU 28a: Coordinator 30: ECC Encoder / Decoder 32, 46: Direct Memory Access Controller 34: Busbar 52, 52a, 52b: Submit queue 54, 62: Selectors 56: Decision Maker 58, 64: Buffer Distributor 60: Logical / Entity Address Translation Table 68: Layout Assigner 70: Command Scheduler S12~S36, S42~S56, S62~S72, S82~S104: Steps W1~W9: Sub-write operations R: Host read operation time r: Compress read operation time R1~R12: Host read operations DTr: Grain read time r1, r2, r3, r4: Compressed read operation D, D0, D1, D2…Dm-1: NAND flash memory chips DTw: Write time to the die Ref: First value

Claims

1. A memory system comprising: a non-volatile memory; and control circuitry electrically connected to the non-volatile memory and configured to: upon receiving a first read request from a host during a current write operation, determine a first ratio between a first duration and a second duration, the first duration being the cumulative time of one or more read operations performed from the completion of the most recently completed write operation to the completion of the current write operation, the second duration being the time required for the current write operation; and determine whether to pause the current write operation based on the first ratio.

2. The memory system according to claim 1, wherein: The control circuit is further configured to: compare the first ratio with the first value; and determine whether to pause the current write operation based on the result of the comparison.

3. The memory system according to claim 2, wherein: The control circuit is configured to use the value specified by the host as the first value.

4. The memory system according to claim 2, wherein: The control circuit is configured to: pause the current write operation when the first ratio is lower than the first value; and not pause the current write operation when the first ratio is equal to or higher than the first value.

5. The memory system according to claim 2, wherein: The control circuit is further configured to: determine whether the first duration is less than a time threshold before performing the comparison; And the comparison between the first ratio and the first value is performed only if the first duration is less than the time threshold.

6. The memory system according to claim 5, wherein: The control circuit is further configured to determine whether to pause the current write operation based on preset rules when the first duration is not less than the time threshold.

7. The memory system according to claim 6, wherein: The preset rule only allows pausing the current write operation when the cumulative number of pauses for the current write operation is less than a predetermined limit.

8. The memory system according to claim 2, wherein: The control circuit is further configured to adjust the comparison in the subsequent second coordination period based on the excess if the first duration in the first coordination period exceeds the second duration multiplied by the first value by an excess amount.

9. The memory system according to claim 8, wherein: The control circuit is configured to adjust the comparison of the subsequent second coordination cycle by subtracting the excess from the cumulative time of the read operations in the subsequent second coordination cycle.

10. The memory system according to claim 1, wherein: The control circuit is further configured to: reset the first duration in response to the completion of the most recently completed write operation; and determine the total cumulative operation time of the one or more read operations corresponding to one or more read requests received from the host as the first duration.

11. The memory system according to claim 10, wherein: The control circuit is further configured to: reset the second duration in response to the completion of the most recently completed write operation; and determine an estimate of the time required for the current write operation as the second duration.

12. The memory system according to claim 1, wherein: The control circuit is further configured to: initiate a compression operation, including one or more compression read operations; determine a third duration, which is the cumulative time of one or more compression read operations performed after the most recently completed write operation; and determine whether to pause the current write operation based on a second ratio between the first duration and the sum of the second and third durations.

13. The memory system according to claim 1, wherein: The control circuit is further configured to resume the current write operation after pausing the current write operation and executing the read operation corresponding to the first read request.

14. The memory system according to claim 13, wherein: The control circuit is further configured to determine whether to resume the current write operation based on whether there are any unexecuted read requests stored in the commit queue.

15. The memory system according to claim 1, wherein: The control circuit is further configured to: determine a third ratio using the cumulative time of one or more read operations performed from the completion of the most recent erase operation to the completion of the current erase operation, and the time required for the current erase operation; and determine whether to pause the current erase operation based on the third ratio.

16. A memory system comprising: a non-volatile memory including a plurality of memory chips; and control circuitry electrically connected to the non-volatile memory and configured to: upon receiving a first read request from a host, determine a first memory chip among the plurality of memory chips as the target of the first read request, the first read request being received during a current write operation for the first memory chip; for the first memory chip, determine a first ratio between a first duration and a second duration, the first duration being the cumulative time of one or more read operations performed in the first memory chip after the most recently completed write operation, the second duration being the time required for the current write operation in the first memory chip; and determine whether to pause the current write operation based on the first ratio.

17. A method for controlling non-volatile memory, the method comprising: receiving a first read request from a host during a current write operation performed in the non-volatile memory; in response to receiving the first read request, determining a first ratio between a first duration and a second duration, the first duration being the cumulative time of one or more read operations performed since the most recently completed write operation, the second duration being the time required for the current write operation; and determining whether to pause the current write operation based on a comparison of the first ratio with a first value specified by the host.

18. The method according to request 17 further comprises: determining that the first ratio is lower than the first value; and in response to determining that the first ratio is lower than the first value, pausing the current write operation.

19. The method according to claim 17 further comprises: determining that the first ratio is equal to or higher than the first value; and in response to determining that the first ratio is equal to or higher than the first value, continuing the current write operation.

20. The method according to claim 17 further comprises: initiating a compression operation, including one or more compression read operations; determining a third duration, the third duration being the cumulative time of one or more compression read operations performed after the most recently completed write operation; and determining whether to pause the current write operation based on a second ratio between the first duration and the sum of the second duration and the third duration.

Images