Storage system and control method of non-volatile memory

Abstract

Embodiments will realize a storage system and a control method of a nonvolatile memory that can reduce data flow throughput. The storage system of the embodiments stores write data received from a host in a shared write buffer shared among a plurality of write target blocks, acquires a plurality of pages of first write data from the shared write buffer, and writes the first write data to a first write target block using a first-stage write operation. In a case where the storage system receives write data from the host in a state where there is no empty area in the shared write buffer, the storage system discards the write data in the shared write buffer in which the first-stage write operation has ended. When the first write data is not present in the shared write buffer at a point in time at which a second-stage write operation of the first write data should be performed, the storage system sends a request to acquire the first write data to the host.

Description

[0001] Information related to divisional application

[0002] This case is a divisional application. The parent application of this divisional application is the invention patent application filed on December 14, 2018, with application number 201811532009.1 and title "Storage System and Control Method for Non-Volatile Memory".

[0003] [Related Applications]

[0004] This application claims priority to Japanese Patent Application No. 2018-78422 (filed on April 16, 2018). This application incorporates the entire contents of that basic application by reference. Technical Field

[0005] The embodiments of the present invention relate to a storage system and a control method for a non-volatile memory. Background Technology

[0006] In recent years, storage systems with non-volatile memory have become widely used. As one such storage system, solid-state drives (SSDs) based on NAND (NOT AND) flash memory technology are known.

[0007] Even in data center servers, storage systems like SSDs are used as storage devices.

[0008] In computer systems like servers that utilize storage systems, the computer system acts as the host of the storage system. Therefore, as the number of end users (clients) using the computer system increases, the storage system is required to handle more I / O. Furthermore, as the number of clients increases, there is a tendency for the amount of data transferred from the host computer system to the storage system—that is, the data throughput—to increase. Summary of the Invention

[0009] The embodiments of the present invention provide a control method for a storage system and a non-volatile memory that can reduce the data flow between the host and the storage system.

[0010] According to an embodiment, a host-connectable storage system includes: a non-volatile memory comprising a plurality of blocks; and a controller electrically connected to the non-volatile memory, comprising at least a first-stage write operation and a second-stage write operation, configured to perform a multi-stage write operation that repeatedly transfers multiple pages of the same data to the non-volatile memory. Each time the controller receives a write request from the host, it receives write data stored in any one of a plurality of write buffers on the host's memory corresponding to a plurality of write object blocks allocated from the plurality of blocks, and stores the received write data in a shared write buffer shared among the plurality of write object blocks. The controller obtains a first write data of multiple pages corresponding to a first write object block within the plurality of write object blocks from the shared write buffer, and writes the first write data to the first write object block using the first-stage write operation. When the controller receives write data from the host while there is no empty space in the shared write buffer, it discards the write data in the shared write buffer where the first-stage write operation has ended, thus ensuring the empty space in the shared write buffer. When the controller is supposed to perform the second phase of the write operation for the first write data, but the first write data is not present in the shared write buffer, it retrieves the first write data from the host by sending a request to the host to retrieve the first write data. Attached Figure Description

[0011] Figure 1 This is a block diagram illustrating the relationship between the storage system and the host in the implementation method.

[0012] Figure 2 This is a diagram showing the relationship between the multiple write buffers (host-side write buffers) set on the host and the shared write buffers (shared device-side write buffers) set on the storage system of this embodiment.

[0013] Figure 3 This is a block diagram illustrating the data write operations performed on each write object block using the storage system of this implementation via a shared write buffer.

[0014] Figure 4 This diagram illustrates the blurry-fine write operation when moving back and forth between three word lines.

[0015] Figure 5 This diagram illustrates the blurry-fine write operation when moving back and forth between two word lines.

[0016] Figure 6 This is a block diagram illustrating an example configuration of a computer system including the storage system and the host of this embodiment.

[0017] Figure 7 This is a block diagram illustrating the relationship between multiple channels and multiple NAND flash memory chips used in the storage system of this embodiment.

[0018] Figure 8 This is a diagram illustrating an example of the configuration of a superblock used in the storage system of this embodiment.

[0019] Figure 9 This is a block diagram illustrating an example of write processing performed by the host and the storage system of this implementation.

[0020] Figure 10 This is a block diagram illustrating another example of write processing performed by the host and the storage system of this embodiment.

[0021] Figure 11 This diagram illustrates the fuzzy-fine write operation performed using a shared write buffer through the storage system of this embodiment, and the process of discarding data from the shared write buffer.

[0022] Figure 12 This diagram illustrates the process performed by the storage system of this embodiment to discard write data in the shared write buffer after a fuzzy write operation has ended, assuming there are no unused areas in the shared write buffer.

[0023] Figure 13 This is a flowchart illustrating the sequence of data write processes performed by the storage system of this embodiment.

[0024] Figure 14 This diagram illustrates the data writing process where the host specifies the write target block and the storage system of this implementation determines the write target page, as well as the data reading process where the host specifies the block address and page address.

[0025] Figure 15 This is a diagram used to illustrate the block allocation instruction (block allocation request) used in the storage system of this implementation.

[0026] Figure 16 This is a diagram used to illustrate the response to block allocation instructions.

[0027] Figure 17 This is a diagram illustrating the write instructions used in the storage system of this implementation.

[0028] Figure 18 This is a diagram used to illustrate the response to a write command.

[0029] Figure 19 This is a diagram used to illustrate the read instructions applied in the storage system of this embodiment.

[0030] Figure 20 This is a block diagram illustrating the data write operations performed on each write target block by the storage system of this embodiment using a shared write buffer containing one or more SLC blocks. Detailed Implementation

[0031] Hereinafter, the embodiments will be described with reference to the accompanying drawings.

[0032] First, refer to Figure 1 This explains the relationship between the host and the storage system of the implementation method.

[0033] This storage system is a semiconductor storage device configured to write data to and read data from non-volatile memory. This storage system has been implemented as a flash memory device based on NAND flash memory technology.

[0034] The host (host device) 2 is configured to control multiple flash storage devices 3. The host 2 is implemented as a computer configured to use a flash array including multiple flash storage devices 3 as storage.

[0035] The following example illustrates a scenario where a computer, such as a server computer, functions as host 2.

[0036] The host (server computer) 2 and multiple flash storage devices 3 are interconnected (internal interconnection) via interface 50. The interface 50 used for this interconnection is not limited to this, but may include PCI Express (PCIe) (registered trademark), NVM Express (NVMe) (registered trademark), Ethernet (registered trademark), NVMe over Fabrics (NVMeOF), etc.

[0037] As a typical example of a server computer that functions as host 2, a server computer (hereinafter referred to as a server) in a data center can be cited.

[0038] In an example where host 2 is implemented via a server within a data center, host 2 (server) 2 can connect to multiple end-user terminals (clients) 61 via network 51. Host 2 can provide a wide variety of services to these end-user terminals 61.

[0039] Examples of services that can be provided by the host (server) 2 include (1) Platform-as-a-Service (PaaS) which provides the system operating platform to each client (each end-user terminal 61) and (2) Infrastructure-as-a-Service (IaaS) which provides infrastructure such as virtual servers to each client (each end-user terminal 61).

[0040] Multiple virtual machines can also run on the physical server that functions as host (server) 2. Each of these virtual machines running on host (server) 2 can function as a virtual server that provides various services to a number of corresponding clients (end-user terminals 61).

[0041] The host (server) 2 includes: storage management functions, which manage multiple flash storage devices 3 that constitute a flash array; and front-end functions, which provide various services including storage access to the end user terminal 61.

[0042] Flash storage device 3 contains non-volatile memory such as NAND flash memory. A flash storage device 3 manages multiple write target blocks allocated from multiple blocks within the non-volatile memory. A write target block represents a block of data that should be written to it.

[0043] Multiple write object blocks managed by flash storage device 3 are used by multiple clients utilizing host (server) 2. Data corresponding to a particular client is written to one or more write object blocks, while data corresponding to other clients is written to other write object blocks.

[0044] If only one flash storage device 3 is connected to host (server) 2, then the same or more blocks as the total number of clients using host (server) 2 are allocated as write target blocks.

[0045] On the other hand, if multiple flash storage devices 3, such as 8 flash storage devices 3, are connected to the host (server) 2, and the host 2 allocates the same number of clients equally to the 8 flash storage devices 3, then 1 / 8 of the total number of clients can be allocated as write target blocks in each flash storage device 3.

[0046] Most NAND flash memories are configured to write multiple bits of data to each memory cell, and multiple pages of data to be written to each write target block must be held in advance in each write target block.

[0047] Therefore, in an environment where multiple write object blocks exist in the flash storage device 3 that can be used simultaneously, a write buffer must be prepared that is the same number as the number of such write object blocks. In other words, a write buffer must be prepared that is the same number as the number of clients.

[0048] Furthermore, in modern NAND flash memory, there are instances where multi-stage write operations are performed, such as transferring multiple pages of the same data multiple times to reduce program interference. These multi-stage write operations include at least a first-stage write operation and a second-stage write operation.

[0049] In the first stage of the write operation, multiple pages of data are transferred to the NAND flash memory and written to multiple memory cells within the NAND flash memory that are connected to the same word lines within a specific write target block. In the second stage of the write operation, the same multiple pages of data are again transferred to the NAND flash memory and written to the same multiple memory cells within the write target block. Thus, a typical example of a multi-stage write operation involving multiple transfers of the same data can be categorized as a fuzzy-fine write operation.

[0050] Typically, in multi-stage write operations such as fuzzy-fine write operations, data write operations are performed back and forth across multiple word lines to reduce program interference caused by interference between word lines.

[0051] For example, in a three-word-line round trip fuzzy-fine write operation, the first write data of multiple pages is transferred to the NAND flash memory, and the first write data is written to the initial physical page (multiple memory cells connected to the initial word line) (fuzzy write operation). Next, the second write data of multiple pages is transferred to the NAND flash memory, and the second write data is written to the next physical page (multiple memory cells connected to the next word line) (fuzzy write operation). Next, the third write data of multiple pages is transferred to the NAND flash memory, and the third write data is then written to the next physical page (and subsequently to the next word line) (fuzzy write operation). Then, the physical page to be written to is returned to the initial physical page (multiple memory cells connected to the initial word line), and the first write data of multiple pages is transferred to the NAND flash memory again, and the first write data is written to the initial physical page (multiple memory cells connected to the initial word line) (fine write operation).

[0052] Therefore, in instances of multi-stage write operations such as fuzzy-fine write operations, the number of page copies, represented by the product of the number of bits to be written to a memory cell and the number of round-trip word lines, must be kept in the write buffer for each write object block.

[0053] Therefore, in practice, it is quite common to write data to multiple blocks in parallel in order to achieve higher data write speeds. In this case, the amount of data in each write buffer must be increased.

[0054] Therefore, in instances where flash storage device 3 is shared by multiple clients, a write buffer must be prepared for each client, i.e., for each write object block, and a large number of page-sized data must be maintained in each write buffer.

[0055] However, the capacity of the random access memory in flash storage device 3 is usually limited, so there are situations where it is difficult to prepare a sufficient number of write buffers, each with sufficient capacity, on the random access memory in flash storage device 3.

[0056] Moreover, even if the flash storage device 3 is equipped with a large capacity of random access memory, when the number of clients sharing the flash storage device 3 is small, the large capacity of random access memory is wasted.

[0057] Therefore, in this embodiment, a specific storage area on the host 2's memory is used as multiple write buffers, each corresponding to a multiple write object block. Since each multiple write object block corresponds to a multiple client, each of these multiple write buffers corresponds to a different client.

[0058] The number of write buffers prepared on the memory of host 2 is the same as the number of write object blocks, i.e., the number of clients. In host 2, write data from a certain client is stored in the write buffer corresponding to that client. Moreover, write data from another client is stored in another write buffer corresponding to that other client.

[0059] Typically, while flash storage device 3 can manage a maximum storage capacity of around 4 GB, host 2 can manage a maximum storage capacity of several terabytes. Therefore, using the write buffer on the host 2 side can easily increase the number of clients utilizing host 2 compared to using the write buffer on the flash storage device 3 side.

[0060] However, most modern NAND flash memories require multi-stage write operations, such as fuzzy-fine write operations, to reduce program interference, as mentioned above. In this case, the same data must be transferred to the NAND flash memory multiple times. Therefore, if a configuration using only the host-side write buffer is employed, the same data must be transferred from host 2 to flash memory 3 multiple times. This increases the amount of data that must be transferred from host 2 to flash memory 3, i.e., the data throughput between host 2 and flash memory 3, potentially leading to poor I / O performance.

[0061] Therefore, in this embodiment, a write buffer (shared device-side write buffer) shared among multiple write target blocks is prepared in the flash storage device 3. Furthermore, the controller located in the flash storage device 3 receives write data stored in any one of the multiple write buffers on the host side from the host 2 each time a write request is received, and stores the received write data in the shared device-side write buffer within the flash storage device 3. That is, regardless of the write target block to which write data should be written, any write data is stored in this shared device-side write buffer.

[0062] The controller of flash memory device 3 writes data to the NAND flash memory through multi-stage write operations, such as fuzzy-fine write operations. A multi-stage write operation includes at least a first-stage write operation, such as a fuzzy write operation, and a second-stage write operation, such as a fine write operation. A fuzzy write operation roughly sets the threshold distribution for each memory cell, while a fine write operation adjusts the threshold distribution for each memory cell. Furthermore, intermediate write operations can be performed between fuzzy and fine write operations.

[0063] The controller of flash storage device 3 obtains multiple pages of write data (first write data) that should be written to a certain write object block within multiple write object blocks from the write buffer on the shared device side, and writes the first write data to the write object block through a first-stage write operation such as a fuzzy write operation.

[0064] To efficiently store write data from host 2 into the shared device-side write buffer, when new write data is received from host 2 while there is no empty space in the shared device-side write buffer, the controller of flash storage device 3 discards the write data (fuzzy write data) in the shared device-side write buffer where the first-stage write operation, such as a fuzzy write operation, has ended, ensuring empty space in the shared device-side write buffer for storing new write data. For example, if the shared device-side write buffer is entirely filled with a large amount of fuzzy write data, and new write data is received from host 2, the controller of flash storage device 3 selects specific write data to be discarded from this fuzzy write data and discards the selected write data. Thus, a large amount of write data corresponding to multiple clients can be efficiently stored in the shared device-side write buffer, which has a limited capacity.

[0065] Furthermore, when the flash storage device 3 is scheduled to perform a second-stage write operation, such as a fine-grained write operation of the first write data, and the first write data is not present in the write buffer on the shared device side, the flash storage device 3 obtains the first write data from the host 2 by sending a request (transfer request) to the host 2 to obtain the first write data.

[0066] The shared device-side write buffer has a limited capacity, but if the number of clients using host 2 is below a certain fixed number, the probability (medium probability) that the first write data exists in the shared device-side write buffer at the time when the second stage write operation should be performed is relatively high. Therefore, multi-stage write operations such as fuzzy-fine write operations can be performed without transferring the same write data from host 2 to flash storage device 3 multiple times. This reduces the data throughput between host 2 and flash storage device 3, thus improving the I / O performance of flash storage device 3 compared to transferring the same write data from host 2 to flash storage device 3 multiple times for each data write.

[0067] As the number of clients utilizing host 2 increases, the rate of write buffer usage on the shared device side decreases. However, in the event that the first write data is not present in the shared device side write buffer (failure), the controller of flash storage device 3 retrieves the first write data from host 2 by sending a transfer request to host 2. Therefore, even with an increased number of clients, multi-stage write operations such as fuzzy-fine write operations can still be performed normally.

[0068] Therefore, the flash storage device 3 can flexibly cope with the increase in the number of clients sharing the flash storage device 3, and can reduce the amount of data flow between the host 2 and the flash storage device 3.

[0069] Figure 2 This indicates the relationship between the multiple write buffers (host-side write buffers) set in host 2 and the shared write buffer (shared device-side write buffer) set in flash storage device 3.

[0070] In host (server) 2, multiple virtual machines 401 are executed, each corresponding to a different end user (multiple client). In hyperscale user-intensive data centers, there may be a large number of clients, such as hundreds or thousands, utilizing host (server) 2. These clients are also referred to as tenants.

[0071] Host (server) 2 provides independent services to each client (tenant). Figure 2 The example illustrates the use of host (server) 2 by clients #0, #1, #2, ..., #m-1.

[0072] In each virtual machine 401, the operating system 402 and user application 403 used by the corresponding client (tenant) are executed.

[0073] Furthermore, in host (server) 2, multiple I / O services 404 corresponding to multiple clients are executed. These I / O services 404 may also include block I / O services, key-value storage services, and various other I / O services. Each I / O service 404 includes an address translation table, i.e., a lookup table (LUT) 404A, which manages the mapping between each tag used to identify data and each physical address of the flash storage device 3. Here, the tag can be either a logical address specifying a location in the logical address space (e.g., a logical block address: LBA) or a key in the key-value store.

[0074] For example, in an LBA-based block I / O service, a LUT can be used to map the logical management addresses (LBAs) to the physical addresses of the flash storage device 3.

[0075] Furthermore, in key-value storage services, a LUT (Local Level Adapter) can be used to manage the mapping between each key and the physical address of the flash storage device 3 storing data corresponding to that key. This LUT can also manage the correspondence between tags, the physical address storing data identified by the tag, and the data length of that data.

[0076] Instead of converting each tag (e.g., LBA) from the user application 403 into an LBA for the flash storage device 3, each LUT404A converts each tag (e.g., LBA) from the user application 403 into a physical address of the flash storage device 3.

[0077] Furthermore, host (server) 2 is equipped with multiple host-side write buffers (#0, #1, #2, ..., #m-1) 405 corresponding to multiple clients (multiple tenants) #0, #1, #2, ..., #m-1 respectively. These host-side write buffers 405 are located in the memory of host (server) 2.

[0078] Each host-side write buffer 405 is used to temporarily hold write data from the corresponding client. For example, write data from client (tenant) #0 is stored in host-side write buffer #0, write data from client (tenant) #1 is stored in host-side write buffer #1, write data from client (tenant) #2 is stored in host-side write buffer #2, and similarly, write data from client (tenant) #m-1 is stored in host-side write buffer #m-1.

[0079] The transmission of instructions from host (server) 2 to flash storage device 3 and the return of responses from flash storage device 3 to host (server) 2 are performed via multiple I / O queues 500 corresponding to multiple clients respectively.

[0080] The controller of flash storage device 3 manages and shares multiple write object blocks (flash blocks #0, #1, #2, ..., #m-1) 602 corresponding to multiple clients (multiple tenants) #0, #1, #2, ..., #m-1 of flash storage device 3. Furthermore, flash storage device 3 includes a shared device-side write buffer 601 shared by the multiple write object blocks (flash blocks) 602. The shared device-side write buffer 601 is used to temporarily store write data that should be written to any one of the multiple write object blocks (flash blocks #0, #1, #2, ..., #m-1) 602. In other words, write data from all clients is stored in the shared device-side write buffer 601.

[0081] Figure 3 This indicates that the data write operation to each write object block is performed by the flash storage device 3 using the shared device-side write buffer 601.

[0082] Here, in flash storage device 3, we envision a scenario where data writing to each write target block is performed by the aforementioned fuzzy-fine write operation.

[0083] In order to support fuzzy-fine write operations, each host-side write buffer 405 must have sufficient capacity.

[0084] For example, with a page size of 16KB, 2 planes per NAND flash memory chip, 3 round-trip word lines, 4 bits written per memory cell (QLC), and 32 NAND flash memory chips accessed in parallel, the capacity of each host-side write buffer 405 becomes 12MB (16KB × 2 × 3 × 4 × 32). The number of host-side write buffers 405 is the same as the number of clients (tenants); therefore, the total capacity of these host-side write buffers 405 is represented by the product of 12MB and the number of clients (tenants). For example, if the number of clients (tenants) is 1000, then the total capacity of these host-side write buffers 405 becomes 12GB.

[0085] Write data requested by the user application corresponding to client (tenant) #0 is stored in host-side write buffer (#0) 405, write data requested by the user application corresponding to client (tenant) #1 is stored in host-side write buffer (#1) 405, and similarly, write data requested by the user application corresponding to client (tenant) #m-1 is stored in host-side write buffer (#m-1) 405.

[0086] Each of these host-side write buffers (#m-1) 405 is used as a cache. Therefore, when data specified by a read request from a client exists in the host-side write buffer 405 corresponding to that client, that data can also be read from the host-side write buffer 405.

[0087] The capacity of the shared device-side write buffer 601 configured in the flash storage device 3 is determined according to the required write speed. For example, the capacity of the shared device-side write buffer 601 can be any capacity in the range of 2M bytes to 3G bytes, or any capacity in the range of 2M bytes to 4G bytes.

[0088] In flash storage device 3, all write data received from host 2 is stored in the write buffer 601 on the shared device side.

[0089] The controller of flash storage device 3 writes the write data to each write object block 602 through fuzzy-fine write operations.

[0090] Each write request received from host 2 contains the block address of the write object block to which write data should be written. The controller of flash storage device 3 obtains multi-page write data (first write data) associated with one or more write requests within a specified write object block 602 (e.g., write object block #0 here) from the shared device-side write buffer 601 as write data (fuzzy data) for fuzzy write operations. For example, when the page size is 16KB and 4 bits of data are written to a storage cell, the controller of flash storage device 3 obtains 64KB of write data to be written to write object block #0 from the shared device-side write buffer 601 as fuzzy data. This 64KB of write data can also be write data associated with a write request containing the block address of the specified write object block #0. Alternatively, when the size of the write data associated with each write request containing the block address of the specified write object block #0 is less than 64KB, the 64KB of write data can also be a collection of multiple write data portions associated with several write requests respectively.

[0091] Furthermore, the controller of the flash storage device 3 writes the first write data (fuzzy data) to multiple storage cells (multiple first storage cells) connected to a word line of the write object within the write object block #0 via a fuzzy write operation.

[0092] In the same manner, the controller of flash storage device 3 performs a fuzzy write operation on one or more subsequent word lines within the write target block #0.

[0093] Furthermore, at the time when the fine write operation for the first write data should be performed, the controller of the flash storage device 3 requests the first write data from the shared device-side write buffer 601. If the first write data (click) exists in the shared device-side write buffer 601, the controller of the flash storage device 3 obtains the first write data from the shared device-side write buffer 601 as the write data (fine data) for the fine write operation. Moreover, the controller of the flash storage device 3 writes the first write data (fine data) to the aforementioned plurality of first storage cells within the write target block #0 through the fine write operation.

[0094] For each of the other write object blocks 602, the fuzzy-fine write operation is performed in the same way.

[0095] The controller of the flash storage device 3, for example, obtains multi-page write data (second write data) associated with one or more write requests to a specified write target block #m-1 from the shared device-side write buffer 601 as write data (fuzzy data) for fuzzy write operations. Furthermore, the controller of the flash storage device 3 writes this second write data (fuzzy data) to multiple storage cells (multiple second storage cells) connected to a word line of the write target within the write target block #m-1 via fuzzy write operations.

[0096] In the same manner, the controller of flash storage device 3 performs a fuzzy write operation on the subsequent word line within the write target block #m-1.

[0097] Furthermore, at the time when a fine write operation for the second write data should be performed, the controller of the flash storage device 3 requests the second write data from the shared device-side write buffer 601. In this embodiment, when new write data is received from the host 2 without any empty space in the shared device-side write buffer 601, a certain write data in the shared device-side write buffer 601 whose fuzzy write operation has ended is discarded. This ensures an empty space in the shared device-side write buffer 601 for storing the new write data. The new write data is stored in this ensured empty space. In other words, a certain write data in the shared device-side write buffer 601 whose fuzzy write operation has ended is cleared from the shared device-side write buffer 601, and the contents of the area in the shared device-side write buffer 601 storing that write data are replaced by the new write data.

[0098] Therefore, when there are a relatively large number of clients using host 2, there is a possibility that the write data used for the fuzzy write operation on a word line may have been discarded at the point in time when a fine write operation is performed on a certain word line.

[0099] If the second write data is not present in the shared device-side write buffer 601 (failure), the controller of the flash storage device 3 obtains the second write data from the host-side write buffer #m-1 of the host 2 as write data (fine data) for fine write operations by sending a request (transfer request) to the host 2 to retrieve the second write data. In this case, the second write data is transferred from the host-side write buffer #m-1 of the host 2 to the flash storage device 3. This transferred second write data may also be stored in the shared device-side write buffer 601.

[0100] Furthermore, the controller of the flash storage device 3 writes the second write data (fine data) to the aforementioned plurality of second storage cells within the write target block #m-1 through a fine write operation.

[0101] Here, we envision a scenario where each host-side write buffer 405 required to support fuzzy-fine write operations has a capacity of 12 MB, resulting in a total capacity of 12 GB for the 1000 host-side write buffers 405 corresponding to 1000 clients (tenants). With the shared device-side write buffer 601's capacity (total capacity) set to 3 GB, if the number of clients (tenants) is less than 250, multi-stage write operations like fuzzy-fine write operations can be performed without repeatedly transferring the same write data from host 2 to flash storage device 3.

[0102] As the number of clients utilizing host 2 increases, the frequency of the shared device-side write buffer 601 gradually decreases. However, in the event that the write data required for a fine-grained write operation is not present in the shared device-side write buffer 601 (failure), the flash storage device 3 retrieves the write data from host 2 by sending a transfer request to host 2. Therefore, even if the number of clients (tenants) increases to 1000, multi-stage write operations such as fuzzy-fine-grained write operations can still be performed normally.

[0103] Moreover, even if there are 1000 clients (tenants), if, for example, 8 flash storage devices 3 are connected to host (server) 2, and host 2 distributes the same number of clients equally to these 8 flash storage devices 3, then each flash storage device 3 only needs to have a 1.5G byte shared device-side write buffer 601, so that the same write data does not need to be transferred from host 2 to flash storage device 3 multiple times, and multi-stage write operations such as fuzzy-fine write operations can be performed.

[0104] Figure 4 This indicates a fuzzy-fine write operation with three word lines traveling back and forth.

[0105] Here, we assume that the NAND flash memory within flash storage device 3 is a QLC-flash memory that stores 4 bits of data in each storage cell. A fuzzy-fine write operation to a specific write target block within a NAND flash memory is performed as follows.

[0106] (1) First, four pages (P0 to P3) of write data are transferred to the NAND flash memory. For multiple memory cells connected to word lines WL0 in the write target block, a fuzzy write operation is performed to write the four pages (P0 to P3) of write data.

[0107] (2) Next, the next 4 pages (P4 to P7) of write data are transferred to the NAND flash memory, and a fuzzy write operation is performed on the multiple memory cells connected to the word line WL1 in the write target block to write the 4 pages (P4 to P7) of write data.

[0108] (3) Next, the next 4 pages (P8 to P11) of write data are transferred to the NAND flash memory, and a fuzzy write operation is performed on the multiple memory cells connected to the word line WL2 in the write target block to write the 4 pages (P8 to P11) of write data.

[0109] (4) When the fuzzy write operation for the multiple memory cells connected to word line WL2 has ended, the word line of the write target returns to word line WL0, thereby enabling the fine write operation for the multiple memory cells connected to word line WL0. Furthermore, four pages (P0 to P3) of write data, identical to the four pages (P0 to P3) of write data used in the fuzzy write operation on word line WL0, are again transferred to the NAND flash memory. A fine write operation is then performed on the multiple memory cells connected to word line WL0 within the write target block to write these four pages (P0 to P3) of write data. Thus, the fuzzy-fine write operation for pages P0 to P3 ends. As a result, data can be read from pages P0 to P3.

[0110] (5) Next, the next 4 pages (P12 to P15) of write data are transferred to the NAND flash memory, and a fuzzy write operation is performed on the multiple memory cells connected to the word line WL3 in the write target block to write the 4 pages (P12 to P15) of write data.

[0111] (6) When the fuzzy write operation for the multiple memory cells connected to word line WL3 has ended, the word line of the write target returns to word line WL1, thereby enabling the fine write operation for the multiple memory cells connected to word line WL1. Furthermore, four pages (P4 to P7) of write data, identical to the four pages (P4 to P7) of write data used in the fuzzy write operation on word line WL1, are transferred again to the NAND flash memory. A fine write operation is then performed on the multiple memory cells connected to word line WL1 within the write target block to write these four pages (P4 to P7) of write data. Thus, the fuzzy-fine write operation for pages P4 to P7 ends. As a result, data can be read from pages P4 to P7.

[0112] Figure 5 This indicates a fuzzy-fine write operation that involves two word lines traveling back and forth.

[0113] Here, we assume that the NAND flash memory within flash storage device 3 is a QLC-flash memory that stores 4 bits of data in each storage cell. A fuzzy-fine write operation to a specific write target block within a NAND flash memory is performed as follows.

[0114] (1) First, four pages (P0 to P3) of write data are transferred to the NAND flash memory. For multiple memory cells connected to word lines WL0 in the write target block, a fuzzy write operation is performed to write the four pages (P0 to P3) of write data.

[0115] (2) Next, the next 4 pages (P4 to P7) of write data are transferred to the NAND flash memory, and a fuzzy write operation is performed on the multiple memory cells connected to the word line WL1 in the write target block to write the 4 pages (P4 to P7) of write data.

[0116] (3) When the fuzzy write operation for multiple memory cells connected to word line WL1 has ended, the word line of the write target returns to word line WL0, thereby enabling the fine write operation for multiple memory cells connected to word line WL0. Furthermore, four pages (P0 to P3) of write data, identical to the four pages (P0 to P3) of write data used in the fuzzy write operation on word line WL0, are again transferred to the NAND flash memory. A fine write operation is then performed on the multiple memory cells connected to word line WL0 within the write target block to write these four pages (P0 to P3) of write data. Thus, the fuzzy-fine write operation for pages P0 to P3 ends. As a result, data can be read from pages P0 to P3.

[0117] (4) Next, the next 4 pages (P8 to P11) of write data are transferred to the NAND flash memory, and a fuzzy write operation is performed on the multiple memory cells connected to the word line WL2 in the write target block to write the 4 pages (P8 to P11) of write data.

[0118] (5) When the fuzzy write operation for multiple memory cells connected to word line WL2 has ended, the word line of the write target returns to word line WL1, thereby enabling the fine write operation for multiple memory cells connected to word line WL1. Furthermore, four pages (P4 to P7) of write data, identical to the four pages (P4 to P7) of write data used in the fuzzy write operation on word line WL1, are again transferred to the NAND flash memory. A fine write operation is then performed on the multiple memory cells connected to word line WL1 within the write target block to write these four pages (P4 to P7) of write data. Thus, the fuzzy-fine write operation for pages P4 to P7 ends. As a result, data can be read from pages P4 to P7.

[0119] (6) Next, the next 4 pages (P12 to P15) of write data are transferred to the NAND flash memory, and a fuzzy write operation is performed on the multiple memory cells connected to the word line WL3 in the write target block to write the 4 pages (P12 to P15) of write data.

[0120] Figure 6 This section describes a configuration example of a computer system 1 that includes the flash storage device 3 and the host 2 of this embodiment.

[0121] The flash memory device 3 includes a controller 4 and non-volatile memory (NAND flash memory) 5. The flash memory device 3 may also include random access memory, such as DRAM 6.

[0122] The NAND flash memory 5 comprises a memory cell array having multiple memory cells arranged in a matrix. The NAND flash memory 5 can be either a 2D NAND flash memory or a 3D NAND flash memory.

[0123] The NAND flash memory 5 has a cell array containing multiple blocks BLK0 to BLKm-1. Each block BLK0 to BLKm-1 is arranged with a large number of pages. Erasing operations are performed on a block-by-block basis, while data writing and reading operations are performed on a page-by-page basis.

[0124] The NAND flash memory 5 can be implemented using a multi-value flash memory capable of storing multiple bits in a single cell, such as a quad-level cell (QLC) flash memory capable of storing 4 bits in a single cell. In this case, the NAND flash memory 5 can selectively use a write mode (QLC mode) for writing 4 bits to a single cell and a write mode (Single Level Cell (SLC) mode) for writing 1 bit to a single cell in each block. A block using QLC mode can be referred to as a QLC block, and a block using SLC mode can be referred to as an SLC block. The multi-stage write operation described above is used for writing data to QLC blocks. Furthermore, the NAND flash memory 5 is not limited to being implemented using QLC flash memory; it can also be implemented using a multi-level cell (MLC) flash memory capable of storing 2 bits in a single cell, or a triple-level cell (TLC) flash memory capable of storing 3 bits in a single cell. The multi-stage write operation described above can be used to write data to blocks that are applicable to MLC mode (MLC blocks) or blocks that are applicable to TLC mode (TLC blocks).

[0125] The controller 4 is electrically connected to the NAND flash memory 5, which is a non-volatile memory, via a NAND interface 13 such as a ToggleNAND interface or an Open NAND Flash Component Interface (ONFI). The controller 4 operates as a memory controller configured to control the NAND flash memory 5. The controller 4 can also be implemented by a circuit such as a System-on-a-chip (SoC).

[0126] NAND flash memory 5 can also be like Figure 7The diagram shows multiple NAND flash memory chips (NAND flash memory dies). Each NAND flash memory chip can operate independently. Therefore, the NAND flash memory chips function as units that can operate in parallel. Figure 7 The example illustrates a scenario where NAND interface 13 has 16 channels Ch.1 to Ch.16, each connected to two NAND flash memory chips. In this case, the 16 NAND flash memory chips #1 to #16 connected to channels Ch.1 to Ch.16 can be arranged as memory bank #0, and the remaining 16 NAND flash memory chips #17 to #32 connected to channels Ch.1 to Ch.16 can also be arranged as memory bank #1. The memory bank functions as a unit that allows multiple memory modules to operate in parallel through memory bank interleaving. Figure 7 In this configuration example, up to 32 NAND flash memory chips can be operated in parallel through 16 channels and memory bank interleaving using 2 memory banks.

[0127] Erasing operations can be performed on a single block (physical block) or on a superblock, which comprises a set of multiple blocks that can be executed in parallel. A superblock is not limited to this, but can contain a total of 32 blocks selected sequentially from NAND flash memory chips #1 to #32. Furthermore, each of the NAND flash memory chips #1 to #32 can have a multi-plane configuration. For example, if each of the NAND flash memory chips #1 to #32 has a multi-plane configuration containing two planes, a superblock can also contain a total of 64 blocks selected sequentially from the 64 planes corresponding to the NAND flash memory chips #1 to #32.

[0128] exist Figure 8 The example illustrates a superblock (SB) containing 32 blocks (this is block BLK2 in NAND flash memory chip #1, block BLK3 in NAND flash memory chip #2, block BLK7 in NAND flash memory chip #3, block BLK4 in NAND flash memory chip #4, block BLK6 in NAND flash memory chip #5, ... block BLK3 in NAND flash memory chip #32).

[0129] As Figure 2 , Figure 3 The write object blocks 602 described herein can also be a set of multiple blocks that can be accessed in parallel, i.e., a superblock.

[0130] Next, regarding Figure 6 The configuration of controller 4 will be explained.

[0131] The controller 4 includes a host interface 11, a CPU 12, a NAND interface 13, and a DRAM interface 14. These host interfaces 11, CPU 12, NAND interface 13, and DRAM interface 14 are interconnected via a bus 10.

[0132] The host interface 11 is a host interface circuit configured to perform communication with the host 2. The host interface 11 can also be, for example, a PCIe controller (NVMe controller). Alternatively, in a configuration where the flash storage device 3 is connected to the host 2 via Ethernet (trademark), the host interface 11 can also be an NVMe over Fabrics (NVMeOF) controller. The configuration where the flash storage device 3 is connected to the host 2 via Ethernet (trademark) allows for easy expansion of the number of flash storage devices 3 as needed. Furthermore, it also allows for easy expansion of the number of hosts 2.

[0133] Host interface 11 receives various requests (instructions) from host 2. These requests (instructions) include write requests (write instructions), read requests (read instructions), and various other requests (instructions).

[0134] CPU 12 is a processor configured with a host interface 11, a NAND interface 13, and a DRAM interface 14. In response to power-on of flash memory 3, CPU 12 loads a control program (firmware) from NAND flash memory 5 or a ROM (not shown) into DRAM 6, and performs various processes by executing the firmware. Alternatively, the firmware can also be loaded into SRAM (not shown) within controller 4. CPU 12 can perform instruction processing, etc., to process various instructions from host 2. The operation of CPU 12 is controlled by the aforementioned firmware executed by CPU 12. Furthermore, some or all of the instruction processing can also be performed by dedicated hardware within controller 4.

[0135] CPU 12 can function as a write control unit 21 and a read control unit 22. These write control units 21 and read control units 22 can also be implemented using dedicated hardware within the controller 4.

[0136] The write control unit 21 is configured to write data to multiple memory cells in the NAND flash memory 5 connected to the same word lines within a specific write target block through a multi-stage write operation, which includes at least a first-stage write operation that transfers multiple pages of data to the NAND flash memory 5 and a second-stage write operation that transfers the data to the NAND flash memory 5. A typical example of this multi-stage write operation, as described above, includes a fuzzy-fine write operation.

[0137] The write control unit 21 manages multiple write target blocks allocated from multiple blocks contained in the NAND flash memory 5. The write control unit 21 receives a write request from the host 2 containing the block address of any one of the multiple write target blocks. Whenever a write request is received from the host 2, the write control unit 21 receives write data stored in any one of the multiple host-side write buffers 405 on the host 2's memory 33, and stores the received write data in the shared device-side write buffer 601 on the DRAM 6.

[0138] The write control unit 21 obtains multi-page write data (first write data) associated with one or more write requests for a specified write object block (first write object block) from the shared device side write buffer 601. Furthermore, the write control unit 21 writes the first write data to multiple memory cells (multiple first memory cells) connected to a word line of the write object within the first write object block through a first-stage write operation such as a fuzzy write operation.

[0139] When the first write data exists in the shared device-side write buffer 601 at the time when the second stage write operation of the first write data should be performed, the write control unit 21 obtains the first write data from the shared device-side write buffer 601 and writes the first write data to the aforementioned plurality of first storage cells of the first write object block through the second stage write operation.

[0140] When the first write data is not present in the shared device-side write buffer 601 at the time when the second stage write operation of the first write data should be performed, the write control unit 21 obtains the first write data from the host 2 by sending a request (transmission request) to the host 2 to obtain the first write data, and writes the first write data to the aforementioned plurality of first storage cells of the first write object block through the second stage write operation.

[0141] Furthermore, the write control unit 21 ensures an empty area in the shared device-side write buffer 601 by discarding a certain write data in the shared device-side write buffer 601.

[0142] For example, when the fine write operation of a certain write data has ended, that is, when the multi-stage write operation of the write data has ended, the write control unit 21 discards the write data from the shared device side write buffer 601.

[0143] Furthermore, when there is no empty space in the shared device side write buffer 601, and new write data is received from the host 2, the write control unit 21 selects the write data to be discarded from the write data in the shared device side write buffer 601 where the fuzzy write operation has ended, and discards the selected write data from the shared device side write buffer 601.

[0144] An algorithm for selecting which write data should be discarded can be used to select the earliest write data in the shared device-side write buffer 601 after the first-stage write operation, such as a fuzzy write operation, has ended, selecting the first-in-first-out write data.

[0145] Compared to the speed of data write operations for write object blocks corresponding to clients (tenants) with low data write volumes, the speed of data write operations for write object blocks corresponding to clients (tenants) with high data write volumes tends to be faster. Therefore, the frequency of sending transmission requests to host 2 can be reduced by using methods such as discarding the earliest write data in the shared device-side write buffer 601 after the first-stage write operation, such as a fuzzy write operation, has been completed.

[0146] Furthermore, the algorithm used to select which write data should be discarded from the write data in the shared device-side write buffer 601 after the first-stage write operation, such as a fuzzy write operation, has ended is not limited to first-in-first-out, and other algorithms such as LRU or random can also be used.

[0147] The read control unit 22 receives a read request (read instruction) with a specified block address and page address from the host 2, reads the data stored in the physical storage location of the NAND flash memory 5 specified by the block address and page address, and sends the read data back to the host 2.

[0148] NAND interface 13 is a memory control circuit configured to control NAND flash memory 5 under the control of CPU 12.

[0149] DRAM interface 14 is a DRAM control circuit configured to control DRAM 6 under the control of CPU 12. A portion of the storage area of ​​DRAM 6 is used as a shared device-side write buffer 601.

[0150] Host 2 is a computer like a server, and as its hardware components, it includes a host storage controller 31, a processor 32, and a memory 33. The host storage controller 31, processor 32, and memory 33 can also be interconnected via a bus 30. The memory 33 is a random access memory like DRAM, and a portion of the storage area of ​​the memory 33 is used as multiple host-side write buffers 405.

[0151] Host 2 also has a network interface 34.

[0152] Figure 9 This represents an example of data write processing performed by host 2 and flash storage device 3.

[0153] Here, we envision a scenario where, at the point in time when a fine-grained write operation should be performed on a word line of a certain write object block, the write data to be written to multiple memory cells connected to that word line exists in the shared device-side write buffer 601.

[0154] (1) If a user application corresponding to a client (tenant) requests data to be written, the data (write data) is stored in the host-side write buffer 405 corresponding to that client (tenant). Furthermore, a write request is sent from host 2 to flash storage device 3. The write request may also include, for example, a block address and a tag. As mentioned above, the tag is information used to identify the write data (e.g., LBA, key of key-value storage, etc.). The write request may also include storage location information indicating the location within the host-side write buffer 405 where the write data is stored. The write request may be sent from host 2 to flash storage device 3 when conditions are met, such as a specific amount of write data being stored in the host-side write buffer 405. In the controller 4 of flash storage device 3, when the write control unit 21 receives a write request from host 2, the write control unit 21 receives the write data stored in the host-side write buffer 405 from host 2. This write data may be sent from host 2 to flash storage device 3 along with the write request, or the write control unit 21 may obtain the write data from host 2 by sending a request (transmission request) to host 2 to obtain the write data. The transmission request contains the aforementioned storage location information indicating the location within the host-side write buffer 405.

[0155] (2)(3) In the controller 4 of the flash memory device 3, the write control unit 21 determines the page address of the page of the write object within the write object block specified by the block address in the received write request. The write control unit 21 obtains the first write data of the multi-page portion to be written to the write object block from the shared device-side write buffer 601. The write control unit 21 writes the first write data of the multi-page portion to the write object block via the NAND interface 13 through a fuzzy write operation. In this case, the write control unit 21 transmits the first write data (fuzzy data) of the multi-page portion to the NAND flash memory chip containing the write object block, and writes the first write data to a plurality of first memory cells connected to the word lines of the write object within the write object block (fuzzy write operation).

[0156] (4) After the completion of the fuzzy write operation for one or more subsequent word lines, reaching the time point when the fine write of the first write data should be performed, the write control unit 21 requests multiple pages of the first write data from the shared device-side write buffer 601. If the multiple pages of the first write data exist in the shared device-side write buffer 601, the write control unit 21 obtains the multiple pages of the first write data from the shared device-side write buffer 601. The write control unit 21 writes the multiple pages of the first write data to the write target block via the NAND interface 13 through a fine write operation. In this case, the write control unit 21 transmits the multiple pages of the first write data (fine data) to the NAND flash memory chip containing the write target block, and writes the first write data to the aforementioned multiple first memory cells within the write target block (fine write operation).

[0157] (5) The write control unit 21 discards the first write data (write data in fine state) of the multi-page copy of the fine write operation that has ended from the shared device side write buffer 601, thereby increasing the empty area of ​​the shared device side write buffer 601.

[0158] Figure 10 This represents another example of data write processing performed via host 2 and flash storage device 3.

[0159] Here, we envision a scenario where, at the point in time when a fine-grained write operation should be performed on a word line of a certain write object block, the write data to be written to multiple memory cells connected to that word line does not exist in the shared device-side write buffer 601.

[0160] (1') If a user application corresponding to a certain client (tenant) requests data to be written, the data (write data) is stored in the host-side write buffer 405 corresponding to that client (tenant). Furthermore, a write request is sent from host 2 to flash storage device 3. The write request may also include, for example, a block address and a tag. As described above, the tag is information used to identify the write data (e.g., LBA, key of key-value storage, etc.). The write request may also include storage location information indicating the location within the host-side write buffer 405 where the write data is stored. The write request may be sent from host 2 to flash storage device 3 when conditions are met, such as a specific amount of write data being stored in the host-side write buffer 405. In the controller 4 of flash storage device 3, when the write control unit 21 receives a write request from host 2, the write control unit 21 receives the write data stored in the host-side write buffer 405 from host 2. This write data may be sent from host 2 to flash storage device 3 along with the write request, or the write control unit 21 may obtain the write data from host 2 by sending a request (transmission request) to host 2 to obtain the write data. The transmission request contains the aforementioned storage location information indicating the location within the host-side write buffer 405.

[0161] (2')(3') In the controller 4 of the flash memory device 3, the write control unit 21 determines the page address of the page of the write object within the write object block specified by the block address in the received write request. The write control unit 21 obtains the first write data of the multi-page portion to be written to the write object block from the shared device-side write buffer 601. The write control unit 21 writes the first write data of the multi-page portion to the write object block via the NAND interface 13 through a fuzzy write operation. In this case, the write control unit 21 transmits the first write data of the multi-page portion (fuzzy data) to the NAND flash memory chip containing the write object block, and writes the first write data to a plurality of first memory cells connected to the word lines of the write object within the write object block (fuzzy write operation).

[0162] (4') After the fuzzy write operation for one or more subsequent word lines is completed and the fine write of the first write data is performed, the write control unit 21 requests multiple pages of the first write data from the shared device-side write buffer 601. If the multiple pages of the first write data do not exist in the shared device-side write buffer 601, the write control unit 21 obtains the first write data from the host 2 by sending a request (transfer request) to the host 2 to obtain the first write data. The transfer request includes the aforementioned storage location information indicating the location within the host-side write buffer 405. The obtained first write data may also be stored in the shared device-side write buffer 601.

[0163] (5') The write control unit 21 writes the first write data of the multi-page portion to the write target block via the NAND interface 13 through a fine write operation. In this case, the write control unit 21 transmits the first write data (fine data) of the multi-page portion to the NAND flash memory chip containing the write target block, and writes the first write data to the aforementioned plurality of first memory cells in the write target block (fine write operation).

[0164] (6') The write control unit 21 discards the first write data (write data in fine state) of the multi-page copy of the fine write operation that has ended from the shared device side write buffer 601, thereby increasing the empty area of ​​the shared device side write buffer 601.

[0165] Figure 11 This indicates the fuzzy-fine write operation performed by the controller 4 of the flash storage device 3 using the shared device-side write buffer 601, and the process of discarding the data in the shared device-side write buffer 601.

[0166] Figure 11 For simplicity, the illustration shows a shared device-side write buffer 601 comprising regions 101 to 109. Each of these regions 101 to 109 has, for example, a size of 4 pages. Furthermore, in... Figure 11 Imagine a scenario where a fuzzy-fine write operation is performed while moving back and forth between three word lines (WL).

[0167] If the write data D1 and D2 stored in regions 101 and 102 of the write buffer 601 on the shared device side are write data to be written to the write target block BLK11, the controller 4 (1) writes 4 pages of write data D1 to pages P0-P3 (multiple storage units connected to word line WL0) of the write target block BLK11 through a fuzzy write operation, and (2) writes 4 pages of write data D2 to pages P4-P7 (multiple storage units connected to word line WL1) of the write target block BLK11 through a fuzzy write operation.

[0168] If the write data D11, D12, and D13 stored in regions 103, 104, and 105 of the write buffer 601 on the shared device side are write data to be written to the write target block BLK101, the controller 4 (3) writes 4 pages of write data D11 to pages P0-P3 (multiple storage units connected to word line WL0) of the write target block BLK101 through a fuzzy write operation, (4) writes 4 pages of write data D12 to pages P4-P7 (multiple storage units connected to word line WL1) of the write target block BLK101 through a fuzzy write operation, and (5) writes 4 pages of write data D13 to pages P8-P11 (multiple storage units connected to word line WL2) of the write target block BLK101 through a fuzzy write operation.

[0169] After the fuzzy write operation of 4 pages of write data D13 on word line WL2 is completed, controller 4 (6) writes 4 pages of write data D11 to pages P0-P3 (multiple memory cells connected to word line WL0) of write target block BLK101 through a fine write operation. After the fine write operation of write data D11 is completed, the state of write data D11 changes from fuzzy state to fine state. Moreover, controller 4 (7) discards the write data D11 (the write data in the fine state) after the fine write operation is completed from the shared device-side write buffer 601, making region 103 an empty region.

[0170] Figure 12 This indicates that the write data in the shared device-side write buffer 601 has been discarded when a fine write operation performed by receiving new write data from the host 2 has ended, provided that there is no empty space in the shared device-side write buffer 601.

[0171] Figure 12 The upper part indicates that the entire shared device side write buffer 601 is filled with fuzzy state write data (D21~D23, D31~D33, D41~D43) in which the fuzzy write operation has ended, and there is no empty area in the shared device side write buffer 601.

[0172] In this state, when new write data is received from host 2, the controller 4 of flash storage device 3, as follows: Figure 12 As shown in the middle, from the write data that has ended in the fuzzy write operation (write data in the fuzzy state), the earliest write data (write data D11 in this case) is selected as the write data to be discarded, and the earliest write data (write data D11 in this case) is discarded from the write buffer 601 on the shared device side.

[0173] Moreover, the controller 4 of the flash storage device 3, such as Figure 12As shown in the lower part, the newly received write data (here, write data D51) is stored in region 101, which becomes an empty area by discarding the write data D11.

[0174] Figure 13 The flowchart illustrates the sequence of data write processes performed by the controller 4 of the flash storage device 3.

[0175] In step S101, controller 4 receives a write request from host 2 containing the block address of any one of a plurality of write object blocks 602. Whenever controller 4 receives a write request from host 2, controller 4 receives write data stored in any one of a plurality of host-side write buffers 405 from host 2, and stores the received write data in the shared device-side write buffer 601.

[0176] In step S102, the controller 4 obtains the first write data of a multi-page copy associated with one or more write requests for a specified write object block from the shared device side write buffer 601.

[0177] In steps S103 and S104, the controller 4 transmits the acquired first write data of multiple pages to the NAND flash memory 5, and writes the first write data to multiple first memory cells connected to a word line of the write object in the write object block through a fuzzy write operation.

[0178] In step S105, at the time when the fine write operation of the first write data should begin, the controller 4 determines whether the first write data exists in the write buffer 601 on the shared device side.

[0179] If the first write data exists in the shared device-side write buffer 601 (YES in step S106), then in step S107, the controller 4 retrieves the first write data from the shared device-side write buffer 601. In steps S108 and S109, the controller 4 transfers the retrieved multi-page copies of the first write data to the NAND flash memory 5, and writes the first write data to the aforementioned plurality of first memory cells within the write target block through a fine write operation. Furthermore, in step S110, the controller 4 discards the first write data from the shared device-side write buffer 601.

[0180] If the first write data does not exist in the shared device-side write buffer 601 (NO in step S106), then in step S111, the controller 4 sends a transfer request to the host 2, and stores the first write data transferred from the host-side write buffer 405 to the flash storage device 3 in the shared device-side write buffer 601. Furthermore, the controller 4 executes the processing steps S107 to S110.

[0181] Figure 14 This diagram illustrates the data write operation where host 2 specifies the write target block and flash storage device 3 determines the write target page, and the data read operation where host 2 specifies the block address and page address.

[0182] Host 2's I / O service 404 sends block allocation requests and write requests to flash storage device 3.

[0183] The controller 4 of the flash storage device 3 includes a block allocation unit 701 and a page allocation unit 702. The block allocation unit 701 and the page allocation unit 702 may also be included in... Figure 6 The write control unit 21 is described in the text.

[0184] Data write operations are performed in the following order.

[0185] (1) When the I / O service 404 of host 2 needs to write data (write data) to flash storage device 3, the I / O service 404 may also request flash storage device 3 to allocate a free block that can be used as a write target block. When the block allocation unit 701 receives the request (block allocation request), the block allocation unit 701 allocates a free block from the free block group as a write target block to host 2, and notifies host 2 of the block address (BLK#) of the allocated write target block.

[0186] (2) Host 2's I / O service 404 sends a write request to flash storage device 3, which includes the block address of the allocated write object block, a tag identifying the write data, and the data length of the write data. Furthermore, I / O service 404 transmits the write data to flash storage device 3.

[0187] (3) When the page allocation unit 702 receives a write request, the page allocation unit 702 determines the page address of the write object page in the block (write object block) that represents the block address specified by the write request. The controller 4 obtains write data from the write buffer 601 on the shared device side and writes the write data to the determined write object page in the write object block.

[0188] (4) Controller 4 notifies host 2 of the page address representing the page to be written as a response to the write request. Alternatively, controller 4 may also notify host 2 of the tag contained in the write request, the block address contained in the write request, and the determined page address as a response to the write request. In host 2, LUT404A is updated in a manner that maps the physical address (block address, page address) representing the physical storage location where the write data is written to the tag of the write data.

[0189] Data read operations are performed in the following order.

[0190] (1) When host 2 needs to read data from flash storage device 3, host 2 obtains the physical address (block address, page address) corresponding to the tag of the data to be read from LUT404A by referring to LUT404A.

[0191] (2) Host 2 sends a read request specifying the block address and page address to flash storage device 3. When controller 4 of flash storage device 3 receives the read request from host 2, controller 4 reads data from the physical storage location of the read object within the block of the read object based on the block address and page address.

[0192] Figure 15 This indicates the block allocation instruction applicable in flash storage device 3.

[0193] A block allocation instruction is an instruction (block allocation request) that requests the allocation of a write object block (free block) to flash storage device 3. Host 2 requests the allocation of a write object block from flash storage device 3 by sending the block allocation instruction to flash storage device 3, thereby obtaining the block address (the block address of the allocated write object block).

[0194] Figure 16 This indicates a response to a block allocation instruction.

[0195] When a block allocation instruction is received from host 2, flash storage device 3 selects a free block from the list of free blocks to be allocated to host 2, allocates the selected free block as a write target block, and returns a response containing the block address of the write target block to host 2.

[0196] Figure 17 This indicates the write command applicable to flash storage device 3.

[0197] A write command is an instruction that requests data to be written to the flash storage device 3. This write command may also include an instruction ID, block address, tag, length, etc.

[0198] The instruction ID is the ID (instruction code) that indicates that the instruction is a write instruction, and the write instruction contains the instruction ID used for writing.

[0199] The block address is the physical address of the write object block that specifies the data to be written to.

[0200] A label is an identifier used to identify the data to be written. As mentioned above, this label can be a logical address like an LBA or a key in a key-value store. When the label is a logical address like an LBA, the logical address (starting LBA) contained in the write instruction represents the logical location (initial logical location) within the logical address space where the data to be written should be located.

[0201] The length indicates the length of the data to be written.

[0202] The write command may further include storage location information indicating the location within the host-side write buffer 405 where the write data is stored.

[0203] When a write command is received from host 2, controller 4 determines the write object location (write object page) within the write object block that has the block address specified by the write command. This write object page is determined after considering constraints such as page write order and bad pages. Then, controller 4 writes the data from host 2 to the write object location (write object page) within the write object block.

[0204] Figure 18 Indicates to Figure 17 The response to the write command.

[0205] The response includes the page address and length. The page address is the physical address representing the physical storage location within the write object block where the data was written. This physical address can be represented by an offset within the block (i.e., the combination of the page address and the offset within the page). The length indicates the length of the data written.

[0206] Alternatively, the response may include not only the page address (offset within a block) and length, but also tags and block addresses. Tags are... Figure 17 The tag contained in the write instruction. The block address is Figure 17 The block address contained in the write instruction.

[0207] Figure 19 This indicates the read command applicable to flash storage device 3.

[0208] A read command is an instruction that requests data to be read from flash storage device 3. This read command includes an instruction ID, tag, block address, page address, and length.

[0209] The instruction ID is the ID (instruction code) that indicates that the instruction is a read instruction, and the read instruction contains the instruction ID used for the read instruction.

[0210] The block address specifies the block containing the data to be read. The page address specifies the page containing the data to be read. This page address can also be represented by an offset within the block (i.e., a combination of the page address and the offset within the page) indicating the physical storage location within the block containing the data to be read. The length indicates the length of the data to be read.

[0211] Figure 20 This represents an example of a shared device-side write buffer 601 implemented by more than one SLC-block.

[0212] So far, the description has envisioned a scenario where the shared device-side write buffer 601 is implemented using a random access memory such as DRAM6. However, since the data write speed of an SLC block that writes 1 bit of data per memory cell is faster than the data write speed of a block that writes multiple bits of data per memory cell, the shared device-side write buffer 601, as... Figure 20 The above can also be implemented by one or more SLC blocks that write 1 bit of data to each memory cell. In this case, each write target block can also be implemented by a block that writes multiple bits of data to each memory cell, such as a QLC block that writes 4 bits of data to each memory cell.

[0213] As explained above, according to this embodiment, whenever a write request is received from the host 2, the write data stored in any of the plurality of host-side write buffers 405 corresponding to the plurality of write object blocks 602 is stored in a shared device-side write buffer 601 shared among the plurality of write object blocks 602. Furthermore, the first write data, consisting of multiple pages of the first write object block to be written to within the plurality of write object blocks 602, is obtained from the shared device-side write buffer 601, and the first write data is written to the first write object block through a first-stage write operation such as a fuzzy write operation. When write data is received from the host 2 when there is no empty space in the shared device-side write buffer 601, the write data in the shared device-side write buffer 601 after the first-stage write operation has been completed is discarded, ensuring an empty space in the shared device-side write buffer 601. When the first write data is not present in the shared device-side write buffer 601 at the time when a second-stage write operation, such as a fine write operation of the first write data, should be performed, the first write data is obtained from the host 2 by sending a request to the host 2 to obtain the first write data.

[0214] Therefore, if the first write data is not present in the write buffer 601 on the shared device side, the same write data can be transferred from the host 2 to the flash storage device 3 again. Thus, it can flexibly cope with the increase in the number of write objects, i.e., the increase in the number of clients sharing the flash storage device 3, and can reduce the data flow between the host 2 and the flash storage device 3.

[0215] Furthermore, in this embodiment, a multi-stage write operation such as a fuzzy-fine write operation is described using a shared device-side write buffer 601. However, the configuration and write processing order of this embodiment are not limited to multi-stage write operations, and can also be applied to a full-sequence write operation that transfers multiple pages of data to the NAND flash memory 5 in one go.

[0216] Furthermore, in this embodiment, a write request (write instruction) containing the block address of the specified write target block is described as the write instruction applicable to the flash storage device 3, but a write request (write instruction) containing both the block address and the page address may also be used.

[0217] Furthermore, in this embodiment, NAND flash memory is exemplified as a non-volatile memory. However, the functionality of this embodiment can also be applied to various other non-volatile memories such as MRAM (Magnetoresistive Random Access Memory), PRAM (Phase Change Random Access Memory), ReRAM (Resistive Random Access Memory), or FeRAM (Ferroelectric Random Access Memory).

[0218] Several embodiments of the present invention have been described, but these embodiments are provided as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other ways, and omissions, substitutions, and modifications may be made without departing from the spirit of the invention. These embodiments or variations thereof are included within the scope or spirit of the invention, and are included within the scope of the invention as set forth in the claims and its equivalents.

[0219] [Explanation of Symbols]

[0220] 2 host

[0221] 3. Flash storage devices

[0222] 4 Controller

[0223] 5 NAND flash memory

[0224] 21 Write to control unit

[0225] 405 Host-side write buffer

[0226] 601 Shared device side write buffer

Claims

1. A storage system connectable to a host, the host including a data buffer, the storage system comprising: A non-volatile memory comprising a plurality of word lines, wherein the plurality of word lines include at least a first word line and a second word line, and each of the plurality of word lines connects to a plurality of memory cells; Write to the buffer; and The controller is electrically connected to the non-volatile memory and is configured to perform a multi-stage write operation to write data to the non-volatile memory, wherein the multi-stage write operation includes at least a first-stage write operation and a second-stage write operation. The controller is further configured as follows: The first data is transferred from the data buffer of the host to the write buffer; The first data is transferred from the write buffer to the non-volatile memory; The first stage write operation of the first data is performed on the plurality of memory cells connected to the first word line; The second data is transferred from the data buffer of the host to the write buffer; The second data is transferred from the write buffer to the non-volatile memory; The first stage write operation of the second data is performed on the plurality of memory cells connected to the second word line; and In the second stage of the write operation for the first data, It is determined that the first data is not stored in the write buffer; In response to determining that the first data is not stored in the write buffer, the first data is transferred again from the data buffer of the host to the write buffer; The first data is then transferred again from the write buffer to the non-volatile memory; and The second-stage write operation of the first data is performed on the plurality of storage cells connected to the first word line.

2. The storage system according to claim 1, wherein The controller is further configured as follows: In the second stage of the write operation for the first data, It is determined that the first data is stored in the write buffer; In response to determining that the first data is stored in the write buffer, the first data is not transferred from the host's data buffer to the write buffer again, and the first data is transferred from the write buffer to the non-volatile memory again; and The second-stage write operation of the first data is performed on the plurality of storage cells connected to the first word line.

3. The storage system according to claim 1, wherein The controller is further configured as follows: It is determined that the write buffer storing the first data does not contain unused space for storing the second data; and In response to determining that the write buffer storing the first data does not contain unused space for storing the second data, the first data is discarded from the write buffer before the second data is transferred from the data buffer of the host to the write buffer.

4. The storage system according to claim 1, wherein The controller is further configured as follows: It is determined that the write buffer does not contain unused space for storing the third data; and In response to determining that the write buffer does not contain unused space for storing the third data, the earliest data in the write buffer from which the first-stage write operation has ended is discarded from the write buffer.

5. The storage system according to claim 1, wherein The controller is further configured as follows: It is determined that the write buffer does not contain unused space for storing the third data; and In response to determining that the write buffer does not contain unused space for storing the third data, the least recently used data in the write buffer from which the first-stage write operation has ended is discarded from the write buffer.

6. The storage system according to claim 1, wherein The controller is further configured as follows: After the second phase of the write operation of the first data has been completed, the first data is discarded from the write buffer.

7. The storage system according to claim 1, wherein The non-volatile memory further comprises a plurality of blocks, each of which is a unit of data erasure operation. The plurality of blocks include at least a first block and a second block, the first block containing the first word line, and the second block containing the second word line. The host's data buffer comprises multiple regions corresponding to the plurality of blocks, wherein the plurality of regions include at least region 1 and region 2, and The controller is further configured as follows: By sending a request to the host, the first data is transferred from the first area of ​​the data buffer to the write buffer, the first area corresponding to the first block; and The second data is transferred from the second area of ​​the data buffer to the write buffer, the second area corresponding to the second block, by sending another request to the host.

8. The storage system according to claim 1, wherein The first-stage write operation includes: an operation for programming data of a first size into the plurality of memory cells connected to one of the plurality of word lines to set the threshold voltage distribution of the plurality of memory cells connected to the one word line. The second-stage write operation includes: an operation for programming the data having the first size into the plurality of memory cells connected to the one word line to adjust the threshold voltage distribution of the plurality of memory cells connected to the one word line.

9. The storage system according to claim 1, wherein The write buffer is implemented using random access memory.

10. The storage system according to claim 1, wherein The non-volatile memory further comprises a plurality of blocks, each of which is a unit of data erasure operation, and the plurality of blocks at least comprises a first block and a second block, the first block comprising the first word line and the second word line. At least one storage unit in the first block is configured to store a first number of bits of data. At least one storage unit in the second block is configured to store a second number of bits of data, the second number being less than the first number, and The write buffer is implemented by the second block.

11. A method for controlling a non-volatile memory in a storage system, the storage system being connectable to a host including a data buffer, the storage system including a write buffer and the non-volatile memory, the non-volatile memory including a plurality of word lines, the plurality of word lines including at least a first word line and a second word line, each of the plurality of word lines connecting to a plurality of memory cells, the method comprising: The first data is transferred from the data buffer of the host to the write buffer; The first data is transferred from the write buffer to the non-volatile memory; Perform a first-stage write operation of the first data on the plurality of memory cells connected to the first word line; The first-stage write operation is one of the multi-stage write operations used to write the data to the non-volatile memory; The second data is transferred from the data buffer of the host to the write buffer; The second data is transferred from the write buffer to the non-volatile memory; The first stage write operation of the second data is performed on the plurality of memory cells connected to the second word line; and In the second-stage write operation of the first data, the second-stage write operation is another operation in the multi-stage write operation. It is determined that the first data is not stored in the write buffer; In response to determining that the first data is not stored in the write buffer, the first data is transferred again from the data buffer of the host to the write buffer; The first data is then transferred again from the write buffer to the non-volatile memory; and The second-stage write operation of the first data is performed on the plurality of storage cells connected to the first word line.

12. The method of claim 11, further comprising: In the second stage of the write operation for the first data, It is determined that the first data is stored in the write buffer; In response to determining that the first data is stored in the write buffer, the first data is not transferred from the data buffer of the host to the write buffer again, and the first data is transferred from the write buffer to the non-volatile memory again; and The second-stage write operation of the first data is performed on the plurality of storage cells connected to the first word line.

13. The method of claim 11, further comprising: It is determined that the write buffer storing the first data does not contain unused space for storing the second data; and In response to determining that the write buffer storing the first data does not contain unused space for storing the second data, the first data is discarded from the write buffer before the second data is transferred from the data buffer of the host to the write buffer.

14. The method of claim 11, further comprising: It is determined that the write buffer does not contain unused space for storing the third data; and In response to determining that the write buffer does not contain unused space for storing the third data, the earliest data in the write buffer from which the first-stage write operation has ended is discarded from the write buffer.

15. The method of claim 11, further comprising: It is determined that the write buffer does not contain unused space for storing the third data; and In response to determining that the write buffer does not contain unused space for storing the third data, the least recently used data in the write buffer from which the first-stage write operation has ended is discarded from the write buffer.

16. The method of claim 11, further comprising: After the second phase of the write operation of the first data has been completed, the first data is discarded from the write buffer.

17. The method of claim 11, wherein The non-volatile memory further comprises a plurality of blocks, each of which is a unit of data erasure operation. The plurality of blocks include at least a first block and a second block, the first block containing the first word line, and the second block containing the second word line. The host's data buffer comprises multiple regions corresponding to the plurality of blocks, wherein the plurality of regions include at least region 1 and region 2, and The method further includes: The first data is transferred from the first area of ​​the data buffer to the write buffer by sending a request to the host, the first area corresponding to the first block; and The second data is transferred from the second area of ​​the data buffer to the write buffer, the second area corresponding to the second block, by sending another request to the host.

18. The method of claim 11, wherein The first-stage write operation includes: an operation for programming data of a first size into the plurality of memory cells connected to one of the plurality of word lines to set the threshold voltage distribution of the plurality of memory cells connected to the one word line. The second-stage write operation includes: an operation for programming the data having the first size into the plurality of memory cells connected to the one word line to adjust the threshold voltage distribution of the plurality of memory cells connected to the one word line.

19. The method of claim 11, wherein The write buffer is implemented using random access memory.

20. The method of claim 11, wherein The non-volatile memory further comprises a plurality of blocks, each of which is a unit of data erasure operation, and the plurality of blocks at least comprises a first block and a second block, the first block comprising the first word line and the second word line. At least one storage unit in the first block is configured to store a first number of bits of data. At least one storage unit in the second block is configured to store a second number of bits of data, the second number being less than the first number, and The write buffer is implemented by the second block.

Images