Performance optimization for storing data in storage services configured to the data storage capacity of a data storage device

Abstract

Techniques for improving performance when storing data to memory addresses implemented in a data storage capacity of a storage subsystem. A connection from the storage subsystem to a host system supports both a cache-coherent memory access protocol to a storage device implemented in the data storage capacity and a data storage access protocol. The storage subsystem can use a cache to cache pages of the storage device for access over the connection. A data storage access queue can be configured to provide a command that is configured to store data at a memory address in the storage device.Such commands can be queued when the cache memory is temporarily unavailable, or to cause the storage subsystem to move a cached page from the cache memory to the data storage capacity.

Description

RELATED REGISTRATIONS

[0001] The present application claims priority from US patent application serial no. 18 / 439,623, filed on February 12, 2024, which claims priority from provisional US patent application serial no. 63 / 485,131, filed on February 15, 2023, the entirety of which disclosures are hereby incorporated by reference. TECHNICAL AREA

[0002] At least some embodiments disclosed herein relate to storage systems in general and, in particular, to storage systems configured to be accessible to storage services and data storage services. GENERAL STATE OF THE ART

[0003] A storage subsystem can include one or more storage devices that store data. These storage devices can be, for example, non-volatile or volatile. Generally, a host system can use a storage subsystem to store data in and retrieve data from the storage devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The embodiments are illustrated as examples and not as limitations in the figures of the accompanying drawings, where the same reference numerals indicate similar elements. Fig. Figure 1 illustrates an exemplary computing system that includes a memory subsystem according to some embodiments of the present disclosure. Fig. Figure 2 shows a storage subsystem configured according to one embodiment to provide both storage and data storage services to a host system via a physical connection. Fig. Figure 3 shows a storage subsystem configured according to one embodiment to provide storage services using a portion of the non-volatile storage capacity of the storage subsystem. Fig. Figure 4 illustrates the operations configured in a memory subsystem according to one embodiment to respond to a memory access request. Fig. Figure 5 illustrates a configuration to enable, according to one embodiment, that a portion of the non-volatile data storage capacity of the storage subsystem is addressable via both storage services and data storage services. Fig. Figure 6 shows a technique for using a write command to implement, according to one embodiment, a request to store data at a memory address implemented in non-volatile memory. Fig. Figure 7 shows a storage subsystem configured with data storage access queues to implement storage requests of the storage according to one embodiment. Fig. Figure 8 shows a storage subsystem that includes data storage access queues that are addressable by a host system via storage services to implement storage requests of the storage system according to one embodiment. Fig. Figure 9 shows a host system that has data storage access queues which a storage subsystem can access to implement storage requests of the storage according to one embodiment. Fig. Figure 10 shows a method for storing data via storage services implemented in a data storage capacity of a storage subsystem according to one embodiment. DETAILED DESCRIPTION

[0005] At least some aspects of this disclosure relate to techniques of a storage subsystem that provides storage and data storage services over a physical connection to a host system. The storage subsystem can be configured to use a portion of its fast memory as cache memory for a host system that accesses a portion of its storage capacity via a cache-coherent memory access protocol. Furthermore, the host system can use data storage access commands to optimize the performance of the host system, which stores data in the storage subsystem using the cache-coherent memory access protocol.

[0006] For example, a host system and a storage subsystem (such as a solid-state drive (SSD)) can be connected via a physical link according to a Compute Express Link (CXL) connection standard for computer components. Compute Express Link (CXL) includes protocols for data storage access (such as cxl.io) and protocols for cache-coherent memory access (such as cxl.mem and cxl.cache). Thus, a storage subsystem can be configured to provide both data storage and storage services to the host system over the physical connection using Compute Express Link (CXL).

[0007] A typical solid-state drive (SSD) is configured or designed as a non-volatile storage device that retains the entire data set received from a host system in the event of an unexpected power failure. The solid-state drive may have volatile memory (e.g., SRAM or DRAM) that acts as a buffer for processing data storage access messages received from a host system (e.g., read commands, write commands). To prevent data loss during a power failure, the solid-state drive is typically configured with an internal backup power source, allowing the solid-state drive to continue operating for a limited period of time during a power outage to save the data buffered in the volatile memory (e.g., SRAM or DRAM) to the non-volatile memory (e.g., NAND). If the limited time is sufficient to retain the data in the volatile memory (e.g., SRAM or DRAM), the drive can then be powered on to the non-volatile memory (e.g., NAND).To ensure the continued operation of the volatile memory (e.g., SRAM or DRAM) during a power outage, the volatile memory protected by the backup power source can be considered non-volatile from the host system's perspective. Typical backup power source implementations (e.g., capacitors, battery packs) limit the amount of volatile memory (e.g., SRAM or DRAM) configured in the solid-state drive to maintain its non-volatile properties as a data storage device. If the functions of such volatile memory are implemented using fast non-volatile memory, the solid-state drive can forgo a backup power source.

[0008] When a solid-state drive (SSD) is configured with a host interface that supports Compute Express Link protocols, a portion of the SSD's fast, volatile memory can optionally be configured to provide cache-coherent storage services to the host system. These storage services can be accessed via load / store instructions executed at the byte level (e.g., 64B or 128B) in the host system over the Compute Express Link connection. Another portion of the SSD's volatile memory can be reserved for internal use by the SSD as buffer memory to facilitate data storage services for the host system. These data storage services can be accessed via read / write commands provided by the host system at the logical block level (e.g., 4 KB) over the Compute Express Link connection.

[0009] When such a solid-state drive (SSD) is connected to a host system via a Computer ExpressLink connection, the solid-state drive can be connected to and used by the host system as both a storage device and a data storage device. The data storage device provides data storage capacity that is addressable by the host system via block-level read and write instructions for database records; and the storage device provides physical storage that is addressable by the host system via byte-level load and store instructions.

[0010] A solid-state drive (SSD) can have a small amount of volatile memory (e.g., DRAM or SRAM) and a large amount of non-volatile memory (e.g., NAND). The volatile memory is faster than the non-volatile memory. A portion of the volatile memory and a portion of the non-volatile memory can be used to implement a storage device that a host system can access via a Computer Express Link (CXL) connection. The storage device provided by the SSD can have an addressable memory area larger than that which can be implemented using the SSD's volatile memory.

[0011] The storage device provided by the solid-state drive can be configured to have some pages of the memory area residing in volatile memory and thus addressable via physical memory addresses in the portion of volatile memory allocated to the storage device. The remaining pages of the memory area can be swapped out to non-volatile memory.

[0012] When a memory access request (e.g., as a result of executing a load instruction or a store instruction) is addressed to a page that is not currently in volatile memory, a paging system of the solid-state drive can pull the page's contents from non-volatile memory into volatile memory. For example, the solid-state drive can allocate a page from volatile memory, retrieve the contents of the accessed page from non-volatile memory, and store the contents in the allocated page.

[0013] If a storage access request is configured to load data from the side of the storage area provided by the storage device, the solid-state drive can serve the request using the data from the appropriate physical addresses of the memory cells in the allocated side.

[0014] If a storage access request is configured to store data in the page of the storage area provided by the storage device, the solid-state drive can serve the request by storing the data in memory cells at the appropriate physical addresses in the page allocated by volatile memory. If the storage access request causes a page to be allocated by volatile memory to be the page of the storage area being accessed, the data can be stored in parallel with the retrieval of the page's contents from non-volatile memory.After retrieving the page content from non-volatile memory, the portion of the content located outside the physical addresses updated by the memory access request can be stored in memory cells within the allocated page without storing the corresponding portion that was updated by the memory access request. Alternatively, the memory access request data can be buffered for combination with the page content retrieved from non-volatile memory before the modified page content is stored in the allocated page of volatile memory.

[0015] The solid-state drive can be configured to periodically write the contents of pages in the volatile memory, or in the event of a power failure, or when a page of the volatile memory needs to be reassigned to another page in the memory area for active memory operations, to corresponding pages of the memory area in the non-volatile memory.

[0016] For example, if the portion of volatile memory allocated to the storage device is full (e.g., because it was allocated to represent some pages of the memory area hosted in non-volatile memory), the solid-state drive can select a page to swap back to non-volatile memory when the host system requests access to another page of the memory area that has not yet been moved to volatile memory. For instance, the solid-state drive can select a least recently used (LRU) page and write the selected page from volatile memory to non-volatile memory. Alternatively, another page swap technique, such as first-in, first-out (FIFO), optimal page swap, etc., can be used.

[0017] Optionally, the solid-state drive's paging system can be configured to proactively back up a page in the background, which can then be selected for paging out the output according to a page-swap technique. For example, while the host system is accessing some of the pages that have been moved to volatile memory, the paging system can select a page as a candidate for paging out and write the page to non-volatile memory if the selected page contains changes that would render the corresponding page in non-volatile memory obsolete. When the page in volatile memory and the corresponding page in non-volatile memory contain the same content, the page in volatile memory is considered clean and can be reused.If the host system requests access to another page that has not yet been loaded into volatile memory, a purged page can be immediately allocated to represent the requested page, since the purged page's content can be deleted instantly without needing to be saved. This reduces or minimizes the delay in responding to the memory request.

[0018] Optionally, a portion of the non-volatile memory allocated to implement the storage device is also configured to be accessible as part of the data storage device. This allows the host system to access data in the portion of non-volatile memory via a memory access protocol and to access the data via a data storage access protocol. For example, the storage device's memory area can be accessed via logical block addresses in a namespace of the memory area on the solid-state drive using a data storage access protocol, and it can be accessed via memory addresses in the memory area implemented by the storage device using a cache-coherent memory access protocol.

[0019] Optionally, the host system can send a configuration request to the solid-state drive to customize the storage services provided by the solid-state drive over the computer ExpressLink connection. The configuration request can, for example, identify resource allocations for implementing the storage device, such as the size of the storage area provided by the storage device, the amount of volatile memory allocated to existing pages of the storage area, the amount of non-volatile memory allocated to host pages of the storage area, the range of memory addresses for accessing the storage area, a data storage area namespace for accessing the data in the storage area via a data storage access protocol, and so on.

[0020] A host system and a solid-state drive can be configured to use data store commands to optimize the storage of data in the storage device, which is implemented using the non-volatile memory of the solid-state drive and is connected to the host system via a Computer Express Link connection through the solid-state drive.

[0021] For example, when the host system executes a memory instruction, it can send a memory request over the Computer Express Link connection using the Cache-Coherent Memory Access Protocol to store a data item at a memory address in the storage device provided by the solid-state drive. In response to the memory's memory request, the solid-state drive can generate a command in a buffer for the request. This command is configured to be executable on the solid-state drive to write the data item to the non-volatile memory allocated on the solid-state drive to implement the memory address of the storage device.

[0022] Optionally, multiple commands resulting from multiple storage requests can be combined to improve efficiency and performance. For example, the data provided by the multiple storage access requests can be buffered and combined before being written to the non-volatile memory of the solid-state drive allocated to implement the storage device. Combining writes to the non-volatile memory can improve efficiency and reduce write gain.

[0023] For example, a portion of the solid-state drive's DRAM or SRAM can be configured as a buffer to host one or more queues for data storage messages. In response to a storage request, the solid-state drive can submit a command to the queues to write the data item from the storage request to a location corresponding to the memory address defined in the storage request. The solid-state drive can then retrieve the command from the queues and execute it to write the data item to the drive's non-volatile memory (e.g., NAND).

[0024] Typically, the size of the data element to be stored for a single memory storage request is smaller than a page of memory cells in the non-volatile memory (e.g., NAND) configured to be programmed together. To execute a command for a partial update of a page containing a memory address of the memory storage request, the solid-state drive can be configured to read the page of memory cells, update the data retrieved from the page using the data element from the memory storage request, allocate a free page of memory cells, and program the allocated page to store the updated data.

[0025] For improved performance, the solid-state drive can be configured to aggregate commands in the data storage message queues that write to the same side of memory cells. The data from the aggregated commands can be combined in the DRAM or SRAM before being written to non-volatile memory (e.g., NAND) via a programming operation. This aggregation can reduce the number of writes to non-volatile memory (e.g., NAND) to improve efficiency and performance and reduce write gain.

[0026] In some applications, the host system might, for example, store data in random memory addresses located across many pages. The available storage capacity of the solid-state drive's DRAM or SRAM allows for the simultaneous caching of a subset of these pages. Since a large portion of the pages are not overwritten, it can be more efficient to use a portion of the DRAM or SRAM to store the instructions for writing to non-volatile memory, rather than rotating some of the pages in and out of the DRAM or SRAM to cache a subset at a time. Reducing the frequency of page loading and unloading from the DRAM or SRAM can improve the solid-state drive's peak performance when handling random storage requests.

[0027] Optionally, a portion of the solid-state drive's DRAM or SRAM can be allocated as part of the storage device, which is connected to the host system via the solid-state drive. The host system can optionally input the appropriate commands directly into the data storage message queues configured in the portion of DRAM or SRAM used to implement the storage device, instead of sending the storage requests to the storage system for conversion into commands by the solid-state drive.

[0028] Optionally, the host system and / or the solid-state drive can be configured to selectively use the command queues or the cached pages to implement the storage of data in memory addresses implemented in the non-volatile memory of the storage device provided by the solid-state drive over the Computer Express Link connection.

[0029] In some applications, the host system can use a command entered into the queues as a hint to assist the solid-state drive in selecting a page to swap from cache memory to non-volatile memory.

[0030] Optionally, the host system can configure one or more datastore access queues in its memory to provide the solid-state drive with the commands configured to write data to memory addresses implemented in the non-volatile memory.

[0031] It is advantageous for a host system to use a communication protocol to query the solid-state drive about its storage connectivity capabilities, such as whether the solid-state drive can provide cache-coherent storage services, the amount of storage the solid-state drive can connect to the host system when providing storage services, how much of the storage that can be connected to provide the storage services can be considered non-volatile (e.g., implemented via non-volatile memory or backed up with a backup power source), the access time of the storage that can be allocated to the storage services by the solid-state drive, etc.

[0032] The query result can be used to configure memory allocation in the solid-state drive (SSD) to provide cache-coherent storage services. For example, a portion of the SSD's fast storage can be allocated to the host system for cache-coherent storage access, while the remaining portion can be reserved by the SSD for internal use. Partitioning the SSD's fast storage for different services can be configured to balance the benefit of the storage services offered to the host system by the SSD with the performance of the storage services implemented by the SSD for the host system.Optionally, the host system can explicitly request the solid-state drive to release a requested portion of its fast, volatile memory as memory accessible over a connection by using a cache-coherent memory access protocol according to Computer Express Link.

[0033] For example, if the solid-state drive is connected to the host system to provide data storage services via a Computer Express Link connection, the host system can send a command to the solid-state drive to query the storage interface capabilities of the solid-state drive.

[0034] For example, the command to query memory connection capabilities can be configured with a command identifier different from a read command; and in response, the solid-state drive is configured to provide a response indicating whether the solid-state drive is capable of acting as a storage device to provide storage services accessible via load and store instructions. Furthermore, the response can be configured to identify an amount of available memory that can be allocated and connected as a storage device accessible via the computer's ExpressLink connection. Optionally, the response can also be configured to include an identification of an amount of available memory that the host system considers non-volatile and can use as a storage device.The non-volatile portion of the storage device connected by the solid-state drive can be implemented via non-volatile memory or volatile memory supported by a backup power source and the non-volatile storage capacity of the solid-state drive.

[0035] Optionally, the solid-state drive (SSD) can be configured with more volatile memory than is protected by the backup power source. In the event of a power failure, the backup power source is sufficient to retain data from a portion of the SSD's volatile memory up to its capacity, but not all of the data in the volatile memory up to its capacity. Therefore, the response to the memory availability query can include an indication of the ratio of volatile to non-volatile portions of memory that can be allocated to storage services by the SSD. Optionally, the response can also include an identification of the access time of the memory that can be allocated to cache-coherent storage services by the SSD.For example, if the host system requests data from the solid-state drive via a cache-coherent protocol over the Compute Express link, the solid-state drive can provide the data in a timeframe no longer than the access time.

[0036] Optionally, a preconfigured response to such a query can be stored at a predetermined location in the data storage device connected to the host system via the solid-state drive. The predetermined location could be, for example, a predefined logical block address within a predefined namespace. The preconfigured response could also be configured as part of the solid-state drive's firmware. The host system can then use a read command to retrieve the response from the predetermined location.

[0037] Optionally, if the solid-state drive (SSD) has the capability to act as a storage device, it can automatically allocate a predetermined amount of its fast, volatile memory as a storage device connected to the host system via the Computer Express Link connection. The predetermined amount can be a minimum or standard amount configured in a production SSD facility, or an amount defined by configuration data stored on the SSD. Subsequently, the memory portability query can optionally be implemented in the cache-coherent memory access protocol instruction set (instead of the datastore access protocol instruction set); and the host system can use the query to retrieve parameters defining the SSD's memory portability.For example, the solid-state drive can store the parameters at predetermined memory addresses in the storage device, and the host can retrieve the parameters by executing load commands with the corresponding memory addresses.

[0038] It is advantageous if a host system individually adapts aspects of the storage services of the storage subsystem (e.g., a solid-state drive) to the storage and data storage usage patterns of the host system.

[0039] For example, the host system can define a size of memory offered by the solid-state drive for connection to the host system, so that a set of physical memory addresses configured according to the size can be addressed by executing load / store instructions in the processing device(s) of the host system.

[0040] Optionally, the host system can define the time requirements for accessing the storage device over the Compute Express Link (CXL) connection. For example, if the cache requests access to the storage location over the connection, the solid-state drive must provide a response within the access time defined by the host system when configuring the solid-state drive's storage services.

[0041] Optionally, the host system can define how much of the storage device connected to the solid-state drive (SSD) must be non-volatile to prevent data loss in the non-volatile portion of the storage device connected to the host system if the SSD's external power supply fails. The non-volatile portion can be implemented by the SSD via non-volatile memory or volatile memory with a backup power source to allow data copying operations from volatile memory to non-volatile memory to continue during an interruption of the SSD's external power supply.

[0042] Optionally, the host system can define whether the solid-state drive should connect a storage device to the host system via the Compute Express Link (CXL) connection.

[0043] The solid-state drive (SSD) can, for example, have a region configured to store the configuration parameters of the storage device to be connected to the host system via the Compute Express Link (CXL) connection. When the SSD restarts, boots up, or powers on, it can allocate a portion of its storage resources as a storage device for connection to the host system, according to the configuration parameters stored in this region. Once the SSD has configured the storage services according to the parameters stored in this region, the host system can access them via the cache by executing load and store instructions that identify the corresponding physical memory addresses.The solid-state drive can configure its remaining storage resources to provide storage services over the Computer Express Link (CXL) connection. For example, a portion of its random-accessible volatile memory can be allocated as buffer memory reserved for the solid-state drive's processing device(s); and this buffer memory is inaccessible and unaddressable to the host system via load / store instructions.

[0044] When the solid-state drive is connected to the host system via a Computer Express Link connection, the host system can send commands to adjust the configuration parameters stored in the space for the connectable storage device. The host system can then request the solid-state drive to restart in order to connect a storage device with storage services configured according to the configuration parameters via the Computer Express Link to the host system.

[0045] For example, the host system may be configured to issue a write command (or save commands) to save the configuration parameters to a predetermined logical block address (or predetermined memory addresses) in the range to customize the setting of the storage device configured to provide storage services over the computer Express Link connection.

[0046] Alternatively, a command with a command identifier that differs from a write command (or a store instruction) can be configured in the write / read protocol (or in the load / store protocol) to instruct the solid-state drive to adjust the configuration parameters stored in the space.

[0047] Fig. Figure 1 illustrates an exemplary computing system 100 that includes a storage subsystem 110 according to some embodiments of the present disclosure. The storage subsystem 110 may include computer-readable storage media, such as one or more volatile storage devices (e.g., storage device 107), one or more non-volatile storage devices (e.g., storage device 109), or a combination thereof.

[0048] In Fig. 1. The storage subsystem 110 is configured as a manufactured product (e.g., a solid-state drive) that can be used as a component installed in a computing device.

[0049] The storage subsystem 110 also includes a host interface 113 for a physical connection 103 with a host system 120.

[0050] The host system 120 can have a interconnection 121 that connects a cache 123, a memory 129, a memory controller 125, a processing device 127 and a memory manager 101 configured to set up the memory services of the memory subsystem 110.

[0051] The memory manager 101 in the host system 120 can be implemented at least partially via instructions executed by the processing device 127, or via a logic circuit, or both. The memory manager 101 in the host system 120 can send configuration parameters to the storage subsystem to customize or control a storage device connected to the host system 120 via the storage subsystem 110. Optionally, the memory manager 101 in the host system 120 is implemented as part of the operating system 135 of the host system 120, or of a device driver configured to operate the storage subsystem 110, or as a combination of such software components.

[0052] Connection 103 can conform to the Compute Express Link (CXL) standard or other communication protocols that support cache-coherent memory access and datastore access. Optionally, multiple physical connections 103 are configured to support communication for cache-coherent memory access and datastore access communication.

[0053] The processing device 127 can be a microprocessor configured as the central processing unit (CPU) of a computing device. Instructions (e.g., load instructions, store instructions) executed in the processing device 127 can access the memory 129 via the memory controller (125) and the cache 123. Furthermore, if the storage subsystem 110 connects a storage device to the host system via the connection 103, instructions (e.g., load instructions, store instructions) executed in the processing device 127 can access the storage device via the memory controller (125) and the cache 123 in a similar manner to accessing the memory 129.

[0054] For example, in response to the execution of a load instruction in the processing device 127, the memory controller 125 can convert a logical memory address defined by the instruction into a physical memory address to request memory access from the cache 123 to retrieve data. The physical memory address might be located, for example, in the memory 129 of the host system 120 or in the storage device connected to the host system 120 via the connection 103 through the storage subsystem 110. If the data at the physical memory address is not already in the cache 123, the cache 123 can load the data from the corresponding physical address as cached content 131. The cache 123 can then provide the cached content 131 to fulfill the request for memory access to the physical memory address.

[0055] For example, in response to the execution of a memory instruction in the processing device 127, the memory controller 125 can convert a logical memory address defined by the instruction into a physical memory address to request the cache 123 to access memory and store data. The cache 123 can hold the data of the memory instruction as cached content 131 and indicate that the corresponding data at the physical memory address is stale. When the cache 123 needs to free a cache block (e.g., to load new data from different memory addresses or to hold data from memory instructions with different memory addresses), the cache 123 can flush the cached content 131 from the cache block to the corresponding physical memory addresses (e.g.,in the memory 129 of the host system or in the storage device connected to the host system 120 via the connection 103 through the storage subsystem 110).

[0056] The connection 103 between the host system 120 and the storage subsystem 110 can support a protocol for cache-coherent memory access. Cache coherence ensures that changes to one copy of the data corresponding to a memory address are propagated to other copies of the data corresponding to that memory address; and that load / store accesses to the same memory address are seen by the processing devices (e.g., 127) in the same order.

[0057] The operating system 135 can include routines of instructions programmed for processing memory access requests from applications.

[0058] In some implementations, the host system 120 configures a portion of its memory (e.g., 129) to function as a queue 133 for memory access messages. Such memory access messages can contain read commands, write commands, erase commands, and so on. A memory access command (e.g., read or write) can define a logical block address for a block of data in a storage device (which is connected to the host system 120, for example, via the memory subsystem 110 and the connection 103). The storage device can retrieve the messages from the queues 133, execute the commands, and make the results available in the queues 133 for further processing by the host system 120 (e.g., using routines in the operating system 135).

[0059] A data block addressed by a memory access command (e.g., read or write) is typically much larger than a data unit accessed by a memory access instruction (e.g., load or store). Thus, memory access commands, using the routines in Operating System 135, can be suitable for batch processing of large amounts of data (e.g., data in a file managed by a file system) simultaneously and in the same way. Memory access instructions can be efficient for random access to small amounts of data without the overhead of Operating System 135 routines.

[0060] The storage subsystem 110 has an interconnection 111 which connects the host interface 113, a controller 115 and storage resources, such as storage devices 107, ..., 109.

[0061] The controller 115 of the storage subsystem 110 can control the operations of the storage subsystem 110. The operations of the storage subsystem 110 can, for example, respond to memory access messages in the queues 133 or to memory access requests from the cache 123.

[0062] In some implementations, each of the storage devices (e.g., 107, ..., 109) comprises one or more integrated circuit devices, each enclosed in a separate integrated circuit package. In other implementations, each of the storage devices (e.g., 107, ..., 109) is configured on a single integrated circuit chip; and the storage devices (e.g., 107, ..., 109) may be configured in the same integrated circuit device, enclosed within the same integrated circuit package. In still other implementations, the storage subsystem 110 is implemented as an integrated circuit device with an integrated circuit package that includes the storage devices 107, ..., 109, the controller 115, and the host interface 113.

[0063] For example, a storage device 107 of the storage subsystem 110 may have volatile random-access memory 138 that is faster than the non-volatile memory 139 of a storage device 109 of the storage subsystem 110. Thus, the non-volatile memory 139 can be used to provide the storage capacity of the storage subsystem 110 for data retention. At least a portion of the storage capacity can be used to provide data storage services to the host system 120. Optionally, a portion of the volatile random-access memory 138 can be used to provide cache-coherent storage services to the host system 120. The remaining portion of the volatile random-access memory 138 can be used to provide buffer services for the controller 115 when processing the storage access messages in the queues 133 and when performing other operations (e.g.,provide wear compensation, memory cleanup, error detection and correction, encryption).

[0064] If the volatile random-access memory 138 is used to buffer data received from the host system 120 before it is stored in the non-volatile memory 139, the data in the volatile random-access memory 138 may be lost if the power supply to the storage device 107 is interrupted. To prevent data loss, the storage subsystem 110 may have a backup power source 105 sufficient to power the storage subsystem 110 for a period of time to allow the controller 115 to transfer the buffered data from the volatile random-access memory 138 to the non-volatile memory 139 if the external power supply to the storage subsystem 110 is interrupted.

[0065] Optionally, the fast memory 138 can be implemented using non-volatile memory (e.g., crosspoint memory); and the backup power source 105 can be omitted. Alternatively, a combination of fast non-volatile memory and fast volatile memory can be configured in the storage subsystem 110 for storage services and buffer services.

[0066] Host system 120 can send a memory connectivity query to storage subsystem 110 via connection 103. In response, storage subsystem 110 can provide a reply identifying: whether storage subsystem 110 can provide cache-coherent storage services over connection 103, the amount of storage that can be connected to provide the storage services over connection 103, how much of the storage available to host system 120 for the storage services is considered non-volatile (e.g., implemented via non-volatile memory or secured by a backup power source 105), the access time of the storage that can be allocated to host system 120 for the storage services, and so on.

[0067] Host system 120 can send a request to storage subsystem 110 via connection 103 to configure the storage services provided to host system 120 by storage subsystem 110. In the request, host system 120 can define: whether storage subsystem 110 should provide cache-coherent storage services via connection 103; the amount of storage to be provided as storage services via connection 103; how much of the storage provided via connection 103 is considered non-volatile (e.g., implemented via non-volatile memory or backed by a backup power source 105); the access time of the storage provided as storage services to host system 120; and so on. In response, storage subsystem 110 can allocate its resources (e.g., storage devices 107, ..., 109) partition and provide the requested storage services via connection 103.

[0068] If a portion of memory 138 is configured to provide storage services over connection 103, the host system 120 can access a cached portion 132 of memory 138 via load and store instructions and the cache 123. Non-volatile memory 139 can be accessed via read and write commands transmitted through the queues 133 configured in the memory 129 of the host system 120.

[0069] The storage manager 101 in the storage subsystem 110 can implement the storage services provided via connection 103 as a storage device attached to the host system 120 that utilizes the resources of the storage subsystem 110. For example, the storage manager 101 can allocate a portion of the fast, volatile memory 138 as cache memory to access a memory area hosted in the slow, non-volatile memory 139. Optionally, this memory area can overlap with a portion of the data storage area provided to the host system 120 by the storage subsystem 110. Thus, a portion of the non-volatile memory 139 can be accessed via both the storage services and the data storage services.

[0070] In general, the memory manager 101 can be implemented in the host system 120 or in the memory subsystem 110, or partly in the host system 120 and partly in the memory subsystem 110. The memory manager 101 in the memory subsystem 110 can be implemented at least partly via instructions (e.g., firmware) executed by the processing device 117 of the controller 115 of the memory subsystem 110, or via a logic circuit, or both.

[0071] Fig. Figure 2 shows a storage subsystem configured, according to one embodiment, to provide both storage and data storage services to a host system via a physical connection. For example, the storage subsystem 110 and the host system 120 of the Fig. 2 be implemented in the same way as the computer system 100 of the Fig. 1.

[0072] In Fig. 2. The storage resources (e.g., the storage devices 107, ..., 109) of the storage subsystem 110 are divided into a loadable part 141 and a readable part 143 (and an optional part for buffer storage 149 in some cases, as in Fig. 5) subdivided. A physical connection 103 between the host system 120 and the storage subsystem 110 can support a protocol 145 for load instructions and store instructions to access storage services provided in the loadable part 141. For example, the load instructions and store instructions can be executed via the cache 123. The connection 103 can further support a protocol 147 for read instructions and write instructions to access the data storage services provided in the readable part 143. For example, the read instructions and write instructions can be provided via the queues 133 configured in the memory 129 of the host system 120. For example, a physical connection 103 that supports a computer express link can be used to connect the host system 120 and the storage subsystem 110.

[0073] Fig. Figure 2 illustrates an example of a single physical connection 103 (e.g., a Computer Express Link connection) configured to facilitate both storage access communication according to one protocol 145 and data storage access communication according to another protocol 147. In general, separate physical connections can be used to provide the host system 120 with storage access according to one protocol 145 and data storage access according to another protocol 147.

[0074] Fig. Figure 3 shows a storage subsystem configured according to one embodiment to provide storage services using a portion of the non-volatile storage capacity of the storage subsystem. For example, the storage services of the Fig. 3 in the computer systems 100 of the Fig. 1 and Fig. 2 be implemented.

[0075] In Fig. 3. The memory subsystem 110 has a non-volatile storage capacity 151. The non-volatile storage capacity 151 can be implemented using non-volatile memory (e.g., 139) of storage devices (e.g., 109) of the memory subsystem 110.

[0076] A loadable portion 141 of the non-volatile storage capacity 151 can be allocated to provide a storage area to a storage device connected to the host system 120 via the connection 103 through the storage subsystem 110. The host system 120 can use a cache-coherent memory access protocol (e.g., 145) to access the loadable portion 141 via the connection 103, as shown in Fig. 2.

[0077] A readable portion 143 of the non-volatile storage capacity 151 can be allocated to provide a data storage area to a data storage device connected to the host system 120 via the storage subsystem 110 and connection 103. The host system 120 can use a data storage access protocol (e.g., 147) to access the readable portion 143 via connection 103, as shown in Fig. 2.

[0078] A portion of the volatile random access memory 138 of the storage subsystem 110 can be allocated as cache memory 157 to support the storage services provided to the host system 120 via connection 103 through the storage subsystem 110.

[0079] A memory manager 101 of the memory subsystem 110 can be configured to use the cache memory 157 to support and accelerate memory operations that address active pages of the loadable part 141. The memory manager 101 can be implemented via instructions executed in the processing device 117 of the memory subsystem 110, or via logic circuits, or both.

[0080] The remaining portion of the volatile random access memory 138 of the memory subsystem 110 can be used by the memory subsystem 110 as buffer memory 149 when running the firmware 153 and the memory manager 101.

[0081] When a page of the loadable part 141 is used by the host system 120, the memory manager 101 can assign a page in the cache memory 157 as a proxy or cache for the page of the loadable part 141. The memory manager 101 can operate an address card 155 to identify the dynamic correlation between the pages in the cache memory 157 and the pages in the loadable part 141.

[0082] If the address card 155 indicates that a page of the loadable part 141 has a corresponding page of the cache memory 157, memory access requests addressed to the page of the loadable part 141 can be made on the corresponding page of the cache memory 157. For example, if the host system 120 uses the cache-coherent memory access protocol 145 to store data in memory addresses that identify the page of the loadable part 141, the memory manager 101 can identify the corresponding addresses of memory cells in the corresponding page of the cache memory 157 for the memory subsystem 110 to first store the data in the corresponding page of the cache memory 157. The memory subsystem 110 can (e.g.,(using the address card 155) indicate that the corresponding page of the cache memory 157 is contaminated with data to be saved to the corresponding page of the loadable part 141. After the data in the page of the cache memory 157 has been saved to the page of the loadable part 141, the page of the cache memory 157 is cleared, since it contains the same content as the corresponding page of the loadable part 141.

[0083] Cache memory 157 contains fewer pages than the loadable part 141. When all the pages of cache memory 157 are used to represent certain active pages in the loadable part 141, cache memory 157 becomes full. Memory manager 101 can identify one or more pages in cache memory 157 as candidates to represent alternative pages of the loadable part 141 that are actively used by the host system 120. Memory manager 101 can be configured, for example, to identify candidate pages using techniques such as LRU (least recently used), First In First Out (FIFO), optimal page swapping, etc.

[0084] If the address card 155 indicates that a candidate page is contaminated, the memory manager 101 can proactively clean the page by writing its contents to the appropriate page in the loadable part 141.

[0085] If the host system 120 uses the cache-coherent protocol 145 to access a page of the loadable part 141 that is not already represented by a corresponding page in the cache memory 157, the memory manager 101 can subsequently update the address card 155 to use a purged candidate page of the cache memory 157 to represent the page of the loadable part 141 being accessed. The memory manager 101 can read the page of the loadable part 141 being accessed to retrieve page data and store the page data in the purged candidate page of the cache memory 157, discarding the existing content of the purged candidate page. No data is lost because the existing content is the same as in the page previously represented by the candidate page.Address card 155 can be updated to identify the page being accessed as represented by the candidate page.

[0086] A memory access that causes the candidate page to be assigned to represent the page of the loadable part 141 being accessed may request a retrieval of data from the accessed page. In response to such a memory access, the memory manager 101 may be configured to store the page data retrieved from the loadable part 141 into the candidate page. Subsequently, the candidate page can be addressed to serve the memory access as if the candidate page were the page being accessed.

[0087] Because retrieving data from the loadable part 141 takes longer than servicing data from the cache memory 157, a significant delay can occur in servicing the memory access. This delay causes the candidate page to be allocated and configured to represent a page in the loadable part 141. Optionally, the memory manager 101 can indicate an error in a response to the memory access while the page data is being retrieved from the loadable part 141. If the host system 120 subsequently makes the same memory access, the candidate page can be ready to represent the page being accessed, and the memory manager 101 can use the candidate page to serve the memory access as if the candidate page were the page being accessed.Operating memory access using the cache memory 157 is faster than operating memory access using the loadable part 141 in the non-volatile memory capacity 151.

[0088] A memory access that assigns the candidate page to represent the page of the loadable part 141 being accessed may request data to be saved to the accessed page. In response to such a memory access, the memory manager 101 may be configured to save the combined data representing the page data being updated by the memory access to the candidate page. The candidate page is then polluted (e.g., via an entry in the address card 155) until the changes to the corresponding page in the loadable part 141 are saved.

[0089] For example, after memory manager 101 has allocated a candidate page of cache memory 157 to represent a page of the loadable part 141 being accessed in response to a memory access to store data at a memory address, memory manager 101 can store the data in the corresponding memory address in cache memory 157 in parallel with reading the page of the loadable part 141 being accessed. Once the access page data is available, memory manager 101 can write the data to the remaining addresses in the candidate page, skipping the memory address where the data provided by the host system 120 through the memory access was stored.

[0090] Alternatively, the memory manager 101 can temporarily hold the data received from the host system 120 during memory access in the buffer 149 while the page data is retrieved from the page in the loadable part 141 being accessed. When the page data is available, the memory manager 101 can update the page data (e.g., in the buffer 149) and move the updated page data to the candidate page of the cache 157. Optionally, the update can be performed in the candidate page of the cache 157.

[0091] In some implementations, the non-volatile memory 139 of the memory subsystem 110 has a structure consisting of pages of memory cells and blocks of memory cell pages. A page of memory cells is the smallest unit for programming memory cells to store data. The memory cells of a page are configured to be programmed together in an atomic programming operation. A block of memory cell pages is the smallest unit for erasing memory cells, enabling individual memory cell pages within the block to be programmed to store data. The memory cell pages in a block are configured to be erased together in an atomic erasure operation.

[0092] The pages of the memory area in the loadable part 141, which are to be represented by the pages in the cache memory 157, can be configured to correspond to memory cell pages of the non-volatile memory 139. If a corrupted page in the cache memory 157 is stored in the loadable part 141, the number of programming operations required to save the data from the page in the cache memory 157 is thus minimized. For example, a page in the cache memory 157 can be configured to represent a memory cell page in the loadable part 141. If the page in the cache memory 157 becomes corrupted, the corresponding memory cell page can be marked as no longer in use and thus deleted.If the page of cache memory 157 is to be stored back into the loadable part, the memory subsystem 110 can allocate a free memory cell page that has been erased and program the allocated memory cell page to store the data of the page of cache memory 157.

[0093] The firmware 153 can, for example, include a flash translation layer (FTL) configured to translate logical memory addresses into physical memory cell addresses in the non-volatile storage capacity 151. Instead of mapping a page of the cache memory 157 to a fixed memory cell page, the memory manager 101 can configure the address card 155 to map the page of the cache memory 157 to a logical data storage page, which can then be mapped by the flash translation layer (FTL) to a dynamically allocated memory cell page to store the page's data in the cache memory 157.

[0094] Optionally, the loadable part 141 can also be configured to be accessible via the memory access protocol 147 over connection 103. For example, the memory subsystem 110 can create a namespace of the non-volatile data storage capacity 151 and assign the namespace to the loadable part 141. Memory locations in the namespace, and thus the loadable part 141, can be addressed via logical block addresses in the namespace; and the host system 120 can use write and read commands in the memory access protocol 147 to write data to and retrieve data from the loadable part 141.Furthermore, locations in the loadable part 141 can be addressed via memory addresses that are mapped to the namespace via the address card 155 (and the Flash Translation Layer (FTL) address card); and the host system 120 can use load instructions and store instructions to access cache-coherent memory access via the protocol 145 to store data in and load data from the loadable part 141.

[0095] Fig. Figure 4 illustrates the operations configured in a memory subsystem according to one embodiment to respond to a memory access request. For example, the operations of Fig. 4 in a storage subsystem 110 of the Fig. 3 in computer systems 100 of the Fig. 1 and Fig. 2 be implemented.

[0096] In Fig. 4. A memory access request 161 is transmitted from a host system 120 via a connection 103 to a host interface 113 of the storage subsystem 110. For example, the memory access request 161 can correspond to a protocol 145 for cache-coherent memory access.

[0097] Memory access request 161 identifies a memory address 163, which represents a memory area in a storage device connected to the host system 120 via connection 103 through the memory subsystem 110.

[0098] The memory subsystem 110 manages an address card 155 which identifies the correlations between pages (e.g. 177) of the cache memory 157 and pages (e.g. 167) of the data storage memory in the loadable part 141.

[0099] The cache memory 157 is faster than the loadable part 141, but has fewer pages than the loadable part 141. The pages (e.g., 177) of the cache memory 157 can be used to represent a portion of the data store pages (e.g., 167) that are actively used by the host system 120.

[0100] The memory area in the loadable part 141 is pre-divided into pages 167. Thus, the memory subsystem 110 can calculate a page identification 165 of the data storage memory page 167 from the memory address 163, which contains a memory location identified by the memory address 163.

[0101] The memory subsystem 110 can determine whether the identification 165 of the data storage memory page is contained in the address card 155 and whether it is assigned to a page identification 175 of a cache memory page 177.

[0102] If a cache memory page 177 has been assigned to represent the data store memory page 167, the address card 155 contains data that maps the identification 165 of the data store memory page and the page identification 175 of the cache memory. Then the memory subsystem 110 can translate the memory address 163 in the data store memory page 167 to the corresponding memory address 173 in the corresponding cache memory page 177.

[0103] For example, if memory address 163 identifies a memory location at memory cells 168 in data storage page 167, the memory subsystem 110 can determine the memory address 173 of the corresponding memory cells 178, which represent memory cells 168. Thus, the memory access request 161 is applied for the corresponding memory operations in memory cells 178 at memory address 173.

[0104] For example, if memory access request 161 involves loading data from memory address 163, the memory subsystem 110 can retrieve data from memory cells 178 to provide a response to request 161.

[0105] For example, if the memory access request 161 concerns storing data in memory address 163, the memory subsystem 110 can store the data in memory cells 178 and update the page status 171 to indicate that the cache memory page 177 is contaminated, indicating that page 177 contains data that should be stored back in the data store memory page 167 identified by the data store memory page identification 165.

[0106] If cache page 177 is contaminated but not actively used by host system 120 via memory access requests (e.g., 161), storage subsystem 110 can retrieve the data from cache page 177 and store it in datastore page 167. Then, storage subsystem 110 can update page status 171 to indicate that cache page 177 is cleared, meaning that page 177 contains the same data as the corresponding datastore page 167.

[0107] If the memory access request 161 has a memory address 163 located in a data store memory page 167, but the address card 155 indicates that the data store memory page 167 is not yet represented by a cache memory page (e.g., 177), the memory subsystem 110 can allocate a cleared cache memory page (e.g., 177) (or a free cache memory page that has not yet been allocated to represent a data store memory page) to represent the data store memory page 167.

[0108] To configure cache memory page 177 to represent data store memory page 167, the storage subsystem 110 can retrieve data from data store memory page 167 and store the data in cache memory page 177. Furthermore, the storage subsystem 110 can update the address card 155 to map the page identification 175 of cache memory page 177 to the page identification 165 of data store memory page 167.

[0109] In some implementations, a cache memory page 177 is configured to represent a memory cell page 167 with memory cells 168, ..., 169, which are arranged in a memory device 109 of an integrated circuit such that they are programmed together to store data in an atomic programming operation. Thus, saving the data of the cache memory page 177 to the loadable part 141 can be performed in a single programming operation.

[0110] In a typical implementation, the cache memory 157 has fewer limitations and is faster than the non-volatile memory 139. For example, the memory cells 178, ..., 179 in the cache memory page 177 can be programmed separately to store data via separate programming operations.

[0111] In some implementations, a block of memory cell pages (e.g., 167) in a memory device 109 of an integrated circuit is configured to be erased together to allow the pages (e.g., 167) to be programmed to store data. To avoid unnecessary copying and erasing of data, the memory subsystem 110 can use the identification 165 of the data storage memory page to represent logical pages in the loadable part 141. The logical pages can be further mapped to memory cell pages (e.g., 167).

[0112] Optionally, the Flash Translation Layer (FTL) function of firmware 153 of memory subsystem 110 can be used to facilitate the mapping of logical pages to memory cell pages (e.g. 167).

[0113] Optionally, memory address 163 can be configured based on a logical data storage area of ​​the loadable part 141. For example, a namespace of the non-volatile storage capacity 151 can be assigned to host the loadable part 141. The Flash Translation Layer (FTL) of the firmware 153 can translate a logical block address in the namespace into identifications of one or more memory cell pages (e.g., 167). The memory address space of the loadable part 141 can have a predefined relationship to the logical block addresses in the namespace. Thus, the identification 165 of the data storage memory page can be configured to be based on logical block addresses in the namespace for mapping to memory cell pages (e.g., 167).

[0114] Since the relationship between the memory addresses (e.g., 163) for memory access requests (e.g., 161) and the logical block addresses in the namespace allocated for the loadable part 141 is predetermined, the host system 120 has the ability to address the memory cell pages (e.g., 167) via memory access requests (e.g., 161) using protocol 145 for cache-coherent memory access or via data store access requests using the data store access protocol 147, as described in Fig. 5.

[0115] Fig. Figure 5 illustrates a configuration to enable, according to one embodiment, a portion of the non-volatile data storage capacity of the storage subsystem to be addressable via both storage services and data storage services. For example, the technique of Fig. 5 in a storage subsystem 110 of the Fig. 3 in a computer system 100 of the Fig. 1 and Fig. 2 be implemented.

[0116] In Fig. 5 The non-volatile data storage capacity 151 can have a readable part 143 that is configured to be accessible via data storage access requests (e.g. 181) that use logical block addresses (e.g. 183) to address storage locations according to a data storage access protocol 147 over a connection 103 between a storage subsystem 110 and a host system 120.

[0117] A part 141 of the readable part 143 can be connected to the host system 120 via the memory subsystem 110 as a storage device through connection 103. Thus, the host system can use memory access requests 161 to access memory addresses (e.g., 163) in the loadable part 141 via connection 103 using a protocol 147 for cache-coherent memory access. For example, memory access in Fig. 5 as in Fig. 4 must be implemented.

[0118] Optionally, the logical block addresses (e.g., 183) can be configured to address memory cell pages (e.g., 167) in the non-volatile memory (e.g., 139) of memory subsystem 110. Alternatively, a logical block address (e.g., 183) can be used to address a logical block containing a plurality of memory cell pages (e.g., 167).

[0119] In contrast, a memory address (e.g. 163) is configured to identify a storage unit of a subset of memory cells (e.g. 168) in a memory cell page (e.g. 167).

[0120] Optionally, the storage subsystem 110 can allocate multiple namespaces of the non-volatile data storage capacity 151 to the loadable part 141. This allows different parts of the storage device connected by the storage subsystem 110 to be accessed via different namespaces using the data storage access protocol 147.

[0121] Optionally, the storage subsystem 110 can allocate multiple namespaces of the non-volatile data storage capacity 151, each for multiple loadable parts (e.g., 141). The loadable parts (e.g., 141) can be connected as separate storage areas via the connection 103, which are addressable by the host system 120 via the protocol 145 for cache-coherent memory access.

[0122] In response to memory access request 161 or data store access request 181, the memory subsystem 110 can determine whether data store memory page 167 is cached in cache memory 157 according to address card 155. If so, the memory subsystem 110 can identify cache memory page 177 and serve memory access request 161 using cache memory page 177; otherwise, the memory subsystem 110 can cache data store page 167 in cache memory 157.

[0123] For example, a method for providing storage services using a data storage capacity of a storage subsystem according to one embodiment in a storage subsystem 110 of the Fig. 3 in computer systems of Fig. 1 and Fig. 2 using the techniques of Fig. 4 and Fig. 5 must be implemented.

[0124] For example, the storage subsystem 110 may have a host interface 113 that can be operated via a connection 103 to a host system 120 according to a data storage access protocol (e.g., 147) and a cache-coherent memory access protocol (e.g., 145). The storage subsystem 110 may have a first non-volatile memory (e.g., 139) configured to provide a non-volatile data storage capacity 151 of the storage subsystem 110, and a second volatile memory (e.g., 138) that is faster than the first memory (e.g., 139). A controller 115 of the storage subsystem 110 may be configured to: allocate a first page (e.g., 177) of the second memory (e.g., 138) to a second page (e.g., 138).167) in a memory area provided by a storage device connected to the host system via the connection through the storage subsystem; and operating the first page (e.g. 177) of the second memory (e.g. 138) in response to a memory access request 161 transmitted to the host interface 113 via the connection 103 in accordance with the cache-coherent memory access protocol 145, when the memory access request 161 identifies a memory address 163 in the memory area.

[0125] For example, the controller 115 can be configured via the firmware 153 to implement the operations of a memory manager 101 for swapping pages between the volatile memory 138 and the non-volatile memory 139 and for caching pages in the cache memory 157. Optionally, each page (e.g., 167) cached in the volatile memory (e.g., 138) of the memory subsystem 110 can be configured to have a memory cell page size allocated to host a portion of the memory area addressable by the host system 120 using memory addresses (e.g., 163). The memory cells of each memory cell page are configured in a memory device (e.g., 109) of an integrated circuit to be programmed together in an atomic programming operation to store data.

[0126] Optionally, the storage manager 101 can allocate a portion 141 of non-volatile data storage capacity 151 of the storage subsystem 110 to a namespace, connect the namespace as a storage device to a host system 120 via a Computer Express Link (CXL) connection 103 between a host interface 113 of the storage subsystem 110 and the host system 120, and provide the host system 120 with: data storage access to the namespace using a data storage access protocol 147 over the connection 103 and logical block addresses (e.g., 183) defined in the namespace; and memory access to a memory area corresponding to the namespace using a cache-coherent memory access protocol 145 over the connection 103 and memory addresses (e.g., 163).For example, the memory manager 101 can map the memory area to the namespace according to a predetermined relationship, so that the same data can be stored or retrieved via memory addresses and via logical block addresses.

[0127] In the procedure, a storage subsystem 110 connects a storage device, which has a storage area (e.g. the loadable section 141) configured in a first memory (e.g. 139) of the storage subsystem 110, via a connection 103 from a host interface 113 of the storage subsystem 110 to a host system 120.

[0128] The storage subsystem 110 can, for example, be a solid-state drive that includes volatile random-access memory 138 and non-volatile memory 139.

[0129] In the procedure, the storage subsystem 110 connects a data storage device via the connection 103 from the host interface 113 of the storage subsystem 110 to the host system 120. The data storage device has a data storage area (e.g., the readable section 143) configured in the first memory (e.g., 139) of the storage subsystem 110.

[0130] Optionally, the data storage area (e.g., the readable part 143) matches (or contains) the storage area (e.g., the loadable part 141).

[0131] In the procedure, the memory subsystem 110 allocates a quantity of second memory (e.g. 138) that is faster than the first memory (e.g. 139) to represent pages of the memory area (e.g. the loadable part 141) when handling memory access requests (e.g. 161) in the memory area.

[0132] In the procedure, the memory subsystem 110 manages an address card 155 which is configured to identify correlations between pages (e.g. 177) of the second memory (e.g. 138) and corresponding pages (e.g. 167) of the memory area represented by the pages (e.g. 177) of the second memory (e.g. 138).

[0133] For example, the address card 155 can contain data that assigns a first identification 175 to the first page (e.g. 177) to a second identification 165 to a second page (e.g. 167) of the memory area.

[0134] Optionally, the identification 165 of the second page (e.g. 167) can be based on a logical block address 183 in the data storage area.

[0135] For example, a Flash Translation Layer (FTL) of memory subsystem 110 can be used to map the logical block address 183 to one or more pages (e.g., 167) of memory cells (e.g., 168, ..., 169) in memory subsystem 110. Thus, the physical memory locations of memory address 163 in non-volatile memory 139 can change based on the mapping of the Flash Translation Layer (FTL).

[0136] In the procedure, the memory subsystem 110 operates a first page 177 of the second memory (e.g. 138) in response to a memory access request 161, which is transmitted over the connection 103 according to a protocol 145 for cache-coherent memory access, to identify a memory address 163 in the memory space.

[0137] Optionally, the first page 177 of the second memory (e.g., 138) is configured to represent a memory cell page with memory cells 168, ..., 169, which are configured to be programmed together to store data in an atomic programming operation. Thus, the memory size of a cache memory page 177 is equal to the memory size of a memory cell page 167.

[0138] The memory manager 101 can be configured to swap content from the second page 167 to the first page 177 in response to the host system 120 accessing memory addresses (e.g., 163) in the second page 167 according to the cache-coherent memory access protocol from the first memory (e.g., 139).

[0139] Memory manager 101 can be configured to save the contents of the first page 177 to the first memory location (e.g., 139) in response to a determination that the host system 120 is not actively accessing the second page 167. Saving the contents of the first page 177 can be performed proactively before the host system 120 accesses a third page in the memory area, causing memory manager 101 to use the first page 177 to represent the third page (e.g., based on a page-replacement technique such as LRU (least recently used), First In First Out (FIFO), optimal page replacement, etc.).

[0140] For example, in response to memory access request 161, which identifies memory address 163, memory manager 101 can determine that the second page 167 of the memory area is not yet represented by a page in the second memory 138. In response, memory manager 101 can allocate the first page 177 of the second memory 138, retrieve page data from the memory cell page (e.g., 167), store the page data in the first page 177 of the second memory 138, and update address card 155 to indicate that the first page 177 represents the second page 167.

[0141] If the memory access request 161 is configured to store initial data in memory address 163, the memory manager 101 can store the initial data in the first page 177 of the second memory 138 and update a page status 171 in the address card 155 to indicate that the first page 171 contains content to be stored in the second memory 139.

[0142] Optionally, in response to memory access request 161 being configured to store the initial data at memory address 163, memory manager 101 can store data identifying that memory cell page 167, previously used to host the second page containing memory address 163, is no longer in use. This allows firmware 153 to reclaim the memory space of memory cell page 167 during a background cleanup operation.

[0143] For example, in response to the fact that the host system is actively using other pages of the memory space and thus the second page 167 of the memory space is not actively used, the memory manager 101 can store the contents of the first page 177 in the second memory 139 and update a page status 171 in the address card 155 to indicate that the content in the first page 177 is the same as in a corresponding page in the second memory. Thus, the first page 177 is cleared and can be reassigned to represent another page of the memory space used by the host system 120.

[0144] For example, to save the contents of the first page 177, the flash translation layer can allocate a memory cell page 167; and the memory subsystem 110 can perform an atomic programming operation to store the contents in memory cell page 167. The memory manager 101 can then update the address card 155 to indicate that the cache memory page 177 has been cleared and represents the page hosted in memory cell page 167.

[0145] In the procedure, the storage subsystem 110 operates the first memory (e.g. 139) in response to a memory access request transmitted over connection 103 according to a data storage access protocol 147 to identify a logical block address in the data storage area.

[0146] For example, if the logical block address identifies a data storage location outside the loadable part 141, the flash translation layer (FTL) of the memory subsystem 110 can determine the memory cells used for the logical block address in the non-volatile memory 139 and serve the data storage access request by reading or programming the memory cells.

[0147] If the logical block address identifies a data storage location within the loadable part 141, the memory manager 101 can determine whether a portion of the memory cells in the non-volatile memory 139 addressed by the logical block address is represented by a page (e.g., 177) in the cache memory 157. If so, the memory subsystem 110 can serve the data storage request via the cache memory page (e.g., 177) and the first memory for the remaining portion of the logical block address that is not represented by pages in the cache memory 157.

[0148] Fig. Figure 6 shows a technique for using a write command to implement, according to one embodiment, a request to store data at a memory address implemented in non-volatile memory.

[0149] For example, the technique of Fig. 6 in a storage subsystem 110 of the Fig. 3 in computer systems of Fig. 1 and Fig. 2 be implemented and optionally in conjunction with the techniques of Fig. 4 and Fig. 5 can be used.

[0150] In Fig. 6. A host system 120 can be accessed via a connection 103 (e.g., as in Fig. 1, Fig. 2 and Fig. 3) Send storage requests 191 of the memory, such as a memory access request 161 to store data 193 at a memory address 163, to a memory subsystem 110.

[0151] For example, in response to the host system 120 executing a memory instruction, a memory memory request 191 can be generated over connection 103 using a protocol 145 for cache-coherent memory access. The memory memory request 191 can be configured to request the memory subsystem to store data 193 at a memory address 163.

[0152] In response, memory subsystem 110 can optionally cache a memory page containing memory address 163 if the memory page is implemented in the non-volatile memory 139 of memory subsystem 110.

[0153] For example, a cache memory page 177 can be allocated to and used in the volatile random-access memory 138 of the memory subsystem 110 to store the data 193 of the memory storage request 191, which defines the memory address 163. Optionally, the memory subsystem 110 can swap the current contents of the memory page from the non-volatile memory 139 to the volatile memory 138 for fast access and to combine it with the data 193 of the memory storage request 191. Subsequently, the memory subsystem 110 can swap the contents of the cache memory page 177 back to the non-volatile memory 139 for persistent storage (and / or to free up space in the volatile random-access memory 138 for caching other pages), as described above in conjunction with Fig. 3, Fig. 4 and Fig. 5 discussed.

[0154] Optionally, in response to the memory's memory storage request 191, the memory subsystem 110 can generate a write instruction 192 in the data storage access queues 134. The execution of instruction 192 in the memory subsystem can cause the data 193 from the memory's storage request 191 to be written to the data storage page 167, which contains memory addresses including memory address 163.

[0155] Optionally, the data 193 to be written by the write command 192 can be buffered in the volatile direct-access memory 138 (e.g., in the cache memory 157 or in the buffer memory 149). Alternatively, the data 193 to be written by the write command 192 can be stored in one or more messages in the queues 134.

[0156] Write command 192, used to store the data of a memory storage request 191, can be configured to identify a memory location using memory address 163 (instead of a logical block address 183). Write command 192 can be used to write data with a granularity / data size smaller than the block size required to write to a logical block address 183.

[0157] The data storage access queues 134 can be located in a buffer memory 149 of the storage subsystem 110 (e.g., as in Fig. 7) or be configured in a section of the loadable part 141 that is configured (or cached) in the fast volatile memory 138 of the memory subsystem 110 (e.g., as in Fig. 8) In some implementations, the datastore access queues 133 configured in the memory 129 of the host system 120 can be used by the host system 120 to send write commands (e.g., 192) to the storage subsystem 110 (e.g., as in Fig. 9).

[0158] In some applications, it is more efficient to swap the contents of data storage page 167 to cache storage page 177 for access and modification by the host system 120. For example, if the storage requests (e.g., 191) from the host system 120 are concentrated on one or more storage pages within a given timeframe, which can be cached simultaneously in the fast, volatile memory 138, it may be efficient to combine the data (e.g., 193) from the storage requests 191 in cache storage page 177 before swapping the contents back to non-volatile memory 139.

[0159] In some cases, the host system 120 can enter a write command 192 as a hint to the memory subsystem 110 to save the contents of the cache memory page 177 to the non-volatile memory 139 (and thus clear the cache memory page 177 and make it ready to cache another page).

[0160] For example, based on a memory access pattern, host system 120 can determine or predict that the most recent accesses to the memory addresses cached in page 177 have completed and that page 177 will not be accessed for the next period. In response, host system 120 can issue write command 192 to queue 134 to write the contents of cache memory page 177 to non-volatile memory 139. Write command 192 can serve as a signal to memory subsystem 110 to save the contents of cache memory page 177 to non-volatile memory 139 (for example, to prepare cache memory page 177 for caching another memory page).

[0161] In some implementations, the data store access queues 134 are configured in a loadable part 141 of the storage subsystem 110 (e.g., as in Fig. 8) Thus, the host system 120 can store data corresponding to the command 192 in the queues 134 via connection 103 using protocol 145 for cache-coherent memory access, in order to write the contents of the cache memory page 177 to the non-volatile memory 139. Alternatively, the host system 120 can use the queues 133 configured in its memory 129 to provide the command 192 (e.g., as in Fig. 9); and the storage subsystem 110 can retrieve the command 192 via connection 103 using the data storage access protocol 147.

[0162] In some applications, it is more efficient to use write commands (e.g., 192) to implement memory storage requests (191) rather than using cache memory pages (e.g., 177). For example, if the host system (120) randomly updates small portions of a large number of pages that cannot be hosted simultaneously in the cache memory (157), it may be advantageous to convert the memory storage requests (191) into write commands (e.g., 192) whose data (e.g., 193) is buffered in the queues (134) or in the buffer memory (149), without using the cache memory (157) to host the respective pages in their entirety.

[0163] For example, if the storage subsystem 110 detects such a pattern of storage requests 191 from the memory, the storage subsystem 110 can generate the write instruction 192 to implement the requests 191, instead of swapping pages from the cache memory 157 to the non-volatile memory 139 to make room for cache pages that are modified by the requests 191. For example, the data store access queues 134 can be configured in the buffer memory 149 of the storage subsystem 110, as shown in Fig. 7.

[0164] Optionally, the processing device 117 of the memory subsystem 110 (e.g. via firmware 153) can be configured to combine multiple write commands addressing the same memory page, so that the page can be reprogrammed to include the data from the multiple commands (e.g. 192).

[0165] Storing write commands 192 in queues 134 allows the memory subsystem 110 to replay memory storage requests 191 in the cache memory 157 over time to improve peak memory access performance.

[0166] Optionally, the host system 120 can be configured to detect such a pattern of storing data in memory addresses scattered across many memory pages. Instead of sending the memory storage requests 191 over connection 103 using the cache-coherent protocol 145, the host system 120 can alternatively generate the write commands (e.g., 192) and inject them into the queues 134 over connection 103 using the cache-coherent memory access protocol 145. The storage subsystem 110 can then retrieve the write commands 192 from the queues 134 using a local connection, without using the datastore access protocol 147 over connection 103. For example, the data storage access queues 134 in the loadable part 141 can be configured for direct access by the host system 120 via connection 103, as shown in Fig. 8. Alternatively, the host system can use 120 datastore access queues 129 configured in its memory to provide commands, as described in Fig. 9; and the storage subsystem 110 can retrieve the commands from the queues 133 using the data storage access protocol 147 via the connection 103.

[0167] Fig. Figure 7 shows a storage subsystem configured with data storage access queues to implement storage requests according to one embodiment. For example, some of the above in conjunction with Fig. 6 techniques discussed in the storage subsystem 110 of the Fig. 7 must be implemented.

[0168] The memory subsystem 110 of the Fig. 7 has a loadable part 141 and a readable part 143 configured in the non-volatile memory capacity 151, as shown in Fig. 3. Optionally, the loadable part 141 within the readable part 143 can be configured to allow access via both the cache-coherent storage protocol 145 and the data storage access protocol 147, as described in Fig. 5.

[0169] In Fig. 7. The data storage access queues 134 are configured in the buffer memory 149 of the storage subsystem 110. The data storage access queues 134 can be used by the processing device 117 of the storage subsystem 110 to implement storage requests 191 received via the connection 103 to store data in the loadable part 141. For example, the requests 191 can be converted into write commands 192 for execution 195 over time to improve peak performance, as described above in conjunction with Fig. 6 discussed.

[0170] For example, in response to a memory storage request 191, the processing device 117 of the memory subsystem 110 can optionally input a write command 192 into the queues 134 and buffer the data 193 of the request 191 into the buffer memory 149. If, for example, the memory page containing the memory address 163 defined by the request 161 is not already present in the cache memory 157, and the cache memory 157 does not have a cleared cache memory page (e.g., 177) that can be used immediately, the processing device 117 of the memory subsystem 110 can generate a write command 192 for the request 191 and input the command into the queues 134 for subsequent execution 195. Such an approach can be faster than swapping the contents of a cache memory page (e.g., 177) to the loadable part 141 to access the cache memory page (e.g.,177) to release the data 193 of the storage request 191 of the memory into the cache memory 157. Thus, the peak performance of the storage subsystem 110 when handling requests 191 to store data in the loadable part 141 can be improved.

[0171] In some implementations, the datastore access queues 134 can be configured in a loadable part that is connected to the host system 120 via the storage subsystem 110 through the connection 103, as in Fig. 8. Such a configuration allows the host system 120 to use the data store access queues 134 for storage operations.

[0172] Fig. Figure 8 shows a storage subsystem that includes data storage access queues addressable by a host system via storage services to implement storage requests according to one embodiment. For example, at least some of the above in conjunction with Fig. 6 and Fig. 7 techniques discussed in the storage subsystem 110 of the Fig. 8 will be implemented.

[0173] In Fig. 8 is a loadable part 142 configured in a fast memory (e.g. 138) of the memory subsystem 110.

[0174] For example, the memory subsystem 110 can connect a combination of part 142 of its fast memory (e.g., 138) and part 141 of its slow memory (e.g., 139) as a storage device that the host system 120 can access via connection 103 using a cache-coherent protocol 145. In this way, the host system 120 can generate write commands (e.g., 192) and / or buffer data (e.g., 193) to be written to the loadable part 142 via the write commands (e.g., 192), instead of sending the memory requests 191 to the memory with addresses (e.g., 163) located in the slow memory (e.g., 139). The processing device 117 of the storage subsystem 110 can be configured via the firmware 153 to execute the write commands (e.g. 192) in the queues 134 (e.g. as in conjunction with Fig. 6 discussed), to store the buffered data (e.g. 193) of the write commands (e.g. 192) in the loadable part 141.

[0175] Optionally, the memory subsystem 110 can allocate another portion of its faster memory (e.g., 138) as cache memory 157 to cache memory pages with memory addresses in the loadable portion 141, which is hosted in the non-volatile storage capacity 151 (e.g., as in Fig. 3).

[0176] Optionally, the loadable part 142 in the cache memory 157 can be implemented as one or more cached pages of the loadable parts 141 in the non-volatile storage capacity 151.

[0177] Optionally, the storage subsystem 110 can operate one or more data storage access queues in a buffer memory 149, as shown in Fig. 7, in addition to and separate from the queues 134 shared with the host system 120.

[0178] Optionally, the host system can use 120 configured datastore access queues 133 in its memory to send commands 192, as described in Fig. 9, to optimize the performance of storage subsystem 110 when providing storage services over connection 103.

[0179] Fig. Figure 9 shows a host system that has data storage access queues which a storage subsystem can access to implement storage requests according to one embodiment. For example, at least some of the above in conjunction with Fig. 6, Fig. 7 and Fig. 8 techniques discussed in the memory subsystem 110 of the Fig. 9 will be implemented.

[0180] For example, the storage subsystem 110 can allocate a cache memory 157 to cache the contents of a memory page hosted in the loadable part 141, as in Fig. 3.

[0181] In Fig. 9. The host system 120 can use a command 192 in the queues 133 to indicate that the contents of a cached memory page 177 can be swapped back to the non-volatile storage capacity 151. In response, the processing device 117 can write the contents of the cached memory page 177 back to the non-volatile storage capacity 151 without having to wait for page 177 to be identified for swapping based on a page swap technique (e.g., LRU (least recently used), First In First Out (FIFO), optimal page swap, etc.).

[0182] Optionally, the loadable part 141 is configured to be at least partially located within the readable part 143; and the host system 120 can use a write command (e.g., a datastore access request 181) to write the contents of a cache memory page 177 into the loadable part 141, as shown in Fig. 5.

[0183] Optionally, the host system 120 can be configured to send the command 192 to the storage subsystem via a datastore access queue 134 configured in a loadable part 142 of the storage subsystem 110, as shown in Fig. 8.

[0184] Optionally, the storage subsystem 110 can operate one or more data storage access queues in a buffer memory 149, as shown in Fig. 7.

[0185] Fig. Figure 10 shows a method for storing data via storage services implemented in a data storage capacity of a storage subsystem according to one embodiment. For example, the method of Fig. 10 in a storage subsystem 110 of the Fig. 3, Fig. 7, Fig. 8 or Fig. 9 in computer systems of Fig. 1 and Fig. 2 using the techniques of Fig. 4, Fig. 5 and Fig. 6 must be implemented.

[0186] In block 201, a storage subsystem 110 connects a first part (e.g. 141) of the storage resources of the storage subsystem 110 as a storage device via a connection 103 from a host interface 113 of the storage subsystem 110 to a host system 120, which the host system 120 can access via the connection 103 using a first protocol 145 for cache-coherent memory access.

[0187] For example, connection 103 can be configured according to a Computer Express Link (CXL) standard.

[0188] In block 203, the storage subsystem 110 connects a second part (e.g. 143) of the storage resources of the storage subsystem 110 as a storage device via connection 103, which the host system 120 can access via connection 103 using a second protocol 147 for data storage access.

[0189] For example, the memory subsystem 110 can be configured to allocate a first portion of its volatile random-access memory 138 as cache memory 157, a second portion of its volatile random-access memory 138 as buffer memory 149, and optionally a third portion of its volatile random-access memory 138 as part of the storage device. The majority of the storage device (e.g., the loadable portion 141) can be configured in the non-volatile storage capacity 151 provided by a non-volatile memory 139 of the memory subsystem 110. The non-volatile memory 139 is slower than the volatile random-access memory 138.

[0190] To improve the performance of the storage device's storage services, the storage subsystem 110 can use the cache memory 157 to cache recently accessed pages of the storage device. Optionally, a portion of the storage device (for example, used to host the datastore access queues 134) can be pinned to the cache memory 157.

[0191] For example, if sufficient space is available in cache memory 157, the storage subsystem 110 can process storage requests (e.g., 191) from the storage by writing the data from requests 191 to cache memory pages (e.g., 177). After the storage subsystem 110 has received, for example, from the host system 120 via connection 103 using the first protocol 145 for cache-coherent memory access, one or more second requests (e.g., 181) to store data in the storage device, the storage subsystem 110 can process the one or more second requests (e.g., 181) by operating one or more pages (e.g., 167) of the storage device cached in the cache memory (e.g., 157).

[0192] In block 205, the storage subsystem 110 receives a first request (e.g. 191) via connection 103 according to the first protocol 145 for cache-coherent memory access, to store data (e.g. 193) at a memory address (e.g. 163) in the storage device.

[0193] In block 207, memory subsystem 110 generates an instruction 192 according to the first request (e.g., 191).

[0194] In block 209, the memory subsystem 110 inserts the instruction 192 into a queue 134.

[0195] For example, the storage subsystem 110 can buffer the data 193 in queue 134 (or at another location in the buffer memory 149). Thus, an execution 195 of the instruction writes the data 193 from queue 134 (or buffer memory 149) to the storage device.

[0196] In response to a determination that the cache memory 157 does not have sufficient capacity to cache a page of the storage device at the memory address (e.g. 163) at the time of receiving the first request (e.g. 191), the storage subsystem 110 can, for example, generate an instruction (e.g. 192) in the data storage access queues 134 for the request 191.

[0197] For example, in response to the detection of a pattern of random data storage in more than a threshold number of pages of the storage device (e.g., more than the number of pages that can be cached simultaneously in the cache memory 157), the storage subsystem 110 can convert the random data storage into a sequence of instructions in the data storage access queues 134, instead of allocating cache memory pages 177 for the random data storage requests. The storage subsystem 110 can execute the instructions (e.g., 192) from the queues in a reasonable time and in an efficient manner.

[0198] For example, memory subsystem 110 can identify a multitude of instructions (e.g., 192) in queue 134 that have addresses on the same memory page (e.g., 167) in the storage device. In response, memory subsystem 110 can aggregate the data of the multitude of instructions (e.g., 192) to generate a memory page content (e.g., 167). For example, memory subsystem 110 can read the current content of the memory page (e.g., 167), update the content of the memory page in buffer memory 149 according to the data (e.g., 193) of the instructions (e.g., 192), allocate a free page of memory cells, and program the free page of memory cells in non-volatile memory 139 to store the aggregated data (e.g., 193) of the multitude of instructions (e.g., 192).

[0199] In block 211, the memory subsystem 110 executes instruction 192 from queue 134 to store the data (e.g. 193) at the memory address (e.g. 163) in the storage device.

[0200] In some implementations, queue 134 is configured in buffer memory 149. Alternatively, queue 134 is configured in cache memory 157 (e.g., as a cached part of loadable part 141) or in a loadable part 142 hosted in volatile random-access memory 138 of storage subsystem 110.

[0201] If the queue 134 is configured in the cache memory 157 or in the loadable part 142, the host system 120 can use the cache-coherent memory access protocol 145 to input commands (e.g., 192) into the queue 134.

[0202] For example, in response to the detection of a pattern of random data storage in more than a threshold number of pages of the storage device (e.g., more than the number of pages that can be cached simultaneously in the cache memory 157), the host system 120 can input the write commands (e.g., 192) into the queues to write data to the storage device, instead of sending the storage requests 191 to the memory with their memory addresses 163. The storage subsystem 110 can retrieve the commands 192 locally from the queues 134 for execution 195 at one time and in a manner with an improved or optimized performance level.

[0203] Optionally, the host system 120 can use its storage host queues 133 to send the commands (e.g. 192) over the connection 103 using the data storage access protocol 147 to the storage subsystem 110.

[0204] Optionally, the host system 120 can issue a command to queues 133 or 134 to cause the storage subsystem 110 to swap the contents of a cached page from cache memory 157 to non-volatile memory 138. The storage subsystem 110 can retrieve such a command from queue 133 via connection 103 using the data store access protocol 146, or from queue 134 using a local connection without using connection 103 to the host system 120.

[0205] In general, a memory subsystem 110 can be a data storage device, a memory module, or a combination of a data storage device and a memory module. Examples of a storage device include a solid-state drive (SSD), a flash drive, a universal serial bus (USB) flash drive, an embedded multimedia controller (eMMC) drive, a universal flash memory (UFS) drive, a secure digital (SD) card, and a hard disk drive (HDD). Examples of memory modules include a dual in-line memory module (DIMM), a small-outline DIMM (SO-DIMM), and various types of non-volatile dual in-line memory modules (NVDIMM).

[0206] The Computing System 100 can be a computing device such as a desktop computer, a laptop computer, a network server, a mobile device, a vehicle (e.g., an airplane, a drone, a train, an automobile, or another means of transport), an Internet of Things (IoT)-enabled device, an embedded computer (e.g., one included in a vehicle, industrial plant, or networked commercial device), or such a computing device that includes a storage and processing device.

[0207] The computing system 100 can include a host system 120 that is coupled to one or more storage subsystems 110. Fig. Figure 1 illustrates an example of a host system 120 coupled to a storage subsystem 110. In the sense used herein, “coupled to” or “coupled with” generally refers to a connection between components, which may be an indirect communicative connection or a direct communicative connection (e.g., without intervening components), both wired and wireless, including connections such as electrical, optical, magnetic, etc.

[0208] For example, the host system 120 can include a processor chipset (e.g., the processing device 127) and a software stack executed by the processor chipset. The processor chipset can include one or more cores, one or more caches (e.g., 123), a memory controller (e.g., controller 125) (e.g., NVDIMM controller), and a data storage protocol controller (e.g., PCIe controller, SATA controller). The host system 120 uses the memory subsystem 110, for example, to write data to and read data from the memory subsystem 110.

[0209] The host system 120 can be coupled to the storage subsystem 110 via a physical host interface 113. Examples of a physical host interface include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, a Fibre Channel, a Serial Attached SCSI (SAS) interface, a Double Data Rate (DDR) memory bus interface, a Small Computer System Interface (SCSI), a Dual Inline Memory Module (DIMM) interface (e.g., a DIMM socket interface that supports Double Data Rate (DDR)), an Open NAND Flash (ONFI) interface, a Double Data Rate (DDR) interface, a Low Power Double Data Rate (LPDDR) interface, a Compute Express Link (CXL) interface, or any other interface.The physical host interface can be used to transfer data between the host system 120 and the storage subsystem 110. The host system 120 can also use an NVM Express (NVMe) interface to access components (such as the storage devices 109) when the storage subsystem 110 is connected to the host system 120 via the PCIe interface. The physical host interface can provide an interface for forwarding control, address, data, and other signals between the storage subsystem 110 and the host system 120. Fig. Figure 1 illustrates a storage subsystem 110 as an example. In general, the host system 120 can access multiple storage subsystems via a single communication link, multiple separate communication links, and / or a combination of communication links.

[0210] The processing device 127 of the host system 120 can be, for example, a microprocessor, a central processing unit (CPU), a processor core, an execution unit, etc. In some cases, the controller 125 can be described as a memory controller, a memory management unit, and / or an initiator. In one example, the controller 125 manages communications over a bus coupled between the host system 120 and the memory subsystem 110. Generally, the controller 125 can send commands or requests to the memory subsystem 110 for the desired access to the memory devices 109 and 107. The controller 125 can also include an interface circuit for communicating with the memory subsystem 110. The interface circuit can convert responses received from the memory subsystem 110 into information for the host system 120.

[0211] The controller 125 of the host system 120 can communicate with the controller 115 of the storage system 110 to perform operations such as reading, writing, or erasing data on the storage devices 109, 107, and other such operations. In some cases, the controller 125 is integrated into the same housing as the processing device 127. In other cases, the controller 125 is separate from the housing of the processing device 127. The controller 125 and / or the processing device 127 may include hardware such as one or more integrated circuits (ICs) and / or discrete components, a buffer memory, a cache memory, or a combination thereof. The controller 125 and / or the processing device 127 may include a microcontroller, a special logic circuit (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.).) or another suitable processor.

[0212] The memory devices 109 and 107 can include any combination of different types of non-volatile memory components and / or volatile memory components. The volatile memory devices (e.g., memory device 107) can include, among others, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0213] Some examples of non-volatile memory components include negative AND (or NOT-AND) (NAND) flash memory and write-in-place memory, such as three-dimensional crosspoint ("3D crosspoint") memory. A crosspoint array of non-volatile memory can perform bit storage based on a change in volume resistance, in conjunction with a stackable cross-grid data access array. Furthermore, unlike many flash-based memories, non-volatile crosspoint memory can perform a write-in-place operation, where a non-volatile memory cell can be programmed without first erasing it. NAND-type flash memory includes, for example, two-dimensional NAND (2D-NAND) and three-dimensional NAND (3D-NAND).

[0214] Each of the storage devices 109 can contain one or more arrays of memory cells. One type of memory cell, for example, single-level cells (SLC), can store one bit per cell. Other types of memory cells, such as multi-level cells (MLCs), triple-level cells (TLCs), quad-level cells (QLCs), and penta-level cells (PLCs), can store multiple bits per cell. In some embodiments, each of the storage devices 109 can contain one or more arrays of memory cells, such as SLC, MLC, TLC, QLC, PLC, or any combination thereof. In some embodiments, a specific storage device can contain an SLC section, an MLC section, a TLC section, a QLC section, and / or a PLC section of memory cells. The memory cells of the storage devices 109 can be grouped as pages, which can refer to a logic unit of the storage device used to store data.With some types of memory (e.g., NAND), pages can be grouped into blocks.

[0215] Although non-volatile memory devices such as 3D crosspoint and NAND-type memory (e.g., 2D NAND, 3D NAND) are described, the memory device 109 can be based on any other type of non-volatile memory, such as read-only memory (ROM), phase change memory (PCM), self-selecting memory, other chalcogenide-based memory, ferroelectric transistor random-access memory (FeTRAM), ferroelectric random-access memory (FeRAM), magneto random-access memory (MRAM), spin-transfer-torque (STT) MRAM, conductive-bridging RAM (CBRAM), resistive random-access memory (RRAM), oxide-based RRAM (OxRAM), negative-OR (NOR) flash memory, and electrically erasable programmable memory. Read-only memory (Electrically Erasable Programmable Read-Only Memory - EEPROM).

[0216] A controller 115 of the storage subsystem (or, for simplicity, controller 115) can communicate with the storage devices 109 to perform operations such as reading, writing, or erasing data from the storage devices 109, and other such operations (e.g., in response to instructions scheduled on an instruction bus by the controller 125). The controller 115 can include hardware such as one or more integrated circuits (ICs) and / or discrete components, a buffer memory, or a combination thereof. The hardware can include digital circuits with dedicated (i.e., hard-coded) logic to perform the operations described herein. The controller 115 can be a microcontroller, a special-purpose logic circuit (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.), or another suitable processor.

[0217] The controller 115 can include a processing device 117 (a processor) configured to execute instructions stored in a local memory 119. In the illustrated example, the local memory 119 of the controller 115 includes embedded memory configured to store instructions for performing various processes, operations, logical sequences, and routines that control the operation of the memory subsystem 110, including handling communications between the memory subsystem 110 and the host system 120.

[0218] In some embodiments, the local memory 119 can include memory registers in which memory pointers, retrieved data, etc., are stored. The local memory 119 can also include a read-only memory (ROM) for storing microcode. While the exemplary memory subsystem 110 in Fig. As illustrated in Figure 1, which includes the controller 115, a storage subsystem 110 in another embodiment of the present disclosure may not include a controller 115 and may instead rely on an external controller (e.g. provided by an external host or by a processor or controller separate from the storage subsystem).

[0219] In general, the controller 115 can receive commands or operations from the host system 120 and can translate these commands or operations into instructions or appropriate commands to achieve the desired access to the storage devices 109. The controller 115 may also be responsible for other operations, such as wear compensation operations, garbage collection operations, error-correcting code (ECC) operations, encryption operations, caching operations, and address translations between a logical address (e.g., a logical block address (LBA), a namespace) and a physical address (e.g., a physical block address) assigned to the storage devices 109. The controller 115 may also include a host interface circuit to communicate with the host system 120 via the physical host interface.The host interface circuit can convert commands received from the host system into command instructions to access the storage devices 109, and can convert responses assigned to the storage devices 109 into information for the host system 120.

[0220] The memory subsystem 110 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 110 may include a cache or buffer (e.g., DRAM) and an addressing circuit (e.g., a row decoder and a column decoder) that can receive an address from the controller 115 and decode the address to access the memory devices 109.

[0221] In some embodiments, the storage devices 109 include local media controllers 137 that work in conjunction with the controller 115 of the storage subsystem to perform operations on one or more memory cells of the storage devices 109. An external controller (e.g., the controller 115 of the storage subsystem) can manage the storage device 109 externally (e.g., perform media management operations on the storage device 109). In some embodiments, a storage device 109 is a managed storage device, which is a raw storage device combined with a local controller (e.g., the local media controller 137) for media management within the same storage device housing. An example of a managed storage device is a managed NAND (MNAND) device.

[0222] In one embodiment, an exemplary machine may be a computer system in which a set of instructions can be executed to cause the machine to perform one or more of the procedures discussed herein. In some embodiments, the computer system may correspond to a host system (e.g., Host System 120 of the Fig. 1), which is a storage subsystem (e.g., storage subsystem 110 of the Fig. 1) includes, is linked to, uses, or can be used to perform the operations discussed above (e.g., to execute instructions to perform operations that refer to the Fig. (corresponding to the processes described in section 1). In alternative embodiments, the machine can be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine can function as a server or client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment.

[0223] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, a switch or bridge, a network-attached storage device, or any machine capable of executing a set of instructions (sequentially or otherwise) that define actions to be performed by that machine. Furthermore, although a single machine is illustrated, the term "machine" is also to be understood as including any assembly of machines that, individually or collectively, execute a set (or multiple sets) of instructions to perform any one or more of the procedures discussed herein.

[0224] The exemplary computer system includes a processing device, main memory (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), static random access memory (SRAM), etc.), and a data storage system, which communicate with each other via a bus (which may include multiple buses).

[0225] The processing device is one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. In particular, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or processors implementing a combination of instruction sets.The processing device may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processing device is configured to execute instructions for performing the operations and steps discussed herein. The computer system may also include a network interface device for communication over the network.

[0226] The data storage system may include a machine-readable storage medium (also known as a computer-readable medium) on which one or more sets of instructions or software are stored that implement one or more of the procedures or functions described herein. The instructions may also reside wholly or at least partially in the main memory and / or the processing device while being executed by the computer system, the main memory and the processing device also being machine-readable storage media. The machine-readable storage medium, the data storage system, and / or the main memory may be connected to the memory subsystem 110 of the Fig. 1 corresponds.

[0227] In one embodiment, the instructions include instructions for implementing the functions discussed above (e.g., those relating to Fig.(1 described processes). While the machine-readable medium is shown as a single medium in an exemplary embodiment, the term "machine-readable storage medium" should be understood to include a single medium or multiple media that store one or more sets of instructions. The term "machine-readable storage medium" should also be understood to include any medium capable of storing or encoding a set of instructions for execution by the machine, causing the machine to perform one or more of the procedures of this disclosure. Accordingly, the term "machine-readable storage medium" should be understood to include, among other things, solid-state storage media, optical media, and magnetic media.

[0228] Some sections of the preceding detailed descriptions have been presented with respect to algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the ways used by a data processing specialist to most effectively communicate the substance of the work to another specialist. An algorithm is conceived here, and generally, as a self-consistent sequence of operations that leads to a desired result. The operations are those that require physical manipulations of physical quantities. Typically, but not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated.It has sometimes proven practical, mainly for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0229] It should be borne in mind, however, that all these and similar terms must be associated with the corresponding physical quantities and are merely convenient designations for these quantities. The present disclosure may describe the action and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities within the registers and memories of the computer system and transforms them into other data similarly represented as physical quantities within the memories or registers or other such information storage systems of the computer system.

[0230] The present disclosure also relates to a device for carrying out the operations contained herein. This device may be specially designed for the intended purposes or may include a general-purpose computer that is selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of hard disk, including floppy disks, optical disks, CD-ROMs and magnetic-optical disks, read-only memory (ROM), random-access memory (RAM), EPROM, EEPROM, magnetic or optical cards, or any type of media suitable for storing electronic instructions and each coupled to a computer system bus.

[0231] The algorithms and displays presented herein do not necessarily relate to any specific computer or other device. Various general-purpose systems can be used with programs according to the teachings contained herein, or it may prove advantageous to construct a more specialized device to carry out the procedure. The setup for a number of these systems is shown as indicated in the following description. Furthermore, the present disclosure is not described with reference to any specific programming language. It is understood that a number of programming languages ​​can be used to implement the teachings of the disclosure described herein.

[0232] The present disclosure may be provided as a computer program product or software that may include a machine-readable medium containing instructions stored thereon that can be used to program a computer system (or other electronic devices) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form that can be read by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a storage medium readable by a machine (e.g., a computer), such as read-only memory (“ROM”), random-access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory components, etc.

[0233] In this description, various functions and processes may be described as being performed or initiated by computer instructions, in order to simplify the description. However, the person skilled in the art will recognize that such expressions mean that the functions result from the execution of the computer instructions by one or more controllers or one or more processors, such as a microprocessor. Alternatively or in combination, the functions and processes may be implemented using specialized circuits with or without software instructions, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). Embodiments may be implemented using hard-wired circuits without software instructions or in combination with software instructions.Thus, the techniques are not limited to a specific combination of hardware circuits and software, nor to a particular source for the instructions executed by the data processing system.

[0234] In the foregoing description, embodiments of the disclosure have been described with reference to specific exemplary embodiments thereof. It is understood that various modifications can be made to these without deviating from the overall nature and scope of embodiments of the disclosure as set forth in the following claims. Accordingly, the description and the drawings are to be understood in an illustrative and not a limiting sense. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 18 / 439,623

[0001] US 63 / 485,131

[0001]

Claims

[1] Procedure, encompassing: Connecting, through a storage subsystem via a connection from a host interface of the storage subsystem to a host system, a first part of the storage subsystem's memory resources as a storage device that the host system can access via the connection using a first protocol for cache-coherent memory access; Connecting a second portion of the storage subsystem's storage resources as a data storage device, which the host system can access via the connection using a second data storage access protocol, through the storage subsystem via the connection; Receiving an initial request to store data at a memory address in the storage device by the storage subsystem over the connection according to the first protocol for cache-coherent memory access; generating a command according to the initial request by the storage subsystem; entering the command into a queue by the storage subsystem; and Execution of the command from the queue by the memory subsystem to store the data at the memory address in the storage device. [2] Method according to claim 1, wherein the connection conforms to a Computer Express Link (CXL) standard. [3] The method of claim 2, further comprising: Buffering the data in the queue, with one execution of the command writing the data from the queue to the storage device. [4] The method of claim 3, further comprising: Allocating a first portion of a volatile random-access memory of the storage subsystem as cache memory by the storage subsystem, wherein the first portion of memory resources of the storage subsystem includes a non-volatile random-access memory that is slower than the volatile random-access memory; Configuring the cache memory by the storage subsystem to cache one or more pages of a memory area provided by the storage device; Receiving one or more second requests to store data in the storage device, in the storage subsystem from the host system over the connection using the first protocol for cache-coherent memory access; and Processing one or more second requests by the storage subsystem via operation of one or more pages of the storage device that are cached in the cache memory. [5] Method according to claim 4, wherein the generation of the instruction according to the first request is carried out in response to a determination that the cache memory does not have sufficient capacity to cache a page of the storage device at the memory address at the time of receiving the first request. [6] Method according to claim 4, wherein the generation of the instruction according to the first requirement is carried out in response to the detection of a pattern of random storage of data in more than a threshold number of pages of the storage device. [7] Method according to claim 4, further comprising: Allocating a second portion of the volatile random-access memory of the storage subsystem as buffer memory, separate from the storage device and the cache memory, by the storage subsystem; and Configuring the queue in the buffer memory by the storage subsystem. [8] Method according to claim 4, wherein the queue is configured in the cache memory. [9] Method according to claim 4, wherein the first part of memory resources of the memory subsystem includes a third part of the volatile random access memory; and the queue is configured in the third part of the volatile random access memory, which implements a predetermined part of the memory area. [10] Storage subsystem comprising: a host interface that can be operated on a connection to a host system according to a data storage access protocol and a cache-coherent memory access protocol; a first memory configured to provide non-volatile storage capacity of the storage subsystem, wherein at least part of the first memory is configured to implement a storage device connected to the host system via the link; a second memory that is faster than the first memory; and a controller configured to do the following: Retrieving a command configured to identify a memory address in the storage device from a queue; and Executing the command to store data at the memory address in the storage device. [11] Storage subsystem according to claim 10, wherein the controller is further configured as follows: Receiving a request to store data at the memory address over the connection from the host system using the cache-coherent memory access protocol; Generating the command in response to a request; and Enter the command into the queue. [12] Storage subsystem according to claim 10, wherein the controller is further configured as follows: Identifying a large number of commands in the queue that have addresses in a memory page in the storage device; Summarizing data from a multitude of commands to generate content for the memory page; and Programming a page of memory cells in the first memory to store the content for the multitude of instructions. [13] Storage subsystem according to claim 12, wherein the connection is a Computer Express Link (CXL) connection. [14] Storage subsystem according to claim 10, wherein the queue is configured in the part of the first memory that is configured to implement the storage device connected to the host system via the connection; and wherein the controller is further configured to: Receiving the command in the queue over the connection from the host system using the cache-coherent memory access protocol. [15] Storage subsystem according to claim 10, wherein the queue is configured in a memory of the host system; and the controller is configured to retrieve the command over the connection from the host system using the data storage access protocol. [16] Non-transient computer storage medium that stores instructions which, when executed in a computing system, cause the computing system to perform a procedure comprising: Establishing a connection between a host system and a storage subsystem, wherein the connection is operable according to a data storage access protocol and a cache-coherent storage access protocol; Connecting an initial memory of the storage subsystem via the connection to the host system as a storage device that can be accessed via the cache-coherent memory access protocol; Entering a command configured to identify a memory address in the storage device into a data storage access queue; and Execution of the command by the memory subsystem to store data at the memory address in the storage device. [17] Non-transient computer storage medium according to claim 16, wherein the method further comprises: Receiving a request to store data at the memory address in the storage subsystem via the connection from the host system using the cache-coherent memory access protocol; and Generating the instruction by the memory subsystem in response to the request; where the connection is a Computer Express Link connection. [18] Non-transient computer storage medium according to claim 17, wherein the storage subsystem is configured to cache a page of the storage device; and the command is configured to cause the storage subsystem to swap out some content of the page from a cache memory to the first memory. [19] Non-transient computer storage medium according to claim 18, wherein the data storage access queue is configured in a memory of the host system; the host system is configured to input the command into the data storage access queue; and the storage subsystem is configured to retrieve the command from the queue via the connection according to the data storage access protocol. [20] Non-transient computer storage medium according to claim 18, wherein the data storage access queue is configured in a second memory of the storage subsystem; the host system is configured to input the command into the data storage access queue via the connection using the cache-coherent memory access protocol; and the storage subsystem is configured to retrieve the command from the data storage access queue without using the connection.

Images