System and method for data command processing

Abstract

Systems and methods for data command processing are disclosed. A storage device includes a first storage medium, a second storage medium, and a processor. The processor may be configured to: receive a first command associated with first data from a computing device; searching the first storage medium to process the first command; and sending a message to the computing device based on the search for the first storage medium. Based on the transmission of the message, the storage device may be configured to receive a second command associated with the first data from the computing device.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 682,704, filed August 13, 2024, entitled “COMPUTE EXPRESS LINK (CXL) SOLID STATE DRIVE (SSD) RETRY MECHANISM TO IMPROVE HOST CPU UTILIZATION AND REDUCE CPU STALL SITUATION,” the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] One or more aspects of embodiments in accordance with the present disclosure relate to storage devices, and more specifically, to processing data store and data load commands. BACKGROUND

[0004] Applications can interact with storage or memory devices (collectively referred to as storage devices) for reading (or loading) and writing (or storing) data. Latency is typically involved in accessing storage devices. The type of latency involved can depend on the storage medium included in the storage device. Certain storage media have lower latency than other storage media.

[0005] The above information disclosed in this Background section is only for enhancing the understanding of the background of the present disclosure, and therefore, it can contain information that does not form the prior art. SUMMARY

[0006] One or more embodiments of the present disclosure relate to a storage device including a first storage medium, a second storage medium, and a processor. The processor can be configured to receive, from a computing device, a first command associated with first data, search the first storage medium for processing the first command, and send, based on the search of the first storage medium, a message to the computing device. Based on the sending of the message, the storage device can be configured to receive, from the computing device, a second command associated with the first data.

[0007] In some embodiments, the first command includes a command to load the first data. In some embodiments, based on the search of the first storage medium, the processor is further configured to retrieve the first data from the second storage medium and store the first data in the first storage medium.

[0008] In some embodiments, the processor is further configured to, based on the receiving of the second command, retrieve the first data from the first storage medium and send the first data to the computing device.

[0009] In some embodiments, the first command comprises a command to store first data. In some embodiments, based on the search of the first storage medium, the processor is further configured to: identify second data in the second storage medium that is associated with an address of the first data; write the second data to the first storage medium; and update the second data with the first data.

[0010] In some embodiments, the message comprises a criterion. The processor can be further configured to receive the second command based on the computing device detecting satisfaction of the criterion.

[0011] In some embodiments, the processor is further configured to: determine an estimated latency of the storage device; and determine the criterion based on the estimated latency.

[0012] In some embodiments, the message comprises a flag. The processor can be further configured to receive the second command based on the computing device detecting the flag.

[0013] In some embodiments, the second command comprises a command to load or store first data.

[0014] In some embodiments, the second command comprises a command to receive a status of a write of first data.

[0015] In some embodiments, the first storage medium comprises volatile memory and the second storage medium comprises non-volatile memory.

[0016] One or more embodiments of the present disclosure relate to a method comprising: receiving, by a storage device, a first command associated with first data; searching, by the storage device, a first storage medium of the storage device for processing the first command; and based on searching the first storage medium, sending a message to a computing device, wherein based on the message, the computing device is configured to send a second command associated with the first data.

[0017] These and other features, aspects, and advantages of embodiments of the present disclosure will become better understood when the following detailed description is considered in conjunction with the accompanying drawings. The intended scope of the application is evidenced by the attached claims. BRIEF DESCRIPTION OF DRAWINGS

[0018] Non-limiting and non-exhaustive embodiments of the present embodiments are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views. The drawings are in simplified form and are not to precise scale.

[0019] Figure 1 A block diagram of a system for processing data commands is depicted in accordance with one or more embodiments;

[0020] Figure 2 depicted is a block diagram of a host coupled to a storage controller over one or more data communication links, in accordance with one or more embodiments;

[0021] Figure 3 depicted is a conceptual diagram of a retry command generated by a response engine, in accordance with one or more embodiments;

[0022] Figure 4 depicted is a flow diagram of a process for data command processing by a storage device, in accordance with one or more embodiments;

[0023] Figure 5 depicted is another flow diagram of a process for data command processing by a storage device, in accordance with one or more embodiments;

[0024] Figure 6 depicted is a block diagram of a process for identifying a retry latency, in accordance with one or more embodiments; and

[0025] Figure 7 depicted is a flow diagram of a process for sending commands to and receiving commands from a storage device, in accordance with one or more embodiments. DETAILED DESCRIPTION

[0026] Hereinafter, example embodiments will be described in greater detail by referring to the accompanying drawings, in which like numerals refer to like elements throughout. The present disclosure may, however, be embodied in various forms and should not be construed as being limited to only the embodiments set forth herein. Rather, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects and features of the present disclosure to those skilled in the art, and therefore, processes, elements and techniques that are not necessary to understand the present disclosure are not described in detail. Unless otherwise specified, like numerals in the drawings and written description denote like elements, and as such, the description of the same can not be repeated. In addition, in the drawings, relative sizes of elements, layers, and regions can be exaggerated and / or simplified for clarity.

[0027] Embodiments of the present disclosure are described below with reference to block diagrams and flowcharts. As such, it will be understood that each block of the block diagrams and flowcharts, and combinations of blocks in the block diagrams and flowcharts, can be implemented in a computer program product, a fully functioning hardware embodiment, a combination of hardware and computer program products, and / or an apparatus, system, computing device, computing entity, etc., that performs instructions, operations, steps, and like used interchangeably with the terms executable instructions, instructions for execution, program code, etc., on a computer readable storage medium. For example, retrieval, loading and execution of the code can be sequential in some examples, where a single instruction is retrieved, loaded, and executed at a time. In some example embodiments, retrieval, loading and / or execution can be in parallel, where multiple instructions are retrieved, loaded, and / or executed together. Thus, such embodiments can produce specially configured machines that perform the specified operations, steps, or functions of the block diagrams and flowcharts. As such, the block diagrams and flowcharts support combinations of embodiments for performing the specified instructions, operations, or steps.

[0028] Furthermore, features of embodiments of the present disclosure can be combined, partially or entirely, with one or more other features, and can operate in various ways, and embodiments can be implemented independently of, or in combination with, one or more other embodiments.

[0029] Generally, an application on a host computing device (referred to as a “host”) can need to store and load data while the application is being executed. If the data to be loaded exists in the host’s cache memory or main memory (collectively referred to as host memory), then there can be no need to access a secondary storage device that can store the data. Data can be retrieved from the host memory with low latency.

[0030] If the data does not exist in the host memory (e.g., a cache miss), then the data can be retrieved from a storage device and / or memory expansion device (e.g., a CXL.mem / CXL.cache device) (collectively referred to as a “storage device”). The latency involved in accessing a storage device can vary depending on the storage medium in which the data is stored. For example, a storage device can have both fast storage media (e.g., dynamic random access memory (DRAM)) and slow storage media (e.g., NAND flash). The latency of the fast storage media can be lower than the latency of the slow storage media.

[0031] The latency experienced by a storage device in processing a data command from a host, such as a data load or store command, can stall the execution of a central processing unit (CPU) core of the host. For example, the host CPU can need to wait for the data requested in a load command before continuing to process another instruction. The speed at which the data is returned can depend on the latency of the storage device in which the data is stored.

[0032] Hosts can employ an out-of-order buffer to improve CPU stalls. The out-of-order buffer can store instructions to be processed by the CPU. If the CPU detects a stall in one instruction, the CPU can move to process another instruction in the buffer. However, even with a large out-of-order buffer, the buffer can become saturated with long-latency instructions, which can cause the CPU to experience stalls, resulting in decreased performance of the host.

[0033] Generally, embodiments of the present invention relate to systems and methods for processing data commands or requests (e.g., load and store commands) from a host. In some embodiments, the storage device includes a response engine configured to determine whether the command results in a cache hit or a cache miss. A cache hit can be determined if a memory location of the data to be loaded or stored is found in a fast storage medium (e.g., DRAM) of the storage device. A cache miss can be determined if the memory location is not found in the fast storage medium.

[0034] If the data command results in a cache hit, the storage device can retrieve the data from or store the data to the fast storage medium and return an appropriate response to the requesting host. For example, for a load command, the requested data can be returned to the host. For a store command, the data to be stored can be written to the fast storage medium.

[0035] If the data command results in a cache miss, the storage device can need to access a slow storage medium (e.g., NAND) to load or store the requested data. In one embodiment, the response engine returns a retry message while the storage device continues to take steps to load or store the requested data. The retry command can be used to prompt the host to retry the data command after a retry period. The retry period can be provided by the response engine based on an estimated latency of the storage device. Because the retry message can be returned (e.g., returned quickly) before the access to the slow storage medium has completed, the latency of the storage device in the case of a cache miss can be similar to the latency of a cache hit (e.g., the same or lower than the device DRAM latency). This allows the latency of the response of the storage device to be substantially the same regardless of whether the data command results in a cache hit or a cache miss.

[0036] In some embodiments, the host CPU receives the retry message and removes or flushes the instruction associated with the message from its instruction buffer, which helps to reduce CPU stalls that can be encountered as the CPU waits for the data to be retrieved from the storage medium. This can help to avoid a decrease in the performance of the host CPU due to such stalls.

[0037] In some embodiments, the host CPU waits for the amount of time indicated in the retry message and then resends the data command. A retry period can be set to give the storage device sufficient time to fetch data from the slow storage medium to the fast storage medium, so that the retried data command can be completed from the fast storage medium. In this way, the latency of processing the retried data command can be the latency of retrieving or writing data to the fast storage medium. Because the retry period is calculated and provided by the storage device based on predicted storage device latency, the CPU's sending of retry commands is executed more efficiently and helps improve CPU core utilization.

[0038] Figure 1 A block diagram of a system for processing load commands and store commands (collectively, data commands) according to one or more embodiments is depicted. The system may include a host computing device (“host”) 100 coupled to an attached storage device 102 via one or more data communication links 104. In some embodiments, the data communication link 104 may include various general-purpose interfaces, such as Ethernet, Universal Serial Bus (USB), and / or any wired or wireless data communication link.

[0039] Host 100 may include processor 106, main memory 108 (e.g., which may be referred to as main memory), and host interface controller 110. Processor 106 may include one or more central processing unit (CPU) cores 112 configured to run one or more applications 114 based on computer program instructions stored in main memory 108. Main memory 108 may include volatile memory (e.g., random access memory (RAM)) and / or non-volatile memory (e.g., read-only memory (ROM)). For example, main memory 108 may include dynamic random access memory (DRAM) for storing computer program instructions and / or data generated by storage device 102.

[0040] Application 114 can be any application configured to send commands (e.g., load and store commands) to storage device 102. For example, application 114 could be a big data analytics application, an e-commerce application, a database application, a machine learning application, etc. The results of the data commands can be used by application 114 to generate output.

[0041] In some embodiments, data commands are processed by load / store unit 116 during the execution of application 114. In some embodiments, data commands or instructions associated with data commands (collectively referred to as data commands) are placed in command buffer 117. Command buffer 117 may include CPU out-of-order buffer, load queue, store queue, etc. Because command buffer 117 may have a set depth, command buffer 117 may be at risk of saturation (e.g., becoming full) if there is a delay in removing or clearing existing commands from buffer 117, and this may prevent further commands from being placed in the buffer for processing by load / store unit 116.

[0042] In some implementations, load / store unit 116 interfaces with cache memory 118 (also simply referred to as "memory," "host cache memory," or "cache") to process data commands. Cache memory 118 may be dedicated to one CPU core in CPU core 112, or it may be shared by various CPU cores within the CPU cores.

[0043] Cache memory 118 may include, for example, a Level 1 (L1) cache coupled to a Level 2 (L2) cache, which in turn is coupled to a Level 3 (L3) cache. The L3 cache may in turn be coupled to main memory 108. In some embodiments, one or more of the L1, L2, or L3 caches may be included as part of load / store unit 116 and / or main memory 108.

[0044] To enable application 114 to use data generated by storage device or memory expander 102, data can be loaded into cache memory 118, and application 114 can consume data from cache memory 118. If the data to be consumed is not already in cache memory 118, load / store unit 116 can query other memory devices in the memory hierarchy to find the data. For example, if the data being sought is not in L1 cache, load / store unit 116 can query L2 cache, and if not in L2 cache, query L3 cache, and if not in L3 cache, query main memory 108. If the data is not in main memory 108, load / store unit 116 can request the data from storage device or memory expander 102 via host interface controller 110.

[0045] The host interface controller 110 may include physical connections and software instructions executable by the processor 106. In some embodiments, the host interface controller 110 allows the host 100 and the storage device 102 to send and receive data using protocols such as CXL, but the embodiments are not limited thereto.

[0046] In addition to or in place of CXL, the host interface controller 110 can use other protocols, such as Cache Coherent Interconnect for Accelerators (CCIX), Dual In-line Memory Module (DIMM) interface, Small Computer System Interface (SCSI), Non-Volatile Memory Fast (NVMe), Peripheral Component Interconnect Fast (PCIe), Remote Direct Memory Access over Ethernet (RDMA), Serial Advanced Technology Attachment (SATA), Fibre Channel, Serial Attached SCSI (SAS), Architecture-based NVMe (NVMe-oF), iWARP protocol, InfiniBand protocol, 5G wireless protocol, Wi-Fi protocol, Bluetooth protocol, etc.

[0047] Load / store unit 116 can generate a data command in response to the execution of instructions from application 114 that uses data. A cache miss may occur if the data is not available in cache memory 118 or main memory 108. In this case, load / store unit 116 can request the data from storage device 102.

[0048] Storage device 102 may take the form of a solid-state drive (SSD), persistent storage, etc. In some embodiments, storage device 102 includes (or is embodied as) an SSD with cache coherency and / or computing power.

[0049] In some embodiments, storage device 102 includes a storage controller 120, a fast storage medium 122, and a slow storage medium 124 (e.g., non-volatile memory (NVM), such as NAND flash memory). In some embodiments, storage device 102 is configured to present memory space accessible to host 100 using memory load / store commands, and the size of the memory space may be based on the size of slow storage medium 124. In such embodiments, storage device 102 may be referred to as a "memory extender" or "memory expansion device" (e.g., because slow storage medium 124 is used to expand the size of the memory). Storage device 102 can fetch data from slow storage medium 124 to fast storage medium 122 to reduce data access latency.

[0050] The fast storage medium 122 may be a high-performance memory of the storage device 102 and may include (or may be) volatile memory, such as DRAM, but this disclosure is not limited thereto, and the fast storage medium 122 may be any suitable type of high-performance volatile or non-volatile memory. Although a single fast storage medium 122 is depicted for simplicity, those skilled in the art will recognize that the storage device 102 may include other local memory for temporarily storing other data of the storage device.

[0051] In some embodiments, the fast storage medium 122 is used and managed as a cache memory. In this regard, the fast storage medium (also referred to as a cache) 122 may store copies of data stored in the slow storage medium 124. For example, data requested by the load / store unit 116 may be copied from the slow storage medium 124 to the fast storage medium 122 (if not already there) to allow data retrieval from the fast storage medium 122 instead of the slow storage medium 124. In some embodiments, the fast storage medium 122 has a lower access latency than the slow storage medium 124.

[0052] Slow storage medium 124 can persistently store, for example, data received from host 100. Slow storage medium 124 may include, for example, NAND flash memory, but this disclosure is not limited thereto, and slow storage medium 124 may include any suitable type of memory (e.g., disk, magnetic tape, optical disk, etc.) for storing data according to the implementation of storage device 102.

[0053] Storage controller 120 can be connected to slow storage medium 124 and fast storage medium 122 via one or more storage interfaces 126a, 126b. Storage controller 120 can receive data commands from host 100 and send commands to slow storage medium 124 and / or fast storage medium 122 to implement those commands. In this regard, storage controller 120 may include at least one processing component embedded thereon for interfacing with host 100, fast storage medium 122, and slow storage medium 124. The processing component may include, for example, digital circuitry (e.g., microcontroller, microprocessor, digital signal processor, or logic device (e.g., field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), etc.)) capable of executing data access instructions (e.g., via firmware and / or software) to provide access to data stored in fast storage medium 122 or slow storage medium 124 according to the data access instructions.

[0054] In some embodiments, the storage controller 120 receives a data command from the host 100 and checks whether the address of the data is in the fast storage medium 122. In some embodiments, if the address is not in the fast storage medium 122, the storage controller 120 sends a retry message to the host 100.

[0055] In some embodiments, host 100 includes a retry engine 119 configured to execute instructions stored in memory 108 for processing retry messages. Retry engine 119 may signal to load / store unit 116 to remove or clear the initial data command that caused the retry message from command buffer 117. In some embodiments, load / store unit 116 tracks initial data commands (e.g., in a separate buffer) and updates the status of the initial commands from "pending" to "presubmit".

[0056] In some embodiments, the retry engine 119 identifies a retry delay in a received retry message and sets a retry time (e.g., a retry timer) based on the retry delay to signal the load / store unit 116 to send a second command. In some embodiments, the second command is a data command targeting the same data as the initial data command (e.g., an initial load command). In some embodiments, the signal identifies the initial command held by the load / store unit 116 in a "pre-commit" state and updates the state of that command to "pending" when the second command is sent. In some embodiments, the second command is a different command from the initial command. For example, if the initial command is a store command, the second command is a command for status confirmation (e.g., the state of the initial store command).

[0057] Figure 2 A block diagram is depicting a host 100 coupled to a storage controller 120 via one or more data communication links 104 according to one or more embodiments. In some embodiments, the storage controller 120 includes a device interface controller 202 configured to receive commands from the host 100 (e.g., a host interface controller 110). In this regard, the device interface controller 202 may include physical connections and software instructions for sending and receiving data from the host 100 using protocols such as CXL, but embodiments are not limited thereto.

[0058] In some embodiments, the storage controller 120 includes a device metadata unit 204 configured to insert metadata into commands sent by the storage controller or to read metadata from commands received by the storage controller. For example, metadata to be inserted by the device metadata unit 204 may include retry latency values ​​for retry commands generated by the storage device 102. Metadata to be read by the device metadata unit 204 may include command codes (e.g., MemWrFwd, MemInv, and / or other op codes) requesting status acknowledgments from the storage device 102.

[0059] In some embodiments, the device interface controller 202 receives a data command to load data from / store data to a specified memory address. The command may be received by a cache controller 210. The cache controller 210 may be configured to determine whether the requested memory address is found in the fast storage medium 122 (e.g., a cache hit), and, depending on the determination, to issue an appropriate request to a first controller 216 or a second controller 218. The cache controller 210 may also be configured to manage the use of the fast storage medium 122 according to a cache management algorithm.

[0060] In some embodiments, the memory address specified in the data command is retrieved from the fast storage medium 122 or the slow storage medium 124 via the first controller 216 or the second controller 218, respectively, and returned for storage in the host cache memory 118. In some embodiments, the memory address specified in the data command is translated into the memory address of the fast storage medium 122 or the slow storage medium 124 via a memory mapping table.

[0061] In some embodiments, if the requested memory address is found in the fast storage medium 122, data is retrieved from that address via the first controller 216 (if the command is a load command) and returned to the host 100 for storage in the host cache memory 118. In some embodiments, a response packet may be returned along with the retrieved data. The response packet may include an indicia indicating that the requested data was returned in response to the load command. This indicia may be added to the metadata field of the response by the device metadata unit 204. The response may conform to, for example, the CXL.mem protocol, but embodiments are not limited thereto.

[0062] In some embodiments, if the requested memory address is in slow storage medium 124 instead of in host cache memory 118 (e.g., a cache miss), cache controller 210 returns a retry message (or retry command) to host 100. Cache controller 210 may include a response engine 212 that generates the retry command. The retry command may conform to, for example, the CXL.mem protocol, but embodiments are not limited thereto.

[0063] In some embodiments, the retry command includes a retry flag and a retry latency (e.g., a criterion). In some embodiments, the retry latency is set based on a latency estimate calculated by the queue monitoring engine 214. The queue monitoring engine 214 can estimate the latency for the storage controller 120 to process the current command based on the number of requests in one or more queues of the storage device 102 and the estimated average execution time for processing the current request. The estimated average execution time can be based on one or more historical execution times. The range or category of the retry latency can be selected based on the estimated latency. The range of the retry latency can include, for example, 10-100 microseconds, 100-200 microseconds, etc.

[0064] In some embodiments, no additional data is returned to host 100 along with the retry command. For example, if the data command received from the host is a load command, the data requested by the host is not returned with the retry command. In another example, if the data command received from the host is a write command, no write acknowledgment is returned with the retry command. Although no additional data is returned with the retry command, the storage controller 120 continues the process of reading from the slow storage medium 124 to the fast storage medium 122, or writing data to the slow storage medium (e.g., when data is to be evicted from the fast storage medium 122).

[0065] In some embodiments, host 100 (e.g., processor 106) includes a host metadata unit 220 configured to receive commands from device interface controller 202 and process metadata inserted in the commands. The commands may conform to the CXL.mem protocol, but embodiments are not limited thereto. For example, the metadata of a retry command transmitted by device interface controller 202 may include a retry flag and a retry delay. Host metadata unit 220 may be configured to identify a retry command based on the retry flag being set and forward the retry command to retry engine 119 to take corresponding retry actions.

[0066] In some embodiments, the retry engine 119 is configured to communicate with the load / store unit 116 to remove or clear data commands associated with received retry commands from the command buffer 117. In this way, CPU core 112 pauses due to the CPU core waiting for data commands to complete processing can be reduced or avoided, and the CPU core can turn to processing other commands in the command buffer 117.

[0067] The retry engine 119 can schedule the sending of the second command based on a retry delay (e.g., a criterion) in the received retry command. In some embodiments, the retry engine 119 is configured to detect the satisfaction of the criterion and send the second command based on the detection of criterion satisfaction. In this regard, the retry engine 119 can wait for a specified amount of time within a specified retry delay and send a signal to the load / store unit 116 to send the second command. The expiration of the time can be considered as the satisfaction of the criterion. The second command can be the same as or similar to the first command. For example, the second command can be a request to load the same data as the first command. If the first command is a store command, the second command can be a request for confirmation of the status of the stored procedure.

[0068] Although the hypothesis Figure 2 One or more components are individual components, but those skilled in the art will recognize that, without departing from the spirit and scope of the inventive concept, the functions of the components can be combined or integrated into a single component, or further subdivided into more sub-components.

[0069] Figure 3 A conceptual diagram of a retry command 300 generated by response engine 212 according to one or more embodiments is depicted. The retry command 300 may include a retry flag 302 and a retry delay identifier (ID) 304. The retry flag 302 may include a single bit that can be set or unset depending on whether a retry is requested. The retry delay ID 304 may include three bits for identifying a retry category or period. Retry periods may include less than 1 microsecond (retry delay ID 000), between 1 and 10 microseconds (retry delay ID 001), etc. Retry periods may be identified based on an estimated delay for the storage controller 120 to process the request, where the estimated delay is provided by queue monitoring engine 214. In some embodiments, a longer estimated delay results in a longer retry delay.

[0070] In some embodiments, a retry command 300 is generated according to the CXL.mem protocol and sent to host 100. At this point, the retry command 300 is sent via the subordinate-to-master (S2M) non-data response (NDR) channel provided by the CXL.mem protocol. In some embodiments, the retry command 300 is included in a reserved field of the S2M NDR message.

[0071] Figure 4A flowchart illustrating a process for processing data commands performed by a storage device according to one or more embodiments is provided. The process begins, and in action 400, the storage controller 120 receives a first request or command associated with first data from a computing device (e.g., host 100). This command may be a load or store command for data stored in a memory location.

[0072] In action 402, the storage controller 120 searches for a first storage medium (e.g., fast storage medium 122) for processing the first command. In some embodiments, the cache controller 210 determines whether the fast storage medium 122 includes a memory location contained in the first command.

[0073] Based on the search of the first storage medium, in action 404, the storage controller 120 sends a message to the computing device (e.g., host 100). This message may be, for example, a retry message. Based on this message, the computing device may send a second command associated with the first data to the storage device.

[0074] Figure 5 Another flowchart depicts a process for data command processing by a storage device according to one or more embodiments. The process begins, and in action 500, the storage controller 120 identifies a load or store request or load or store command from an application 114 running on the host 100. The load or store request may be associated with a memory address.

[0075] Data commands can be processed by device metadata unit 204 to determine the type of command. In some embodiments, a load command or store command is sent to cache controller 210 to determine in action 502 whether the command results in a cache hit. In some embodiments, a cache hit can be determined when the requested memory address is found in fast storage medium 122.

[0076] If the answer is yes, then in action 504 the data command is processed from the fast storage medium 122. At this point, if the data command is a load command, the requested data is retrieved from the fast storage medium 122 and sent to the host 100 as a response to the load command. If the data command is a store command, the data to be written is retrieved from the host 100's memory 108 and written to the fast storage medium 122. In some cases, it may be necessary to evict data from the fast storage medium 122 to make room for new data to be stored.

[0077] Referring again to action 502, if the command does not result in a cache hit, the response engine 212 generates and sends a retry message. In some embodiments, the retry latency is determined based on the latency for processing pending commands estimated by the queue monitoring engine 214.

[0078] In action 508, the storage controller 120 continues processing the data command, retrieves the data needed to complete the data command from the slow storage medium 124, and stores the data in the fast storage medium 122. In this way, when the host 100 resends the data command after the retry period has expired, the requested memory location is expected to be present in the fast storage medium 122.

[0079] Figure 6 A block diagram depicts a process for identifying retry latency according to one or more embodiments. In some embodiments, queue monitoring engine 214 calculates a movement or runtime latency value for storage device 102 based on latency experienced while processing one or more previous requests, latency experienced while processing the current request, and pending requests in one or more queues of the storage device. The latency of a request can be determined based on the execution time of the request. For example, the execution time can be the time spent by storage device 102 retrieving data from slow storage medium 124, referred to as round-trip time.

[0080] The process begins, and in action 600, the queue monitoring engine 214 identifies a request. This request could be, for example, a request to load or store data, identified by the storage controller 120.

[0081] In action 602, the queue monitoring engine 214 updates the average execution time based on the request. In some embodiments, the execution time is based on the round-trip counter for the request. The average execution time or round-trip time t can be calculated based on the following formula. n :

[0082] t n =(1-a)(t n-1 )+a(1 / fc * c i )

[0083] Where a is a coefficient or weight value between 0 and 1, t n-1 It is the history of previous requests (n-1) or the previous execution time, c i This is a counter for the number of requested round-trip clock cycles, and fc is the clock frequency. Although the above formula considers the immediately preceding execution time t... n-1 The average execution time can be calculated using this method, but those skilled in the art should recognize that two or more other historical execution times (e.g., t) can be used. n-2 t n-3 (etc.) to calculate the average execution time.

[0084] counter c iIt can start at 0 and increase each clock cycle until the request has been satisfied (e.g., when the requested data has been retrieved from slow storage medium 124). A coefficient 'a' can be set to give more or less weight to the current execution time relative to previous execution times when calculating the average execution time. For example, if a=0, the current execution time is ignored, and historical execution times are used to calculate the average execution time. If a=1, historical execution times are ignored, and the current execution time is used to calculate the average execution time.

[0085] In action 604, it is determined whether a retry command should be generated. If the answer is no, the queue monitoring engine 214 continues to update the runtime average latency based on the requests processed by the storage controller 120.

[0086] If the answer is yes, then queue monitoring engine 214 calculates the estimated latency in action 606 based on the current average execution time. In some embodiments, the estimated latency lat is calculated based on the following formula:

[0087] lat = q c *t n

[0088] Where q c This refers to the number of commands in one or more queues of storage device 102. At this point, storage device 102 may buffer load or store commands from host 100 in one or more queues for processing by storage controller 120. The one or more queues may include a read request queue, a write request queue, a NAND read queue, a NAND write queue, etc. The NAND queues may be maintained separately by the second controller 218 for requests to read data from slow storage medium 124 and requests to store data on slow storage medium 124 (e.g., in the event of a cache miss).

[0089] In action 608, the queue monitoring engine 214 (or response engine 212) can select a retry delay value based on the calculated latency. In this regard, the retry delay category or range into which the latency value falls can be identified, and the value of the identified category or range can be set as the retry delay. Selecting the retry delay based on the expected latency of the storage device allows the host 100 to efficiently schedule retries of initially sent commands based on latency feedback.

[0090] Figure 7A flowchart illustrating a process for sending and receiving commands to and from storage device 102 according to one or more embodiments is provided. The process begins, and in action 700, host 100 sends a first data command. In some embodiments, application 114 running on processor 106 may place the first data command (or a request associated with the first data command) in command buffer 117, and load / store unit 116 may retrieve the first data command and send it to storage device 102.

[0091] In action 702, processor 106 receives a retry command in response to a first data command that caused a cache miss, resulting in increased latency for a portion of storage device 102 when the command is completed. In some embodiments, the retry command is included in a reserved field of an S2M NDR message. This message may identify the first command to which the retry command pertains.

[0092] In some embodiments, the host metadata unit 220 receives a message from the storage device 102 and determines that a retry flag has been set. The host metadata unit 220 may send a signal to the retry engine 119 based on the retry flag being set, the identifier of the first data command included in the message, and the retry delay.

[0093] In action 704, the retry engine 119 sends a signaling instruction to the load / store unit 116 to remove the first data command from the command buffer 117 based on the retry command. Removing the first data command, which is expected to have increased latency, allows the release of resources occupied by such long-latency commands—which could otherwise saturate the command buffer and prevent further commands from being processed—thereby improving processor core utilization.

[0094] In action 706, retry engine 119 checks the retry delay to determine if it is time to retry the initial command. For example, retry engine 119 may wait until the retry delay period expires (e.g., the minimum or maximum retry time of the retry delay range) before attempting to retry the command.

[0095] If it is time for a retry, the retry engine 119 sends a second command to the storage device 102 in action 708. In some embodiments, the second request is the same as the initial request. For example, if the initial command is a load command for data stored in a memory location, the second command is also a load command for the same data. If the initial command is a store command, the second command may be a command for status confirmation of the initial store command. The second command may be sent via a master-to-slave (M2S) channel that sets the command code of the message to a command code requesting status confirmation from the storage device 102 (e.g., MemWrFwd, MemInv, and / or other op codes).

[0096] In action 710, host 100 receives a response to the second command. For example, for a retry load command, host 100 receives the requested data from the volatile memory (or fast storage medium (122)) of storage device 102. For a second command requesting status confirmation of an initial storage command, the response could be confirmation that the initial data has been stored. Waiting for the retry period and sending the second command can reduce the latency from NAND read latency to device memory latency.

[0097] Load / store unit 116, retry engine 119, device metadata unit 204, response engine 212, queue monitoring engine 214, and host metadata unit 220 can be implemented using software, firmware, hardware, or a combination of software, firmware, or hardware. For example, one or more of the engines or units can be implemented via a processing component (e.g., a microcontroller, microprocessor, digital signal processor, or logic device (e.g., a field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), etc.)) capable of executing instructions (e.g., via firmware and / or software) to implement the described functions.

[0098] One or more embodiments of this disclosure can be implemented in one or more processors. The term processor can refer to one or more processors and / or one or more processing cores. One or more processors can be housed in a single device or distributed across multiple devices (e.g., on a cloud system). Processors can include, for example, application-specific integrated circuits (ASICs), general-purpose or special-purpose central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), and programmable logic devices (such as field-programmable gate arrays (FPGAs)). In a processor, as used herein, each function is performed by hardware configured (i.e., hardwired) to perform that function, or by more general-purpose hardware (such as a CPU) configured to execute instructions stored in a non-transitory storage medium (e.g., memory). Processors can be fabricated on a single printed circuit board (PCB) or distributed across several interconnected PCBs. Processors can include other processing circuitry; for example, processing circuitry can include two processing circuits, an FPGA and a CPU, interconnected on a PCB.

[0099] It should be understood that although the terms "first," "second," "third," etc., may be used herein to describe various elements, components, regions, layers, and / or portions, these elements, components, regions, layers, and / or portions should not be limited by these terms. These terms are used only to distinguish one element, component, region, layer, or portion from another element, component, region, layer, or portion. Therefore, without departing from the spirit and scope of the inventive concept, the first element, component, region, layer, or portion discussed herein may be referred to as the second element, component, region, layer, or portion.

[0100] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the inventive concept. Furthermore, unless explicitly stated otherwise, the embodiments described herein are not mutually exclusive. Aspects of the embodiments described herein can be combined in several implementations.

[0101] As used herein, the terms “substantially,” “about,” and similar terms are used as approximate terms rather than terms of degree and are intended to explain the inherent biases of measurements or calculations that will be recognized by one of ordinary skill in the art.

[0102] As used herein, the singular forms “a” and “an” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of…” modify the entire list of elements when following a list of elements and do not modify individual elements in the list. Furthermore, the use of “may” when describing embodiments of the inventive concept means “one or more embodiments of this disclosure.” Additionally, the term “exemplary” is intended to refer to an example or illustration. As used herein, the terms “use,” “using,” and “used” can be considered synonymous with the terms “utilize,” “utilizing,” and “utilized,” respectively.

[0103] Although exemplary embodiments of systems and methods for data command processing have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. Therefore, it should be understood that systems and methods for data command processing constructed in accordance with the principles of this disclosure may be implemented in ways other than those specifically described herein. This disclosure is also defined in the appended claims and their equivalents.

[0104] The system and method for data command processing may include one or more combinations of the features set forth in the following statements.

[0105] Statement 1. A storage device comprising: a first storage medium; a second storage medium; and a processor configured to: receive a first command associated with first data from a computing device; search the first storage medium for processing the first command; and send a message to the computing device based on the search of the first storage medium, wherein based on the sending of the message, the storage device is configured to receive a second command associated with the first data from the computing device.

[0106] Statement 2. The storage device of claim 1, wherein the first command includes a command to load the first data, wherein, based on a search of the first storage medium, the processor is further configured to retrieve the first data from the second storage medium and store the first data in the first storage medium.

[0107] Statement 3. The storage device of claim 2, wherein the processor is further configured to: retrieve the first data from the first storage medium based on receiving the second command, and send the first data to the computing device.

[0108] Statement 4. The storage device of claim 1, wherein the first command includes a command to store the first data, wherein, based on a search of the first storage medium, the processor is further configured to: identify second data in the second storage medium associated with an address of the first data; write the second data to the first storage medium; and update the second data with the first data.

[0109] Statement 5. The storage device of claim 1, wherein the message includes a standard, and wherein the processor is further configured to receive the second command based on the computing device detecting that the standard is satisfied.

[0110] Statement 6. The storage device of claim 5, wherein the processor is further configured to: determine an estimated latency of the storage device; and determine the criterion based on the estimated latency.

[0111] Statement 7. The storage device of claim 1, wherein the message includes a flag, and wherein the processor is further configured to receive the second command based on the computing device detecting the flag.

[0112] Statement 8. The storage device of claim 1, wherein the second command includes a command to load or store the first data.

[0113] Statement 9. The storage device of claim 1, wherein the second command includes a command for receiving a state of writing the first data.

[0114] Statement 10. The storage device of claim 1, wherein the first storage medium comprises volatile memory, and the second storage medium comprises non-volatile memory.

[0115] Statement 11. A method includes: receiving a first command associated with first data by a storage device; searching a first storage medium of the storage device for processing the first command by the storage device; and sending a message to a computing device based on the search of the first storage medium, wherein, based on the message, the computing device is configured to send a second command associated with the first data.

[0116] Statement 12. The method of claim 11, wherein the first command includes a command to load the first data, and the method further includes: retrieving the first data from a second storage medium of the storage device based on searching the first storage medium and storing the first data in the first storage medium.

[0117] Statement 13. The method of claim 12, further comprising: receiving the second command by the storage device; and retrieving the first data from the first storage medium and sending the first data to the computing device.

[0118] Statement 14. The method of claim 11, wherein the first command includes a command to store the first data, the method further comprising: based on the search of the first storage medium: identifying second data in a second storage medium of the storage device associated with an address of the first data; writing the second data to the first storage medium; and updating the second data with the first data.

[0119] Statement 15. The method of claim 11, wherein the message includes a standard, and the method further includes: detecting by the computing device that the standard is satisfied; and sending the second command by the computing device based on the detection that the standard is satisfied.

[0120] Statement 16. The method of claim 15, further comprising: determining an estimated latency of the storage device; and determining the criterion based on the estimated latency.

[0121] Statement 17. The method of claim 11, wherein the message includes a flag, and the method further includes: generating the second command based on detecting the flag.

[0122] Statement 18. The method of claim 11, wherein the second command includes a command to load or store the first data.

[0123] Statement 19. The method of claim 11, wherein the second command includes a command for receiving a status of writing the first data.

[0124] Statement 20. The method of claim 11, wherein the first storage medium comprises volatile memory, and the second storage medium comprises non-volatile memory.

Claims

1. A storage device comprising: a first storage medium; a second storage medium; and a processor configured to: receive, from a computing device, a first command associated with first data; search the first storage medium for processing the first command; and based on the search of the first storage medium, send a message to the computing device, wherein based on the sending of the message, the storage device is configured to receive, from the computing device, a second command associated with the first data. the first command comprises a command to load the first data, wherein based on the search of the first storage medium, the processor is further configured to retrieve the first data from the second storage medium and store the first data in the first storage medium. the processor is further configured to:

2. The storage device of claim 1, wherein, based on receiving the second command, retrieve the first data from the first storage medium and send the first data to the computing device.

3. The storage device of claim 2, wherein, the first command comprises a command to store the first data, wherein based on the search of the first storage medium, the processor is further configured to: identify second data associated with an address of the first data in the second storage medium; 4. The storage device of claim 1, wherein, write the second data to the first storage medium; and update the second data with the first data.

5. The storage device of claim 1, wherein the message comprises a criterion, wherein the processor is further configured to receive the second command based on the computing device detecting satisfaction of the criterion. the processor is further configured to: determine an estimated latency of the storage device; and 6. The storage device of claim 5, wherein, determine the criterion based on the estimated latency. the message comprises a flag, wherein the processor is further configured to receive the second command based on the computing device detecting the flag. the second command comprises a command to load or store the first data.

7. The storage device of claim 1, wherein, the second command comprises a command to receive a status of a write to the first data.

8. The storage device of claim 1, wherein, the first storage medium comprises a volatile memory and the second storage medium comprises a non-volatile memory.

9. The storage device of claim 1, wherein, 11. A method comprising:

10. The storage device of claim 1, wherein, receiving, by a storage device, a first command associated with first data; searching, by the storage device, a first storage medium of the storage device for processing the first command; and based on searching the first storage medium, sending a message to a computing device, wherein based on the message, the computing device is configured to send a second command associated with the first data. the first command comprises a command to load the first data, the method further comprising: based on searching the first storage medium, retrieving the first data from a second storage medium of the storage device and storing the first data in the first storage medium.

13. The method of claim 12, further comprising:

12. The method of claim 11, wherein, receiving, by the storage device, the second command; and retrieving the first data from the first storage medium and sending the first data to the computing device. the first command comprises a command to store the first data, the method further comprising: based on searching the first storage medium: ​ ​ 14. The method of claim 11, wherein, ​ ​ identifying second data associated with an address of the first data in a second storage medium of the storage device; writing the second data to the first storage medium; and updating the second data with the first data.

15. The method of claim 11, wherein, the message includes a criterion, the method further comprising: detecting, by the computing device, satisfaction of the criterion; and sending, by the computing device, the second command based on detecting satisfaction of the criterion.

16. The method of claim 15, further comprising: determining an estimated latency of the storage device; and determining the criterion based on the estimated latency.

17. The method of claim 11, wherein, the message includes a flag, the method further comprising: generating the second command based on detecting the flag.

18. The method of claim 11, wherein, the second command includes a command to load or store the first data.

19. The method of claim 11, wherein, the second command includes a command to receive a status of writing the first data.

20. The method of claim 11, wherein, the first storage medium includes volatile memory and the second storage medium includes non-volatile memory.

Images