Real-time (JIT) scheduler for memory subsystems

Abstract

The invention relates to a real-time (JIT) scheduler for a memory subsystem. A memory subsystem of the present disclosure discloses a real-time JIT scheduling system and method. In one embodiment, a system receives a request to perform a memory operation using a hardware resource associated with a memory device. The system identifies a traffic category corresponding to the memory operation. The system determines a number of available quality of service (QoS) credits for the traffic category during a current scheduling time frame. The system determines a number of QoS credits associated with the type of memory operation. In response to determining that the number of QoS credits associated with the type of memory operation is less than the number of available QoS credits, the system submits the memory operation to be processed at a memory device.

Description

[0001] Information about divisional applications

[0002] This application is a divisional application of the Chinese invention patent application with application number 202111543519.0, application date December 16, 2021, and invention name “Real-time (JIT) scheduler for memory subsystem”. Technical Field

[0003] Embodiments of the present disclosure relate generally to a memory subsystem, and more particularly, to a just-in-time (JIT) scheduler for a memory subsystem. Background Art

[0004] The memory subsystem may include one or more memory devices that store data. The memory devices may be, for example, non-volatile memory devices and volatile memory devices. Generally speaking, the host system may utilize the memory subsystem to store data at the memory devices and retrieve data from the memory devices. Summary of the Invention

[0005] In one aspect, the present disclosure relates to a system comprising: a memory device; and a processing device operatively coupled to the memory device to perform operations comprising: receiving, by a just-in-time (JIT) scheduler, a request to perform a memory operation using hardware resources associated with the memory device; identifying a traffic class corresponding to the memory operation; determining a number of available quality of service (QoS) credits for the traffic class during a current scheduling time frame; determining a number of QoS credits associated with a type of the memory operation; and in response to determining that the number of QoS credits associated with the type of the memory operation is less than the number of available QoS credits, submitting the memory operation to the memory device.

[0006] In yet another aspect, the present disclosure relates to a method comprising: receiving, by a just-in-time (JIT) scheduler, a request to perform a memory operation using hardware resources associated with a memory device; identifying a business class corresponding to the memory operation; determining a type of the memory operation; determining, based on the business class and the type of the memory operation, whether the memory operation is to be processed during a current scheduling time frame; and submitting the memory operation to the memory device in response to determining that the memory operation is to be processed during the current scheduling time frame.

[0007] In another aspect, the present disclosure relates to a non-transitory machine-readable storage medium comprising instructions that, when accessed by a processing device, cause the processing device to perform one or more operations including: receiving, by a just-in-time (JIT) scheduler, a request to perform a memory operation using hardware resources of a memory device associated with the memory device; identifying a traffic class corresponding to the memory operation; determining a number of available quality of service (QoS) credits for the traffic class during a current scheduling time frame; determining a number of QoS credits associated with a type of the memory operation; and in response to determining that the number of QoS credits associated with the type of the memory operation is less than the number of available QoS credits, submitting the memory operation to the memory device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure will be more fully understood from the detailed description provided below and from the accompanying drawings of various embodiments of the present disclosure. However, the drawings should not be considered to limit the present disclosure to the specific embodiments, but are only for explanation and understanding.

[0009] Figure 1 An example computing system including a memory subsystem in accordance with some embodiments is described.

[0010] Figure 2 A block diagram illustrating a just-in-time (JIT) scheduling component according to one embodiment.

[0011] Figure 3 An example of a two-dimensional array of scheduling pools according to an embodiment is illustrated.

[0012] Figure 4 A block diagram illustrating a Quality of Service (QoS) traffic manager according to an embodiment.

[0013] Figure 5 A block diagram illustrating a memory resource manager according to an embodiment.

[0014] Figure 6 is a flowchart of an example method for scheduling memory operations according to an embodiment.

[0015] Figure 7 is a flowchart of an example method for scheduling memory operations according to an embodiment.

[0016] Figure 8 A block diagram illustrating a delay simulator according to one embodiment.

[0017] Figure 9 A latency profile according to an embodiment is described.

[0018] Figure 10 An example of a two-dimensional array of scheduling pools according to an embodiment is illustrated.

[0019] Figure 11 A block diagram illustrating a Quality of Service (QoS) traffic manager according to an embodiment.

[0020] Figure 12 is a flowchart of an example method for scheduling memory operations according to an embodiment.

[0021] Figure 13 is a flowchart of an example method for scheduling memory operations according to an embodiment.

[0022] Figure 14 is a block diagram of an example computer system in which embodiments of the present disclosure may operate. DETAILED DESCRIPTION

[0023] Various aspects of the present disclosure are directed to a real-time scheduler for a memory subsystem. The memory subsystem may be a storage device, a memory module, or a combination of a storage device and a memory module. Figure 1 Examples of storage devices and memory modules are described. Generally speaking, a host system can utilize a memory subsystem that includes one or more components, such as memory devices, that store data. The host system can provide data to be stored at the memory subsystem and can request data to be retrieved from the memory subsystem.

[0024] The memory controller of a memory subsystem can perform different types of memory access operations, such as host-initiated memory access operations and background (memory controller-initiated) memory access operations. Background operations can temporarily render the memory subsystem unavailable to the host system. Background operations can include media management operations, such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, and encryption operations. In other cases, prioritizing host-initiated memory access operations can starve background memory access operations. In some cases, multiple memory operations may attempt to access the same die during overlapping time periods, which can cause die contention. In other cases, multiple memory operations may direct their outputs to the same memory buffer, which can cause head-of-line blocking at the memory buffer. Die contention and head-of-line blocking can result in increased memory command latency.

[0025] The example memory subsystem of the present disclosure addresses the above and other deficiencies by implementing a JIT scheduling component that manages buffer allocation, data transfer, and memory operation submission from the front end (Flash Translation Layer (FTL) firmware) to the memory subsystem back end (e.g., memory device). The JIT scheduler can process requests from the front end via a cursor. A cursor refers to a module between the front end and the back end that can be used to track the physical memory address of the next block or page to be written to the NAND device. The JIT scheduler can allocate data buffers and communicate with the host system to retrieve data from the host system. The JIT scheduler can use multiple scheduling pools to ensure that no blocking occurs regarding data transfer with the host system or with back end resources. Multiple scheduling pools can prioritize the requests by classifying them into one or more business categories. Business classification is the process of classifying business according to various parameters, such as the origin or type of business. Some business categories include host reads, host writes, and background operations.

[0026] An example memory subsystem may include a Quality of Service (QoS) traffic manager that can be used to maintain the number of available QoS credits for each traffic class for any given scheduling timeframe. A QoS credit refers to the unit of count in the QoS counting mechanism. For example, when a memory controller processes a memory operation, it may use one or more QoS credits from the available QoS credits for the corresponding traffic class. QoS credits can be used as a counting mechanism to manage the number of outstanding memory operations per traffic class for any given scheduling timeframe without starving lower-priority traffic classes. QoS credits can represent the timing, power, or thermal load of a memory operation. A JIT scheduler can identify the traffic class of a memory operation. The JIT scheduler can request QoS credits for each traffic class from the QoS traffic manager. Based on the response (success or failure) from the request, the JIT scheduler can determine whether the memory operation can be sent to the backend memory device or whether the memory operation should remain in the scheduling pool until the next available scheduling timeframe. The QoS traffic manager reduces expected host command latency by ensuring that a minimum number of each traffic class is sent to the backend memory device in a given scheduling timeframe without starving lower-priority traffic classes.

[0027] The instance memory subsystem may include a backend resource manager module. The JIT scheduler may query the backend resource manager to determine whether backend hardware resources (parallelizable hardware memory resources) are available to complete requests at any given time. Parallelizable backend resources refer to memory devices and / or processing units that can be accessed simultaneously by the memory controller.

[0028] Memory resources (or back-end resources) for a memory device can be parallelizable NAND units, such as NAND cells (e.g., NAND dies, NAND planes, blocks, or pages), or they can be parallelizable controller-specific resources, such as logical command processors (LCPs). LCPs can be implemented by a memory controller to process incoming commands on one or more data buses. For example, a NAND device with 16 channels can have 16 available LCPs, one for each channel. Back-end resources can be hardware architecture-specific. By identifying back-end resource availability, a back-end resource manager can improve worst-case host command latency by reducing memory resource contention.

[0029] It is desirable to understand the impact on system-level performance of various workloads submitted to the JIT scheduling component. An example simulator may include a JIT scheduler and a QoS traffic manager of the JIT scheduling component to simulate the scheduling of different workloads, where the workloads comprise sequences of incoming commands. The simulator may simulate various expected latencies for various requests in various workloads to tune various default values for QoS credits used by the QoS traffic manager, as discussed further below.

[0030] Various aspects of the above-referenced methods and systems are described in detail below by way of example and not by way of limitation.

[0031] Figure 1 An example computing system 100 is illustrated that includes a memory subsystem 110 according to some embodiments of the present disclosure. Memory subsystem 110 may include media such as one or more volatile memory devices (e.g., memory device 140), one or more non-volatile memory devices (e.g., memory device 130), or a combination thereof.

[0032] The memory subsystem 110 may be a storage device, a memory module, or a combination of a storage device and a memory module. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage (UFS) drives, secure digital (SD) cards, and hard disk drives (HDDs). Examples of memory modules include dual inline memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and various types of non-volatile dual inline memory modules (NVDIMMs).

[0033] The computing system 100 may be a computing device, such as a desktop computer, a laptop computer, a network server, a mobile device, a vehicle (e.g., an airplane, drone, train, car, or other transportation), a device with Internet of Things (IoT) capabilities, an embedded computer (e.g., an embedded computer included in a vehicle, industrial equipment, or networked commercial device), or such a computing device that includes a memory and a processing device.

[0034] The computing system 100 may include a host system 120 coupled to one or more memory subsystems 110. In some embodiments, the host system 120 is coupled to multiple memory subsystems 110 of different types. Figure 1 An example of a host system 120 coupled to one memory subsystem 110 is illustrated. As used herein, "coupled to" or "coupled with" generally refers to a connection between components, which can be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical, optical, magnetic, etc.

[0035] The host system 120 may include a processor chipset and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more caches, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). The host system 120 uses the memory subsystem 110, for example, to write data to the memory subsystem 110 and read data from the memory subsystem 110.

[0036] The host system 120 can be coupled to the memory subsystem 110 via a physical host interface. Examples of the physical host interface include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, Fibre Channel, Serial Attached SCSI (SAS), a Double Data Rate (DDR) memory bus, a Small Computer System Interface (SCSI), a Dual In-line Memory Module (DIMM) interface (e.g., a DIMM socket that supports Double Data Rate (DDR)), and the like. The physical host interface can be used to transfer data between the host system 120 and the memory subsystem 110. When the memory subsystem 110 is coupled to the host system 120 via a physical host interface (e.g., a PCIe bus), the host system 120 can further utilize an NVM Express (NVMe) interface to access components (e.g., the memory device 130). The physical host interface can provide an interface for passing control, address, data, and other signals between the memory subsystem 110 and the host system 120. Figure 1Memory subsystem 110 is illustrated as an example. In general, host system 120 can access multiple memory subsystems via the same communication connection, multiple individual communication connections, and / or a combination of communication connections.

[0037] Memory devices 130 and 140 may include any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 140) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0038] Some examples of non-volatile memory devices (e.g., memory device 130) include non-and-disconnect (NAND)-type flash memory and write-in-place memory, such as a three-dimensional cross-point ("3D cross-point") memory device, which is a cross-point array of non-volatile memory cells. The cross-point array of non-volatile memory cells can be combined with a stackable cross-grid data access array to store bits based on changes in bulk resistance. In addition, compared to many flash-based memories, cross-point non-volatile memory can perform write-in-place operations, where non-volatile memory cells can be programmed without first erasing the non-volatile memory cells. NAND-type flash memory includes, for example, two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0039] Each of the memory devices 130 may include one or more memory cell arrays. One type of memory cell, such as a single-level cell (SLC), may store one bit per cell. Other types of memory cells, such as multi-level cells (MLC), triple-level cells (TLC), quad-level cells (QLC), and penta-level cells (PLC), may store multiple bits per cell. In some embodiments, each of the memory devices 130 may include one or more memory cell arrays, such as SLC, MLC, TLC, QLC, PLC, or any combination thereof. In some embodiments, a particular memory device may include an SLC portion of memory cells, and an MLC portion, a TLC portion, a QLC portion, or a PLC portion. The memory cells of the memory device 130 may be grouped into pages, which may refer to logical units of a memory device for storing data. In the case of some types of memory (e.g., NAND), pages may be grouped to form blocks.

[0040] Although nonvolatile memory components such as a 3D cross-point nonvolatile memory cell array and NAND-type flash memory (e.g., 2D NAND, 3D NAND) are described, the memory device 130 may be based on any other type of nonvolatile memory, such as read-only memory (ROM), phase-change memory (PCM), selectable memory, other chalcogenide-based memory, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin transfer torque (STT)-MRAM, conductive bridging RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), or non-OR flash memory, or electrically erasable programmable read-only memory (EEPROM).

[0041] The memory subsystem controller 115 (or, for simplicity, controller 115) can communicate with the memory device 130 to perform operations, such as reading data, writing data, or erasing data at the memory device 130 and other such operations. The memory subsystem controller 115 can include hardware, such as one or more integrated circuits and / or discrete components, buffer memory, or a combination thereof. The hardware can include digital circuitry with dedicated (i.e., hard-coded) logic to perform the operations described herein. The memory subsystem controller 115 can be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.), or another suitable processor.

[0042] The memory subsystem controller 115 may include a processing device including one or more processors (e.g., processor 117) configured to execute instructions stored in local memory 119. In the illustrated example, the local memory 119 of the memory subsystem controller 115 includes embedded memory configured to store instructions for performing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 110, including handling communications between the memory subsystem 110 and the host system 120.

[0043] In some embodiments, local memory 119 may include memory registers that store memory pointers, fetched data, etc. Local memory 119 may also include read-only memory (ROM) for storing microcode. Figure 1 The example memory subsystem 110 in FIG. 1 has been described as including a memory subsystem controller 115, but in another embodiment of the present disclosure, the memory subsystem 110 does not include a memory subsystem controller 115 and may instead rely on external control (e.g., provided by an external host or by a processor or controller separate from the memory subsystem).

[0044] Generally speaking, the memory subsystem controller 115 may receive commands or operations from the host system 120 and convert them into instructions or appropriate commands to achieve the desired access to the memory device 130. The memory subsystem controller 115 may be responsible for other operations such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical addresses (e.g., logical block addresses (LBAs), namespaces) and physical addresses (e.g., physical MU addresses, physical block addresses) associated with the memory device 130. The memory subsystem controller 115 may further include host interface circuitry to communicate with the host system 120 via a physical host interface. The host interface circuitry may convert commands received from the host system into command instructions to access the memory device 130, and convert responses associated with the memory device 130 into information for the host system 120.

[0045] The memory subsystem 110 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 110 may include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoders and column decoders) that receive addresses from the memory subsystem controller 115 and decode the addresses to access the memory device 130.

[0046] The memory subsystem 110 may include a JIT scheduling component 113 that manages buffer allocation, data transfer, and back-end memory operations submitted by the Flash Translation Layer (FTL) firmware to the back-end memory devices. The FTL firmware is a module that maps logical block addresses on the host side to physical addresses of the memory devices. In some embodiments, the memory subsystem controller 115 includes at least a portion of the JIT scheduling component 113. In some embodiments, the JIT scheduling component 113 is part of the host system 110, an application, or an operating system. In other embodiments, the local media controller 135 includes at least a portion of the JIT scheduling component 113 and is configured to perform the functions described herein. Further details regarding the operation of the JIT scheduling component 113 are described below.

[0047] Figure 2 A block diagram illustrating JIT scheduling component 113 according to one embodiment. JIT scheduling component 113 may include JIT scheduler 210, QoS traffic manager 207, memory resource manager 209, and response router 211. Each of modules 207 through 211 may be implemented by software, hardware, or a combination thereof.

[0048] The JIT scheduler 210 retrieves and schedules incoming requests by service class within the current scheduling timeframe, which may be one or more clock cycles of the memory subsystem. In one embodiment, incoming requests are stored in a scheduling pool. In other embodiments, incoming requests may be stored in a first-in-first-out (FIFO) queue, a first-in-last-out (FILO) queue, or a combination thereof. FIFO / FILO describes the order in which access requests are made. For a FIFO queue, when a request is received, it is added to the tail of the queue. The pending request is then removed from the head of the queue.

[0049] In one embodiment, incoming requests may be stored in a pool of linked lists (single or double linked lists), where each pool has one linked list for incoming requests. Figure 3 , the JIT scheduler 210 stores requests in a two-dimensional array of scheduling pools 300, where each pool 302 in the array 300 is intended for a specific business class and a specific backend resource. When a request is received from the read / write / erase mailboxes 201 to 203, the JIT scheduler may attempt to retrieve any available node from the free pool 301, fill in the request with the appropriate information (memory operation, business class, memory operation type, etc.), and place the node in the pool in the array 300 according to the business class and backend resource corresponding to the request. The mailbox is a buffer for the front end 200 to place requests, and the JIT scheduler 210 will eventually retrieve these requests. Here, the nodes are linked to a linked list in the array 300, and the nodes can store information for the request. The linked list can store one or more nodes, where each node contains a reference to the next node in the list. The JIT scheduler 210 can iterate through each pool in the array 300, such as pool [1][1] ... pool [m][n], to determine whether a particular pool has any requests and whether it can process the request. In one embodiment, the JIT scheduler 210 iterates through each pool in a round-robin fashion. Round-robin is an arrangement in which requests are sequentially selected from all pools in a predetermined order. The predetermined order defines the priorities of the service classes. For example, the service class host reads may have a higher priority than host writes, and host writes may have a higher priority than background operations. Thus, the predetermined order may prioritize host reads over host writes, and host writes over background operations. In another embodiment, the rotation is a weighted round-robin. Weighted round-robin is similar to round-robin, except that requests are selected based on weights associated with each pool. If a request in a pool can be processed (i.e., if QoS credits and NAND resources are available, as discussed further below), the requesting node is removed from the pool, forwarded to the backend (e.g., a memory device), and placed back into the free pool 301. In summary, the array 300 can be used to ensure that when requests for different combinations of service classes and backend resources are present, requests are submitted from different service class / resource combinations (or based on weights in a weighted round-robin).

[0050] The QoS traffic manager 207 may implement a Quality of Service (QoS) policy for the request pool. In one embodiment, the implemented QoS policy may be based on the traffic class of the request within the scheduling time frame and manage the requests using a credit-based scheme to avoid starving the pool of lower priority requests. For example, upon initialization, a credit-based scheme may specify a preconfigured number of available QoS credits for each traffic class. The JIT scheduler 210 then consumes QoS credits from a particular traffic class before forwarding the request to the backend. A request may be submitted only when there are the requisite number of available QoS credits to submit the request, and the number of available QoS credits is then decremented to reflect the submission. In this way, when there are requests from different traffic classes, the submission of requests from different traffic classes is managed by the number of available QoS credits for the traffic class within a given scheduling time frame. As Figure 4 As described in

[15] , in one embodiment, the QoS traffic manager 207 may include modules such as a reset credit module 401, an obtain credit module 403, an add credit module 405, a traffic class credit table 407, and a memory operation type table 409. Tables 407-409 may be initialized by the QoS traffic manager 207 and may be stored in the local memory 119. The traffic class credit table 407 may specify various traffic classes, a preconfigured number of credits (default credits), and available credits for the corresponding traffic class. In one embodiment, traffic classes may include host read (JIT_HOST_READ), host write (JIT_HOST_WRITE), and background request (JIT_BACKGROUND_REQUEST). The default credit columns for the corresponding traffic classes may be DEFAULT_HOST_READ_CREDITS, DEFAULT_HOST_WRITE_CREDITS, and DEFAULT_BACKGROUND_REQUEST_CREDITS, respectively. Default credits are credits that can be set to the available credits (a1, a2, and a3) at the beginning of each new scheduling time frame. The Available Credits column can store a count of available credits for a traffic class. Reset Credits module 401, Get Credits module 403, and Add Credits module 405 can each modify (reset, subtract, and add, respectively) credits from the Available Credits column.

[0051] The memory operation type table 409 may specify the number of memory operation types and their corresponding credit requirements. For example, the memory operation type may include different operations (read or write) for different types of memory cells. Some example types are: single-level cell (SLC) read (SLC_READ), SLC write (SLC_WRITE), QLC LP write (QLC_LP_WRITE), QLC UPXP write (QLC_UPXP_WRITE), QLC TP write (QLC_TP_WRITE). It should be noted that a quad-level cell (QLC) may have four pages referred to as lower page (LP), upper page (UP), extra page (XP), and top page (TP). Different memory operation types may represent different operations used to access a page of an SLC or a page of QLC memory cells. Reference Figure 4 , the credits required to complete these operations can be preset as: b1, b2, b3, b4, and b5, respectively. Different memory operation types may have different processing requirements: timing requirements, power requirements, thermal load, or a combination thereof. Although only three traffic classes and five memory operation types are shown, QoS traffic manager 207 can specify any number of traffic classes and memory operation types.

[0052] For one example, at the beginning of a scheduling time frame or in response to a power-up event, the QoS traffic manager 207 may set the available credit column of table 407 to be equal to the default credit column of table 407 (via the reset credit module 401). The JIT scheduler 210 may decrement some credits from the available credits for a particular traffic class when it submits a memory operation request associated with that traffic class to the backend (via the get credit module 403). The QoS traffic manager 207 then uses the available credit column of table 407 to determine how many credits are available for the traffic class and grants the request submission if there are still enough available credits to service the request, or denies the request submission if the number of available credits is less than the number of credits required to service the request. If the memory operation subsequently fails for other reasons (e.g., backend resources are busy, etc.), the add credit module 405 may add the granted credits back to the available credits for the traffic class.

[0053] The backend resource manager 209 can manage the availability of backend resources for incoming requests. As previously described, a backend resource can be a software or hardware parallelizable unit that can be accessed by the memory controller at any given time. Examples of backend resources include NAND dies, planes, blocks, pages, or logical command processors (LCPs). Backend resources can be designated based on the architecture of the backend memory device or memory controller. For example, if the backend memory device can be accessed by the memory controller in parallel channels, the backend resource can be designated as an LCP.

[0054] refer to Figure 5 In one embodiment, the backend resource manager 209 includes a get availability module 501, a memory operation completion module 503, and an in-flight memory operation table 505. These modules may be implemented as software-based or hardware-based modules. The get availability module 501 tracks the availability of backend resources and may determine whether a particular backend resource is available. The memory operation completion module 503 tracks the completion status of memory operations and may determine whether a previously submitted memory operation has been completed. The backend resource manager 209 may store a list of previously submitted memory operation requests that are to be completed at the in-flight memory operation table 505. The in-flight memory operation table 505 may be stored as Figure 1 portion of the local memory 119 in the .

[0055] In one embodiment, the JIT scheduler 210 requests the availability of a given backend resource and memory operation type (via the get availability module 501). The backend resource manager 209 may compare the requested backend resource with the backend resources of the in-flight memory operation to determine the availability of the requested backend resource. Depending on the availability, the backend resource manager 209 may return a pass / fail status code. For example, if the requested backend resource is a specific page of a block, the backend resource manager 209 may compare it with the pages of the in-flight memory operation to determine whether the specific page is currently being accessed.

[0056] In another example, if the requested backend resource is a specific LCP, the backend resource manager 209 can compare the LCPs used by in-flight memory operations to determine whether the specific LCP is available. In another example, if the number of outstanding memory operations for each LCP is below a predetermined threshold, then the LCP is available. Thus, the backend resource manager 209 can manage the availability of backend resources in a manner that does not compromise the performance of the memory subsystem.

[0057] In another embodiment, the backend resource manager 209 implements a prediction model to predict whether an LCP is available. The prediction model can base the prediction on one or more parameters of the last memory operation (e.g., submission time and type). For example, if the last memory operation was submitted t nanoseconds ago and was of a type of memory operation expected to complete within t nanoseconds, then the last memory operation is considered complete. Therefore, the current memory operation request can be submitted before the completion of the last memory operation is confirmed by the backend resource manager 209 via the response router 211. Therefore, this prediction model can be used to keep the backend pipeline more efficiently loaded.

[0058] refer to Figure 2 , operations 1 to 10 are described with respect to an example scheduling process flow according to one embodiment.

[0059] Operation 1. The JIT scheduler 210 receives a read / write / erase request from the NAND cursor 205. The request may be a host read, host write, or background memory operation received from the front end 200. A background memory operation may be a background read, background write, or background erase operation. Background operations may represent media management operations initiated by the memory controller to maintain predefined data state metrics of the memory device within a specified range. The NAND cursor 205 may receive these requests from the read mailbox 201, the write mailbox 202, or the erase mailbox 203. In another embodiment, the JIT scheduler 210 may receive memory operation requests directly from mailboxes 201 and 203. The requested memory operation may utilize a logical or physical address. For commands utilizing a logical address, the JIT scheduler 210 may request the NAND cursor 205 to translate the logical address into a corresponding physical address using the logical-to-physical translation table from the data flash manager 204. The NAND cursor 205 then sends the physical address back to the JIT scheduler 210. Mailboxes 201 - 203 may be memory buffers that store requested command operations.

[0060] Operation 2. The JIT scheduler 210 manages the incoming requests from the NAND cursor 205. For example, the JIT scheduler 210 may store the incoming requests in a two-dimensional (m×n) array of a scheduling pool according to the business class and backend resources corresponding to the incoming requests, such as Figure 3 Each pool corresponds to a specific combination of business class and backend resources. The JIT scheduler 210 may use round-robin or weighted round-robin to select a round of requests from the scheduling pool for submission to the backend 230. For example, for one scheduling time frame, the JIT scheduler 210 may traverse the m×n array of scheduling pools in a predetermined order (e.g., traversing the pools in a specified dimension first).

[0061] Operation 3. For an incoming request, the JIT scheduler 210 may retrieve credits from the QoS service manager 207. For example, based on the requested service class and memory operation type, the JIT scheduler 210 determines whether sufficient credits are available to service the request. Based on this determination, the QoS service manager 207 returns a success or failure status. Upon receiving a failure status, the incoming request is placed back into the scheduling pool for submission in a future scheduling timeframe. For a success status, the JIT scheduler 210 proceeds to check whether a backend is available to service the request.

[0062] Operation 4. The JIT scheduler 210 may send the request to the backend resource manager 209 to check the availability of backend resources for servicing the request. For example, the backend resource manager 209 may determine whether any outstanding requests are currently using the requested backend resource. Here, the backend resource manager 209 may track outstanding (or in-flight) requests and compare them with the current request. If the comparison yields a match, for example, the backend resource corresponding to the outstanding request is the requested backend resource, then the requested backend resource may be deemed unavailable. In one embodiment, for LCPs as backend resources, the backend resource manager 209 may compare the FPA corresponding to the outstanding request with the FPA corresponding to the current request for overlap. The backend resource manager 209 may determine that the backend resource is available if there is no overlap. In another embodiment, if the number of outstanding requests for the LCP is below a certain threshold, then the backend resource manager 209 deems the backend resource available. In another embodiment, the backend resource manager 209 predicts whether the backend resource is available by analyzing the last request submitted. The backend resource manager 209 returns a status code reflecting the availability of the specified resource.

[0063] If the specified backend resource is not available, the JIT scheduler 210 returns a credit to the QoS traffic manager 207 (via the add credit module 405) and appends the request back to the scheduling pool to be processed at a future time. Otherwise (i.e., if the specified backend resource is available), the backend resource manager 209 stores the request in the in-flight memory operations table (e.g., Figure 5 505) and marks the request as an in-flight request.

[0064] Operation 5. If the request is a host write, the JIT scheduler 210 allocates a data buffer for the request and waits for the host data transfer engine 220 to provide data. Other request types proceed to Operation 8.

[0065] Operation 6. For a host write request, the JIT scheduler 210 may request data from the front end 200 via the host data transfer engine 220. The data request may be asynchronous so that the JIT scheduler 210 may process the next request concurrently with the current request.

[0066] Operation 7. The host data transfer engine 220 may store the requested data to the allocated data buffer within the JIT scheduler 210 and send a transfer completion response to the JIT scheduler 210 .

[0067] Operation 8. Once the request is ready to be submitted, it is submitted to the backend 230.

[0068] Operation 9. The response router 211 receives an operation completion response from the backend 230 .

[0069] Operation 10. The response router 211 routes the response to the backend resource manager 209 and / or the frontend 200 to notify the backend resource manager 209 and / or the frontend 200 that the operation is completed.

[0070] Figure 6 6 is a flow chart of an example method 650 for scheduling memory operation requests according to some embodiments of the present disclosure. The method 650 may be performed by processing logic, which may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 650 is performed by Figure 1 JIT scheduling component 113 or Figure 2 The JIT scheduler 210 is executed. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0071] At operation 651, processing logic receives a request to perform a memory operation using hardware resources associated with a memory device. In one embodiment, processing logic enqueues the request in an inbound queue. The inbound queue can be represented by an (m×n) array of scheduling pools. Each scheduling pool in the (m×n) array of scheduling pools corresponds to a specific hardware resource and a specific traffic class. The scheduling pool can be implemented as a linked list (single or double linked list), where information for the request can be stored in a node in the linked list. During a scheduling time frame, processing logic can select one or more rounds of requests from the scheduling pool for processing according to a predetermined order through round-robin or weighted round-robin.

[0072] At operation 653, processing logic identifies a traffic class corresponding to the memory operation. For example, the traffic class may be one of: host read, host write, and background operation.

[0073] At operation 655, processing logic determines the number of available quality of service (QoS) credits for the traffic class during the current scheduled time frame. At the beginning of the scheduled time frame, or when the memory subsystem is powered on, a number of credits equal to its default credits is allocated to a data buffer (or table) storing the available credits for the traffic class. Each traffic class has a predetermined number of default credits. As processing logic obtains the number of QoS credits for memory operations from the available QoS credits, the available QoS credits are decremented.

[0074] At operation 657, processing logic determines a number of QoS credits associated with the type of memory operation. Each memory operation corresponds to a memory operation type. Each memory operation type is associated with a predetermined number of credits to complete the type of memory operation. Some example types may be: SLC read, SLC write, QLC LP write, QLC UPXP write, and QLC TP write. Here, a quad-level cell (QLC) may have four pages referred to as a lower page (LP) and an upper page (UP, XP, and TP). Different memory operation types may have different processing requirements: timing latency, power requirements, thermal load, or a combination thereof. QoS credits may represent these processing requirements. Therefore, different memory operation types may be associated with different QoS credits.

[0075] At operation 659, in response to determining that the number of QoS credits associated with the type of memory operation is less than the number of available QoS credits, processing logic submits the memory operation to be processed at the memory device during the current scheduled time frame and decrements the available QoS credits by the number of QoS credits associated with the memory operation. If the number of QoS credits associated with the memory operation is greater than the available QoS credits, processing logic places the request back into a scheduling pool (e.g., a queue) for requests to be processed at a later scheduled time frame. That is, processing logic can manage the number of memory operations of a traffic class submitted within a scheduled time frame. Available QoS credits provide a limit on how many memory operations of a traffic class can be submitted within a scheduled time frame to avoid starvation of lower priority requests.

[0076] The processing logic may further determine the availability of the hardware resource before submitting the request to the memory device. The processing logic may use information about previously submitted but not yet completed in-flight (incomplete) memory operations to determine the hardware resource availability. In one embodiment, based on a statistical model, using information for in-flight requests, the processing logic may determine a count of outstanding requests for the hardware resource. If the count is below a predetermined threshold, the processing logic determines that the hardware resource is available and the processing logic submits the request to the memory device. It should be noted that the hardware resource may be a NAND die, a NAND plane, a NAND block, a NAND page, an LCP, etc.

[0077] The processing logic may also use a prediction model to determine the availability of hardware resources. The prediction model may predict when the last submitted request associated with the hardware resource will complete based on information about the last submitted request. The information about the last submitted request may include the type of the last submitted memory operation and the submission timestamp of the last submission. Based on this information, the processing logic may predict when the memory operation will complete.

[0078] Figure 77 is a flow chart of an example method 700 for scheduling memory operation requests according to some embodiments of the present disclosure. The method 700 may be performed by processing logic, which may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 700 is performed by Figure 1 JIT scheduling component 113 or Figure 2 The JIT scheduler 210 is executed. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0079] At operation 701, processing logic receives a request to perform a memory operation using hardware resources associated with a memory device. At operation 703, processing logic identifies a service class corresponding to the memory operation. Some example service classes may be: host read, host write, and background operation. At operation 705, processing logic determines the type of memory operation corresponding to the memory operation. Some example types may be: SLC read, SLC write, QLC LP write, QLC UPXP write, and QLCTP write.

[0080] At operation 707, processing logic determines whether the memory operation is to be processed during the current scheduled time frame based on the traffic class and the type of memory operation. For example, processing logic determines the available QoS credits for the traffic class and the required QoS credits to complete the request for the memory operation type. If there are more available QoS credits than the required QoS credits, then the memory operation is to be processed during the current scheduled time frame, and processing logic decrements the available QoS credits. Otherwise, the memory operation is queued for processing at a future scheduled time frame. At operation 709, in response to determining that the memory operation is to be processed during the current scheduled time frame, processing logic submits the memory operation to be processed at the memory device.

[0081] The scheduling of memory operations can be simulated before deployment. For example, a latency simulator can simulate the system-level performance of various workloads submitted to the JIT scheduler by simulating the expected latency of memory operations for each service class and memory operation type. Information about the expected latency of memory operations based on the service class and memory operation type of the memory operations can be used to simulate the performance of the JIT scheduler. This simulator can estimate the system-level performance impact that can result from using different NAND types and background operation algorithms. Therefore, the amount of available QoS credits for each service class and the QoS credits required to complete certain memory operations can be further adjusted by the operator to meet the required scheduling latency for various workloads.

[0082] Figure 8 A block diagram illustrating a latency simulator according to one embodiment. Latency simulator 800 can simulate the latency model of a JIT scheduler. Simulator 800 may include an input module 801, a JIT scheduler 810, a QoS service manager 807, and a response router / output 811. Each of modules 801 through 811 may be a hardware-based or software-based module of the latency simulator.

[0083] The input module 801 may receive the expected delay profile, for example, via an input / output (I / O) interface or a graphical user interface (GUI). Figure 9 As described in the configuration file 809 of FIGURE 8, the expected latency profile represents the expected time period for processing memory operations according to the memory operation type of the memory operation for various service classes. The expected latency profile may reflect the characteristics of a specific NAND device and how long it takes for a memory operation of that type to be performed on the NAND device. The expected latency profile may be in the form of a binary metadata file, a text file, or a spreadsheet.

[0084] The input module 801 may receive a workload list that specifies a list of memory operations for various business classes that may be forwarded to the JIT scheduler 810 at various times to test the performance of the JIT scheduler 810. An example workload list may include 10 SLC read memory operations to be submitted at scheduled time frame 1; 20 SLC write memory operations to be submitted at scheduled time frame 2; and so on. A workload profile may specify repetition of specific memory operations or may have a unique set of memory operations for each scheduled time frame. The workload list may be in the form of a binary metadata file, a text file, or a spreadsheet generated by an operator.

[0085] In one embodiment, the input module 801 includes a read / write / erase buffer (not shown) for the JIT scheduler 810, where various types of memory operations (read, write, erase) are stored in the buffer. The JIT scheduler 810 can then retrieve the memory operations from the read / write / erase buffer for processing. Figure 2 Read / write / erase mailbox functionality.

[0086] The JIT scheduler 810 may schedule memory operations at various scheduling time frames specified by the workload list. In some embodiments, the JIT scheduler 810 may (similar to Figure 2 The JIT scheduler 210 of FIG. 210 schedules memory operations using a first-in-first-out (FIFO) queue, a first-in-last-out (FIFO), a pool, or a combination thereof. In one embodiment, the JIT scheduler 210 may schedule memory operations using a first-in-first-out (FIFO) queue, a first-in-last-out (FIFO), a pool, or a combination thereof. Figure 10 The scheduling pool 1000 described in the above may include one or more linked lists (single linked list or double linked list). In one embodiment, similar to Figure 3 In the scheduling pool 300, pool 1000 is a two-dimensional array of scheduling pools, wherein each pool 1002 in pool 1000 is intended for a specific service class and a specific simulated backend resource, for example, service classes 1...m and simulated backend resources 1...n. In one embodiment, the simulated backend resources 1...n are simulated as always available.

[0087] In one example, when a memory operation is received from the input module 801, the JIT scheduler 810 may retrieve a node from the free pool 1003, add the appropriate information from the memory operation to the node, and place the node into one of the pools 1000 based on the traffic class corresponding to the memory operation and the simulated backend resources. The JIT scheduler 810 may iterate through each pool in the pools 1000, e.g., pool [1] [1] ... pool [m] [n], to determine whether a particular pool has any memory operations and whether the memory operations can be processed. In one embodiment, the JIT scheduler 810 iterates through each pool in a predetermined order using a round-robin method. In another embodiment, the JIT scheduler 810 iterates through each pool in a predetermined order using a weighted round-robin method. The predetermined order may define priorities for various traffic classes. For example, a host read traffic class may have a higher priority than a host write traffic class, etc. If the memory operation in the pool can be processed (i.e., if QoS credits are available), the node for the memory operation is removed from the pool, forwarded to the response router 811, and the node is placed back into the free pool 1001. Thus, pool 1000 may be used to ensure that memory operations are selected among different traffic classes according to their portion of available QoS credits (or selected according to weights based on a weighted round-robin algorithm).In some embodiments, it is assumed that the simulated backend resources are available.

[0088] refer to Figure 8 , the QoS traffic manager 807 may implement a Quality of Service (QoS) policy for the scheduling pool, similar to Figure 2 QoS business manager 207. In one embodiment, the implemented QoS policy may manage memory operations according to their business class within a scheduling time frame using a QoS credit-based scheme to avoid starving the pool of lower priority requests. For example, upon initialization, the credit-based scheme may specify a preconfigured number of available QoS credits for each business class. The JIT scheduler 810 then consumes QoS credits from a particular business class before forwarding the memory operation to the response router 811. The memory operation may be submitted only when there are the requisite number of available QoS credits to submit the memory operation, and then the number of available credits is decremented to reflect the submission. In this way, when memory operations of different business classes are present, the submission of memory operations from different business classes is managed by the number of available credits for the business class within a given scheduling time frame. As Figure 11 As described in

[0066] , in one embodiment, the QoS traffic manager 807 may include modules such as a reset credit module 401, an obtain credit module 403, a memory operation type table 409, and a traffic class credit table 1101. Tables 409 and 1101 may be initialized by the QoS traffic manager 807 and may be stored in memory. Traffic class credit table 1101 may specify various traffic classes, a preconfigured number of credits (default credits), and available credits for the corresponding traffic class. In one embodiment, traffic classes may include host reads (JIT_HOST_READ), host writes (JIT_HOST_WRITE), and background memory operations (JIT_BACKGROUND_READ, JIT_BACKGROUND_WRITE, and JIT_BACKGROUND_ERASE). The default credit columns for the corresponding service classes may be DEFAULT_HOST_READ_CREDITS, DEFAULT_HOST_WRITE_CREDITS, DEFAULT_BACKGROUND_READ_CREDITS, DEFAULT_BACKGROUND_WRITE_CREDITS, and DEFAULT_BACKGROUND_ERASE_CREDITS, respectively. The default credits may be the credits to which the available QoS credits (a1, a2, a3, ...) may be set at the beginning of each new scheduling time frame. The Available Credits column may store a count of available QoS credits for the service class. The Reset Credits module 401 and the Get Credits module 403 may each modify (reset and subtract, respectively) the credits from the Available Credits column.

[0089] Memory operation type table 409 may specify the number of memory operation types and their corresponding QoS credit requirements. For example, memory operation types may include different operations (read, write, or erase) for different types of memory cells. Some example types are: single-level cell (SLC) read (SLC_READ), SLC write (SLC_WRITE), QLC LP write (QLC_LP_WRITE), QLC UPXP write (QLC_UPXP_WRITE), and QLC TP write (QLC_TP_WRITE). Note that a quad-level cell (QLC) may have four pages, referred to as lower page (LP), upper page (UP), extra page (XP), and top page (TP). Different memory operation types may represent different operations for accessing a page of SLC or a page of QLC memory cells. Examples of the credits required to complete these operations may be preset as values: b1, b2, b3, b4, and b5, respectively. Different memory operation types may have different processing requirements: timing requirements, power requirements, thermal load, or a combination thereof. Although only three traffic classes and five memory operation types are shown, QoS traffic manager 807 may specify any number of traffic classes and memory operation types.

[0090] For one example, at the beginning of a scheduling time frame, the QoS traffic manager 807 may set the Available Credits column of table 1101 to be equal to the Default Credits column of table 1101 (via the Reset Credits module 401). The JIT scheduler 810 may decrement some credits from the Available Credits for a particular traffic class (via the Get Credits module 403) when it submits a memory operation request associated with that traffic class to the response router 811. The QoS traffic manager 807 then uses the Available Credits column of table 1101 to determine how many credits are available for the traffic class and either grant the memory operation submission if there are still enough available credits to service the memory operation, or deny the memory operation submission if the number of available credits is less than the credits required to service the memory operation.

[0091] Response router / output 811 can simulate a receiver for memory operation submissions submitted from JIT scheduler 810. Response router / output 811 can retrieve the expected latency of a memory operation based on its business class and memory operation type. Response router / output 811 can indicate a memory operation as serviced after a time equal to the expected latency elapsed from submission by transmitting a memory operation completion notification back to JIT scheduler 810. Response router / output 811 can output a file specifying some statistical data on the performance of the JIT scheduler. The output may include the worst-case latency of the scheduled timeframe, the number of scheduled timeframes used for a workload, and so on. The output file may also include graphs that can be used to analyze potential bottlenecks and die contention within the scheduled timeframe.

[0092] refer to Figure 8 , operations 11 to 15 are described with respect to an example simulation according to one embodiment.

[0093] Operation 11. The JIT scheduler 810 retrieves read / write / erase memory operations at various times simulated from the input module 801. The memory operations may be of a host read, host write, or background memory operation business class. The background memory operations may be background read, background write, or background erase operations. The memory operations may include associated business class data and / or simulated backend resource data. The memory operations may be part of a workload profile (e.g., a workload profile) provided to the input module 801 from an operator, wherein the workload profile may include various read / write / erase memory operations at various scheduling time frames to simulate workloads handled by the JIT scheduler. For example, the JIT scheduler 810 may store incoming memory operations in a two-dimensional (m×n) scheduling pool based on business class and simulated backend resources.

[0094] Operation 12. The JIT scheduler 810 manages incoming memory operations from the input module 801. For example, the JIT scheduler 810 may store incoming memory operations in a two-dimensional (m×n) scheduling pool based on the business class and the emulated backend resources. The JIT scheduler 210 may select memory operations from the scheduling pool using round-robin or weighted round-robin. For example, during a given scheduling time frame, the JIT scheduler 810 may traverse the m×n scheduling pool one or more times to retrieve memory operations from these pools.

[0095] Operation 13. To process the memory operation specified by the incoming request, the JIT scheduler 810 may retrieve available QoS credits from the QoS service manager 807. For example, the type of memory operation is performed based on the requested service class and the required QoS credits. The JIT scheduler 810 may determine whether there are sufficient credits to service the request. For example, if the available QoS credits are greater than the required QoS credits, then sufficient credits are available.

[0096] Operation 14. The QoS service manager 807 returns a success or failure status based on the determination. Upon receiving a failure status, the simulated request is placed back into the scheduling pool for submission in a future scheduling time frame. For a success status, the JIT scheduler 810 proceeds to operation 15.

[0097] Operation 15. The JIT scheduler 810 may submit the memory operation to the response router 811. The response router 811 may return a status indicating that the simulated memory operation is complete after the expected latency of the memory operation has elapsed.

[0098] By simulating the timing of workloads of different memory operations, output 811 can generate an output of latency statistics indicating the performance of the workload. The output can include the worst-case latency period of the scheduled time frame, and how many scheduled time frames are required to complete a particular workload. For example, output 811 can calculate the worst-case latency of the scheduled time frame by calculating the latency for each business class. Output 811 can calculate the business class latency for a business class by adding together the memory operation latencies for the memory operations according to the business class. Output 811 can then apply a maximum function to the different business class latencies to find which business class has the worst latency within the scheduled time frame. In another embodiment, output 811 calculates the worst-case latency of the scheduled time frame by determining the average latency of the memory operations in the business class, and calculating the worst-case latency by multiplying the average latency by the number of memory operations for the business class within the scheduled time frame.

[0099] In one embodiment, the number of frames for a workload can be calculated based on the number of scheduled time frames required to complete the memory operations in the workload profile. In another embodiment, the number of scheduled time frames for a workload is estimated based on the number of memory operations in the workload.

[0100] Simulating different workloads allows analyzing the latency of the JIT scheduler using different available QoS credits and required QoS credits for various types of memory operations. The analysis can provide the worst-case latency for various workloads.

[0101] Figure 12 is a flow chart of an example method for scheduling memory operations according to an embodiment. Method 1200 may be performed by processing logic, which may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, method 1200 is performed by Figure 8 The simulator 800 is performed. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0102] At operation 1201, processing logic receives a workload profile specifying a sequence of memory operations, wherein each memory operation is associated with a type of memory operation. An example memory operation may be an SLC read, address P1. The memory operation is associated with a service class (e.g., host read) and belongs to a memory operation type (e.g., SLC read).

[0103] At operation 1203, processing logic identifies a traffic class associated with each memory operation in the sequence of memory operations. The traffic class may include at least one of: host read, host write, background read and background write, and background erase.

[0104] At operation 1205, processing logic enqueues each memory operation in the sequence of memory operations in a scheduling pool of a plurality of scheduling pools based on a traffic class associated with the memory operation. The scheduling pool may be a two-dimensional array of pools, where each pool includes a linked list, and each link in the linked list represents a memory operation request.

[0105] At operation 1207, processing logic selects one or more memory operations from a plurality of scheduling pools to be serviced within a scheduling time frame based on a quality of service (QoS) policy. The selecting may further include: determining a number of available quality of service (QoS) credits for a traffic class for the scheduling time frame; determining a number of QoS credits associated with the memory operation based on a memory operation type of the memory operation; and in response to determining that the number of QoS credits associated with the memory operation is less than the number of available QoS credits for the traffic class, subtracting the number of QoS credits from the available QoS credits and indicating that the memory operation has been serviced. The memory operation type may include at least one of: a single-level cell (SLC) read, an SLC write, a quad-level cell (QLC) lower page (LP) write, a QLC upper page (UP) write, a QLC extra page (XP) write, and a QLC top page (TP) write.

[0106] At operation 1209, processing logic determines a latency period for each of the one or more memory operations based on the latency profile. In one embodiment, processing logic further determines a memory operation latency period for each of the one or more memory operations based on the latency profile. Processing logic determines a service class latency period by summing the memory operation latency periods of the memory operations corresponding to the service class. Processing logic determines a latency period for the scheduling time frame by selecting a maximum service class latency period from among service class latency periods for the plurality of service classes. The latency profile may include a plurality of latency periods, wherein each latency period in the plurality of latency periods corresponds to a service class and a memory operation type.

[0107] In one embodiment, the processing logic further determines the number of scheduled time frames required to service the sequence of memory operations. The processing logic determines a total latency period for the number of scheduled time frames. For example, the total latency period may be equal to the latency period of the scheduled time frames multiplied by the number of scheduled time frames.

[0108] Figure 13 is a flow chart of an example method for scheduling memory operations according to an embodiment. Method 1300 may be performed by processing logic, which may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, method 1300 is performed by Figure 8 The simulator 800 is performed. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0109] At operation 1301, processing logic receives a memory operation to be simulated. An example memory operation may be an SLC read, address P1. The memory operation is associated with a service class (eg, host read) and belongs to a memory operation type (eg, SLC read).

[0110] At operation 1303, processing logic determines a traffic class corresponding to the memory operation. The traffic class includes at least one of: host read, host write, background read and background write, and background erase.

[0111] At operation 1305, processing logic determines a number of quality of service (QoS) credits associated with the memory operation based on a memory operation type of the memory operation. The memory operation type includes at least one of: a single-level cell (SLC) read, an SLC write, a quad-level cell (QLC) lower page (LP) write, a QLC upper page (UP) write, a QLC extra page (XP) write, and a QLC top page (TP) write.

[0112] At operation 1307, processing logic determines the number of available QoS credits for the traffic class for the scheduled time frame.

[0113] At operation 1309, in response to determining that the number of QoS credits associated with the memory operation is less than the number of available QoS credits for the traffic class, processing logic subtracts the number of QoS credits from the available QoS credits to indicate that the number of QoS credits is no longer available for the traffic class. At operation 1311, processing logic indicates that the memory operation has been serviced.

[0114] In one embodiment, processing logic determines a latency period for a memory operation based on a latency profile, and marks the memory operation as serviced after the latency period for the memory operation has elapsed. In another embodiment, processing logic determines a memory operation latency period for each of one or more memory operations based on the latency profile. Processing logic determines a service class latency period by summing the memory operation latencies of the memory operations corresponding to the service class. Processing logic determines a latency period for a scheduling time frame by selecting a maximum service class latency period for a plurality of service classes. The latency profile includes a number of latency periods, wherein each of the plurality of latency periods corresponds to a service class and a memory operation type.

[0115] In one embodiment, processing logic enqueues a memory operation in one of a plurality of scheduling pools based on a service class of the memory operation, and the processing logic selects the memory operation from the plurality of scheduling pools by round-robin or weighted round-robin with respect to a scheduling time frame. In one embodiment, the processing logic determines the number of scheduling time frames required to service the plurality of memory operations. The processing logic determines a total latency period for the number of scheduling time frames based on a latency profile.

[0116] Figure 14 An example machine illustrating a computer system 600 may be executed within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system 600 may correspond to a host system (e.g., Figure 1 120) that includes, is coupled to, or utilizes a memory subsystem (e.g., Figure 1 memory subsystem 110) or can be used to perform controller operations (e.g., execute an operating system to perform operations corresponding to Figure 1 JIT scheduling component 113 or Figure 8 In some embodiments, the machine may be connected (e.g., using a network) to other machines. The machine may operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0117] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by the machine. Further, while a single machine is described, the term "machine" shall also be taken to include any collection of machines that individually or collectively execute a set (or multiple sets of instructions) to perform any one or more of the methodologies discussed herein.

[0118] The example computer system 600 includes a processing device 602, a main memory 604 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM) or RDRAM, etc.), a static memory 606 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 618, which communicate with each other via a bus 630.

[0119] Processing device 602 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements another instruction set, or a processor that implements a combination of instruction sets. Processing device 602 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. Processing device 602 is configured to execute instructions 626 for performing the operations and steps discussed herein. Computer system 600 may further include a network interface device 608 for communicating over a network 620.

[0120] The data storage system 618 may include a machine-readable storage medium 624 (also referred to as a computer-readable medium) on which is stored one or more sets of instructions 626 or software embodying any one or more of the methodologies or functions described herein. The instructions 626 may also reside, completely or at least partially, within the main memory 604 and / or within the processing device 602 during execution thereof by the computer system 600, with the main memory 604 and the processing device 602 also constituting machine-readable storage media. The machine-readable storage medium 624, the data storage system 618, and / or the main memory 604 may correspond to Figure 1 Memory subsystem 110.

[0121] In one embodiment, instructions 626 include implementing a block selection component (e.g., Figure 1 JIT scheduling component 113 or Figure 8800) of the simulator. Although machine-readable storage medium 624 is shown as a single medium in the example embodiment, the term "machine-readable storage medium" should be considered to include a single medium or multiple media that store one or more sets of instructions. The term "machine-readable storage medium" should also be considered to include any medium capable of storing or encoding a set of instructions for execution by a machine and causing the machine to perform any one or more of the methods of the present disclosure. Thus, the term "machine-readable storage medium" should be considered to include, but not be limited to, solid-state memory, optical media, and magnetic media.

[0122] Some portions of the previous detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means by which those skilled in the data processing arts most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. Operations are those requiring physical manipulation of physical quantities. Typically, but not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. At times, it has proven convenient, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0123] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities within the computer system's registers and memories and transforms that data into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0124] The present disclosure also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the intended purpose, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored on a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0125] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used with programs according to the teachings herein, or it may prove convenient to construct more specialized devices for performing the methods. The structures for various such systems will be presented as described below. Additionally, the present disclosure is not described with reference to any particular programming language. It will be appreciated that the teachings of the present disclosure as described herein can be implemented using various programming languages.

[0126] The present disclosure may be provided as a computer program product or software, which may include a machine-readable medium having stored thereon instructions that can be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory component, or the like.

[0127] In the foregoing description, embodiments of the present disclosure have been described with reference to specific example embodiments thereof. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

Claims

1. A method comprising: Receiving, by a processing device, a request to perform a memory operation using hardware resources associated with a memory device; determining a type of the memory operation; identifying a traffic class corresponding to the memory operation; determining whether the memory operation is to be processed during a current scheduled time frame based on the traffic class and the type of the memory operation; as well as Responsive to determining that the memory operation is to be processed during the current scheduled time frame, the memory operation is submitted to the memory device.

2. The method according to claim 1, further comprising: In response to receiving the request, the request is enqueued in a scheduling pool among a plurality of scheduling pools, wherein the scheduling pool is associated with the traffic class and the hardware resource.

3. The method according to claim 2, further comprising: The requests are dequeued from the scheduling pool using a round-robin algorithm. 4 . The method of claim 1 , wherein committing the memory operation is performed in response to determining that a count of outstanding requests for the hardware resource is below a predetermined threshold. The method of claim 1 , wherein submitting the memory operation is performed in response to predicting that the hardware resource is available based on a previous request. 6 . The method of claim 5 , wherein predicting that the hardware resource is available based on the previous request is based on a type of memory operation of the previous request and a time when the memory operation of the previous request was submitted.

7. The method of claim 1, wherein the traffic class is one of: host read, host write, and background operation.

8. A system comprising: memory device; as well as a processing device operably coupled to the memory device, the processing device configured to: receiving a request to perform a memory operation using hardware resources associated with a memory device; determining a type of the memory operation; identifying a traffic class corresponding to the memory operation; determining whether the memory operation is to be processed during a current scheduled time frame based on the traffic class and the type of the memory operation; as well as Responsive to determining that the memory operation is to be processed during the current scheduled time frame, the memory operation is submitted to the memory device.

9. The system of claim 8, wherein the processing device is further configured to: In response to receiving the request, the request is enqueued in a scheduling pool among a plurality of scheduling pools, wherein the scheduling pool is associated with the traffic class and the hardware resource.

10. The system of claim 9, wherein the processing device is further configured to: The requests are dequeued from the scheduling pool using a round-robin algorithm.

11. The system of claim 8, wherein committing the memory operation is performed in response to determining that a count of outstanding requests for the hardware resource is below a predetermined threshold.

12. The system of claim 8, wherein submitting the memory operation is in response to predicting that the hardware resource is available based on a previous request.

13. The system of claim 12, wherein predicting that the hardware resource is available based on the previous request is based on a type of memory operation of the previous request and a time when the memory operation of the previous request was submitted.

14. The system of claim 8, wherein the traffic class is one of: host read, host write, and background operation.

15. A non-transitory machine-readable storage medium comprising executable instructions that, when executed by a processing device, cause the processing device to: receiving a request to perform a memory operation using hardware resources associated with a memory device; determining a type of the memory operation; identifying a traffic class corresponding to the memory operation; determining whether the memory operation is to be processed during a current scheduled time frame based on the traffic class and the type of the memory operation; as well as Responsive to determining that the memory operation is to be processed during the current scheduled time frame, the memory operation is submitted to the memory device.

16. The non-transitory machine-readable storage medium of claim 15, further comprising executable instructions that, when executed by the processing device, cause the processing device to: In response to receiving the request, the request is enqueued in a scheduling pool among a plurality of scheduling pools, wherein the scheduling pool is associated with the traffic class and the hardware resource.

17. The non-transitory machine-readable storage medium of claim 16, further comprising executable instructions that, when executed by the processing device, cause the processing device to: The requests are dequeued from the scheduling pool using a round-robin algorithm.

18. The non-transitory machine-readable storage medium of claim 15, wherein committing the memory operation is performed in response to determining that a count of outstanding requests for the hardware resource is below a predetermined threshold.

19. The non-transitory machine-readable storage medium of claim 15, wherein committing the memory operation is in response to predicting that the hardware resource is available based on a previous request.

20. The non-transitory machine-readable storage medium of claim 19, wherein predicting that the hardware resource is available based on the previous request is based on a type of memory operation of the previous request and a time when the memory operation of the previous request was submitted.

Images