Just-in-time (JIT) scheduler for the memory subsystem

Abstract

The present disclosure relates to a real-time (JIT) scheduler for a memory subsystem. The memory subsystem of the present disclosure discloses a real-time JIT scheduling system and method. In one embodiment, a system receives a request to perform a memory operation using hardware resources associated with a memory device. The system identifies a business class corresponding to the memory operation. The system determines the number of available quality of service (QoS) credits for the business class during a current scheduling time frame. The system determines the number of QoS credits associated with the type of the memory operation. In response to determining that the number of QoS credits associated with the type of the memory operation is less than the number of available QoS credits, the system submits the memory operation to be processed at the memory device.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to a memory subsystem, and more particularly, to a just-in-time (JIT) scheduler for a memory subsystem. Background Art

[0002] The memory subsystem may include one or more memory devices that store data. The memory devices may be, for example, non-volatile memory devices and volatile memory devices. In general, the host system may utilize the memory subsystem to store data at the memory devices and retrieve data from the memory devices. Summary of the invention

[0003] In one aspect, the present disclosure relates to a system comprising: a memory device; and a processing device operatively coupled to the memory device to perform operations comprising: receiving a request by a just-in-time (JIT) scheduler to perform a memory operation using hardware resources associated with the memory device; identifying a business class corresponding to the memory operation; determining a number of available quality of service (QoS) credits for the business class during a current scheduling time frame; determining a number of QoS credits associated with a type of the memory operation; and in response to determining that the number of QoS credits associated with the type of the memory operation is less than the number of available QoS credits, submitting the memory operation to the memory device.

[0004] In yet another aspect, the present disclosure relates to a method comprising: receiving, by a real-time (JIT) scheduler, a request to perform a memory operation using hardware resources associated with a memory device; identifying a business class corresponding to the memory operation; determining a type of the memory operation; determining whether the memory operation is to be processed during a current scheduling time frame based on the business class and the type of the memory operation; and in response to determining that the memory operation is to be processed during the current scheduling time frame, submitting the memory operation to the memory device.

[0005] In another aspect, the present disclosure relates to a non-transitory machine-readable storage medium comprising instructions that, when accessed by a processing device, cause the processing device to perform one or more operations including: receiving, by a real-time (JIT) scheduler, a request to perform a memory operation using hardware resources of a memory device associated with the memory device; identifying a service class corresponding to the memory operation; determining a number of available quality of service (QoS) credits for the service class during a current scheduling time frame; determining a number of QoS credits associated with a type of the memory operation; and in response to determining that the number of QoS credits associated with the type of the memory operation is less than the number of available QoS credits, submitting the memory operation to the memory device. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The present disclosure will be more fully understood from the detailed description provided below and from the accompanying drawings of various embodiments of the present disclosure. However, the drawings should not be considered to limit the present disclosure to the specific embodiments, but are only for explanation and understanding.

[0007] Figure 1 An example computing system including a memory subsystem in accordance with some embodiments is described.

[0008] Figure 2 A block diagram illustrating a just-in-time (JIT) scheduling component according to one embodiment.

[0009] Figure 3 An example of a two-dimensional array of scheduling pools according to an embodiment is described.

[0010] Figure 4 A block diagram illustrating a Quality of Service (QoS) traffic manager according to an embodiment.

[0011] Figure 5 A block diagram illustrating a memory resource manager according to an embodiment.

[0012] Figure 6 is a flowchart of an example method for scheduling memory operations according to an embodiment.

[0013] Figure 7 is a flowchart of an example method for scheduling memory operations according to an embodiment.

[0014] Figure 8 A block diagram illustrating a delay simulator according to one embodiment.

[0015] Fig. 9 A latency profile according to an embodiment is described.

[0016] Fig.10 An example of a two-dimensional array of scheduling pools according to an embodiment is described.

[0017] Fig.11 A block diagram illustrating a Quality of Service (QoS) traffic manager according to an embodiment.

[0018] Fig.12 is a flowchart of an example method for scheduling memory operations according to an embodiment.

[0019] Fig.13 is a flowchart of an example method for scheduling memory operations according to an embodiment.

[0020] Fig.14 is a block diagram of an example computer system in which embodiments of the present disclosure may operate. DETAILED DESCRIPTION

[0021] Various aspects of the present disclosure are directed to a real-time scheduler for a memory subsystem. The memory subsystem may be a storage device, a memory module, or a combination of a storage device and a memory module. Figure 1 Examples of storage devices and memory modules are described. In general, a host system may utilize a memory subsystem that includes one or more components, such as a memory device, that stores data. The host system may provide data to be stored at the memory subsystem and may request data to be retrieved from the memory subsystem.

[0022] The memory controller of the memory subsystem may perform different types of memory access operations, such as host-initiated memory access operations and background (memory controller-initiated) memory access operations. Background operations may temporarily render the memory subsystem unavailable to the host system. Background operations may include media management operations, such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, and encryption operations, among others. In other cases, prioritizing host-initiated memory access operations may starve background memory access operations. In some cases, multiple memory operations may attempt to access the same die in overlapping time periods, which may cause die contention. In other cases, multiple memory operations may direct their outputs to the same memory buffer, which may cause head-of-line blocking at the memory buffer. Die contention and head-of-line blocking may result in increased memory command latency.

[0023] The example memory subsystem of the present disclosure solves the above and other deficiencies by implementing a JIT scheduling component that manages buffer allocation, data transfer, and memory operation submission from the front end (flash translation layer (FTL) firmware) to the memory subsystem back end (e.g., memory device). The JIT scheduler can process requests from the front end via a cursor. A cursor refers to a module between the front end and the back end that can be used to track the physical memory address of the next block or page to be written to the NAND device. The JIT scheduler can allocate data buffers and communicate with the host system to retrieve data from the host system. The JIT scheduler can use multiple scheduling pools to ensure that no blocking occurs regarding data transfer with the host system or with back end resources. Multiple scheduling pools can prioritize the requests by classifying the requests into one or more business categories. Business classification is the process of classifying business according to various parameters (e.g., the origin or type of business). Some business categories include host reads, host writes, and background operations.

[0024] An example memory subsystem may include a quality of service (QoS) traffic manager that can be used to maintain the number of available QoS credits for each traffic class for any given scheduling time frame. A QoS credit refers to a unit of count of a QoS counting mechanism. For example, when a memory controller processes a memory operation, one or more QoS credits may be used from the available QoS credits of the corresponding traffic class. The QoS credits may be used as a counting mechanism to manage the number of outstanding memory operations per traffic class for any given scheduling time frame without starving lower priority traffic classes. The QoS credits may represent the timing, power, or thermal load of the memory operation. A JIT scheduler may identify a traffic class for a memory operation. The JIT scheduler may request QoS credits from the QoS traffic manager for a traffic class. Based on a response (success or failure) from the request, the JIT scheduler may determine whether the memory operation can be sent to a back-end memory device, or whether the memory operation should remain in the scheduling pool until the next available scheduling time frame. The QoS traffic manager reduces expected host command latency by ensuring that a minimum number of each traffic class is sent to the back-end memory device in a given scheduling time frame without starving lower priority traffic classes.

[0025] The example memory subsystem may include a backend resource manager module. The JIT scheduler may query the backend resource manager to determine whether backend hardware resources (parallelizable hardware memory resources) are available to complete requests for any given time. Parallelizable backend resources refer to memory devices and / or processing units that can be accessed simultaneously by the memory controller.

[0026] The memory resources (or back-end resources) for the memory device may be parallelizable NAND units, such as NAND cells (e.g., NAND dies, NAND planes, blocks, or pages), or the memory resources may be parallelizable controller-specific resources, such as logical command processors (LCPs), etc. The LCP may be implemented by the memory controller for processing incoming commands in one or more data buses. For example, a NAND device with 16 channels may have 16 available LCPs, one for each channel. The back-end resources may be hardware architecture specific. By identifying the back-end resource availability, the back-end resource manager may improve the worst case host command latency by reducing memory resource contention.

[0027] It is desirable to know the impact on system-level performance of various workloads submitted to the JIT scheduling component. An example simulator may include a JIT scheduler and a QoS traffic manager of the JIT scheduling component to simulate the scheduling of different workloads, where the workloads include sequences of incoming commands. The simulator may simulate various expected latencies for various requests for various workloads to tune various default values ​​of QoS credits used by the QoS traffic manager, as discussed further below.

[0028] Various aspects of the above-referenced methods and systems are described in detail below by way of example and not by way of limitation.

[0029] Figure 1 An example computing system 100 is illustrated that includes a memory subsystem 110 according to some embodiments of the present disclosure. Memory subsystem 110 may include media such as one or more volatile memory devices (e.g., memory device 140), one or more non-volatile memory devices (e.g., memory device 130), or a combination of such.

[0030] The memory subsystem 110 may be a storage device, a memory module, or a combination of a storage device and a memory module. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash memory (UFS) drives, secure digital (SD) cards, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and various types of non-volatile dual in-line memory modules (NVDIMMs).

[0031] The computing system 100 may be a computing device, such as a desktop computer, a laptop computer, a network server, a mobile device, a vehicle (e.g., an airplane, a drone, a train, a car, or other transportation), a device with Internet of Things (IoT) capabilities, an embedded computer (e.g., an embedded computer included in a vehicle, industrial equipment, or a networked commercial device), or such a computing device that includes a memory and a processing device.

[0032] The computing system 100 may include a host system 120 coupled to one or more memory subsystems 110. In some embodiments, the host system 120 is coupled to multiple memory subsystems 110 of different types. Figure 1 An example of a host system 120 coupled to one memory subsystem 110 is illustrated. As used herein, "coupled to" or "coupled with" generally refers to a connection between components, which may be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical, optical, magnetic, etc.

[0033] The host system 120 may include a processor chipset and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more caches, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). The host system 120 uses the memory subsystem 110, for example, to write data to the memory subsystem 110 and read data from the memory subsystem 110.

[0034] The host system 120 may be coupled to the memory subsystem 110 via a physical host interface. Examples of the physical host interface include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, a Fibre Channel, a Serial Attached SCSI (SAS), a Double Data Rate (DDR) memory bus, a Small Computer System Interface (SCSI), a Dual In-line Memory Module (DIMM) interface (e.g., a DIMM socket interface supporting Double Data Rate (DDR)), etc. The physical host interface may be used to transfer data between the host system 120 and the memory subsystem 110. When the memory subsystem 110 is coupled to the host system 120 through a physical host interface (e.g., a PCIe bus), the host system 120 may further utilize an NVM Express (NVMe) interface to access components (e.g., the memory device 130). The physical host interface may provide an interface for passing control, address, data, and other signals between the memory subsystem 110 and the host system 120. Figure 1Memory subsystem 110 is illustrated as an example. In general, host system 120 can access multiple memory subsystems via the same communication connection, multiple individual communication connections, and / or a combination of communication connections.

[0035] The memory devices 130, 140 may include any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 140) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0036] Some examples of non-volatile memory devices (e.g., memory device 130) include non-and (NAND) type flash memory and write-in-place memory, such as a three-dimensional cross-point ("3D cross-point") memory device, which is a cross-point array of non-volatile memory cells. The cross-point array of non-volatile memory cells can be combined with a stackable cross-grid data access array to perform bit storage based on changes in body resistance. In addition, compared to many flash-based memories, cross-point non-volatile memory can perform write-in-place operations, where non-volatile memory cells can be programmed without pre-erasing the non-volatile memory cells. NAND type flash memory includes, for example, two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0037] Each of the memory devices 130 may include one or more memory cell arrays. One type of memory cell, such as a single-level cell (SLC), may store one bit per cell. Other types of memory cells, such as multi-level cells (MLC), three-level cells (TLC), four-level cells (QLC), and five-level cells (PLC) may store multiple bits per cell. In some embodiments, each of the memory devices 130 may include one or more memory cell arrays, such as SLC, MLC, TLC, QLC, PLC, or any combination of such. In some embodiments, a particular memory device may include an SLC portion of a memory cell, and an MLC portion, a TLC portion, a QLC portion, or a PLC portion. The memory cells of the memory device 130 may be grouped into pages that may refer to logical units of a memory device for storing data. In the case of some types of memory (e.g., NAND), pages may be grouped to form blocks.

[0038] Although nonvolatile memory components such as a 3D cross-point nonvolatile memory cell array and NAND-type flash memory (e.g., 2D NAND, 3D NAND) are described, the memory device 130 may be based on any other type of nonvolatile memory, such as read-only memory (ROM), phase change memory (PCM), self-selected memory, other chalcogenide-based memory, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin transfer torque (STT)-MRAM, conductive bridging RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), or non-(NOR) flash memory, or electrically erasable programmable read-only memory (EEPROM).

[0039] The memory subsystem controller 115 (or, for simplicity, controller 115) may communicate with the memory device 130 to perform operations, such as reading data, writing data, or erasing data and other such operations at the memory device 130. The memory subsystem controller 115 may include hardware, such as one or more integrated circuits and / or discrete components, buffer memory, or a combination thereof. The hardware may include digital circuitry with dedicated (i.e., hard-coded) logic to perform the operations described herein. The memory subsystem controller 115 may be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or another suitable processor.

[0040] The memory subsystem controller 115 may include a processing device including one or more processors (e.g., processor 117) configured to execute instructions stored in a local memory 119. In the illustrated example, the local memory 119 of the memory subsystem controller 115 includes an embedded memory configured to store instructions for performing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 110, including handling communications between the memory subsystem 110 and the host system 120.

[0041] In some embodiments, local memory 119 may include memory registers that store memory pointers, fetched data, etc. Local memory 119 may also include read-only memory (ROM) for storing microcode. Figure 1 The example memory subsystem 110 in FIG. 1 has been illustrated as including a memory subsystem controller 115, but in another embodiment of the present disclosure, the memory subsystem 110 does not include a memory subsystem controller 115 and may instead rely on external control (e.g., provided by an external host or by a processor or controller separate from the memory subsystem).

[0042] In general, the memory subsystem controller 115 may receive commands or operations from the host system 120 and may convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory device 130. The memory subsystem controller 115 may be responsible for other operations such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical addresses (e.g., logical block addresses (LBA), namespaces) and physical addresses (e.g., physical MU addresses, physical block addresses) associated with the memory device 130. The memory subsystem controller 115 may further include a host interface circuit system to communicate with the host system 120 via a physical host interface. The host interface circuit system may convert commands received from the host system into command instructions to access the memory device 130, and convert responses associated with the memory device 130 into information for the host system 120.

[0043] The memory subsystem 110 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 110 may include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoders and column decoders) that may receive addresses from the memory subsystem controller 115 and decode the addresses to access the memory device 130.

[0044] The memory subsystem 110 may include a JIT scheduling component 113, which can manage buffer allocation, data transfer, and back-end memory operations submitted to the back-end memory device by the flash translation layer (FTL) firmware. The FTL firmware refers to a module that maps the logical block address of the host side to the physical address of the memory device. In some embodiments, the memory subsystem controller 115 includes at least a portion of the JIT scheduling component 113. In some embodiments, the JIT scheduling component 113 is part of the host system 110, the application program, or the operating system. In other embodiments, the local media controller 135 includes at least a portion of the JIT scheduling component 113 and is configured to perform the functions described herein. Other details about the operation of the JIT scheduling component 113 are described below.

[0045] Figure 2 A block diagram illustrating JIT scheduling component 113 according to one embodiment. JIT scheduling component 113 may include JIT scheduler 210, QoS traffic manager 207, memory resource manager 209, and response router 211. Each of modules 207 to 211 may be implemented by software, hardware, or a combination thereof.

[0046] The JIT scheduler 210 can retrieve and schedule incoming requests by business class within the current scheduling time frame, which can be one or more of the clock cycles of the memory subsystem. In one embodiment, the incoming requests are stored in a scheduling pool. In other embodiments, the incoming requests can be stored in a first-in-first-out (FIFO) queue, a first-in-last-out (FILO) queue, or a combination thereof. The FIFO / FILO describes the order of access requests. For a FIFO queue, when a request is received, the request is added to the tail of the queue. The pending request is then taken out from the head of the queue.

[0047] In one embodiment, incoming requests may be stored in a pool of linked lists (single or double linked lists), where each pool has one linked list of incoming requests. Figure 3 , the JIT scheduler 210 stores the requests in a two-dimensional array of scheduling pools 300, where each pool 302 in the array 300 is intended for a specific business class and a specific backend resource. When a request is received from the read / write / erase mailboxes 201 to 203, the JIT scheduler may attempt to retrieve any available node from the free pool 301, fill in the request with the appropriate information (memory operation, business class, memory operation type, etc.), and place the node in the pool in the array 300 according to the business class and backend resource corresponding to the request. The mailbox is a buffer for the front end 200 to place the requests, and the JIT scheduler 210 will eventually retrieve these requests. Here, the nodes are linked to a linked list in the array 300, and the nodes can store information for the request. The linked list can store one or more nodes, each of which contains a reference to the next node in the list. The JIT scheduler 210 can iterate through each pool in the array 300, such as pool [1] [1] ... pool [m] [n], to determine whether a particular pool has any requests and whether the request can be processed. In one embodiment, the JIT scheduler 210 iterates through each pool by round-robin. Round-robin is an arrangement for selecting requests from all pools sequentially in a predetermined order. The predetermined order defines the priority of the business class. For example, the business class host read may have a higher priority than the host write, and the host write may have a higher priority than the background operation. Therefore, the predetermined order may give priority to the host read over the host write, and give priority to the host write over the background operation. In another embodiment, the rotation is a weighted round-robin. In addition to selecting requests according to the weight associated to each pool, the weighted round-robin is similar to the round-robin. If the request in the pool can be processed (that is, if the QoS credit and NAND resources are available, as further discussed below), then the node of the request is removed from the pool, forwarded to the back end (e.g., a memory device), and the node is placed back in the free pool 301. In summary, the array 300 can be used to ensure that when there are requests for different combinations of business classes and back-end resources, requests are submitted from different business class / resource combinations (or selected according to the weights in the weighted round-robin).

[0048] The QoS traffic manager 207 may implement a quality of service (QoS) policy for the request pool. In one embodiment, the implemented QoS policy may manage the requests according to the traffic class of the requests within a scheduling time frame and using a credit-based scheme to avoid starving the lower priority request pool. For example, after initialization, a credit-based scheme may specify a preconfigured number of available QoS credits for each traffic class. The JIT scheduler 210 then consumes QoS credits from a specific traffic class before forwarding the request to the backend. The request may be submitted only when there are the required number of available QoS credits to submit the request, and then the number of available QoS credits is decremented to reflect the submission. In this way, when there are requests for different traffic classes, the submission of requests from different traffic classes is managed by the number of available QoS credits for the traffic class within a given scheduling time frame. As Figure 4 As described in FIG. 4 , in one embodiment, the QoS service manager 207 may include modules such as a reset credit module 401, a get credit module 403, an add credit module 405, a service class credit table 407, and a memory operation type table 409. Tables 407 to 409 may be initialized by the QoS service manager 207 and may be stored in the local memory 119. The service class credit table 407 may specify various service classes, a preconfigured number of credits (default credits), and available credits for the corresponding service class. In one embodiment, the service classes may include host read (JIT_HOST_READ), host write (JIT_HOST_WRITE), and background request (JIT_BACKGROUND_REQUEST). The default credit columns for the corresponding service classes may be DEFAULT_HOST_READ_CREDITS, DEFAULT_HOST_WRITE_CREDITS, and DEFAULT_BACKGROUND_REQUEST_CREDITS, respectively. Default credit is the credit that can set the available credit (a1, a2 and a3) to the beginning of each new scheduling time frame. The available credit column can store a count of the available credits for the service class. The reset credit module 401, the obtain credit module 403 and the add credit module 405 can each modify (reset, subtract and add respectively) the credit from the available credit column.

[0049] The memory operation type table 409 may specify the number of memory operation types and their corresponding credit requirements. For example, the memory operation type may include different operations (read or write) for different types of memory cells. Some example types are: single-level cell (SLC) read (SLC_READ), SLC write (SLC_WRITE), QLC LP write (QLC_LP_WRITE), QLC UPXP write (QLC_UPXP_WRITE), QLC TP write (QLC_TP_WRITE). It should be noted that a quad-level cell (QLC) may have four pages called lower page (LP), upper page (UP), extra page (XP), and top page (TP). Different memory operation types may represent different operations used to access a page of an SLC or a page of a QLC memory cell. Reference Figure 4 , the credits required to complete these operations can be preset as: b1, b2, b3, b4, and b5, respectively. Different memory operation types can have different processing requirements: timing requirements, power requirements, thermal loads, or a combination thereof. Although only three traffic classes and five memory operation types are shown, the QoS traffic manager 207 can specify any number of traffic classes and memory operation types.

[0050] For one example, at the beginning of a scheduling time frame or in response to a power-on event, the QoS traffic manager 207 may set the available credit column of the table 407 to be equal to the default credit column of the table 407 (via the reset credit module 401). The JIT scheduler 210 may decrement some credits from the available credits of a particular traffic class when it submits a memory operation request associated with that traffic class to the backend (via the get credit module 403). The QoS traffic manager 207 then uses the available credit column of the table 407 to determine how many credits are available for the traffic class, and grants the request submission if there are still enough available credits to service the request, or denies the request submission if the number of available credits is less than the credits needed to service the request. If the memory operation subsequently fails for other reasons (e.g., the backend resources are busy, etc.), the add credit module 405 may add the granted credits back to the available credits for the traffic class.

[0051] The back-end resource manager 209 can manage the availability of back-end resources for incoming requests. As previously described, a back-end resource can be a software or hardware parallelizable unit that can be accessed by a memory controller at any given time. Examples of back-end resources include NAND dies, planes, blocks, pages, or logical command processors (LCPs), etc. Back-end resources can be specified based on the architecture of the back-end memory device or memory controller. For example, if the back-end memory device can be accessed by the memory controller in parallel channels, the back-end resource can be specified as an LCP.

[0052] refer to Figure 5 In one embodiment, the backend resource manager 209 includes a get availability module 501, a memory operation completion module 503, and an in-flight memory operation table 505. These modules may be implemented as software-based or hardware-based modules. The get availability module 501 tracks the availability of backend resources and may determine whether a particular backend resource is available. The memory operation completion module 503 tracks the completion status of memory operations and may determine whether a previously submitted memory operation has been completed. The backend resource manager 209 may store a list of previously submitted memory operation requests that will be completed at the in-flight memory operation table 505. The in-flight memory operation table 505 may be stored as Figure 1 The portion of local memory 119 in FIG.

[0053] In one embodiment, the JIT scheduler 210 requests the availability of a given backend resource and memory operation type (via the get availability module 501). The backend resource manager 209 may compare the requested backend resource with the backend resources of the in-flight memory operation to determine the availability of the requested backend resource. Depending on the availability, the backend resource manager 209 may return a pass / fail status code. For example, if the requested backend resource is a specific page of a block, the backend resource manager 209 may compare it with the pages of the in-flight memory operation to determine whether the specific page is currently being accessed.

[0054] In another example, if the requested backend resource is a specific LCP, the backend resource manager 209 can compare the LCPs used by the in-flight memory operations to determine whether the specific LCP is available. In another example, if the number of outstanding memory operations per LCP is below a predetermined threshold, the LCP is available. Thus, the backend resource manager 209 can manage the availability of backend resources in a manner that does not impair the performance of the memory subsystem.

[0055] In another embodiment, the backend resource manager 209 implements a prediction model to predict whether the LCP is available. The prediction model may base the prediction on one or more parameters of the last memory operation (e.g., submission time and type). For example, if the last memory operation was submitted t nanoseconds ago and is a memory operation type that is expected to complete within t nanoseconds, then the last memory operation is considered complete. Therefore, the current memory operation request may be submitted before the completion of the last memory operation is confirmed by the backend resource manager 209 via the response router 211. Therefore, this prediction model can be used to keep the pipeline of the backend more efficiently loaded.

[0056] refer to Figure 2 , operations 1 to 10 are described with respect to an example scheduling process flow according to one embodiment.

[0057] Operation 1. The JIT scheduler 210 receives read / write / erase requests from the NAND cursor 205. The request may be a host read, host write, or background memory operation received from the front end 200. The background memory operation may be a background read, background write, or background erase operation. Background operations may represent media management operations initiated by the memory controller to maintain predefined data state metrics of the memory device within a specified range. The NAND cursor 205 may receive these requests from the read mailbox 201, the write mailbox 202, or the erase mailbox 203. In another embodiment, the JIT scheduler 210 may receive memory operation requests directly from the mailboxes 201 to 203. The requested memory operation may utilize a logical or physical address. For commands utilizing a logical address, the JIT scheduler 210 may request the NAND cursor 205 to translate the logical address into a corresponding physical address using a logical-to-physical translation table from the data flash manager 204. The NAND cursor 205 then sends the physical address back to the JIT scheduler 210. Mailboxes 201-203 may be memory buffers that store requested command operations.

[0058] Operation 2. The JIT scheduler 210 manages the incoming requests from the NAND cursor 205. For example, the JIT scheduler 210 may store the incoming requests in a two-dimensional (m×n) array of the scheduling pool according to the business class and backend resources corresponding to the incoming requests, such as Figure 3 Each pool corresponds to a specific combination of business classes and backend resources. The JIT scheduler 210 may use round-robin or weighted round-robin to select a round of requests from the scheduling pool for submission to the backend 230. For example, for one scheduling time frame, the JIT scheduler 210 may traverse the m×n array of scheduling pools in a predetermined order (e.g., traversing the pools in a specified dimension first).

[0059] Operation 3. For an incoming request, the JIT scheduler 210 may retrieve credits from the QoS service manager 207. For example, based on the requested service class and the type of memory operation, the JIT scheduler 210 determines whether there are enough credits to service the request. The QoS service manager 207 returns a success or failure status based on the determination. Upon receiving a failure status, the incoming request is placed back into the scheduling pool for submission in a future scheduling time frame. For a success status, the JIT scheduler 210 proceeds to check whether the backend is available for the request.

[0060] Operation 4. The JIT scheduler 210 may send a request to the backend resource manager 209 to check the availability of the backend resources for servicing the request. For example, the backend resource manager 209 may determine whether any of the outstanding requests is using the requested backend resources. Here, the backend resource manager 209 may track outstanding (or in-flight) requests and compare the outstanding requests with the current requests. If the comparison produces a match, for example, the backend resource corresponding to the outstanding request is the requested backend resource, then the requested backend resource may be considered unavailable. In one embodiment, for LCP as a backend resource, the backend resource manager 209 may compare the FPA corresponding to the outstanding request with the FPA corresponding to the current request for overlap. The backend resource manager 209 may determine that the backend resource is available in the absence of overlap. In another embodiment, if the number of outstanding requests for LCP is below a certain threshold, the backend resource manager 209 considers the backend resource to be available. In another embodiment, the backend resource manager 209 predicts whether the backend resource is available by analyzing the last request submitted. The backend resource manager 209 returns a status code reflecting the availability of the specified resource.

[0061] If the specified backend resource is not available, then the JIT scheduler 210 returns a credit to the QoS traffic manager 207 (via the add credit module 405) and appends the request back to the scheduling pool to be processed at a future time. Otherwise (i.e., if the specified backend resource is available), the backend resource manager 209 stores the request in the in-flight memory operation table (e.g., Figure 5 Table 505) and marks the request as an in-flight request.

[0062] Operation 5. If the request is a host write, the JIT scheduler 210 allocates a data buffer for the request and waits for the host data transfer engine 220 to provide data. Other request types proceed to Operation 8.

[0063] Operation 6. For a host write request, the JIT scheduler 210 may request data from the front end 200 via the host data transfer engine 220. The data request may be asynchronous, so that the JIT scheduler 210 may process the next request concurrently with processing the current request.

[0064] Operation 7. The host data transfer engine 220 may store the requested data to the allocated data buffer within the JIT scheduler 210 , and send a transfer completion response to the JIT scheduler 210 .

[0065] Operation 8. Once the request is ready to be submitted, it is submitted to the backend 230.

[0066] Operation 9. The response router 211 receives an operation completion response from the backend 230 .

[0067] Operation 10. The response router 211 routes the response to the backend resource manager 209 and / or the frontend 200 to notify the backend resource manager 209 and / or the frontend 200 that the operation is completed.

[0068] Figure 6 6 is a flowchart of an example method 650 for scheduling memory operation requests according to some embodiments of the present disclosure. The method 650 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 650 is performed by Figure 1 The JIT scheduling component 113 or Figure 2 The JIT scheduler 210 of FIG. 210 is performed. Although shown in a specific order or sequence, unless otherwise specified, the order of the processes may be modified. Therefore, it should be understood that the illustrated embodiments are only examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in each embodiment. Other process flows are possible.

[0069] At operation 651, processing logic receives a request to perform a memory operation using hardware resources associated with a memory device. In one embodiment, processing logic enqueues the request in an inbound queue. The inbound queue may be represented by an (m×n) array of scheduling pools. Each scheduling pool in the (m×n) array of scheduling pools corresponds to a specific hardware resource and a specific traffic class. The scheduling pool may be implemented as a linked list (single linked list or double linked list), where information for the request may be stored in a node in the linked list. During a scheduling time frame, processing logic may select one or more rounds of requests from the scheduling pool for processing according to a predetermined order by round robin or weighted round robin.

[0070] At operation 653, processing logic identifies a traffic class corresponding to the memory operation. For example, the traffic class may be one of: host read, host write, and background operation.

[0071] At operation 655, processing logic determines the number of available quality of service (QoS) credits for a traffic class during the current scheduling time frame. At the beginning of the scheduling time frame, or when the memory subsystem is powered on, a data buffer (or table) storing available credits for a traffic class is assigned credits equal to its default credits. Each traffic class has a predetermined number of default credits. When processing logic obtains the number of QoS credits for memory operations from the available QoS credits, the available QoS credits are decremented.

[0072] At operation 657, processing logic determines the number of QoS credits associated with the type of memory operation. Each memory operation corresponds to a memory operation type. Each memory operation type is associated with a predetermined number of credits to complete the type of memory operation. Some example types may be: SLC read, SLC write, QLC LP write, QLC UPXP write, and QLC TP write. Here, a quad-level cell (QLC) may have four pages referred to as a lower page (LP) and an upper page (UP, XP, and TP). Different memory operation types may have different processing requirements: timing latency, power requirements, thermal load, or a combination thereof. QoS credits may represent these processing requirements. Therefore, different memory operation types may be associated with different QoS credits.

[0073] At operation 659, in response to determining that the number of QoS credits associated with the type of memory operation is less than the number of available QoS credits, processing logic submits the memory operation to be processed at the memory device during the current scheduling time frame and decrements the available QoS credits by the number of QoS credits associated with the memory operation. If the number of QoS credits associated with the memory operation is greater than the available QoS credits, then processing logic places the request back into a scheduling pool (e.g., a queue) for requests to be processed at a later scheduling time frame. That is, processing logic can manage the number of memory operations of a business class that are submitted within a scheduling time frame. Available QoS credits provide a limit on how many memory operations of a business class can be submitted within a scheduling time frame to avoid starvation of lower priority requests.

[0074] The processing logic may further determine the availability of the hardware resource before submitting the request to the memory device. The processing logic may use information of previously submitted but not yet completed in-flight (incomplete) memory operations to determine the hardware resource availability. In one embodiment, based on a statistical model, using information for in-flight requests, the process logic may determine a count of incomplete requests for the hardware resource. If the count is below a predetermined threshold, the processing logic determines that the hardware resource is available and the processing logic submits the request to the memory device. It should be noted that the hardware resource may be a NAND die, a NAND plane, a NAND block, a NAND page, an LCP, etc.

[0075] The processing logic may also use the prediction model to determine the availability of the hardware resource. The prediction model may predict when the last submitted request associated with the hardware resource will be completed based on the information of the last submitted request. The information of the last submitted request may include: the type of memory operation last submitted and the submission timestamp of the last submission. Based on this information, the processing logic may predict when the memory operation will be completed.

[0076] Figure 7700 is a flowchart of an example method 700 for scheduling memory operation requests according to some embodiments of the present disclosure. The method 700 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 700 is performed by Figure 1 The JIT scheduling component 113 or Figure 2 The JIT scheduler 210 of FIG. 210 is performed. Although shown in a specific order or sequence, unless otherwise specified, the order of the processes may be modified. Therefore, it should be understood that the illustrated embodiments are only examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in each embodiment. Other process flows are possible.

[0077] At operation 701, processing logic receives a request to perform a memory operation using hardware resources associated with a memory device. At operation 703, processing logic identifies a business class corresponding to the memory operation. Some example business classes may be: host read, host write, and background operation. At operation 705, processing logic determines a type of memory operation corresponding to the memory operation. Some example types may be: SLC read, SLC write, QLC LP write, QLC UPXP write, and QLCTP write.

[0078] At operation 707, processing logic determines whether the memory operation is to be processed during the current scheduled time frame based on the traffic class and the type of memory operation. For example, processing logic determines the available QoS credits for the traffic class and the required QoS credits to complete the request for the type of memory operation. If there are more available QoS credits than the required QoS credits, the memory operation is to be processed during the current scheduled time frame, and processing logic decrements the available QoS credits. Otherwise, the memory operation is queued for processing at a future scheduled time frame. At operation 709, in response to determining that the memory operation is to be processed during the current scheduled time frame, processing logic commits the memory operation to be processed at the memory device.

[0079] The scheduling of memory operations can be simulated before deployment. For example, a latency simulator can simulate the system-level performance of various workloads submitted to the JIT scheduler by simulating the expected latency of memory operations for each business class and memory operation type. Information based on the business class and memory operation type of the expected latency for memory operations can be used to simulate the performance of the JIT scheduler. This simulator can estimate the system-level performance impact that can be generated by using different NAND types and background operation algorithms. Therefore, the amount of available QoS credits for each business class and the QoS credits required to complete certain memory operations can be further adjusted by the operator for the required scheduling latency for various workloads.

[0080] Figure 8 A block diagram illustrating a latency simulator according to one embodiment. The latency simulator 800 may simulate a latency model of a JIT scheduler. The simulator 800 may include an input module 801, a JIT scheduler 810, a QoS service manager 807, and a response router / output 811. Each of the modules 801 to 811 may be a hardware-based or software-based module of the latency simulator.

[0081] The input module 801 may receive the expected delay profile, for example, via an input / output (I / O) interface or a graphical user interface (GUI). Fig. 9 As described in the configuration file 809 of , the expected latency profile represents the expected time period for processing the memory operation according to the memory operation type of the memory operation of various business classes. The expected latency profile can reflect the characteristics of a specific NAND device and how long it takes for the memory operation to perform the type of memory operation on the NAND device. The expected latency profile can be in the form of a binary metadata file, a text file, or a spreadsheet.

[0082] The input module 801 may receive a workload list, which may specify a list of memory operations of various business classes that may be forwarded to the JIT scheduler 810 at various times to test the performance of the JIT scheduler 810. An example workload list may include 10 SLC read memory operations to be submitted at scheduled time frame 1; 20 SLC write memory operations to be submitted at scheduled time frame 2, etc. A workload profile may specify repetition of specific memory operations, or may have a unique set of memory operations for each scheduled time frame. The workload list may be in the form of a binary metadata file, a text file, or a spreadsheet generated by an operator.

[0083] In one embodiment, the input module 801 includes a read / write / erase buffer (not shown) of the JIT scheduler 810, where various types of memory operations (read, write, erase) are stored in the buffer. The JIT scheduler 810 can then retrieve the memory operations from the read / write / erase buffer for processing. Figure 2 The read / write / erase mailbox functionality.

[0084] The JIT scheduler 810 may schedule memory operations at various scheduling time frames specified by the workload list. In some embodiments, the JIT scheduler 810 may (similar to Figure 2 The operation of the JIT scheduler 210 of the embodiment of the present invention) uses a first-in-first-out (FIFO) queue, a first-in-last-out (FIFO), a pool, or a combination thereof to schedule memory operations. In one embodiment, the operation of the JIT scheduler 210 of the embodiment of the present invention may be performed in a manner such as Fig.10 The scheduling pool 1000 described in the above description may schedule memory operations. The scheduling pool 1000 may include one or more linked lists (single linked lists or double linked lists). In one embodiment, similar to Figure 3 The scheduling pool 300 of the embodiment of the present invention is shown in FIG. 3 , wherein the pool 1000 is a two-dimensional array of scheduling pools, wherein each pool 1002 in the pool 1000 is intended for a specific business class and a specific simulated backend resource, for example, business class 1 ... m and simulated backend resource 1 ... n. In one embodiment, the simulated backend resource 1 ... n is simulated to be always available.

[0085] In one example, when a memory operation is received from input module 801, JIT scheduler 810 may retrieve a node from free pool 1003, add appropriate information from the memory operation to the node, and place the node into one of pools 1000 according to the business class corresponding to the memory operation and the simulated backend resources. JIT scheduler 810 may iterate through each pool in pool 1000, such as pool [1] [1] ... pool [m] [n], to determine whether a particular pool has any memory operations, and whether the memory operations can be processed. In one embodiment, JIT scheduler 810 iterates through each pool in a predetermined order by round robin. In another embodiment, JIT scheduler 810 iterates through each pool in a predetermined order by weighted round robin. The predetermined order may define priorities for various business classes. For example, a business class of host reads may have a higher priority than host writes, etc. If the memory operation in the pool can be processed (i.e., if QoS credits are available), the node for the memory operation is removed from the pool, forwarded to response router 811, and the node is placed back into free pool 1001. Thus, pool 1000 may be used to ensure that memory operations are selected among different traffic classes according to their portion of available QoS credits (or selected according to weights based on a weighted round robin algorithm).In some embodiments, it is assumed that the simulated backend resources are available.

[0086] refer to Figure 8 , the QoS traffic manager 807 may implement a quality of service (QoS) policy for the scheduling pool, similar to Figure 2 QoS business manager 207. In one embodiment, the implemented QoS policy may manage the memory operations according to their business class within the scheduling time frame using a QoS credit-based scheme to avoid starving the lower priority request pool. For example, after initialization, the credit-based scheme may specify a preconfigured number of available QoS credits for each business class. The JIT scheduler 810 then consumes QoS credits from a specific business class before forwarding the memory operation to the response router 811. The memory operation can only be submitted when there are the required number of available QoS credits to submit the memory operation, and then the number of available credits is decremented to reflect the submission. In this way, when there are memory operations of different business classes, the submission of memory operations from different business classes is managed by the number of available credits for the business class within a given scheduling time frame. As Fig.11 As described in FIG. 1 , in one embodiment, the QoS traffic manager 807 may include modules such as a reset credit module 401, an obtain credit module 403, a memory operation type table 409, and a traffic class credit table 1101. Tables 409 and 1101 may be initialized by the QoS traffic manager 807 and may be stored in a memory. The traffic class credit table 1101 may specify various traffic classes, a preconfigured number of credits (default credits), and available credits for the corresponding traffic classes. In one embodiment, the traffic classes may include host reads (JIT_HOST_READ), host writes (JIT_HOST_WRITE), and background memory operations (JIT_BACKGROUND_READ, JIT_BACKGROUND_WRITE, and JIT_BACKGROUND_ERASE). The default credit columns for the corresponding service classes may be DEFAULT_HOST_READ_CREDITS, DEFAULT_HOST_WRITE_CREDITS, DEFAULT_BACKGROUND_READ_CREDITS, DEFAULT_BACKGROUND_WRITE_CREDITS, and DEFAULT_BACKGROUND_ERASE_CREDITS, respectively. The default credits are credits that may set the available QoS credits (a1, a2, a3, ...) to the beginning of each new scheduling time frame. The available credit column may store a count of available QoS credits for the service class. The reset credit module 401 and the obtain credit module 403 may each modify (reset and subtract, respectively) the credits from the available credit column.

[0087] The memory operation type table 409 may specify the number of memory operation types and their corresponding QoS credit requirements. For example, the memory operation types may include different operations (read, write, or erase) for different types of memory cells. Some example types are: single-level cell (SLC) read (SLC_READ), SLC write (SLC_WRITE), QLC LP write (QLC_LP_WRITE), QLC UPXP write (QLC_UPXP_WRITE), QLC TP write (QLC_TP_WRITE). It should be noted that a quad-level cell (QLC) may have four pages called lower page (LP), upper page (UP), extra page (XP), and top page (TP). Different memory operation types may represent different operations for accessing a page of an SLC or a page of a QLC memory cell. Examples of the credits required to complete these operations may be preset to values: b1, b2, b3, b4, and b5, respectively. Different memory operation types may have different processing requirements: timing requirements, power requirements, thermal load, or a combination thereof. Although only three traffic classes and five memory operation types are shown, QoS traffic manager 807 may specify any number of traffic classes and memory operation types.

[0088] For one example, at the beginning of a scheduling time frame, the QoS traffic manager 807 may set the Available Credits column of the table 1101 equal to the Default Credits column of the table 1101 (via the Reset Credits module 401). The JIT scheduler 810 may decrement some credits from the Available Credits for a particular traffic class (via the Get Credits module 403) when it submits a memory operation request associated with that traffic class to the response router 811. The QoS traffic manager 807 then uses the Available Credits column of the table 1101 to determine how many credits are available for the traffic class, and grants the memory operation submission if there are still enough available credits to service the memory operation, or denies the memory operation submission if the number of available credits is less than the credits needed to service the memory operation.

[0089] The response router / output 811 may simulate a receiver for memory operation submissions submitted from the JIT scheduler 810. The response router / output 811 may retrieve the expected latency of the memory operation based on the business class of the memory operation and the memory operation type. The response router / output 811 may indicate the memory operation as serviced after a time equal to the expected latency that has elapsed from submission by passing a memory operation completion notification back to the JIT scheduler 810. The response router / output 811 may output a file specifying some statistics of the performance of the JIT scheduler. The output may include the worst case latency of the scheduling time frame, how many scheduling time frames are used for the workload, etc. The output file may also include a graph that can be used to analyze potential bottlenecks in the scheduling time frame, die contention.

[0090] refer to Figure 8 , operations 11 to 15 are described with respect to an example simulation according to one embodiment.

[0091] Operation 11. The JIT scheduler 810 retrieves read / write / erase memory operations at various times simulated from the input module 801. The memory operation may be a host read, host write, or background memory operation business class. The background memory operation may be a background read, background write, or background erase operation. The memory operation may include associated business class data and / or simulated backend resource data. The memory operation may be part of a profile (e.g., a workload profile) of a workload provided to the input module 801 from an operator, wherein the workload profile may include various read / write / erase memory operations at various scheduling time frames to simulate workloads handled by the JIT scheduler. For example, the JIT scheduler 810 may store incoming memory operations in a two-dimensional (m×n) scheduling pool according to the business class and the simulated backend resources.

[0092] Operation 12. The JIT scheduler 810 manages incoming memory operations from the input module 801. For example, the JIT scheduler 810 may store incoming memory operations in a two-dimensional (m×n) scheduling pool according to the business class and the simulated backend resources. The JIT scheduler 210 may select memory operations from the scheduling pool using round-robin or weighted round-robin. For example, during a given scheduling time frame, the JIT scheduler 810 may traverse the m×n scheduling pool one or more times to retrieve memory operations from these pools.

[0093] Operation 13. To process the memory operation specified by the incoming request, the JIT scheduler 810 may retrieve available QoS credits from the QoS service manager 807. For example, the memory operation type is completed according to the requested service class and the required QoS credits. The JIT scheduler 210 may determine whether there are enough credits to service the request, for example, if the available QoS credits are greater than the required QoS credits, then there are enough credits.

[0094] Operation 14. The QoS service manager 807 returns a success or failure status based on the determination. Upon receiving a failure status, the simulated request is placed back into the scheduling pool to be submitted in a future scheduling time frame. For a success status, the JIT scheduler 810 proceeds to operation 15.

[0095] Operation 15. The JIT scheduler 810 may submit the memory operation to the response router 811. The response router 811 may return a status of the simulated memory operation completion after the expected latency of the memory operation has elapsed.

[0096] By simulating the timing of the workload of different memory operations, output 811 can generate an output of latency statistics indicating the performance of the workload. The output can include the worst-case latency period of the scheduling time frame, and how many scheduling time frames are needed to complete a specific workload, etc. For example, output 811 can calculate the worst-case latency of the scheduling time frame by calculating the latency of each business class. Output 811 can calculate the business class latency for the business class by adding the memory operation latency together for the memory operation according to the business class. Output 811 can then apply the maximum function to different business class latency to find which business class has the worst latency within the scheduling time frame. In another embodiment, output 811 calculates the worst-case latency of the scheduling time frame by the following operations: determining the average latency of the memory operations in the business class, and calculating the worst-case latency by multiplying the average latency by the number of memory operations of the business class within the scheduling time frame.

[0097] In one embodiment, the number of frames for a workload may be calculated based on the number of scheduled time frames required to complete the memory operations in the workload profile. In another embodiment, the number of scheduled time frames for a workload is estimated based on the number of memory operations in the workload.

[0098] Simulating different workloads allows analyzing the latency of the JIT scheduler using different available QoS credits and required QoS credits for various types of memory operations. The analysis can provide the worst case latency for various workloads.

[0099] Fig.12 1200 is a flowchart of an example method for scheduling memory operations according to an embodiment. The method 1200 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 1200 is performed by Figure 8 The simulator 800 of FIG. 1 is performed. Although shown in a particular order or sequence, unless otherwise specified, the order of the processes may be modified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in each embodiment. Other process flows are possible.

[0100] At operation 1201, processing logic receives a workload profile specifying a sequence of memory operations, where each memory operation is associated with a type of memory operation. An example memory operation may be an SLC read, address P1. The memory operation is associated with a service class (e.g., host read) and belongs to a memory operation type (e.g., SLC read).

[0101] At operation 1203, processing logic identifies a traffic class associated with each memory operation in the sequence of memory operations. The traffic class may include at least one of: host read, host write, background read and background write, and background erase.

[0102] At operation 1205, processing logic enqueues each memory operation in the sequence of memory operations in a scheduling pool of a plurality of scheduling pools based on a traffic class associated with the memory operation. The scheduling pool may be a two-dimensional array of pools, where each pool includes a linked list, and each link in the linked list represents a memory operation request.

[0103] At operation 1207, processing logic selects one or more memory operations from a plurality of scheduling pools to be serviced within a scheduling time frame based on a quality of service (QoS) policy. The selecting may further include: determining a number of available quality of service (QoS) credits for a traffic class for the scheduling time frame; determining a number of QoS credits associated with the memory operation based on a memory operation type of the memory operation; and in response to determining that the number of QoS credits associated with the memory operation is less than the number of available QoS credits for the traffic class, subtracting the number of QoS credits from the available QoS credits and indicating that the memory operation has been serviced. The memory operation type may include at least one of: a single level cell (SLC) read, an SLC write, a quad level cell (QLC) lower page (LP) write, a QLC upper page (UP) write, a QLC extra page (XP) write, and a QLC top page (TP) write.

[0104] At operation 1209, the process logic determines a latency period for each of the one or more memory operations based on the latency profile. In one embodiment, the processing logic further determines a memory operation latency period for each of the one or more memory operations based on the latency profile. The processing logic determines a service class latency period by summing the memory operation latency periods of the memory operations corresponding to the service class. The processing logic determines a latency period for the scheduling time frame by selecting a maximum service class latency period among service class latency periods of a plurality of service classes. The latency profile may include a plurality of latency periods, wherein each of the plurality of latency periods corresponds to a service class and a memory operation type.

[0105] In one embodiment, the processing logic further determines the number of scheduled time frames required to service the sequence of memory operations. The processing logic determines the total latency period of the number of scheduled time frames. For example, the total latency period may be equal to the latency period of the scheduled time frame multiplied by the number of scheduled time frames.

[0106] Fig.13 1300 is a flowchart of an example method for scheduling memory operations according to an embodiment. Method 1300 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, method 1300 is performed by Figure 8 The simulator 800 of FIG. 1 is performed. Although shown in a particular order or sequence, unless otherwise specified, the order of the processes may be modified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in each embodiment. Other process flows are possible.

[0107] At operation 1301, processing logic receives a memory operation to be simulated. An example memory operation may be an SLC read, address P1. The memory operation is associated with a service class (eg, host read) and is of a memory operation type (eg, SLC read).

[0108] At operation 1303, processing logic determines a traffic class corresponding to the memory operation. The traffic class includes at least one of: host read, host write, background read and background write, and background erase.

[0109] At operation 1305, processing logic determines a number of quality of service (QoS) credits associated with the memory operation based on a memory operation type of the memory operation. The memory operation type includes at least one of: a single level cell (SLC) read, an SLC write, a quad level cell (QLC) lower page (LP) write, a QLC upper page (UP) write, a QLC extra page (XP) write, and a QLC top page (TP) write.

[0110] At operation 1307, processing logic determines the number of available QoS credits for the traffic class for the scheduled time frame.

[0111] At operation 1309, in response to determining that the number of QoS credits associated with the memory operation is less than the number of available QoS credits for the traffic class, processing logic subtracts the number of QoS credits from the available QoS credits to indicate that the number of QoS credits is no longer available for the traffic class. At operation 1311, processing logic indicates that the memory operation has been serviced.

[0112] In one embodiment, processing logic determines a latency period for a memory operation based on a latency profile, and marks the memory operation as serviced after the latency period for the memory operation has elapsed. In another embodiment, processing logic determines a memory operation latency period for each of one or more memory operations based on the latency profile. Processing logic determines a service class latency period by summing the memory operation latencies of the memory operations corresponding to the service class. Processing logic determines a latency period for a scheduling time frame by selecting a maximum service class latency period for a plurality of service classes. The latency profile includes a number of latency periods, wherein each latency period in the plurality of latency periods corresponds to a service class and a memory operation type.

[0113] In one embodiment, processing logic enqueues the memory operation in one of a plurality of scheduling pools based on the business class of the memory operation, and the processing logic selects the memory operation from the plurality of scheduling pools by round robin or weighted round robin for the scheduling time frame. In one embodiment, processing logic determines the number of scheduling time frames required to service the plurality of memory operations. Processing logic determines a total latency period for the number of scheduling time frames based on a latency profile.

[0114] Fig.14 An example machine illustrating a computer system 600 may be executed within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system 600 may correspond to a host system (e.g., Figure 1 1) a host system 120 that includes, is coupled to, or utilizes a memory subsystem (e.g., Figure 1 The memory subsystem 110 of the controller may be used to perform the operation of the controller (for example, execute the operating system to perform the corresponding Figure 1 The JIT scheduling component 113 or Figure 8 In some embodiments, the machine may be connected (e.g., using a network) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine may operate in the capacity of a server or a client machine in a client-server network environment, or in the capacity of a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0115] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by the machine. In addition, while a single machine is described, the term "machine" should also be taken to include any collection of machines that individually or collectively execute a set (or multiple sets of instructions) to perform any one or more of the methodologies discussed herein.

[0116] The example computer system 600 includes a processing device 602, a main memory 604 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM) or RDRAM, etc.), a static memory 606 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 618, which communicate with each other via a bus 630.

[0117] The processing device 602 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. The processing device 602 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processing device 602 is configured to execute instructions 626 for performing the operations and steps discussed herein. The computer system 600 may further include a network interface device 608 to communicate over a network 620.

[0118] The data storage system 618 may include a machine-readable storage medium 624 (also referred to as a computer-readable medium) on which is stored one or more sets of instructions 626 or software embodying any one or more of the methodologies or functions described herein. The instructions 626 may also reside, completely or at least partially, within the main memory 604 and / or within the processing device 602 during execution thereof by the computer system 600, the main memory 604 and the processing device 602 also constituting machine-readable storage media. The machine-readable storage medium 624, the data storage system 618, and / or the main memory 604 may correspond to Figure 1 Memory subsystem 110.

[0119] In one embodiment, instructions 626 include implementing a block selection component (e.g., Figure 1 The JIT scheduling component 113 or Figure 8The term "machine-readable storage medium" may be a medium that stores or encodes a set of instructions for execution by a machine and causes the machine to perform any one or more of the methods of the present disclosure. Thus, the term "machine-readable storage medium" may be a medium that stores or encodes a set of instructions for execution by a machine and causes the machine to perform any one or more of the methods of the present disclosure.

[0120] Some portions of the previous detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively convey the substance of their work to others skilled in the art. Here, and in general, an algorithm is conceived to be a self-consistent sequence of operations that produces a desired result. Operations are those requiring physical manipulation of physical quantities. Typically, but not necessarily, these quantities are in the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. At times, it has proven convenient, primarily for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0121] It should be borne in mind, however, that all of these and similar terms should be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities within the computer system's registers and memories and transforms that data into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0122] The present disclosure also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the intended purpose, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magnetic optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0123] The algorithms and displays presented herein are not inherently related to any particular computer or another device. Various general purpose systems may be used with programs according to the teachings herein, or it may prove convenient to construct a more specialized device to perform the method. The structures for various these systems will be presented as described below. In addition, the present disclosure is described without reference to any particular programming language. It should be appreciated that the teachings of the present disclosure as described herein may be implemented using various programming languages.

[0124] The present disclosure may be provided as a computer program product or software, which may include a machine-readable medium having stored thereon instructions that can be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory component, etc.

[0125] In the foregoing description, embodiments of the present disclosure have been described with reference to specific example embodiments thereof. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of embodiments of the present disclosure as set forth in the appended claims. Accordingly, the present description and drawings should be viewed in an illustrative rather than a restrictive sense.

Claims

1. A system comprising: Memory device; as well as a processing device operatively coupled to the memory device to perform operations including: receiving, by a real-time JIT scheduler, a request to perform a memory operation using hardware resources associated with the memory device; identifying a service class corresponding to the memory operation; determining a number of available quality of service (QoS) credits for the traffic class during a current scheduling time frame; determining a number of QoS credits associated with the type of memory operation; determining that the number of QoS credits associated with the type of the memory operation is less than the number of available QoS credits; predicting that the hardware resource is available based on a type of a previous memory operation in response to a previous request and a time when the previous memory operation was submitted; as well as The memory operation is submitted to the memory device.

2. The system of claim 1 , wherein receiving the request further comprises: Enqueuing the request in a scheduling pool among a plurality of scheduling pools, wherein the scheduling pool is associated with the service class and the hardware resource; as well as The request is dequeued from the scheduling pool of the plurality of scheduling pools using a round-robin algorithm. 3 . The system of claim 1 , wherein committing the memory operation is performed in response to determining that a count of outstanding requests for the hardware resource is below a predetermined threshold.

4. The system of claim 1, wherein the operations further comprise: In response to determining that the number of QoS credits associated with the memory operation exceeds the available QoS credits, enqueuing the request to be processed at a subsequent scheduled time frame.

5. The system of claim 1, wherein the operations further comprise: The number of available QoS credits for the traffic class is reset at a subsequent scheduling time frame. 6 . The system of claim 1 , wherein the traffic class is one of: host read, host write, and background operation.

7. A non-transitory machine-readable storage medium comprising instructions that, when accessed by a processing device, cause the processing device to perform one or more operations comprising: receiving, by the real-time JIT scheduler, a request to perform a memory operation using hardware resources of the memory device associated with the memory device; identifying a traffic class corresponding to the memory operation; determining a number of available quality of service (QoS) credits for the traffic class during a current scheduling time frame; determining a number of QoS credits associated with the type of memory operation; determining that the number of QoS credits associated with the type of the memory operation is less than the number of available QoS credits, predicting that the hardware resource is available based on a type of previous memory operation in response to a previous request and a time when the previous memory operation was submitted; as well as The memory operation is submitted to the memory device.

8. The non-transitory machine-readable storage medium of claim 7, wherein receiving the request further comprises: Enqueuing the request in a scheduling pool among a plurality of scheduling pools, wherein the scheduling pool is associated with the service class and the hardware resource; as well as The request is dequeued from the scheduling pool of the plurality of scheduling pools using a round-robin algorithm.

9. The non-transitory machine-readable storage medium of claim 7, wherein committing the memory operation is performed in response to determining that a count of outstanding requests for the hardware resource is below a predetermined threshold.

10. The non-transitory machine-readable storage medium of claim 7, wherein the operations further comprise: In response to determining that the number of QoS credits associated with the memory operation exceeds the available QoS credits, enqueuing the request to be processed at a subsequent scheduled time frame.

11. The non-transitory machine-readable storage medium of claim 7, wherein the operations further comprise: The number of available QoS credits for the traffic class is reset at a subsequent scheduling time frame.

12. The non-transitory machine-readable storage medium of claim 7, wherein the traffic class is one of: host read, host write, and background operation.

13. A method comprising: receiving, by a real-time JIT scheduler, a request to perform a memory operation using hardware resources associated with a memory device; identifying a traffic class corresponding to the memory operation; determining a number of available quality of service (QoS) credits for the traffic class during a current scheduling time frame; determining a number of QoS credits associated with the type of memory operation; determining that the number of QoS credits associated with the type of the memory operation is less than the number of available QoS credits, predicting that the hardware resource is available based on a type of previous memory operation in response to a previous request and a time when the previous memory operation was submitted; as well as The memory operation is submitted to the memory device.

14. The method of claim 13, wherein receiving the request further comprises: Enqueuing the request in a scheduling pool among a plurality of scheduling pools, wherein the scheduling pool is associated with the service class and the hardware resource; as well as The request is dequeued from the scheduling pool of the plurality of scheduling pools using a round-robin algorithm.

15. The method of claim 13, wherein committing the memory operation is performed in response to determining that a count of outstanding requests for the hardware resource is below a predetermined threshold.

16. The method of claim 13, wherein the operations further comprise: In response to determining that the number of QoS credits associated with the memory operation exceeds the available QoS credits, enqueuing the request to be processed at a subsequent scheduled time frame.

17. The method of claim 13, wherein the operations further comprise: The number of available QoS credits for the traffic class is reset at a subsequent scheduling time frame.

18. The method of claim 13, wherein the traffic class is one of: host read, host write, and background operation.

Images