Managing data dependencies in hybrid DIMMs for out-of-order processing

Abstract

Systems and methods are disclosed that include a processing device operably coupled to first and second memory devices. The processing device may receive a set of data access requests from a host system in a first order and execute the set of data access requests in a second order. The processing device may further identify a delayed data access request in the set of data access requests and determine whether a data structure in a local memory associated with the processing device includes a previously uncompleted data access request corresponding to an address associated with the delayed data access request. In response to determining that the data structure includes an indication of a previously uncompleted data access request corresponding to the address associated with the delayed data access request, a type of data dependency associated with the previously uncompleted data access request is identified and one or more operations associated with the type of data dependency are performed.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to memory subsystems, and more particularly, to managing data dependencies in a hybrid dual in-line memory module (DIMM) for out-of-order processing. Background Art

[0002] The memory subsystem may include one or more memory devices that store data. The memory devices may be, for example, non-volatile memory devices and volatile memory devices. In general, the host system may utilize the memory subsystem to store data at the memory devices and retrieve data from the memory devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The present disclosure will be more fully understood from the detailed description given below and the accompanying drawings of various embodiments of the present disclosure.

[0004] Figure 1 An example computing system including a memory subsystem according to some embodiments of the present disclosure is described.

[0005] Figure 2 is a flow chart of an example method for performing operations in response to detecting a data dependency context according to some embodiments of the present disclosure.

[0006] Figure 3 Flowchart of another example method for performing operations in response to detecting a data dependency context according to some embodiments of the present disclosure.

[0007] Figure 4 Flowchart of an example method for performing operations in response to detecting a read-before-write situation according to some embodiments of the present disclosure.

[0008] Figure 5 is a block diagram of an example computer system in which embodiments of the present disclosure may operate. DETAILED DESCRIPTION

[0009] Various aspects of the present disclosure relate to managing data dependencies in a hybrid dual in-line memory module for out-of-order processing. The memory subsystem may be a storage device, a memory module, or a mixture of a storage device and a memory module. Figure 1 Examples of storage devices and memory modules are described. In general, a host system may utilize a memory subsystem that includes one or more components, such as memory devices, that store data. The host system may provide data for storage at the memory subsystem and may request retrieval of data from the memory subsystem.

[0010] The memory subsystem may include both nonvolatile and volatile memory devices. One example of a nonvolatile memory device is a NAND memory device. Another example is a three-dimensional cross-point ("3D cross-point") memory device, which is a cross-point array of nonvolatile memory cells. Figure 1 Other examples of non-volatile memory devices are described. A non-volatile memory device is a package of one or more dies. The dies in a package may be assigned to one or more channels for communication with a memory subsystem controller. Each die may include a set of memory cells ("cells"). A cell is an electronic circuit that stores information. Depending on the cell type, a cell may store one or more binary bits of information and have various logical states associated with the number of bits stored. The logical states may be represented by binary values ​​(e.g., "0" and "1") or combinations of such values.

[0011] The non-volatile memory device may include a three-dimensional cross-point ("3D cross-point") memory device that is a cross-point array of non-volatile memory cells and may perform bit storage based on changes in bulk resistance in conjunction with a stackable cross-grid data access array. Additionally, in contrast to many flash-based memories, the 3D cross-point memory device may perform write-in-place operations, where non-volatile memory cells may be programmed without pre-erasing the non-volatile memory cells. Such non-volatile memory devices may group pages on a die and channel to form management units (MUs).

[0012] The memory subsystem may be a hybrid DIMM that includes a first type of memory device (e.g., 3D crosspoint media) and a second type of memory device (e.g., dynamic random access memory (DRAM)) in a single DIMM package. The first type of memory device (i.e., first memory type) may have a larger storage capacity but a high access latency, while the second type of memory device (i.e., second memory type) has a smaller amount of volatile memory but a lower access latency. The cache manager may manage the retrieval, storage, and delivery of data to and from the first type of memory device and the second type of memory device. Data transfer between the first type of memory device (e.g., 3D crosspoint) and the second type of memory device (e.g., DRAM) requires more time to process compared to the processing speed of the cache manager processing data access requests (e.g., read access requests and write access requests) from the host system.

[0013] The cache manager allows the second type of memory to act as a cache memory of the first memory type. Therefore, if the cache hit rate is high, the high latency of the first memory type can be masked by the low latency of the second memory type. For example, a DRAM memory device or other volatile memory can be used as a cache memory for a 3D crosspoint memory device or other non-volatile memory device (e.g., a storage class memory (SCM)). The host system can utilize a hybrid DIMM to retrieve and store data at the 3D crosspoint memory. The hybrid DIMM can be coupled to the host system via a bus interface (e.g., a DIMM connector). The DIMM connector can be a synchronous or asynchronous interface between the hybrid DIMM and the host system. When the host system provides a data access request, such as a read access request, the corresponding data can be returned to the host system from the 3D crosspoint memory or from another memory device of the hybrid DIMM that serves as a cache memory for the 3D crosspoint memory.

[0014] In a conventional memory system, a host system may send a read access request and / or a write access request to a memory subsystem. These data access requests (e.g., read access requests and write access requests) may be queued and processed in the order received by the memory subsystem (e.g., first-in, first-out, hereinafter "FIFO") to prevent problems associated with data dependencies. Data dependency is a situation in which a data access request refers to data operated by a previous data access request. For example, a memory subsystem controller may receive a write access request for data stored at the same physical address from a physical address, followed by a read access request for the data. If the read access request is executed before the write access request, the read access request returns incorrect data to the host system because the write access request has not yet been processed. However, queuing and processing data access requests in the order received may be undesirable because not all data access requests have data dependencies and most data access requests may be issued and completed out of order. Completing data access requests out of order may reduce latency experienced when read and write operations frequently switch and when switching to a different block or die while outstanding data access requests to that block or die are still queued.

[0015] Aspects of the present disclosure address the above and other deficiencies by implementing a set of schemes for managing data dependencies. In an illustrative example, a second type of memory device (e.g., DRAM) may be configured as a cache memory that stores recently accessed and / or highly accessed data so that such data may be quickly accessed by a host system. Read access requests and write access requests for data in the cache memory may be processed generally out of order (e.g., the memory subsystem need not process a first request received before a second request) unless there are data dependencies, such as when multiple data access requests correspond to the same data (e.g., data stored at the same physical address). Examples of data dependencies include a read access request after a write access request (RAW), a write access request after a read access request (WAR), and a write access request after a write access request (WAW). A write access content addressable memory (CAM) may be used to track all outstanding write access requests before the data transfer of the write access request is completed. The read access CAM may be used to track all outstanding read access requests, and the WAR CAM may be used to track all write access requests that have a read access request preceding them. When a read access request or a write access request is received, a write CAM lookup may be performed by the cache manager to determine whether there are any outstanding write access requests to the same physical address that precede the newly received read access request or write access request. In addition, for each write access request received, a read access CAM lookup may be performed to determine whether any outstanding read access request to the same physical address was executed before the new write access request. In response to determining that the write access CAM and / or the read access CAM contain an indication of an outstanding data (read or write) access request corresponding to an address associated with the received read or write access request, the cache manager may determine which type of data dependency scenario exists and perform operations associated with the determined data dependency scenario. Depending on which type of data dependency scenario is detected, the cache manager may reorder the received data access requests and / or the outstanding data access requests accordingly.

[0016] Advantages of the present disclosure include, but are not limited to, improved performance of a hybrid DIMM resulting in a higher quality of service for a host system. For example, cache operations between a first memory device and a second memory device may be internal to the hybrid DIMM. Thus, when data is transferred from a 3D crosspoint memory to be stored at a DRAM data cache, the transfer of data will not utilize an external bus or interface that is also used by the host system when receiving and transmitting write operations and read operations. Additionally, the present disclosure allows read commands and write commands to be serviced out of order by a cache manager while still allowing any data dependencies to be maintained with minimal latency increase.

[0017] Figure 1 An example computing system 100 is illustrated that includes a memory subsystem 110 in accordance with some embodiments of the present disclosure. Memory subsystem 110 may include media such as one or more volatile memory devices (e.g., memory device 140), one or more non-volatile memory devices (e.g., memory device 130), or a combination of such memory devices.

[0018] The memory subsystem 110 may be a storage device, a memory module, or a mixture of storage devices and memory modules. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage (UFS) drives, secure digital (SD) cards, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and various types of non-volatile dual in-line memory modules (NVDIMMs).

[0019] The computing system 100 can be a computing device, such as a desktop computer, a laptop computer, a network server, a mobile device, a vehicle (e.g., an airplane, drone, train, car, or other transportation vehicle), an Internet of Things (IoT)-enabled device, an embedded computer (e.g., a computer included in a vehicle, industrial equipment, or a networked commercial device), or such computing device that includes a memory and a processing device.

[0020] The computing system 100 may include a host system 120 coupled to one or more memory subsystems 110. In some embodiments, the host system 120 is coupled to memory subsystems 110 of different types. Figure 1 An example of a host system 120 coupled to one memory subsystem 110 is illustrated. As used herein, "coupled to" or "coupled with" generally refers to a connection between components, which can be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical connections, optical connections, magnetic connections, etc.

[0021] The host system 120 may include a processor chipset and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more cache memories, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). The host system 120 uses the memory subsystem 110, for example, to write data to the memory subsystem 110 and read data from the memory subsystem 110.

[0022] The host system 120 may be coupled to the memory subsystem 110 via a physical host interface. Examples of the physical host interface include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, a Fibre Channel, a Serial Attached SCSI (SAS), a Double Data Rate (DDR) memory bus, a Small Computer System Interface (SCSI), a Dual In-line Memory Module (DIMM) interface (e.g., a DIMM socket interface supporting Double Data Rate (DDR)), etc. The physical host interface may be used to transfer data between the host system 120 and the memory subsystem 110. When the memory subsystem 110 is coupled to the host system 120 through a physical host interface (e.g., a PCIe bus), the host system 120 may further utilize an NVM Express (NVMe) interface to access components (e.g., the memory device 130). The physical host interface may provide an interface for passing control, address, data, and other signals between the memory subsystem 110 and the host system 120. Figure 1 Memory subsystem 110 is illustrated as an example. In general, host system 120 can access multiple memory subsystems via the same communication connection, multiple separate communication connections, and / or a combination of communication connections.

[0023] The memory devices 130, 140 may include any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 140) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0024] Some examples of non-volatile memory devices (e.g., memory device 130) include NAND type flash memory and write-in-place memory, such as a three-dimensional cross-point ("3D cross-point") memory device, which is a cross-point array of non-volatile memory cells. The cross-point array of non-volatile memory can perform bit storage based on changes in body resistance in conjunction with a stackable cross-grid data access array. In addition, in contrast to many flash-based memories, cross-point non-volatile memory can perform write-in-place operations, where non-volatile memory cells can be programmed without pre-erasing the non-volatile memory cells. NAND type flash memory includes, for example, two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0025] Each of the memory devices 130 may include one or more memory cell arrays. One type of memory cell, for example, a single-level cell (SLC) may store one bit per cell. Other types of memory cells, such as multi-level cells (MLC), three-level cells (TLC), four-level cells (QLC), and five-level cells (PLC) may store multiple bits per cell. In some embodiments, each of the memory devices 130 may include one or more memory cell arrays, such as SLC, MLC, TLC, QLC, PLC, or any combination thereof. In some embodiments, a particular memory device may include an SLC portion of a memory cell, as well as an MLC portion, a TLC portion, a QLC portion, or a PLC portion. The memory cells of the memory device 130 may be grouped into pages, which may refer to logical units of a memory device for storing data. For some types of memory (e.g., NAND), pages may be grouped to form blocks.

[0026] Although nonvolatile memory components such as a 3D cross-point array of nonvolatile memory cells and NAND-type flash memory (e.g., 2D NAND, 3D NAND) are described, the memory device 130 may be based on any other type of nonvolatile memory, such as read-only memory (ROM), phase-change memory (PCM), self-selected memory, other chalcogenide-based memories, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin transfer torque (STT)-MRAM, conductive bridging RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), non-OR (NOR) flash memory, and electrically erasable programmable read-only memory (EEPROM).

[0027] The memory subsystem controller 115 (or simply controller 115) can communicate with the memory device 130 to perform operations, such as reading data, writing data, or erasing data at the memory device 130, and other such operations. The memory subsystem controller 115 can include hardware, such as one or more integrated circuits and / or discrete components, buffer memory, or a combination thereof. The hardware can include digital circuitry with dedicated (i.e., hard-coded) logic to perform the operations described herein. The memory subsystem controller 115 can be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or other suitable processor.

[0028] The memory subsystem controller 115 may be a processing device that includes one or more processors (e.g., processor 117) configured to execute instructions stored in a local memory 119. In the illustrated example, the local memory 119 of the memory subsystem controller 115 includes an embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 110, including handling communications between the memory subsystem 110 and the host system 120.

[0029] In the illustrated example, the local memory 119 of the memory subsystem controller 115 includes embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 110, including handling communications between the memory subsystem 110 and the host system 120.

[0030] In some embodiments, local memory 119 may include memory registers that store memory pointers, fetched data, etc. Local memory 119 may also include read-only memory (ROM) for storing microcode. Figure 1 The example memory subsystem 110 in has been illustrated as including a memory subsystem controller 115, but in another embodiment of the present disclosure, the memory subsystem 110 does not include a memory subsystem controller 115 and may instead rely on external control (e.g., provided by an external host or a processor or controller separate from the memory subsystem).

[0031] In general, the memory subsystem controller 115 may receive commands or operations from the host system 120, and may convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory device 130. The memory subsystem controller 115 may be responsible for other operations, such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical addresses (e.g., logical block addresses (LBA), name space) and physical addresses (e.g., physical MU addresses, physical block addresses) associated with the memory device 130. The memory subsystem controller 115 may further include a host interface circuit system to communicate with the host system 120 via a physical host interface. The host interface circuit system may convert commands received from the host system into command instructions to access the memory device 130, and convert responses associated with the memory device 130 into information for the host system 120.

[0032] The memory subsystem 110 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 110 may include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoders and column decoders) that may receive addresses from the memory subsystem controller 115 and decode the addresses to access the memory device 130.

[0033] In some embodiments, the memory device 130 includes a local media controller 135 that operates in conjunction with the memory subsystem controller 115 to perform operations on one or more memory cells of the memory device 130. An external controller (e.g., the memory subsystem controller 115) can manage the memory device 130 externally (e.g., perform media management operations on the memory device 130). In some embodiments, the memory subsystem 110 is a managed memory device that includes a raw memory device 130 with control logic on the die (e.g., the local controller 132) and a controller for media management within the same memory device package (e.g., the memory subsystem controller 115). An example of a managed memory device is a managed NAND (MNAND) device.

[0034] In one embodiment, the memory subsystem 110 includes a cache manager 113, which can be used to track and manage data in the memory device 130 and the memory device 140, as well as data access commands (e.g., read access commands, write access commands, etc.) received from the host system 120. In some embodiments, the memory subsystem controller 115 includes at least a portion of the cache manager 113. In some embodiments, the cache manager 113 is part of the host system 120, an application, or an operating system. In other embodiments, the local media controller 135 includes at least a portion of the cache manager 113 and is configured to perform the functionality described herein. The cache manager 113 can communicate directly with the memory devices 130 and 140 via a synchronous interface. In addition, the transfer of data between the memory device 130 and the memory device 140 can be completed within the memory subsystem 110 without accessing the host system 120.

[0035] Memory device 140 may include a data cache that stores data from memory device 130 so that future requests for data can be serviced more quickly. A cache line is the basic unit for cache storage and may contain multiple bytes and / or words of data. Smaller cache line sizes have higher hit rates but require more tag memory than larger cache line sizes. A tag is a unique identifier for a set of data that can be used to distinguish between different areas of mapped memory.

[0036] In some embodiments, all data stored by the memory subsystem 110 may be stored at the memory device 130. Certain data stored at the memory device 130 may also be stored at the data cache of the memory device 140. For example, data determined to be more frequently or more recently accessed by the host system 120 may be stored at the data cache for faster host access. When the host system 120 provides a read access command for data stored at the data cache (i.e., a cache hit), the data may be retrieved from the data cache rather than from the memory device 130. The bandwidth or ability to retrieve data at the data cache may be faster than the bandwidth or ability to retrieve data at the memory device 130.

[0037] The cache manager 113 may include executable code that implements one or more of the components and modules and may be implemented within the controller 115, hardware, firmware, or a combination thereof. The cache manager 113 may include a read access content addressable memory (CAM) 152, a write access CAM 154, and a read-before-write (WAR) CAM 156. A CAM is a special type of computer memory that can compare input search data (e.g., a tag) to a table of stored data and return the address of the matching data. Specifically, data stored on the CAM can be accessed by searching the content itself, and the cache manager 113 can retrieve the address where the content was found.

[0038] Unless data dependencies exist, the cache manager 113 may execute multiple read access requests and / or write access requests for data in a cache memory (e.g., memory device 140) out of order from one or more queues (e.g., FIFO queues). A read access request may be queued in a read queue (e.g., a read FIFO). A write access request may be queued in a write queue (e.g., a write FIFO). The cache manager 113 may select from the read queue or the write queue for processing. Examples of data dependencies include, but are not limited to, a read access request after a write access request (hereinafter “RAW”), a write access request after a read access request (hereinafter “WAR”), and a write access request after an existing write access request (hereinafter “WAW”). When such data dependencies occur, they may be referred to as data dependency scenarios (e.g., RAW scenarios, WAR scenarios, and WAW scenarios).

[0039] The write access CAM 154 may include a write queue and may track all outstanding write access requests stored in the write queue before the data transfer of the write access request is executed. The size of the write access CAM 154 may be determined by the total number of outstanding write access requests that can be supported by the cache memory. Each write access request may then be assigned a write access identification (WID). The cache manager 113 may use the write access CAM 154 to identify and address any read or write access requests subsequent to the outstanding write access request, as will be explained in more detail below.

[0040] The read access CAM 152 may include a read queue and may track all outstanding read access requests stored in the read queue. The read access CAM 152 may also track all outstanding read access requests in a data pipeline (e.g., between a sequence FIFO and a read FIFO). The sequence FIFO stores write requests and read requests in the order issued by the host. A data pipeline is a set of data processing elements connected in series, where the output of one element is the input of the next element. The size of the read access CAM 152 may be determined by the total number of outstanding read access requests that can be supported by the cache memory. Each read access request may then be assigned a read access identification (RID).

[0041] The WAR CAM 156 may be used to track all write access requests that have been preceded by a read access request. The size of the WARCAM may be determined by the total number of outstanding write access requests that may be supported by the cache memory.

[0042] The cache manager 113 may receive data access requests (read access requests and write access requests) from, for example, the host system 120. In response to receiving the data access request from the host system 120, the cache manager 113 may perform a lookup in the write access CAM 154 to determine whether the physical address being accessed by the data access request has an outstanding write access request. If the write access CAM 154 lookup hits and the data access request is a read access request (e.g., RAW context), the cache manager 113 may execute the write access request before executing the read access request. If the write access CAM 154 lookup hits and the data access request is a write access request (e.g., WAW context), the WID for which the write access CAM 154 lookup hits may be stored in a separate hit WID queue and ignored instead of executed. In some embodiments, in response to a write access CAM 154 lookup hit or miss, a WID may be written to the write access CAM 154 to become an outstanding write access request, and a miss indication may also be stored in a hit WID queue.

[0043] It should be noted that a miss indication may be used because the write access address and the write access data may be sent at different times with the write access data being sent several clock cycles after the write access address. The write access data may pass through an error checking process, wherein the error checking information is stored in an error checking queue. A write access request may fail due to a data error, and the cache manager 113 may use the error checking information stored in the error checking queue to remove the hit WID (associated with the write access request) that the write access CAM 154 search hits. Once the transfer of the write access request is performed, the cache manager 113 may remove the incomplete write access request stored in the write access CAM 154, as indicated by releasing its WID. When the incomplete write access request is being removed from the write access CAM 154, the write access CAM 154 search is not performed. This means that a read access request or a write access request in the cache queue may occur at least one cycle earlier than when the cache manager 113 selects the hit WID for processing. In another example, the cache manager 113 performs a write access CAM 154 lookup even when a write access request fails. This means that the write access CAM 154 lookup latency can be less than or equal to the write cache lookup latency.

[0044] In the RAW scenario, in response to a write access CAM 154 lookup hit from a read access request, the cache manager 113 may use the data of the write access request in the write buffer to service the read access request. The cache lookup for the read access request associated with the write CAM lookup hit may be invalidated by the cache manager 113. Once a write access CAM 154 lookup hit occurs, the cache manager 113 may transfer the data from the write buffer (where the data of all write access requests is stored) to the read buffer (where the data of all read access requests is stored before being read by the host system 120). The cache manager 113 may recycle (or release) the hit WID from the write access CAM 154 after executing the write access request. It should be noted that the execution of a write access request may take longer than the execution of a read access request.

[0045] In some embodiments of the RAW scenario, the cache manager 113 sends the hit WID to the write buffer so that the cache manager 113 knows which location will be read. The cache manager 113 may also send the RID to the write buffer so that the RID and the write buffer data read can be sent together to the host system 120. The cache manager 113 may use a FIFO queue to indicate whether the read access request is a hit or a miss (by looking up the write access CAM 154). In response to a hit, when the read access request is serviced by the cache manager 113, the read access request will be discarded because it has already been sent to the write buffer.

[0046] For a write access request that follows a write access request (WAW scenario), the cache manager 113 may set a delete flag for the hit WID. When a write access request is serviced by the cache manager 113, the cache manager 113 may check whether the associated delete flag is set. If set, the hit WID will be released. The cache manager 113 may also check whether the write access request is associated with the hit WID. If so, the cache manager 113 will clear the delete flag of the hit WID. In addition, the cache manager 113 may send a read access request that follows a write access request for the same physical address to the write buffer until the previous write access request is executed. For example, when a RAW scenario occurs, the cache manager 113 may retrieve the write access CAM 154 to find the hit data, and return the data of the hit write access request in the write buffer for the read access request. Because the data of the write access request may be current data, retrieving from the write buffer provides correct data with a faster access time. Data may be moved from the write buffer to the read buffer after a write CAM 154 hit occurs, so that the data may be quickly read before a write access request is executed. When a WAW scenario occurs, the cache manager 113 may delete the write access request that hits in the write access CAM 154. The new write access request may have the latest version of the data, and therefore, the cache manager 113 may ignore the previously incomplete write access request.

[0047] In one embodiment, when the cache manager 113 reads a WID in the write queue without an error indication, the cache manager 113 may set a delete WID flag in the delete WID table at a location indexed by the hit WID. When a data transfer is about to occur for a hit WID write access request, the flag may be checked. If the delete WID flag is set, the cache manager 113 may fail the data transfer and release the WID. In response to the cache manager 113 executing the write access request, the cache manager 113 may clear the delete WID flag associated with the write access request.

[0048] The cache manager 113 may perform a lookup in the read access CAM 152 for all received write access requests to determine if the write access request is accessing the same physical address as any outstanding read access request (e.g., WAR context). If a write access request has a read access CAM 152 lookup hit, the write access request may be stored in the read queue and the write queue along with a hit indication. The hit indication may prevent the write access request from exceeding the outstanding read access request on which it depends. The write access request may also be stored in the WAR CAM 156 so that the next write access request with the same physical address may be stored in the read queue when its WAR CAM 156 lookup hits. A write access request with a WARCAM 156 lookup hit may also replace a hit entry in the WAR CAM 156.

[0049] In some embodiments, a write access request from the write queue may be selected by the cache manager 113 for processing only when its write data error status is available. However, a write access request associated with a WAR CAM 156 lookup hit may not be selected for processing by the cache manager 113, but its error status may be stored in the WAR queue. Write access requests outside the read queue may not be selected for processing by the cache manager 113. Alternatively, the read access request may be stored in another WAR queue. When both WAR queues have available data, the cache manager 113 may select the write access request for processing.

[0050] Figure 2 2 is a flow chart of an example method 200 illustrating a process performed in response to detecting a data dependency situation according to some embodiments of the present disclosure. The method 200 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 200 is performed by Figure 1 113. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0051] At operation 210, processing logic may receive a set of data access requests in a first order. The data access requests may include one or more read access requests, one or more write access requests, or any combination thereof. The first order may be a sequential order and correspond to times at which the memory subsystem 110 receives various requests in the set of data access requests. For example, in the first order, a first data access request may be received at a first time, and a second data access request may be received at a second time, wherein the second time occurs after the first time. In one embodiment, the set of data access requests is received at the memory subsystem 110 from a host system, such as the host system 120. In another embodiment, the set of data access requests includes one or more requests for memory management operations (e.g., garbage collection, wear leveling, etc.), which are generated within the memory subsystem 110, such as by the memory subsystem controller 115 or some other component.

[0052] At operation 220, processing logic may execute the set of data access requests in a second order. The second order may be a non-sequential order. For example, processing logic may execute the set of data access requests in a last-in-first-out (LIFO) order, may execute read access requests first, may execute write access requests first, and so on. Thus, even if a second request in the set of data access requests is received after a first request, as described above, in one embodiment, the second request may be processed before the first request is processed. In another embodiment, the second order is the same as the first order.

[0053] At operation 230, processing logic may identify a delayed data access request in the set of data access requests. A delayed data access request may be any data access request that is not a first data access request received in the set of data access requests. For example, if the set of data access requests includes a plurality of data access requests, wherein the first data access request includes a first data access request received in a first order, then any of the second, third, fourth, etc. data access requests received in the first order may be considered a delayed data access request.

[0054] At operation 240, processing logic may determine whether a data structure in a local memory associated with the processing device includes an indication of a previously incomplete data access request corresponding to the physical address associated with the deferred data access request. The data structure may be at least one of the read access CAM 152, the write access CAM 154, the WAR CAM 156, or any combination thereof. In an example, processing logic may perform a lookup of at least one of the read access CAM 152, the write access CAM 154, or the WAR CAM 156 to determine whether the physical address being accessed by the data access request has an incomplete data access request. For example, the data structure may have a plurality of entries, each corresponding to a previously incomplete data access request, and each having an associated physical address. In one embodiment, the cache manager 113 may compare the physical address of the deferred data access request with the physical address associated with each entry in the data structure. When the physical address of the deferred data access request matches a physical address associated with at least one of the entries in the data structure, the cache manager 113 may determine that the physical address of the deferred data access request has an outstanding data access request.

[0055] At operation 250, processing logic may insert the delayed data access request into the data structure as a new outstanding data access request in response to determining that the data structure does not include a previously outstanding data access request. For example, the delayed data access request may be placed in one or more of CAM 152, write access CAM 154, WAR CAM 156, or any combination thereof.

[0056] At operation 260, processing logic may identify a type of data dependency (e.g., a data dependency context) associated with a previously uncompleted data access request in response to determining that the data structure includes a previously uncompleted data access request and perform one or more operations associated with the type of data dependency. The data dependency context may include a RAW context, a WAW context, or a WAR context. The RAW context occurs when the deferred data access request is a read access request and the uncompleted data access request is a write access request. The WAW context occurs when the deferred data access request is a write access request and the uncompleted data access request is a write access request. The WAR context occurs when the deferred data access request is a write access request and the uncompleted data access request is a read access request.

[0057] Depending on which type of data dependency scenario is detected, processing logic may reorder the deferred data access request and / or the outstanding data access request accordingly. In the case of a WAR scenario, processing logic may place the deferred data access request in a queue to be executed after the previously outstanding data access request. In the case of a RAW scenario, processing logic may execute the deferred data access request by reading data associated with the previously outstanding data access request from the data structure before executing the deferred read access request. In the case of a WAW scenario, processing logic may delete the previously outstanding data access request and insert the deferred data access request into the data structure as a new outstanding data access request.

[0058] Figure 3 Flowchart of an example method 300 illustrating a process performed in response to detecting a data dependency context according to some embodiments of the present disclosure. The method 300 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 300 is performed by Figure 1 113. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0059] At operation 310, processing logic may maintain a set of read access requests in a first data structure. In one example, the first data structure may be a read access CAM 152. At operation 320, processing logic may maintain a set of write access requests in a second data structure. In one example, the second data structure may be a write access CAM 154. The set of read requests and the set of write requests may be maintained in the read access CAM 152 and the write access CAM 154, respectively, in a first order. The first order may be a sequential order and corresponds to the time at which the memory subsystem 110 receives the various data access requests. For example, in the first order, a first data access request may be received at a first time, and a second data access request may be received at a second time, wherein the second time occurs after the first time. In one embodiment, the set of data access requests is received at the memory subsystem 110 from a host system, such as the host system 120. In another embodiment, the set of data access requests includes one or more requests for memory management operations (eg, garbage collection, wear leveling, etc.) generated within the memory subsystem 110, such as by the memory subsystem controller 115 or some other component.

[0060] The processing logic may execute the set of read access requests and the set of write access requests in a second order. The second order may be a non-sequential order. For example, the processing logic may execute the set of data access requests in a last-in-first-out (LIFO) order, may execute the read access request first, may execute the write access request first, and so on. Thus, even if a second request in the set of data access requests is received after the first request, as described above, in one embodiment, the second request may be processed before the first request is processed. In another embodiment, the second order is the same as the first order.

[0061] At operation 330, processing logic may receive a new data access request from the host system 120. The new data access request may be a read access request or a write access request. The new data access request may be any data access request that is not a first data access request in the set of read access requests in the first data structure or the write access requests in the second data structure. For example, if the set of data access requests in the first data structure and / or the second data structure includes a plurality of data access requests, wherein the first data access request includes a first data access request received in a first order, then any of the second, third, fourth, etc. data access requests received in the first order may be considered a new data access request.

[0062] At operation 340, processing logic may determine whether an outstanding data access request corresponding to the address associated with the new data access request is queued in at least one of the first data structure or the second data structure. For example, processing logic may perform a lookup of at least one of the read access CAM 152 or the write access CAM 154 to determine whether the physical address being accessed by the data access request has an outstanding data access request. For example, the first data structure and the second data structure may have multiple entries, each entry corresponding to a previously outstanding data access request, and each having an associated physical address. In one embodiment, the cache manager 113 may compare the physical address of the new data access request with the physical address associated with each entry in the first data structure and / or the second data structure. When the physical address of the new data access request matches the physical address associated with at least one of the entries in the first data structure or the second data structure, the cache manager 113 may determine that the physical address of the new data access request has an outstanding data access request.

[0063] At operation 350, in response to determining that the first data structure and the second data structure do not include an outstanding data access request, the processing logic may insert the new data access request into the first data structure when the new data access request is a read access request, or insert the new data access request into the second data structure when the new data access request is a write access request.

[0064] At operation 360, in response to determining that the first data structure and / or the second data structure includes an outstanding data access request, processing logic may identify a type of data dependency (e.g., a data dependency context) associated with the outstanding data access request and perform one or more operations associated with the type of data dependency. The data dependency context may include a RAW context, a WAW context, or a WAR context. The RAW context occurs when the new data access request is a read access request and the outstanding data access request is a write access request. The WAW context occurs when the new data access request is a write access request and the outstanding data access request is a write access request. The WAR context occurs when the new data access request is a write access request and the outstanding data access request is a read access request.

[0065] Depending on which type of data dependency scenario is detected, the processing logic may reorder the new data access request and / or the outstanding data access request accordingly. In certain embodiments, in response to determining that the new data access request is a write access request and detecting the outstanding data access request in the first data structure, the processing logic may place the new data access request in a queue to be executed after the outstanding data access request. In response to determining that the new data access request is a read access request and detecting the outstanding data access request in the second data structure, the processing logic may execute the new data access request by reading data associated with the outstanding data access request from the second data structure before executing the new read access request. In response to determining that the new data access request is a write access request and detecting the outstanding data access request in the second data structure, the processing logic may delete the outstanding data access request and insert the new data access request into the second data structure as a new outstanding data access request.

[0066] Figure 4 4 is a flow chart illustrating an example method 400 of a process performed in response to detecting a WAR context according to some embodiments of the present disclosure. The method 400 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 400 is performed by Figure 1 113. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, it should be understood that the illustrated embodiments are merely examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0067] At operation 410, processing logic may perform a lookup in the read access CAM 152 for all received write access requests to determine whether the write access request is accessing the same physical address as any outstanding read access request. In response to a write access request having a read access CAM 152 lookup hit, processing logic may store the write access request along with a hit indication in a read queue of the read CAM 152 and a write queue of the write CAM 154 at operation 420. The hit indication may prevent the write access request from exceeding the outstanding read access request on which it depends.

[0068] At operation 430, processing logic may store the write access request in the WAR CAM 156. This enables the next write access request with the same physical address to be stored in the read queue when its WAR CAM 156 lookup hits. Processing logic may also replace the hit entry in the WAR CAM 156 with the write access request associated with the WAR CAM 156 lookup hit.

[0069] Figure 5 An example machine of a computer system 500 is illustrated, within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system 500 may correspond to a host system (e.g., Figure 1 A host system 120 that includes or utilizes a memory subsystem (e.g., Figure 1 The memory subsystem 110 of the controller may be used to execute the operation of the controller (for example, execute the operating system to execute the corresponding Figure 1 In some embodiments, the machine may be connected (e.g., using a network) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine may operate in the capacity of a server or a client machine in a client-server network environment, or in the capacity of a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0070] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by the machine. In addition, while a single machine is described, the term "machine" shall also be taken to include any collection of machines that individually or collectively execute one (or more) sets of instructions to perform any one or more of the methodologies discussed herein.

[0071] The example computer system 500 includes a processing device 502, a main memory 504 (e.g., a read-only memory (ROM), a flash memory, a dynamic random access memory (DRAM) such as a synchronous DRAM (SDRAM) or a Rambus DRAM (RDRAM)), a static memory 506 (e.g., a flash memory, a static random access memory (SRAM), etc.), and a data storage system 518, which communicate with each other via a bus 530. The processing device 502 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, etc. More specifically, the processing device can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. The processing device 502 can also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. The processing device 502 is configured to execute instructions 526 for performing the operations and steps discussed herein. Computer system 500 may further include a network interface device 508 to communicate via a network 520 .

[0072] The data storage system 518 may include a machine-readable storage medium 524 (also referred to as a computer-readable medium) on which is stored one or more sets of instructions 526 or software embodying any one or more of the methodologies or functions described herein. The instructions 526 may also reside, completely or at least partially, within the main memory 504 and / or within the processing device 502 during execution thereof by the computer system 500, with the main memory 504 and the processing device 502 also constituting machine-readable storage media. The machine-readable storage medium 524, the data storage system 518, and / or the main memory 504 may correspond to Figure 1 Memory subsystem 110.

[0073] In one embodiment, instructions 526 include instructions for implementing Figure 1 13. Although the machine-readable storage medium 524 is shown as a single medium in the example embodiment, the term "machine-readable storage medium" should be considered to include a single medium or multiple media storing one or more sets of instructions. The term "machine-readable storage medium" should also be considered to include any medium capable of storing or encoding a set of instructions for execution by a machine and causing the machine to perform any one or more of the methods of the present disclosure. The term "machine-readable storage medium" should therefore be considered to include, but not be limited to, solid-state memory, optical media, and magnetic media.

[0074] Some portions of the previous detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively convey the substance of their work to others skilled in the art. In this document, and generally, an algorithm is conceived to be a self-consistent sequence of operations that produces a desired result. An operation is one that requires physical manipulation of physical quantities. Typically (but not necessarily), these quantities take the form of electrical or magnetic signals that can be stored, combined, compared, and otherwise manipulated. It has proven convenient at times, primarily for common reasons, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, and the like.

[0075] It should be borne in mind, however, that all of these and similar terms will be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within a computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0076] The present disclosure also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the desired purpose, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including a floppy disk, an optical disk, a CD-ROM, and a magnetic optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card or an optical card, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0077] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general purpose systems may be used in conjunction with the programs according to the teachings herein, or it may prove convenient to construct a more specialized device to perform the method. The structures of various such systems will be presented as set forth in the description below. In addition, the present disclosure is not described with reference to any particular programming language. It should be appreciated that the teachings of the present disclosure as described herein may be implemented using various programming languages.

[0078] The present disclosure may be provided as a computer program product or software, which may include a machine-readable medium having stored thereon instructions that can be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form that is readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory device, etc.

[0079] In the foregoing description, embodiments of the present disclosure have been described with reference to specific example embodiments thereof. It will be apparent that various modifications may be made to the present disclosure without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. Therefore, the description and drawings should be viewed in an illustrative sense rather than a restrictive sense.

Claims

1. A system comprising: a first memory device; a second memory device coupled to the first memory device, wherein the second memory device has a lower access latency than the first memory device and serves as a cache memory for the first memory device; as well as a processing device operatively coupled to the first memory device and the second memory device to perform operations comprising: receiving a set of data access requests in a first order; executing the set of data access requests in a second order; identifying a delayed data access request among the set of data access requests; determining whether a data structure in a local memory associated with the processing device contains an indication of a previously outstanding data access request corresponding to an address associated with the delayed data access request, wherein the data structure comprises a content addressable memory (CAM); as well as In response to determining that the data structure includes an indication of a previously outstanding data access request corresponding to the address associated with the delayed data access request, a type of data dependency associated with the previously outstanding data access request is identified and one or more operations associated with the type of data dependency are performed.

2. The system of claim 1, wherein the processing device further performs operations comprising: identifying a read-after-write RAW data dependency in response to determining that the deferred data access request is a read access request and the previously outstanding data access request is a write access request; identifying a write-after-write (WAW) data dependency in response to determining that the deferred data access request is a write access request and the previously outstanding data access request is a write access request; as well as In response to determining that the deferred data access request is a write access request and the previously outstanding data access request is a read access request, a read-before-write WAR data dependency is identified.

3. The system of claim 2, wherein in response to identifying the RAW data dependency, performing the one or more operations comprises performing the delayed data access request by reading data associated with the previously outstanding data access request from the data structure.

4. The system of claim 2, wherein in response to identifying the WAW data dependency, performing the one or more operations comprises deleting the previously outstanding data access request and inserting the deferred data access request into the data structure as a new outstanding data access request.

5. The system of claim 2, wherein in response to identifying the WAR data dependency, performing the one or more operations comprises placing the delayed data access request in a queue to be executed after the previously outstanding data access request.

6. The system of claim 1, wherein the processing device further performs operations comprising: In response to determining that the data structure does not include the indication of the previously incomplete data access request, the indication of the deferred data access request is inserted into the data structure as a new incomplete data access request.

7. A method comprising: maintaining a set of read access requests in a first data structure; maintaining a set of write access requests in a second data structure, wherein the first data structure and the second data structure comprise a content addressable memory CAM; receiving new data access requests; determining whether an outstanding data access request corresponding to an address associated with the new data access request is queued in at least one of the first data structure or the second data structure; as well as In response to determining that the first data structure and the second data structure include indications of outstanding data access requests, a type of data dependency associated with the outstanding data access request is identified and one or more operations associated with the type of data dependency is performed.

8. The method according to claim 7, further comprising: In response to determining that the new data access request is a read access request and the outstanding data access request is a write access request, identifying a read-before-write RAW data dependency; identifying a write-after-write (WAW) data dependency in response to determining that the new data access request is a write access request and the outstanding data access request is a write access request; as well as In response to determining that the new data access request is a write access request and the outstanding data access request is a read access request, a read-before-write WAR data dependency is identified.

9. The method of claim 8, wherein in response to identifying the RAW data dependency, executing the one or more operations comprises executing the new data access request by reading data associated with the outstanding data access request from the data structure.

10. The method of claim 8, wherein in response to identifying the WAW data dependency, performing the one or more operations comprises deleting the outstanding data access request and inserting the new data access request into the data structure as a new outstanding data access request.

11. The method of claim 8, wherein in response to identifying the WAR data dependency, performing the one or more operations comprises placing the new data access request in a queue to be executed after the outstanding data access request.

12. The method according to claim 7, further comprising: In response to determining that the data structure does not include the indication of the previously incomplete data access request, the indication of the new data access request is inserted into the data structure as a new incomplete data access request.

13. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device operably coupled to a first memory device and a second memory device, perform operations comprising: receiving a set of data access requests in a first order; executing the set of data access requests in a second order; identifying a delayed data access request among the set of data access requests; determining whether a data structure in a local memory associated with the processing device contains an indication of a previously outstanding data access request corresponding to an address associated with the delayed data access request, wherein the data structure comprises a content addressable memory (CAM); as well as In response to determining that the data structure includes an indication of a previously outstanding data access request corresponding to the address associated with the delayed data access request, a type of data dependency associated with the previously outstanding data access request is identified and one or more operations associated with the type of data dependency are performed.

14. The non-transitory computer-readable storage medium of claim 13, wherein the processing device further performs operations comprising: identifying a read-after-write RAW data dependency in response to determining that the deferred data access request is a read access request and the previously outstanding data access request is a write access request; identifying a write-after-write (WAW) data dependency in response to determining that the deferred data access request is a write access request and the previously outstanding data access request is a write access request; as well as In response to determining that the deferred data access request is a write access request and the previously outstanding data access request is a read access request, a read-before-write WAR data dependency is identified.

15. The non-transitory computer-readable storage medium of claim 14, wherein in response to identifying the RAW data dependency, performing the one or more operations comprises executing the delayed data access request by reading data associated with the previously outstanding data access request from the data structure.

16. The non-transitory computer-readable storage medium of claim 14, wherein in response to identifying the WAW data dependency, performing the one or more operations comprises deleting the previously outstanding data access request and inserting the deferred data access request into the data structure as a new outstanding data access request.

17. The non-transitory computer-readable storage medium of claim 14, wherein in response to identifying the WAR data dependency, performing the one or more operations comprises placing the delayed data access request in a queue to be executed after the previously outstanding data access request.

18. The non-transitory computer-readable storage medium of claim 13, wherein the processing device further performs operations comprising: In response to determining that the data structure does not include the indication of the previously incomplete data access request, the indication of the deferred data access request is inserted into the data structure as a new incomplete data access request.

Images