Non-volatile memory module error recovery

Abstract

The memory controller includes a command queue, a memory interface queue, at least one storage queue, and a replay control circuit. The command queue has a first input for receiving memory access commands. The memory interface queue receives selected commands from the command queue and couples to a heterogeneous memory channel coupled to at least one non-volatile storage class memory (SCM) module. The at least one storage queue stores the memory access commands placed in the memory interface queue. The replay control circuit detects that an error requiring a recovery sequence has occurred and initiates the recovery sequence in response to the error. In the recovery sequence, the replay control circuit transmits the selected memory access commands from the at least one storage queue by grouping non-volatile read commands separately from all pending volatile reads, volatile writes, and non-volatile writes.

Description

[Background technology]

[0001] Computer systems typically use inexpensive, high-density dynamic random access memory (DRAM) chips for main memory. Most DRAM chips sold today are compatible with various Double Data Rate (DDR) DRAM standards promulgated by the Joint Electron Devices Engineering Council (JEDEC). A DDR memory controller is used to manage the interface between the various memory access agents and the DDR DRAM in accordance with the published DDR standards.

[0002] A Non-Volatile Dual In-line Memory Module with Persistent Storage (NVDIMM-P) is a storage class memory that can be used in place of a standard DDR DIMM, but includes persistent memory. The use of persistent or "non-volatile" memory with a memory channel that supports volatile memory such as DRAM presents several new problems. Reading from the non-volatile memory of a non-volatile DIMM is a slower process than reading from DRAM. Non-volatile memory reads typically complete in a non-deterministic time, as opposed to DRAM reads, which complete faster in a deterministic, known time. Addressing these differences presents various challenges in designing a memory controller that can interact with non-volatile DIMMs. [Brief description of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram illustrating an accelerated processing unit (APU) and memory system known in the prior art. [Diagram 2] 2 is a block diagram of a memory controller suitable for use in an APU such as FIG. 1, according to some embodiments. [Diagram 3] 1 is a block diagram of a data processing system according to some embodiments. [Figure 4]FIG. 4 is a flow diagram of a process for handling memory access requests according to some embodiments. [Diagram 5] FIG. 1 is a flow diagram of a process for handling errors according to some embodiments. [Figure 6] FIG. 6 is a diagram showing the process of FIG. 5. [Figure 7] FIG. 6 illustrates a further portion of the process of FIG. 5. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004] In the following description, the use of the same reference numbers in different figures indicates similar or identical items. Unless otherwise stated, the term "coupled" and its related verb forms include both direct and indirect electrical connections by means known in the art, and unless otherwise stated, reference to a direct connection implies alternative embodiments that also use an appropriate form of indirect electrical connection.

[0005] The memory controller includes a command queue, a memory interface queue, at least one storage queue, and a replay control circuit. The command queue has a first input for receiving memory access commands including volatile read, volatile write, non-volatile read, and non-volatile write, and an output, and has a plurality of entries. The memory interface queue has an input for receiving a command selected from the command queue, and an output for coupling to a heterogeneous memory channel to which the at least one non-volatile storage class memory (SCM) module is coupled. The at least one storage queue stores the memory access commands placed in the memory interface queue. The replay control circuit detects that an error has occurred that requires a recovery sequence, and initiates the recovery sequence in response to the error. In the recovery sequence, the replay control circuit transmits the memory access command selected from the at least one storage queue by grouping the non-volatile read command separately from all pending volatile read, volatile write, and non-volatile write.

[0006] A method is responsive to an error in a memory system. The method includes receiving a plurality of memory access requests including a volatile memory read, a volatile memory write, a non-volatile memory read, and a non-volatile memory write. Memory access commands for satisfying the memory access requests are placed in a memory interface queue. The memory access commands are transmitted from the memory interface queue to a heterogeneous memory channel coupled to at least one non-volatile SCM module. The memory access commands placed in the memory interface queue are stored in at least one storage queue. The method detects that an error has occurred that requires a recovery sequence, and in response, performs a recovery sequence that includes transmitting selected memory access commands from the at least one storage queue by grouping transmission of all non-volatile reads separately from all pending volatile reads, volatile writes, and non-volatile writes.

[0007] The data processing system includes a central processing unit, a data fabric coupled to the central processing unit, and a memory controller coupled to the data fabric for satisfying memory access requests from the central processing unit. The memory controller includes a command queue, a memory interface queue, at least one storage queue, and a replay control circuit. The command queue has a first input for receiving memory access commands including volatile reads, volatile writes, non-volatile reads, and non-volatile writes, and an output, and has a plurality of entries. The memory interface queue has an input for receiving a command selected from the command queue and an output for coupling to a heterogeneous memory channel to which the at least one non-volatile SCM module is coupled. The at least one storage queue stores the memory access commands placed in the memory interface queue. The replay control circuit detects that an error has occurred that requires a recovery sequence, and initiates the recovery sequence in response to the error. In the recovery sequence, the replay control circuit transmits the selected memory access command from the at least one storage queue by grouping the non-volatile read command separately from all pending volatile reads, volatile writes, and non-volatile writes.

[0008] 1 is a block diagram illustrating an accelerated processing unit (APU) 100 and memory system 130 known in the prior art. APU 100 is an integrated circuit suitable for use as a processor in a host data processing system and generally includes a central processing unit (CPU) core complex 110, a graphics core 120, a set of display engines 122, a memory management hub 140, a data fabric 125, a set of peripheral controllers 160, a set of peripheral bus controllers 170, and a system management unit (SMU) 180.

[0009] The CPU core complex 110 includes a CPU core 112 and a CPU core 114. In this embodiment, the CPU core complex 110 includes two CPU cores, but in other embodiments, the CPU core complex 110 may include any number of CPU cores. Each of the CPU cores 112 and 114 is bidirectionally connected to a system management network (SMN) that forms a control fabric and a data fabric 125, and is capable of providing memory access requests to the data fabric 125. Each of the CPU cores 112 and 114 may be a single core or a core complex including two or more single cores that share a particular resource such as a cache.

[0010] Graphics core 120 is a high-performance graphics processing unit (GPU) capable of performing graphics operations such as vertex processing, fragment processing, shading, texture blending, etc. in a highly integrated and parallel manner. Graphics core 120 is bidirectionally connected to SMN and data fabric 125 and can provide memory access requests to data fabric 125. In this regard, APU 100 can support either a unified memory architecture in which CPU core complex 110 and graphics core 120 share the same memory space, or a memory architecture in which CPU core complex 110 and graphics core 120 share a portion of the memory space, while graphics core 120 also uses a private graphics memory that is inaccessible to CPU core complex 110.

[0011] Display engine 122 renders and rasterizes objects generated by graphics core 120 for display on a monitor. Graphics core 120 and display engine 122 are bidirectionally connected to a common memory management hub 140 for uniform translation to appropriate addresses in memory system 130, which is bidirectionally connected to data fabric 125 for generating such memory accesses and receiving read data returned from the memory system.

[0012] Data fabric 125 includes a crossbar switch for routing memory access requests and responses between any memory accessing agent and memory management hub 140. Data fabric 125 also includes a system memory map defined by the system's basic input / output system (BIOS) and buffers for each virtual connection for determining the destination of memory accesses based on the system configuration.

[0013] Peripheral controllers 160 include a Universal Serial Bus (USB) controller 162 and a Serial Advanced Technology Attachment (SATA) interface controller 164, each of which is bidirectionally connected to a system hub 166 and the SMN bus. These two controllers are merely examples of peripheral controllers that may be used in APU 100.

[0014] The peripheral bus controller 170 includes a system controller or "south bridge" (SB) 172 and a peripheral component interconnect express (PCIe) controller 174, each of which is bidirectionally connected to an input / output (I / O) hub 176 and the SMN bus. The I / O hub 176 is also bidirectionally connected to the system hub 166 and the data fabric 125. Thus, for example, a CPU core can program registers in the USB controller 162, the SATA interface controller 164, the SB 172, or the PCIe controller 174 through accesses routed by the data fabric 125 through the I / O hub 176. The software and firmware of the APU 100 is stored in a system data drive or a system BIOS memory (not shown), which can be any of a variety of non-volatile memory types, such as read-only memory (ROM), flash electrically erasable programmable ROM (EEPROM), etc. Typically, the BIOS memory is accessed via the PCIe bus and the system data drive is accessed via the SATA interface.

[0015] SMU 180 is a local controller that controls the operation of resources on APU 100 and synchronizes communications between them. SMU 180 manages the power-up sequencing of the various processors on APU 100 and controls multiple off-chip devices via reset, enable, and other signals. SMU 180 includes one or more clock sources (not shown), such as phase-locked loops (PLLs), that provide clock signals to each of the components of APU 100. SMU 180 may also manage power to the various processors and other functional blocks, and may receive measured power consumption values ​​from CPU cores 112, 114 and graphics core 120 to determine appropriate power states.

[0016] Memory management hub 140 and associated physical interfaces (PHYs) 151, 152 are integrated with APU 100 in this embodiment. Memory management hub 140 includes memory channels 141, 142 and power engine 149. Memory channel 141 includes host interface 145, memory channel controller 143, and physical interface 147. Host interface 145 bidirectionally connects memory channel controller 143 to data fabric 125 via a serial presence detect link (SDP). Physical interface 147 bidirectionally connects memory channel controller 143 to PHY 151 and conforms to the DDR PHY Interface (DFI) specification. Memory channel 142 includes host interface 146, memory channel controller 144, and physical interface 148. Host interface 146 bidirectionally connects memory channel controller 144 to data fabric 125 via another SDP. The physical interface 148 bidirectionally connects the memory channel controller 144 to the PHY 152 and is compliant with the DFI specification. The power engine 149 is bidirectionally connected to the SMU 180 via the SMN bus, bidirectionally connected to the PHYs 151 and 152 via an Advanced Peripheral Bus (APB), and also bidirectionally connected to the memory channel controllers 143 and 144. The PHY 151 is bidirectionally connected to the memory channel 131. The PHY 152 is bidirectionally connected to the memory channel 133.

[0017] Memory management hub 140 and associated physical interfaces (PHYs) 151, 152 are integrated with APU 100 in this embodiment. Memory management hub 140 includes memory channels 141, 142 and power engine 149. Memory channel 141 includes host interface 145, memory channel controller 143, and physical interface 147. Host interface 145 bidirectionally connects memory channel controller 143 to data fabric 125 via a serial presence detect link (SDP). Physical interface 147 bidirectionally connects memory channel controller 143 to PHY 151 and conforms to the DDR PHY Interface (DFI) specification. Memory channel 142 includes host interface 146, memory channel controller 144, and physical interface 148. Host interface 146 bidirectionally connects memory channel controller 144 to data fabric 125 via another SDP. The physical interface 148 bidirectionally connects the memory channel controller 144 to the PHY 152 and is compliant with the DFI specification. The power engine 149 is bidirectionally connected to the SMU 180 via the SMN bus, bidirectionally connected to the PHYs 151 and 152 via an Advanced Peripheral Bus (APB), and also bidirectionally connected to the memory channel controllers 143 and 144. The PHY 151 is bidirectionally connected to the memory channel 131. The PHY 152 is bidirectionally connected to the memory channel 133.

[0018] Memory system 130 includes memory channel 131 and memory channel 133. Memory channel 131 includes a set of dual in-line memory modules (DIMMs) connected to DDRx bus 132, including representative DIMMs 134, 136, and 138, which in this example correspond to separate ranks. Similarly, memory channel 133 includes a set of DIMMs connected to DDRx bus 139, including representative DIMMs 135, 137, and 139.

[0019] The APU 100 acts as the central processing unit (CPU) of the host data processing system and provides a variety of buses and interfaces useful in modern computer systems, including two double data rate (DDRx) memory channels, a PCIe root complex for connecting to a PCIe link, a USB controller for connecting to a USB network, and an interface to a SATA mass storage device.

[0020] The APU 100 also implements various system monitoring and power saving functions. One system monitoring function in particular is temperature monitoring. For example, if the APU 100 gets too hot, the SMU 180 can reduce the frequency and voltage of the CPU cores 112, 114 and / or the graphics core 120. If the APU 100 gets too hot, it can be shut down completely. Temperature events can also be received by the SMU 180 from external sensors over the SMN bus, and the SMU 180 can reduce the clock frequency and / or power supply voltage accordingly.

[0021] Figure 2 is a block diagram of a memory controller 200 suitable for use in an APU such as Figure 1. The memory controller 200 generally includes a memory channel controller 210 and a power controller 250. The memory channel controller 210 generally includes an interface 212, a memory interface queue 214 ("memory interface queue", "queue"), a command queue 220, an address generator 222, a content addressable memory (CAM) 224, a replay queue 230, a replay control logic 231, a refresh logic block 232, a timing block 234, a page table 236, an arbiter 238, an error correction code (ECC) check circuit 242, an ECC generation block 244, a data buffer 246, a non-volatile (NV) buffer 247, and a NV queue 248.

[0022] Interface 212 has a first bidirectional connection via an external bus to data fabric 125 and has an output. In memory controller 200, this external bus is compatible with the Advanced Extensible Interface version 4, known as "AXI4," defined by ARM Holdings, PLC of Cambridge, England, but may be other types of interfaces in other embodiments. Interface 212 translates memory access requests from a first clock domain, known as the FCLK (or MEMCLK) domain, to a second clock domain internal to memory controller 200, known as the UCLK domain. Similarly, memory interface queue 214 provides memory accesses from the UCLK domain to the DFICLK domain associated with the DFI interface.

[0023] Address generator 222 decodes addresses of memory access requests received from data fabric 125 via the AXI4 bus. The memory access requests include an access address in a physical address space expressed in a normalized format. Address generator 222 converts the normalized address into a format that can be used to address actual memory devices in memory system 130 and efficiently schedule the associated access. This format includes a region identifier that associates the memory access request with a particular rank, row address, column address, bank address, and bank group. At startup, the system BIOS queries the memory devices in memory system 130 to determine their size and configuration, and programs address generator 222 and a set of associated configuration registers. Address generator 222 converts the normalized address into the appropriate format using the configuration stored in the configuration registers. Address generator 222 decodes the address range of the memory, including the NVDIMM-P memory, and stores a decoded signal in command queue 220 that indicates whether the memory access request is a request to an NVDIMM-P. The arbiter 238 can then prioritize the NVDIMM-P requests with appropriate priority over other requests. The command queue 220 is a queue of memory access requests received from memory access agents in the APU 100, such as the CPU cores 112, 114 and the graphics core 120. The command queue 220 stores the address fields decoded by the address generator 222, as well as other address information that allows the arbiter 238 to efficiently select memory accesses, including access types and quality of service (QoS) identifiers. The CAM 224 contains information to implement ordering rules, such as write-after-write (WAW) and read-after-write (RAW) ordering rules.

[0024] An error correcting code (ECC) generation block 244 determines the ECC of the write data sent to the NVDIMM-P. An ECC check circuit 242 checks the received ECC against the incoming ECC.

[0025] Replay queue 230 is a temporary queue for storing selected memory accesses that have been selected by arbiter 238 waiting for a response, such as an address and command parity response. Replay control logic 231 accesses ECC check block 242 to determine whether the returned ECC is correct or indicates an error. Replay control logic 231 initiates and controls a replay sequence in which an access is replayed in the event of a parity or ECC error in one of these cycles. Replayed commands are placed in memory interface queue 214.

[0026] The refresh logic 232 includes state machines for various power-down, refresh, and termination resistor (ZQ) calibration cycles that are generated separately from normal read and write memory access requests received from memory access agents. For example, when a memory rank is in precharge power-down, it must be periodically woken up to perform refresh cycles. The refresh logic 232 periodically generates refresh commands to prevent data errors caused by charge leakage from storage capacitors of memory cells in DRAM chips. In addition, the refresh logic 232 periodically calibrates ZQ to prevent mismatches in on-die termination resistors due to temperature changes in the system.

[0027] Arbiter 238 is bidirectionally connected to command queue 220 and is the heart of memory channel controller 210. Arbiter 238 improves efficiency by intelligently scheduling accesses to increase memory bus utilization. Using timing block 234, arbiter 238 enforces proper timing relationships by determining whether a particular access in command queue 220 is eligible for issue based on DRAM timing parameters. For example, each DRAM has a "t RC There is a minimum specified time between active commands, known as a "timeout". Timing block 234 maintains a set of counters that determine eligibility based on this and other timing parameters defined in the JEDEC specifications, and is bidirectionally connected to replay queue 230. Page table 236 maintains state information about active pages in each bank and rank of the memory channel to arbiter 238, and is bidirectionally connected to replay queue 230.

[0028] NV buffer 247 stores NV read commands in NV queue 248, both for use in replay sequences and for managing NV read responses. NV buffer 247 is bidirectionally connected to memory interface queue 214 for processing RD_RDY and SEND commands, as described further below.

[0029] In response to a write memory access request received from interface 212, ECC generation block 244 calculates an ECC according to the write data. Data buffer 246 stores the write data and ECC for the received memory access request. Data buffer 246 outputs the combined write data / ECC to memory interface queue 214 when arbiter 238 selects the corresponding write access for dispatch to the memory channel.

[0030] The power controller 250 generally includes an interface 252 to the Advanced Extensible Interface version 1 (AXI), an APB interface 254, and a power engine 260. The interface 252 has a first bidirectional connection to the SMN, including an input for receiving an event signal labeled "EVENT_n" as shown separately in FIG. 2, and an output. The APB interface 254 has an input connected to an output of the interface 252 and an output for connecting to the PHY via the APB. The power engine 260 has an input connected to an output of the interface 252 and an output connected to an input of the memory interface queue 214. The power engine 260 includes a set of configuration registers 262, a microcontroller (μC) 264, a self-refresh controller (SLFREF / PE) 266, and a reliable read / write timing engine (RRW / TE) 268. The configuration registers 262 are programmed via the AXI bus and store configuration information to control the operation of various blocks in the memory controller 200. Thus, configuration register 262 has outputs connected to these blocks not shown in detail in Figure 2. Self-refresh controller 266 is an engine that allows for manual generation of refreshes in addition to the automatic generation of refreshes by refresh logic 232. Reliable read / write timing engine 268 provides a continuous stream of memory accesses to memory or I / O devices, such as for maximum read latency (MRL) training and loopback testing of DDR interfaces.

[0031] The memory channel controller 210 includes circuitry that allows it to select memory accesses for dispatch to the associated memory channel. To make the desired arbitration decisions, the address generator 222 decodes address information into pre-decode information including rank, row address, column address, bank address, and bank group in the memory system, and the command queue 220 stores the pre-decode information. The configuration registers 262 store configuration information to determine how the address generator 222 decodes the received address information. The arbiter 238 uses the decoded address information, timing eligibility information indicated by the timing block 234, and active page information indicated by the page table 236 to efficiently schedule memory accesses while adhering to other criteria such as quality of service (QoS) requirements. For example, the arbiter 238 avoids the overhead of pre-charge and activate commands required to implement the basic settings for accesses to open pages and modify memory pages by interleaving them with read and write accesses to one bank, hiding the overhead accesses to the other bank. In particular, during normal operation, the arbiter 238 typically holds open pages in different banks until those pages are required to be precharged before selecting a different page.

[0032] FIG. 3 is a block diagram of a data processing system 300, according to some embodiments. Data processing system 300 includes APU 310 and memory system 330. To focus on the memory arrangement, various other parts of the system are not shown. APU 310 includes a memory controller, such as memory controller 200 (FIG. 2), that supports a heterogeneous memory channel to interface with memory system 330. In addition to a normal DDRx memory channel, APU 310 supports NVDIMM-P 338 with a heterogeneous memory channel 330 with both normal registered DIMMs or RDIMMs 334, 336 and NVDIMM-P 338, in addition to a homogeneous memory channel 340 with only RDIMMs 344, 346, 348 connected via bus 342. Other DIMM types, such as LRDIMMs and UDIMMs, are supported in some embodiments. In this embodiment, heterogeneous memory channel 330 connects to both NVDIMM-P and RDIMMs, but in some embodiments, the heterogeneous memory channel has the ability to interface with all NVDIMM-P type DIMMs.

[0033] In accordance with the draft NVDIMM-P standard, transactions between the memory controller of the APU 310 and the NVDIMM-P 338 are protected by "link" ECC. Link ECC ensures data integrity of data transfers between the memory controller and the NVDIMM over the bus 332. It protects against data corruption on the link caused by random or transient errors according to known ECC mechanisms. The protection depends on the ECC code used. ECC may, for example, allow single-bit correction with multi-bit error detection. In response to detection of an uncorrectable error, the memory controller may replay the transaction to ensure that the transient or random error does not persist, and may also report both correctable and uncorrectable errors to the operating system.

[0034] While an NVDIMM-P type DIMM is described in this embodiment, other embodiments use the techniques herein to interface with other types of storage class memory (SCM) modules through heterogeneous memory channels. As used herein, SCM refers to a memory module with non-volatile memory addressable in the system memory space. The non-volatile memory in the SCM module can be buffered in RAM and / or paired with RAM mounted on the SCM module. The SCM memory address map is viewed from the perspective of the operating system (OS) alongside the traditional DRAM population. The OS typically recognizes that the address range defined in SCM is a "different" type of memory than traditional memory. This difference is to inform the OS that this memory is more potential and may have a persistent quality. The OS can map the SCM memory as direct access memory or file system access memory. Direct access means that the OS accesses the SCM address range as physical, addressable memory. File system access means that the OS manages the persistent memory as part of the file system and manages access to the SCM through a file-based API. Eventually the request reaches the memory controller within the SCM address range, regardless of how the higher level OS manages the access.

[0035] FIG. 4 is a flow diagram of a process 400 for processing memory access commands, according to some embodiments. Process 400 focuses on processing non-volatile read commands and is suitable for implementation with memory controller 200 of FIG. 2 or other memory controller devices. Process 400 begins at block 402 with receiving a number of memory access requests including a volatile memory read, a volatile memory write, a non-volatile memory read, and a non-volatile memory write. At block 404, memory access commands are scheduled and placed in a memory interface queue to satisfy the requests. Block 404 generally includes decoding a memory access command for the memory access request and may include holding the memory access command in a command queue before being scheduled and placed in the memory interface queue by an arbiter, such as arbiter 238 (FIG. 2).

[0036] At block 406, the process 400 stores the non-volatile read command placed in the memory interface queue in a non-volatile command queue (NV queue). At block 408, the memory access command from the memory interface queue is sent over a heterogeneous memory channel coupled to at least one non-volatile dual in-line memory module (DIMM). In some embodiments, the memory channel is coupled to at least one volatile DIMM.

[0037] As shown in block 410, for a non-volatile read command sent over a heterogeneous memory channel, the non-volatile DIMM generally responds after a non-deterministic period due to an unpredictable process of reading the requested data, which may be in the non-volatile memory of the non-volatile DIMM, the DRAM of the non-volatile DIMM, or the cache of the media controller. During the non-deterministic period, other memory access commands are generally sent from the memory interface queue. When the media controller of the non-volatile DIMM completes the process of reading the requested data, it sends a ready response signal "RD_RDY" to the memory controller. The process waits to receive RD_RDY for each non-volatile read. Generally, the RD_RDY signal is sent and received on a sub-channel of the heterogeneous memory channel that is separate from the sub-channel on which the memory interface queue receives responses to memory access commands. For example, with an NVDIMM-P memory channel, the RD_RDY signal is generally sent on a "RSP_R" line of the memory channel that is separate from the "CMD" and "DQ" lines on which the command and data are sent.

[0038] At block 412, a RD_RDY signal is received from the non-volatile DIMM indicating that response data is available for an associated non-volatile read command. Control circuitry, which in this embodiment is NV buffer 247 (FIG. 2), receives the RD_RDY signal. In response, at block 414, NV buffer 247 places a SEND command in the memory interface queue, whereby the SEND command is scheduled or queued for transmission to the non-volatile DIMM.

[0039] Upon receiving the SEND command, the non-volatile DIMM media controller sends response data read with the non-volatile read command back to the memory controller, including the command and associated identifier. The associated identifier in this embodiment is the read identifier "RID" of the read command. At block 416, the response data and associated identifier are received from the non-volatile DIMM of the memory controller. In response, the NV buffer of the memory controller uses the associated identifier to identify a non-volatile read command in the NV queue that has the same associated identifier. At block 418, the response data is provided to fulfill the associated non-volatile read request for which the non-volatile ready command was generated. This fulfills the request and removes the associated non-volatile read command from the NV queue.

[0040] In some embodiments, process 400 at block 404 includes scheduling the memory access commands with an arbiter, such as arbiter 238 (FIG. 2). In one example, the process groups the nonvolatile read command with other nonvolatile read commands or volatile read commands before placing the memory access command in the memory interface queue. In some embodiments, process 400 at block 414 further includes grouping the SEND command with a group of nonvolatile or volatile read commands before placing the SEND command in the memory interface queue. Because the response time of the SEND command is critical, memory interface queue 214 can mix SEND commands with other commands, such as regular DDRx reads and writes and nonvolatile writes to volatile memory.

[0041] FIG. 5 is a flow diagram of a process for handling errors, according to some embodiments. FIGS. 6 and 7 are a series of diagrams 600, 700 illustrating the process of FIG. 5. Referring to FIGS. 5-7, process 500 generally handles storing commands and providing a recovery sequence in which channels and non-volatile DIMMs are reset and recent commands are replayed to correct errors. Although blocks are shown in a particular order, this order is not limiting and some blocks occur in parallel on an ongoing basis. Process 500 is suitable for execution by memory controller 200 (FIG. 2) or other memory controllers with appropriate NV and replay queues and error detection capabilities.

[0042] At block 502, a copy of the non-volatile read command is stored in the NV queue where it is placed in the memory interface queue for transmission to the respective non-volatile DIMM. This is illustrated in diagram 600 by arrow 601 which shows the command going to the memory interface queue if selected for transmission, and arrow 602 which shows the copy of the non-volatile read command stored in the NV queue. Other types of commands have copies stored in the replay queue, as shown in block 504, including non-volatile writes, volatile writes, volatile reads, SEND commands, and other memory access commands. Arrow 603 in diagram 600 shows other commands stored in the replay queue. Blocks 502 and 504 occur continually as the memory controller processes memory access requests.

[0043] If no errors are detected, process 500 continues to store commands in the NV and replay queues, where they are held until executed and removed from their respective queues. Process 500 at block 506 detects whether there was an error in one of the DIMMs on the memory channel that requires a recovery sequence, initiates a recovery sequence, and proceeds to either block 507, 508, or 509 depending on the nature of the error detected. If the error detected is a command parity error, process 500 proceeds from block 506 to block 507, where it sends a command to clear the parity error in each DIMM on the memory channel. If a write or read ECC error is detected, process 500 proceeds to block 508, where it clears the write or read ECC status. If both a command parity error and a write / read ECC error are detected, process 500 proceeds to block 509, where it sends a command to clear the parity error in each DIMM on the channel, and proceeds to block 510, where it clears the write or read ECC status. In some embodiments, if the process cannot determine the error type, block 509 also proceeds to block 510 upon clearing the errors for both error types to ensure that the error status is completely cleared. Process 500 then proceeds to block 511 to continue the recovery sequence.

[0044] In block 511, if a multi-purpose register (MPR) mode is currently active, it is disabled. The memory controller resets the first-in-first-out (FIFO) buffer of the PHY in block 512. In block 514, all read IDs (RIDs) are reset to the non-volatile DIMMs on the memory channel and channel buffers. In some embodiments, block 514 includes sending a reset RID (RST_RID) command, waiting for a ready (RDY) response, sending a SEND command, and waiting for the resulting data packets to ensure that all outstanding reads have been reset so that the non-volatile DIMMs do not send RDY responses for pending read commands.

[0045] If write credits are needed, they are requested and obtained at block 516. In some embodiments, block 516 includes sending a write status command to determine the number of write credits available to the non-volatile DIMM, determining whether more write credits are needed, and then requesting and obtaining more write credits. The request may include looping through multiple write credit requests until sufficient write credits are received.

[0046] If MPR mode was active prior to the recovery sequence, it is re-enabled at block 518 in order to place the non-volatile DIMM in the same state as if the error had occurred and replay the necessary commands.

[0047] At this point, the recovery sequence is reset and various portions of the channel and non-volatile DIMM are cleared to begin replaying the commands. At block 520, process 500 begins replaying the commands starting with a selected command from the replay queue. In some embodiments, the selected command includes any volatile reads, multi-purpose register (MPR) related commands, SEND commands associated with MPR related commands, volatile writes, and non-volatile writes that are in the replay queue. SEND commands associated with non-volatile reads are stored in the replay queue for reporting and debugging purposes, but are not sent at block 520. FLUSH commands that are in the replay queue are also not replayed.

[0048] Preferably, blocks 506 through 520 are executed under the control of replay control logic 231 (FIG. 2) or similar replay control circuitry. The process then passes control to NV buffer 247 to complete the replay of the non-volatile read command.

[0049] At block 522, process 500 includes replaying all non-volatile reads stored in the NV queue by sending them to the memory interface queue. Preferably, this replay occurs in groups after sending all selected memory access commands stored in the replay queue. In other embodiments, the non-volatile reads are replayed in groups before the commands are replayed at block 520. As described with respect to FIG. 4, the response time of non-volatile reads is non-deterministic, that is, the SEND command first sent following a RD_RDY response of a non-volatile read is not necessarily sent again in the same order. To handle this ordering, process 500 includes skipping the SEND command stored in the replay queue associated with the non-volatile read and responding to the RD-RDY response when it arrives for the non-volatile read at block 524 by generating a new SEND command in response to the read ready (RD_RDY) response received from the non-volatile DIMM during the recovery sequence. At this point, the replay sequence is complete and the memory controller terminates the replay sequence and returns to normal operating conditions.

[0050] Thus, the memory controller and data processing system as described herein improves the memory controller's ability to interface with non-volatile DIMMs. Additionally, the memory controller herein reduces the length of the memory interface queue by eliminating the need for the memory interface queue to hold non-deterministic and potentially long latency non-volatile read commands until they are executed.

[0051] The memory controller 200 or any portion thereof, such as the arbiter 238 of FIG. 2, may be described or represented by a computer-accessible data structure in the form of a database or other data structure that can be read by a program and used directly or indirectly to manufacture an integrated circuit. For example, this data structure may be a behavioral or register transfer level (RTL) description of the hardware functionality in a high-level design language (HDL) such as Verilog or VHDL. The description may be read by a synthesis tool, which may synthesize the description to generate a netlist that includes a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware that comprises the integrated circuit. The netlist may then be placed and routed to generate a data set that describes the geometry to be applied to a mask. The mask may then be used in various semiconductor manufacturing processes to generate the integrated circuit. Alternatively, the database on the computer-accessible storage medium may be a netlist (with or without a synthesis library) or a data set, or a Graphics Data System (GDS) II data, as appropriate.

[0052] While specific embodiments have been described, various modifications to these embodiments will be apparent to those skilled in the art. For example, the internal architecture of memory channel controller 210 and / or power engine 250 may vary in different embodiments. Memory controller 200 may interface to other types of memory other than NVDIMM-P memory, such as, for example, high bandwidth memory (HBM), RAM bus DRAM (RDRAM), etc. While the illustrated embodiment shows each rank of memory corresponding to a separate DIMM, in other embodiments, each DIMM may support multiple ranks. Additionally, a channel may be filled entirely with non-volatile DIMMs, although heterogeneous memory channels are generally supported. Additionally, while two separate queues are described to achieve recovery and replay, in some embodiments, a single dedicated storage queue is used.

[0053] It is therefore intended by the appended claims to cover all modifications of the disclosed embodiments that fall within the scope of the disclosed embodiments.

Claims

1. 1. A memory controller, comprising: a command queue having a first input for receiving memory access commands including volatile reads, volatile writes, non-volatile reads and non-volatile writes, and an output, the command queue having a plurality of entries; a memory interface queue having an input for receiving a command selected from the command queue and an output for coupling to a heterogeneous memory channel coupled to at least one non-volatile storage class memory (SCM) module; at least one storage queue for storing memory access commands placed in the memory interface queue; a replay control circuit for detecting an occurrence of an error requiring a recovery sequence and for initiating the recovery sequence in response to the error, the replay control circuit transmitting selected memory access commands from the at least one storage queue by grouping non-volatile read commands separately from all pending volatile reads, volatile writes and non-volatile writes. Memory controller.

2. the replay control circuit transmits selected memory access commands from the at least one storage queue by delaying transmission of all non-volatile reads until all of the pending volatile reads, volatile writes, and non-volatile writes have been transmitted. The memory controller of claim 1 .

3. the at least one storage queue includes a non-volatile command queue (NV queue) coupled to an output of the command queue for storing non-volatile read commands placed in the memory interface queue, and a replay queue coupled to an output of the command queue for storing other selected memory access commands placed in the memory interface queue. The memory controller of claim 1 .

4. the recovery sequence includes skipping a SEND command stored in the replay queue associated with a non-volatile read; and generating a new SEND command in response to a read ready (RD_RDY) response received from the at least one non-volatile SCM module during the recovery sequence. The memory controller of claim 3.

5. the recovery sequence includes requesting and obtaining a buffer write credit on the at least one non-volatile SCM module; The memory controller of claim 1 .

6. 1. A method comprising: receiving a plurality of memory access requests including a volatile memory read, a volatile memory write, a non-volatile memory read, and a non-volatile memory write; placing a memory access command for satisfying the memory access request in a memory interface queue and transmitting the memory access command from the memory interface queue to a heterogeneous memory channel coupled to at least one non-volatile storage class memory (SCM) module; storing the memory access commands placed in the memory interface queue in at least one storage queue; detecting that an error has occurred requiring a recovery sequence, and in response to the error, executing the recovery sequence including transmitting selected memory access commands from the at least one storage queue by grouping transmission of all non-volatile reads separately from all pending volatile reads, volatile writes, and non-volatile writes. method.

7. transmitting selected memory access commands from the at least one storage queue by delaying transmission of all non-volatile reads until all of the pending volatile reads, volatile writes, and non-volatile writes have been transmitted. The method of claim 6.

8. the at least one storage queue includes a non-volatile command queue (NV queue) for storing non-volatile read commands placed in the memory interface queue, and a replay queue for storing selected memory access commands placed in the memory interface queue. The method of claim 6.

9. the recovery sequence includes skipping a SEND command stored in the replay queue associated with a non-volatile read; and generating a new SEND command in response to a read ready (RD_RDY) response received from the at least one non-volatile SCM module during the recovery sequence.

9. The method of claim 8.

10. the recovery sequence includes resetting non-volatile reads of all read identifiers (RIDs) in the heterogeneous memory channel buffers prior to transmitting the selected memory access command; The method of claim 6.

11. the error requiring the recovery sequence is one of a command parity error, a write command error correcting code (ECC) error associated with the at least one non-volatile SCM module, and a read command ECC error associated with the at least one non-volatile SCM module. The method of claim 6.

12. 1. A data processing system comprising: A central processing unit; a data fabric coupled to the central processing unit; a memory controller coupled to the data fabric to satisfy memory access requests from the central processing unit; The memory controller includes: a command queue having a first input for receiving memory access commands including volatile reads, volatile writes, non-volatile reads and non-volatile writes, and an output, the command queue having a plurality of entries; a memory interface queue having an input for receiving a command selected from the command queue and an output for coupling to a heterogeneous memory channel having at least one non-volatile storage class memory (SCM) module coupled thereto; at least one storage queue for storing memory access commands placed in the memory interface queue; a replay control circuit for detecting an occurrence of an error requiring a recovery sequence and for initiating the recovery sequence in response to the error, the replay control circuit transmitting selected memory access commands from the at least one storage queue by grouping non-volatile read commands separately from all pending volatile reads, volatile writes and non-volatile writes. Data processing system.

13. the replay control circuit transmits selected memory access commands from the at least one storage queue by delaying transmission of all non-volatile reads until all of the pending volatile reads, volatile writes, and non-volatile writes have been transmitted.

13. The data processing system of claim 12.

14. the at least one storage queue includes a non-volatile command queue (NV queue) coupled to an output of the command queue for storing non-volatile read commands placed in the memory interface queue, and a replay queue coupled to an output of the command queue for storing selected memory access commands placed in the memory interface queue.

13. The data processing system of claim 12.

15. the recovery sequence includes skipping a SEND command stored in the replay queue associated with a non-volatile read; and generating a new SEND command in response to a read ready (RD_RDY) response received from the at least one non-volatile SCM module during the recovery sequence.

15. The data processing system of claim 14.

Images