Non-volatile memory module error reporting

Abstract

The memory controller includes a command queue, a memory interface queue, and a non-volatile error reporting circuit. The command queue receives memory access commands including a volatile read, a volatile write, a non-volatile read, and a non-volatile write, and has an output. The memory interface queue has an input coupled to the output of the command queue and an output for coupling to a non-volatile storage class memory (SCM) module. The non-volatile error reporting circuit maps the error conditions from a first number of possible error conditions associated with the non-volatile SCM module to a second, smaller number of virtual error types to identify and report error conditions associated with the non-volatile SCM module to an error monitoring module of the host operating system, the mapping being based at least on a classification of the error conditions as either adversely affecting or not adversely affecting executable processes executing on the host operating system.

Description

[Background technology]

[0001] Computer systems typically use inexpensive, high-density dynamic random access memory (DRAM) chips for main memory. Most DRAM chips sold today are compatible with various Double Data Rate (DDR) DRAM standards promulgated by the Joint Electron Device Engineering Council (JEDEC). A DDR memory controller is used to manage the interface between the various memory access agents and the DDR DRAM according to the published DDR standards.

[0002] Non-volatile dual in-line memory modules with persistent storage (NVDIMM-P) are storage class memories that can be used in place of standard DDR DIMMs in some applications, but contain persistent memory. However, these memories contain several types of error conditions that are different from DDR error conditions. Furthermore, error conditions associated with NVDIMM-P have different impacts on the operating system and running processes that use the memory than the impact of error conditions associated with standard DDR DIMMs. [Brief description of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram illustrating an accelerated processing unit (APU) and memory system known in the prior art. [Diagram 2] 2 is a block diagram illustrating a memory controller suitable for use in an APU, such as the APU of FIG. 1, in accordance with some embodiments. [Diagram 3] 1 is a block diagram illustrating a data processing system according to some embodiments. [Figure 4] FIG. 2 is a block diagram illustrating a non-volatile buffer according to some embodiments. [Diagram 5] 1 is a block diagram illustrating a method for using non-volatile memory in a computer system according to some embodiments. [Figure 6] FIG. 1 is a flow diagram of a process for error reporting, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004] In the following description, the use of the same reference numbers in different figures indicates similar or identical items. Unless otherwise stated, the term "coupled" and its related verb forms include both direct and indirect electrical connections by means known in the art, and unless otherwise stated, reference to a direct connection connotes alternative embodiments that also use an appropriate form of indirect electrical connection.

[0005] The memory controller includes a command queue, a memory interface queue, and a non-volatile error reporting circuit. The command queue has a first input for receiving memory access commands including volatile read, volatile write, non-volatile read, and non-volatile write, and an output, and has a plurality of entries. The memory interface queue has an input coupled to the output of the command queue and an output for coupling to a non-volatile storage class memory (SCM) module. The non-volatile error reporting circuit maps error conditions from a first number of possible error conditions associated with the non-volatile SCM module to a second, smaller number of virtual error types to identify and report error conditions associated with the non-volatile SCM module to an error monitoring module of a host operating system. The mapping is based at least on a classification of the error conditions as having or not having a detrimental effect on executable processes executing on the host operating system.

[0006] The method includes receiving a plurality of memory access requests including a volatile memory read, a volatile memory write, a non-volatile memory read, and a non-volatile memory write. The method places memory access commands for satisfying the memory access requests in a memory interface queue and transmits the memory access commands from the memory interface queue to a memory channel coupled to a non-volatile storage class memory (SCM) module. Identifying an error condition associated with the memory channel based on the information received over the memory channel. The error conditions are mapped from a first number of possible error conditions associated with the non-volatile SCM module to a second, smaller number of virtual error types for reporting to an error monitoring module of a host operating system, the mapping based at least on a classification of whether the error conditions adversely affect or do not adversely affect executable processes executing on the host operating system.

[0007] The data processing system includes a central processing unit, a data fabric coupled to the central processing unit, and a memory controller coupled to the data fabric for fulfilling memory access requests made by the data fabric. The memory controller includes a command queue, a memory interface queue, and a non-volatile error reporting circuit. The command queue has a first input for receiving memory access commands including volatile read, volatile write, non-volatile read, and non-volatile write, and an output, and has a plurality of entries. The memory interface queue has an input coupled to the output of the command queue and an output for coupling to a non-volatile storage class memory (SCM) module. The non-volatile error reporting circuit maps error conditions from a first number of possible error conditions associated with the non-volatile SCM module to a second, smaller number of virtual error types to identify and report error conditions associated with the non-volatile SCM module to an error monitoring module of a host operating system, the mapping being based on a classification of at least whether the error conditions adversely affect or do not adversely affect executable processes executing on the host operating system.

[0008] 1 is a block diagram illustrating an accelerated processing unit (APU) 100 and memory system 130 known in the prior art. APU 100 is an integrated circuit suitable for use as a processor in a host data processing system and generally includes a central processing unit (CPU) core complex 110, a graphics core 120, a set of display engines 122, a memory management hub 140, a data fabric 125, a set of peripheral controllers 160, a set of peripheral bus controllers 170, and a system management unit (SMU) 180.

[0009] CPU core complex 110 includes CPU core 112 and CPU core 114. In this example, CPU core complex 110 includes two CPU cores, but in other embodiments, CPU core complex 110 may include any number of CPU cores. Each of CPU cores 112, 114 is bidirectionally connected to a system management network (SMN) and data fabric 125 that form a control fabric, and is capable of providing memory access requests to data fabric 125. Each of CPU cores 112, 114 may be a single core, or may be a core complex with two or more single cores that share certain resources, such as a cache.

[0010] Graphics core 120 is a high-performance graphics processing unit (GPU) capable of performing graphics operations such as vertex processing, fragment processing, shading, texture blending, etc. in a highly integrated and parallel manner. Graphics core 120 is bidirectionally connected to SMN and data fabric 125 and can provide memory access requests to data fabric 125. In this regard, APU 100 can support either a unified memory architecture in which CPU core complex 110 and graphics core 120 share the same memory space, or a memory architecture in which CPU core complex 110 and graphics core 120 share a portion of the memory space, while graphics core 120 also uses a private graphics memory that is inaccessible to CPU core complex 110.

[0011] Display engine 122 renders and rasterizes objects generated by graphics core 120 for display on a monitor. Graphics core 120 and display engine 122 are bidirectionally connected to a common memory management hub 140 for uniform translation to appropriate addresses in memory system 130, which in turn is bidirectionally connected to data fabric 125 for generating such memory accesses and receiving read data returned from the memory system.

[0012] Data fabric 125 includes a crossbar switch for routing memory access requests and responses between any memory accessing agent and memory management hub 140. Data fabric 125 also includes a system memory map defined by a basic input / output system (BIOS) to determine the destination of memory accesses based on the system configuration, and buffers for each virtual connection.

[0013] Peripheral controllers 160 include a Universal Serial Bus (USB) controller 162 and a Serial Advanced Technology Attachment (SATA) interface controller 164, each of which is bidirectionally connected to a system hub 166 and the SMN bus. These two controllers are only examples of peripheral controllers that may be used in APU 100.

[0014] Peripheral bus controller 170 includes a system controller or "south bridge" (SB) 172 and a peripheral component interconnect express (PCIe) controller 174, each of which is bidirectionally connected to an input / output (I / O) hub 176 and an SMN bus. I / O hub 176 is also bidirectionally connected to system hub 166 and data fabric 125. Thus, for example, a CPU core can program registers of USB controller 162, SATA interface controller 164, SB 172, or PCIe controller 174 by accesses routed by data fabric 125 through I / O hub 176. Software or firmware for APU 100 is stored in a system data drive or system BIOS memory (not shown), which may be any of a variety of non-volatile memory types, such as read only memory (ROM), flash electrically erasable programmable ROM (EEPROM), etc. Typically, the BIOS memory is accessed via the PCIe bus and the system data drive is accessed via the SATA interface.

[0015] SMU 180 is a local controller that controls the operation of APU 100's resources and synchronizes communication between them. SMU 180 manages the sequencing of power-up of the various processors of APU 100 and controls multiple off-chip devices via reset, enable, and other signals. SMU 180 includes one or more clock sources (not shown), such as a phase-locked loop (PLL), to provide clock signals to each of APU 100's components. SMU 180 may also manage the power of the various processors and other functional blocks, receive measured power consumption values ​​from CPU cores 112, 114 and graphics core 120, and determine appropriate power states.

[0016] In this embodiment, memory management hub 140 and associated physical interfaces (PHYs) 151, 152 are integrated with APU 100. Memory management hub 140 includes memory channels 141, 142 and power engine 149. Memory channel 141 includes host interface 145, memory channel controller 143 and physical interface 147. Host interface 145 bidirectionally connects memory channel controller 143 to data fabric 125 through a serial presence detect link (SDP). Physical interface 147 bidirectionally connects memory channel controller 143 to PHY 151 and conforms to the DDR PHY Interface (DFI) specification. Memory channel 142 includes host interface 146, memory channel controller 144 and physical interface 148. Host interface 146 bidirectionally connects memory channel controller 144 to data fabric 125 through another SDP. Physical interface 148 bidirectionally connects memory channel controller 144 to PHY 152 and is compliant with the DFI specification. Power engine 149 is bidirectionally connected to SMU 180 through the SMN bus, to PHYs 151 and 152 through the APB, and to memory channel controllers 143 and 144. PHY 151 has a bidirectional connection to memory channel 131. PHY 152 has a bidirectional connection to memory channel 133.

[0017] Memory management hub 140 is an instantiation of a memory controller having two memory channel controllers and uses a shared power engine 149 to control the operation of both memory channel controller 143 and memory channel controller 144 in a manner further described below. Each of memory channels 141, 142 can connect to state-of-the-art DDR memories (such as DDR version 4 (DDR4), low power DDR4 (LPDDR4), graphics DDR version 5 (gDDR5), and high bandwidth memory (HBM)) and can accommodate future memory technologies. These memories provide high bus bandwidth and high speed operation. At the same time, they also provide low power modes to conserve power for battery-powered applications such as laptop computers, and built-in temperature monitoring.

[0018] Memory system 130 includes memory channel 131 and memory channel 133. Memory channel 131 includes a set of dual in-line memory modules (DIMMs) connected to Dedra bus 132, including, in this example, representative DIMMs 134, 136, and 138 corresponding to separate ranks. Similarly, memory channel 133 includes a set of DIMMs connected to DDRx bus 129, including representative DIMMs 135, 137, and 139.

[0019] The APU 100 acts as the central processing unit (CPU) of the host data processing system and provides a variety of buses and interfaces useful in modern computer systems, including two double data rate (DDRx) memory channels, a PCIe root complex for connecting to a PCIe link, a USB controller for connecting to a USB network, and an interface to a SATA mass storage device.

[0020] The APU 100 also implements various system monitoring and power saving functions. One system monitoring function in particular is temperature monitoring. For example, if the APU 100 becomes too hot, the SMU 180 can reduce the frequency and voltage of the CPU cores 112, 114 and / or the graphics core 120. If the APU 100 becomes too hot, the APU 100 can be shut down completely. Temperature events can also be received by the SMU 180 from external sensors over the SMN bus, and in response, the SMU 180 can reduce the clock frequency and / or the power supply voltage.

[0021] Figure 2 is a block diagram illustrating a memory controller 200 suitable for use in an APU such as Figure 1. The memory controller 200 generally includes a memory channel controller 210 and a power controller 250. The memory channel controller 210 includes an interface 212, a memory interface queue 214, a command queue 220, an address generator 222, a content addressable memory (CAM) 224, a replay control logic 231 including a replay queue 230, a refresh logic block 232, a timing block 234, a page table 236, an arbiter 238, an error correction code (ECC) check circuit 242, an ECC generation block 244, a data buffer 246, a non-volatile (NV) buffer 247, and a NV queue 248.

[0022] Interface 212 has a first bidirectional connection to data fabric 125 through an external bus and has an output. In memory controller 200, this external bus is compatible with the Advanced Extensible Interface version 4, known as "AXI4," specified by ARM Holdings, PLC (Cambridge, England), but in other embodiments may be other types of interfaces. Interface 212 moves memory access requests from a first clock domain, known as the FCLK (or MEMCLK) domain, to a second clock domain internal to memory controller 200, known as the UCLK domain. Similarly, memory interface queue 214 provides memory accesses from the UCLK domain to the DFICLK domain associated with the DFI interface.

[0023] Address generator 222 decodes addresses of memory access requests received from data fabric 125 over the AXI4 bus. The memory access requests include an access address in a physical address space expressed in a standardized format. Address generator 222 converts the standardized address into a format that can be used to address actual memory devices in memory system 130 as well as efficiently schedule associated accesses. This format includes a region identifier that associates the memory access request with a particular rank, row address, column address, bank address, and bank group. At startup, the system BIOS queries the memory devices in memory system 130 to determine their size and configuration and programs a set of configuration registers associated with address generator 222. Address generator 222 converts the standardized address into the appropriate format using the configuration stored in the configuration registers. Address generator 222 decodes the address range of the memory, including the NVDIMM-P memory, and stores a decoded signal in command queue 220 indicating whether the memory access request is a request to an NVDIMM-P. The arbiter 238 can then prioritize the NVDIMM-P requests with appropriate priority relative to other requests. The command queue 220 is a queue of memory access requests received from memory access agents in the APU 100, such as the CPU cores 112, 114 and the graphics core 120. The command queue 220 stores address fields decoded by the address generator 222 and other address information that allows the arbiter 238 to efficiently select memory accesses, including access types and quality of service (QoS) identifiers. The CAM 224 contains information that implements ordering rules, such as write-after-write (WAW) and read-after-write (RAW) ordering rules.

[0024] An error correction code (ECC) generation block 244 determines the ECC of the write data sent to the NVDIMM-P. An ECC check circuit 242 checks the received ECC against the incoming ECC. Thus, the memory controller and data processing system described herein extends the scope of the data integrity check and provides an end-to-end check by leveraging a limited number of user bits stored in the NVDIMM-P device and available for comparison when the corresponding data is later read.

[0025] Replay queue 230 is a temporary queue for storing selected memory accesses chosen by arbiter 238 awaiting responses, such as address responses and command parity responses. Replay control logic 231 accesses ECC check circuitry 242 to determine whether the returned ECC is correct or indicates an error. Replay control logic 231 initiates and controls a replay sequence in which an access is replayed in the event of a parity or ECC error in one of these cycles. Replayed commands are placed in memory interface queue 214.

[0026] The refresh logic 232 includes state machines for various power-down, refresh, and termination resistor (ZQ) calibration cycles that are generated separately from normal read and write memory access requests received from memory access agents. For example, when a memory rank is in precharge power-down, the memory rank must be periodically woken up to perform refresh cycles. The refresh logic 232 periodically generates refresh commands to prevent data errors caused by charge leakage from storage capacitors of memory cells of DRAM chips. In addition, the refresh logic 232 periodically calibrates ZQ to prevent mismatches in on-die termination resistors due to temperature changes in the system.

[0027] Arbiter 238 is bidirectionally connected to command queue 220 and is the heart of memory channel controller 210. Arbiter 238 improves efficiency by intelligently scheduling accesses to improve memory bus utilization. Arbiter 238 enforces proper timing relationships using timing block 234 by determining whether a particular access in command queue 220 is eligible for issue based on DRAM timing parameters. For example, each DRAM has a minimum specified time between active commands, known as "tRC." Timing block 234 maintains a set of counters that determine eligibility based on this timing parameter and other timing parameters specified in the JEDEC specification, and is bidirectionally connected to replay queue 230. Page table 236 maintains state information regarding active pages in each bank and rank of the memory channel to arbiter 238, and is bidirectionally connected to replay queue 230.

[0028] NV buffer 247 stores NV read commands in NV queue 248 for both use in replay sequences and for managing NV read responses. NV buffer 247 is bidirectionally connected to memory interface queue 214 for processing RD_RDY and SEND commands, as further described below.

[0029] In response to a write memory access request received from interface 212, ECC generation block 244 calculates an ECC according to the write data. Data buffer 246 stores the write data and ECC for the received memory access request. Data buffer 246 outputs the combined write data / ECC to memory interface queue 214 when arbiter 238 selects the corresponding write access for dispatching to a memory channel.

[0030] The power controller 250 generally includes an interface 252 to an Advanced Extensible Interface version 1 (AXI), an Advanced Peripheral Bus (APB) interface 254, and a power engine 260. The interface 252 includes a first bidirectional connection to the SMN and has an input for receiving an event signal labeled "EVENT_n" shown separately in FIG. 2, and an output. The APB interface 254 has an input connected to an output of the interface 252 and an output for connecting to the PHY through the APB. The power engine 260 has an input connected to an output of the interface 252 and an output connected to an input of the memory interface queue 214. The power engine 260 includes a set of configuration registers 262, a microcontroller (μC) 264, a self-refresh controller (SLFREF / PE) 266, and a reliable read / write timing engine (RRW / TE) 268. The configuration registers 262 are programmed through the AXI bus and store configuration information that controls the operation of various blocks in the memory controller 200. Thus, configuration register 262 has outputs that are connected to these blocks not shown in detail in Figure 2. Self-refresh controller 266 is an engine that allows manual generation of refreshes in addition to the automatic generation of refreshes by refresh logic 232. High reliability read / write timing engine 268 provides a continuous stream of memory accesses to memory or I / O devices for purposes such as DDR interface maximum read latency (MRL) training and loopback testing.

[0031] The memory channel controller 210 includes circuitry that allows it to select memory accesses for dispatch to the associated memory channel. To make the desired arbitration decisions, the address generator 222 decodes the address information into pre-decode information including rank, row address, column address, bank address, and bank group in the memory system, and the command queue 220 stores the pre-decode information. The configuration registers 262 store configuration information to determine how the address generator 222 decodes the received address information. The arbiter 238 uses the decoded address information, timing eligibility information indicated by the timing block 234, and active page information indicated by the page table 236 to efficiently schedule memory accesses while complying with other criteria such as quality of service (QoS) requirements. For example, the arbiter 238 implements a baseline for accesses to open pages, avoiding the overhead of pre-charge and activate commands required to modify memory pages, and hiding the overhead accesses to one bank by interleaving them with read and write accesses to another bank. In particular, during normal operation, the arbiter 238 typically keeps pages open in different banks until a page is required to be precharged before selecting a different page.

[0032] 3 is a block diagram illustrating a data processing system 300, according to some embodiments. Data processing system 300 includes a memory system 330 and an APU 310, and a baseboard management controller (BMC) 340. APU 310 includes a memory controller, such as memory controller 200 (FIG. 2), that supports a heterogeneous memory channel to interface with memory system 330. In addition to a regular DDRx memory channel, APU 310 supports NVDIMM-P 338 in a heterogeneous memory channel 331 having both regular registered DIMMs (or RDIMMs) 334, 336 connected through a bus 332, in addition to a homogeneous memory channel 333 having only RDIMMs 335, 337, 339 connected through a bus 329. In this embodiment, heterogeneous memory channel 331 connects to both NVDIMM-P and RDIMMs, although in some embodiments, the heterogeneous memory channel is capable of interfacing with all NVDIMM-P type DIMMs.

[0033] In accordance with the draft NVDIMM-P standard, transactions between the memory controller of the APU 310 and the NVDIMM-P 338 are protected by "link" ECC. Link ECC ensures data integrity of data transfers between the memory controller and the NVDIMM over the bus 332. In accordance with known ECC mechanisms, it protects against data corruption on the link caused by random or transient errors. The protection varies depending on the ECC code used. For example, ECC may allow single-bit correction with multi-bit error detection. In response to detection of an uncorrectable error, the memory controller may replay the transaction so that the transient or random error does not persist, and may also report both correctable and uncorrectable errors to the operating system.

[0034] BMC 340 is a specialized processor mounted on the host circuit board of data processing system 300 and connected to APU 310 to provide control and monitoring capabilities. BMC 340 monitors the status of APU 310 and various other components (not separately shown) of data processing system 300 by receiving error reports and monitoring status registers and sensors. BMC 340 connects to system communication buses such as the illustrated Peripheral Component Interconnect Express (PCIe) bus and Universal Serial Bus (USB), and may also monitor registers of various system components via an inter-integrated circuit bus to poll for error reports. External access to BMC functions is typically provided for remote monitoring and control via a dedicated network interface or by connection to a network interface of data processing system 300.

[0035] Although an NVDIMM-P type DIMM is described in this embodiment, other embodiments use the techniques herein to interface with other types of storage class memory (SCM) modules through heterogeneous memory channels. As used herein, SCM refers to a memory module with non-volatile memory addressable in the system memory space. The non-volatile memory in an SCM module can be buffered in RAM and / or paired with RAM in an on-board SCM module. The SCM memory address map is viewed from an operating system (OS) perspective along with a traditional DRAM population. The OS typically recognizes that the address range defined in SCM is a "different" type of memory than conventional memory. This difference is to inform the OS that this memory may be more potential and has a persistent quality. The OS can map the SCM memory as direct access memory or file system access memory. Direct access implies that the OS accesses the SCM address range as physically addressable memory. File system access implies that the OS manages the persistent memory as part of a file system and manages access to the SCM through a file-based API. Ultimately, the request is sent to the memory controller within the SCM address range, regardless of how the higher-level OS manages access.

[0036] 4 is a block diagram illustrating an NV buffer 400, according to some embodiments. NV buffer 400 is an example instantiation of NV buffer 247 (FIG. 2). NV buffer 400 includes a non-volatile command queue (NV QUEUE) 402, a non-volatile buffer control circuit (NV BUFFER CTRL) 404, a non-volatile error reporting circuit (NV ERROR REPORT) 406, and a non-volatile error lookup table (NV ERROR LUT) 408.

[0037] NV queue 402 is coupled to the output of command queue 220 (FIG. 2) for receiving non-volatile read commands and for storing the commands while they wait to be executed. NV queue 402 stores non-volatile read commands in order to provide the command for replay in the event that a recovery sequence is triggered that requires the command to be replayed.

[0038] The NV buffer CTRL 404 processes a ready response signal (RD_RDY) received from the non-volatile DIMM to indicate that response data is available for an associated one of the non-volatile read commands. In response, the NV buffer CTRL 404 causes a SEND command to be placed in the memory interface queue for issuance to the non-volatile DIMM and causes it to send the response data. If the response data is received, the NV buffer CTRL 404 uses the read ID (RID) to identify the associated non-volatile read command in the NV queue 402 and removes the associated non-volatile read command from the non-volatile queue.

[0039] The NV error reporting circuit 406 is connected to the ECC check circuit 242 and the replay control logic 231 to receive error codes indicative of data errors and link errors associated with the NVDIMM-P or other non-volatile DIMM. Errors are mapped from a first number of possible error conditions identified by error type identifiers associated with the non-volatile DIMM to a second, smaller number of virtual error types for reporting to an error monitoring module of the host operating system. In this embodiment, the mapping is accomplished using a NV error LUT 408 that is indexed for lookup using the error type identifiers describing the data errors or link errors. The result of the lookup is a virtual error type for reporting to the error monitoring module, as described further below with respect to Figures 5 and 6.

[0040] Figure 5 is a block diagram illustrating non-volatile memory usage in a computer system 500, according to some embodiments. The illustrated software and hardware components are suitable for use in a data processing system, as described with respect to Figures 2-4. System 500 includes a user space 510 hosted by an operating system kernel (OS kernel) 520, a memory controller 530, and a non-volatile DIMM (in this embodiment, an NVDIMM-P 540).

[0041] The OS kernel 520 is typically a server OS, but in some implementations is another type of OS, such as a personal computer OS or an OS specialized for a particular computing task. The OS kernel 520 runs on a system processor, such as the APU 310 (FIG. 3), and hosts a user space 510 that runs user processes 512 and user applications 514. The OS kernel 520 includes error monitoring modules, such as a machine check architecture 522, a non-volatile DIMM driver (NV driver) 524, and a file system 526. Many other parts of the OS kernel 520 are not shown to focus on the components that access the non-volatile DIMM. In some implementations, multiple OS kernels exist as virtual machines hosted by a hypervisor layer.

[0042] In this embodiment, machine check architecture 522 is shown, but in other embodiments, other types of error reporting modules are used. Machine check architecture 522 is a known mechanism for providing error reporting from error detection logic in many parts of data processing system 500 to higher level software, such as OS kernel 520, virtual machine manager, applications, processes, etc. Machine check architecture 522 can handle errors and recover from certain errors by determining whether correction data can be obtained and executing software routines to correct the error. In certain conditions, machine check architecture 522 determines that execution of a particular process cannot proceed based on determining that correction data cannot be obtained. In such cases, machine check architecture 522 can handle the error by passing control to higher level software or by forcing a reset.

[0043] NV driver 524 is a driver module that runs on OS kernel 520 to provide access to memory of NVDIMM-P 540. As shown, NV driver 524 executes memory access instructions from processes 512 and applications 514 (having one or more executing processes). To execute the instructions, NV driver 524 formats the instructions as appropriate requests to NVDIMM-P 540 via memory controller 530. For example, non-volatile read and non-volatile write requests from NV driver 524 may result in XREAD and XWRITE commands at memory controller 530.

[0044] The file system 526 runs on the OS kernel 520 and manages data files including data stored on the NVDIMM-P 540. The file system 526 may have native support for interfacing with the NVDIMM-P 540 or may also use the NV driver 524 to handle the access. The OS kernel 520 also accesses the NVDIMM-P 540 via the memory controller 530 according to its own memory access needs.

[0045] Memory controller 530 of this embodiment is an implementation of memory controller 200 (FIG. 2) or a similar memory controller suitable for accessing NVDIMM-P 540, and includes NV error reporting logic 532 that corresponds to NV error reporting logic 406 (FIG. 4). In some embodiments, NV error reporting logic 532 reports errors to machine check architecture 522 using error reporting information determined according to the process of FIG. 6, which is described further below.

[0046] The use of persistent memory such as NVDIMM-P introduces many challenges in addressing error reporting and associated reliability, accessibility and serviceability (RAS) issues. Such challenges stem from the many new error types that occur in non-volatile DIMMs compared to DRAM DIMMs, which have significantly fewer error types. While error types are defined for a particular non-volatile DIMM such as NVDIMM-P, the manner in which errors are generally handled and reported to the system is not specified, for example, in the draft NVDIMM-P standard.

[0047] 6 is a flow diagram of a process 600 for reporting an error, according to some embodiments. Process 600 is suitable for implementation with memory controller 200 of FIG. 2 or other memory controller configurations, and in this embodiment is performed by NV error reporting logic 406 (FIG. 4) in conjunction with replay logic control circuitry 231 (FIG. 2).

[0048] In general, process 600 handles errors detected in the memory controller and reports the errors in a manner appropriate for error and fault management within the system. Error management describes the actions required of operational software (e.g., an operating system) to manage running programs that may be affected by the error. Error management is controlled by an error monitoring module of the operating system, such as machine check architecture 522, and typically includes a limited set of responses to an error, such as taking no action, terminating a single affected process, program, or virtual machine, and terminating system operation. As such, the error monitoring module only needs enough error information to determine the scope and severity of the error and to decide what immediate action to take. Fault management describes optional actions aimed at diagnosing, repairing, and reconfiguring the underlying hardware. Fault management functions are found in various parts of the data processing system and are directed by BMC 340 in the case of critical faults (FIG. 3).

[0049] Referring to process 600, the process begins at block 602 with receiving an error notification from a non-volatile DIMM or detecting an error at the memory controller, such as a link ECC error detected by ECC check circuitry 242 (FIG. 2). Errors reported by the non-volatile DIMM include various types of DRAM errors, media errors (related to the non-volatile media), cache errors (related to the cache of the non-volatile DIMM), and temperature errors. Some non-volatile DIMMs report other types of errors, such as firmware initialization errors. Link errors include transmission errors through the memory channel, such as link ECC errors. Host errors are detected at the memory controller and include non-volatile read command timeouts and duplicate RID errors.

[0050] Generally, there are three overall categories of errors reported from a non-volatile DIMM, such as an NVDIMM-P: alert, emergency, and interrupt. However, these categories are insufficient to report the error management or fault management errors described above. These various types of errors provide dozens of individual error identifiers, each representing an error condition associated with the non-volatile DIMM. In block 604, the error identifiers are used to classify the error based on the specific hardware condition associated with the error. Such classification may be performed in advance, and the results are maintained in a table in the memory controller, such as the NV error LUT 408. The classification is based at least on whether the error condition adversely affects or does not affect executable processes running on the host operating system. Typically, such a determination is based on whether the error indicates that data has been lost and cannot be recovered, for example, due to a buffer overflow or a bit error in the memory media. The mapping may also be based on the type of impact the error condition has on the executable process. For example, if instruction data is lost, the process may be resumed based on the saved user data. If user data is lost, such an option may not be available.

[0051] At block 606, error conditions are mapped from a first number of possible error conditions associated with the non-volatile DIMM to a second, smaller number of virtual error types. The virtual error types are selected based on virtual errors used by an error monitoring module of the data processing system, such as machine check architecture 522 (FIG. 5). In this embodiment, the virtual error types have numeric assigned severity levels that are used to report to the error monitoring module at block 608. The virtual error types are then used to determine the error management response to be made by the error management module.

[0052] The virtual error type is used for reporting to the error management module, but in block 610, the original error identifier based on the error condition is reported to BMC 340. Associated data, such as an associated address or instruction, may also be reported. The reporting mechanism typically loads the information into a register that is polled on behalf of BMC 340, although in other embodiments, other suitable reporting mechanisms are used. BMC 340 uses the information to perform error logging and fault management functions, such as diagnosing hardware faults and repairing or reconfiguring hardware to mitigate the fault condition.

[0053] Thus, the memory controller and data processing system described herein improve the memory controller's ability to handle errors associated with non-volatile DIMMs on heterogeneous memory channels. The techniques herein also provide a mechanism for reporting non-volatile DIMM error conditions to a system error monitoring module in a manner that can be used with the DIMM's existing error reporting mechanisms. Additionally, the techniques herein provide an error handling capability that does not rely on variable or unknown methods that may be used by various non-volatile DIMM vendors to report errors. Instead, errors are classified by the memory controller, which appropriately reports the errors to an error management mechanism and / or a fault management mechanism.

[0054] Any portion of the memory controller 200 or arbiter 238 of FIG. 2 may be described or represented by a computer-accessible data structure in the form of a database or other data structure that can be read by a program and used directly or indirectly to manufacture an integrated circuit. For example, this data structure may be a behavioral level description or a register transfer level (RTL) description of the hardware functionality in a high-level design language (HDL) such as Verilog or VHDL. The description may be read by a synthesis tool, which may synthesize the description to generate a netlist that includes a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware that comprises the integrated circuit. The netlist may then be placed and routed to generate a data set that describes the geometry to be applied to a mask. The mask may then be used in various semiconductor manufacturing steps to generate the integrated circuit. Alternatively, the database on the computer-accessible storage medium may be a netlist (with or without a synthesis library) or a data set, or a Graphics Data System (GDS) II data, as appropriate.

[0055] While specific embodiments have been described, various modifications to these embodiments will be apparent to those skilled in the art. For example, the internal architecture of memory channel controller 210 and / or power engine 250 may vary in different embodiments. Memory controller 200 may interface to other types of memory besides NVDIMM-P and DDRx, such as, for example, high bandwidth memory (HBM), RAM bus DRAM (RDRAM), etc. While the illustrated embodiment shows each rank of memory corresponding to a separate DIMM, in other embodiments, each DIMM may support multiple ranks. Additionally, while heterogeneous memory channels are generally supported, a heterogeneous channel may be filled entirely with non-volatile DIMMs.

[0056] It is therefore intended by the appended claims to cover all modifications of the disclosed embodiments that fall within the scope of the disclosed embodiments.

Claims

1. a command queue having a first input for receiving memory access commands including volatile reads, volatile writes, non-volatile reads and non-volatile writes, and an output, the command queue having a plurality of entries; a memory interface queue having an input coupled to an output of the command queue and an output for coupling to a non-volatile storage class memory (SCM) module; a non-volatile error reporting circuit operable to map error conditions associated with the non-volatile SCM module from a first number of possible error conditions to a second, smaller number of virtual error types for identifying and reporting error conditions associated with the non-volatile SCM module to an error monitoring module of a host operating system, the mapping being based at least on a classification of the error conditions as either adversely affecting or not adversely affecting executable processes executing on the host operating system. Memory controller.

2. the non-volatile error reporting circuitry is operable to report the virtual error type to the error monitoring module of the host operating system. The memory controller of claim 1 .

3. the non-volatile error reporting circuitry is operable to report the error condition to a baseboard management controller of a host data processing system. The memory controller of claim 1 .

4. The error conditions include an interrupt type error and an emergency type error. The memory controller of claim 1 .

5. the non-volatile error reporting circuit includes a lookup table indexed by at least an error code associated with the error condition, the lookup table including a virtual error type associated with the error code; The memory controller of claim 1 .

6. the mapping includes determining a type of effect that the error condition has on the executable process. The memory controller of claim 1 .

7. A method executed by a memory controller of a data processing system, comprising: receiving a plurality of memory access requests including a volatile memory read, a volatile memory write, a non-volatile memory read, and a non-volatile memory write; placing a memory access command in a memory interface queue to satisfy the memory access request, and transmitting the memory access command from the memory interface queue to a memory channel coupled to a non-volatile storage class memory (SCM) module; identifying an error condition associated with the memory channel based on information received via the memory channel; mapping the error conditions from a first number of possible error conditions associated with the non-volatile SCM module to a second, smaller number of virtual error types for reporting to an error monitoring module of a host operating system, the mapping being based at least on a classification of the error conditions as either adversely affecting or not adversely affecting executable processes executing on the host operating system. method.

8. reporting the virtual error type to the error monitoring module of the host operating system. The method of claim 7.

9. reporting the error condition to a baseboard management controller. The method of claim 7.

10. The error conditions include an interrupt type error and an emergency type error. The method of claim 7.

11. the mapping includes determining a type of effect that the error condition has on the executable process. The method of claim 7.

12. The mapping includes determining a severity level to be reported to the error monitoring module. The method of claim 7.

13. A central processing unit; a data fabric coupled to the central processing unit; a memory controller coupled to the data fabric to satisfy memory access requests made by the data fabric; The memory controller includes: a command queue having a first input for receiving memory access commands including volatile reads, volatile writes, non-volatile reads and non-volatile writes, and an output, the command queue having a plurality of entries; a memory interface queue having an input coupled to an output of the command queue and an output for coupling to a non-volatile storage class memory (SCM) module; a non-volatile error reporting circuit operable to map error conditions associated with the non-volatile SCM module from a first number of possible error conditions to a second, smaller number of virtual error types for identifying and reporting error conditions associated with the non-volatile SCM module to an error monitoring module of a host operating system, the mapping being based at least on a classification of the error conditions as either adversely affecting or not adversely affecting executable processes executing on the host operating system. Data processing system.

14. the non-volatile error reporting circuitry is operable to report the virtual error type to the error monitoring module of the host operating system.

14. The data processing system of claim 13.

15. the non-volatile error reporting circuit includes a lookup table indexed by an error code associated with at least the error condition, the lookup table including an associated virtual error type for the error code; 14. The data processing system of claim 13.

Images