Data line (DQ) backup with adaptive error correction coding (ECC) mode switching

Abstract

The system provides DQ level backup to backup for a failure of a data signal (DQ) line of a memory bus. The data bus has a plurality of data dynamic random access memory (DRAM) devices and at least one error correction code (ECC) DRAM device coupled thereto. The error manager may be located in a memory controller or in a platform error controller. And the error manager is used for detecting DQ failure and immediately and dynamically switching the ECC mode. The error manager may map data bits of the DQ and remap ECC bits of the at least one ECC DRAM device to the mapped data bits of the DQ.

Description

[0001] Priority claim

[0002] This application claims the benefit of priority of Patent Cooperation Treaty (PCT) application No. PCT / CN2023 / 103032 filed on June 28, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The description generally relates to memory systems and, more particularly, to memory sparing. Background Art

[0004] Memory errors are one of the main errors that cause server downtime in modern data centers. With error correction coding (ECC), memory errors can be divided into two categories: 1) correctable errors (CE), which are errors that can be corrected by ECC, so this does not have serious consequences; and 2) uncorrectable errors (UE), which are errors that cannot be corrected by ECC (such as single-device UE) or are not covered by ECC (such as multi-device UE), which causes workload failure or server downtime.

[0005] DQ failure can affect a wide range of memory and can flip bits across multiple banks and bank groups, which is one of the main memory failure modes leading to UE in current memory devices. According to some test / validation data, DQ failure may be the leading cause of UE in some memory systems.

[0006] To avoid UE, the system uses sparing techniques to allow on-the-fly failover from a failed component to another component. Modern sparing techniques can spar failed memory at different levels, including nibble-level sparing through partial cacheline sparing (PCLS), row-level sparing through postpackage repair (PPR), bank sparing through adaptive double device data correction (ADDDC), and other sparing techniques.

[0007] In row sparing (e.g., PPR), if the platform or operating system (OS) discovers a memory row with defects (one or more defective bits) during system operation, the system can trigger the setting of fuses in the memory device to map a spare row into the array to replace the defective row. With row sparing, the system maps the defective row out of the memory array to not use it.

[0008] Bank spare techniques such as ADDDC can map out banks of memory devices (e.g., banks of physical memory chips) with hard errors by placing failed memory areas in a virtual lockstep mode. It should be understood that bank spares have been used to spare bit line / data signal (DQ) failures, but sparing banks for failed bit lines affects much larger memory areas than failures. Therefore, bank spares for single bit line failures have a significant negative impact because it significantly reduces the total available memory capacity relative to the failure. ADDDC may have a significant runtime performance impact. Therefore, bank spares either have a significant performance impact or require a system restart.

[0009] PCLS is a spare technology that detects single-bit or single-nibble hard errors within a cache line and then replaces the entire nibble (e.g., 4 bits) within spare capacity in a central processing unit (CPU). For example, an integrated memory controller (iMC) of a CPU may implement PCLS.

[0010] Row sparing (e.g., PPR), nibble sparing (e.g., PCLS), and page off-line cannot provide spare coverage for bitline / DQ failures. While bank sparing (e.g., ADDDC) can provide spare coverage for bitline / DQ, the performance and / or capacity impact is significant. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The following description includes a discussion of the accompanying drawings with diagrams given by way of examples of implementations. The accompanying drawings should be understood by way of example and not limitation. As used herein, reference to one or more examples should be understood to describe a specific feature, structure or characteristic included in at least one implementation of the present invention. Phrases such as "in one example" or "in an alternative example" that appear herein provide examples of implementations of the present invention, and do not necessarily all refer to the same implementation. However, they are not necessarily mutually exclusive.

[0012] Figure 1is a block diagram of an example of a system with DQ spare.

[0013] Figure 2A is a block diagram of an example of a memory subsystem for DQ sparing.

[0014] Figure 2B is a block diagram of an example of a memory device input / output architecture.

[0015] Figure 3 is a block diagram of an example of a DQ standby control architecture.

[0016] Figure 4 is a block diagram of an example of a system architecture for DQ spare.

[0017] Figure 5 is a block diagram of an example of a system in which a controller maintains a defective DQ inventory.

[0018] Fig. 6A is a block diagram of an example of uncorrectable error analysis training.

[0019] Figure 6B is a block diagram of an example of backup based on uncorrectable error analysis.

[0020] Figure 7 is a block diagram of an example of a system that reduces ECC and remaps for failed DQs.

[0021] Fig. 8A is a block diagram of an example of a system that reduces ECC and remaps for failed DQs.

[0022] Figure 8B is a block diagram of an example of a system that reduces ECC and remaps for two failed DQs.

[0023] Fig. 9 is a block diagram of an example of a system that reduces ECC and remaps for three failed DQs.

[0024] Fig.10 is a block diagram of an example of a data read / write flow with DQ sparing.

[0025] Fig.11 is a flow chart of an example of a process for DQ spare.

[0026] Fig.12 is a block diagram of an example of a memory subsystem in which DQ sparing may be implemented.

[0027] Figures 13A-13B is a block diagram of an example of a CAMM system in which DQ spare may be implemented.

[0028] Fig.14is a block diagram of an example of a computing system in which DQ sparing may be implemented.

[0029] Fig.15 is a block diagram of an example of a multi-node network in which DQ standby may be implemented.

[0030] Below is a description of certain details and implementations, including a non-limiting description of the accompanying drawings, which may depict some or all examples, as well as other potential implementations. DETAILED DESCRIPTION

[0031] As described herein, a system provides DQ level backup to backup for failures of data signal (DQ) lines of a memory bus. The data bus has multiple data dynamic random access memory (DRAM) devices and at least one error correction code (ECC) DRAM device coupled thereto. The error manager can be located in a memory controller, in a platform error controller, or distributed between a memory controller and a platform error controller. The error manager can detect DQ failures and dynamically switch ECC modes in real time. The error manager can map out data bits of the DQ and remap the ECC bits of at least one ECC DRAM device to the mapped data bits of the DQ.

[0032] The memory subsystem can include DQ fault management to enable DQ sparing with dynamic switching of ECC modes. Reducing the ECC mode means that fewer ECC bits are needed to implement ECC, freeing up bits in the memory. DQ fault management can remap the freed ECC bits, reusing them as data bits to replace the data bits from the spared DQ.

[0033] The system can perform ECC degradation and repurpose these bits on the fly during runtime of the memory subsystem. Since DQ failures can significantly increase the number of correctable errors (CE) per cell of a data burst and increase the risk of multi-device uncorrectable errors (UE), incorporating dynamic DQ spare functionality into the server platform improves the reliability, availability, and serviceability (RAS) of the server system. A system having the described features can avoid UEs due to DQ failures.

[0034] Figure 11 is a block diagram of an example of a system with DQ sparing. System 100 shows a memory coupled to a host. Dual inline memory module (DIMM) 110 represents a memory module including a plurality of memory devices, such as DRAM device 140 [0: (N-1)], or collectively referred to as DRAM device 140 and DRAM device 150 [0: (N-1)], or collectively referred to as DRAM device 150. N can be any integer greater than 2, where there is at least one ECC device.

[0035] The memory controller 120 represents a host, which may be part of a computing platform, such as a central processing unit (CPU) system on a chip (SOC) or other host processing element SOC, such as a graphics processing unit (GPU). The memory controller 120 includes hardware interconnects and driver / receiver hardware to provide interconnections between the memory controller 120 and the DIMM 110. Although the DIMM 110 provides an example of a module having multiple memory devices, the system 100 may alternatively be implemented with a high bandwidth memory (HBM) package having multiple DRAM chips stacked vertically.

[0036] System 100 shows an example of a DIMM 110 having a registered (or registering) clock driver (RCD) 130 and memory devices. RCD 130 represents a controller of DIMM 110. In one example, RCD 130 receives information from memory controller 120 via command / address (C / A) bus 162 and buffers the signal to the memory device via the C / A bus. System 100 represents two separate channels on DIMM 110, where each channel can have one or more ranks of DRAM devices. A rank refers to a group of DRAM devices accessed by a common chip select signal.

[0037] C / A bus 170[A] can be the first channel (channel A) of DRAM device 140. C / A bus 170[B] can be the second channel (channel B) of DRAM device 150. System 100 also shows an enable line (EN) from RCD 130 to various DRAM devices. C / A bus 170[A] and C / A bus 170[B] can be collectively referred to as C / A bus 170. C / A bus 170 represents a bus for providing command encoding and address information for memory access operations. C / A bus 170 is typically a single-sided bus or a unidirectional bus to transmit commands, addresses, and enable information from RCD 130 to DRAM devices in response to commands from memory controller 120.

[0038] The data (DQ) bus 164[A] represents a bus for exchanging data between the DRAM device 140 and the memory controller 120, and the data (DQ) bus 164[B] represents a bus for exchanging data between the DRAM device 150 and the memory controller 120. The DQ bus 164[A] and the DQ bus 164[B] may be collectively referred to as the DQ bus 164. The DQ bus 164 is conventionally a bidirectional, point-to-point bus.

[0039] DRAM devices 140[0:(N-1)] respectively include arrays 142[0:(N-1)], collectively referred to as arrays 142. Arrays 142 store data from DQ bus 164[A] in response to a write command and provide data to DQ bus 164[A] in response to a read command. DRAM devices 140[0:(N-1)] respectively include circuit devices 144[0:(N-1)], collectively referred to as circuit devices 144. Circuit devices 144 represent circuit devices that are coupled to arrays 142 to store data from DQ bus 164[A] in response to a write command and provide data to DQ bus 164[A] in response to a read command.

[0040] DRAM devices 150[0:(N-1)] respectively include arrays 152[0:(N-1)], collectively referred to as arrays 152. Arrays 152 store data from data bus 164[B] in response to a write command and provide data to data bus 164[B] in response to a read command. DRAM devices 150[0:(N-1)] respectively include circuit devices 154[0:(N-1)], collectively referred to as circuit devices 154. Circuit devices 154 represent circuit devices that are coupled to arrays 152 to store data from DQ bus 164[B] in response to a write command and provide data to DQ bus 164[B] in response to a read command.

[0041] DRAM device 140 and DRAM device 150 are shown as having an xM interface with M data bus pins DQ[0:(M-1)]. M can be any integer, and is typically a binary integer, such as 4, 8, or 16. Each DQ interface will transfer data bits over a burst length (BL), such as BL16 for 16 unit intervals or transfer cycle data exchanges. Thus, data transfer is BL x M, such as a x4 interface (e.g., M=4) with BL16 (e.g., 16 transfer unit intervals (UI)) for 4x16=64 bits per device.

[0042] The memory controller 120 receives read data on the DQ bus or sends data corresponding to a command sent on the C / A (command / address) bus 162 on the DQ bus. The command sent on the C / A bus 162 includes a command encoding and address information. In one example, the C / A bus 162 connects the memory controller 120 to the RCD 130, wherein the memory controller 120 can send commands to any channel, and the RCD 130 generates command and enable signals on the DIMM 110. For channel A, the RCD 130 can provide the command and address information on the C / A bus 170 [A] to the DRAM device 140. For channel B, the RCD 130 can provide the command and address information on the C / A bus 170 [B] to the DRAM device 150. The RCD 130 asserts the enable (EN) signal line of the desired DRAM device associated with the access command.

[0043] The memory controller 120 includes a command (CMD) controller 122, which represents logic at the memory controller 120 for sending commands to the DRAM devices. In one example, the memory controller 120 includes error checking and correction (ECC or error correction coding) 124 to identify and correct data errors in data received from the DRAM devices.

[0044] In one example, memory controller 120 includes DQ remapper 126. In response to an error in a particular DQ of one of the memory channels, memory controller 120 may spare the DQ, reduce the ECC applied by ECC 124, and remap data bits from the DQ to the ECC bits freed by reducing the applied ECC.

[0045] In one example, the system 100 includes a DQ manager 180. In one example, the DQ manager 180 is part of the memory controller 120. In one example, the DQ manager 180 is implemented as firmware on the memory controller 120. In one example, the DQ manager 180 is separate from the memory controller 120. In one example where the DQ manager 180 is separate from the memory controller 120, the DQ manager can be part of a controller circuit that monitors errors in the system 100. Such a controller circuit can be arranged on a substrate as part of a chip separate from the memory controller. The circuit can be coupled to the memory controller or directly coupled to the memory bus to detect errors. In one example, the DQ manager 180 is implemented as firmware on an error detector controller. More details of such a controller are provided with reference to subsequent figures.

[0046] In one example, the DQ manager 180 determines that a DQ in the system 100 has experienced an uncorrectable error (UE) and determines to spare a failed DQ. Based on such a determination, the DQ manager 180 can trigger the ECC 124 to reduce the level of applied ECC. Reduction of ECC refers to dynamically switching the ECC to a lower overwrite mode. Reducing the level of applied ECC will free up bits previously used for ECC. Sparing a failed DQ and determining to reduce ECC can occur instantly. The DQ remapper 126 enables the memory controller 120 to reuse the freed ECC bits to store data from the retired / spare DQ.

[0047] In one example, the DQ remapper 126 is part of the DQ manager 180. When the DQ manager 180 is part of the memory controller 120, the DQ remapper 126 may be part of both the memory controller 120 and the DQ manager 180. When the DQ manager 180 is separate from the memory controller 120, the DQ remapper 126 may be considered a distributed component of the DQ manager 180 located in the memory controller 120 to enable remapping of data bits from the DQs in the system.

[0048] Figure 2A 1 is a block diagram of an example of a memory subsystem for DQ sparing. System 202 represents a memory subsystem according to the example of system 100. System 202 shows a memory controller (CONTRLR) 210 coupled to ten (10) dynamic random access memory (DRAM) devices. System 202 more specifically shows the memory devices as chips [0:9], where each chip is a separate DRAM device.

[0049] The command / address (CA) bus 220 represents a command bus through which the memory controller 210 can provide command encoding and address information to the DRAM devices. The CA bus 220 represents a unidirectional bus for providing information from the memory controller 210 to the DRAM devices. The data (DQ) bus 230 represents a data bus for exchanging data between the DRAM devices and the memory controller 210, wherein the memory controller 210 drives data on the DQ bus 230 for a write operation, and the DRAM devices drive data on the DQ bus 230 for a read operation.

[0050] As shown, chips [0:7] represent data devices to store user data, and chips [8:9] represent ECC devices to store system ECC information. It should be understood that each DRAM device may optionally include on-die ECC, where the on-die ECC circuitry generates and stores ECC bits internally to detect and correct errors in the data provided to the memory controller 210. Such ECC information may be referred to as internal ECC bits. Internal ECC bits are generated and applied only locally at individual devices.

[0051] The ECC information stored in chip [8:9] refers to the system ECC, which is the ECC applied by the memory controller 210 to the data received from the data device set. The system ECC information is generated by the memory controller 210 and applied to the data stored across chip [0:7]. During a write operation, the memory controller 210 generates and writes the ECC data to be stored in chip [8:9] in parallel with the data stored in chip [0:7]. For a read operation, each DRAM device provides its data to the memory controller 210, which knows the mapping of the data.

[0052] As shown, each DRAM device includes an array 242, which represents a memory array that stores data in the chip. The DRAM device includes a local input / output (I / O) 244 and a global input / output (I / O) 246. The local I / O 244 provides interface circuitry to the local subdivisions of the array 242. The global I / O 246 interconnects the various subdivisions to the transceiver circuits 248 that interface with the DQ lines of the DQ bus 230.

[0053] The transceiver circuit 248 can map specific data paths to specific I / O pins in the I / O pins 250 on the chip. As shown, chip [0:9] is a x4 device, which means it has four (4) I / O pins connected to the DQ bus. Other devices can be x8, x16, or other interface widths. The system 202 shows the internal I / O count of the transceiver circuit 248 of each device as 0, 1, 2, and 3 to reference the four I / O pins of each chip. The system 202 shows I / O pins 0, 1, 2, 3, 4, 5, ..., 38, 39 to reference the total DQ pin interface from the collective DRAM devices to the DQ bus 230. The numbering is for convenience of the description given below.

[0054] Utilizing eight (8) data devices and two (2) ECC devices, system 202 can apply various levels of ECC at the system level. References to applying different levels of ECC will be understood throughout as references to application of system ECC.

[0055] The different ECC modes that can be applied by the system 202 include 1) 10x4, 128-bit (or 125-bit) ECC mode; 2) 10x4, 96-bit ECC mode; 3) 9x4, 64-bit ECC mode. Mode 1 refers to the use of 8 data devices and 2 ECC devices, wherein the 2 ECC devices store 128-bit or 125-bit ECC information. Mode 2 refers to the use of 8 data devices and 2 ECC devices, wherein the 2 ECC devices store 96-bit ECC information. Mode 3 refers to the use of 8 data devices and 1 ECC device, wherein the ECC device stores 64-bit ECC information.

[0056] In one example, the system 202 can perform a runtime ECC mode change, thereby changing the level of system ECC that is applied immediately. Lowering the ECC level frees up bits, which the system can then reuse for the failed DQ. The memory controller 210 can remap how it interprets bits received from various DRAM devices, including ignoring bits from failed DQs and remapping bits from one or more ECC devices to data bits for the failed DQ. In one example, the memory controller 210 maps all bits of the failed DQ to a single ECC device. In one example, the memory controller 210 maps bits of the failed DQ to multiple ECC devices.

[0057] Assume that system 202 is an application of 10x4 double data rate version 5 (DDR5) DIMMs with 128-bit ECC mode, where a DQ failure is detected. In one example, system 202 can spare up to 2 failed DQs (16 bits per DQ in DDR5, 2 DQs = 32 bits) by changing the ECC mode from 128-bit ECC to 96-bit ECC and reusing the 32 bits released from the ECC device to store data from the spare DQ. In one example, system 202 can spare up to 4 failed DQs by switching the ECC from 10x4 128-bit ECC mode to 9x4 64-bit ECC mode and reusing the released 64 bits to store data from the spare DQ.

[0058] Figure 2B is a block diagram of an example of a memory device input / output architecture. Memory 204 illustrates components of a DRAM device with a specific mapping of I / O to portions of a memory array according to any example of system 202 .

[0059] Memory 204 includes a bank 260 that includes multiple rows of memory cells. The vertical rectangles of the figure represent rows. In one example, bank 260 is organized into multiple sub-arrays 262. A sub-array 262 or array can refer to a group of rows 240. An access command (read command or write command) triggers a command signal line interpreted by row decode 270 to select one or more rows for operation. In one example, row decode 270 includes sub-array or array decode (DEC) 272 to select a specific sub-array 262 of bank 260.

[0060] The memory 204 includes a column decode 274 to select a column of data, where a signal line from a particular subarray can be output to a sense amplifier and routed to a global I / O 246 using a local I / O 244. In one example, the column (COL) decode 274 includes a multiplexer to selectively connect certain columns to a common I / O connector. The local I / O 244 refers to a routing circuit device for transmitting data of a particular subarray 262. The global I / O 246 refers to a routing circuit device that couples the local I / O to an external I / O connector (I / O pin) of the memory 204.

[0061] In one example, local I / O 244 includes logic for mapping specific signal lines from sub-array 262 to specific I / O paths or data paths connected to specific global I / O connectors. A data path may include cables and circuits for switching or selecting a path that connects a memory cell to the I / O of a device. A data path may refer to cables, traces, logic gates, and other circuit devices used to transfer data between a sub-array and an I / O. A data path may refer to all connection logic and I / O logic used to couple a data array to an externally facing connector or pad of a device package.

[0062] It should be understood that any part of the I / O may experience a failure. A failure in column decode 274 may cause a DQ failure by interrupting the data path to the I / O connector. A failure in local I / O 244 may also cause a DQ failure by interrupting the data path to the I / O connector. A failure in global I / O 246 may also cause a DQ failure by interrupting the data path to the I / O connector. Failures that cause DQ failures in any level of the data path may be addressed by DQ standby, as described above with reference to system 202.

[0063] Figure 3 is a block diagram of an example of a DQ spare control architecture. System 300 illustrates a system according to an example of system 100 or system 202. System 300 illustrates the logical location of components of DQ management, including a DQ manager, an address checker, and a DQ remapper.

[0064] System 300 shows a central processing unit (CPU) 310, which represents a processing unit that generates memory access requests to memory represented by subchannel 330. Subchannel 330 includes data DRAM 0, data DRAM 1, ..., and data DRAM 7, as well as ECC DRAM 0 and ECC DRAM 1, for a total of 10 DRAM devices. As shown, each DRAM device has a x4 interface for a total of 40 DQ signal lines that interface with memory controller 320.

[0065] The memory controller 320 manages access to the subchannel 330. In one example, the memory controller 320 is part of the CPU 310. The system 300 does not show all the details of the memory controller 320. In one example, the memory controller 320 includes an address checker 322, which may optionally include a register R, an ECC algorithm 324, and a DQ remapper 326. The ECC algorithm 324 represents the system ECC performed by the memory controller 320. The address checker 322 can identify whether a read or write is for a spare area. The DQ remapper 326 enables the memory controller 320 to determine how to receive and how to send data without using a spare area.

[0066] The system 300 includes a DQ manager 340, which represents a component that manages DQ spare for the system 300. In one example, the DQ manager 340 is located within the memory controller 320. In one example, the DQ manager 340 is implemented as firmware of the memory controller 320. In one example, the DQ manager 340 is a state machine or hardware component of the memory controller 320. In one example, the DQ manager 340 is separate from the memory controller 320, such as part of a chipset or a system error manager. When the DQ manager 340 is separate from the memory controller 320, it can be implemented as hardware or firmware of another host component.

[0067] The DQ manager 340 may include DQ spare results for each bank of the subchannel 330. The DQ manager 340 may manage the ECC mode associated with the DQ spare results. In one example, the DQ manager 340 represents a microcontroller or firmware logic executed on a microcontroller. The DQ manager 340 may read memory subsystem information (e.g., configuration) and record DIMM configuration and ECC mode information.

[0068] In one example, a microcontroller, firmware, or software of system 300 notifies DQ manager 340 of a DQ failure. The microcontroller may be an error detector controller. The firmware may be firmware of memory controller 320 or another platform component. The software may be software executed on CPU 310, such as a host operating system (OS).

[0069] In response to a DQ failure, in one example, the DQ manager 340 checks the feasibility of a spare DQ. If the DQ failure can be spared (spare), the address checker 322 can determine whether the read / write operation is in the spare DQ position. The DQ remapper 326 can reorder the DQ bits from the spare DQ to the released ECC bit space to store data in the released position or read data from the released position.

[0070] Prior to DQ sparing, the DQ remapper 326 and the address checker 322 may be transparent to read and write operations, allowing reads and writes to be performed normally. Once a DQ sparing is inferred, the DQ manager 340 may notify the address checker 322 and the DQ remapper 326 of the sparing. The cache lines in the corresponding memory bank should be refreshed using the new ECC mode. The system 300 may refresh the memory bank containing the DQ failure by one of two actions. For a fast and immediate action, the memory controller 320 may suspend read and write requests from / to the memory bank associated with the DQ failure until all cache lines of the memory bank are refreshed.

[0071] An alternative action is to refresh the memory bank during the memory controller idle time. In one example, register R provides a mechanism for the address checker 322 to record the refresh status in the memory bank. When the system 300 starts refreshing, register R can be initialized with the lowest address in the memory bank. During the system idle time (which includes the memory bank idle time when the memory bank is not accessed), the address checker 322 can refresh the cache line from register R to a higher address and increment register R to the new address. In one example, once the memory controller 320 is busy, the address checker 322 stops refreshing. If a read / write operation accesses a location in the memory bank during the refresh, the address checker 322 can compare the operation address with the address stored in register R. In one example, if the operation accesses a location with a lower address than register R, the address checker 322 applies the new mapping; otherwise, the address checker 322 applies the old mapping.

[0072] As described above, the DQ manager 340 can manage the ECC mode applied to the subchannel 330. In one example, the system 300 can dynamically switch the ECC mode from the 10x4 128-bit ECC mode to the 10x4 96-bit ECC mode. In one example, after the DQ manager 340 has switched the ECC mode to the 10x4 96-bit ECC mode, the system 300 can subsequently dynamically switch the ECC mode to the 9x4 64-bit ECC mode. In one example, the system 300 can dynamically switch the ECC mode from the 10x4 128-bit ECC mode to the 9x4 64-bit ECC mode without first switching to the 10x4 96-bit mode.

[0073] Figure 44 is a block diagram of an example of a system architecture for DQ sparing. System 400 illustrates a computer system according to the example of system 100 or the example of system 202. System 400 includes a host 410 connected to a DIMM 420. Host 410 represents a host hardware platform for a system in which DIMM 420 operates. Host 410 includes a host processor (not explicitly shown), such as a CPU or GPU, to perform operations requesting access to memory of DIMM 420.

[0074] DIMM 420 includes DRAM devices or DRAMs connected in parallel to process access commands. DIMM 420 is more specifically shown as a two-rank DIMM, wherein there are M DRAMs (DRAM [0: M-1]) in each rank (rank 0 and rank 1). M can be any integer. Typically, a row of DRAMs includes a data DRAM for storing user data and an ECC DRAM for storing system ECC bits and metadata. System 400 does not distinguish between DRAM uses. In one example, the DRAM devices of system 400 represent DRAM devices compatible with the double data rate version 5 (doubledata rate version 5, DDR5) standard from JEDEC (Joint Electron Device Engineering Council, now JEDEC Solid State Technology Association).

[0075] DRAMs in a row share command bus and chip select signal lines and have separate data bus interfaces. Command (CMD) 412 represents the CA bus for row 0 and command (CMD) 422 represents the CA bus for row 1. CS0 represents the chip select for the devices in row 0 and CS1 represents the chip select for the devices in row 1. DQ 414 represents the data (DQ) bus for the devices in row 0, where each DRAM contributes B bits, where B is an integer, with a total of B*M bits on the DQ bus. DQ 424 represents the data (DQ) bus for the devices in row 1.

[0076] DRAM 440 provides an example representation of the details of each DRAM device of system 400. DRAM 440 includes control (CTRL) logic 446, which represents the logic for receiving and decoding commands. Control logic 446 provides internal control signals in response to commands received on the command bus. DRAM 440 includes multiple memory banks 442, where the memory banks represent the organization of the memory array of DRAM 440. The memory banks 442 have separate access hardware to allow parallel access or non-blocking access to different memory banks. The portion marked as 450 is a sub-array of the total memory array of DRAM 440.

[0077] The memory array includes rows (ROW) and columns (COL) of memory elements. Sense amplifier (SA) 444 represents a sense amplifier for making data appear (to stage data) for reading from the memory array or for writing to the memory array. Data can be selected into the sense amplifier to allow detection of the value stored in the bit cell or memory cell of the array. The dotted box contains the intersection of the marked rows and columns of the memory array. The dotted portion shows a typical DRAM cell 448, including a transistor as a control element and a capacitor as a storage element. The bit line (BL) is a column signal line, and the word line (WL) is a row signal line.

[0078] The memory controller (MEM CTLR) 418 represents a memory controller that manages access to the memory resources of the DIMM 420. The memory controller 418 provides access commands to the memory devices, including sending data for write commands or receiving data for read commands. The memory controller 418 sends command and address information to the DRAM devices and exchanges data bits with the DRAM devices (to or from the DRAM devices, depending on the command type).

[0079] The host 410 includes an OS 416, which represents a host operating system on the host 410. In one example, the host 410 includes an error control 430 to manage error detection and memory error handling in the system 400. In one example, the error control 430 includes a DQ manager for DQ sparing according to any of the descriptions herein. In one example, some or all of the error control 430 may be part of the memory controller 418.

[0080] In one example, error control 430 includes ECC 432, which represents the ECC at host 410. Error control 430 can be or include a memory fault tracker and detector. Remapper 434 represents the ability of error control 430 to spare DQ, reduce the ECC applied by ECC 432, and remap bits of the spare DQ to freed bits of the ECC. Thus, system 400 can spare DQ according to any example herein.

[0081] Figure 5is a block diagram of an example of a system in which a controller maintains a defective DQ directory. System 500 represents a system according to the example of system 100 or the example of system 202. System 500 shows a row 520 with 10 DRAM chips (chips [0:9]) that are selected using a common chip select (CS) signal. System 500 distributes cache lines 510 across data cells of row 520. Cache line 510 may include 640b, with 512b of data plus 128b ECC on 40DQs, thereby distributing 64b to each data cell.

[0082] DRAM chip 530 represents one of the chips of row 520. Specifically, DRAM chip 530 shows the details of chip [1], where other chips will have similar details. DRAM chip 530 includes 8 memory banks, memory banks [0:7]. Each memory bank includes an array 532 with columns and rows. Row decoder (DEC) 534 selects rows or word lines, and column decoder (DEC) 538 selects columns or bit lines. Row buffer 536 represents a buffer for data read from a single row. Column decoder 538 can select which portion(s) of the row to trigger for access.

[0083] The intersection of the column lines and the row lines represents a memory cell. The memory cells marked with an "X" represent bit errors at these memory cells. Error detector 550 represents hardware in system 500 for detecting errors in array 532. In one example, error detector 550 detects both correctable errors and uncorrectable errors in array 532. A correctable error (CE) may refer to a row with a single error, which can be corrected by on-die ECC. An uncorrectable error (UE) may refer to a row with more than one error, which is generally not correctable by on-die ECC. Such errors can be corrected with system ECC, but the focus in system 500 is on correction by on-die ECC. The arrow from error detector 550 to array 532 identifies a column failure or DQ failure.

[0084] The error detector 550 can provide CE and UE information to the controller 540, which represents the control performed by the hardware components of the system 500. In one example, the controller 540 represents a microcontroller. In one example, the controller 540 represents other hardware in the system, such as a circuit device in a memory controller. The controller 540 may include a failure detector 542 to identify defective memory regions associated with detected errors, and in particular DQ failures according to the discussion of this article. The failure detector 542 can be enabled / configured to detect failures other than DQ failures. The controller 540 may include a table 544 as a defective memory region directory (DMRD) to identify faulty DQs detected by the failure detector 542.

[0085] In one example, table 544 is implemented in nonvolatile memory, as represented by nonvolatile random access memory (NVRAM) 560. NVRAM 560 enables defective region directory information to be retained between system boots. NVRAM 560 provides an example of information that may be stored in table 544. Table 544 may represent a list of defective DQs populated with corresponding identification information. In one example, system 500 may store table 544 in static random access memory (SRAM) as an alternative to NVRAM.

[0086] FDQ0 may represent a first record of a first failed DQ (FDQ). DQs are identified as rank0 / chip1 / bank1 / DQ#. The second defective region indicated by record FDQ1 is identified as rank2 / chip2 / bank2 / DQ#. Both the first and second defective DQs are identified by a specific component address (i.e., a column of a specific device in a row) to enable the system to avoid using the failed DQ.

[0087] Fig. 6A 6 is a block diagram of an example of uncorrectable error analysis training. System 602 represents an element of a training phase or training system for predicting memory failures or analyzing memory failures due to uncorrectable errors. In one example, system 602 can be considered to be offline prediction or analysis model training because data set 610 represents data from past system operations. An online system refers to a system that is currently operating. System 602 is "operable" in the sense that it is operable to generate a model, but generates the model based on historical data rather than real-time or runtime data.

[0088] In one example, system 602 includes data set 610. Data set 610 may represent a large-scale CE and UE failure data set including micro-level memory error information. The micro-level memory error information may include an indication of a failure based on a bit, DQ, row, column, device, row, channel, DIMM, or other configuration, or combination of information. In one example, data set 610 includes a timestamp for indicating when an error occurred. In one example, data set 610 includes hardware configuration information associated with the error data set. The hardware configuration information may include information such as memory device information, DIMM manufacturer part number, CPU model, system board details, or other information or a combination of such information. In one example, data set 610 may represent information collected from a large-scale data center implementation.

[0089] System 602 includes a UE analysis model (UAM) builder 620 for processing data from data set 610 to generate a model indicating a configuration with an error mode that may cause the UE. In one example, UAM builder 620 represents software logic for AI (artificial intelligence) training to generate a model. In this context, AI represents neural network training or other forms of data mining to identify relationship patterns from large data sets. In one example, UAM builder 620 generates UAM 630 for each hardware configuration based on micro-level (e.g., bit, DQ, row, column, device, row) CE mode or indicator. Therefore, UAM 630 can include N different UAMs (UAM [1: N]) based on different configuration information (CONFIG).

[0090] In one example, the UAM 630 includes a separate analysis model for each combination of a CPU model and a DIMM manufacturer or part number. This granularity for different combinations of CPU models and DIMM part numbers can identify faulty hardware modes differently, seeing that different hardware configurations may result in different hardware failure states. For example, DIMMs from the same manufacturer or with the same part number but with different CPU models may implement ECC differently in the memory controller, resulting in the same faulty hardware state of the DIMM exhibiting different observations due to different behaviors of the ECC implementation. The CPU family can provide multiple ECC modes, allowing customers to select ECC based on the customer's selected application. Similarly, for the same CPU model and DIMMs from different manufacturers or with different part numbers, the failure state of the DIMM may exhibit different observations due to different designs and implementations of the DIMM hardware. Therefore, in one example, the system 602 creates an analysis model for each combination of a CPU model and a DIMM manufacturer or part number to provide improved analysis accuracy performance.

[0091] Figure 6B is a block diagram of an example of a backup based on uncorrectable error analysis. System 604 represents an example of a system with UE fault analysis to detect DQ fault information for backup according to system 300. In one example, system 604 implements an example of UAM 630 of system 602 in defect detection 662. However, system 602 can operate based on historical or stored information, and system 604 can be considered a runtime memory failure analysis system because system 604 operates on runtime or real-time parameters that occur as well as on historical information.

[0092] In one example, Fig. 6A The system 602 of the present invention provides a machine learning-based uncorrectable memory error analysis mechanism at the memory device level. In one example, the system 604 utilizes the system 602 to generate a runtime prediction or determination of a faulty component to determine what component is the likely cause of a detected UE or other error. For example, the system 604 can generate a prediction or determination of the cause of the error and trigger a corrective action specific to the cause of the error.

[0093] System 604 includes controller 680, which can be a dedicated microcontroller or other hardware, or can represent firmware for executing on a shared controller or hardware shared with other control or management functions in a computer system. In one example, controller 680 is a controller of a host hardware platform such as hardware 640. The host hardware platform can include a CPU or other host processor 642. Memory 646 can represent multiple memory devices or multiple parallel memory resources. In one example, controller 680 represents a controller arranged on a substrate of a computer system. In one example, the substrate is a motherboard. In one example, the substrate is a memory module board. In one example, the substrate is a logic die of an HBM stack (e.g., a control layer on which a memory die is set).

[0094] According to any of the examples described, the controller 680 executes a memory fault tracker (MFT) 660, which represents an engine for determining the component that caused the error and triggering a runtime backup action for the memory area associated with the faulty component. The hardware 640 represents the hardware of the system to be monitored for memory errors. The hardware 640 provides a hardware configuration (CONFIG) 656 to the MFT 660 for error analysis. The configuration 656 represents a specific hardware component and its features and settings. The hardware 640 may include a host processor 642, peripheral devices 644, and memory 646 representing the processing resources of the computer system.

[0095] Peripheral devices 644 represent components and features of hardware 640 that can change the handling of memory errors. Therefore, hardware components and software / firmware configurations of hardware components that can affect how memory errors are handled can be included in the configuration information for consideration to send to MFT 660 for memory fault analysis. Examples of peripheral configurations can include peripheral control hub (PCH) configuration, management engine (ME) configuration, quick path interconnect (QPI) capabilities, or other components or capabilities.

[0096] Memory 646 represents a memory resource for which errors can be identified. In one example, system 604 monitors memory 646 to determine when correctable errors and uncorrectable errors occur in the memory. For example, such errors can be detected in a scrubbing operation or as part of an error handling routine. CE 652 represents CE data for correctable errors detected in the data of memory 646. UE 654 represents UE data for uncorrectable errors (DUE) detected in the data of memory 646.

[0097] In one example, defect detection 662 represents a UE analyzer that implements information from UAM 630 to identify faulty components in memory 646 based on historical error information correlated with system architecture information. By identifying errors at the hardware component level, memory fault tracker 660 can specifically identify which memory region(s) are defective based on the faulty component.

[0098] The DQ analyzer 664 specifically represents a component for enabling the memory fault tracker 660 to identify faulty DQs in the memory 646 based on detection and prediction by the defect detection 662. In one example, the system 604 stores the faulty DQ information in a DQ failure directory 670 as a log 668. The DQ failure directory 670 can be stored in nonvolatile RAM (NVRAM), flash memory, or other persistent memory, which enables the system 604 to store spare information persistently between boots. Some memory failures persist across power cycles of the system 604.

[0099] DQ correction 666 enables the MFT 660 to determine the corrective action to be implemented to resolve the faulty DQ. As described herein, the corrective action can include DQ sparing by remapping data bits from the faulty DQ to ECC bits freed up by reducing the ECC. In one example, the system reduces the ECC level and then remaps the bits. If the ECC level has been reduced and there are free ECC bits available, the system can remap bits from the faulty DQ without changing the ECC level.

[0100] A system according to system 604 may respond to error detection in a memory based on fault-aware analysis. Fault-aware analysis enables the system to determine, for example, by statistical prediction, the specific hardware element of the memory that caused the error, thereby identifying the faulty component and the faulty memory area associated with the faulty component. In statistical analysis, "prediction" may refer to a conclusion drawn by computational analysis. In a computational sense, a calculated prediction may identify a previous event or a previous cause. The prediction described herein may refer to a future prediction of a component that may cause an uncorrectable error (UE) or a determination of a component-level cause of a component that generated a UE. Thus, the system may prevent the occurrence of a UE, or may provide corrective action in response to the detection of a UE.

[0101] The system according to system 604 (through fault analysis or fault-aware analysis) can consider the circuit-level architecture of the memory, rather than just the number or frequency of correctable errors (CE). Observing error patterns associated with circuit structure can enable the system to confidently predict the component that is the source of the error. Fault prediction for a detected UE or predicted UE can refer to the result of a computational analysis that identifies the most likely cause of an error that occurred previously (i.e., for a detected UE) or for a UE that is expected to occur (i.e., a predicted UE).

[0102] In response to the error, the system may correlate the hardware configuration of the memory device with historical data indicating memory failures of hardware elements of the hardware configuration. Thus, the system may consider the rank, bank, row, column, or other information related to the physical organization and structure of the memory when predicting uncorrectable errors. Based on the determination of the specific component that caused the detected error (whether CE or UE), the system may identify the memory region associated with the detected error and mirror the faulty region to the reserved memory space of the memory device for accessing the data once stored in the faulty region.

[0103] A runtime micro-level fault-aware strategy based on tracking error history can detect defective memory regions (e.g., word lines, bit lines / DQ (data pins), sub-ranges of word lines / bit lines, rows, columns, devices, banks) to infer whether a certain micro-level memory component (e.g., column, row) is faulty. Analysis of the faulty component can be performed by hardware on the system platform.

[0104] Figure 7 7 is a block diagram of an example of a system for reducing ECC and remapping for failed DQs. System 700 shows a specific example of a 10x4 rank with 8 data devices (chips [0:7]) and 2 ECC devices (chips [8:9]). Rank 710 may be an example of a DDR5 DIMM. CA bus 720 represents the command / address bus for rank 710. DQ bus 730 represents the data bus for rank 710.

[0105] System 700 shows the details of the data of the read transaction. The data of the write transaction will be similar to that shown, but can be presented in the opposite order. The data is shown as data bits per DQ of the data bus 730 per unit interval (UI) (UI[0:15]) of the burst length. It can be understood that a system with an 8x4 data device of BL16 generates 8*4*16=512 bits of data. For 32 DQs (DQ[0:31]), the specific data is marked as d0, d1, d2, ..., d511, d512.

[0106] It should be understood that the data bit identification shown in system 700 is not necessarily how the memory controller will interpret the data. Instead, the data bit identification shown is only for the purpose of simplicity of description. It should be noted that although for data devices, d0, d1, d2... are shown in column-major format (DQ0 has d[0:15], DQ1 has d[16:31], DQ2 has d[32:47], etc.), for each ECC device, the ECC bits are described in row-major format. Using the row-major format for each ECC device, in the first row, e[0:3] is distributed across DQ32, DQ33, DQ34, and DQ35, respectively, e[4:7] in the second row, e[8:11] in the third row, and so on. For device 9 with DQ36, DQ37, DQ38, and DQ49, the first row shows e[64:67], the second row shows e[68:71], and so on.

[0107] Consider a DQ failure detected on DQ6, as shown by the 'X' and the box around the data bits of DQ6. Therefore, the data bits d[96:111] cannot be correctly exchanged between the DRAM device and the memory controller. In response to detecting a failed DQ, the system 700 can remap the bits of DQ6 to ECC bits.

[0108] As shown, first the system identifies (ID) the failed DQ (1). Next, the system can ignore the failed DQ (2a) and reduce the ECC (2b). The first diagram of the memory device shows 128 bits of ECC. The reduction shows that bits of device 8 are allocated to user data and the number of ECC bits in device 8 is reduced by half. Therefore, the system 700 finally has 96 bits of ECC. As shown, the system can remap (3) the failed DQ to the freed ECC bits.

[0109] System 700 shows an example where the data bits of a failed DQ6 are remapped to multiple DQs of ECC device 8, where d96, d97, d98, and d99 are located on the first row of DQ32, DQ33, DQ34, and DQ35, respectively, and d[100:111] is similarly distributed across DQ[32:35]. It can be seen that reducing the ECC to 96 bits can free up more data bits than required for DQ6, and therefore, these bits can remain unused across DQ[32:35] for UI[4:7]. ECC bits e[0:31] are distributed across DQ[32:35] on UI[8:15]. Therefore, system 700 can downgrade the ECC mode from 128-bit ECC to 96-bit ECC, and then remap the DQ6 bits to the free ECC bits in device 8. When a cache line is read from a bank affected by a DQ6 fail, a remapping may occur where the data is redirected from the ECC bits back to DQ6.

[0110] Fig. 8A 8 is a block diagram of an example of a system that reduces ECC and remaps for failed DQs. System 802 shows a specific example of a 10x4 rank with 8 data devices (chips [0:7]) and 2 ECC devices (chips [8:9]). Rank 810 can be an example of a DDR5 DIMM. CA bus 820 represents the command / address bus of rank 810. DQ bus 830 represents the data bus of rank 810.

[0111] System 802 shows the details of the data of the read transaction. The data of the write transaction will be similar to that shown, but can be presented in the opposite order. The data is shown as data bits per DQ of the data bus 830 per unit interval (UI) of the burst length (UI[0:15]). It can be understood that the system with 8x4 data devices of BL16 generates 8*4*16=512 bits of data. For 32 DQs (DQ[0:31]), the specific data is marked as d0, d1, d2, ..., d511, d512.

[0112] It should be understood that the data bit identification shown in system 802 is not necessarily how the memory controller will interpret the data. Instead, the data bit identification shown is for descriptive simplicity purposes only. Note that d0, d1, d2, ... are shown in column major format for data devices (DQ0 has d[0:15], DQ1 has d[16:31], DQ2 has d[32:47], etc.), and the ECC bits are also shown in column major format. Thus, DQ32 of device 8 has e[0:15], DQ33 has e[16:31], and so on.

[0113] Consider a DQ failure detected on DQ6, as shown by the 'X' and the box around the data bits of DQ6. Therefore, the data bits d[96:111] cannot be correctly exchanged between the DRAM device and the memory controller. In response to detecting the failed DQ, the system 802 can remap the bits of DQ6 to ECC bits.

[0114] As shown, first the system identifies (ID) the failed DQ (1). Next, the system can ignore the failed DQ (2a) and reduce the ECC (2b). The first diagram of the memory device shows 128 bits of ECC. The reduction shows that the bits of device 8 are allocated to user data, and the number of ECC bits in device 8 is reduced by half. Therefore, the system 802 finally has 96 bits of ECC. As shown, the system can remap (3) the failed DQ to the freed ECC bits.

[0115] System 802 shows an example where the data bits of a failed DQ6 are remapped to a single DQ of ECC device 8, where d[96:111] are remapped to DQ32. It can be seen that reducing the ECC to 96 bits can free up more data bits than needed for DQ6, and therefore, these bits can remain unused in DQ33. ECC bits e[0:15] are then located in DQ34, and e[16:31] are located in DQ35. Therefore, system 802 can downgrade the ECC mode from 128-bit ECC to 96-bit ECC, and then remap the DQ6 bits to the free ECC bits in device 8. When a cache line is read from a memory bank affected by the DQ6 failure, a remapping can occur where data is redirected from the ECC bits back to DQ6.

[0116] Figure 8B 8 is a block diagram of an example of a system that reduces ECC and remaps for two failed DQs. System 804 shows an example of system 802 after a second DQ failure is detected. Therefore, after one DQ is spared in system 802, the system changes from 128-bit ECC to 96-bit ECC to remap the failed DQ to the data bits of the ECC device.

[0117] Now consider a DQ failure detected on DQ29, shown by the "X" and the box around the data bits of DQ29. As a result, data bits d[464:479] cannot be properly exchanged between the DRAM device and the memory controller. In response to detecting the failed DQ, the system 804 can remap the bits of DQ29 to ECC bits that have been freed from the previous ECC reduction.

[0118] As shown, first the system identifies (ID) the failed DQ (4). Next, the system can ignore the failed DQ (5a), but does not need to reduce the ECC because the previous ECC reduction is sufficient to spare the bits of DQ29 (5b). The first figure of the memory device illustrates 96 bits of ECC. The second figure illustrates the same number of ECC bits, but in the first figure DQ33 has 16 unused bits, while in the second figure, 16 bits of DQ33 are used for the second failed DQ. Therefore, the system 804 remaps d[464:479] to DQ33.

[0119] Fig. 9 9 is a block diagram of an example of a system that reduces ECC and remaps for three failed DQs. System 900 shows a specific example of a 10x4 rank with 8 data devices (chip [0:7]) and 2 ECC devices (chip [8:9]). Rank 910 can be an example of a DDR5 DIMM. CA bus 920 represents a command / address bus for rank 910. DQ bus 930 represents a data bus for rank 910.

[0120] System 900 shows the details of the data of the read transaction. The data of the write transaction will be similar to that shown, but can be presented in the opposite order. The data is shown as data bits (UI[0:15]) per DQ of the data bus 930 per unit interval (UI) of the burst length. It can be understood that the system with 8x4 data devices of BL16 generates 8*4*16=512 bits of data. For 32 DQs (DQ[0:31]), the specific data is marked as d0, d1, d2, ..., d511, d512.

[0121] It should be understood that the data bit identification shown in system 900 is not necessarily how the memory controller will interpret the data. Instead, the data bit identification shown is for descriptive simplicity purposes only. Note that d0, d1, d2, ... are shown in column major format for data devices (DQ0 has d[0:15], DQ1 has d[16:31], DQ2 has d[32:47], etc.), and the ECC bits are also shown in column major format. Thus, DQ32 of device 8 has e[0:15], DQ33 has e[16:31], and so on.

[0122] Consider multiple DQ failures detected on DQ6, DQ29, and DQ36, as shown by the "X" and boxes around the data bits of each DQ. Three different DQ failures can be detected at a time, or one DQ failure can be detected at a time to remap different DQs when a failure is detected. As shown, data bits d[96:111], d[464:479], and e[64:79] cannot be properly exchanged between the DRAM device and the memory controller. In response to detecting a failed DQ, the system 900 can remap the bits of the data DQs and can remove the bits of the failed ECC DQ from use.

[0123] As shown, first the system identifies (ID) the failed DQ (1). Next, the system can ignore the failed DQ (2a) and reduce the ECC (2b). The first diagram of the memory device illustrates 128-bit ECC, where 16 ECC bits are unavailable due to the failed DQ36. The ECC reduction illustrates that bits of device 8 are allocated to user data, the number of ECC bits in device 8 is reduced by half, and the number of ECC bits in device 9 is also reduced by half. Therefore, the system 900 finally has 64-bit ECC.

[0124] When three failed DQs are identified in a bank (chip 1 / bank X / DQ6; chip 7 / bank X / DQ29; chip 9 / bank X / DQ35), the microcontroller or firmware logic spares the failed DQ6, DQ29, and DQ35 by switching the 10x4 device from 10x4 128-bit ECC mode to 9x4 64-bit ECC mode (for all cache line data accesses to / from the affected bank X). By downgrading the ECC mode from 10x4 128-bit to 9x4 64-bit, up to 64 ECC bits are freed up and can be reused to store data from the spare DQs. In one example, when the bank is accessed, it is redirected on-the-fly from the spare DQ to the freed ECC bits for data writes and remapped from the ECC bits back to the original DQ locations with the redirected data for reads.

[0125] As shown, the system can remap (3) the failed data DQ to the released ECC bits. System 900 shows an example in which the data bits of failed DQ6 and DQ29 are remapped to DQ32 and DQ33 of ECC device 8, respectively, where d[96:111] is remapped to DQ32 and d[464:479] is remapped to DQ33. The ECC bits of DQ36 are ignored and the bits of DQ37 are released. Therefore, 64 ECC bits can be stored in DQ34, DQ35, DQ38 and DQ39. As shown, system 900 can downgrade the ECC mode from a 128-bit ECC mode to a 64-bit ECC mode, including removing ECC bits from use and remapping the bits of the failed data DQ to the released ECC bits.

[0126] It should be understood that the system 900 can potentially support another DQ failure. If another data DQ fails, it can be remapped to DQ37, which has the released ECC bits. If another ECC DQ fails, the ECC bits can be remapped. Although the system 900 shows DQ6 mapped to DQ32 and DQ29 mapped to DQ33, it should be understood that any type of DQ sequence or order does not need to be maintained when remapping the bits. For example, DQ29 can be remapped to DQ32, and DQ6 can be remapped to DQ33. In addition, although the ECC bits are always shown in order, the remapping can also remap the order of the ECC bits.

[0127] Fig.10 1 is a block diagram of an example of a data read / write flow with DQ spare. System 1000 includes an integrated memory controller (iMC) 1010 or other memory controller that can generate reads and writes of cache lines of data (e.g., 512 bits) representing a bank 1020 of memory resources. Data RD / WR 1014 represents a read or write transaction of a cache line of data.

[0128] System 1000 shows data 1032 and data 1034, where data 1032 represents data of a normal read / write transaction in which no DQ fails, and data 1034 represents data of a read / write transaction in which there is a DQ spare. In one example, when iMC 1010 generates a read or write, it checks whether the accessed storage bank has a faulty DQ that is identified and spared. Faulty DQ 1012 represents a table or directory that identifies spare DQs.

[0129] Row 1020 includes chip [0:9], which is a data chip or data device. The row provides 640 bits of data for each memory access transaction, including 512 bits of data and 128 bits of ECC from 40 DQs. Data 1032 shows data d[0:63] in device 0, data d[64:127] in device 1, ..., data d[464:511] in device 7, ECC bits e[0:64] in device 8, a total of 512 data bits, and e[65:127] in device 9, a total of 128 ECC bits.

[0130] For normal read / write, without identifying and sparing a faulty DQ, the system 1000 can perform normal read / write using the configured ECC mode. As shown, the configured ECC mode is a 128-bit single device data correction (SDDC) mode.

[0131] In one example, the system 1000 determines that DQ5 is faulty, reduces the ECC mode, and remaps the bits of DQ5 to the spare ECC bits of device 8. With the faulty DQ, the system 1000 can implement DQ sparing. As shown, 512 bits of data come from devices [0:8] instead of just from devices [0:7] as in normal read / write. Then, devices [8:9] provide 96 bits of ECC instead of 128 bits.

[0132] In one example, for a data write, if the write goes to a bank with an identified faulty DQ, the iMC 1010 may perform DQ sparing on the fly by switching the ECC mode to 96-bit ECC and redirecting the data from the retired DQ5 to the freed and reused ECC bits of device 8. The iMC 1010 may write the remapped data to the memory device and store the DQ sparing record.

[0133] In one example, for data reads, when reading from a memory bank with spare DQs, the iMC 1010 may switch to 96-bit ECC mode and redirect the data stored in the reused ECC bits to the original location of the spare DQs and return the remapped data to the requesting application / process.

[0134] Fig.111100 is a flow chart of an example of a process for DQ sparing. Process 1100 represents a process for DQ sparing that may be performed by any system described herein. In one example, at 1102, the system may detect a DQ failure and determine whether the DQ has been previously spared for a bank. As described herein, in one example, the detection of a DQ failure may be performed by an error detector. If there has not been a previous DQ sparing, the "no" branch at 1104, then in one example, at 1106, the DQ manager may determine if there is a lower coverage ECC mode that frees up more bits. The lower coverage ECC mode may free up more bits to spare a failed DQ.

[0135] If there has been a previous DQ spare, at the “yes” branch at 1104, then in one example, the DQ manager may determine whether the ECC device has available bits for storing one DQ's worth of data at 1108. For example, if a DQ has been previously spared and two DQs' worth of data are released from one or more ECC devices, there may be available bits to spare another DQ.

[0136] If the system does not have a lower ECC mode available, the "No" branch at 1112, then in one example, DQ sparing is disabled at 1114. If there is a lower ECC mode that can free up bits, the "Yes" branch at 1112, then at 1116, the system can perform DQ fail sparing to send a new DQ mapping to the DQ remapper and a bank index to the address checker. Additionally, if there are available bits to spare another DQ, the "Yes" branch at 1110, then the system can perform DQ fail sparing.

[0137] At 1118, the system can determine whether the controller (e.g., memory controller or iMC) allows suspending access to the memory bank. If the controller allows suspending access to the memory bank, the "yes" branch at 1120, then at 1122, the system can lock the memory bank to refresh all data in the memory bank with the new DQ mapping and set a new ECC mode. After refreshing, at 1124, the system can unlock the memory bank and DQ spare is completed.

[0138] If the controller is not allowed to suspend access to the memory bank, the "No" branch at 1120, then in one example, at 1126, the system initializes register R to the lowest address in the memory bank and increments R by N cache lines. If the memory bank is not in an idle state, the "No" branch at 1128, then at 1130, the system can wait S seconds and check again whether the memory bank is idle. If the memory bank is in an idle state, the "Yes" branch at 1128, then at 1132, the system can refresh N cache lines, increment R, and determine whether R has reached the highest address in the memory bank.

[0139] If register R reaches the highest address in the bank, the "yes" branch at 1134, then at 1136, DQ standby ends. If R has not yet reached the highest address in the bank, the "no" branch at 1134, then at 1128, the process can return to determine whether the bank is idle to continue refreshing.

[0140] Fig.12 1 is a block diagram of an example of a memory subsystem in which DQ sparing may be implemented. System 1200 includes elements of a processor and a memory subsystem in a computing device. System 1200 is an example of a system according to the example of system 100, system 202, system 300, system 400, or the example of system 1000.

[0141] In one example, the system 1200 includes an error manager 1294 or other memory fault tracking engine to determine the component that is the cause of the detected UE. In one example, the error manager 1294 is part of the memory controller 1220. In one example, the error manager 1294 is part of the controller circuit other than the memory controller. In one example, the system 1200 includes a DQ manager 1296, which represents a controller that manages DQ standby when a faulty DQ is detected. DQ standby can be performed according to any of the examples described. In one example, the memory controller 1220 includes a DQ remapper 1292 to work in conjunction with the DQ manager 1296 to implement DQ standby. DQ standby includes reusing the released ECC bits for data of the standby DQ. The DQ remapper 1292 can remap the data to return to the program that triggered the read, or remap the data from the program that triggered the write.

[0142] Processor 1210 represents a processing unit of a computing platform that can execute an operating system (OS) and applications, which can be collectively referred to as a host or a user of a memory. The OS and applications perform operations that result in memory access. Processor 1210 may include one or more separate processors. Each separate processor may include a single processing unit, a multi-core processing unit, or a combination. The processing unit may be a main processor such as a CPU (central processing unit), a peripheral processor such as a GPU (graphics processing unit), or a combination. Memory access may also be initiated by devices such as a network controller or a hard disk controller. Such devices may be integrated with processors in some systems, or attached to processors via a bus (e.g., PCI Express), or a combination. System 1200 may be implemented as a SOC (system on chip), or implemented with separate components.

[0143] References to memory devices may apply to different memory types. Memory devices generally refer to volatile memory technologies. Volatile memory is memory whose state (and therefore the data stored therein) is uncertain if the device power is interrupted. Non-volatile memory refers to memory whose state is certain even if the device power is interrupted. Dynamic volatile memory requires refreshing the data stored in the device to maintain the state. An example of dynamic volatile memory includes DRAM (dynamic random access memory) or some variants, such as synchronous DRAM (SDRAM). The memory subsystem described herein can be compatible with a variety of memory technologies, such as DDR4 (Double Data Rate Version 4, JESD79-4, originally released by JEDEC (Joint Electron Device Engineering Council, now JEDEC Solid State Technology Association in September 2012), LPDDR4 (Low Power DDR Version 4, JESD209-4, originally released by JEDEC in August 2014), WIO2 (Wide I / O 2 (WideIO2), JESD229-2, originally released by JEDEC in August 2014), HBM (High Bandwidth Memory DRAM, JESD235A, originally released by JEDEC in November 2015), DDR5 (DDR Revision 5, originally released by JEDEC in July 2020), LPDDR5 (LPDDR Revision 5, JESD209-5, originally released by JEDEC in February 2019), HBM2 (HBM Revision 2, JESD235C, originally released by JEDEC in January 2020), HBM3 (HBM Revision 3, JESD238, originally released by JEDEC in January 2022), or other memory technologies or combinations of memory technologies, as well as derivative or extended technologies based on such specifications.

[0144] Memory controller 1220 represents one or more memory controller circuits or devices for system 1200. Memory controller 1220 represents control logic that generates memory access commands in response to processor 1210 performing operations. Memory controller 1220 accesses one or more memory devices 1240. Memory device 1240 can be a DRAM device according to any of the above-mentioned. In one example, memory device 1240 is organized and managed as different channels, wherein each channel is coupled to a bus and a signal line, and the bus and the signal line are coupled to multiple memory devices in parallel. Each channel can operate independently. Therefore, each channel is independently accessed and controlled, and for each channel, timing, data transmission, command and address exchange and other operations are separated. Coupling can refer to electrical coupling, communication coupling, physical coupling or a combination of these. Physical coupling can include direct contact. Electrical coupling includes interfaces or interconnections that allow current flow between components, or allow signaling between components, or both. Communication coupling includes connections (including wired or wireless) that enable components to exchange data.

[0145] In one example, the settings for each channel are controlled by separate mode registers or other register settings. In one example, each memory controller 1220 manages a separate memory channel, but the system 1200 can be configured to have multiple channels managed by a single controller, or to have multiple controllers on a single channel. In one example, the memory controller 1220 is part of the main processor 1210, such as logic implemented on the same die as the processor or implemented in the same package space.

[0146] The memory controller 1220 includes an I / O interface logic 1222 to couple to a memory bus, such as the memory channel mentioned above. The I / O interface logic 1222 (and the I / O interface logic 1242 of the memory device 1240) may include pins, pads, connectors, signal lines, traces, or cables, or other hardware for connecting devices, or a combination of these. The I / O interface logic 1222 may include a hardware interface. As shown, the I / O interface logic 1222 includes at least a driver / transceiver for a signal line. Typically, a cable within an integrated circuit interface is coupled to a pad, pin, or connector to engage a signal line or trace or other cable between devices. The I / O interface logic 1222 may include a driver, a receiver, a transceiver, or a terminal, or other circuit devices or a combination of circuit devices to exchange signals on signal lines between devices. The exchange of signals includes at least one of sending or receiving. Although shown as coupling I / O 1222 from memory controller 1220 to I / O 1242 of memory device 1240, it should be understood that in implementations of system 1200 where memory device group 1240 is accessed in parallel, multiple memory devices may include I / O interfaces to the same interface of memory controller 1220. In implementations of system 1200 that include one or more memory modules 1270, I / O 1242 may include interface hardware for the memory modules in addition to interface hardware on the memory devices themselves. Other memory controllers 1220 will include separate interfaces to other memory devices 1240.

[0147] The bus between the memory controller 1220 and the memory device 1240 can be implemented as a plurality of signal lines that couple the memory controller 1220 to the memory device 1240. The bus can generally include at least a clock (CLK) 1232, a command / address (CMD) 1234, and write data (DQ) and read data (DQ) 1236, as well as zero or more other signal lines 1238. In one example, the bus or connection between the memory controller 1220 and the memory can be referred to as a memory bus. In one example, the memory bus is a multi-drop bus. The signal line for CMD can be referred to as a "C / A bus" (or an ADD / CMD bus, or some other name indicating the transmission of command (C or CMD) and address (A or ADD) information), and the signal line for writing and reading DQ can be referred to as a "data bus". In one example, independent channels have different clock signals, C / A buses, data buses, and other signal lines. Therefore, the system 1200 can be considered to have multiple "buses" in the sense that independent interface paths can be considered to be separate buses. It should be understood that, in addition to the lines clearly shown, the bus may also include at least one of a gate command line, an alarm line, an auxiliary line, or other signal lines, or a combination. It should also be understood that serial bus technology can be used for the connection between the memory controller 1220 and the memory device 1240. An example of serial bus technology is 8B10B encoding and transmission of high-speed data with an embedded clock on a single differential signal pair in each direction. In one example, CMD 1234 represents a signal line shared in parallel with multiple memory devices. In one example, multiple memory devices share the encoding command signal line of CMD 1234, and each memory device has a separate chip select (CS_n) signal line to select a separate memory device.

[0148] It should be understood that in the example of system 1200, the bus between the memory controller 1220 and the memory device 1240 includes an auxiliary command bus CMD 1234 and an auxiliary bus DQ 1236 for carrying write and read data. In one example, the data bus may include a bidirectional line for reading data and for writing / command data. In another example, the auxiliary bus DQ 1236 may include a unidirectional write signal line for writing data from the host to the memory, and may include a unidirectional line for reading data from the memory to the host. Depending on the selected memory technology and system design, other signals 1238 may accompany the bus or sub-bus, such as a strobe line DQS. Based on the design of the system 1200 or based on the implementation (if the design supports multiple implementations), the data bus may have more or less bandwidth for each memory device 1240. For example, the data bus may support memory devices with x4 interfaces, x8 interfaces, x16 interfaces, or other interfaces. The convention "xW" represents a plurality of signal lines for exchanging data with the memory controller 1220, where W refers to an integer of the interface size or interface width of the memory device 1240. The interface size of the memory device is a controlling factor as to how many memory devices can be used simultaneously or coupled in parallel to the same signal line per channel in the system 1200. In one example, a high bandwidth memory device, a wide interface device, or a stacked memory configuration, or a combination can implement a wider interface, such as a x128 interface, a x256 interface, a x512 interface, a x1024 interface, or other data bus interface widths.

[0149] In one example, the memory device 1240 and the memory controller 1220 exchange data in a burst or a series of continuous data transmissions through the data bus. A burst corresponds to a plurality of transmission cycles, and the transmission cycle is related to the bus frequency. In one example, a transmission cycle can be the entire clock cycle of the transmission occurring on the same clock or strobe signal edge (e.g., rising edge). In one example, each clock cycle (referring to a cycle of the system clock) is divided into a plurality of unit intervals (UI), wherein each UI is a transmission cycle. For example, a double data rate transmission is triggered on two edges (e.g., rising edge and falling edge) of the clock signal. A burst can last for a configured number of UIs, which can be a configuration stored in a register, or can be triggered instantly. For example, a sequence of eight continuous transmission periods can be considered as a burst length of eight (BL8), and each memory device 1240 can transmit data on each UI. Therefore, an x8 memory device operating on BL8 can transmit 64 bits of data (8 data signal lines multiplied by 8 data bits transmitted on each line on the burst). It should be understood that this simple example is exemplary only and not restrictive.

[0150] Memory devices 1240 represent memory resources of system 1200. In one example, each memory device 1240 is a separate memory die. In one example, each memory device 1240 can be coupled to multiple (e.g., 2) channels of each device or die. Each memory device 1240 includes I / O interface logic 1242, which has a bandwidth determined by the implementation of the device (e.g., x16 or x8 or some other interface bandwidth). I / O interface logic 1242 enables the memory device to be coupled to memory controller 1220. I / O interface logic 1242 may include a hardware interface and may be based on the I / O 1222 of the memory controller, but at the memory device end. In one example, multiple memory devices 1240 are connected in parallel to the same command and data bus. In another example, multiple memory devices 1240 are connected in parallel to the same command bus and to different data buses. For example, system 1200 may be configured with multiple memory devices 1240 coupled in parallel, each of which responds to commands and accesses memory resources 1260 inside each memory device. For a write operation, a separate memory device 1240 may write a portion of a whole data word, and for a read operation, a separate memory device 1240 may retrieve a portion of a whole data word. The remaining bits of the word will be provided or received in parallel by other memory devices.

[0151] In one example, the memory device 1240 is directly arranged on the motherboard or host system platform of the computing device (e.g., a PCB (printed circuit board, printed circuit board) or substrate on which the processor 1210 is arranged). In one example, the memory device 1240 can be organized into a memory module 1270. In one example, the memory module 1270 represents a dual inline memory module (DIMM). In one example, the memory module 1270 represents other organizations of multiple memory devices for sharing at least a portion of an access or control circuit device, which can be a circuit, a separate device, or a separate board separated from the host system platform. The memory module 1270 may include multiple memory devices 1240, and the memory module may include support for multiple separate channels to the included memory devices arranged thereon. In another example, the memory device 1240 may be incorporated into the same package as the memory controller 1220, for example, by a technology such as a multi-chip module (MCM), a stacked package, a through-silicon via (TSV), or other technologies or combinations thereof. Similarly, in one example, multiple memory devices 1240 can be incorporated into a memory module 1270, which itself can be incorporated into the same package as the memory controller 1220. It should be appreciated that for these and other implementations, the memory controller 1220 can be part of the host processor 1210.

[0152] Each memory device 1240 includes one or more memory arrays 1260. The memory array 1260 represents an addressable memory location or a storage location for data. Typically, the memory array 1260 is managed as a data row, which is accessed via word lines (rows) and bit lines (individual bits within a row) control. The memory array 1260 can be organized into separate channels, rows, and banks of memory. A channel can refer to an independent control path to a storage location within the memory device 1240. A row can refer to a common location across multiple memory devices in parallel (e.g., the same row address within different devices). A bank can refer to a sub-array of memory locations within the memory device 1240. In one example, a memory bank is divided into sub-banks, wherein at least a portion of a shared circuit device (e.g., a driver, a signal line, a control logic) is used for a sub-bank to allow separate addressing and access. It should be understood that channels, banks, sub-banks, bank groups, or other organizations of memory locations, and combinations of organizations can overlap in their application to physical resources. For example, the same physical memory location may be accessed through a particular channel as a particular bank, which may also belong to a rank. Thus, the organization of memory resources is to be understood in an inclusive rather than exclusive manner.

[0153] In one example, the memory device 1240 includes one or more registers 1244. Register 1244 represents one or more storage devices or storage locations that provide configuration or settings for the operation of the memory device. In one example, register 1244 can provide a storage location for the memory device 1240 to store data for access by the memory controller 1220 as part of a control or management operation. In one example, register 1244 includes one or more mode registers. In one example, register 1244 includes one or more multi-purpose registers. The configuration of the locations within register 1244 can configure the memory device 1240 to operate in different "modes", where command information can trigger different operations within the memory device 1240 based on the mode. In addition or alternatively, different modes can also trigger different operations from address information or other signal lines based on the mode. The settings of register 1244 can indicate the configuration of I / O settings (e.g., timing, termination or ODT (on-die termination) 1246, driver configuration, or other I / O settings).

[0154] In one example, the memory device 1240 includes an ODT 1246 as part of the interface hardware associated with the I / O 1242. The ODT 1246 can be configured as described above and provides a setting for the impedance of the interface to be applied to the specified signal line. In one example, the ODT 1246 is applied to the DQ signal line. In one example, the ODT 1246 is applied to the command signal line. In one example, the ODT 1246 is applied to the address signal line. In one example, the ODT 1246 can be applied to any combination of the foregoing. The ODT setting can be changed based on whether the memory device is a selected target or a non-target device for the access operation. The ODT 1246 setting can affect the timing and reflection of the signaling on the termination line. Careful control of the ODT 1246 can achieve higher speed operation by improving the matching of the applied impedance and load. The ODT 1246 can be applied to specific signal lines of the I / O interface 1242, 1222 (e.g., ODT for DQ lines or ODT for CA lines), and does not have to be applied to all signal lines.

[0155] The memory device 1240 includes a controller 1250, which represents the control logic within the memory device for controlling the internal operations within the memory device. For example, the controller 1250 decodes the command sent by the memory controller 1220 and generates internal operations to execute or satisfy the command. The controller 1250 can be referred to as an internal controller and is separate from the memory controller 1220 of the host. The controller 1250 can determine what mode to select based on the register 1244, and configure the internal execution of operations for accessing the memory resource 1260 or other operations based on the selected mode. The controller 1250 generates control signals to control the routing of bits within the memory device 1240, thereby providing an appropriate interface for the selected mode and directing the command to the appropriate memory location or address. The controller 1250 includes command logic 1252, which can decode the command encoding received on the command and address signal lines. Therefore, the command logic 1252 can be or include a command decoder. Using the command logic 1252, the memory device can identify the command and generate internal operations to execute the requested command.

[0156] Referring again to the memory controller 1220, the memory controller 1220 includes command (CMD) logic 1224, which represents logic or circuitry for generating commands to send to the memory device 1240. The generation of commands may refer to the scheduling of a previous command, or the preparation of a queued command to be sent. In general, signaling in a memory subsystem includes address information within or accompanying a command to indicate or select one or more memory locations where the memory device should execute the command. In response to the transaction scheduling of the memory device 1240, the memory controller 1220 may issue a command via the I / O 1222 to cause the memory device 1240 to execute the command. In one example, the controller 1250 of the memory device 1240 receives and decodes the command and address information received from the memory controller 1220 via the I / O 1242. Based on the received command and address information, the controller 1250 may control the timing of the operation of the logic and circuitry within the memory device 1240 to execute the command. The controller 1250 is responsible for complying with the standards or specifications within the memory device 1240, such as timing and signaling requirements. The memory controller 1220 may implement compliance with standards or specifications through access scheduling and control.

[0157] The memory controller 1220 includes a scheduler 1230, which represents a logic or circuit device for generating transactions and ordering the transactions to be sent to the memory device 1240. From one perspective, the main function of the memory controller 1220 can be said to be to schedule memory access and other transactions to the memory device 1240. Such scheduling can include generating the transaction itself to implement the processor 1210 request for data and maintain the integrity of the data (for example, using commands related to refresh). A transaction can include one or more commands and result in the transmission of commands or data or both over one or more timing cycles (such as clock cycles or unit intervals). Transactions can be used for access, such as read or write or related commands or combinations, and other transactions can include memory management commands for configuration, setup, data integrity or other commands or combinations.

[0158] The memory controller 1220 typically includes logic such as a scheduler 1230 to allow transactions to be selected and ordered to improve the performance of the system 1200. Thus, the memory controller 1220 can select which outstanding transactions should be sent to the memory device 1240 in what order, which is usually achieved using logic that is much more complex than a simple first-in, first-out algorithm. The memory controller 1220 manages the transmission of transactions to the memory device 1240 and manages the timing associated with the transactions. In one example, the transactions have deterministic timing, which can be managed by the memory controller 1220 and used to determine how to schedule transactions using the scheduler 1230.

[0159] In one example, the memory controller 1220 includes refresh (REF) logic 1226. The refresh logic 1226 can be used for memory resources that are volatile and need to be refreshed to maintain a deterministic state. In one example, the refresh logic 1226 indicates the location for refresh and the type of refresh to be performed. The refresh logic 1226 can trigger a self-refresh within the memory device 1240, or perform an external refresh (which can be referred to as an auto-refresh command) by sending a refresh command, or a combination. In one example, the controller 1250 within the memory device 1240 includes refresh logic 1254 to apply refresh within the memory device 1240. In one example, the refresh logic 1254 generates internal operations to perform refreshes based on external refreshes received from the memory controller 1220. The refresh logic 1254 can determine whether the refresh is for the memory device 1240, and what memory resources 1260 are refreshed in response to the command.

[0160] Figures 13A-13B is a block diagram of an example of a CAMM system in which DQ spare may be implemented.

[0161] Reference Fig.13A , system 1000 includes a memory stack architecture monitored by a memory fault tracker that can perform mirroring. System 1302 is an example of a system according to an example of system 100 , system 202 , system 300 , system 400 , or an example of system 1000 .

[0162] Substrate 1310 illustrates a SOC package substrate or a motherboard or a system board. Substrate 1310 includes contacts 1312, which represent contacts for connecting to a memory. CPU 1314 represents a processor or a central processing unit (CPU) chip or a graphics processing unit (GPU) chip to be arranged on substrate 1310. CPU 1314 performs computing operations in system 1302. In one example, CPU 1314 includes multiple cores (not specifically shown), which can generate operations requesting to read data from a memory and write data to a memory. CPU 1314 may include a memory controller to manage access to a memory device.

[0163] Compression-attached memory module (CAMM) 1330 represents a module with memory devices, which are not specifically shown in system 1302. Substrate 1310 is coupled to CAMM 1330 and its memory devices through compression mount technology (CMT) connector 1320. Connector 1320 includes contact 1322, which is a compression-based contact. Compression-based contacts are compressible pins or devices whose shapes compress as pressure is applied to connector 1320. In one example, contact 1322 represents a C-shaped pin as shown. In one example, contact 1322 represents another compressible pin shape, such as a spring shape, an S shape, or a pin with other shapes that can be compressed.

[0164] CAMM 1330 includes contacts 1332 on one side of the CAMM board that engages with connector 1320. Contacts 1332 are connected to memory devices on the CAMM board. Plate 1340 represents a plate or housing that provides a structure for applying pressure to compress contacts 1322 of connector 1320.

[0165] Reference Fig. 13B , system 1304 is a perspective view of a system according to system 1302. System 1304 shows a DQ manager 1350, which is not specifically shown in system 1302. DQ manager 1350 represents a controller that manages DQ spares in the event that a failed DQ is detected. DQ spares can be performed according to any of the examples described. DQ spares include reusing freed ECC bits for data for spare DQs. A DQ remapper can remap data back to the program that triggered the read, or remap data from the program that triggered the write.

[0166] CAMM 1330 is shown with memory chips or memory dies identified as DRAM 1336 on one or both sides of the PCB of CAMM 1330. DRAM 1336 is coupled to conductive contacts via conductive traces in or on the PCB, which are coupled to contacts 1332, which in turn are coupled to contacts 1322 of connector 1320.

[0167] System 1304 shows holes 1342 in plate 1340 to receive fasteners represented by screws 1344. There are corresponding holes through CAMM 1330, connector 1320, and in substrate 1310. Screws 1344 can compressibly attach CAMM 1330 to substrate 1310 via connector 1320.

[0168] Fig.14 is a block diagram of an example of a computing system in which DQ standby may be implemented. System 1400 represents a computing device according to any example herein, and may be a laptop computer, desktop computer, tablet computer, server, gaming or entertainment control system, embedded computing device, or other electronic device.

[0169] System 1400 is an example of a system according to the example of system 100, system 202, system 300, system 400, or system 1000. In one example, system 1400 includes a DQ manager 1490 that manages a controller for DQ spare when a failed DQ is detected. DQ spare can be performed according to any of the examples described. DQ spare includes reusing the freed ECC bits for data of the spare DQ. The DQ remapper can remap the data back to the program that triggered the read, or remap the data from the program that triggered the write.

[0170] The system 1400 includes a processor 1410, which may include any type of microprocessor, central processing unit (CPU), graphics processing unit (GPU), processing core, or other processing hardware, or combination, to provide processing or execution of instructions for the system 1400. The processor 1410 may be a host processor device. The processor 1410 controls the overall operation of the system 1400 and may be or include one or more programmable general or special purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), or a combination of such devices.

[0171] System 1400 includes boot / configuration 1416, which represents a storage device for storing boot code (e.g., basic input / output system (BIOS)), configuration settings, security hardware (e.g., trusted platform module (TPM)), or other system-level hardware operating outside of the host OS. Boot / configuration 1416 may include non-volatile storage devices such as read-only memory (ROM), flash memory, or other memory devices.

[0172] In one example, the system 1400 includes an interface 1412 coupled to the processor 1410, which may represent a higher speed interface or a high throughput interface for system components (e.g., the memory subsystem 1420 or the graphics interface component 1440) that require a higher bandwidth connection. The interface 1412 represents an interface circuit, which may be a separate component or integrated onto a processor die. The interface 1412 may be integrated onto the processor die as a circuit or integrated onto a system on a chip as a component. Where present, the graphics interface 1440 interfaces with the graphics component to provide a visual display to a user of the system 1400. The graphics interface 1440 may be a separate component or integrated onto a processor die or onto a system on a chip. In one example, the graphics interface 1440 may drive a high definition (HD) display or an ultra high definition (UHD) display that provides output to a user. In one example, the display may include a touch screen display. In one example, the graphics interface 1440 generates a display based on data stored in the memory 1430 or based on operations performed by the processor 1410 or both.

[0173] Memory subsystem 1420 represents the main memory of system 1400 and provides storage for code executed by processor 1410 or data values ​​for executing routines. Memory subsystem 1420 may include one or more random-access memories (RAM), such as DRAM, 3DXP (three-dimensional crosspoint), or other memory devices, or a combination of such devices. Memory 1430 stores and hosts operating system (OS) 1432 to provide a software platform for executing instructions in system 1400. In addition, application 1434 can be executed on the software platform of OS 1432 from memory 1430. Application 1434 represents a program with its own operating logic to implement the execution of one or more functions. Process 1436 represents an agent or routine that provides auxiliary functions to OS 1432 or one or more applications 1434 or a combination. OS 1432, application 1434 and process 1436 provide software logic to provide functions for system 1400. In one example, the memory subsystem 1420 includes a memory controller 1422, which is a memory controller for generating commands and issuing commands to the memory 1430. It should be understood that the memory controller 1422 can be a physical part of the processor 1410 or a physical part of the interface 1412. For example, the memory controller 1422 can be an integrated memory controller, integrated into a circuit with the processor 1410, such as integrated into a processor die or a system on a chip.

[0174] Although not specifically shown, it should be understood that system 1400 may include one or more buses or bus systems between devices, such as a memory bus, a graphics bus, an interface bus, or other buses. A bus or other signal line may communicate or electrically couple components together, or both communicate and electrically couple these components. A bus may include a physical communication line, a point-to-point connection, a bridge, an adapter, a controller, or other circuit device or a combination thereof. A bus may include, for example, a system bus, a peripheral component interconnect (PCI) bus, a HyperTransport or an industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB) or other buses or one or more of a combination thereof.

[0175] In one example, system 1400 includes interface 1414, which can be coupled to interface 1412. Interface 1414 can be an interface that is slower than interface 1412. In one example, interface 1414 represents an interface circuit, which can include independent components and integrated circuit devices. In one example, multiple user interface components or peripheral components or both are coupled to interface 1414. Network interface 1450 provides system 1400 with the ability to communicate with remote devices (e.g., servers or other computing devices) through one or more networks. Network interface 1450 can include an Ethernet adapter, a wireless interconnect component, a cellular network interconnect component, a USB (universal serial bus), or other interfaces based on wired or wireless standards or proprietary. Network interface 1450 can exchange data with a remote device, which can include sending data stored in a memory or receiving data to be stored in a memory.

[0176] In one example, the system 1400 includes one or more input / output (I / O) interfaces 1460. The I / O interfaces 1460 may include one or more interface components (e.g., audio, alphanumeric, tactile / touch, or other interfaces) through which a user interacts with the system 1400. The peripheral interfaces 1470 may include any hardware interface not specifically mentioned above. A peripheral device generally refers to a device that is connected to the system 1400 in a dependent manner. A dependent connection is a connection over which the system 1400 provides a software platform or a hardware platform or both on which operations are performed and with which the user interacts.

[0177] In one example, system 1400 includes a storage subsystem 1480 for storing data in a non-volatile manner. In one example, in certain system implementations, at least some components of storage 1480 may overlap with components of memory subsystem 1420. Storage subsystem 1480 includes storage device 1484, which may be or include any conventional media for storing large amounts of data in a non-volatile manner, such as one or more magnetic, solid-state, NAND, 3DXP, or optical based disks, or a combination. Storage 1484 saves code or instructions and data 1486 in a persistent state (i.e., the value is retained despite power interruption to system 1400). Although memory 1430 is typically an execution or operating memory that provides instructions to processor 1410, storage 1484 may be generally considered to be "memory." Although storage 1484 is non-volatile, memory 1430 may include volatile memory (i.e., if power to system 1400 is interrupted, the value or state of the data is indeterminate). In one example, the storage subsystem 1480 includes a controller 1482 to interface with the storage device 1484. In one example, the controller 1482 is a physical part of the interface 1414 or the processor 1410, or may include circuits or logic in both the processor 1410 and the interface 1414.

[0178] The power supply 1402 provides power to the components of the system 1400. More specifically, the power supply is typically coupled to one or more power supply devices 1404 in the system 1400 to provide power to the components of the system 1400. In one example, the power supply device 1404 includes an AC to DC (alternating current to direct current) adapter to plug into a wall outlet. This AC power can be a renewable energy (e.g., solar) power supply 1402. In one example, the power supply 1402 includes a DC power source, such as an external AC-DC converter. In one example, the power supply 1402 or the power supply device 1404 includes wireless charging hardware to charge via a proximity charging field. In one example, the power supply 1402 may include an internal battery or fuel cell source.

[0179] Fig.15 15 is a block diagram of an example of a multi-node network in which DQ standby can be implemented. In one example, system 1500 represents a data center. In one example, system 1500 represents a server farm. In one example, system 1500 represents a data cloud or a processing cloud.

[0180] Node 1530 of system 1500 represents a system according to the example of system 100, system 202, system 300, system 400, or system 1000. In one example, node 1530 includes a DQ manager 1590, which represents a controller that manages DQ spare when a failed DQ is detected. DQ spare can be performed according to any of the examples described. DQ spare includes reusing the released ECC bits for data of the spare DQ. The DQ remapper can remap the data back to the program that triggered the read, or remap the data from the program that triggered the write.

[0181] One or more clients 1502 send requests to the system 1500 through the network 1504. The network 1504 represents one or more local networks, or wide area networks, or a combination. The client 1502 can be a human or machine client that generates a request to perform an operation on the system 1500. The system 1500 performs the application or data computing task requested by the client 1502.

[0182] In one example, system 1500 includes one or more racks, one or more racks representing structural and interconnection resources for housing and interconnecting multiple computing nodes. In one example, rack 1510 includes multiple nodes 1530. In one example, rack 1510 hosts multiple blade components, blade 1520[0], ..., blade 1520[N-1], collectively referred to as blade 1520. Hosting refers to providing power, structural or mechanical support, and interconnection. Blade 1520 can refer to a computing resource on a printed circuit board (PCB), where the PCB houses the hardware components of one or more nodes 1530. In one example, blade 1520 does not include a chassis or housing or other "box" other than that provided by rack 1510. In one example, blade 1520 includes a housing with exposed connectors to connect to rack 1510. In one example, system 1500 does not include rack 1510, and each blade 1520 includes a chassis or housing that can be stacked or otherwise reside in close proximity to other blades and allows nodes 1530 to be interconnected.

[0183] The system 1500 includes a fabric 1570, which represents one or more interconnections of the nodes 1530. In one example, the fabric 1570 includes a plurality of switches 1572 or routers or other hardware to route signals between the nodes 1530. Additionally, the fabric 1570 can couple the system 1500 to a network 1504 for access by clients 1502. In addition to routing devices, the fabric 1570 can be considered to also include cables or ports or other hardware devices to couple the nodes 1530 together. In one example, the fabric 1570 has one or more associated protocols to manage the routing of signals through the system 1500. In one example, the one or more protocols depend, at least in part, on the hardware devices used in the system 1500.

[0184] As shown, rack 1510 includes N blades 1520. In one example, in addition to rack 1510, system 1500 also includes rack 1550. As shown, rack 1550 includes M blade components, blade 1560[0], ..., blade 1560[M-1], collectively referred to as blade 1560. M is not necessarily the same as N; therefore, it should be understood that a variety of different hardware device components can be used and coupled together into system 1500 through structure 1570. Blade 1560 can be the same or similar to blade 1520. Node 1530 can be any type of node and is not necessarily all nodes of the same type. System 1500 is not limited to homogeneity, nor is it limited to heterogeneity.

[0185] The nodes in the system 1500 may include computing nodes, memory nodes, storage nodes, accelerator nodes, or other nodes. Rack 1510 is represented by memory nodes 1522 and storage nodes 1524, which represent shared system memory resources and shared persistent storage devices, respectively. One or more nodes of rack 1550 may be memory nodes or storage nodes.

[0186] Node 1530 represents an example of a computing node. For simplicity, only the computing nodes in blade 1520[0] are shown in detail. However, other nodes in system 1500 can be the same or similar. At least some of nodes 1530 are computing nodes, which have a processor (proc) 1532 and a memory 1540. A computing node refers to a node with processing resources (e.g., one or more processors) that executes an operating system and can receive and process one or more tasks. In one example, at least some of nodes 1530 are server nodes, where the server acts as a processing resource represented by processor 1532 and memory 1540.

[0187] Memory node 1522 represents an example of a memory node where system memory is located outside the compute node. The memory node may include a controller 1582, which represents a processor on the node for managing access to memory. The memory node includes memory 1584 as a memory resource to be shared between multiple compute nodes.

[0188] Storage node 1524 represents an example of a storage server, which refers to a node with more storage resources than a computing node, and rather than a storage server having a processor for performing tasks, includes processing resources for managing access to storage nodes within the storage server. A storage node may include a controller 1586 for managing access to a storage device 1588 of the storage node.

[0189] In one example, node 1530 includes an interface controller 1534, which represents logic for controlling access to structure 1570 by node 1530. The logic may include hardware resources for interconnecting with physical interconnect hardware. The logic may include software or firmware logic for managing interconnection. In one example, interface controller 1534 is or includes a host fabric interface, which may be a fabric interface according to any example described herein. The interface controllers of memory node 1522 and storage node 1524 are not explicitly shown.

[0190] Processor 1532 may include one or more individual processors. Each individual processor may include a single processing unit, a multi-core processing unit, or a combination. The processing unit may be a main processor such as a CPU (central processing unit), a peripheral processor such as a GPU (graphics processing unit), or a combination. Memory 1540 may be or include a memory device represented by memory 1540 and a memory controller represented by controller 1542.

[0191] Generally speaking, with respect to the description herein, in one aspect, a memory controller includes: a hardware interface to a data bus for coupling to a plurality of data dynamic random access memory (DRAM) devices and at least one error correction code (ECC) DRAM device; and an error manager for detecting a failure of a data signal (DQ) of one of the plurality of data DRAM devices when coupled to the data bus, dynamically switching ECC modes on the fly, mapping out data bits of the DQ, and remapping ECC bits of at least one ECC DRAM device to the mapped out data bits of the DQ.

[0192] In one example of a memory controller, a data bus has at least two ECC DRAM devices. According to any of the previous examples of the memory controller, in one example, the error manager is used to remap the ECC bits of only one DQ of the ECC device to the mapped DQ. According to any of the previous examples of the memory controller, in one example, the error manager is used to remap the ECC bits of multiple DQs of the ECC device to the mapped DQ. According to any of the previous examples of the memory controller, in one example, the error manager is used to dynamically switch the ECC mode from the 10x4 128-bit ECC mode to the 10x4 96-bit ECC mode. According to any of the previous examples of the memory controller, in one example, the error manager is used to subsequently dynamically switch the ECC mode from the 10x4 96-bit ECC mode to the 9x4 64-bit ECC mode. According to any of the previous examples of the memory controller, in one example, the error manager is used to dynamically switch the ECC mode from the 10x4 128-bit ECC mode to the 9x4 64-bit ECC mode. According to any of the previous examples of the memory controller, in one example, the error manager is used to dynamically switch the ECC mode from the 10x4 128-bit ECC mode to the 9x4 64-bit ECC mode. According to any previous example of a memory controller, in one example, an error manager is used to refresh a memory bank that once contained mapped-out data bits of a DQ, wherein the memory controller is used to suspend read and write requests to the memory bank until all cache lines of the memory bank are refreshed. According to any previous example of a memory controller, in one example, an error manager is used to refresh a memory bank that once contained mapped-out data bits of a DQ, wherein the error manager includes an address checker having a register that is used to track refreshes of cache lines of the memory bank during idle time of the memory bank.

[0193] Generally speaking, with respect to the description herein, in one aspect, a system for managing memory errors includes: a plurality of data dynamic random access memory (DRAM) devices coupled to a data bus; an error correct code (ECC) DRAM device coupled to the data bus; and a data signal (DQ) manager for detecting a failure of a DQ of one of the plurality of data DRAM devices, dynamically switching ECC modes on the fly, mapping out data bits of the DQ, and remapping ECC bits of the ECC DRAM device to the mapped out data bits of the DQ.

[0194] According to an example of the system, the DQ manager includes circuitry of a memory controller coupled to a data bus. According to any previous example of the system, in one example, the DQ manager includes circuitry of a controller disposed on a substrate, the circuitry being separate from the memory controller coupled to the data bus. According to any previous example of the system, in one example, the ECC DRAM device includes a first ECC DRAM device and also includes a second ECC DRAM device. According to any previous example of the system, in one example, the DQ manager is used to remap ECC bits of only one ECC device to a mapped DQ. According to any previous example of the system, in one example, the DQ manager is used to remap ECC bits of multiple ECC devices to the mapped DQ. According to any previous example of the system, in one example, the DQ manager is used to dynamically switch an ECC mode from a 10x4 128-bit ECC mode to a 10x4 96-bit ECC mode. According to any previous example of the system, in one example, the DQ manager is used to dynamically switch an ECC mode from a first ECC mode to a 9x4 64-bit ECC mode. According to any previous example of the system, in one example, the system includes a memory controller coupled to a data bus, wherein a DQ manager is used to refresh a memory bank that once contained mapped-out data bits of a DQ, wherein the memory controller is used to suspend read and write requests to the memory bank until all cache lines of the memory bank are refreshed. According to any previous example of the system, in one example, the DQ manager is used to refresh a memory bank that once contained mapped-out data bits of a DQ, wherein the DQ manager includes an address checker having a register that is used to track refreshes of cache lines of the memory bank during memory bank idle time. According to any previous example of the system, in one example, the system includes one or more of: a display communicatively coupled to a central processing unit (CPU); a network interface communicatively coupled to a host processor; or a battery for powering the system.

[0195] In one aspect, a method includes: detecting a failure of a data signal (DQ) of a data bus coupled to a plurality of data dynamic random access memory (DRAM) devices and at least one error correction code (ECC) DRAM device; dynamically switching an ECC mode on the fly; mapping out data bits of the DQ; and remapping the ECC bits of at least one ECC DRAM device to the mapped data bits of the DQ.

[0196] According to an example of the method, the data bus has at least two ECC DRAM devices. According to any previous example of the method, in one example, remapping the ECC bits includes: remapping the ECC bits of only one DQ of the ECC device to the mapped DQ. According to any previous example of the method, in one example, remapping the ECC bits includes: remapping the ECC bits of multiple DQs of the ECC device to the mapped DQ. According to any previous example of the method, in one example, dynamically switching the ECC mode includes: dynamically switching the ECC mode from a 10x4 128-bit ECC mode to a 10x4 96-bit ECC mode. According to any previous example of the method, in one example, dynamically switching the ECC mode includes subsequently dynamically switching the ECC mode from a 10x4 96-bit ECC mode to a 9x4 64-bit ECC mode. According to any previous example of the method, in one example, dynamically switching the ECC mode includes: dynamically switching the ECC mode from a 10x4 128-bit ECC mode to a 9x4 64-bit ECC mode. According to any previous example of the method, in one example, the method further includes refreshing the memory bank that once contained the mapped data bits of the DQ, including suspending read requests and write requests to the memory bank until all cache lines of the memory bank are refreshed. According to any previous example of the method, in one example, the method further includes: refreshing the memory bank that once contained the mapped data bits of the DQ, and tracking the refresh of the cache lines of the memory bank during the idle time of the memory bank with an address checker, the address checker having a register for tracking the refresh.

[0197] Flowcharts as shown herein provide examples of sequences of various process actions. Flowcharts may indicate operations to be performed by software or firmware routines as well as physical operations. Flowcharts may illustrate examples of implementations of states of finite state machines (FSMs), which may be implemented in hardware and / or software. Although shown in a particular order or sequence, the order of actions may be modified unless otherwise specified. Therefore, the figures shown should be understood only as examples, and the process may be performed in different orders, and some actions may be performed in parallel. In addition, one or more actions may be omitted; therefore, not all implementations will perform all operations.

[0198] With respect to the various operations or functions described herein, they may be described or defined as software code, instructions, configurations, and / or data. The content may be directly executable ("object" or "executable" form), source code, or differential code ("incremental" or "patch" code). The software content described herein may be provided via an article having content stored thereon, or provided via a method of operating a communication interface to send data via a communication interface. A machine-readable storage medium may enable a machine to perform the functions or operations described, and includes any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as a recordable / unrecordable medium (e.g., a read-only memory (ROM), a random access memory (RAM), a disk storage medium, an optical storage medium, a flash memory device, etc.). The communication interface includes any mechanism for engaging with any of hardwired, wireless, optical, and other media to communicate with another device, such as a memory bus interface, a processor bus interface, an Internet connection, a disk controller, etc. The communication interface may be configured by providing configuration parameters and / or sending signals to prepare the communication interface to provide a data signal describing the software content. The communication interface may be accessed via one or more commands or signals sent to the communication interface.

[0199] The various components described herein may be units for performing the described operations or functions. Each component described herein includes software, hardware, or a combination thereof. These components may be implemented as software modules, hardware modules, dedicated hardware (e.g., dedicated hardware, application specific integrated circuits (ASICs), digital signal processors (DSPs), etc.), embedded controllers, hard-wired circuit devices, and the like.

[0200] In addition to what is described herein, various modifications may be made to the disclosure and embodiments of the present invention without departing from its scope. Therefore, the description and examples herein should be interpreted as illustrative rather than restrictive. The scope of the present invention should be measured only by reference to the appended claims.

Claims

1. A memory controller, comprising: a hardware interface to a data bus for coupling to a plurality of data dynamic random access memory (DRAM) devices and at least one error correction code (ECC) DRAM device; as well as An error manager is configured to detect a failure of a data signal (DQ) of one of the plurality of data DRAM devices when coupled to the data bus, dynamically switch an ECC mode on the fly, map out data bits of the DQ, and remap ECC bits of the at least one ECC DRAM device to the mapped data bits of the DQ.

2. The memory controller according to claim 1, wherein: The data bus is used to couple to at least two ECC DRAM devices.

3. The memory controller according to claim 2, wherein: The error manager is used to remap the ECC bits of only one DQ of the ECC device to the mapped out DQ.

4. The memory controller according to claim 2, wherein: The error manager is used to remap ECC bits of a plurality of DQs of the ECC device to the mapped DQs.

5. The memory controller according to claim 2, wherein: The error manager is used to dynamically switch the ECC mode from the 10x4 128-bit ECC mode to the 10x4 96-bit ECC mode.

6. The memory controller according to claim 5, wherein: The error manager is used to subsequently dynamically switch the ECC mode from the 10x4 96-bit ECC mode to the 9x4 64-bit ECC mode.

7. The memory controller according to claim 2, wherein: The error manager is used to dynamically switch the ECC mode from the 10x4 128-bit ECC mode to the 9x4 64-bit ECC mode.

8. The memory controller according to claim 1, wherein: The error manager is used to refresh a memory bank that once contained the mapped data bits of the DQ, wherein the memory controller is used to suspend read requests and write requests to the memory bank until all cache lines of the memory bank are refreshed.

9. The memory controller according to claim 1, wherein: The error manager is configured to refresh a memory bank that contained the mapped-out data bits of the DQ, wherein the error manager includes an address checker having a register configured to track refreshes of cache lines of the memory bank during memory bank idle time.

10. A system for managing memory errors, comprising: a plurality of data dynamic random access memory (DRAM) devices coupled to the data bus; an error correction code (ECC) DRAM device coupled to the data bus; as well as A data signal (DQ) manager, the DQ manager is used to: detect failure of the DQ of one of the multiple data DRAM devices, dynamically switch the ECC mode in real time, map out the data bits of the DQ, and remap the ECC bits of the ECC DRAM device to the mapped data bits of the DQ.

11. The system according to claim 10, wherein: The DQ manager includes circuitry of a memory controller coupled to the data bus.

12. The system according to claim 10, wherein: The DQ manager includes circuitry of a controller disposed on a substrate separate from a memory controller coupled to the data bus.

13. The system according to claim 10, wherein: The ECC DRAM device includes a first ECC DRAM device and also includes a second ECC DRAM device.

14. The system according to claim 13, wherein: The DQ manager is used to remap the ECC bits of only one ECC device to the mapped DQ.

15. The system of claim 13, wherein: The DQ manager is used to remap ECC bits of multiple ECC devices to the mapped DQs.

16. The system of claim 13, wherein: The DQ manager is used to dynamically switch the ECC mode from the 10x4 128-bit ECC mode to the 10x4 96-bit ECC mode.

17. The system of claim 13, wherein: The DQ manager is used to dynamically switch the ECC mode from the first ECC mode to the 9x4 64-bit ECC mode.

18. The system of claim 10, further comprising a memory controller coupled to the data bus, wherein: The DQ manager is used to refresh a memory bank that once contained the mapped data bits of the DQ, wherein the memory controller is used to suspend read requests and write requests to the memory bank until all cache lines of the memory bank are refreshed.

19. The system of claim 10, wherein: The DQ manager is used to refresh a memory bank that contained the mapped-out data bits of the DQ, wherein the DQ manager includes an address checker having a register for tracking refreshes of cache lines of the memory bank during memory bank idle time.

20. The system of claim 10, further comprising one or more of the following: a display communicatively coupled to a central processing unit (CPU); a network interface communicatively coupled to a host processor; or A battery for powering the system.

21. A method for managing memory errors, comprising: detecting a failure of a data signal (DQ) of a data bus coupled to a plurality of data dynamic random access memory (DRAM) devices and at least one error correction code (ECC) DRAM device; Instant dynamic switching of ECC mode; Mapping the data bits of the DQ; as well as The ECC bits of the at least one ECC DRAM device are remapped to the mapped data bits of the DQ.

22. The method according to claim 21, wherein: The data bus has at least two ECC DRAM devices, and wherein remapping the ECC bits includes remapping ECC bits of only one DQ of the ECC device to the mapped out DQ.

23. The method according to claim 21, wherein: The data bus has at least two ECC DRAM devices, and wherein remapping the ECC bits includes remapping ECC bits of a plurality of DQs of the ECC devices to mapped out DQs.

24. The method according to claim 21, wherein: Dynamically switching the ECC mode includes: dynamically switching the ECC mode from the 10x4 128-bit ECC mode to the 10x4 96-bit ECC mode, or dynamically switching the ECC mode from the 10x4 128-bit ECC mode to the 9x4 64-bit ECC mode.

Images