Configuring DRAM refresh for systems with multiple ranks

Abstract

An electronic device includes a memory functional block having a plurality of memory ranks and a memory controller functional block coupled to the memory. The memory controller includes refresh logic that detects two or more memory ranks for which refresh is to be performed during a refresh interval based on buffered memory accesses to each of the memory ranks. The refresh logic determines a refresh order for performing refreshes on the two or more memory ranks during the refresh interval based at least in part on one or more characteristics of the buffered memory accesses to the two or more memory ranks. The memory controller then performs refreshes on the two or more memory ranks during the refresh interval in the refresh order.

Description

Background Art

[0001] Related technologies

[0002] Some electronic devices include dynamic random access memory (DRAM), such as double data rate fourth generation (DDR4) synchronous DRAM. For example, a desktop computer may include multiple gigabytes of DDR4 SDRAM as main memory for storing information (i.e., data, instructions, etc.) for various computing operations. Typically, DRAM is a semiconductor memory that uses integrated circuit elements manufactured on semiconductor integrated circuit chips to store information. For example, some DRAMs use capacitors to store information bits, where the capacitors are charged to store a logic value of 1 and discharged to store a logic value of 0. DRAM is "dynamic" in the sense that the circuit elements used to store information lose charge over time, meaning that if the circuit elements are not recharged periodically, the information stored in the DRAM will eventually be lost. For example, for DRAMs that use capacitors as storage elements, the state of charge of the capacitors is lost due to leakage, so the capacitors must be recharged or "refreshed" in order to maintain the stored information.

[0003] Some electronic devices include memory with a large number of DRAMs, which are typically implemented on multiple separate DRAM integrated circuit chips (or more simply, "DRAM chips") that are combined to form the memory. In these electronic devices, separate DRAM chips in the multiple DRAM chips may be divided into logical groups and / or physical groups for issues such as power control, addressing / memory access, etc. For example, in some electronic devices, several DRAM chips are mounted to a memory module (e.g., a dual in-line memory module or DIMM), and the memory module is mounted in a socket coupled to a communication bus in the electronic device. In such electronic devices, power connections, chip selects, and other power, control, and access signals may be connected to a subset of the DRAM integrated circuit chips on the memory module, thereby logically and / or physically grouping the DRAM chips or portions thereof. For example, in some electronic devices, the DRAM chips on the memory module are included in or may be operated as one of N columns (e.g., 4 columns, 8 columns, etc.), wherein each chip includes M individuals (e.g., 32 individuals), and each individual includes several arrays of DRAM circuit elements for storing information.

[0004] In an electronic device including a plurality of DRAM chips, the above-mentioned refresh operation may be performed for logical and / or physical groups of DRAM chips. For example, some of these electronic devices perform refresh at a column level. In other words, when a refresh operation is performed to recharge the circuit elements in the DRAM chips, these electronic devices refresh one or more DRAM chips in the column as a group. In such electronic devices, in order to refresh the column, the column is first removed or the use of the column is blocked to perform memory access (e.g., read, write, etc.), such as by eliminating the chip availability signal of the DRAM chips in the column. Then the circuit elements in the DRAM chips in the column are refreshed (i.e., rewritten with existing information to refresh the existing information). After the refresh, the DRAM chips of the column are restored for use in performing memory access.

[0005] The process for refreshing circuit elements in the DRAM chips in a column requires hundreds of clock cycles, which can be equivalent to hundreds of nanoseconds, during which time the DRAM chips in the column are unavailable to perform memory accesses. Therefore, due to the refresh operation, entities in the electronic device that rely on the results of the memory access operation, and operations performed thereby, are delayed a significant amount of time before the memory access operation can be completed. Because the DRAM refresh operation causes delays in other operations in the electronic device, improvements in the DRAM refresh operation are beneficial to the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 A block diagram is presented showing an electronic device according to some embodiments.

[0007] Figure 2 A flowchart is presented illustrating a process for selecting a refresh order to be used to perform refreshes on memory ranks according to some embodiments.

[0008] Figure 3 A flow chart is presented illustrating a process for performing refreshes on memory ranks using a refresh sequence according to some embodiments.

[0009] Figure 4 A timeline diagram is presented illustrating performing a refresh operation on a memory rank according to some embodiments.

[0010] Like reference numerals refer to the same figure elements throughout the drawings and the description. DETAILED DESCRIPTION

[0011] The following description is presented to enable any person skilled in the art to make and use the described embodiments, and is provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications. Therefore, the described embodiments are not limited to the embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein.

[0012] the term

[0013] In the following description, various terms are used to describe the embodiments. The following is a simplified and general description of one of these terms. It should be noted that the terms may have important additional aspects, which are not listed herein for clarity and brevity, and thus the description is not intended to limit the terms.

[0014] Functional Block: A functional block refers to a group, set, and / or set of one or more interrelated circuit elements (such as integrated circuit elements, discrete circuit elements, etc.). Circuit elements are "interrelated" because the circuit elements share at least one characteristic. For example, the interrelated circuit elements may be included in, manufactured on, or otherwise coupled to a particular integrated circuit chip or portion thereof, may be involved in the execution of a given function (computational or processing function, memory function, etc.), may be controlled by a common control element, etc. A functional block may include any number of circuit elements, from a single circuit element (e.g., a single integrated circuit logic gate) to millions or billions of circuit elements (e.g., integrated circuit memory), etc.

[0015] Overview

[0016] The described embodiments include an electronic device having a memory functional block and a memory controller functional block. The memory functional block includes multiple columns of dynamic random access memory (DRAM) circuits. The memory circuits are "dynamic" in that the memory circuits lose charge over time and therefore must be refreshed periodically to avoid losing stored information. The memory controller functional block includes refresh logic that determines how and when to perform refresh operations for various memory columns during a refresh interval. For example, the memory controller may determine specific memory columns to refresh and the refresh order in which the memory columns are to be refreshed during a refresh interval based on specified criteria.

[0017] In some embodiments, before the refresh interval, the refresh logic performs an operation for determining a refresh order for the refresh interval. Typically, the refresh order is the order in which the identified memory columns (or more simply, "columns") are to be refreshed during the refresh interval. When determining the refresh order, the refresh logic first identifies the columns to be refreshed during the refresh interval. For example, the refresh logic may identify the columns for which the memory controller currently has more than a specified number of memory accesses for its buffer (where "access" includes memory reads, writes, moves, deletes, etc.). Then, the refresh logic divides the refresh interval into sub-intervals based at least in part on the number of identified columns, each of which will be used to refresh one of the identified columns. For example, if there are five identified columns, the refresh logic may divide the refresh interval into five sub-intervals. Next, based on one or more criteria related to the characteristics of the buffered memory access and / or the interaction between the memory controller and the identified columns, the refresh logic determines the refresh order for refreshing the identified columns. At the end of these operations, the refresh logic has determined both the number of sub-intervals of the refresh interval and the refresh order in which the refreshes are to be performed in the sub-intervals.

[0018] In some embodiments, criteria related to one or more characteristics of the buffered memory accesses used to determine the refresh order include criteria such as: a count of buffered memory accesses for each column (i.e., the total number of memory accesses pending in the memory controller for each column), a type of each of the buffered memory accesses from a set of types (e.g., read, write, move, delete, etc.), a priority of the buffered memory accesses from a set of priorities, etc. For example, in some embodiments, the memory controller maintains a column-by-column record of the number of buffered memory accesses at each of a set of two or more priority levels, and the criteria are the number of buffered memory accesses of a specified priority, the average priority level of the buffered memory accesses for each column, etc. In some embodiments, criteria related to the interaction between the memory controller and the identified columns include criteria such as: the type of commands that have been sent or are to be sent to the columns, the relationship between the columns, etc. For example, the criteria may be the relative timing of the memory controller's previous sending (not sending) of such commands to the identified column and / or other columns. In some embodiments, the memory controller maintains one or more running records (i.e., records that are updated in real time and when events occur) of the values ​​used to determine whether the corresponding criteria are met.

[0019] In some embodiments, after determining the refresh sequence, the refresh logic causes the memory controller to perform a refresh operation in the refresh sequence. For the first column in the refresh sequence, the memory controller sends a refresh in the first subinterval, which means that the buffer memory access to the first column is suspended and the first column is refreshed substantially immediately at the beginning of the refresh interval. For the second and subsequent columns in the refresh sequence, the memory controller performs a buffer memory access to that column while performing a refresh for one or more previous columns in the refresh sequence. In this way, when a subinterval for performing a refresh on each column after the first column occurs, that column is ready for refresh. In other words, the memory controller has (as much as possible) cleared the buffer memory access to the column before refreshing the column. By performing a buffer memory access before performing a refresh as described above, the memory controller can help avoid delays in entities (e.g., processors, input-output devices, etc.) that rely on buffer memory access. In some embodiments, the memory controller also avoids sending a specified command to that column within a specified time for the subinterval for the column to occur, thereby avoiding corresponding delays in refreshing during the subinterval.

[0020] By determining the refresh sequence as described above and performing a buffer memory access before performing a refresh using the refresh sequence, the described embodiments improve the performance of refresh operations. More specifically, the described embodiments can avoid some of the delays experienced in existing systems when refresh operations are performed reactively and without using a refresh sequence, and then sending a suspended buffer storage operation after performing the corresponding refresh operation, resulting in memory access delays. This can help avoid unnecessary delays in executing program code, idle computing elements, and other inefficiencies that occur in existing electronic devices that perform DRAM refreshes. The improved functionality of the electronic device can in turn result in higher user satisfaction.

[0021] Electronic Devices

[0022] Figure 1 A block diagram illustrating an electronic device 100 according to some embodiments is presented. Figure 1 As can be seen in FIG. 1 , the electronic device 100 includes a processing function block 102 and a memory function block 110. Typically, the processing function block 102 and the memory function block 110 are implemented in hardware (i.e., using various circuit elements and devices). For example, the processing function block 102 and the memory function block 110 may be completely manufactured on one or more semiconductor chips (including on one or more separate semiconductor chips for each of the function blocks), may be formed by a semiconductor chip in combination with discrete circuit elements, may be manufactured solely from discrete circuit elements, etc. As described herein, the processing function block 102 and the memory function block 110 perform operations for handling refresh of a dynamic random access memory (DRAM) in the memory function block 110.

[0023] The processing function block 102 is a function block in the electronic device 100 that performs calculations and other operations (eg, control operations, configuration operations, etc.). Figure 1 As can be seen in FIG. 1 , processing block 102 includes processing complexes 104-106. Processing complexes 104-106 are functional blocks that include elements for performing calculations and other operations, such as central processing units (CPUs) or CPU cores, application specific integrated circuits (ASICs), general purpose graphics processing units (GPGPUs), field programmable gate arrays (FPGAs), etc. For example, in some embodiments, one or both of processing complexes 104-106 include multiple CPU cores, caches, and other elements coupled together via high-speed interconnects. Processing block 102 also includes a memory controller 108, which is described in more detail below.

[0024] The memory functional block 110 is a functional block in the electronic device 100 that performs the operation of the memory (e.g., "main" memory) of the electronic device 100. The memory functional block 110 includes: a memory circuit, such as a fourth-generation double data rate synchronous DRAM (DDR4 SDRAM) and / or other types of memory circuits for storing data and instructions for use by functional blocks in the electronic device 100; and a control circuit for handling access to the data and instructions stored in the memory circuit and for performing other control or configuration operations. In the described embodiment, at least some of the memory circuits in the memory functional block 110 are "dynamic" memory circuits. Dynamic memory circuits use circuit elements (such as capacitors) that lose charge over time due to leakage and / or other charge loss mechanisms to store information (e.g., information bits, such as data, instructions, etc.). Because circuit elements lose charge, refresh operations are periodically performed to recharge the circuit elements to avoid losing the stored information. For example, in some embodiments, a given circuit element may need to be refreshed every N microseconds to ensure that the information stored in the circuit element is not lost due to leakage and / or other charge loss mechanisms.

[0025] As in Figure 1 As can be seen in FIG. 1 , the memory functional block 110 includes memory modules 112-118, which are modules including several memory integrated circuit chips (MEM CHIP) 120 (e.g., DDR SDRAM integrated circuit chips) mounted on both sides of a circuit board, such as a dual in-line memory module (DIMM) ( Figure 1 In the depicted embodiment, each memory chip 120 includes a plurality of memory cells, such as cell 122 (for clarity, Figure 1Only one of the memory chips 120 is marked in FIG. 1 , and each of the memory chips 120 includes several memory circuit arrays (not shown). Figure 1 On the side of the memory modules 112-118 shown, the memory chips 120 on the memory modules 112-118 are logically and / or physically divided into eight columns (for clarity, Figure 1 In some embodiments, control signals, power signals, address signals, chip selects, and / or other signals are coupled to each of the columns, respectively, so that the columns can be addressed / accessed, controlled, powered up and down, and / or interacted with, respectively. The described embodiments perform refreshes on the memory circuits in the memory functional block 110 at the resolution of the columns. In other words, when refreshing the DRAM memory circuits, the described embodiments select the columns and perform refreshes on the memory circuits of that column.

[0026] In some embodiments, some or all of the memory chips 120 include at least two separate DRAM memory chips arranged in a stack. In other words, two or more separate memory chips, each including a group, are physically arranged in a stack, which may be referred to as a "3D stack". In these embodiments, the memory chips include communication mechanisms such as through silicon vias for communicating with the memory chips in the stack and between the memory chips, and at least one chip in the stack includes interface circuits for communicating with the memory controller 108 and other entities outside the stack. In some embodiments, the above-mentioned columns are or include at least a portion of the chips in a single memory chip stack.

[0027] The memory controller 108 is a functional block that manages, controls, and otherwise handles the interactions between the processing functional block 102 and the memory functional block 110. For example, the memory controller 108 performs memory accesses (i.e., reads, writes, deletes, moves, etc.) on behalf of the processing functional block 102, performs configuration and control operations on the memory functional block 110, and / or other operations. In some embodiments, the memory controller 108 schedules, causes, and / or controls the execution of refreshes of DRAM memory circuits in the memory functional block 110. In some embodiments, the memory controller 108 communicates with respective individual memory modules in the memory modules 112-118 via channels such as channels 128-130, each of which includes a bus, wires, leads, and / or other communication elements and circuitry.

[0028] In some embodiments, memory accesses are buffered by the memory controller 108. In other words, upon receiving a request, command, or other indication from an entity such as the processing complex 104 that a memory access is to be performed, the memory controller 108 determines that the memory access (i.e., the request, command, etc.—or information based thereon) is to be held in a buffer, queue, or another storage element in the memory controller 108, rather than being sent immediately to the memory functional block 110 for resolution. For example, the memory controller 108 may buffer memory accesses to enable memory access (re)ordering to avoid conflicts with other operations being performed by the rank being accessed, to avoid interference with other traffic on the corresponding channel, etc. In some of these embodiments, the memory controller 108 is limited, such as by an architecture specification or other system rules, to buffer only a predetermined number (e.g., 7, 8, etc.) of memory accesses per rank at a given time, and / or to only hold buffered memory accesses for a specified amount of time before sending the memory accesses to the memory functional block 110.

[0029] The memory controller 108 includes refresh logic (RELO) 132. The refresh logic 132 is a functional block in the memory controller 108 that performs operations for determining a refresh order in which memory ranks are to be refreshed during a refresh interval. As described herein, the refresh logic 132 determines the refresh order based on characteristics of buffered memory accesses and / or interactions between the memory controller 108 and the memory ranks, and then causes the memory controller to send a corresponding refresh to each rank to be refreshed based on the refresh order.

[0030] The electronic device 100 is simplified for illustrative purposes. However, in some embodiments, the electronic device 100 includes additional or different functional blocks, subsystems, elements and / or communication paths. For example, the electronic device 100 may include a display subsystem, a power subsystem, an input-output (I / O) subsystem, etc. The electronic device 100 generally includes functional blocks sufficient to perform the operations described herein, etc.

[0031] The electronic device 100 may be any device that performs computing operations or may be included therein. For example, the electronic device 100 may be a desktop computer, a laptop computer, a wearable computing device, a tablet computer, a piece of virtual or augmented reality equipment, a smart phone, an artificial intelligence (AI) or machine learning device, a server, a network appliance, a toy, a piece of audio-visual equipment, a home appliance, a vehicle, etc. and / or a combination thereof or may be included therein.

[0032] Refresh Operation

[0033] In the described embodiments, the dynamic memory circuits in the memory functional blocks are refreshed during a "refresh operation" to avoid losing the stored information. In some embodiments, the refresh operation is performed at the resolution of the memory column, so that all memory circuits in the memory column are refreshed during a single refresh operation. In some embodiments, during the refresh operation, the memory column to be refreshed is first removed or its effective use is prevented, so that memory access and other operations involving information stored in the memory circuits of the memory column are suspended, terminated or otherwise delayed for the duration of the refresh operation. For example, in some embodiments, the memory controller stores the pending memory access operation in a buffer, queue or other memory circuit associated with the column until the refresh operation is completed. The memory column is then refreshed. For example, in some embodiments, for a group of multiple rows of storage elements in the column, during the refresh operation, until each row has been refreshed, the memory controller stores the value of a group of bits from the next row of memory circuits in the row buffer, and then precharges the memory circuits of that row of memory circuits, and then writes the stored values ​​from the self buffer back to that row of memory circuits. Once the refresh operation is complete, ie, the memory circuits in the rank are all refreshed, the memory rank is released back to an active use state so that memory accesses and other operations involving information stored in the memory circuits (including buffered memory accesses) are performed normally.

[0034] Perform refresh on multiple memory ranks using refresh sequence

[0035] In the described embodiment, refresh logic in a memory controller determines an order in which to refresh dynamic memory circuits in memory ranks from among a plurality of memory ranks. Figure 2 A flow chart illustrating a process for selecting a refresh order to be used to perform refresh on a memory column according to some embodiments is presented. It should be noted that Figure 2 The operations shown in are presented as general examples of operations performed by some embodiments. The operations performed by other embodiments include different operations and / or operations performed in a different order. Figure 2 In the example of FIG. 1 , an electronic device having a similar internal arrangement to the electronic device 100 is described as performing various operations. However, in some embodiments, electronic devices with different arrangements perform the described operations. For example, there may be different numbers of columns, memory modules, etc. in the electronic device.

[0036] Figure 2The illustrated operation begins when refresh logic (e.g., refresh logic 132) in a memory controller (e.g., memory controller 108) detects a memory rank for which a refresh is to be performed (step 200). For this operation, the refresh logic performs one or more checks to determine that a refresh is to be performed during the next refresh interval based on information about, associated with, or otherwise related to the memory rank. For example, in some embodiments, the refresh logic detects a memory rank that is at or near a specified state, such as a rank for which the memory controller has buffered (and possibly intentionally buffered / delayed) a sufficient number of memory accesses, at least some of which should be sent during the next refresh interval to avoid violating corresponding memory access rules. In this case, the limit on the allowable number of buffered memory accesses may be determined by an architectural specification, a rule in the electronic device, and / or by a timing constraint—because a buffered memory access may need to be sent before it expires, before a requesting device times out, etc.

[0037] Then, the refresh logic divides the refresh interval into a number of sub-intervals based at least in part on the number of memory columns on which refresh is to be performed (step 202). During this operation, in some embodiments, the refresh logic divides the refresh interval into a number of sub-intervals, the number of sub-intervals being equal to the number of memory columns on which refresh is to be performed. For example, if five of eight columns are to be refreshed during the refresh interval, the refresh logic may divide the refresh interval into five equal sub-intervals, such that a refresh interval of N microseconds (e.g., 4 microseconds, 2 microseconds, etc.) is divided into sub-intervals of N / 5 microseconds. In some embodiments, in addition to the number of memory columns on which refresh is to be performed, the refresh logic also uses one or more other inputs or factors to determine the number, duration, and / or arrangement of sub-intervals. For example, the current state of each memory column (e.g., the number of buffer memory accesses, etc.) may be used to divide the refresh interval into sub-intervals of correspondingly different time lengths.

[0038] In some embodiments, a minimum and / or maximum number of subintervals are used. For example, one may choose to specify a minimum number of memory columns as memory columns on which refresh is to be performed, regardless of whether at least the minimum number of memory columns has been determined to require refresh in step 200. In this case, steps 200-202 will involve at least the minimum number of columns—and therefore at least the minimum number of subintervals. In these embodiments, when it is determined that less than the minimum number of columns require refresh, a default or initial refresh order as described in more detail below may be used to determine at least some of the memory columns to be refreshed. As another example, one may choose to specify a maximum number of columns (e.g., as limited by the time it takes to refresh each column within a refresh interval) as memory columns on which refresh is to be performed—and steps 200-202 will involve at most the maximum number of columns and therefore at most the maximum number of subintervals.

[0039] Next, the refresh logic determines a refresh order for performing refreshes on the memory ranks based on one or more characteristics of the buffer memory accesses of the memory ranks and / or the operation of the memory controller (step 204). Typically, for this operation, the refresh logic determines (i.e., generates, selects, etc.) a refresh order based on the current status of the buffer memory accesses, the ranks, and / or the memory controller. The determined refresh order causes refresh operations for the ranks to be ordered in a manner that helps reduce the impact on the overall performance of the electronic device performing the refresh, given the buffer memory accesses and other current status. In other words, the refresh order is determined for refreshes that are performed in a manner that reduces the latency of performing the buffer memory accesses (which may otherwise prevent the performance of the buffer memory accesses, particularly in existing electronic devices). This means that entities that rely on the buffer memory accesses (e.g., processing complexes 104-106, etc.) are less likely to be idle, in whole or in part, waiting for the completion of the buffer memory accesses that were delayed due to the refresh.

[0040] In some embodiments, the characteristic of the buffered memory accesses used to determine the refresh order is the number of buffered memory accesses for each memory rank to be refreshed. For example, in some embodiments, memory ranks with more than N buffered memory accesses (e.g., 5, 7, etc.) are to be refreshed, and the refresh order is selected based on the actual number of buffered memory accesses for each memory rank. For example, assuming N=5, a maximum of 7 buffered memory accesses can be maintained for any memory rank according to the architecture specification, and four memory ranks have the following number of buffered memory accesses: Rank 1 = 5; Rank 2 = 5; Rank 3 = 6; Rank 4 = 7; Rank 5 = 8; Rank 6 = 9; Rank 7 = 10; Rank 8 = 11; Rank 9 = 12; Rank 10 = 13; Rank 11 = 14; Rank 12 = 15; Rank 13 = 16; Rank 14 = 17; Rank 15 = 18; Rank 16 = 19; Rank 17 = 20 0 =5, columns 1 =7, columns 2 =6, and the columns 3 = 5. In this case, the possible refresh order is: 0 ,List 3 ,List2 and column 1 , so that the column 0 First in the refresh order, column 3 Second in the refresh order, and so on.

[0041] In some embodiments, the characteristic of the buffered memory accesses used to determine the refresh order is the priority of the memory accesses buffered for each memory column to be refreshed. For example, in some embodiments, the memory controller assigns corresponding priorities from a set of two or more priorities to memory accesses from various entities in the electronic device. For example, in these embodiments, memory accesses from the I / O subsystem may be assigned a lower priority than memory accesses from the processing complex, memory accesses from specific software entities (e.g., browsers, word processors, etc.) may be assigned a lower priority than memory accesses from other software entities (e.g., operating systems, firmware, etc.), and so on. In these embodiments, the refresh order is selected based on: a count of buffered memory accesses with a specified priority, an average or sum of the priorities of the buffered memory accesses, and / or another characteristic of the priorities of the buffered memory accesses. For example, assume that four memory columns are to be refreshed, the priorities include at least a lowest priority and a highest priority, and the four memory columns have the following highest priority buffered memory accesses: Column 0 =0, column 1 =3, columns 2 =1, and the column 3 = 1. In this case, the possible refresh order is: 1 ,List 2 ,List 3 and column 0 , so that the column 1 First in the refresh order, column 2 Second in the refresh order, and so on.

[0042] In some embodiments, the characteristic of the buffered memory accesses used to determine the refresh order is the type of memory accesses buffered for each memory column to be refreshed. For example, in some embodiments, the type of memory accesses includes memory reads, memory writes, memory deletes, memory moves, etc. In these embodiments, the refresh order is selected based on: a count of buffered memory accesses of a specified type, an average or sum of buffered memory accesses of one or more types, and / or another characteristic of the type of buffered memory accesses. For example, assume that five memory columns are to be refreshed, the types include at least a write type and a read type, and the five memory columns have buffered memory accesses of the following write type: Column 0 =5, columns 1 =2, columns 2 =6, columns3 = 2, and the columns 4 = 3. In this case, the possible refresh order is: column 2 , column 0 , column 4 , column 1 and column 3 , such that column 2 is first in the refresh order, column 0 is second in the refresh order, and so on.

[0043] In some embodiments, the operation of the memory controller for determining the refresh order is a command that the memory controller previously sent to the memory column. For example, in some embodiments, when the memory controller sends a command (e.g., a page open command, etc.) to the memory column, the memory column performs several operations and then waits for a delay period (e.g., N microseconds) before the memory column processes subsequent operations and thus processes the command. In some embodiments, the refresh order is selected based on whether a command is sent to the memory column during the delay period. For example, assume that three memory columns are to be refreshed: column 0 , column 1 and column 2 , and a command from the specified commands was sent to column 0 before M nanoseconds, where M << N. In this case, the possible refresh order is: column 1 , column 2 , column 3 and column 0 , such that column 1 is first in the refresh order, column 2 is second in the refresh order, and so on.

[0044] In some embodiments, a combination of two or more of the above-described characteristics of the buffered memory access of the memory column and / or the operation of the memory controller is used to determine the refresh order. For example, in some embodiments, a combined value or average of two or more characteristics (e.g., the count of buffered memory accesses and the priority of buffered memory accesses for each column) is used to determine the refresh order. In some embodiments, mathematical, logical, and / or other functions are used to determine the combined value.

[0045] In some embodiments, refresh logic or another entity maintains a running record of one or more characteristics of buffered memory accesses of memory ranks and / or the operation of the memory controller used to determine the refresh order. For example, assuming an embodiment in which the characteristics of the buffered memory accesses include the number of memory accesses buffered for each memory rank, the refresh logic, memory controller, and / or another entity may maintain a counter of buffered memory accesses for each memory rank, the counter incrementing when the memory access is buffered (e.g., stored in a buffer or queue in the memory controller) and decrementing when the memory access is debuffered, executed, or otherwise handled. As another example, assuming an embodiment in which the operation of the memory controller is used to determine the refresh order, the refresh logic, memory controller, and / or another entity may maintain a timer that is initiated when one or more specified commands are sent to each memory rank, stores a timestamp when one or more specified commands are sent to each memory rank, and so on. In some embodiments, refresh logic or another entity generates or otherwise determines a record of one or more characteristics of buffered memory accesses of memory ranks and / or the operation of the memory controller used to determine the refresh order as needed. For example, flush logic or other entity may use a tail pointer to calculate a count of buffered memory accesses, etc., as needed for the buffer in which the memory accesses are stored.

[0046] In some embodiments, a static / unchanging default or initial refresh order may be used for some or all refresh intervals. For example, a default refresh order may be used when none of the columns are operating in or near the above-specified state (where at least some of the buffered memory accesses should be sent during the next refresh interval to avoid violating the corresponding memory access rules). In some embodiments, a default or initial refresh order is provided by an external entity, such as in a configuration file or firmware. In some embodiments, the default or initial refresh order is used in the event that the columns are not in a state where at least some of the buffered memory accesses should be sent during the next refresh interval to avoid violating the corresponding memory access rules. Figure 2 The refresh order determined in is a modification of the default or initial refresh order.

[0047] In some embodiments, one or more rules, criteria, thresholds, functions, etc. are used to determine the refresh order based on the characteristics of the buffered memory accesses of the above-mentioned memory ranks and / or the operation of the memory controller. In some of these embodiments, the one or more rules, criteria, thresholds, functions, etc. can be received from an external source such as a software entity, a configuration file, firmware, etc.

[0048] Then, the refresh logic causes the memory controller to perform a refresh on the memory ranks in the refresh sequence (step 206). For this operation, the refresh logic communicates the refresh sequence to the memory controller. Then, the memory controller uses the refresh sequence to perform a refresh during the corresponding refresh interval. Figure 3A flow chart illustrating a process for performing a refresh on a memory column using a refresh sequence according to some embodiments is presented. Note that Figure 3 The operations shown are presented as general examples of operations performed by some embodiments. Operations performed by other embodiments include different operations and / or operations performed in a different order. Figure 3 In the example of FIG. 1 , an electronic device having a similar internal arrangement to electronic device 100 is described as performing various operations. However, in some embodiments, an electronic device having a different arrangement performs the described operations.

[0049] Figure 3 The operations shown begin when the memory controller pauses buffered memory accesses for the first memory column in a refresh sequence and performs a refresh in a first subinterval of a set of subintervals in a refresh interval (step 300). Generally, and as described in more detail herein, one of the benefits of the described embodiments is that the memory columns can be prepared for a pending refresh by sending the corresponding buffered memory accesses prior to the subinterval in which the refresh is performed. However, in some embodiments, for the first memory column in the refresh sequence, there is no time in the refresh interval to send the corresponding buffered memory accesses. Instead, the buffered memory accesses are "paused" by holding the buffered memory accesses in a buffer in the memory controller, and the buffered memory accesses are not performed until after the refresh is complete. The refresh is performed "substantially immediately" in that the refresh of the first memory column in the refresh sequence is performed at the beginning of the refresh interval (i.e., at the beginning of the first subinterval of the refresh interval) or shortly after the beginning of the refresh interval. In some embodiments, the refresh can be performed in this manner and with such timing. Figure 2 Operation of: causing the refresh sequence to be generated / prepared just in time for the refresh of the memory ranks to be started in the refresh sequence during the refresh interval. Thus, once the refresh sequence is prepared, the refresh may be performed on the first memory rank in the first subinterval—that is, the refresh may be performed “substantially immediately”.

[0050] Figure 4 A timeline diagram is presented showing a refresh operation performed on a memory column according to some embodiments. More specifically, Figure 4 Presented as Figure 3 FIG. 1 is a diagram of the refresh performed based on the refresh order shown. It should be noted that Figure 4 An example of a refresh operation of one embodiment is presented, but other embodiments operate in different ways, such as refreshing a different number of memory ranks and / or refreshing memory ranks in a different order, performing buffer memory accesses at different times, etc. As shown in Figure 4 As can be seen in FIG. 1 , the refresh order is as follows: (1) memory rank (RANK) 400, (2) memory rank 402, (3) memory rank 406, and (4) memory rank 404. Figure 3 As described in step 300 of , a refresh operation is performed substantially immediately on memory rank (RANK) 400 (which is the first memory rank in the refresh sequence) at the beginning of a refresh interval (i.e., in sub-interval (SUB-INT) 408). Execution of a buffered memory access of memory rank 400 does not occur until after sub-interval 408 in the refresh interval, i.e., during (or possibly after) sub-interval 410.

[0051] return Figure 3 , for the second and subsequent memory ranks in the refresh sequence, the memory controller performs corresponding buffered memory accesses while performing a refresh on the previous memory rank (step 302). During this operation, the memory controller performs as many buffered memory accesses for a given memory rank as can be performed before initiating a refresh on the given memory rank. In other words, buffered memory reads, writes, etc. are sent to the memory rank for execution therein, thereby completing the memory accesses. In some cases, most, if not all, of the buffered memory accesses to a given memory rank are executed before initiating a refresh on the given memory rank during the corresponding subinterval. Return Figure 4 And as mentioned above Figure 3 As depicted in step 302 of , a buffer memory access is performed on column 402 during subinterval 408, a buffer memory access is performed on column 404 during subintervals 410-412, and a buffer memory access is performed on column 406 during subintervals 408-410.

[0052] Through Figure 3 to Figure 4 As shown, buffered memory accesses are performed before the refresh begins, and the described embodiments can avoid delaying the operation of other entities in the electronic device (e.g., processing complexes 104-106, etc.) that depend on the results of the buffered memory accesses. However, it should be noted that other entities in the electronic device can generate new memory accesses that will be buffered by the memory controller during the refresh of a given memory rank, including memory accesses generated before a subinterval but within a threshold time of the subinterval. Memory accesses generated in this manner / at these times are suspended during the refresh.

[0053] return Figure 3 , for the second and subsequent memory ranks, the memory controller performs refresh in corresponding subintervals in a set of subintervals in the refresh interval (step 304). Figure 4 As can be seen in the above Figure 3As described in step 304, a refresh operation is performed on memory column 402 (the second memory column in the refresh sequence) in sub-interval 410, a refresh operation is performed on memory column 406 (the third memory column in the refresh sequence) in sub-interval 412, and a refresh operation is performed on memory column 412 (the fourth memory column in the refresh sequence) in sub-interval 414.

[0054] The described embodiments can improve the performance of an electronic device (compared to existing electronic devices) when multiple ranks are operating at or near full capacity in terms of processing memory accesses. This is true because refreshing such as Figures 2 to 4 The arrangement shown is such that buffer memory stores are performed as far as possible before a corresponding refresh is performed. This is in contrast to existing electronic devices which are reactive to the columns reaching a state where a buffer memory access must be sent, and lack organization and efficiency in the method of refreshing.

[0055] Refresh by volume

[0056] The embodiments described herein and Figures 2 to 4 In the examples in , the refresh operation is performed at the column level (i.e., at column resolution). However, the specific number of bodies and DRAM chips in each column varies depending on the configuration of the electronic device, the DRAM memory technology or architecture used, the arrangement of the memory modules, etc. For example, in some embodiments, the column includes only one DRAM chip, including one or more bodies, etc. Therefore, when a "column" is refreshed, one or more bodies within the column may be refreshed. In some embodiments, the number and arrangement of bodies refreshed during the refresh operation include a portion of the bodies in the column - that is, only some of the bodies in the column are refreshed during a given refresh interval (or sub-interval). For example, in some embodiments, one or more bodies within a multi-body column may be refreshed separately. In these embodiments, the operations described for refreshing a column may be adapted to apply to one or more bodies within a column. For example, the refresh interval and / or refresh order may be determined at the body level and / or at the column and body level, and the refresh may be performed accordingly.

[0057] In some embodiments, the electronic device (e.g., electronic device 100 and / or a portion thereof) uses code and / or data stored on a non-transitory computer-readable storage medium to perform some or all of the operations described herein. More specifically, when performing the described operations, the electronic device reads code and / or data from a computer-readable storage medium and executes the code and / or uses the data. A computer-readable storage medium may be any device, medium, or combination thereof that stores code and / or data for use by an electronic device. For example, a computer-readable storage medium may include, but is not limited to, volatile memory or non-volatile memory, including flash memory, random access memory (eDRAM, RAM, SRAM, DRAM, DDR, DDR2 / DDR3 / DDR4 SDRAM, etc.), read-only memory (ROM), and / or magnetic or optical storage media (e.g., disk drives, tapes, CDs, DVDs).

[0058] In some embodiments, one or more hardware modules are configured to perform the operations described herein. For example, a hardware module may include, but is not limited to, one or more processors / cores / central processing units (CPUs), application specific integrated circuits (ASIC) chips, field programmable gate arrays (FPGAs), computing units, embedded processors, graphics processors (GPUs) / graphics cores, pipelines, accelerated processing units (APUs), system management units, power controllers, and / or other programmable logic devices. When such hardware modules are activated, the hardware modules perform some or all of the operations. In some embodiments, a hardware module includes one or more general-purpose circuits that are configured to perform operations by executing instructions (program code, firmware, etc.).

[0059] In some embodiments, data structures representing some or all of the structures and mechanisms described herein (e.g., processing function block 102, memory function block 110, and / or some portion thereof) are stored on a non-transitory computer-readable storage medium, which includes a database or other data structure that can be read by an electronic device and used directly or indirectly to manufacture hardware including the structures and mechanisms. For example, the data structure can be a behavioral level description or a register transfer level (RTL) description of the hardware functionality in a high-level design language (HDL) such as Verilog or VHDL. The description can be read by a synthesis tool, which can synthesize the description to generate a netlist including a list of gate / circuit elements from a synthesis library representing the functionality of the hardware including the above-mentioned structures and mechanisms. The netlist can then be placed and routed to generate a data set describing the geometry to be applied to the mask. The mask can then be used in various semiconductor manufacturing steps to produce one or more semiconductor circuits (e.g., integrated circuits) corresponding to the above-mentioned structures and mechanisms. Alternatively, the database on the computer accessible storage medium may be a netlist (with or without a synthesis library) or a data set (as desired), or Graphic Data System (GDS) II data.

[0060] In this specification, variables or unspecified values ​​(i.e., general descriptions of values ​​without specific examples of values) are represented by letters such as N. As used herein, although similar letters may be used in different places in this specification, variables and unspecified values ​​are not necessarily the same in each case, and there may be different amounts and values ​​of variables intended for some or all of the general variables and unspecified values. In other words, in this specification, N and any other letters used to represent variables and unspecified values ​​are not necessarily related to each other.

[0061] As used herein, the expression "et cetera / etc." is intended to present one and / or circumstances, i.e., equivalents of "at least one" of the elements in the list associated with et cetera. For example, in the sentence "the system performs a first operation, a second operation, etc.", the system performs at least one of the first operation, the second operation, and the other operation. Furthermore, the elements in the list associated with et cetera are merely examples of a set of examples—and at least some of the examples may not appear in some embodiments.

[0062] The foregoing description of the embodiments is presented for the purpose of illustration and description only. The foregoing description is not intended to be exhaustive or to limit the embodiments to the disclosed forms. Therefore, many modifications and variations will be apparent to those skilled in the art. In addition, the above disclosure is not intended to limit the embodiments. The scope of the embodiments is defined by the appended claims.

Claims

1. An electronic device, comprising: a memory, the memory comprising a plurality of memory columns; as well as a memory controller coupled to the memory, the memory controller comprising refresh logic configured to: detecting two or more memory ranks for which refresh is to be performed during a refresh interval based on buffered memory accesses to each of the plurality of memory ranks; determining a refresh order for performing refreshes on the two or more memory ranks during the refresh interval based at least in part on one or more characteristics of buffered memory accesses to the two or more memory ranks, wherein each memory rank is included in a respective position in the refresh order relative to other memory ranks from the two or more memory ranks based at least in part on the one or more characteristics of the buffered memory accesses; and Refreshing is performed on the two or more memory ranks during the refresh interval in the refresh order.

2. The electronic device of claim 1 , wherein when the refresh is performed on the two or more memory ranks during the refresh interval in the refresh order, the refresh logic is configured to: For a given memory rank that is first in the refresh order among the two or more memory ranks, corresponding buffer memory access is suspended and the refresh is performed on the given memory rank after the refresh interval starts.

3. The electronic device of claim 1 , wherein when refreshing is performed on the two or more memory ranks during the refresh interval in the refresh order, the refresh logic is configured to: A corresponding buffer memory access is performed on a given memory rank of the plurality of memory ranks while a refresh is performed on a previous memory rank of the plurality of memory ranks in the refresh sequence, thereby preparing the given memory rank for a subsequent refresh.

4. The electronic device of claim 3, wherein the refresh logic is further configured to: While performing the refresh on the preceding memory rank of the plurality of memory ranks in the refresh sequence, sending of a specified memory command to the given memory rank is suspended, thereby preparing the given memory rank for the subsequent refresh.

5. The electronic device of claim 1 , wherein the refresh logic is further configured to: The refresh interval is divided into a number of sub-intervals based at least in part on a number of the plurality of memory ranks to be refreshed during the refresh interval, wherein a refresh is to be performed on a different one of the two or more memory ranks during each sub-interval.

6. The electronic device as claimed in claim 1, wherein: The one or more characteristics of the buffer memory accesses include a count of the buffer memory accesses; and Memory ranks with more buffered memory accesses are included in the refresh order later than memory ranks with fewer buffered memory accesses.

7. The electronic device as claimed in claim 1, wherein: The one or more characteristics of the buffered memory accesses include a type of each of the buffered memory accesses; and Memory ranks having more buffered memory accesses of a given type are included in the refresh order earlier than memory ranks having fewer buffered memory accesses of the given type.

8. The electronic device of claim 7, wherein the type of each of the buffered memory accesses comprises one of a memory read or a memory write.

9. The electronic device as claimed in claim 1, wherein: The one or more characteristics of the buffered memory access include a priority of the buffered memory access; and Memory ranks having more higher priority buffer memory accesses are included in the refresh order earlier than memory ranks having fewer higher priority buffer memory accesses.

10. The electronic device of claim 1, wherein the refresh logic is further configured to: A refresh order for performing a refresh on at least one of the plurality of memory ranks during the refresh interval is determined based on operations performed or to be performed by the memory controller in addition to the one or more characteristics of the buffered memory accesses.

11. The electronic device of claim 1, wherein a refresh is to be performed on a given memory rank when a specified number of memory accesses are buffered in the given memory rank.

12. The electronic device of claim 1, wherein: Each rank includes one or more memory banks; and The detecting, determining, and executing operations are performed at a resolution of memory banks within the plurality of memory ranks such that only a portion of one or more memory banks in a given memory rank are refreshed during the refresh interval.

13. A method for operating an electronic device having a memory including a plurality of memory ranks and a memory controller including refresh logic, the method comprising: detecting, by the refresh logic, two or more memory ranks for which refresh is to be performed during a refresh interval based on buffer memory accesses to each of the plurality of memory ranks; determining, by the refresh logic, a refresh order for performing refreshes on the two or more memory ranks during the refresh interval based at least in part on one or more characteristics of buffer memory accesses to the two or more memory ranks, wherein each memory rank is included in a respective position in the refresh order relative to other memory ranks from the two or more memory ranks based at least in part on the one or more characteristics of the buffer memory accesses; as well as Refreshing the two or more memory ranks is performed by the refresh logic during the refresh interval in the refresh order.

14. The method of claim 13, wherein performing the refresh on the two or more memory ranks during the refresh interval in the refresh order comprises: For a given memory rank that is first in the refresh order among the two or more memory ranks, corresponding buffer memory access is suspended and the refresh is performed on the given memory rank after the refresh interval starts.

15. The method of claim 13, wherein performing refresh on the two or more memory ranks during the refresh interval in the refresh order comprises: A corresponding buffer memory access is performed on a given memory rank of the plurality of memory ranks while a refresh is performed on a previous memory rank of the plurality of memory ranks in the refresh sequence, thereby preparing the given memory rank for a subsequent refresh.

16. The method of claim 15, further comprising: While performing the refresh on the preceding memory rank of the plurality of memory ranks in the refresh sequence, sending of a specified memory command to the given memory rank is suspended, thereby preparing the given memory rank for the subsequent refresh.

17. The method of claim 13, further comprising: The refresh interval is divided into a number of sub-intervals based at least in part on a number of the plurality of memory ranks to be refreshed during the refresh interval, wherein a refresh is to be performed on one of the two or more memory ranks during each sub-interval.

18. The method of claim 13, wherein: The one or more characteristics of the buffer memory accesses include a count of the buffer memory accesses; and Memory ranks with more buffered memory accesses are included in the refresh order later than memory ranks with less buffered memory accesses.

19. The method of claim 13, wherein: The one or more characteristics of the buffered memory accesses include a type of each of the buffered memory accesses; and Memory ranks having more buffered memory accesses of a given type are included in the refresh order earlier than memory ranks having fewer buffered memory accesses of the given type.

20. The method of claim 13, wherein: The one or more characteristics of the buffered memory access include a priority of the buffered memory access; and Memory ranks having more higher priority buffered memory accesses are included in the refresh order earlier than memory ranks having less higher priority buffered memory accesses.

21. The method of claim 13, wherein the refresh logic is further configured to: A refresh order for performing a refresh on at least one of the plurality of memory ranks during the refresh interval is determined based on operations performed or to be performed by the memory controller in addition to the one or more characteristics of the buffered memory accesses.

Images