Non-deterministic memory protocols

Abstract

The present invention includes apparatus and methods related to non-deterministic memory protocols. An example apparatus may perform operations on a memory device based on commands received from a host according to a protocol, wherein the protocol includes non-deterministic timing of the operations. The memory device may be a non-volatile dual in-line memory module (NVDIMM) device.

Description

[0001] Divisional Application Information

[0002] This invention patent application is a divisional application of the invention patent application with application date of April 27, 2017, application number 201780026941.4 and invention name “Non-deterministic Memory Protocol”. Technical Field

[0003] The present invention relates generally to memory devices, and more particularly, to apparatus and methods for non-deterministic memory protocols. Background Art

[0004] Memory devices are typically provided as internal semiconductor integrated circuits in computers or other electronic devices. There are many different types of memory including volatile and non-volatile memory. Volatile memory may require power to maintain its data, and includes random access memory (RAM), dynamic random access memory (DRAM), and synchronous dynamic random access memory (SDRAM), among others. Non-volatile memory may provide persistent data by retaining stored data when not powered, and may include NAND flash memory, NOR flash memory, read-only memory (ROM), electrically erasable programmable ROM (EEPROM), erasable programmable ROM (EPROM), and resistance variable memory, such as phase change random access memory (PCRAM), resistive random access memory (RRAM), and magnetoresistive random access memory (MRAM), among others.

[0005] Memory is also used in a wide range of electronic applications as both volatile and nonvolatile data storage devices. For example, nonvolatile memory can be used in personal computers, portable memory sticks, digital cameras, cellular phones, portable music players such as MP3 players, movie players, and other electronic devices. Memory cells can be arranged in an array, where the array is used in a memory device.

[0006] The memory may be part of a memory module, such as a dual in-line memory module (DIMM), used in a computing device. The memory module may include volatile memory, such as DRAM, and / or nonvolatile memory, such as flash memory or RRAM. A DIMM may use primary memory in a computing system. Summary of the invention

[0007] According to one aspect of the present application, a method is provided, comprising: receiving at least one read command (244) and at least one write command (362) from a host (102) via a command / address bus of a non-volatile dual in-line memory module (NVDIMM) (110-1); receiving data to be written to the NVDIMM from the host via a data bus of the NVDIMM; and transmitting a write credit signal (332) from a set of pins of the NVDIMM designated for write credit signals to the host based at least in part on writing the data received from the host.

[0008] According to another aspect of the present application, a device is provided, comprising: a command / address bus configured to transmit a read command and a write command from a host (102) to a volatile memory or a non-volatile memory, or to both the volatile memory and the non-volatile memory; a data bus configured to transmit a read command (244) or a write command between the host and the volatile memory or the non-volatile memory (110-1, ..., 110-Y), or between the host and the volatile memory and the non-volatile memory; The present invention relates to a device for communicating data associated with a command (362); and a set of pins, which is independent of the command / address bus, the set of pins being coupled to the volatile memory or the non-volatile memory, or to both the volatile memory and the non-volatile memory, the set of pins being configured to transmit a write credit signal to the host to provide write credit feedback based at least in part on data from the host that is written to the volatile memory or the non-volatile memory, or to both the volatile memory and the non-volatile memory.

[0009] According to another aspect of the present application, a device is provided, comprising: a volatile memory configured as an element (110-1, ..., 110-Y) of a dual in-line memory module (DIMM); a non-volatile memory configured as an element (110-1, ..., 110-Y) of the DIMM; a first bus coupled to the volatile memory or the non-volatile memory, or coupled to both the volatile memory and the non-volatile memory, and the first bus is configured to receive a command or an address, or both a command and an address, from a host (102); a second bus coupled to the volatile memory or the non-volatile memory, or coupled to the volatile memory and the non-volatile memory The DIMM is configured to transmit write credit feedback from the DIMM to the host via a write credit signal based at least in part on data from the host written to the volatile memory or the nonvolatile memory, or based on data from the host written to both the volatile memory and the nonvolatile memory.

[0010] According to yet another aspect of the present invention, a method is provided, comprising: transmitting at least one read command (244) and at least one write command (362) from a host (102) to a non-volatile dual in-line memory module (NVDIMM) via a command / address bus; transmitting data to be written to the NVDIMM from the host to the NVDIMM via a data bus; and receiving write credit feedback at the host from a set of pins of the NVDIMM designated for write credit signals based at least in part on transmitting the data to be written to the NVDIMM. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram of a device in the form of a computing system including a memory system according to several embodiments of the invention.

[0012] Figure 2A to 2B is a diagram of several read operations according to several embodiments of the present invention.

[0013] Figure 3 is a diagram of several write operations according to several embodiments of the present invention.

[0014] Figure 4 is a diagram of several read and write operations according to several embodiments of the present invention. DETAILED DESCRIPTION

[0015] The present invention includes apparatus and methods related to non-deterministic memory protocols. Example apparatus may perform operations on a memory device based on commands received from a host according to a protocol, wherein the protocol includes non-deterministic timing of the operations. For example, the memory device may be a non-volatile dual in-line memory module (NVDIMM) device.

[0016] In one or more embodiments of the present invention, a protocol may be used to perform operations on an NVDIMM device. The protocol may allow the host to have direct access to the NVDIMM device, so the host can send commands to the NVDIMM device for execution by the NVDIMM device. The non-volatile nature of the NVDIMM device may allow the NVDIMM device to execute commands from the host with non-deterministic timing. The protocol may be used by the host and NVDIMM to send, receive, and execute commands. The protocol may include signals that achieve the following: execution of non-deterministic timing of commands; returning data associated with read commands in an order different from the order in which the read commands were issued; and using a write buffer, cache memory, or similar entity when executing a write command. Moreover, the protocol according to an embodiment of the present invention may be used with a previous protocol (e.g., compatible), so that the host can use the protocol of the present invention and other previous protocols on the same channel to send commands to the DIMM device. For example, the host may use the protocol according to an embodiment of the present invention to send commands to the NVDIMM device and also use the previous protocol on the same channel to send commands to the DDR4 SDRAM device.

[0017] The signals included in the protocol of the present invention may be a read ready (R_RDY) signal, a read send (R_SEND) signal, a write credit increment (WC_INC) signal, and a read identification (RID) signal. The read ready signal may be sent from the NVDIMM device to the host to indicate that the NVDIMM device is ready to send data associated with a read command to the host. The read send command may be sent from the host to the NVDIMM device in response to receiving the read ready command to indicate that the host is ready to receive data associated with the read command. The read identification (RID) signal may be sent from the NVDIMM device to the host when the data associated with the read command is transmitted to the host. The RID signal may be used by the host to identify which read command is associated with the data transmitted from the NVDIMM device to the host. The write credit increment signal may be sent from the NVDIMM device to the host in response to the completion of the execution of the write command or the write in the write buffer has been consumed (this indicates that the write buffer space associated with the request is available). The write increment signal may be used by the host to increment the number of available write commands that can be sent to the NVDIMM device. For example, each time the host sends a write command to the NVDIMM device, the host decrements the number of available write commands that can be sent to the NVDIMM device, and when the host receives a write count increment signal from the NVDIMM device, the host may increment the number of available write commands that can be sent to the NVDIMM device.

[0018] In the following detailed description of the invention, reference is made to the accompanying drawings which form a part of the invention and in which are shown by way of illustration how several embodiments of the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments of the invention, and it is understood that other embodiments may be utilized and process, electrical and / or structural changes may be made without departing from the scope of the invention. As used herein, the designator "N" indicates that several of the particular features so designated may be included with several embodiments of the invention.

[0019] As used herein, "a number of" something may refer to one or more of such things. For example, a number of memory devices may refer to one or more of the memory devices. Additionally, as used herein, designators such as "N" (particularly with respect to reference numbers in the drawings) indicate that a number of the particular features so designated may be included with several embodiments of the invention.

[0020] Each figure herein follows a numbering convention in which the first digit or first few digits correspond to the figure number of the diagram and the remaining digits identify the elements or components in the diagram. Similar elements or components between different figures can be identified by using similar numbers. As will be appreciated, the elements shown in the various embodiments herein can be added, exchanged and / or eliminated in order to provide several additional embodiments of the present invention. In addition, the proportions and relative scales of the elements provided in the various figures are intended to illustrate various embodiments of the present invention and are not used in a limiting sense.

[0021] Figure 1 is a functional block diagram of a computing system 100 including an apparatus in the form of a number of memory systems 104-1 ... 104-N according to one or more embodiments of the present invention. As used herein, "apparatus" may refer to, but is not limited to, any of a variety of structures or combinations of structures, such as one or more circuits, one or more dies, one or more modules, one or more devices, or one or more systems, for example. Figure 1 In the embodiment illustrated in , the memory system 104-1 ... 104-N may include one or more dual in-line memory modules (DIMMs) 110-1, ..., 110-X, 110-Y. The DIMMs 110-1, ..., 110-X, 110-Y may include volatile memory and / or non-volatile memory. In several embodiments, the memory system 104-1, ..., 104-N may include a multi-chip device. The multi-chip device may include several different memory types and / or memory modules. For example, the memory system may include non-volatile or volatile memory on any type of module. Figures 1 to 4 The examples described in this connection use DIMMs as memory modules, but the protocol of the present invention can be used on any memory system where the memory can execute non-deterministic commands. Figure 1, a memory system 104-1 coupled to a host via a channel 112-1 may include DIMMs 110-1, ..., 110-X, where DIMMs 110-1 are NVDIMMs and 110-X are DRAM DIMMs. In this example, each DIMM 110-1, ..., 110-X, 110-Y includes a controller 114. The controller 114 may receive commands from the host 102 and control the execution of the commands on the DIMMs. Moreover, in several embodiments, the protocol of the present invention may be implemented by a memory device (e.g., a DIMM) that does not have a controller, and the execution of commands using the protocol of the present invention may be built into the memory device. The host 102 may send commands to the DIMMs 110-1, ..., 110-X, 110-Y using the protocol of the present invention and / or the previous protocol, depending on the type of memory in the DIMM. For example, a host may use the protocol of the present invention to communicate with an NVDIMM on the same channel (e.g., channel 112-1) and use a previous protocol to communicate with a DRAM DIMM, both of which are on the same memory system. The host and NVDIMM may communicate via a read ready (R_RDY) signal, a read send (R_SEND) signal, a write credit increment (WC_INC) signal, and a read identification (RID) signal according to the protocol of the present invention. The read ready (R_RDY) signal, the read send (R_SEND) signal, the write credit increment (WC_INC) signal, and the read identification (RID) signal may be sent via pins that are not used in the previous protocol (e.g., DDR4) or pins from the previous protocol (e.g., DDR4) that are repurposed (e.g., used in a different manner) such that the present protocol is compatible with the previous protocol. Moreover, pins may be assigned to a read ready (R_RDY) signal, a read send (R_SEND) signal, a write credit increment (WC_INC) signal, and a read identify (RID) signal in protocols being developed (e.g., DDR5).

[0022] like Figure 1 As illustrated in , a host 102 may be coupled to memory systems 104-1...104-N. In several embodiments, each memory system 104-1...104-N may be coupled to the host 102 via a channel. Figure 11, memory system 104-1 is coupled to host 102 via channel 112-1 and memory system 104-N is coupled to host 102 via channel 112-N. Host 102 may be a laptop computer, a personal computer, a digital camera, a digital recording and playback device, a mobile phone, a PDA, a memory card reader, an interface hub, and other host systems, and may include a memory access device (e.g., a processor). One of ordinary skill in the art will appreciate that "a processor" may mean one or more processors, such as a parallel processing system, a number of co-processors, etc.

[0023] The host 102 includes a host controller 108 to communicate with the memory systems 104-1 ... 104-N. The host controller 108 sends commands to the DIMMs 110-1, ..., 110-X, 110-Y via channels 112-1 ... 112-N. The host controller 108 can communicate with the controllers 114 on the DIMMs 110-1, ..., 110-X, 110-Y and / or each of the DIMMs 110-1, ..., 110-X, 110-Y to read, write and erase data and perform other operations. The physical host interface may provide an interface for passing control, address, data and other signals between the memory systems 104-1 ... 104-N and the host 102 having a compatible acceptor for the physical host interface. Signals may be communicated between 102 and DIMMs 110-1, ..., 110-X, 110-Y via channels 112-1 ... 112-N over a number of buses (eg, a data bus and / or an address bus).

[0024] The host controller 108 and / or the controller 114 on the DIMM may include control circuitry, such as hardware, firmware, and / or software. In one or more embodiments, the host controller 108 and / or the controller 114 may be an application specific integrated circuit (ASIC) coupled to a printed circuit board including a physical interface. Likewise, each DIMM 110-1, ..., 110-X, 110-Y may include a buffer 116 of volatile and / or non-volatile memory. The buffer 116 may be used to buffer data used during the execution of a read command and / or a write command. The buffer 116 may be split into a write buffer and a read buffer. The amount of space dedicated to the write buffer and the amount of space dedicated to the read buffer may be controlled by the host controller 108. The host may control the amount of space dedicated to the write buffer and the read buffer in the buffer 116 based on the type of command sent to the particular DIMM. In several embodiments, the DIMM may have a fixed write buffer size and / or a fixed read buffer size.

[0025] DIMMs 110-1, ..., 110-X, 110-Y may provide primary memory for a memory system or may be used as additional memory or storage for the entire memory system. Each DIMM 110-1, ..., 110-X, 110-Y may include one or more arrays of memory cells (e.g., non-volatile memory cells). For example, the arrays may be flash arrays with a NAND architecture. Embodiments are not limited to a particular type of memory device. For example, the memory device may include RAM, ROM, DRAM, SDRAM, PCRAM, RRAM, and flash memory, among other memories.

[0026] Figure 1 104-N may include additional circuitry that is not illustrated so as not to obscure the embodiments of the invention. For example, the memory system 104-1 ... 104-N may include address circuitry to latch address signals provided via I / O connections by I / O circuitry. The address signals may be received and decoded by a row decoder and a column decoder to access the memory devices 110-1, ..., 110-N. Those skilled in the art will appreciate that the number of address input connections may depend on the density and architecture of the DIMMs 110-1, ..., 110-X, 110-Y.

[0027] Figure 2A to 2B 2 is a diagram of several read operations according to several embodiments of the present invention. The host and NVDIMM can communicate via commands 222, DQ+ECC pins 224, read ready (R_RDY) signal 228, read send (R_SEND) signal 230, write credit increment (WC_INC) signal 232 and read identification (RID) signal 226. Figure 2A The diagram illustrates an example of performing two read operations. A command signal 222 may be sent from the host to the NVDIMM. The command signal may include an activate command and a read command. Figure 2A In the embodiment of the present invention, a first activation command 242-1, a first read command 244-1, a second activation command 242-2, and a second read command 244-2 are sent from the host to the NVDIMM. In the protocol of the present invention, read commands can be sent one after another without considering the timing parameters. This allows commands to be sent faster than the commands must be spaced apart due to clock timing parameters, so the NVDIMM can start processing commands faster, which can reduce the latency of the commands.

[0028] In several embodiments, when sending an activate command linked by and / or immediately following a read command or a write command, there may be additional redundant bits between the activate command and the read or write command that may be repurposed for additional addressing of the command. Since non-volatile memory typically has more capacity than DRAM, it may be desirable to have additional addressing of the command. Bits in the read or write command that are redundant with the bits in the activate command may be used to provide additional row (page) addressing and / or additional column addressing for the read or write command.

[0029] exist Figure 2A , once the NVDIMM has received the read commands 244-1 and 244-2, the NVDIMM may check the buffer and / or cache memory of the NVDIMM and the non-volatile memory of the NVDIMM to locate the data associated with the read commands 244-1 and 244-2. Once the controller locates the data and prepares the data to be sent back to the host, the NVDIMM controller may send a read ready command to the host. When the NVDIMM receives several read commands, the NVDIMM controller does not have to respond to the read commands in the order in which the NVDIMM receives the read commands. For example, if the NVDIMM controller locates the data associated with the command before it locates the data associated with another command received before the command, the NVDIMM may send a read ready command indicating that the data associated with the command is ready to be sent to the host without having to wait for the NVDIMM controller to locate the data associated with the other command. When the data associated with the read command is located in the buffer, an out-of-order response to the read command may occur for some but not all of the read commands.

[0030] exist Figure 2AIn the embodiment of the present invention, a read ready indicator 228 may be sent to the host by switching (e.g., transitioning) a read ready signal from a low binary state to high or from a high binary state to low to indicate that the data associated with the read command 244-1 is ready to be sent to the host. The host may detect the read ready indicator 228 by detecting the transition of the read ready signal. In response to receiving the read ready indicator 228, the host may send a read send indicator 230 to the NVDIMM controller to indicate that the host is ready to receive the data associated with the read command 224-1. The read send indicator 230 may be sent by the host to the NVDIMM controller at any time and without regard to timing parameters. The host may control the traffic on the data bus by timing when the read send command is sent to the NVDIMM controller. The read send indicator 230 may include switching the read send signal from a low binary state to a high, which is detected by the NVDIMM controller. The NVDIMM controller may transmit data (V) 246-1 associated with the write command on the DQ+ECC pin 224 in response to receiving the read transmit indicator 230. send )250-1 and then sends data (V) 246-1. Moreover, when data (V) 246-1 is sent to the host, a read identification (RID) signal 248-1 is sent to the host on the RID pin 226. The RID signal 248-1 is used to identify the read command associated with the data (V) 246-1 sent to the host. Since NVDIMM commands can respond to read commands in an order different from the order in which the read commands are received, the host will not know which read command a particular data transfer is associated with without the RID signal. The RID signal may include n bits, where n is 1 or more bits. For example, the RID signal may be transmitted on a 2-bit RID bus. The number of outstanding reads that the host can issue may depend on the number of bits in the RID signal, the number of beats in the data packet, and the number of reads that the NVDIMM controller can support. For example, if the data burst is 8 beats and the RID signal is 2 bits, the number of outstanding reads that the host can issue is 2. 2*8 = 65,536. Furthermore, the NVDIMM may have a register that can be read by the host to further limit the number of outstanding read commands that the host can issue and the host may track the number of outstanding reads that the host has issued.

[0031] In several embodiments, the read send signal may be pulsed from low to high to low to indicate that the host is ready to receive data. The pulsed read send signal may be timed by the host to coordinate with other signals sent on a common channel to the DIMMs, which are typically captured at a clock edge, some of which may or may not use the protocol of the present invention.

[0032] In response to receiving the read command 244-2, a read ready indicator 228 may be sent to the host by switching the read ready signal from a high binary state to a low state to indicate that the data associated with the read command 244-2 is ready to be sent to the host. The host may detect the read ready indicator 228 by detecting the change of the read ready signal from high to low. In response to receiving the read ready indicator 228, the host may send a read send indicator 230 to the NVDIMM controller to indicate that the host is ready to receive the data associated with the read command 224-2. The read send indicator 230 may include switching the read send signal from a high binary state to a low state, which is detected by the NVDIMM controller. The NVDIMM controller may send data (V) 246-2 associated with the write command on the DQ+ECC pin 224 in response to receiving the read send indicator 230. During the read data transfer time period (t send )250-2 and then send data (V) 246-2. Moreover, when data (V) 246-2 is sent to the host, a read identification (RID) signal 248-2 is sent to the host on the RID pin 226. The RID signal 248-2 is used to identify the read command associated with the data (V) 246-2 sent to the host. In several embodiments, since the NVDIMM can execute read commands in any order, the data (V) 246-2 and the RID signal 248-2 may be first sent to the host in response to a two-state switch from low to high of the read ready command, and the data (V) 246-1 and the RID signal 248-1 may be sent from the host later in response to a two-state switch from high to low of the read ready command.

[0033] Figure 2B The diagram illustrates an example of performing two read operations, where one of the read operations includes a read hit and the other read operation includes a read miss. A read miss may indicate that the NVDIMM controller was unable to find the data associated with the read command in the buffer of the NVDIMM, and a read hit may indicate that the NVDIMM controller was able to find the data associated with the read command in the buffer. Figure 2B In the example, a first activate command 242-1, a first read command 244-1, a second activate command 242-2, and a second read command 244-2 are sent from the host to the NVDIMM.

[0034] exist Figure 2B, once the NVDIMM has received the read commands 244-1 and 244-2, the NVDIMM may check the buffer and / or cache memory of the NVDIMM and the non-volatile memory of the NVDIMM to locate the data associated with the read commands 244-1 and 244-2. The data associated with the read command 244-1 is a "read hit" and is located in the buffer of the NVDIMM. A read ready indicator 228 may be sent to the host by switching the read ready signal from a low binary state to a high state to indicate that the data associated with the read command 244-1 is ready to be sent to the host. The host may detect the read ready indicator 228 by detecting the change of the read ready signal from a low to a high state. In response to receiving the read ready indicator 228, the host may send a read send indicator 230 to the NVDIMM controller to indicate that the host is ready to receive the data associated with the read command 224-1. The read send indicator 230 may include switching the read send signal from a low binary state to a high state, which is detected by the NVDIMM controller. The NVDIMM controller may transmit data (V) 246-1 associated with the read command on the DQ+ECC pin 224 in response to receiving the read transmit indicator 230. send ) 250-1 and then sends the data (V) 246-1. Also, when the data (V) 246-1 is sent to the host, a read identification (RID) signal 248-1 is sent to the host on the RID pin 226. The RID signal 248-1 is used to identify the read command associated with the data (V) 246-1 sent to the host.

[0035] In response to receiving the read command 244-2, the NVDIMM may check the buffer and / or cache memory of the NVDIMM and the non-volatile memory of the NVDIMM to locate the data associated with the read commands 244-1 and 244-2. The data associated with the read command 244-2 is a "read miss" and is located in the non-volatile memory of the NVDIMM. Since the read command 244-2 is a "read miss", the NVDIMM controller cannot send a read ready command as fast as a "read hit" command. After the time period generated by locating the data associated with the read command 244-2, a read ready indicator 228 may be sent to the host by switching the read ready signal from a high binary state to a low state to indicate that the data associated with the read command 244-2 is ready to be sent to the host. The NVDIMM may wait to send the read ready command until the data associated with the read command is ready to be sent to the host, because the read ready command may be sent without considering the timing parameters. The host may detect the read ready indicator 228 by detecting the change of the read ready signal from high to low. In response to receiving the read ready indicator 228, the host may send a read send indicator 230 to the NVDIMM controller to indicate that the host is ready to receive data associated with the read command 224-2. The read send indicator 230 may include switching the read send signal from a high binary state to a low state, which is detected by the NVDIMM controller. The NVDIMM controller may send data (V) 246-2 associated with the write command on the DQ+ECC pin 224 in response to receiving the read send indicator 230. During the read data transfer time period (t send )250-2. Moreover, when the data (V) 246-2 is sent to the host, a read identification (RID) signal 248-2 is sent to the host on the RID pin 226. The RID signal 248-2 is used to identify the read command associated with the data (V) 246-2 sent to the host. In several embodiments, a priority (e.g., a priority bit can be enabled) can be included in the read command, which indicates to the NVDIMM that the read command should be executed with priority. The NVDIMM can send the data associated with the read command in which the priority bit is enabled before sending the data associated with other read commands. In several embodiments, since the NVDIMM can execute the read commands in any order, the data (V) 246-2 and the RID signal 248-2 can be sent to the host first in response to the two-state switching of the read ready command from low to high, and the data (V) 246-1 and the RID signal 248-1 can be sent from the host later in response to the two-state switching of the read ready command from high to low.

[0036] Figure 33 is a diagram of several write operations according to several embodiments of the present invention. The host and NVDIMM can communicate via commands 322, DQ+ECC pins 324, read ready (R_RDY) signal 328, read send (R_SEND) signal 330, write credit increment (WC_INC) signal 332 and read identification (RID) signal 326. Figure 3 The diagram illustrates an example of performing two write operations. A command signal 322 may be sent from the host to the NVDIMM. The command signal may include an activate command and a write command. Figure 3 In FIG. 1 , a first activate command 342-1 and a first write command 362-1 are sent from the host to the NVDIMM. The activate command and the write command may be sent without a timing delay therebetween.

[0037] exist Figure 3 Once the host has sent the write command 362-1, the host can write data transfer time period (t send ) 352-2 and then sends the data 346-1 associated with the write command to the NVDIMM, wherein the data associated with the write command is transferred to the NVDIMM. In the protocol of the present invention, a subsequent write command may be sent after the write data transfer time period and while the data associated with the write command is being transferred to the NVDIMM. Figure 3 In the second activation command 342-2 and the second write command 362-2, the write data transmission time period (t send ) 350-1 is sent from the host to the NVDIMM. Once the host has sent the write command 362-2, the host can write the data transfer time period (t send ) 352-2 then sends the data 346-2 associated with the write command 362-2 to the NVDIMM.

[0038] A write credit increment (WC_INC) indicator 332 may be used to indicate that write data associated with a write command has been written to the NVDIMM. The write credit increment indicator 332 may be sent from the NVDIMM to the host in response to the NVDIMM retiring the write command from the write buffer and writing the data associated with the write command to the cache memory and / or non-volatile memory of the NVDIMM. In embodiments where a write credit increment signal is sent from the NVDIMM to the host in response to the NVDIMM writing the data associated with the write command to the non-volatile memory of the NVDIMM, the write credit increment signal may be used to indicate to the host the persistence of the data (e.g., the data is stored in the non-volatile memory). In embodiments where the write credit increment command indicates the persistence of the data, the write commands are executed by the NVDIMM in the order in which the write commands are received by the NVDIMM. Moreover, the write credit increment signal may be used as a write buffer release signal.

[0039] In embodiments where a write credit increment command is sent from the NVDIMM to the host in response to the NVDIMM retiring the write command from the write buffer, the write credit increment command does not indicate to the host the persistence of the data (e.g., the data is stored in non-volatile memory), and the write commands may be executed by the NVDIMM in an order different from the order in which the write commands were received by the NVDIMM. For example, the data associated with the write commands is stored in a buffer and then written to the non-volatile memory of the NVDIMM in an order and time as determined by the NVDIMM controller.

[0040] In several embodiments, a write identification (WID) signal may be sent from the NVDIMM to the host when a write credit increment signal is sent to the host. The WID signal may be used to identify which write commands have completed execution and may be sent on several WID pins. The WID signal may be used with the write credit increment signal indicating the persistence of data to allow the NVDIMM to execute write commands in an order different from the order in which the write commands were received.

[0041] The number of outstanding write commands that the host can issue may depend on the size of the write buffer on the NVDIMM. The size of the write buffer on the NVDIMM may be controlled by the host. The host and NVDIMM controller may track the number of outstanding write commands that the host may issue. When there are no outstanding write commands issued by the host, the write count is at its maximum value. Each time a write command is sent by the host to the NVDIMM, the host decrements the write counter on the host, and each time a write command is received by the NVDIMM from the host, the NVDIMM decrements the write counter on the NVDIMM.

[0042] In several embodiments, two write credit increment signals may be used to track the number of outstanding write commands that the host may issue. For example, the WC_INCv signal may be toggled in response to a write command being retired (e.g., released) from a write buffer and the WC_INCnv signal may be toggled in response to a persistent write being completed in a non-volatile memory. The WC_INCv and WC_INCNV signals may be sent on different pins. The WC_INCv signals associated with write commands that are retired (e.g., space in the write buffer for data associated with the write command is free) from the write buffer may be sent to the host in an order different from the order in which the host sends the write commands. The WC_INCnv signals associated with persistent writes completed in the non-volatile memory may be sent to the host in the order in which the host sends the write commands.

[0043] exist Figure 3 In the example embodiment, before the write command is sent, the write count is at a maximum value (WC=Max) 360-M. In response to sending the write command 362-1, the write counter in the host is decremented by 1 (WC=Max-1) 360-M-1. In response to sending the write command 362-2, the write counter in the host is decremented by 1 again (WC=Max-2) 360-M-2. The NVDIMM may also include a write counter that is decremented upon receiving the write command. Once the NVDIMM has written the data 346-1 associated with the write command 362-1 to the buffer and / or non-volatile memory of the NVDIMM, a write count increment indicator 332 may be sent to the host. The write credit increment indicator 332 may switch a write credit increment signal from a low binary state to a high state to indicate that the data associated with the write command 362-1 has been written to the NVDIMM. The host may detect the write count increment indicator 332 by detecting a change in the write credit increment signal from low to high. In response to receiving the write credit increment indicator 332, the host may increment a write counter in the host by 1 (WC=Max-1) 360-M-1. Once the NVDIMM has written the data 346-2 associated with the write command 362-2 to the buffer and / or non-volatile memory of the NVDIMM, the write count increment indicator 332 may be sent to the host. The write credit increment indicator 332 may toggle the read ready signal from a high binary state to a low state to indicate that the data associated with the write command 362-2 has been written to the NVDIMM. The host may detect the write count increment indicator 332 by detecting a change in the write credit increment signal from high to low. In response to receiving the write credit increment indicator 332, the host may increment a write counter in the host by 1 (WC=Max) 360-M. The NVDIMM may also include a write counter that is incremented upon sending a write credit increment command.

[0044] Figure 4 is a diagram of several read and write operations according to several embodiments of the present invention. Figure 4 The diagram illustrates an example of performing two read operations and one write operation. Figure 4 In the embodiment of the present invention, a first activation command 442-1, a first read command 444-1, a second activation command 442-2 and a second read command 444-2, a third activation command 442-3, and a first write command 462-1 are sent from the host to the NVDIMM. In the protocol of the present invention, read commands can be sent one after another without considering the timing parameters and write commands can follow the read commands. This allows commands to be sent faster than the commands must be spaced apart due to clock timing parameters, so the NVDIMM can start processing commands faster, which can reduce the latency of the commands.

[0045] exist Figure 4 Once the host has sent the write command 462-1, the host can write data transfer time period (t send ) 452-1, the data 446-1 associated with the write command is sent to the NVDIMM. In response to sending the write command 462-1, the write counter in the host is decremented by 1 (WC=Max-1) 460-M-1. During the time when the host is sending the data 446-1 associated with the write command 462-1, the NVDIMM may check the buffer and / or cache memory of the NVDIMM and the non-volatile memory of the NVDIMM to locate the data associated with the read commands 444-1 and 444-2. Once the controller prepares the data associated with the read command to be sent back to the host, the NVDIMM controller may send a read ready command to the host. Figure 4 In the embodiment of the present invention, a read ready indicator 428 may be sent to the host by switching the read ready signal from a low binary state to a high state to indicate that the data associated with the read command 444-1 is ready to be sent to the host. The host may detect the read ready indicator 428 by detecting the change of the read ready signal from a low to a high state. In response to receiving the read ready indicator 428, the host may send a read send indicator 430 to the NVDIMM controller to indicate that the host is ready to receive the data associated with the read command 444-1. The read send indicator 430 may be sent by the host to the NVDIMM controller at a time determined by the host. The host may time the read send indicator to avoid data collision with write data on the data bus. The read send indicator 430 may include switching the read send signal from a low binary state to a high state, which is detected by the NVDIMM controller. The NVDIMM controller may send data (V) 446-2 associated with the read command on the DQ+ECC pin 424 in response to receiving the read send indicator 430. During the read data transfer time period (tsend ) 450-1 and then sends data (V) 446-2. Also, when data (V) 446-2 is sent to the host, a read identification (RID) signal 448-1 is sent to the host on the RID pin 426. The RID signal 448-1 is used to identify the read command associated with the data (V) 446-2 sent to the host.

[0046] Moreover, once the controller is ready to send the data associated with the read command 444-2 back to the host, the NVDIMM controller may send a read ready command to the host. A read ready indicator 428 may be sent to the host by switching the read ready signal from a high binary state to a low state to indicate that the data 446-3 associated with the read command 444-2 is ready to be sent to the host. The host may detect the read ready indicator 428 by detecting the change of the read ready signal from high to low. In response to receiving the read ready indicator 428, the host may send a read send indicator 430 to the NVDIMM controller to indicate that the host is ready to receive the data 446-3 associated with the read command 444-2. The read send indicator 430 may include switching the read send signal from a high binary state to a low state, which is detected by the NVDIMM controller. The NVDIMM controller may send the data 446-3 associated with the read command 444-2 on the DQ+ECC pin 424 in response to receiving the read send indicator 430. During the read data transfer time period (t send ) 450-2 and then send data 446-3. Also, when data (V) 446-3 is sent to the host, a read identification (RID) signal 448-2 is sent to the host on the RID pin 426. The RID signal 448-2 is used to identify the read command associated with the data (V) 446-3 sent to the host.

[0047] exist Figure 44, the NVDIMM may continue by executing read commands 444-1 and 444-2 while completing execution of write command 462-1 by writing data 446-1 to a write buffer on the NVDIMM and / or non-volatile memory of the NVDIMM. Once the NVDIMM has written data 446-1 associated with write command 462-1 to the buffer, cache memory, and / or non-volatile memory of the NVDIMM, a write count increment indicator 432 may be sent to the host. The write credit increment indicator 432 may toggle a write credit increment signal from a low binary state to a high state to indicate that the data associated with write command 462-1 has been written to the NVDIMM. The host may detect the write count increment indicator 432 by detecting a change in the write credit increment signal from low to high. In response to receiving the write credit increment indicator 432, the host may increment a write counter in the host by 1 (WC=Max) 460-M.

[0048] In several embodiments, several read and write commands may be sent to the NVDIMM. The several read and write commands may be executed in an order and timing controlled by the NVDIMM and the host.

[0049] Although specific embodiments have been illustrated and described herein, it will be appreciated by those skilled in the art that arrangements calculated to achieve the same results may replace the specific embodiments shown. The present invention is intended to cover changes or variations of the various embodiments of the present disclosure. It should be understood that the above description is made in an illustrative manner and not in a restrictive manner. After reviewing the above description, it will be clear to those skilled in the art that the combination of the above embodiments and other embodiments not specifically described herein. The scope of the various embodiments of the present invention includes other applications in which the above structures and methods are used. Therefore, the scope of the various embodiments of the present invention should be determined with reference to the attached claims together with the full scope of equivalents to which such claims are authorized.

[0050] In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of simplifying the invention. This method of the invention should not be interpreted as reflecting an intention that the disclosed embodiments of the invention must use more features than those explicitly stated in each claim. Rather, as the appended claims reflect: the inventive subject matter lies in less than all the features of a single disclosed embodiment. Therefore, the appended claims are hereby incorporated into the detailed description, with each claim independently serving as a separate embodiment.

Claims

1. A method comprising: Receiving at least one read command (244) and at least one write command (362) from a host (102) via a command / address bus of a non-volatile dual in-line memory module NVDIMM (110-1); Receiving data to be written to the NVDIMM from the host via a data bus of the NVDIMM; Based at least in part on writing the data received from the host, transmitting a write credit signal (332) from a set of pins of the NVDIMM designated for write credit signals to the host; sending a read ready indicator (428) to the host to indicate that data associated with the at least one read command is ready to be sent to the host; receiving a read send indicator (430) from the host at a time determined by the host to indicate that the host is ready to receive the data associated with the at least one read command in response to sending the read ready indicator; as well as Data in response to the at least one read command is transmitted to the host via the data bus of the NVDIMM in a same data packet as the write credit signal.

2. The method of claim 1, wherein the write credit signal comprises write credit feedback provided to the host to increment an indication of a write credit count at the host.

3. The method of claim 1, wherein the write credit signal comprises an indication to execute the at least one write command.

4. The method according to claim 1, further comprising: The at least one write command is received at a write buffer of the NVDIMM, wherein the write credit signal includes an indication of available space in the write buffer.

5. The method according to claim 4, further comprising: The write credit signal is transmitted in response to another command from the host.

6. A device comprising: a command / address bus configured to transmit read commands and write commands from a host (102) to the volatile memory or the non-volatile memory, or to both the volatile memory and the non-volatile memory; a data bus configured to communicate data associated with a read command (244) or a write command (362) between the host and the volatile memory or the nonvolatile memory (110-1, . . ., 110-Y), or between the host and both the volatile memory and the nonvolatile memory; as well as a set of pins, independent of the command / address bus, coupled to the volatile memory or the nonvolatile memory, or to both the volatile memory and the nonvolatile memory, the set of pins being configured to transmit a write credit signal to the host to provide write credit feedback based at least in part on data from the host that is written to the volatile memory or the nonvolatile memory, or to both the volatile memory and the nonvolatile memory, wherein the command / address bus is configured to send a read ready indicator (428) to the host to indicate that data associated with a read command is ready to be sent to the host, wherein in response to receiving the read ready indicator, the host is configured to send a read send indicator (430) to the volatile memory or the non-volatile memory, or to both the volatile memory or the non-volatile memory at a time determined by the host to indicate that the host is ready to receive data associated with the read command, and Wherein the set of pins is configured to transmit the write credit feedback in a same data packet as data communicated on the data bus in response to the read command.

7. The apparatus of claim 6, wherein the set of pins is configured to transmit write credit feedback, the write credit feedback comprising an indication to the host to increment a write credit count at the host.

8. The apparatus of claim 6 , wherein the set of pins is configured to transmit write credit feedback, the write credit feedback comprising an indication that data from the host communicated via the data bus has been written after receiving at least one write command received from the host via the command / address bus.

9. The apparatus according to claim 6, further comprising: A write buffer is coupled to the volatile memory or the nonvolatile memory, or to both the volatile memory and the nonvolatile memory, wherein the set of pins is configured to transmit write credit feedback including an indication of available space in the write buffer.

10. The apparatus of claim 9, wherein the set of pins is configured to transmit write credit feedback in response to another command received from the host via the command / address bus.

11. A device comprising: a volatile memory configured as an element (110-1, . . ., 110-Y) of a dual in-line memory module DIMM; a non-volatile memory configured as an element of the DIMM (110-1, . . ., 110-Y); a first bus coupled to the volatile memory or the non-volatile memory, or to both the volatile memory and the non-volatile memory, and configured to receive commands or addresses, or both commands and addresses, from a host (102); a second bus coupled to the volatile memory or the nonvolatile memory, or to both the volatile memory and the nonvolatile memory, and configured to communicate data between the volatile memory or the nonvolatile memory and the host, or between both the volatile memory and the nonvolatile memory and the host; as well as a set of pins, the set of pins being different from the first bus and configured to transmit write credit feedback from the DIMM to the host via a write credit signal based at least in part on data from the host being written to the volatile memory or the non-volatile memory, or based on data from the host being written to both the volatile memory and the non-volatile memory, wherein the first bus is configured to send a read ready indicator (428) to the host to indicate that data associated with a read command in the commands is ready to be sent to the host, wherein in response to receiving the read ready indicator, the host is configured to send a read send indicator (430) to the volatile memory or the non-volatile memory, or to both the volatile memory or the non-volatile memory at a time determined by the host to indicate that the host is ready to receive data associated with the read command, and Wherein the second bus and the set of pins are configured to transmit a data packet from the DIMM including write credit feedback and data in response to the read command received via the first bus.

12. The apparatus according to claim 11, further comprising: A write buffer is configured as an element of the DIMM and is coupled to the volatile memory or the non-volatile memory, or to both the volatile memory and the non-volatile memory, wherein the set of pins is configured to transmit write credit feedback including an indication of available space in the write buffer.

13. A method comprising: transmitting at least one read command (244) and at least one write command (362) from a host (102) to a non-volatile dual in-line memory module (NVDIMM) via a command / address bus; Transmitting data to be written to the NVDIMM from the host to the NVDIMM via a data bus; receiving, at the host, write credit feedback from a set of pins of the NVDIMM designated for write credit signals based at least in part on transmitting the data to be written to the NVDIMM; receiving a read ready indicator (428) at the host to indicate that data associated with the at least one read command is ready to be sent to the host; in response to receiving the read ready indicator, sending a read send indicator from the host to the NVDIMM at a time determined by the host to indicate that the host is ready to receive the data associated with the at least one read command; as well as A data packet including the write credit feedback and a response to the at least one read command is received at the host from the NVDIMM.

14. The method according to claim 13, further comprising: Based at least in part on receiving the write credit feedback, a write credit count is incremented at the host.

15. The method of claim 13, wherein the write credit feedback comprises an indication of execution of the at least one write command at the NVDIMM.

16. The method according to claim 13, further comprising: The at least one write command is transmitted to a write buffer of the NVDIMM, wherein the write credit feedback includes an indication of available space in the write buffer.

Images