Base die for stacked memory device with memory controllers and interconnect

Abstract

A stacked memory device comprises a stack of dies including a base die comprising interface logic and a set of memory dies including memory banks. The interface logic includes a host-side interface to a host device that communicates packetized serial commands and data, an interconnect, and a set of memory controllers corresponding to respective memory channels. The host-side interface may include one or more independent serial links including at least one command link, at least one response link, and at least one data link. The memory controllers operate in parallel to receive the packetized commands and perform memory operations with respect to the memory banks. The interconnect may comprise a full crossbar switch that enables switching between any of the serial links of the host-side interface and any of the memory channels controlled by the respective memory controllers.

Description

BASE DIE FOR STACKED MEMORY DEVICE WITH MEMORY CONTROLLERS ANDINTERCONNECTBRIEF DESCRIPTION OF THE DRAWINGS

[0001] The teachings of the embodiments herein can be readily understood by considering the following detailed description in conjunction with the accompanying drawings.

[0002] FIG. 1 is an example embodiment of a memory system according to an embodiment.

[0003] FIG. 2 is an example embodiment of an interconnect for a memory device according to an embodiment.

[0004] FIG. 3 is a flowchart illustrating an example embodiment of a process for performing a store operation in a memory system according to an embodiment.

[0005] FIG. 4 is a flowchart illustrating an example embodiment of a process for performing a load operation in a memory system according to an embodiment.

[0006] FIG. 5 is a first example physical structure of a memory device according to an embodiment.

[0007] FIG. 6 is a second example physical structure of a memory device according to an embodiment.DETAILED DESCRIPTIONA stacked memory device comprises a stack of dies including a base die having interface logic and a set of memory dies comprising a set of memory banks. The interface logic includes a hostside interface to a host device for receiving packetized serial commands and data, an interconnect, and a set of memory controllers corresponding to respective memory channels. The host-side interface may include one or more independent serial links including at least one command link, at least one response link, and at least one data link. The memory controllers may operate in parallel to receive the packetized commands and perform memory operations with respect to the memory banks. The interconnect may comprise a full crossbar switch that enables switching between any of the serial links of the host-side interface and any of the memory channels controlled by the respective memory controllers. The mapping between the serial links of the host-side interface and the respective memory channels may be dynamically configurable depending on the operating state.

[0008] FIG. 1 illustrates an example embodiment of a memory system 100. The memory system 100 includes a host device 110 and one or more memory devices 130 coupled by a1 RAMB-024WO / 11433WO01communication link 120. In the illustrated example, a single host device 110 and single memory device 130 are shown. In various applications, the memory system 100 can include any number of memory devices 130 coupled to the same host device 110 (via different serial communication links 120 or via a shared link), and / or the memory system 100 can include multiple host devices 110 accessing the same memory device 130.

[0009] The memory device 130 may include interface logic 140 and a set of memory banks 160 organized into memory channels (each comprising one or more banks 160). The interface logic 140 includes a host-side physical interface 142 (PHY), an interconnect 150, a plurality of memory controllers 152, and a set of channel interfaces 154. The host-side PHY 142 comprises a physical interface for communicating with the host device 110 via the communication link 120. In one implementation, the host-side PHY 132 comprises a plurality of serial links that facilitate packetized serial communication over the communication link 120. The serial links may perform deserialization of incoming serial communications and perform serialization of outgoing communications. The serial links may include one or more command links 144 for receiving packetized commands from the host device 110, one or more response links 146 for sending responses from the memory device 130 to the host device 110, and one or more data links 148 for communicating data between the memory device 130 and the host device 110.

[0010] The various serial links 144, 146, 148 may be unidirectional or bidirectional links in different configurations. For example, in one implementation the host-side PHY 142 includes separate unidirectional command and response links 144, 146 that may operate at least partially independently and may communicate concurrently (e.g., the response link 146 may communicate a response to a prior command with timing that may overlap with the command link 144 communicating a subsequent command). In another implementation, the command and response links 144, 146 may be merged in a single bidirectional link. In this case, various timing logic may be employed to avoid collisions between commands and responses. The data links 148 may similarly comprise either unidirectional, bidirectional links or a combination thereof. For example, in one implementation, a set of bidirectional links allow data to be communicated from the host device 110 to the memory device 130 (e.g., when writing to the memory device 130) and for data to be communicated from the memory device 130 to the host device 110 (e.g., when reading from the memory device 130). In other embodiments, the data links 148 may include pairs of unidirectional links, in which case data may be sent from the host device 110 to the memory device 130 and sent from the memory device 130 to the host device 110 on separate unidirectional link, which may operate concurrently. In other embodiments, the data links 148 may include a combination of unidirectional and bidirectional links.

[0011] The host-side PHY 142 may include different numbers of command links 144, response2 RAMB-024WO / 11433WO01links 146, and data links 148 in different configurations. In some implementations, the host-side PHY 142 includes a greater number of data links 148 than command and response links 144, 146 to accommodate high data bandwidths. The host-side PHY 142 may also include more command links 144 than response links 146 to accommodate protocols in which command bandwidth is higher than response bandwidth. In an example implementation the host-side PHY 142 may include three command links 134, one response link 136, and seven data links 138 per memory device 130. However, these numbers may vary depending on the configuration and design considerations.

[0012] The serial links 144, 146, 148 may operate at least partially independently such that packets may be communicated in parallel on two or more links 144, 146, 148. For example, multiple commands may be transmitted concurrently on different command links 144. Furthermore, data may be communicated on data links 148 in parallel with commands on the command link 144 and / or responses on the response link 146. A full load / sore transfer may utilize the command link 144, response link 146, and data links 148. The selection of which specific command link 144, response link 146, and data link(s) 148 are used for a specific transfer may be determined independently of each other and independently of which combination of links 144, 146, 148 are selected in prior transfers.

[0013] Communications over the host-side PHY 142 may conform to a packetized serial communication protocol at the physical and data link layers such as a coherent accelerator processor interface (e.g., OpenCAPI) protocol, a compute express link (CXL) protocol, a peripheral component interconnect express (PCI-e) protocol, a universal chiplet interconnect express (UCIe) protocol, or other serial communication protocol suitable for communicating packetized commands and data in a memory system 100.

[0014] The interconnect 150 facilitates switching of communications (e.g., command, responses, and data) between the host-side PHY 142 and the plurality of memory controllers 152. In one implementation, the interconnect 150 comprises a full crossbar switch such that any of the serial links 144, 146, 148 may be dynamically switched between any of the memory controllers 152. In some implementations, switching between the serial links 144, 146, 148 and the memory controllers may be dependent on a memory address of a command being executed for incoming commands to the memory device 130 or an identifier of the requestor at the device 110 for outgoing responses from the memory device 130. For example, each memory controller 152 may be associated with a particular address range and the interconnect 150 operates to switch incoming commands and incoming data (for store operations) to a particular memory controller 152 based on the address specified in the command. The interconnect may similarly output responses and / or data (for load operations) to a particular serial link of the host-side PHY 1423 RAMB-024WO / 11433WO01dependent on the associated requestor identifier. The interconnect 150 may also monitor congestion levels of the serial links 144, 146, 148 and dynamically select which response link 146 and / or data links 148 are used for sending data and responses to host device 110 depending on congestion levels of the serial link 146, 148. For example, the interconnect 150 may perform switching in a manner that optimizes latency, throughput, or some other performance criteria. In other embodiments, the interconnect 150 may perform dynamic switching based on some other predefined parameters or a combination of different parameters. The interconnect 150 may furthermore dynamically enable or disable memory channels and / or communication links 144, 146, 148, depending on congestion, bandwidth considerations, or other conditions. In an example embodiment, the interconnect 150 operates according to an Advanced extensible Interface (AXI) protocol. In other embodiments, a different communication protocol may be employed.

[0015] The memory controllers 152 facilitate memory operations for respective memory channels. Each channel may include a memory controller 152, a corresponding bank interface 154, and one or more memory banks 160 associated with the memory channel. For example, the memory controllers 152 may issue write commands to one or more memory banks 160 to facilitate writing of data to the memory banks 160 and may issue read commands to one or more memory banks 160 to facilitate reading of data from the memory banks 160. The memory controller 152 may further facilitate other control and command operations associated with the memory banks 160 such as activate commands, precharge commands, refresh commands, register read / write commands, or other memory operations.

[0016] The channel interface 154 between the memory controllers 152 and memory banks 160 may comprise a set of independent interfaces each including a command / address link and one or more data links that may operate as independent channels. In one implementation, each channel interface 154 may operate according to a high bandwidth interface (HBI) protocol or other protocol for communicating between memory controllers 152 and memory banks 160. In other embodiments, different types of memory and / or different interface protocols may be employed.

[0017] In one implementation, the memory banks 160 may comprise dynamic random access memory (DRAM) banks and the memory controllers 152 may operate according to DRAM specifications. The memory banks 160 may be organized into one or more bank groups, and each memory bank may include an array of memory cells arranged in rows and columns. In other example, the memory banks 160 may include one or more different types of memories such as static random access memory (SRAM), non-volatile core memory (such as flash), conductive bridging random core memory (CBRAM — a.k.a., programmable metallization cell — PMC), resistive random core memory (a.k.a., RRAM or ReRAM), or magneto-resistive random- access memory (MRAM),4 RAMB-024WO / 11433WO01and the like.

[0018] The memory controllers 152 and the memory banks 160 may operate with various timing constraints associated with different types of memory operations or sequences of operations. For example, a read / write timing constraint may enforce a minimum time between a row of a memory bank 160 being activated and performance of a read or write operation associated with that row. Furthermore, the memory banks 160 may operate such that only a single row per bank may be activated at a given time. In another example, a timing constraint may enforce a minimum time between writing data to a memory location and reading the data from that memory location.

[0019] The memory controllers 152 may operate to translate packetized commands received from the host device 110 (via the host-side PHY 142 and interconnect 150) into the memory commands. The packetized commands may comprise high-level commands such as load and store commands that do not necessarily follow the same command set employed between the memory controllers 152 and memory banks 160 and do not necessarily conform to the timing specifications of the memory operations. In some instances, the memory controller 152 may issue a sequence of memory commands in response to a single packetized command. For example, the memory controller 152 may receive a store command and issue a sequence of memory operations including an activate command to activate a row (if not already activated), a write command and the write data, and a precharge command to precharge the row. The memory controller 152 may also optionally issue an acknowledgement response to the host device. In response to a load command, the memory controller 152 may similarly issue an activate command to activate a row (if not already activated), issue a read command, obtain the read data and output it to the host device 110 (via the interconnect 150 and host-side PHY 142), and may subsequently issue a precharge command to precharge the row. The memory controller 152 may also optionally issue acknowledgement responses or error messages to the host device 110 responsive to load or store commands. In such operations, the memory controller 152 may control timing of the command sequence to meet timing specifications of the memory controller 152 and memory banks 160.

[0020] The memory banks 160, memory controllers 152, and channel interfaces 154 may operate as multiple memory channels, where each memory channel represents an independent data pathway that may operate in parallel. For example, each channel may include one memory controller 152, a dedicated set of interface pins of a channel interface 154 (e.g., an HBI including a command / address link and a set of data links), and one or more banks of memory banks 160. In one implementation, the memory device 130 includes 16 memory controllers 152 for respectively controlling 16 channels. In other implementations, the memory device 130 may5 RAMB-024WO / 11433WO01include a different number of channels and corresponding memory controllers 152 (e.g., 4 channels, 8 channels, 16 channels, 32 channels, 64 channels, or any other number of channels). In some embodiments, the number of channels may be dynamically configured. For example, the interconnect 150 may dynamically reassign routing between address ranges and channels to dynamically select which channels (and corresponding memory controllers 152, channel interfaces 154, and memory banks 160) are active. Furthermore, the number of channels may be independent of the number of host-side serial links 144, 146, 148 (as enabled by the interconnect 150).

[0021] In the above-described implementation, the interface logic 140 may include any number of serial links 144, 146, 148 and any number of memory controllers 152 (which may be different than the number of serial links 144, 146, 148). For example, a relatively smaller number of serial links 144, 146, 148 may facilitate communications for a relatively larger number of memory channels, each driven by one of the memory controllers 152. Furthermore, the number of active serial links 134, 136, 138 and active memory controllers 152 may dynamically change. The interconnect 150 may include configuration logic for enabling or disabling memory controllers 152 and / or serial links 144, 146, 148 depending on a state of the memory device 130. For example, if a failure occurs in a memory controller 152 or a serial link 144, 146, 148, the memory controller 152 or serial link 144, 146, 148 may be deactivated and the interconnect 150 operates to utilize remaining available memory controllers 152 and serial links 144, 146, 148. In other cases, one or more memory controllers 152 and / or serial links 144, 146, 148 may be disabled in low bandwidth situations for power savings. In further embodiments, one or more serial links 144, 146, 148 may be dynamically quiesced or slowed depending on overall bandwidth requirements.

[0022] In one implementation, the memory device 130 may comprise a stacked memory device that includes multiple dies arranged in a stack. In one such arrangement, the interface logic 140 resides on a base die and the memory banks 160 reside in one or more memory dies. The channel interface 154 may include interconnects such as through silicon vias (TSVs) to provide connectivity between the dies in the stack. In one implementation, the base die resides at the bottom of the stack. However, in other implementations, the base die may be at any position within the stack. Examples of physical implementations of the memory device 130 are illustrated below with respect to FIG. 5-6.

[0023] The host device 110 may comprise host processor (CPU) 112, a network-on-chip (NoC) 114, a bridge 116, and a host PHY 118. The host processor 112 may comprise a general-purpose processor, a graphics processor, a neural processing unit, or any other type of processor or combination thereof. The host processor 112 may operate as part of a cloud computing and / or6 RAMB-024WO / 11433WO01storage system, an enterprise server system, or any other computing environment. The NoC device 114 facilitates data transfer between various components of the host device 110 (such as the one or more processors 112 or other components of the host device 110) and the memory device 130. The NoC device 114 may perform functions such as packet-switching, routing, dynamic bandwidth allocation, address translation, packeting scheduling, arbitration, etc. between multiple memory devices 130 and / or between different channels of one or more memory devices 130. In one such implementation, the NoC device 114 may operate according to an AXI protocol. Alternatively, another communication may be used.

[0024] The bridge 116 provides an interconnect between the NoC device 114 and the communication link 120 and may facilitate translations between the native communication protocol of the NoC device 114 (e.g., an AXI protocol) and the communication protocol of the communication link 120 described above. The bridge 116 may be implemented as a separate chip from the NoC device 114 or the NoC device 114 and the bridge 116 may be integrated in a single chip. In an alternative embodiment, the NoC device 114 may natively include an interface compatible with the communication link 120 and the bridge 116 may therefore be omitted.

[0025] Similar to the interconnect 150 described above, the bridge 116 may operate to dynamically select which command link 144 and / or data links 148 are used for each transfer to the memory device 130. Selection of serial links may be based on an address range associated with commands, monitored congestion levels of the serial links 144, 146, or other performancebased criteria.

[0026] The host PHY 118 comprises a network-based communication interface or a direct serial communication interface. Communications over the host PHY 118 may conform to a packetized serial communication protocol at the physical and data link layers such as a an OpenCAPI protocol, a CXL protocol, a PCI-e protocol, a UCIe protocol, or another serial communication protocol suitable for communicating packetized commands and data in a memory system 100.

[0027] FIG. 2 illustrates an example embodiment of an interconnect 150. The interconnect 150 includes input / output (I / O) buffers 202, mapping logic 204, transaction tracking logic 206, and switching logic 208. The I / O buffers 202 buffer commands and data received from the communication link 120 and furthermore buffers responses and data to be transmitted over the communication link 120. The mapping logic 204 controls switching based on a mapping between incoming commands and the memory channels. For example, the mapping logic 204 may perform address-based mapping to map different address ranges to a specific memory channel. When an incoming command is received, the mapping logic 204 maps the address of the corresponding memory channel and controls the switching logic 208 to switch the command to the memory controller 152 associated with that channel. This mapping may be dynamically7 RAMB-024WO / 11433WO01configurable such that mapping between addresses and memory channels may change depending on the state of the device 130. In another embodiment, the mapping logic 204 may change routing to and from the serial links 144, 146, 148 based on monitored congestion of the various channels . In further embodiments, the mapping logic 204 may control enabling or disabling of memory channels depending on the state of the memory device 130 as described above.

[0028] The transaction tracking logic 206 tracks transaction identifiers associated with commands, data, and responses to link communications relating to the same transaction. The transaction tracking logic 206 may furthermore maintain one or more state parameters associated with a state of a transaction (e.g., indicative of whether write data has been sent, whether a response has been sent, whether read data has been sent, etc.) For example, when a store command is received, the transaction tracking logic 206 may store a transaction identifier associated with the command and map the transaction identifier to the channel on which the command is sent. When write data associated with the same transaction identifier is subsequently received, the transaction tracking logic 206 may then map the transaction identifier in the write data to the previously stored channel in order to send the write data over the same channel as the matching command. Responses and read data (for load commands) may similarly be mapped to the appropriate transaction based on the transaction identifier.

[0029] FIG. 3 is a flowchart illustrating an example embodiment of a process for performing a store command in a memory system 100. A store command and write data is received 302 at the host-side PHY 142. The interconnect 150 routes 304 the store command and write data to one of the memory controllers 152 associated with a channel mapped to the command. The memory controller 152 performs 306 one or more memory operations to write the data to the memory banks 160. For example, the memory controller 152 may translate the store command into one or more memory commands which are sequenced to one or more memory banks 160 together with the write data via the channel interface 154 in accordance with the command set and timing specifications of the memory controller 152 and memory banks 160. The memory controller 152 may furthermore issue 308 a response, which may be routed by the interconnect 150 to an appropriate serial link of the host-side PHY 142 for transmitting to the host device 110.

[0030] FIG. 4 is a flowchart illustrating an example embodiment of a process for performing a load command in a memory system 100. A load command is received 402 at the host-side PHY 142. The interconnect 150 routes 404 the load command to one of the memory controllers 152 based on channel mapping logic. The memory controller 152 performs 406 one or more memory operations to load the data from the memory banks 160. For example, the memory controller 152 may translate the load command into one or more memory commands which are sequenced to one or more memory banks 160 via the channel interface 154 in accordance with the command8 RAMB-024WO / 11433WO01set and timing specifications of the memory controller 152 and memory banks 160. The memory controller 152 obtains read data and sends 408 the read data and a response to the host device 110 via the interconnect 150 and host-side PHY 142.

[0031] FIG. 5 illustrates a first example embodiment of a physical structure of a memory device 130. In this example, the interface logic 140 (including the host-side PHY 142, interconnect 150, memory controllers 152, and channel interface 154) resides entirely on a base die 510. One or more memory dies 520 (each including one or more memory banks 160) are vertically stacked over the base die 510. The base die 510 and memory die 520 may be coupled using TSVs or other interconnects.

[0032] FIG. 6 illustrates a second example embodiment of a physical structure of a memory device 130. In this example, the host-side PHY 142, interconnect 150, and memory controllers 152 reside on a load / store base die 610 that may be combined with a high bandwidth memory (HBM) stack 640, which includes an HBM base die 630 and one or more memory dies 520. The HBM base die 630 includes a set of channel interfaces 154 for interfacing with the memory banks 160 on the memory die 520 and a set of corresponding HBI interfaces 650 that interface with the memory controllers 152. The channel interfaces 154, HBI interfaces 650, and memory controllers 152 may be structured as vertical interconnects to enable stacking of the memory banks 160 directly over the memory controllers 152. Alternatively, the channel interfaces 154 and / or HBI interfaces 650 of the HBM base die 630 may be structured to include horizontal interconnects such that the memory banks 160 of the memory die 520 are horizontally offset from the memory controllers 152.

[0033] Upon reading this disclosure, those of ordinary skill in the art will appreciate still alternative structural and functional designs and processes for the described embodiments, through the disclosed principles of the present disclosure. Thus, while embodiments and applications of the present disclosure have been illustrated and described, it is to be understood that the disclosure is not limited to the precise construction and components disclosed herein. Various modifications, changes and variations which will be apparent to those skilled in the art may be made in the arrangement, operation and details of the method and apparatus of the present disclosure herein without departing from the scope of the disclosure as defined in the appended claims.9 RAMB-024WO / 11433WO01

Claims

WHAT IS CLAIMED IS:

1. A memory device comprising: a set of memory dies arranged in a stack and comprising a set of memory banks logically associated with a plurality of memory channels; and a base die in the stack, the base die including interface logic comprising: a host-side interface to receive load and store commands from a host device; a plurality of memory controllers to communicate over the one or more memory channels to facilitate memory operations of the memory banks; and an interconnect to facilitate switching of communications between the hostside interface and the plurality of memory controllers.

2. The memory device of claim 1, wherein the host-side interface comprises a plurality of serial links including one or more command links, one or more data links, and one or more response links.

3. The memory device of claim 2, wherein the host-side interface includes a greater number of the one or more data links than the one or more command links.

4. The memory device of claim 2, wherein the interconnect enables switching between any of the one or more command links, any of the one or more response links and any of the one or more data links with any of the plurality of memory controllers.

5. The memory device of claim 2, wherein the one or more data links comprise bidirectional links.

6. The memory device of claim 1, wherein the host-side interface includes one or more bidirectional command / response links for communicating command and address information from the host device to the plurality of memory controllers and for communicating responses from the plurality of memory controllers to the host device.

7. The memory device of claim 1, wherein the host-side interface is configured to receive the load and store commands from the host device without memory-specific timing constraints, and wherein the plurality of memory controllers facilitate the memory operations by issuing memory commands conforming to the memory-specific timing constraints.

8. The memory device of claim 1, wherein the interconnect includes:10 RAMB-024WO / 11433WO01mapping logic to control switching of the communications between the plurality of memory controllers and the host-side interface based on an address range associated with a command in the communications or based on a requestor identifier associated with a response in the communications.

9. The memory device of claim 1, wherein the interconnect includes: mapping logic to obtain congestion information associated with the host-side interface, and to control switching of responses and data to serial links of the host-side interface based on the congestion information.

10. The memory device of claim 1, wherein the interconnect further includes: mapping logic to control enabling or disabling of individual serial links of the host-side interface and to control enabling or disabling of each of the plurality of memory channels.

11. The memory device of claim 1, wherein the base die further comprises: a channel interface comprising through silicon vias to vertically connect between the plurality of memory controllers and the set of memory banks.

12. The memory device of claim 1, further comprising: an interface die between the base die and the set of memory dies, wherein the interface die includes a channel interface comprising through silicon vias to vertically connect between the plurality of memory controllers and the set of memory banks.

13. The memory device of claim 12, wherein the channel interface of the interface die further includes one or more horizontal interconnects between the plurality of memory controllers and the set of memory banks.

14. A memory system comprising: a host device including a host processor and network interface to communicate packetized load and store commands and packetized data via a serial communication interface; at least one memory device comprising: a set of memory dies arranged in a stack and comprising a set of memory banks logically associated with a plurality of memory channels; and a base die in the stack, the base die including interface logic comprising: a host-side interface to communicate with the host device via the serial communication interface;11 RAMB-024WO / 11433WO01a plurality of memory controllers to communicate over the one or more memory channels to facilitate memory operations of the memory banks; and an interconnect to facilitate switching of communications between the host-side interface and the plurality of memory controllers.

15. The memory system of claim 14, wherein the host-side interface comprises a plurality of serial links including one or more command links, one or more data links, and one or more response links.

16. The memory system of claim 15, wherein the interconnect enables switching between any of the one or more command links, any of the one or more response links, and any of the one or more data links with any of the plurality of memory controllers.

17. The memory system of claim 15, wherein the interconnect includes mapping logic to dynamically configure mapping between the plurality of serial links and the plurality of memory channels.

18. The memory system of claim 14, wherein the host-side interface is configured to receive the load and store commands from the host device without memory-specific timing constraints, and wherein the plurality of memory controllers facilitate the memory operations by issuing memory commands conforming to the memory-specific timing constraints.

19. A method for operating a memory device comprising: receiving a load or store command at a host-side interface of a base die of a stacked memory device; routing by an interconnect on the base die, the load or store command to a memory controller of a plurality of memory controllers residing on the base die, each of the plurality of memory controllers logically associated with a different memory channel; and performing, by the memory controller residing on the base die, one or more memory operations with respect to one or more memory banks residing on one or more memory dies of the stacked memory device.

20. The method of claim 19, wherein the host-side interface comprises a plurality of serial links including one or more command links, one or more data links, and one or more response links, and wherein the in interconnect enables switching between any of the one or more12 RAMB-024WO / 11433WO01command links, any of the one or more response links, and any of the one or more data links with any of the plurality of memory controllers.13 RAMB-024WO / 11433WO01

Images