Expandable Dual In-line Memory Module

Abstract

An expandable dual in-line memory module (DIMM) includes: a dynamic random access memory (DRAM) having a basic memory space; a DIMM memory controller coupled to the DRAM; a memory interface configured to couple the DIMM to a DIMM connector of a computing device; and a first expansion interface configured to couple the DIMM to a first remote memory module having a first remote memory space, wherein the DIMM memory controller is configured to map the DIMM memory space, the DIMM memory space including the basic memory space of the DRAM and the first remote memory space of the first remote memory module. When the DIMM is coupled to a computing device, the DIMM memory space can be accessed by the computing device through the memory interface.

Description

Technical Field

[0001] The present application relates to an expandable memory, and more particularly to an expandable memory dual in-line memory module (DIMM) for an expandable memory subsystem. Background Art

[0002] The memory of a computing device is often limited by physical space constraints. For example, because the space on the motherboard of the computing device is limited, only a fixed number of slots / slots of memory DIMMs can be accommodated on the motherboard. Once each of all memory DIMM slots / slots has been populated with a memory DIMM, the memory subsystem of the computing device will be "expanded" to the maximum capacity. Therefore, increasing the memory capacity of each memory chip on the memory DIMM has become a typical solution for increasing the capacity of the memory subsystem. However, despite the subsequent emergence of new generations of memory DIMMs with larger memory capacities, the computing device may not be able to accommodate a mix of heterogeneous types of memory DIMMs. Basically, the user must accept the limited system memory capacity and, accordingly, settle for computing performance that is lower than the optimal computing performance of the computing device. Summary of the invention

[0003] According to one aspect, the present disclosure provides an expandable dual in-line memory module (DIMM), the DIMM comprising: a dynamic random access memory (DRAM), the DRAM having a basic memory space; a DIMM memory controller, the DIMM memory controller coupled to the DRAM; a memory interface, the memory interface being configured to couple the DIMM to a DIMM connector of a computing device; and a first expansion interface, the first expansion interface being configured to couple the DIMM to a remote memory module, the remote memory module comprising a first remote memory space, wherein the DIMM memory controller is configured to map the DIMM memory space, the DIMM memory space comprising the basic memory space of the DRAM and the first remote memory space of the remote memory module, wherein when the DIMM is coupled to the computing device, the DIMM memory space can be accessed by the computing device through the memory interface.

[0004] Optionally, as described above, the DIMM memory controller is further configured to use a first address to map to a second address, wherein the first address is determined as an address of the DIMM memory space, and the second address is determined as an address of the first remote memory space.

[0005] According to another aspect, the present disclosure provides an expandable memory subsystem, comprising: a dual in-line memory module (DIMM) as described above; and a first remote memory module coupled to the DIMM via a first expansion interface of the DIMM.

[0006] Optionally, as described above, the expandable memory subsystem, the first remote memory module also includes: a first remote memory controller, the first remote memory controller is configured to define a first remote memory space, the first remote memory controller is also configured to use a second address to map to a third address, wherein the second address is determined to be an address within the first remote memory space, and wherein the second address is converted from the first address by the DIMM memory controller.

[0007] Optionally, in the expandable memory subsystem as described above, the DIMM memory controller is further configured to provide a memory map, wherein the memory map is aggregated by the first remote memory space and the basic memory space.

[0008] Optionally, the expandable memory subsystem as described above also includes: a second remote memory module, the second remote memory module defines a second remote memory space, the second remote memory module is coupled to the first remote memory module via a second expansion interface, wherein the first remote memory module also includes: a first remote memory controller, the first remote memory controller is configured to use a second address to map to a third address, wherein the third address is determined to be an address within the second remote memory space, and the second address is determined to be an address within the first remote memory space of the first remote memory module, and wherein the second address is converted from the first address by the DIMM memory controller.

[0009] Optionally, in the scalable memory subsystem as described above, the first remote memory controller is configured to provide a memory map, wherein the memory map is aggregated by the first remote memory space and the second remote memory space.

[0010] According to another aspect, the present disclosure provides an expandable memory module, characterized in that the memory module includes: a dynamic random access memory (DRAM), the DRAM includes a DRAM memory space; a non-volatile memory (NVM), the NVM includes an NVM memory space; and a memory controller, the memory controller is configured to map a first memory space, the first memory space includes a DRAM memory space and an NVM memory space, wherein the memory controller is also configured to determine a first address within the first memory space.

[0011] Optionally, the memory module further comprises: a first interface configured to couple the memory module to a dual in-line memory module (DIMM) interface of a computing device.

[0012] Optionally, the memory module further includes: a second interface, wherein the second interface is configured to couple the memory module to a second memory module, and the second memory module has a second memory space.

[0013] Optionally, as described above in the memory module, the memory controller is further configured to map a first aggregate memory space, where the first aggregate memory space includes the first memory space and the second memory space.

[0014] Optionally, the memory module further includes an address translation table, which may be stored in the DRAM, and is configured to provide address mapping between the first aggregate memory space and the second memory space.

[0015] Optionally, as described above in the memory module, the memory controller is further configured to provide address mapping between the first aggregate memory space and one of the first memory space and the second memory space.

[0016] Optionally, in the memory module as described above, the second memory space includes a memory space of a DRAM device.

[0017] Optionally, in the memory module as described above, the second memory space includes storage space of a solid-state storage device.

[0018] Optionally, in the memory module as described above, the second memory space includes storage space of a hard disk drive.

[0019] Optionally, in the memory module as described above, the second interface comprises a network interface circuit, and the network interface circuit is configured to couple the memory module to another storage device.

[0020] According to another aspect, the present disclosure provides a memory subsystem, which includes: a first memory module, the first memory module includes: at least one memory device, the memory device provides a first memory space; a first memory controller, the first memory controller is configured to provide a first memory map of the first memory space; and a first interface, the first interface is configured to couple the first memory module to a dual in-line memory module (DIMM) interface; and a second memory module, the second memory module is coupled to the first memory module, the second memory module includes: at least one second memory device, the second memory device provides a second memory space; and a second memory controller, the second memory controller is configured to provide a second map of the second memory space to the first memory controller, wherein the first memory controller is configured to provide an aggregate memory map, the aggregate memory map includes the first memory map and the second memory map.

[0021] Optionally, as described above in the memory subsystem, the memory controller is further configured to use a first address of the first aggregate memory to map to a second address, where the second address is an address of one of the first memory map and the second memory map.

[0022] According to another aspect, the present disclosure provides a memory subsystem of a computing device, the memory subsystem comprising: a dual in-line memory module (DIMM), the DIMM comprising: a memory interface, the memory interface being configured to be coupled to a system memory controller of the computing device; and a DIMM memory controller, the DIMM memory controller being configured to map a DIMM memory space, the system memory controller being able to use the DIMM memory space through the memory interface; and cascaded remote memory modules, each remote memory module providing a remote memory space respectively, wherein the cascaded remote memory modules are coupled to the DIMMs, and such that the DIMM memory space is at least the cumulative sum of the corresponding remote memory spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A scalable memory subsystem is shown according to one embodiment.

[0024] Figure 2 A scalable memory subsystem according to another embodiment is schematically illustrated.

[0025] Figure 3A , Figure 3B and Figure 3C Schematically shows Figure 2 The cumulative memory space of the memory subsystem.

[0026] Figure 4A method of configuring a scalable memory subsystem according to one embodiment is shown.

[0027] Figure 5A and Figure 5B A method of operating using a scalable memory subsystem according to one embodiment is shown.

[0028] Figure 6 The address mapping of a DIMM memory controller according to one embodiment is schematically shown.

[0029] Figure 7 The address mapping to the first remote memory space according to another embodiment is schematically shown.

[0030] Figure 8 The address mapping to the second remote memory space according to another embodiment is schematically shown.

[0031] Fig. 9 The address is schematically shown to be mapped to the memory space upstream of the second remote memory space.

[0032] Fig.10 An expandable memory DIMM is shown according to one embodiment.

[0033] Fig.11 A remote memory module according to one embodiment is shown for use with Fig.10 Expandable memory DIMM connections in.

[0034] Fig.12 An expandable memory subsystem is shown including a plurality of memory DIMMs, wherein at least one memory DIMM is Fig.10 Expandable memory DIMMs in the .

[0035] Fig.13 Shown with Fig.11 Remote memory module coupling Fig.10 Expandable memory DIMMs in the .

[0036] Fig.14 Schematically showing the corresponding Fig.13 The memory space of the expandable memory subsystem in . DETAILED DESCRIPTION

[0037] It will be readily understood that, in addition to the example embodiments described herein, the components of the embodiments may be arranged and designed in a variety of different configurations as generally described and illustrated in the drawings herein. Therefore, the following more detailed description of the example embodiments represented in conjunction with the drawings is not intended to limit the scope of the claimed embodiments, but is merely representative of the example embodiments.

[0038] References throughout this specification to "one embodiment," "another embodiment," or "an embodiment" (or the like) mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment. Thus, the phrases "in one embodiment" or "in an embodiment," etc., appearing in various places throughout this specification are not necessarily all referring to the same embodiment.

[0039] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a detailed understanding of the embodiments. Those skilled in the relevant art will recognize that various embodiments can be implemented without one or more specific details, or in the case of other methods, components, materials, etc. In other cases, some or all known structures, materials, or operations may not be shown or described in detail to avoid confusion.

[0040] A computing device includes: a central processing unit (CPU) on a motherboard of the computing device. The CPU is equipped with an operating system for executing instructions and / or manipulating data. The CPU can be coupled to a solid-state drive and / or a hard disk drive via a peripheral component interconnect express (PCIe) bus or other suitable bus. The computing device can be regarded as a computing node. In order to communicate with another computing node in a network, the computing device is configured to transmit data via a network protocol. Examples of network protocols include a transmission control protocol / internet protocol (TCP / IP) network protocol and a remote direct memory access (RDMA) network protocol.

[0041] The CPU may also be configured to function as a system memory controller. Alternatively, the computing device may additionally provide a system memory controller configured to manage a memory subsystem. The memory subsystem defines and manages one or more memory spaces that can be used by the CPU. The memory subsystem enables data (including instructions) to be temporarily stored in the memory space so that the CPU can execute instructions and / or manipulate data.

[0042] The memory subsystem may include one memory dual in-line memory module (memory DIMM) or multiple memory dual in-line memory modules. Each memory DIMM includes multiple dynamic random access memories (DRAMs) arranged on a printed circuit board. Along one edge of the printed circuit board are multiple groups of contacts. Each group of contacts leads to the corresponding DRAM of the memory DIMM. The edge of the printed circuit board can be accommodated by a memory DIMM slot / slot on the motherboard. The memory DIMM slot / slot is coupled to the system memory controller. In this way, the system memory controller can directly access the corresponding DRAM. It is understood that the number of DRAMs on the memory DIMM is limited by the number of contacts that can be accommodated along the edge of the printed circuit board and the length of the corresponding memory DIMM slot / slot. Due to physical limitations, computing devices provide a limited number of memory DIMM slots / slots. Each memory DIMM slot / slot is configured to accommodate a memory DIMM, so in a conventional memory subsystem, the maximum memory space available to the CPU is limited by physical constraints.

[0043] In order to overcome the difficulties of the prior art, the present invention provides an expandable memory subsystem suitable for use with a computing device. According to an embodiment of the present disclosure, Figure 1As shown, the computing device 100 may include a system memory controller 102, wherein the system memory controller is configured to manage an expandable memory subsystem 101. The system memory controller may be in the form of an integrated circuit (IC) chip as a separate device from the CPU 103, or the system memory controller may form a part of the CPU chip. The CPU may include one or more processor cores, processor caches, buses or interconnects, logic, memory controllers and / or other components. The expandable memory subsystem is configured to store data (including instructions) for use by the CPU. For example, the system memory controller may cache data in a memory space to speed up access. In some embodiments, the memory space is provided by at least one expandable memory dual in-line memory module (expandable memory DIMM) 120 inserted or insertable into a memory DIMM connector 110. An example of a memory DIMM connector is configured for a double data rate (DDR) slot / slot. A standard memory DIMM slot / slot, such as one of the Joint Electron Devices Engineering Council (JEDEC) standard, can be used with embodiments of the present invention without modification. In some embodiments, the expandable memory DIMM includes a first interface 121 suitable for coupling the expandable memory DIMM to a memory DIMM connector. It will be understood that the standard memory DIMM connector provided in the computing device can accommodate the expandable memory DIMM according to the embodiment of the present disclosure, or it can accommodate a conventional memory DIMM. It should also be understood that with changing or new standards, the expandable memory DIMM of the present disclosure can be operably coupled with the memory DIMM connector of the changed or new standard to be used as part of the memory subsystem without exceeding the scope of the protection claimed in the present disclosure. This advantageously prepares for conventional computing devices to be upgraded to have an expandable memory subsystem without replacing the existing system memory controller or the existing memory DIMM connector. It should be understood that the expandable memory DIMM according to the embodiment of the present disclosure can also be configured to be operably coupled with other standard or non-standard connectors for communicating with the system memory controller.

[0044] The expandable memory DIMM 120 includes a dynamic random access memory (DRAM) 123 mounted to a printed circuit board 122. The DRAMs of the expandable memory DIMM collectively provide a basic memory space. When the expandable memory DIMM is coupled to the system memory controller 102, the expandable memory DIMM provides the basic memory space to the system memory controller, wherein the basic memory space is presented as one of the expandable capacities. The expandable memory DIMM includes a second interface 124, including the second interface 124 being configured to couple with a remote memory module 130. The remote memory module is configured to provide a first remote memory space. The expandable memory DIMM and the remote memory module are configured such that for the system memory controller, the memory space accessible through the DIMM connector / first interface 110, 121 is the cumulative total memory space including the basic memory space and the first remote memory space.

[0045] In other words, the expandable memory DIMM can be configured to provide a total expandable memory DIMM space to the system memory controller, wherein the total memory space accessible to the system memory controller can be changed without increasing the number of memory DIMM slots / slots in the computing device. The system memory controller can send data (including instructions) to the expandable memory DIMM via the memory bus through the memory DIMM interface. The memory bus can follow a standard (e.g., DDR4) or another pre-set protocol. In other words, the system memory controller and the memory DIMM connector coupled to the system memory controller do not need to be specially adjusted before they can be used with the expandable memory DIMM of the present disclosure.

[0046] To help understand, you can also refer to Figure 2 , FIG. 3A to FIG. 3C Yet another embodiment of the expandable memory subsystem 200 may include a plurality of memory spaces 212, 222, 232, provided or defined by an expandable memory DIMM 210 and remote memory modules 220, 230, respectively, which are coupled to each other in a cascade 201 as shown. For ease of reference, the "downstream" end of the cascade refers to the end where the expandable memory DIMM 210 is located. Alternatively, the downstream of the cascade leads to a system memory controller 202.

[0047] For example, in some embodiments, the expandable memory DIMM does not pass through the second interface ( Figure 3A ) is coupled to another memory module. The total expandable memory DIMM space 301 perceived by the system memory controller 202 is the same as the basic memory space 212 contributed by the expandable memory DIMM 210 .

[0048] In some embodiments, such as in Figure 3BIn the example of , the expandable memory DIMM 210 also includes a second interface / first expansion interface 231. The expandable memory DIMM can be coupled to the first remote memory module 220 through the first expansion interface 231. The expandable memory DIMM and the first remote memory module can communicate through the first expansion interface according to the expansion memory bus. The first expansion interface 231 can be considered as an example of an upstream interface (relative to the expandable memory DIMM 210), and the expandable memory DIMM communicates upstream with the first remote memory module 220. In the example of Figure 3B In the example shown, only one remote memory module 220 and one expandable memory DIMM 210 are coupled to the system memory controller 202. The first remote memory module is configured to provide a first remote memory space contribution 222. The total expandable memory DIMM space 300′ presented to the system memory controller by the expandable memory DIMM or perceived by the system memory controller at the memory DIMM interface 211 is the cumulative sum or aggregation of the base memory space contribution 212 and the first remote memory space contribution 222.

[0049] In some embodiments, Figure 3C As schematically shown, the expandable memory DIMM 210 is coupled to the first remote memory module 220 via the first expansion interface 231 according to the expansion memory bus for communication, and the first remote memory module 220 is also coupled to the second remote memory module 230 via the second expansion interface 241 (according to the expansion memory bus). The second memory module provides a second remote memory space contribution 232. In this case, the total expandable memory DIMM space 300' is the cumulative sum or aggregation of the basic memory space contribution 212, the first remote memory space contribution 222, and the second remote memory space contribution 232.

[0050] In some embodiments, the expandable memory DIMM is coupled to the first remote memory module via a first expansion interface (according to an expansion memory bus); the first remote memory module is also coupled to the second remote memory module via a second expansion interface (according to an expansion memory bus); the second remote memory module is also coupled to the third remote memory module via a third expansion interface (according to an expansion memory bus). The third memory module provides a third remote memory space. In these examples, the total expandable memory DIMM space is an aggregation of the basic memory space, the first remote memory space, the second remote memory space, and the third remote memory space. The expandable memory DIMM can therefore be configured to be coupled to n remote memory modules, and the total expandable memory DIMM space can be an aggregation of the basic memory space and all n remote memory spaces.

[0051] In some embodiments, the nth remote memory module may be coupled to the (n-1)th remote memory module (downstream) via the nth expansion interface. The nth remote memory module is configured to pass data downstream to the n-1th remote memory module, and ultimately to the expandable memory DIMM and to the system memory controller. The nth remote memory module may also be coupled to the (n+1)th remote memory module (upstream) via the (n+1)th expansion interface, and so on. According to the nth remote memory module, the upstream memory capacity will be represented as the sum of all memory capacities of all upstream remote memory modules (i.e., the maximum memory size).

[0052] According to an embodiment of the present disclosure, each memory module (including expandable memory DIMMs and any remote memory modules in cascaded memory modules) perceives the upstream memory space (if any) as one memory space, even though the upstream memory space is an aggregation of multiple upstream memory spaces. Each memory module is configured to provide a memory space to the next memory module downstream, which memory space includes the memory space perceived upstream and its own memory space. Whether the memory space presented to the downstream is equal to or greater than the memory space contributed by the memory module depends on whether the memory module is coupled to at least one memory module upstream. Therefore, in this article, the terms "memory space" and "memory space contribution" are used interchangeably to refer to the memory space perceived by the downstream / provided to the downstream, because the downstream memory controller is not aware of the upstream configuration.

[0053] Now refer to Figure 4 An exemplary method of configuring a system with expandable memory DIMMs is described. The expandable memory DIMMs may include a DIMM memory controller. In the method 400, when the system is powered on, a power-on self-test (POST) initialization 401 occurs. The computing device may be configured to perform the following operations as part of the POST. The computing device may be configured to first determine whether the device coupled to the memory bus is an expandable memory DIMM 402. If the device coupled to the memory bus is not an expandable memory DIMM, the total memory space available to the system memory controller through the memory DIMM interface is equal to the total memory space provided by the memory space of the non-expandable memory DIMM. The memory DIMM will provide a total memory map to the system memory controller, wherein the total memory map will only include the memory space actually available on the memory DIMM 403.

[0054] If the system memory controller is coupled to the expandable memory DIMM through the memory DIMM interface, the expandable memory DIMM will determine whether at least one first remote memory module is coupled to the first expansion interface 405 during initialization 404. If the DIMM memory controller determines that the expandable memory DIMM is not coupled to the at least one first remote memory module, the DIMM memory controller will provide the system memory controller with a total expandable memory DIMM space (map) equal to the basic memory space (map) 403.

[0055] Therefore, it can be understood that the system memory controller does not need to be configured to distinguish between non-expandable memory DIMMs and expandable memory DIMMs. Therefore, to avoid confusion, in this document, the terms "total memory DIMM space" and "total expandable memory DIMM space" are interchangeable terms with respect to the total memory space provided to the system memory controller through the memory DIMM interface.

[0056] The DIMM memory controller may be configured to determine whether the expandable memory DIMM is coupled to at least one remote memory module. If the expandable memory DIMM is coupled to at least one remote memory module (a first remote memory module) via a first expansion interface, the DIMM memory controller provides a reset signal to reset the first remote memory module 406. The trigger or an equivalent trigger causes the first remote memory module to undergo initialization. In the presence of a second remote memory module coupled to the first remote memory module, the reset signal will also cause the second remote memory module to undergo initialization.

[0057] The first remote memory module includes a first remote memory controller, wherein the first remote memory controller is configured to determine whether another remote memory module (second remote memory module) is coupled to the first remote memory module. That is, the first remote memory controller is configured to determine whether another remote memory module is coupled to the first expansion interface.

[0058] The first remote memory controller is configured such that, if the first remote memory module is not coupled to the second remote memory module, the first remote memory controller will reply to the DIMM memory controller 407 with a memory map that includes only memory locations on the first remote memory module. The memory space provided by the first remote memory module is referred to as the first remote memory space.

[0059] The first remote memory controller is configured such that, if the first remote memory module is coupled to the second remote memory module, the first remote memory controller will query 410 the second remote memory controller of the second remote memory module. In response, the second remote memory controller provides a reply 409 to the first remote memory controller, wherein the reply notifies the first remote memory controller 4072 of the second remote memory space. The first remote memory controller is configured to aggregate the second remote memory space with the first remote memory space. The aggregated memory space is stored in the first remote memory controller. The first remote memory controller may be configured such that it stores the aggregated memory space until a reset signal (or refresh instruction) is received 408. Thus, the first remote memory module is configured such that when the first remote memory module is coupled to the second remote memory module, the first remote memory controller will provide a memory space (map) to the DIMM memory controller, the memory space (map) being formed by the aggregation of the first remote memory space and the second remote memory space.

[0060] It should be understood that when the second remote memory module is coupled to the first remote memory module and is not coupled to any other remote memory module, the memory space presented by the second remote memory controller to the first remote memory controller is only the second remote memory space. In the case where the second remote memory module is coupled "downstream" to the first remote memory module (i.e., through the first expansion interface) and is also coupled "upstream" to the third remote memory module (i.e., through the second expansion interface), the memory space presented by the second remote memory controller to the first remote memory controller is an aggregation of the second remote memory space and the third remote memory space. In the case where one or more additional remote memory modules are coupled upstream of the second remote memory module, the memory space presented by the second remote memory controller to the first remote memory controller is an aggregation of the corresponding memory spaces (including the second remote memory space) upstream of the first remote memory space. The terms "upstream" and "downstream" are used in a relative sense. Therefore, when the first remote memory module is coupled to the second remote memory module through an interface, the interface can be upstream of the first remote memory module, and the same interface can be downstream of the second remote memory module.

[0061] If there are n remote memory modules coupled to the system at initialization, the initialization of the remote memory modules may start from the nth remote memory module, i.e., the remote memory module that is logically furthest away or remote relative to the expandable memory DIMM. The cascaded memory modules may be configured such that a reset signal is passed upstream to the nth remote memory module 408. The size of the remote memory space is then aggregated or cascaded from the nth remote memory module down to the first remote memory module to answer 409 the query 410 / 407n directed upstream. Thus, the first remote memory module (or the remote memory module that is logically directly coupled to the expandable memory DIMM) is able to provide a memory map of the remote memory space 411 to the expandable memory DIMM.

[0062] According to some embodiments, the DIMM memory controller is configured to determine or define the total memory space 405' after querying the first remote memory controller. According to some embodiments, the DIMM memory controller can determine or define the total memory space without querying the first remote memory controller. In both cases, the DIMM memory controller can be configured to calculate the total memory space and provide the total memory space to the system memory controller. Alternatively, or additionally, in some embodiments, the DIMM memory controller can be programmed by a user so that the DIMM memory controller reflects the total memory space available through the memory DIMM interface. Alternatively, or additionally, in some embodiments, the system memory controller can be programmed by a user so that the total memory space available to the system memory controller is known.

[0063] After initialization, the system memory controller perceives the memory space at the memory bus as a total memory space. The system memory controller can perceive the memory space at the memory bus as a total memory space, although the total memory space is actually an aggregation of the expandable memory DIMM space (basic memory space), the first remote memory space, the second remote memory space, etc. Regardless of the exact number of physical devices directly and / or indirectly coupled to the expandable memory DIMM, the system memory controller perceives the available memory space of one expandable memory DIMM as one memory space. The system memory controller can remain unaware of the number and type of memory coupled to the expandable memory DIMM.

[0064] Reference Figure 5A and Figure 5B An exemplary method 500 for operating with an expandable memory DIMM is described. Figures 6 to 9 , Figures 6 to 9Examples of different configurations that can be implemented using this method are schematically shown. In a memory subsystem, a system memory controller provides a first address 601 to an expandable memory DIMM (such as one described above) 501. The first address can be provided relative to data (including instructions) retrieved or stored from / to a memory location. The first address is determined to be an address within the total memory space perceived by the system memory controller. In this article, "an address is determined to be an address within a certain memory space" and "an address is determined based on a certain memory space" are used interchangeably, for example, the first address can be determined based on the total memory space perceived by the system memory controller. The system memory controller does not need to know whether the first address points to a memory location in the basic memory space 630 of the memory DIMM or to a memory location in the remote memory space 640. The system memory controller does not need to know whether the total memory space is provided only by the basic memory space 630 or by the aggregated memory space 610. The aggregated memory space (if any) can include the remote memory space 640 and the basic memory space 630, and / or the remote memory space itself is an aggregated memory space. Optionally, the system memory controller need not differentiate between memory locations physically located in the expandable memory DIMM and memory locations physically located in a remote memory module or elsewhere.

[0065] The expandable memory DIMM includes a DIMM memory controller, which, upon receiving a first address (the first address is determined as an address within the total memory space, or the first address is determined based on the total memory space), uses the first address to determine whether the first address falls within the basic memory space of the expandable memory DIMM 502, 503. For example, Figure 6 As shown, the first address may be an address corresponding to a memory location 631 in a DRAM on the expandable memory DIMM. If the first address is found in the basic memory space, the DIMM memory controller may continue to read / write from / to the memory location in the basic memory space 504, where the memory location is the location corresponding to the first address.

[0066] The DIMM memory controller may determine whether the first address corresponds to an address in the base memory space by performing a lookup or translating the first address. An example of performing the determination is using an address translation table (ACT) or a memory map, wherein each first address in the total memory space may be mapped to an address in the base memory space, or to a second address in the remote (aggregate) memory space.

[0067] The expandable memory DIMM ACT may be configured in the DIMM memory controller itself. Alternatively, the expandable memory DIMM ACT may be provided in other memory devices disposed on the expandable memory DIMM. The expandable memory DIMM ACT may be stored in DRAM or non-volatile memory (NVM).

[0068] In some examples, such as Figure 7 As shown, the expandable memory DIMM ACT can be configured to map or translate 505 the first address 601 to the second address 641, and the second address is not found in the basic memory space, but is found in the remote (aggregate) memory space 640 upstream of the basic memory space 630. The expandable memory DIMM ACT of the expandable memory DIMM can be configured so that if the second address is not found in the basic memory space, the DIMM memory controller only knows that the second address 641 is found in the remote (aggregate) memory space 640, without distinguishing which remote memory space (if there are multiple remote memory spaces available). If the DIMM memory controller determines that the second address falls within the remote memory space, the DIMM memory controller provides the second address to the first remote memory controller 505.

[0069] If the first remote memory module is not coupled to at least one other remote memory module upstream of itself, the second address 641 received by the first remote memory controller should correspond to the first remote memory space 740 ( Figure 8 ) at memory location 741.

[0070] If the first remote memory module is coupled to at least one remote memory module upstream of itself, the first remote memory controller uses the second address to determine whether the second address is found in the first remote memory space 740, or whether the second address is found in a remote (aggregate) memory space 840 upstream of the first remote memory space. The first remote memory controller may be configured to perform such a determination by performing a lookup or translation of the second address 506. One example of performing the determination may involve using a first remote address translation table (ACT), wherein each second address received by the first remote memory controller may be mapped to an address in the first remote memory space, or to a third address in a remote (aggregate) memory space upstream of the first remote memory space.

[0071] The first remote ACT may be configured in the first remote memory DIMM controller itself. Alternatively, the first remote ACT may be provided in other memory devices disposed on the first remote memory module. The first remote ACT may be stored in DRAM or non-volatile memory (NVM).

[0072] If the first remote memory controller determines that the second address falls within the first remote memory space 507, the first remote memory controller enables a read / write operation from / to the corresponding memory location in the first remote memory space 508 ( Figure 8 ). If data is retrieved from the corresponding memory location in the first remote memory space, the data is passed back to the expandable memory DIMM through the first expansion interface 509. The system memory controller will receive data from the expandable memory DIMM through the memory bus as if the data had been stored in the memory location of the expandable memory DIMM itself. Therefore, the system memory controller perceives the total memory space of the expandable memory DIMM as a larger memory space (an aggregation of the basic memory space and the first remote memory space) than the memory space physically available on a single expandable memory DIMM.

[0073] If the first remote memory controller determines that the second address falls within a remote (aggregate) memory space upstream of the first remote memory space, the first remote memory controller may convert the second address into a third address 510. The third address is determined to be an address within a remote (aggregate) memory space 840 upstream of the first remote memory space 740. The third address is determined based on the remote (aggregate) memory space 840 upstream of the first remote memory space 740. The second address is an address determined based on a larger memory space, wherein the larger memory space includes the first remote memory space and the remote (aggregate) memory space upstream of the first remote memory space. The first remote memory controller may be configured to provide the third address to a second remote memory controller, wherein the second remote memory controller is upstream of the first remote memory controller 510. The second remote memory controller may be arranged on a second remote memory module upstream of the first remote memory module. The second remote memory module is coupled to the first remote memory module via a second expansion interface. That is, the first remote memory controller and the second remote memory controller are coupled via the second expansion interface to communicate with each other.

[0074] In one example, the second remote memory module is not coupled to another remote memory module upstream of itself, and the second remote memory controller can use a third address to locate a memory location in a second remote memory space 840, where the second remote memory space is a space physically provided on the second remote memory module 511.

[0075] In yet another example, a second remote memory module having a second remote memory space is coupled to at least one other remote memory module upstream of the second remote memory module, and the second remote memory controller is configured to determine whether a third address corresponds to a memory location in the second remote memory space or to a memory location upstream of the second remote memory space.

[0076] If the second remote memory controller finds the corresponding memory location in the second remote memory space, the second remote memory controller reads / writes data from / to the corresponding memory location 512. If data is read from the corresponding memory location, the data will be transferred from the second remote memory module to the first remote memory module through the second expansion interface, and then transferred from the first remote memory module to the expandable memory DIMM 513 through the first expansion interface. Therefore, the system memory controller can receive data from the expandable memory DIMM as if the data had been stored in the expandable memory DIMM instead of in the remote memory module. Whether the data is read or written from the basic memory space or from the remote memory space is transparent to the system memory controller. Whether the data is read or written from the first remote memory space or the second remote memory space is transparent to the system memory controller.

[0077] If the second remote memory controller determines that the third address (the third address received from the first remote memory controller) does not correspond to a memory location on the second remote memory space 740, the second remote memory controller performs a lookup or translates the third address to a fourth address. The second remote memory controller may translate the third address to the fourth address using a second remote address translation table (second remote ACT).

[0078] In this example, the third address is determined to be an address within a memory space that includes the first aggregate memory space and the aggregation of all remote memory spaces upstream of the first remote memory space; and the fourth address is determined to be an address within a memory space that includes the aggregation of all remote memory spaces upstream of the first remote memory space but does not include the first remote memory space.

[0079] When the second remote memory controller determines that the third address corresponds to a memory location 841 in a remote memory module upstream of the second remote memory module, the second remote memory controller converts the third address to a fourth address and transmits the fourth address to a third remote memory controller, wherein the third remote memory controller is part of a third remote memory module and the third remote memory module is upstream of the second remote memory module ( Fig. 9 ).

[0080] Although the above describes a configuration involving up to three remote memory modules, it is understood that more remote memory modules can be provided and configured in the above manner. Generally speaking, the memory module includes a memory controller, and the memory controller is configured to convert a first address to a second address, wherein the first address is determined as an address within a first memory space and the second address is determined as an address within a second memory space. The first memory space includes a second memory space. The first address is provided through a downstream interface of the memory module. The second address is provided through an upstream interface of the memory module. The memory module is configured to provide a memory space, and the memory module includes a memory controller, which is configured to receive an upstream memory map from an upstream memory module and provide a downstream memory map to a downstream memory module, wherein the upstream memory map corresponds to an aggregation of one or more upstream memory spaces, and wherein the downstream memory map corresponds to an aggregation of the upstream memory map and the memory space. The memory space itself can be an aggregation of memory space and / or storage space. The memory controller is also configured to: when there is no upstream memory module coupled to the memory module, provide a downstream memory map corresponding to the memory map. The memory module is also configured to communicate with the upstream memory module through the upstream interface. The remote memory module is also configured to communicate with the downstream memory device via the downstream interface.

[0081] Optionally, the remote memory modules may be coupled to one another in a series such that the memory spaces defined by the respective remote memory controllers are cascaded downstream. In a series of coupled remote memory modules, the remote memory controller of a selected remote memory module will define a remote memory space that includes at least the cascaded or cumulative sum of the remote memory spaces upstream of the selected remote memory module. The remote memory spaces are cascaded or cumulative in a downstream direction such that the remote memory spaces (defined by the respective remote memory controllers) increase as the distance from the expandable memory DIMM module increases.

[0082] As an example, and as Fig.10 As shown, a computing device according to an embodiment of the present disclosure includes an expandable memory DIMM 1010, which is configured to be coupled to a host CPU 1000 through a memory DIMM interface 1020, such as a standard DIMM slot / slot. The expandable DIMM may include a contact group along a first edge of a printed circuit board, the first edge and the contact group being configured to be coupled to a memory DIMM slot / slot.

[0083] The expandable memory DIMM is equipped with an expandable memory controller 1030 and a bus 1021 / 1022. The host CPU and the expandable memory controller communicate via the DDR-T memory bus and the system management bus 1022. The expandable memory controller is configured to incorporate a serial presence detect (SPD) function 1031. The expandable memory DIMM may include a firmware flash memory 1032 that communicates with the DIMM memory controller via a serial peripheral interface bus. The expandable memory DIMM may also include a battery 1034 (which may be in the form of a supercapacitor) and a power management integrated circuit (PMIC) 1035. The PMIC may be coupled to the DIMM memory controller to implement power management on the DIMM independent of the host CPU. The DIMM memory controller and the PMIC may be configured to operate according to an inter-integrated circuit (I 2 C) bus protocol for communication.

[0084] The expandable memory DIMM is equipped with an expansion memory interface 1040, which is suitable for coupling with a remote memory module 1100. The expansion memory bus can be configured similarly to the memory DIMM bus between the expandable memory DIMM and the system memory controller. The expansion memory bus is configured to transmit data including instructions, addresses, and serial presence detection (SPD) information.

[0085] The expandable memory DIMM includes at least one dynamic random access memory (DRAM) 1050. At least one DRAM physically embodies memory locations that together form a memory space provided solely by the expandable memory DIMM. That is, the memory devices arranged on the expandable memory DIMM together provide a basic memory space. The DIMM memory controller is configured to provide a memory map including the basic memory space. The DIMM memory controller can be configured to provide a memory map of the total memory space or the aggregated memory space to the host CPU, wherein the memory map includes the basic memory space and a remote memory map received or retrieved through the extended memory interface. The expandable memory DIMM includes an address translation table (ACT) 1060. The ACT can be stored in a specified DRAM, or can be stored in a flash memory. Alternatively, the ACT can be stored in the DIMM memory controller.

[0086] like Fig.11As shown, a remote memory module 1100 according to an embodiment of the present disclosure includes a remote memory controller 1110 disposed on the remote memory module. The remote memory module may include a remote address translation table (remote ACT) 1160. The remote ACT may be part of the remote memory controller or a separate IC chip. The remote ACT may be copied to a flash memory or non-volatile (permanent) memory 1162. The remote memory module may also be equipped with a battery and a power management circuit 1164.

[0087] The remote memory module also includes a first expansion interface 1120, which is suitable for an operable coupling between the remote memory module and the expandable memory DIMM as described above. The remote memory module may also include a second expansion interface 1121, which is suitable for coupling to another remote memory module 1122. The remote memory module 1100 may include a first expansion interface and a second expansion interface, the first expansion interface being configured to couple a downstream memory module, such as the expandable memory DIMM described above, and the second expansion interface being configured to couple an upstream memory module. The remote memory module 1100 may also be coupled to other remote memory modules, such that the first expansion interface is coupled to a downstream remote memory module, and the second expansion interface is coupled to an upstream remote memory module.

[0088] The remote memory module 1100 may include one or more memory / storage devices. One or more memory / storage devices may be selected from a group including DRAM 1131, flash memory 1132, and storage drive 1133 (solid state and / or hard disk). The remote memory controller is configured to access the corresponding memory device and / or storage device so that the remote memory controller can define a remote memory space, which includes the memory space of the corresponding memory and / or storage device set on or coupled to the remote memory module. The remote memory module may also be equipped with a network interface circuit 1123, through which the remote memory controller can access cloud storage and / or network storage devices, so that the memory space of cloud storage and / or network storage is displayed as part of the remote memory space. In other words, the remote memory controller is configured to define or provide a remote memory space. The remote memory space corresponds to a memory location in a memory and / or storage device on / coupled to the remote memory module.

[0089] like Fig.12As shown, a memory subsystem 1200 according to an embodiment of the present disclosure includes an expandable memory DIMM 1210 coupled to a remote memory module 1220 via an expansion interface 1221. In this example, the remote memory module provides a remote memory space provided by multiple DRAMs installed in the remote memory module. The remote memory module can be equipped with other types of memory and / or storage devices, including but not limited to another storage device that can be accessed by a remote memory controller via a cloud and / or other network. The other storage device may include a memory device, a memory module and / or any type of memory.

[0090] In an expandable memory subsystem, the system memory controller 1230 of the host / computing device 1240 can be coupled to at least one expandable memory DIMM 1210 via a first memory DIMM interface 1211 , and at the same time, the system memory controller is coupled to at least one conventional memory DIMM 1250 via a second memory DIMM interface 1251 . Fig.12 Also shown is an expandable memory subsystem, wherein a system memory controller is coupled to a plurality of memory DIMMs, at least one of the plurality of memory DIMMs being an expandable memory DIMM. The system memory controller can access a total expandable memory space, the total expandable memory space capacity (maximum available memory and / or storage space) can vary according to user requirements, and the total expandable memory space can be accessed by a selected one of the plurality of memory DIMMs. Specifically, the total expandable memory space is configured to be accessible by the expandable memory DIMM.

[0091] Fig.13 and Fig.14An example of an expandable memory subsystem 1300 is shown, in which a system memory controller 1301 is coupled to an expandable memory DIMM 1302 through a memory DIMM interface 1303, and the expandable memory DIMM is coupled to a remote memory module 1320 through an expansion interface 1321. The total memory space 1400 of the memory subsystem includes a basic memory space 1410 and a remote memory space 1420. In this example, the remote memory space 1420 is an aggregation of multiple DRAM memory spaces 1421, 1422, a solid-state drive (SSD) storage space 1423, and a network storage space 1424. The network storage space can be provided to the remote memory module via a cable or wirelessly as part of a local area network, a wide area network, a cloud storage, etc. The remote memory controller 1322 can access the network storage space through a network interface circuit 1324. The remote memory module is configured to regard the storage / memory capacity of the physical network storage device as part of the aggregated memory space. The aggregated memory space can in turn be aggregated with the basic memory space through the scalable memory controller, so that the system memory controller perceives the network storage space as part of the memory DIMM space.

[0092] A second address 1452 pointing to a memory location on the network storage space may be allocated by a remote memory controller. The second address may be determined as an address within the remote memory space 1420. When the second address is passed to the DIMM memory controller, the DIMM memory controller maps the second address to the first address 1450. The first address is determined as an address within the total memory space 1400, wherein the total memory space is an aggregation of the remote memory space 1420 and the basic memory space 1410. The basic memory space is provided by a memory device (such as a DRAM) disposed on an expandable memory DIMM. The basic memory space itself may be an aggregated memory space. The address passed from the DIMM memory controller to the system memory controller is the first address, i.e., an address within the total memory space. In this example, the total memory space includes the basic memory space and the remote memory space, wherein the remote memory space itself is an aggregation of various memory and / or storage spaces, including but not limited to the space provided by the memory and / or storage device disposed on the remote memory module. Each memory and / or storage space may be accessed at the level of the memory subsystem via the memory bus without passing through the CPU of the computing device. Therefore, in order to access / use memory space that extends beyond the immediate memory DIMMs, the system memory controller only requires a memory map to the total (aggregate) memory space.

[0093] As used herein, unless explicitly stated otherwise, the singular "a", "an" and "an" may be construed as including the plural "one or more".

[0094] The present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limiting. Many modifications and variations will be apparent to those of ordinary skill in the art. The exemplary embodiments are selected and described in order to explain the principles and practical applications, and to enable those of ordinary skill in the art to understand the various embodiments of the present disclosure with various modifications suitable for the particular use contemplated.

[0095] Therefore, although illustrative example embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the description is not limiting and that various other changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope or spirit of the present disclosure.

Claims

1. An expandable dual in-line memory module, characterized in that: The dual in-line memory module comprises: A dynamic random access memory, wherein the dynamic random access memory has a basic memory space; a dual inline memory module memory controller, the dual inline memory module memory controller coupled to the dynamic random access memory; a memory interface configured to couple the dual inline memory module to a dual inline memory module connector of a computing device; and a first expansion interface, the first expansion interface being configured to couple the dual in-line memory module to a first remote memory module, the first remote memory module comprising a first remote memory space, The dual inline memory module memory controller is configured to map a dual inline memory module memory space, wherein the dual inline memory module memory space includes a basic memory space of the dynamic random access memory and a first remote memory space of the first remote memory module, wherein when the dual inline memory module is coupled to the computing device, the dual inline memory module memory space can be accessed by the computing device through the memory interface, and the dual inline memory module memory controller is also configured to use a first address to map to a second address, wherein the first address is determined as an address within the dual inline memory module memory space, and the second address is determined as an address within the first remote memory space.

2. An expandable memory subsystem, characterized in that: The expandable memory subsystem includes: The dual in-line memory module of claim 1; and A first remote memory module is coupled to the dual inline memory module through a first expansion interface of the dual inline memory module.

3. The expandable memory subsystem according to claim 2, wherein: The dual inline memory module memory controller is configured to provide a memory map, wherein the memory map is aggregated by the first remote memory space and the basic memory space.

4. The expandable memory subsystem according to claim 2, wherein: The expandable memory subsystem further includes: A second remote memory module, wherein the second remote memory module defines a second remote memory space, and the second remote memory module is coupled to the first remote memory module via a second expansion interface, wherein the first remote memory module further comprises: A first remote memory controller, wherein the first remote memory controller is configured to use a second address to map to a third address, wherein the third address is determined as an address within the second remote memory space, and the second address is determined as an address within the first remote memory space of the first remote memory module, and wherein the second address is converted from the first address by a dual in-line memory module memory controller.

5. The expandable memory subsystem according to claim 4, wherein: The first remote memory controller is configured to provide a memory map, wherein the memory map is aggregated by the first remote memory space and the second remote memory space.

6. An expandable memory module, characterized in that: The memory module comprises: A dynamic random access memory, wherein the dynamic random access memory comprises a dynamic random access memory memory space; A non-volatile memory, wherein the non-volatile memory includes a non-volatile memory memory space; a memory controller configured to map a first memory space, the first memory space comprising a dynamic random access memory memory space and a non-volatile memory memory space, wherein the memory controller is further configured to determine a first address within the first memory space; a first interface configured to couple the memory module to a dual in-line memory module connector of a computing device; and a second interface, the second interface being configured to couple the memory module to a second memory module, the second memory module having a second memory space, It is characterized in that the memory controller is also configured to map a first aggregate memory space, the first aggregate memory space includes the first memory space and the second memory space, and the memory module also includes an address translation table, the address translation table can be stored in the dynamic random access memory, and the address translation table is configured to provide address mapping between the first aggregate memory space and the second memory space.

7. The memory module according to claim 6, wherein: The memory controller is further configured to provide address mapping between the first aggregate memory space and one of the first memory space and the second memory space.

8. The memory module according to claim 7, wherein: The second memory space includes a memory space of a dynamic random access memory device.

9. The memory module according to claim 7, wherein: The second memory space includes a storage space of a solid-state storage device.

10. The memory module according to claim 7, wherein: The second memory space includes storage space of a hard disk drive.

11. The memory module according to claim 7, wherein: The second interface includes a network interface circuit configured to couple the memory module to another storage device.

12. A memory subsystem, characterized in that: The memory subsystem includes: A first memory module, wherein the first memory module comprises: at least one memory device, the memory device providing a first memory space; a first memory controller configured to provide a first memory map of the first memory space; a first interface configured to couple the first memory module to a dual in-line memory module connector of a computing device; and A second interface; and A second memory module, the second memory module is coupled to the first memory module through the second interface, and the second memory module includes: at least one second memory device, the second memory device providing a second memory space; and a second memory controller, the second memory controller being configured to provide a second memory map of the second memory space to the first memory controller, wherein the first memory controller is configured to provide an aggregate memory map, the aggregate memory map including the first memory map and the second memory map, It is characterized in that the first memory controller is also configured to use the first address of the aggregate memory to map to the second address, and the second address is an address of one of the first memory mapping map and the second memory mapping map.

13. A memory subsystem of a computing device, characterized in that: The memory subsystem includes: A dual in-line memory module, the dual in-line memory module comprising: a memory interface configured to couple to a system memory controller of a computing device; a DIMM memory controller configured to map a DIMM memory space that is accessible to the system memory controller via the memory interface; and A first expansion interface; and Cascaded remote memory modules, each of which includes a memory controller and provides a remote memory space, wherein the cascaded remote memory modules are coupled to the dual in-line memory module through the first expansion interface, and the dual in-line memory module memory space is at least the cumulative sum of the corresponding remote memory spaces, The dual inline memory module memory controller is configured to use a first address in the dual inline memory module memory space to map to an address in the basic memory space of the dual inline memory module or a second address in the corresponding remote memory space, and It is characterized in that each memory controller of the cascaded remote memory modules is also configured to communicate with the corresponding upstream memory module through the corresponding upstream interface, and communicate with the downstream memory device through the corresponding downstream interface, and It is characterized in that the memory controllers of the respective cascaded remote memory modules are also respectively configured to convert addresses provided through corresponding downstream interfaces into addresses provided through corresponding upstream interfaces, and receive an upstream memory map from an upstream memory module, and provide a downstream memory map to a downstream memory module, wherein the upstream memory map corresponds to an aggregation of one or more upstream memory spaces, and wherein the downstream memory map corresponds to an aggregation of the upstream memory map and the memory space, and It is characterized in that the memory controllers of the respective cascaded remote memory modules are also respectively configured to provide corresponding downstream memory mapping diagrams to the downstream memory modules when the corresponding upstream memory modules do not exist.

Images