Computing devices, memory module groups, computing devices, and clusters of computing devices

Abstract

The application provides a computing device, a memory module group, a computing device and a computing device cluster, and belongs to the technical field of chip packaging. The computing device comprises at least one memory module group, a processor and a mainboard. Each memory module in the memory module group is stacked by memory chips with a bit width of at least 8 bits, has a small size and a simple stacking process, and is accessed through at least one memory channel, so that the density of the memory channel per unit area is large. Therefore, the memory module group in the computing device can provide a large bandwidth to meet the demand of AI or large-scale computing scenarios for bandwidth without using HBM which has a large process difficulty and high cost, thereby reducing the engineering difficulty and saving the cost. In addition, the processor and the memory module communicate based on a DRAM interface, and compared with an HBM interface, the delay is smaller, the power consumption generated by the delay is smaller, the data reading and writing efficiency is improved, and the efficiency of random access is obviously improved.

Description

[0001] This application is a divisional application. The original application has the application number 202410585671.2 and the original application date is May 11, 2024. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of chip packaging technology, and in particular to a computing device, a memory module group, a computing equipment, and a computing equipment cluster. Background Technology

[0003] The core architecture of artificial intelligence (AI) is the AI ​​module, which includes accelerators such as graphics processing units (GPUs) or neural network processing units (NPUs) and multiple high-bandwidth memory (HBM) particles. Each HBM particle is composed of 3D heterogeneous multi-layer vertically stacked DRAM and logic chips that provide physical connections, through-silicon vias (TSVs), and design for test (DFT) units. Each HBM particle provides multiple channels for the accelerator, thereby meeting the high bandwidth requirements of AI scenarios.

[0004] In related technologies, multilayer DRAM in HBM particles is stacked using TSV technology. However, 3D TSV technology is difficult to implement, time-consuming, and requires wafer-level packaging (Chip on Wafer on Substrate, CoWoS) to encapsulate with the main chip, resulting in some yield loss and high cost of AI modules. Summary of the Invention

[0005] This application provides a computing device, a memory module group, a computing equipment, and a computing equipment cluster, which can reduce the manufacturing complexity of the computing device and save costs. The technical solution is as follows.

[0006] In a first aspect, a computing device is provided, the computing device including a processor, at least one memory module group and a first substrate; Each memory module in the memory module group is composed of stacked memory chips with a bit width of at least 8 bits. This results in a smaller size, simpler stacking process, and each memory module is connected to at least one memory channel, leading to a higher memory channel density per unit area. Since bandwidth equals the product of bit width and data transfer rate, a higher memory channel density per unit area allows a single memory module to provide a larger bit width. With a constant data transfer rate, a single memory module can provide a larger bandwidth. Therefore, at least one memory module group can provide bandwidth no less than that provided by a single HBM chip, serving as a replacement for HBM chips to meet the high bandwidth requirements of AI or large-scale computing scenarios without the need for the more complex and expensive HBM process. This reduces engineering complexity and saves costs. Furthermore, communication between the processor and memory modules is based on a memory interface, which offers lower latency compared to the HBM interface, resulting in lower power consumption due to latency, improved data read / write efficiency, and a more significant improvement in random access efficiency.

[0007] A memory module is a device that provides data storage functionality and is packaged from multiple memory chips. Memory modules can be installed on the motherboard. Each memory module is connected to at least one memory channel, allowing the processor to access the memory module through at least one memory channel, thereby storing data in or retrieving data from the memory module.

[0008] The processor accesses memory modules in the at least one memory module group through at least one memory interface, wherein different memory interfaces are used to access different memory modules.

[0009] In one possible implementation, the memory interface includes a dynamic memory controller (DMC) and a double data rate physical layer (DDR PHY).

[0010] In the aforementioned computing device, the processor and memory module communicate via a memory interface. Compared to the HBM interface, this interface has lower latency, resulting in lower power consumption due to latency, improved data read / write efficiency, and a more significant improvement in random access efficiency.

[0011] In one possible implementation, the computing device includes at least one first memory module group, wherein the memory modules in each first memory module group are mounted in a flat-mount manner.

[0012] In the aforementioned computing device, the memory modules in the first memory module group are installed in a flat mounting manner, which is simple to install, quick to install, and can improve the packaging efficiency of the computing device.

[0013] In one possible implementation, the computing device includes at least one second memory module group, each of which is pluggable.

[0014] In the aforementioned computing device, the memory modules in the second memory module group are installed in a pluggable manner. When a memory module is damaged, only the second memory module group containing the damaged memory module can be replaced, without replacing the entire computing device, which improves the convenience of maintenance and reduces maintenance costs.

[0015] In one possible implementation, each second memory module group further includes a second substrate on which memory modules from the second memory module group are mounted, the second substrate being pluggably mounted on the first substrate.

[0016] In one possible implementation, at least one memory module group is mounted on the first substrate in the following positions: distributed on both sides of the accelerator, or surrounding the accelerator, or distributed on one side of the accelerator.

[0017] In one possible implementation, the computing device further includes an integrated heat sink that covers at least one memory module group and the accelerator.

[0018] In the aforementioned computing device, a one-piece heat sink is used to dissipate heat from the processor and memory modules, resulting in better heat dissipation.

[0019] In one possible implementation, the integrated heat sink dissipates heat through at least one of liquid cooling and air cooling.

[0020] In one possible implementation, the first substrate is an ultra-high density particle board (UHD).

[0021] In the aforementioned computing device, UHD is used to install the processor and memory modules. UHD can meet the deployment requirements of a large number of lines in scenarios with large bit width. Furthermore, the processor and memory modules can be connected through UHD without the need to use complex and expensive packaging processes such as CoWoS to package the processor and memory modules, which can reduce engineering difficulty and save costs.

[0022] In one possible implementation, the total capacity of at least one memory module group is not less than the capacity of a high-bandwidth memory HBM particle, and the total bandwidth of at least one memory module group is not less than the bandwidth of a HBM particle.

[0023] In the aforementioned computing device, at least one memory module group provides a total bandwidth that is not less than that provided by one HBM particle, which can meet the high bandwidth requirements of AI or large-scale computing scenarios without the need to use HBM, which is more difficult to manufacture and more expensive. Therefore, it can reduce engineering difficulty and save costs.

[0024] In one possible implementation, the processor is any one of a central processing unit (CPU), a graphics processing unit (GPU), a network processing unit (NPU), a tensor processing unit (TPU), or a general purpose computing on graphics processing units (GPGPU).

[0025] In one possible implementation, when the memory module comprises multiple memory chip sets, each memory chip set comprises the same number of memory chips.

[0026] In one possible implementation, when the memory module includes multiple memory chip groups, there are at least two first memory chip groups in the memory module, and the at least two first memory chip groups include different numbers of memory chips.

[0027] In one possible implementation, the subgroup includes at least one data chip and at least one error correcting code (ECC) chip.

[0028] In the aforementioned memory module, since a subgroup connected by a memory channel includes a data chip and an ECC chip, the ECC chip is used to store error correction codes for the data in the data chip, that is, to store redundant data of the data chip. Thus, when the data in the data chip is corrupted and the number of corrupted bits is within the error correction capability of ECC technology, data recovery can be performed, allowing the processor to continue to access the subgroup normally. This avoids the interruption of the processor's access to the subgroup due to a small number of data errors, thereby improving the reliability of the data in the subgroup and the stability of data reading and writing.

[0029] In one possible implementation, the memory chipset includes at least one first subgroup, wherein the bit width ratio of a single data chip to a single ECC chip in the first subgroup is 2:1, and the number ratio of data chips to ECC chips in the first subgroup is 2:1.

[0030] In the aforementioned memory module, the bit width ratio of the data chip to the ECC chip is 2:1, and the quantity ratio of the data chip to the ECC chip is also 2:1. This satisfies the requirement that the total bit width of the data chip to the total bit width of the ECC chip in ECC technology should be 4:1, thereby ensuring that the error correction capability of ECC technology can be utilized when data errors occur.

[0031] In one possible implementation, the memory chipset includes at least one second subgroup, wherein the bit width ratio of a single data chip to a single ECC chip in the second subgroup is 4:1, and the number ratio of data chips to ECC chips in the second subgroup is 1:1.

[0032] In the aforementioned memory module, the bit width ratio of the data chip to the ECC chip is 4:1, and the quantity ratio of the data chip to the ECC chip is 1:1. This satisfies the requirement that the total bit width of the data chip to the total bit width of the ECC chip in ECC technology should be 4:1, thereby ensuring that the error correction capability of ECC technology can be utilized when data errors occur.

[0033] In one possible implementation, the memory chipset includes multiple subgroups, including at least one first subgroup and at least one second subgroup. In the first subgroup, the bit width ratio of a single data chip to a single ECC chip is 2:1, and the number ratio of data chips to ECC chips in the first subgroup is 2:1. In the second subgroup, the bit width ratio of a single data chip to a single ECC chip is 4:1, and the number ratio of data chips to ECC chips in the second subgroup is 1:1.

[0034] In one possible implementation, the capacity ratio of a single data chip to a single ECC chip in the first subgroup is 2:1.

[0035] In the aforementioned memory module, the bit width ratio of the data chip and the ECC chip is 2:1, meaning the ratio of the number of data pins in the data chip to the number of data pins in the ECC chip is 2:1. Setting the capacity ratio of the data chip and the ECC chip to 2:1 also ensures that the number of data pins corresponds to the capacity of the memory chip. In other words, the larger the capacity of the memory chip, the more data pins it has, and the more bits of data can be written to or read from the memory chip at one time, thus avoiding waste of memory chip capacity.

[0036] In one possible implementation, when the memory chipset comprises multiple first subgroups, the data chips and ECC chips in each first subgroup are stacked in the same order.

[0037] In the aforementioned memory modules, since multiple chips in each first subgroup adopt the same stacking order, the orderly packaging of the memory modules can be guaranteed, which helps to reduce processing errors in the memory modules, thereby improving the yield of the memory modules, and also helps the processor to identify data chips and ECC chips.

[0038] In one possible implementation, the capacity ratio of a single data chip to a single ECC chip in the second subgroup is 4:1.

[0039] In the aforementioned memory module, the bit width ratio of the data chip and the ECC chip is 4:1, meaning the ratio of the number of data pins in the data chip to the number of data pins in the ECC chip is 4:1. Setting the capacity ratio of the data chip and the ECC chip to 4:1 also ensures that the number of data pins corresponds to the capacity of the memory chip. In other words, the larger the capacity of the memory chip, the more data pins it has, and the more bits of data can be written to or read from the memory chip at one time, thus avoiding waste of memory chip capacity.

[0040] In one possible implementation, when the memory chipset includes multiple second subgroups, the data chips and ECC chips in each second subgroup are stacked in the same order.

[0041] In the aforementioned memory modules, since multiple chips in each second subgroup adopt the same stacking order, the orderly packaging of the memory modules can be guaranteed, which helps to reduce processing errors in the memory modules, thereby improving the yield of the memory modules, and also helps the CPU to identify data chips and ECC chips.

[0042] In one possible implementation, the memory chipset includes at least one third subgroup, each third subgroup consisting of three data chips and one ECC chip stacked together, wherein one data chip has a bit width of 32 bits, two data chips have a bit width of 16 bits, and one ECC chip has a bit width of 16 bits.

[0043] In one possible implementation, multiple memory chips in each memory chipset are stacked based on redistribution layer (RDL) technology and wire bonding.

[0044] The process involves first rewiring the pads at the center of multiple memory chips to the edge of the memory chips using RDL technology, and then connecting the pads of multiple memory chips to the substrate leads using wire bonding to form a memory module.

[0045] In the aforementioned memory module, the pads are rewired to the edge of the memory chip using RDL technology, which helps to shorten the length of the bonding wires used to connect the pads and the substrate leads. The shorter the bonding wire length, the higher the signal quality transmitted through the bonding wire.

[0046] In one possible implementation, multiple memory chips in each memory chip group are stacked based on wire bonding.

[0047] In the aforementioned memory module, multiple memory chips in the memory chipset are directly packaged using wire bonding, without the need for the more complex RDL technology. Therefore, the packaging process difficulty can be reduced, and costs can be saved.

[0048] In one possible implementation, the memory module does not include a registered clock driver (RCD).

[0049] The memory modules described above do not include an RCD, which reduces product costs compared to RDIMMs in related technologies. A memory module can save approximately $10 in cost compared to an RDIMM. Furthermore, the location of the RCD does not need to be considered when packaging the memory module, resulting in a smaller package size. The memory module can be installed closer to the processor, thereby improving the efficiency of the processor accessing the memory module and enhancing signal quality.

[0050] In a second aspect, a memory module group is provided, the memory module group including at least one memory module and a second substrate, the second substrate being used to mount at least one memory module, and each second memory module group being mounted in a pluggable manner.

[0051] Thirdly, a computing device is provided, which includes a plurality of computing devices and a third substrate on which the plurality of computing devices are mounted.

[0052] Fourthly, a computing device cluster is provided, the computing device cluster including at least one computing device, each computing device including multiple computing units and a third substrate, the third substrate having multiple computing units mounted on it.

[0053] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a memory module 1A provided in an embodiment of this application; Figure 2 This is a schematic diagram of a memory module 2A provided in an embodiment of this application; Figure 3 This is a schematic diagram of a memory module 3A provided in an embodiment of this application; Figure 4 This is a simulation result diagram of a CA eye-mask provided in an embodiment of this application; Figure 5 This is a schematic diagram of a computing device 5B provided in an embodiment of this application; Figure 6 This is a schematic diagram of a memory interface provided in an embodiment of this application; Figure 7 This is an installation diagram of a computing device 5B provided in an embodiment of this application; Figure 8 This is a schematic diagram of a computing device 8B provided in an embodiment of this application; Figure 9 This is an installation diagram of a computing device 8B provided in an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0056] To facilitate understanding of the technical solutions in the embodiments of this application, several technical terms involved in the embodiments of this application will be explained below.

[0057] A memory channel is a physical path connecting the processor and the memory module. A memory channel consists of multiple physical lines, each connected to a data pin of a memory chip, transmitting one bit at a time. The bit width of a memory channel indicates the number of bits that can be transmitted simultaneously. For example, a 64-bit memory channel indicates that it includes 64 physical lines, transmitting 64 bits simultaneously. Multiple memory channels can transmit data in parallel; the more memory channels, the more bits can be transmitted in parallel, resulting in higher processor efficiency in accessing memory. The total bit width of the data chips connected to a memory channel is equal to the bit width of the memory channel itself.

[0058] The bit width of a memory chip refers to the number of data pins included in the memory chip, indicating the number of bits that can be written to or read from the memory chip at one time. For example, a memory chip with a bit width of 8 indicates that the memory chip includes 8 data pins, and the 8 data pins can transmit 8 bits simultaneously at one time. That is, the memory chip can write or read 8 bits simultaneously at one time.

[0059] Processor bit width: refers to the amount of data a processor can read or write to memory in a single memory channel. For example, a processor bit width of 64 indicates that the processor can write or read 64 bits from memory at a time.

[0060] In related technologies, each HBM chip is composed of 3D heterogeneous multilayer vertically stacked DRAM and logic chips providing physical interconnection, TSV, and DFT test units. Specifically, an HBM chip consists of four layers of chips with TSV vias and micro-solder balls (ubumps). TSV technology requires high aspect ratio (10:1) etching and filling. Each DRAM chip layer needs chemical mechanical polishing (CMP) to thin it to less than 50µm during stacking. Such thin silicon wafers are very fragile, and clamping and transferring them is extremely difficult. Chip-to-chip alignment during stacking is very difficult due to the small size and high density of the micro-solder balls. High thermal conductivity underfill material or NCF is used to fill the spaces between DRAM chip layers. High thermal conductivity materials are cutting-edge technologies in the industry. The stacking process inevitably introduces engineering defects, and how to test and repair them requires accumulation and exploration. Because HBM 3D stacking engineering technology is difficult, time-consuming, and has a certain yield loss, its price is 4 to 5 times that of general DRAM per unit density.

[0061] This application provides a computing device comprising at least one memory module group, a processor, and a first substrate. Each memory module in the memory module group is composed of stacked memory chips with a bit width of at least 8 bits. This results in a small size, simple stacking process, and each memory module is connected to at least one memory channel, leading to a high memory channel density per unit area. Since bandwidth equals the product of bit width and data transfer rate, a high memory channel density per unit area allows a single memory module to provide a larger bit width. With a constant data transfer rate, a single memory module can provide a larger bandwidth. Therefore, at least one memory module group can provide bandwidth no less than that provided by a single HBM chip, serving as a replacement for HBM chips to meet the high bandwidth requirements of AI or large-scale computing scenarios without the need for the more complex and expensive HBM process. This reduces engineering complexity and saves costs. Furthermore, communication between the processor and memory modules is based on a memory interface, which, compared to the HBM interface, results in lower latency, leading to lower power consumption due to latency, improved data read / write efficiency, and a more significant improvement in random access efficiency.

[0062] First, we will further introduce the memory modules used in the aforementioned computing device.

[0063] This application provides a memory module comprising at least one memory chip group, each memory chip group comprising at least one subgroup, different subgroups being connected to different memory channels, wherein each subgroup is composed of stacked memory chips with a bit width greater than or equal to 8 bits, and each subgroup contains at least one memory chip with a bit width greater than 8 bits. A memory chip group is a stack of multiple memory chips stacked together, which can be understood as a "stack". Compared to related technologies, the use of memory chips with a larger bit width reduces the number of memory chips connected to a single memory channel, thereby reducing the number of CA lines in the memory channel and lowering the CA signal load on the processor. The number of memory chips connected to a single memory channel is reduced by at least 40% compared to related technologies. The processor can directly drive the clock signal and CA signal to access the memory module without needing to drive it through an RCD relay. Since the number of memory chips connected to a single memory channel is significantly reduced and no RCD configuration is required, product costs can be reduced. Furthermore, the memory module is packaged by stacking memory chips, which significantly reduces the size of the memory module compared to related technologies that flatly mount each memory chip on the PCB to form a memory module or directly on the motherboard. The memory module can be installed closer to the processor, the memory channel connecting the processor and the memory module is shorter, the processor accesses the memory module more efficiently, and the signal quality is better.

[0064] It should be noted that the memory chips in the above memory module are all dies, and the specifications of the memory chips can be the same or different. For example, the memory chips can all be dynamic random access memory (DRAM) chips. In some embodiments, the above DRAM chips can be DDR SDRAM (double data rate synchronous dynamic random access memory) chips, abbreviated as DDR; for example, DDR4, DDR5 or DDR6, etc.

[0065] Those skilled in the art will understand that as the area of ​​storage cells (bits) continues to shrink, the margin of storage cells also decreases, increasing the probability of bit flips in memory chips. To ensure the reliability of data storage in memory modules, multiple memory chips in a memory module include data chips and ECC chips. It should be understood that ECC is an error correction technology that detects and corrects errors in data operations by adding extra bits to the data. ECC technology can tolerate errors in memory operation; even if data errors occur, the memory controller will automatically correct these errors based on ECC, allowing the system to continue operating normally without interruption due to errors. When data is written to the ECC chip, the error correction code corresponding to the written data is stored in the ECC chip. When reading data, the memory controller compares the stored error correction code with the error correction code generated during data reading. If the read error correction code does not match the stored error correction code, the error bit can be determined based on the error correction code, and then that bit is corrected immediately.

[0066] Furthermore, when using ECC technology, the ratio of the total bit width of the data chip to the total bit width of the ECC chip among multiple memory chips connected to a memory channel is typically 4:1. With a CPU bit width of 64 bits, the total bit width of the data chip is 64 bits, and the total bit width of the ECC chip is 16 bits. ECC technology's error correction capability is to correct bit errors on any four data queues (DQs). If the memory chip includes an on-die error correction engine (ODECC), then by employing multi-level joint error correction of on-die ECC and system ECC, the error correction capability can be improved to correct bit errors in eight DQs. That is, if the data chip bit width is 16 bits, half-chipkill of the data chip can be achieved.

[0067] The memory module provided in this application will be further described below through specific embodiments.

[0068] Figure 1 This is a schematic diagram of a memory module 1A provided in an embodiment of this application. Figure 1 As shown, memory module 1A includes memory chipset 11 and memory chipset 12. Memory chipset 11 includes a first subgroup 110, and memory chipset 12 includes a first subgroup 120. Both first subgroups 110 and 120 are composed of six stacked memory chips, of which four are data chips. Figure 1 (The example is filled with white), and the two memory chips are ECC chips. Figure 1(Example shown in black), each data chip has a bit width of 16 bits, and each ECC chip has a bit width of 8 bits. The ratio of data chips to ECC chips is 2:1, and the bit width ratio of a single data chip to a single ECC chip is 2:1. The first subgroup 110 is connected to one memory channel, and the first subgroup 120 is connected to another memory channel. That is, a memory module 1A can be connected to two memory channels simultaneously, and different subgroups within the memory module group are connected to different memory channels. Understandably, Figure 1 The total bit width of the four data chips in the first subgroup shown is 64, which is aligned with the CPU bit width. Figure 1 The total bit width of the two ECC chips in the first subgroup shown is 16, and the ratio of the total bit width of the data chip to the total bit width of the ECC chip is 4:1.

[0069] In some embodiments, multiple memory chips in memory module 1A are stacked and packaged using RDL technology and wire bonding. For example, the pads at the center of multiple memory chips are rewired to the edges of the memory chips using RDL technology, and then the pads of the multiple memory chips are connected to the substrate leads using wire bonding to form the memory module. In some embodiments, when the number of stacked layers of memory chips in the memory module is greater than a preset value, the above-described packaging method is used. The preset value can be determined according to actual needs; for example, the preset value can be 3.

[0070] In some embodiments, the capacity of the data chip and the ECC chip in the memory chipset are equal. For example, the capacity of both the data chip and the ECC chip is 4 gigabytes (GB), 8GB, or 16GB. It should be noted that the above examples of memory chip capacity are merely illustrative, and the embodiments of this application do not limit the capacity of the memory chips.

[0071] In some embodiments, the capacity ratio of the data chip to the ECC chip in the memory chipset is 2:1. For example, the data chip capacity is 16GB and the ECC chip capacity is 8GB; or, for another example, the data chip capacity is 8GB and the ECC chip capacity is 4GB. In the above optional methods, the bit width ratio of the data chip and the ECC chip is 2:1, that is, the ratio of the number of data pins of the data chip to the number of data pins of the ECC chip is 2:1. Setting the capacity ratio of the data chip and the ECC chip to 2:1 also ensures the correspondence between the number of data pins and the memory chip capacity. That is, the larger the capacity of the memory chip, the more data pins it has, and the more bits of data can be written or read from the memory chip at one time, thereby avoiding the waste of memory chip capacity.

[0072] It should be noted that, Figure 1This is an example of the vertical positional relationship and stacking number of multiple memory chips in a memory chipset. It is not a front view, side view, or top view of the memory module and does not limit the actual product form of the memory module 1A.

[0073] It should be noted that the dimensions of the multiple memory chips in the first subgroup may be the same or different. In some embodiments, the dimensions of the multiple memory chips in the first subgroup are all the same, thereby shortening the length of the bonding wire and improving signal quality. This application does not limit the dimensions of the memory chips in the same memory chip group.

[0074] It should be noted that, Figure 1 The memory module 1A shown is illustrated using an example of a memory module comprising two memory chip groups. In some embodiments, the memory module includes one memory chip group. In other embodiments, the memory module includes two or more memory chip groups. This application does not limit the number of memory chip groups included in the memory module. Two or more memory chip groups can be stacked in a single column or multiple columns, and this application does not limit this arrangement. "Single column" means that in a top view, two or more memory chip groups are arranged in a single column; "multiple columns" means that in a top view, two or more memory chip groups are arranged in multiple columns.

[0075] It should be noted that, Figure 1 The memory module 1A shown is illustrated using a memory chipset including a first subgroup as an example. In some embodiments, the memory chipset includes multiple first subgroups. The stacking order of the data chips and memory chips in the multiple first subgroups can be the same or different. For example, in some embodiments, the stacking order of the data chips and ECC chips in the multiple first subgroups is the same; for instance, each first subgroup is stacked in the following order from bottom to top: data chip, data chip, ECC chip, data chip, data chip, ECC chip. In the above embodiments, since multiple chips in multiple subgroups with the same composition adopt the same stacking order, the orderliness of the memory module packaging can be guaranteed, thereby reducing processing errors in the memory module, improving the yield of the memory module, and facilitating the CPU's identification of the data chips and ECC chips. It should be noted that the above... Figure 1 The stacking order of memory chips shown is merely exemplary, and this application does not limit the stacking order. For example, in other embodiments, at least two of the multiple first subgroups of the memory chip set have different stacking orders for the data chips and ECC chips.

[0076] Figure 1The memory module 1A shown is illustrated using an example where the first subgroup includes a 16-bit data chip and an 8-bit ECC chip. In some embodiments, the first subgroup includes a 32-bit data chip and a 16-bit ECC chip. For example, Figure 2 This is a schematic diagram of a memory module 2A provided in an embodiment of this application. Figure 2 As shown, memory module 2A includes a memory chip group 21, which includes a first subgroup 211 and a first subgroup 212. Each first subgroup consists of three stacked memory chips, of which two are data chips. Figure 2 (Examples are shown with white filling), one memory chip is an ECC chip ( Figure 2 (Examples are shown in black). Each data chip has a bit width of 32 bits, and each ECC chip has a bit width of 16 bits. The ratio of data chips to ECC chips is 2:1, and the bit width ratio of a single data chip to a single ECC chip is 2:1. The first subgroup 211 is connected to one memory channel, and the first subgroup 212 is connected to another memory channel. That is, one memory module 2A can be connected to two memory channels simultaneously, and different subgroups within the first memory module 2A are connected to different memory channels. Understandably, Figure 2 The total bit width of the two data chips in the first subgroup shown is 64, which is aligned with the CPU bit width. Figure 2 The total bit width of one ECC chip in the first subgroup shown is 16, and the ratio of the total bit width of the data chip to the total bit width of the ECC chip is 4:1.

[0077] The above Figure 2 Compared to memory module 1A, memory module 2A uses memory chips with a larger bit width. The number of memory chips in a first subgroup is reduced, and the number of stacking layers formed by stacking the memory chips in a first subgroup is also reduced. Therefore, with the same number of stacking layers as memory module 1A, memory module 2A can stack more first subgroups. Since one first subgroup is connected to one memory channel, memory module 2A can be connected to more memory channels with the same number of stacking layers. In other words, stacking the same number of memory chips on the same area can connect to more memory channels, thus increasing the number of memory channels per unit area, which is to say, increasing the memory channel density. Furthermore, since the number of memory channels connected to a memory module increases, the efficiency of the processor accessing the memory module can be improved.

[0078] In some embodiments, the memory module 2A is packaged using wire bonding. In some embodiments, when the number of stacked layers of the memory chips in the memory module is less than or equal to a preset value, wire bonding is used for packaging. The preset value can be determined according to actual needs; for example, the preset value can be 3. In the above embodiments, when the number of stacked layers of the memory chips is small, wire bonding is used directly for packaging without the need for RDL technology, which reduces the complexity of the packaging process and saves costs.

[0079] In some embodiments, the capacity of the data chip and the ECC chip in the memory chipset are equal. For example, the capacity of both the data chip and the ECC chip is 2GB, 4GB, 8GB, or 16GB. It should be noted that the above examples of memory chip capacity are merely illustrative, and the embodiments of this application do not limit the capacity of the memory chips.

[0080] In some embodiments, the capacity ratio of the data chip to the ECC chip in the memory chipset is 2:1. For example, the data chip capacity is 16GB and the ECC chip capacity is 8GB; another example is that the data chip capacity is 8GB and the ECC chip capacity is 4GB; yet another example is that the data chip capacity is 4GB and the ECC chip capacity is 2GB. In the above optional methods, the bit width ratio of the data chip and the ECC chip is 2:1, that is, the ratio of the number of data pins of the data chip to the number of data pins of the ECC chip is 2:1. Setting the capacity ratio of the data chip and the ECC chip to 2:1 also ensures the correspondence between the number of data pins and the memory chip capacity. That is, the larger the capacity of the memory chip, the more data pins it has, and the more bits of data can be written or read from the memory chip at one time, thereby avoiding the waste of memory chip capacity.

[0081] It should be noted that, Figure 2 This is an example of the vertical positional relationship and stacking number of multiple memory chips in a memory chipset. It is not a front view, side view, or top view of the memory module and does not limit the actual product form of the memory module 2A.

[0082] Figure 1 and Figure 2 The memory module shown is illustrated using a 2:1 ratio of the bit width of a single data chip to the bit width of a single ECC chip as an example. In some embodiments, the ratio is 4:1, for example, Figure 3 This is a schematic diagram of a memory module provided in an embodiment of this application. For example... Figure 3As shown, memory module 3A includes memory chipset 31 and memory chipset 32. Memory chipset 31 includes a second subgroup 310, and memory chipset 32 ​​includes a second subgroup 320. Both second subgroups 310 and 320 are composed of four stacked memory chips, of which two are data chips. Figure 3 (The example is filled with white), and the two memory chips are ECC chips. Figure 3 (Example shown in black), each data chip has a bit width of 32 bits, and each ECC chip has a bit width of 8 bits. The ratio of data chips to ECC chips is 1:1, and the bit width ratio of a single data chip to a single ECC chip is 4:1. The second subgroup in memory chip group 31 is connected to one memory channel, and the second subgroup in memory chip group 32 is connected to another memory channel. That is, a memory module 3A can be connected to two memory channels simultaneously, and different subgroups in a memory module 3A are connected to different memory channels. Understandably, Figure 3 The total bit width of the two data chips in one of the second subgroups shown is 64, which is aligned with the CPU bit width. Figure 3 The total bit width of the two ECC chips in one of the second subgroups shown is 16, and the ratio of the total bit width of the data chip to the total bit width of the ECC chip is 4:1.

[0083] In the above embodiments, the above Figure 3 Compared to memory module 1A, memory module 3A uses a 32-bit memory chip, which means fewer memory chips are connected to a memory channel. This means that the number of memory chips in a subgroup is reduced. Therefore, there are fewer stacked layers of memory chips in the memory module. Under the same package size, the engineering difficulty of packaging is reduced, the length of the bonding wire is shortened, and the signal quality is better. In addition, the capacity of the memory module is flexible and configurable.

[0084] In some embodiments, the multiple memory chips in memory module 3A are packaged based on RDL technology and wire bonding.

[0085] In some embodiments, the capacity of the data chip and the ECC chip in the memory chipset are equal. For example, the capacity of both the data chip and the ECC chip is 4GB, 8GB, or 16GB. It should be noted that the above examples of memory chip capacity are merely illustrative, and the embodiments of this application do not limit the capacity of the memory chips.

[0086] In some embodiments, the capacity ratio of the data chip to the ECC chip in the memory chipset is 4:1. For example, the data chip has a capacity of 16GB, and the ECC chip has a capacity of 4GB. In the above optional method, the bit width ratio of the data chip and the ECC chip is 4:1, that is, the ratio of the number of data pins of the data chip to the number of data pins of the ECC chip is 4:1. Setting the capacity ratio of the data chip and the ECC chip to 4:1 also ensures the correspondence between the number of data pins and the capacity of the memory chip. That is, the larger the capacity of the memory chip, the more data pins it has, and the more bits of data can be written or read from the memory chip at one time, thereby avoiding the waste of memory chip capacity.

[0087] It should be noted that, Figure 3 This is an example of the vertical positional relationship and stacking number of multiple memory chips in a memory chipset. It is not a front view, side view, or top view of the memory module and does not limit the actual product form of the memory module 2A.

[0088] It should be noted that, Figure 3 The memory module 3A shown is illustrated using an example of a memory module comprising two memory chip groups. In some embodiments, the memory module includes one memory chip group. In other embodiments, the memory module includes two or more memory chip groups. This application does not limit the number of memory chip groups included in the memory module. Two or more memory chip groups can be stacked in a single column or multiple columns, and this application does not limit this arrangement. "Single column" means that in a top view, two or more memory chip groups are arranged in a single column; "multiple columns" means that in a top view, two or more memory chip groups are arranged in multiple columns.

[0089] It should be noted that, Figure 3 The memory module 3A shown is illustrated using a memory chipset comprising one second subgroup as an example. In some embodiments, the memory chipset comprises multiple second subgroups. The stacking order of the data chips and memory chips in the multiple second subgroups can be the same or different. For example, in some embodiments, the stacking order of the data chips and ECC chips in the multiple second subgroups is the same; for instance, each second subgroup is stacked in the order of data chip, ECC chip, data chip, ECC chip from bottom to top. In the above embodiments, since multiple chips in multiple subgroups with the same composition adopt the same stacking order, the orderliness of the memory module packaging can be guaranteed, thereby reducing processing errors in the memory module, improving the yield of the memory module, and facilitating the CPU's identification of the data chips and ECC chips. It should be noted that the above... Figure 3The stacking order of memory chips shown is merely exemplary, and this application does not limit the stacking order. For example, in other embodiments, at least two of the multiple second subgroups of the memory chip set have different stacking orders for their data chips and ECC chips.

[0090] It should be noted that the above memory modules 1A, 2A and 3A are illustrated using the example that different memory chipsets in the same memory module include the same number of subgroups. In some embodiments, different memory chipsets in the same memory module may include different numbers of subgroups.

[0091] It should be noted that the memory modules 1A, 2A, and 3A described above are illustrated using the example of different memory chip groups within the same memory module having the same composition. In some embodiments, the compositions of different memory chip groups within the same memory module may differ. For example, a memory module may include two memory chip groups, one of which is formed by stacking a first subgroup, meaning that the bit width ratio of the data chip to the ECC chip used in this memory chip group is 2:1; the other memory chip group is formed by stacking a second subgroup, meaning that the bit width ratio of the data chip to the ECC chip used in this memory chip group is 4:1.

[0092] It should be noted that the memory modules 1A, 2A and 3A described above are based on the example of a memory chip group in a memory module that includes only the first subgroup or only the second subgroup. In some embodiments, when the memory chip group includes multiple subgroups, a memory chip group includes both the first subgroup and the second subgroup.

[0093] It should be noted that the memory modules 1A, 2A, and 3A described above are illustrated using the example of multiple data chips and multiple ECC chips in a subgroup of the memory module having equal bit widths. In some embodiments, the bit widths of multiple data chips and multiple ECC chips in a subgroup may not be equal. For example, in some embodiments, the memory chip group includes at least one third subgroup, and each third subgroup is composed of 3 data chips and 1 ECC chip stacked together, wherein 1 data chip has a bit width of 32 bits, 2 data chips have a bit width of 16 bits, and 1 ECC chip has a bit width of 16 bits.

[0094] In the aforementioned embodiments, the use of memory chips with a larger bit width compared to related technologies reduces the number of memory chips connected to a single memory channel, thereby reducing the number of CA lines in the memory channel and lowering the CA signal load on the processor. The number of memory chips connected to a single memory channel is reduced by at least 40% compared to related technologies. The processor can directly drive the clock signal and CA signal to access the memory module without needing to drive it through an RCD relay. Since the number of memory chips connected to a single memory channel is significantly reduced and no RCD configuration is required, product costs can be lowered. Furthermore, the memory module is packaged using stacked memory chips, which significantly reduces the size of the memory module compared to related technologies where each memory chip is flat-mounted on the PCB to form a memory module or directly flat-mounted on the motherboard. The memory module can be installed closer to the processor, the memory channel connecting the processor and the memory module is shorter, the processor accesses the memory module more efficiently, and the signal quality is better. In addition, the memory module provided in this application embodiment can be mounted on the motherboard flat-mounted without using a PCB board and connectors (sockets), improving the signal link environment and further reducing costs.

[0095] The memory module provided in this application has a base area greater than 10mm × 10mm and less than 20mm × 20mm. Compared with related technologies, this significantly reduces the size of the memory module, allowing for a greater number of memory channels within the same area. For example, taking the memory module accessed through two memory channels in this application as an example, the area occupied by 8 memory channels in the related technology's RDIMM can accommodate more than 100 memory channels, significantly improving the bandwidth capacity density per unit area. Furthermore, as... Figure 4 As shown, Figure 4 This is a simulation result diagram of a CA eye-mask provided in an embodiment of this application. Figure 4 The upper middle figure shows the simulation results of the RDIMM memory module in the related technology, where the eye height is 164mV and the eye width is 272ps; Figure 4 The lower figure shows the simulation results of the memory module provided in this application embodiment, where the eye height is 195mV and the eye width is 288ps. It can be seen that the simulation results of the memory module provided in this application embodiment show that both the eye height and eye width indicators are higher than those of related technologies, indicating that this application can directly drive the CA signal through the CPU without relying on an RCD, and the CA signal quality is superior. Since it does not rely on an RCD, each memory module saves at least $10 in cost.

[0096] Based on the memory module described above, a computing device provided in the embodiments of this application will be further described below.

[0097] The computing devices provided in this application can be applied to scenarios such as AI large-scale model training, high-performance computing, graphics processing, data centers, virtual reality (VR) and augmented reality (AR), autonomous driving, and professional image processing. This application does not limit the application scenarios of the computing devices. Different configurations of computing devices can be selected according to the application scenario. For example, in an image processing scenario, if the trained AI large-scale model is used to complete image and video processing and analysis, the computing device used in this scenario may include a GPU and at least one memory module group, with a total bandwidth of 4×1024 for the at least one memory module group. As another example, in an AI large-scale model training scenario, where a large-scale neural network is trained to complete image recognition, speech recognition, or natural language processing, the computing device used in this scenario may include a TPU and at least one memory module group, with a total bandwidth of 4×1024 for the at least one memory module group. It should be noted that the above descriptions of the configurations of computing devices selected for different scenarios are merely exemplary, and this application does not limit them.

[0098] Figure 5 This is a schematic diagram of a computing device 5B provided in an embodiment of this application. Figure 5 As shown, the computing device 5B includes multiple first memory module groups 51-54, a processor 55, and a first substrate 56. The first memory module groups 51-54 and the processor 55 are mounted on the first substrate 56. Multiple memory modules from first memory module groups 51 and 52 are arranged in a row and mounted on one side of the processor 55 on the first substrate 56, while multiple memory modules from first memory module groups 53 and 54 are arranged in a row and mounted on the other side of the processor 55 on the first substrate 56.

[0099] It should be noted that, Figure 5 The computing device 5B shown is illustrated as an example of a computing device including multiple memory module groups. In some embodiments, the computing device includes one memory module group. This application does not limit the number of memory module groups in the computing device. In other embodiments, the computing device includes at least one memory module group and at least one HBM particle.

[0100] Each first memory module group comprises 8 memory modules. Taking memory modules 1A, 2A, or 3A as examples, each memory module independently accesses two memory channels simultaneously. Each memory channel has a bit width of 64 bits, so the bit width of one first memory module group is 8 × 2 × 64 = 1024 bits. Since the bit width of an HBM chip is also 1024 bits, then... Figure 5 The bit width of one of the first memory module groups shown is equal to the bit width of one HBM chip. Taking the memory chip specification in the memory module as DDR5-6400 as an example, Figure 5The bandwidth of a single memory module is 102.4 GBps, and the total bandwidth of a first memory module group is 819.2 GBps, which is equal to the bandwidth of one HBM3 chip. The total bandwidth of four first memory module groups is equal to the total bandwidth of four HBM3 chips. It should be noted that the above description of the bandwidth of memory module groups is merely exemplary. In some embodiments, the total bandwidth of multiple memory module groups is equal to the bandwidth of one HBM chip, or the bandwidth of one memory module group is equal to the bandwidth of one HBM chip. This application does not limit the bandwidth of a memory module group. Furthermore, the capacity of a memory module group can be configured to be no less than the capacity of one HBM chip, or the capacity of multiple memory module groups can be no less than the capacity of one HBM chip, by configuring the number of memory modules in the first memory module group and configuring the capacity of a single memory module. This application does not limit the specific configuration method.

[0101] The above Figure 5 The computing device shown provides a bandwidth no less than that of HBM chips by arranging multiple memory modules. The stacking process of memory chips in the memory modules is simple, and the cost of the memory modules is low. Compared with the expensive 3D TSV stacking process of HBM, it can reduce engineering difficulty and save costs. The cost of 16GB HBM is about $320, while the cost of a memory module of the same capacity is only $48. Larger capacity can be obtained at a lower cost than HBM, which can meet the bandwidth and capacity requirements of AI and large-scale computing, and ensure the availability of high-bandwidth memory.

[0102] The processor 55 can be a general-purpose CPU, GPU, switching module processor unit (SMPU), NPU, microprocessor, TPU, or GPGPU, etc. This embodiment does not limit the type of processor. For example, the processor 55 accesses memory modules in multiple memory module groups through multiple memory interfaces, and these memory interfaces conform to the DDR protocol. The memory interfaces include a DMC and a memory port network layer (DDRPHY). Figure 6 As shown, Figure 6 This is a schematic diagram of a memory interface provided in an embodiment of this application. Figure 6 It includes 16 memory interfaces, one of which is used to access a subgroup of the memory module based on a memory channel (one subgroup is connected to one memory channel, and different subgroups are connected to different memory channels). Figure 6Taking a memory module comprising two subgroups as an example, the processor accesses the memory module through two memory interfaces. Exemplarily, the process of the processor accessing the memory module through the memory interface includes: the processor executes a data read / write instruction, generates a read / write request corresponding to the instruction, and sends the read / write request to the memory management unit; the memory management unit first converts the virtual address carried in the read / write request into a physical address, and then sends the read / write request to the bus; the bus sends the data read / write request to the memory controller responsible for accessing the physical address; the memory controller, based on the data read / write request, reads data from the memory module corresponding to the physical address through the DDR PHY, or writes data to the memory module corresponding to the physical address through the DDR PHY, thereby completing the access to the memory module. Since the processor and the memory module communicate based on the memory interface, compared to the HBM interface, the latency is lower, resulting in lower power consumption due to latency, improved data read / write efficiency, and a more significant improvement in random access efficiency.

[0103] In some embodiments, the first substrate is a UHD (Ultra-High Definition) substrate. In the above embodiments, a UHD substrate is used to mount the processor and memory module. UHD can meet the requirements of deploying numerous lines in scenarios with large bit widths. Furthermore, the processor and memory module can be connected via UHD without requiring complex and expensive packaging processes such as CoWoS, thus reducing engineering complexity and saving costs. In some embodiments, the first substrate is of other types; this application does not limit the type of the first substrate.

[0104] In some embodiments, the memory modules in the first memory module group are mounted in a flat-mount manner.

[0105] In some embodiments, the computing device further includes an integrated heatsink covering the plurality of first memory module groups and the accelerator. The integrated heatsink dissipates heat through at least one of liquid cooling and air cooling. In the above embodiments, the integrated heatsink provides better heat dissipation for the processor and memory modules. Figure 7 This is an installation diagram of a computing device 5B provided in an embodiment of this application, as shown below. Figure 7 As shown, computing device 5B is mounted to the PCB via a connector (socket).

[0106] It should be noted that the above Figure 5 The computing device 5B shown includes four first memory module groups, which is merely exemplary. The number of first memory module groups can be configured according to the number of memory channels of the processor, and this application embodiment does not limit this.

[0107] It should be noted that the above Figure 5The eight memory modules in each memory module group shown are merely exemplary, and the number of memory modules in each first memory module group can be set according to actual needs. For example, in some embodiments, a memory module group includes 12 memory modules, and the total bandwidth of a memory module group is 1.2 TBps, while the bandwidth of an HBM3e chip is 1 TBps. Therefore, the total bandwidth of a memory module group is greater than that of an HBM3e chip. Furthermore, the number of memory modules in a first memory module group can be configured based on the processor's memory channel data, and this application does not limit the number of first memory module groups included in the computing device.

[0108] It should be noted that the DDR5-6400 specification of the memory chip mentioned above is merely exemplary, and this application does not limit the specification of the memory chip in the memory module. In some embodiments, the memory chip is DDR5 or DDR6, which have higher read and write efficiency, resulting in a higher total bandwidth for a memory module group. As the DDR standard evolves, memory chips with higher read and write efficiency can be used, thereby allowing the memory module group to replace higher-standard HBM chips.

[0109] It should be noted that the specifications of the memory modules used in the computing device can be determined according to actual needs. For example, in some embodiments, a smaller computing device is required. Since the size of the memory module 2A is smaller than that of memory modules 1A and 3A, the memory module in the computing device adopts the specifications of the memory module 2A to meet the size requirements of the computing device. In other embodiments, there are no requirements for the size of the computing device. Since the memory chips with a bit width of 16 and a bit width of 8 included in the memory module 1A do not need to be customized separately like the memory chip with a bit width of 32, the memory module in the computing device adopts the specifications of the memory module 1A, thereby reducing the cost of the computing device.

[0110] It should be noted that the above Figure 5 The arrangement of multiple memory modules in each memory module group in a single column shown is merely exemplary. In some embodiments, multiple memory modules in each first memory module group are arranged in multiple columns for installation. This application does not limit this practice.

[0111] It should be noted that the above Figure 5 The multiple first memory module groups shown are mounted on the first substrate at positions on both sides of the processor. In some embodiments, the multiple first memory module groups are mounted on one side of the processor or around the processor. This application does not limit the mounting position of the first memory module groups on the first substrate.

[0112] Figure 8This is a schematic diagram of a computing device 8B provided in an embodiment of this application. Figure 8 As shown, the computing device 8B includes eight second memory module groups 81-88, a processor 89, and a first substrate 810. The second memory module groups 81-88 and the processor 89 are mounted on the first substrate 810. Each second memory module group includes four memory modules, and each second memory module group also includes a second substrate for mounting the memory modules in the second memory module group. Each second memory module group is installed in a pluggable manner. Second memory module groups 81-84 are mounted on one side of the processor 89 on the first substrate 810, and second memory module groups 85 and 88 are mounted on the other side of the processor 89 on the first substrate 810.

[0113] Each second memory module group comprises four memory modules. Taking memory modules 1A, 2A, or 3A as examples, each memory module independently accesses two memory channels simultaneously. Each memory channel has a bit width of 64 bits. Therefore, the bit width of two second memory module groups is 2 × 4 × 2 × 64 = 1024 bits. Since the bit width of an HBM chip is also 1024 bits, ... Figure 8 The bit width of the two second memory module groups shown is equal to the bit width of one HBM chip. Taking the memory chip specification in the memory module as DDR5-6400 as an example, Figure 8 The bandwidth of one memory module is 102.4 GBps. The total bandwidth of two second memory module groups is 819.2 GBps, which is equal to the bandwidth of one HBM3 chip. The total bandwidth of eight second memory module groups is equal to the total bandwidth of four HBM3 chips.

[0114] The above Figure 8 Compared to computing device 5B, the provided computing device 8B has a pluggable second memory module group. When a memory module is damaged, only the second memory module group containing the damaged memory module can be replaced, without replacing the entire computing device, which improves the convenience of maintenance and reduces maintenance costs.

[0115] In some embodiments, the second memory module group is mounted on the first substrate 810 via a connector (socket), thereby ensuring that the second memory module group is pluggable.

[0116] In some embodiments, the computing device further includes an integrated heatsink covering the at least one memory module group and the accelerator. In some embodiments, the integrated heatsink dissipates heat through at least one of liquid cooling and air cooling. In the above embodiments, using an integrated heatsink to dissipate heat from the processor and memory modules results in better heat dissipation. Figure 9 This is an installation diagram of a computing device 5B provided in an embodiment of this application, as shown below. Figure 9 As shown, computing device 8B is mounted to the PCB board via a connector (socket).

[0117] It should be noted that the above Figure 8 The inclusion of four memory modules in each second memory module group is merely exemplary; the number of memory modules in each second memory module group can be set according to actual needs, and this application embodiment does not limit this. Furthermore, the number of memory modules in multiple second memory module groups can be the same or different, and this application embodiment does not limit this. Multiple memory modules in a second memory module group can be arranged in one column or multiple columns on the second substrate, and this application embodiment does not limit this.

[0118] It should be noted that the above Figure 8 Therefore, the second memory module group is pluggable by mounting multiple memory modules on the second substrate and then mounting the second substrate on the first substrate in a pluggable manner. In other embodiments, the memory modules in the second memory module group are mounted on the first substrate in a flat-insertion manner without first mounting them on the second substrate. This allows the computing device to be repaired at the memory module level. Compared with the computing device 5B, which is repaired at the second memory module group level, this further improves the convenience of repair and further reduces the repair cost.

[0119] It should be noted that the above Figure 8 In some embodiments, the mounting positions of the multiple second memory module groups on the first substrate 810 are on both sides of the processor. However, in other embodiments, the mounting positions are on one side of the processor or around the processor. This application does not limit the mounting positions of the second memory module groups on the first substrate.

[0120] It should be noted that the above-described computing device 5B is illustrated with the example of all memory modules being the first memory module group, and the above-described computing device 8B is illustrated with the example of all memory modules being the second memory module group. In some embodiments, the memory module group in the computing device includes both the first memory module group and the second memory module group, and the embodiments of this application do not limit this.

[0121] This application provides a computing device, which includes multiple computing units and a third substrate on which the multiple computing units are mounted. The third substrate can be a PCB. The third substrate and a first substrate adopt a mother-daughter board structure, with the first substrate being the daughter board and the third substrate being the mother board.

[0122] This application provides a computing device cluster, which includes at least one computing device. Each computing device includes multiple computing units and a third substrate, on which the multiple computing units are mounted. The third substrate can be a PCB. The third substrate and a first substrate adopt a mother-daughter board structure, with the first substrate being the daughter board and the third substrate being the mother board.

[0123] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with substantially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of various examples, a first memory module group can be referred to as a second memory module group, and similarly, a second memory module group can be referred to as a first memory module group. Both the first and second memory module groups can be node memory module groups, and in some cases, they can be separate and distinct memory module groups.

[0124] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. The terms "system" and "network" are often used interchangeably.

[0125] It should also be understood that the term "if" can be interpreted as meaning "when" or "upon" or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrase "if determination..." or "if detection [the stated condition or event]" can be interpreted as meaning "when determination..." or "in response to determination..." or "when detection [the stated condition or event]" or "in response to detection [the stated condition or event]."

[0126] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0127] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A computing device, comprising: include: At least one memory module group, each memory module group includes at least one memory module, each memory module includes at least one memory chip group, each memory chip group includes at least one subgroup, different subgroups are connected to different memory channels, each subgroup is composed of multiple memory chips stacked together, and the bit width of the memory chip in each subgroup includes a first bit width and a second bit width, the first bit width is N times the second bit width, the second bit width is greater than or equal to 8 bits, and N is a positive integer greater than 1; The processor is configured to access memory modules in the at least one group of memory modules via at least one memory interface, wherein different memory interfaces are used to access different memory modules, and the memory interfaces conform to the Double Data Rate Synchronous Dynamic Random Access Memory (DDR) protocol.

2. The computing device of claim 1, wherein, The memory interface includes a memory controller (DMC) and a memory port physical layer (DDR PHY).

3. The computing device of claim 1, wherein, The computing device includes at least one first memory module group, each of the first memory module groups being mounted in a flat-mount manner.

4. The computing device of claim 1, wherein, The computing device includes at least one second memory module group, each of the second memory module groups being installed in a pluggable manner.

5. The computing device of claim 4, wherein, The processor and the at least one memory module group are mounted on a first substrate. Each second memory module group further includes a second substrate on which the memory modules of the second memory module group are mounted. The second substrate is mounted on the first substrate in a pluggable manner.

6. The computing device of any of claims 1 to 4, wherein, The processor and the at least one memory module group are mounted on a first substrate. The at least one memory module group is mounted on the first substrate in the following positions: distributed on both sides of the accelerator, or surrounding the accelerator, or distributed on one side of the accelerator.

7. The computing device of any of claims 1 to 6, wherein, The computing device also includes an integrated heat sink that covers the at least one memory module group and the accelerator.

8. The computing device of claim 7, wherein, The integrated heat sink dissipates heat through at least one of liquid cooling and air cooling.

9. The computing device of any of claims 1-8, wherein, The processor and the at least one memory module group are mounted on a first substrate, which is an ultra-high density board (UHD).

10. The computing device of any of claims 1 to 9, wherein, The total capacity of at least one of the memory module groups is not less than the capacity of a high-bandwidth memory HBM particle, and the total bandwidth of at least one of the memory module groups is not less than the bandwidth of a HBM particle.

11. The computing device of any of claims 1 to 10, wherein, The processor is any one of a central processing unit (CPU), a graphics processing unit (GPU), a network processing unit (NPU), a tensor processor (TPU), or a general-purpose graphics processing unit (GPGPU).

12. The computing device of any of claims 1-11, wherein, Each of the subgroups includes at least one data chip and at least one error correction code (ECC) chip.

13. The computing device of claim 12, wherein, The memory chipset includes at least one first subgroup, wherein the bit width ratio of a single data chip to a single error correction code chip in the first subgroup is 2:1, and the number ratio of data chips to ECC chips in the first subgroup is 2:

1.

14. The computing device of claim 12, wherein, The memory chipset includes at least one second subgroup, wherein the bit width ratio of a single data chip to a single ECC chip in the second subgroup is 4:1, and the number ratio of data chips to ECC chips in the second subgroup is 1:

1.

15. The computing device of claim 12, wherein, The memory chipset includes multiple subgroups, including at least one first subgroup and at least one second subgroup. In the first subgroup, the bit width ratio of a single data chip to a single ECC chip is 2:1, and the number ratio of data chips to ECC chips in the first subgroup is 2:

1. In the second subgroup, the bit width ratio of a single data chip to a single ECC chip is 4:1, and the number ratio of data chips to ECC chips in the second subgroup is 1:

1.

16. The computing device of any of claims 1 to 15, wherein, Multiple memory chips in each of the memory chipsets are stacked based on redistribution layer (RDL) technology and wire bonding.

17. The computing device of any of claims 1 to 15, wherein, The memory chips in each of the memory chipsets are stacked based on wire bonding.

18. The computing device of any of claims 1-17, wherein, The memory module does not include a registered clock driver (RCD).

19. The computing device according to any one of claims 1 to 17, further comprising: The first substrate is used to mount the processor and at least one memory module group.

20. A memory module bank, comprising: include: At least one memory module; The second substrate is used to mount the at least one memory module in a pluggable manner.

21. A computing device, characterized in that, The computing device includes multiple computing units and a third substrate; The plurality of computing devices are mounted on the third substrate.

22. A computing device cluster, characterized in that, The computing device cluster includes at least one computing device, and each computing device includes multiple computing units and a third substrate; The plurality of computing devices are mounted on the third substrate.

Images