Computing apparatus, memory module group, computing device and computing device cluster

Abstract

The present application relates to the technical field of chip packaging, and provides a computing apparatus, a memory module group, a computing device, and a computing device cluster. The computing apparatus comprises at least one memory module group, a processor, and a mainboard. Each memory module in the memory module group is formed by stacking memory chips having a bit width of at least 8 bits, the size is small, the stacking process is simple, each memory module is accessed through at least one memory channel, and the density of the memory channels in unit area is large, so that the memory module group in the computing apparatus can provide large bandwidth to meet the requirements of AI or large-scale computing scenarios on bandwidth, without adopting a HBM having high process difficulty and high costs, thereby reducing engineering difficulty and saving costs. In addition, the processor and the memory modules communicate on the basis of a DRAM interface, and compared with an HBM interface, there is less delay, resulting in lower power consumption generated by the delay, thereby improving the data read-write efficiency, and significantly improving the random access efficiency.

Description

Computing device, memory module group, computing device, and computing device cluster

[0001] The present application claims priority to the Chinese patent application No. 202410585671.2, filed on May 11, 2024, and entitled "Computing device, memory module group, computing device, and computing device cluster", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the chip packaging technical field, in particular to a computing device, a memory module group, a computing device, and a computing device cluster. BACKGROUND

[0003] The core architecture of artificial intelligence (AI) is an AI module, and the AI module includes an accelerator such as a graphics processing unit (GPU) or a neural network processing unit (NPU) and a plurality of high-bandwidth memory (HBM) particles. Each HBM particle is stacked by a 3D heterogeneous multi-layer vertical stack of DRAM and a logic chip that provides physical connection, through silicon via (TSV), and design for test (DFT) units. Each HBM particle provides a plurality of channels for the accelerator, thereby meeting the demand for large bandwidth in the AI scenario.

[0004] In related technologies, the multi-layer DRAM in the HBM particle is stacked by TSV technology. However, the 3D TSV technology has high process difficulty, long time consumption, and must be combined with the main chip by using the chip on wafer on substrate (CoWoS) process, which has a certain yield loss, resulting in high cost of the AI module. SUMMARY

[0005] Embodiments of the present application provide a computing device, a memory module group, a computing device, and a computing device cluster, which can reduce the process difficulty of the computing device and save costs. The technical solution is as follows.

[0006] In a first aspect, a computing device is provided, which includes a processor, at least one memory module group, and a first substrate.

[0007] Each memory module in the memory module group is stacked by memory chips with a bit width of at least 8 bits, has a small size and a simple stacking process, and is connected to at least one memory channel, so that the density of the memory channels per unit area is large. Since the bandwidth is equal to the product of the bit width and the data transmission rate, when the density of the memory channels per unit area is large, one memory module can provide a larger bit width, and under the condition that the data transmission rate is unchanged, one memory module can provide a larger bandwidth. Therefore, at least one memory module group composed of memory modules can provide a bandwidth not less than that of one HBM particle, and can be used as a substitute for HBM particles to meet the demand for large bandwidth in AI or large-scale computing scenarios without using HBM with high process difficulty and high cost. Therefore, the engineering difficulty can be reduced and the cost can be saved. In addition, the processor and the memory module communicate based on a memory interface, and compared with an HBM interface, the delay is smaller, the power consumption caused by the delay is smaller, the data read-write efficiency is improved, and the efficiency of random access is improved more obviously.

[0008] The memory module is a device providing data storage function packaged by a plurality of memory chips. The memory module can be installed on a mainboard, one memory module is connected to at least one memory channel, and the processor can access the memory module through the at least one memory channel to store data in the memory module or read data from the memory module.

[0009] The processor accesses the memory modules in the at least one memory module group through at least one memory interface, and different memory interfaces are used to access different memory modules.

[0010] In a possible implementation, the memory interface includes a dynamic memory controller (DMC) and a double data rate physical layer (DDR PHY).

[0011] In the above computing device, the processor and the memory module communicate based on a memory interface, and compared with an HBM interface, the delay is smaller, the power consumption caused by the delay is smaller, the data read-write efficiency is improved, and the efficiency of random access is improved more obviously.

[0012] In a possible implementation, the computing device includes at least one first memory module group, and the memory modules in each first memory module group are installed in a flat manner.

[0013] In the above computing device, the memory modules in the first memory module group are installed in a flat manner, so that the installation method is simple, the time consumption is short, and the packaging efficiency of the computing device can be improved.

[0014] In a possible implementation, the computing device comprises at least one second memory module group, each second memory module group being installed in a pluggable manner.

[0015] In the computing device, the memory modules in the second memory module group are installed in a pluggable manner, and when a memory module is damaged, only the second memory module group in which the damaged memory module is located needs to be replaced, without the need to replace the entire computing device, thereby improving the convenience of maintenance and reducing the maintenance cost.

[0016] In a possible implementation, each second memory module group further comprises a second substrate on which the memory modules in the second memory module group are installed, and the second substrate is installed on the first substrate in a pluggable manner.

[0017] In a possible implementation, the installation positions of the at least one memory module group on the first substrate are distributed on both sides of the accelerator, or surround the accelerator, or are distributed on one side of the accelerator.

[0018] In a possible implementation, the computing device further comprises an integrated heat sink covering the at least one memory module group and the accelerator.

[0019] In the computing device, the processor and the memory modules are cooled by the integrated heat sink, and the cooling effect is better.

[0020] In a possible implementation, the integrated heat sink is cooled by at least one of liquid cooling and air cooling.

[0021] In a possible implementation, the first substrate is an ultra high density particle board (UHD).

[0022] In the computing device, the processor and the memory modules are installed by using the UHD, the UHD can meet the layout requirement of a large number of lines in a large bit width scenario, and the processor and the memory modules can be connected by the UHD without the need to use a packaging process such as CoWoS, which is difficult and expensive, to package the processor and the memory modules, thereby reducing the engineering difficulty and saving the cost.

[0023] In a possible implementation, the total capacity of the at least one memory module group is not less than the capacity of one high bandwidth memory (HBM) particle, and the total bandwidth of the at least one memory module group is not less than the bandwidth of one HBM particle.

[0024] The total bandwidth provided by the at least one memory module group is not less than the bandwidth provided by one HBM particle, which can meet the demand for large bandwidth in AI or large-scale computing scenarios, and the HBM with high process difficulty and high cost is not required, thereby reducing engineering difficulty and saving cost.

[0025] In a possible implementation, the processor is any one of a central processing unit (CPU), a graphics processing unit (GPU), a network processing unit (NPU), a tensor processing unit (TPU), and a general purpose computing on graphics processing units (GPGPU).

[0026] In a possible implementation, when the memory module includes a plurality of memory chip groups, each memory chip group includes the same number of memory chips.

[0027] In a possible implementation, when the memory module includes a pluralityity of memory chip groups, there are at least two first memory chip groups in the memory module, and the at least two first memory chip groups include different numbers of memory chips.

[0028] In a possible implementation, the sub-group includes at least one data chip and at least one error correcting code (ECC) chip.

[0029] In the memory module, the sub-group connected by one memory channel includes a data chip and an ECC chip, the ECC chip is used to store error correcting codes of data in the data chip, that is, redundant data of the data chip, and when data in the data chip is erroneous and the number of erroneous bits is within the error correction capability of the ECC technology, data recovery can be performed, so that the processor can continue to normally access the sub-group, avoiding interruption of the processor accessing the sub-group due to a small amount of erroneous data, and the reliability of data in the sub-group and the stability of data reading and writing can be improved.

[0030] In a possible implementation, the memory chip group includes at least one first sub-group, a bit width ratio of a single data chip to a single ECC chip in the first sub-group is 2:1, and a quantity ratio of the data chip to the ECC chip in the first sub-group is 2:1.

[0031] In the memory module, the bit width ratio of the data chip and the ECC chip is 2:1, and the quantity ratio of the data chip and the ECC chip is also 2:1, which can meet the requirement that the total bit width ratio of the data chip and the ECC chip is 4:1 in the ECC technology, and further ensure the error correction capability of the ECC technology when data error occurs.

[0032] In a possible implementation, the memory chip set comprises at least one second sub-group, and the bit width ratio of a single data chip and a single ECC chip in the second sub-group is 4:1, and the quantity ratio of the data chip and the ECC chip in the second sub-group is 1:1.

[0033] In the memory module, the bit width ratio of the data chip and the ECC chip is 4:1, and the quantity ratio of the data chip and the ECC chip is 1:1, which can meet the requirement that the total bit width ratio of the data chip and the ECC chip is 4:1 in the ECC technology, and further ensure the error correction capability of the ECC technology when data error occurs.

[0034] In a possible implementation, the memory chip set comprises a plurality of sub-groups, and the plurality of sub-groups comprise at least one first sub-group and at least one second sub-group, the bit width ratio of a single data chip and a single ECC chip in the first sub-group is 2:1, the quantity ratio of the data chip and the ECC chip in the first sub-group is 2:1, the bit width ratio of a single data chip and a single ECC chip in the second sub-group is 4:1, and the quantity ratio of the data chip and the ECC chip in the second sub-group is 1:1.

[0035] In a possible implementation, the capacity ratio of a single data chip and a single ECC chip in the first sub-group is 2:1.

[0036] In the memory module, the bit width ratio of the data chip and the ECC chip is 2:1, that is, the quantity ratio of the data pins of the data chip and the quantity ratio of the data pins of the ECC chip is 2:1, and the capacity ratio of the data chip and the ECC chip is also set to 2:1, which can ensure the correspondence between the quantity of the data pins and the capacity of the memory chip, that is, the greater the capacity of the memory chip, the more the quantity of the data pins of the memory chip, and the more the data bits written or read from the memory chip at a time, thereby avoiding the waste of the capacity of the memory chip.

[0037] In a possible implementation, when the memory chip set comprises a plurality of first sub-groups, the stacking sequence of the data chip and the ECC chip in each first sub-group is the same.

[0038] In the memory module, the plurality of chips in each first sub-group are stacked in the same stacking order, which ensures the order of the memory module package, thereby facilitating reduction of processing errors of the memory module, improving the yield of the memory module, and facilitating identification of the data chips and the ECC chips by the processor.

[0039] In a possible implementation, the capacity ratio of the single data chip to the single ECC chip in the second sub-group is 4:1.

[0040] In the memory module, the bit width ratio of the data chip to the ECC chip is 4:1, that is, the number of data pins of the data chip to the number of data pins of the ECC chip is 4:1, and the capacity ratio of the data chip to the ECC chip is also set to 4:1, which ensures the correspondence between the number of data pins and the capacity of the memory chip, that is, the greater the capacity of the memory chip, the greater the number of data pins of the memory chip, and the greater the number of data bits written or read from the memory chip at a time, thereby avoiding waste of the capacity of the memory chip.

[0041] In a possible implementation, when the memory chip group includes a plurality of second sub-groups, the stacking order of the data chip and the ECC chip in each second sub-group is the same.

[0042] In the memory module, the plurality of chips in each second sub-group are stacked in the same stacking order, which ensures the order of the memory module package, thereby facilitating reduction of processing errors of the memory module, improving the yield of the memory module, and facilitating identification of the data chip and the ECC chip by the CPU.

[0043] In a possible implementation, the memory chip group includes at least one third sub-group, each third sub-group is stacked by three data chips and one ECC chip, wherein the bit width of one data chip is 32 bits, the bit width of two data chips is 16 bits, and the bit width of one ECC chip is 16 bits.

[0044] In a possible implementation, the plurality of memory chips in each memory chip group are stacked based on a redistribution layer (RDL) technology and wire bonding.

[0045] In the method, first, the pads in the center of the plurality of memory chips are redistributed to the edges of the memory chips by the RDL technology, and then the pads of the plurality of memory chips are connected to the leads of the substrate by wire bonding, to form the memory module.

[0046] In the memory module, the pads are rewired to the edge of the memory chip by the RDL technology, so as to shorten the length of the bonding wire for connecting the pad and the lead of the substrate, and the shorter the length of the bonding wire, the higher the signal quality transmitted through the bonding wire.

[0047] In a possible implementation, the plurality of memory chips in each memory chip group are stacked in a manner of wire bonding.

[0048] In the memory module, the plurality of memory chips in the memory chip group are packaged in a manner of wire bonding directly, without the RDL technology with high process difficulty, so as to reduce the process difficulty of packaging and save the cost.

[0049] In a possible implementation, the memory module does not include a register clock driver (RCD).

[0050] In the memory module, compared with the RDIMM in the related art, the product cost can be saved, and about 10 dollars of cost can be saved for one memory module compared with one RDIMM; and the packaging size of the memory module is smaller, and the memory module can be installed closer to the processor, so as to improve the efficiency of the processor accessing the memory module and improve the signal quality.

[0051] In a second aspect, a memory module group is provided, which includes at least one memory module and a second substrate for mounting the at least one memory module, and each second memory module group is mounted in a plug-in manner.

[0052] In a third aspect, a computing device is provided, which includes a plurality of computing apparatuses and a third substrate on which the plurality of computing apparatuses are mounted.

[0053] In a fourth aspect, a computing device cluster is provided, which includes at least one computing device, and each computing device includes a plurality of computing apparatuses and a third substrate on which the plurality of computing apparatuses are mounted.

[0054] On the basis of the implementation manners of the aspects provided in the present application, further combinations can be made to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0055] FIG. 1 is a schematic diagram of a memory module 1A provided in an embodiment of the present application;

[0056] FIG. 2 is a schematic diagram of a memory module 2A provided in an embodiment of the present application;

[0057] FIG. 3 is a schematic diagram of a memory module 3A provided in an embodiment of the present application;

[0058] Figure 4 is a CA eye-mask simulation result diagram provided by an embodiment of the present application;

[0059] Figure 5 is a schematic diagram of a computing device 5B provided by an embodiment of the present application;

[0060] Figure 6 is a schematic diagram of a memory interface provided by an embodiment of the present application;

[0061] Figure 7 is a schematic diagram of the installation of a computing device 5B provided by an embodiment of the present application;

[0062] Figure 8 is a schematic diagram of a computing device 8B provided by an embodiment of the present application;

[0063] Figure 9 is a schematic diagram of the installation of a computing device 8B provided by an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will make a further detailed description of the embodiments of the present application in combination with the drawings.

[0065] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will make a further detailed description of the embodiments of the present application in combination with the drawings.

[0066] Memory channel: refers to the physical path connecting the processor and the memory module, a memory channel includes a plurality of physical lines, a physical line is connected with a data pin of a memory chip, and 1 bit is transmitted each time. The bit width of the memory channel indicates the number of bits transmitted simultaneously through the memory channel. For example, the bit width of the memory channel is 64 bits, indicating that the memory channel includes 64 physical lines, and 64 bits are transmitted simultaneously each time. A plurality of memory channels can perform data transmission in parallel, the more the number of memory channels, the more the number of bits that can be transmitted in parallel, and the higher the efficiency of the processor accessing the memory. The total bit width of the data chip connected with a memory channel is equal to the bit width of the memory channel.

[0067] Memory chip bit width: refers to the number of data pins included in the memory chip, indicating the number of bits that can be written or read at a time for the memory chip. For example, the bit width of the memory chip is 8, indicating that the memory chip includes 8 data pins, and 8 bits can be transmitted simultaneously at a time, that is, the memory chip can write or read 8 bits simultaneously at a time.

[0068] Processor bit width: refers to the amount of data that the processor reads or writes to the memory at a time through a memory channel. For example, the processor bit width is 64, indicating that the processor can write or read 64 bits from the memory at a time.

[0069] In the related art, each HBM grain is stacked by 3D heterogeneous multi-layer DRAM and logic chips providing physical connection, TSV and DFT test units. Specifically, one HBM grain is stacked by 4 layers of chips with TSV through holes and micro bumps. The TSV technology requires high aspect ratio (10:1) etching and filling. Each DRAM chip needs to be thinned to within 50 um by chemical mechanical polishing (CMP) when stacked. Such a thin silicon wafer is very fragile, and how to clamp and transfer it is very difficult. When stacked, the alignment of chip to chip is very difficult due to the small size and high density of the micro bumps of the chip. The underfill material or NCF with high thermal conductivity needs to be filled between the layers of DRAM chips. Such materials with high thermal conductivity are the frontiers in the industry. The stacking process will definitely cause engineering defects, and how to test and repair them needs to be accumulated and explored. Because of the great difficulty, long time and certain yield loss of the HBM 3D stacking engineering technology, the price of HBM is 4-5 times that of the general DRAM per unit density.

[0070] The embodiment of the present application provides a computing device, which comprises at least one memory module group, a processor and a first substrate. Each memory module in the memory module group is stacked by a memory chip with a bit width of at least 8 bits. The size is small, and the stacking process is simple. Each memory module is connected with at least one memory channel. The density of the memory channel per unit area is large. Since the bandwidth is equal to the product of the bit width and the data transmission rate, when the density of the memory channel per unit area is large, one memory module can provide a large bit width. Under the condition that the data transmission rate is unchanged, one memory module can provide a large bandwidth. Therefore, the at least one memory module group composed of memory modules can provide a bandwidth not less than that provided by one HBM grain, and can be used as a substitute for the HBM grain to meet the demand for large bandwidth in the AI or large-scale computing scenarios, without using the HBM with great process difficulty and high cost. Therefore, the engineering difficulty can be reduced, and the cost can be saved. In addition, the processor and the memory module communicate based on a memory interface. Compared with the HBM interface, the delay is smaller, the power consumption caused by the delay is smaller, the data read-write efficiency is improved, and the efficiency of random access is improved more obviously.

[0071] First, the memory module used in the above computing device is further introduced.

[0072] The embodiment of the present application provides a memory module, which comprises at least one memory chip set, each memory chip set comprises at least one sub-group, different sub-groups are connected with different memory channels, wherein each sub-group is stacked by memory chips with a bit width greater than or equal to 8 bits, and at least one memory chip with a bit width greater than 8 bits exists in each sub-group. Wherein, one memory chip set is a plurality of memory chips stacked together, which can be understood as "a pile". Compared with the related art, the number of memory chips connected with one memory channel is reduced by using the memory chip with a larger bit width, and the number of CA lines of the memory channel is reduced, the CA signal load of the processor is reduced, the number of memory chips connected with one memory channel is reduced by at least 40% compared with the related art, and the processor can directly drive the clock signal and the CA signal to access the memory module without driving through the RCD relay, since the number of memory chips connected with one memory channel is significantly reduced and the RCD does not need to be configured, the product cost can be reduced; and the stacked memory chip is used to package the memory module, compared with the related art that each memory chip is flatly attached on the PCB to form a memory stick or is directly flatly attached on the motherboard, the size of the memory module is obviously reduced, the memory module can be installed in a position closer to the processor, the memory channel connecting the processor and the memory module is shorter, the efficiency of the processor accessing the memory module is higher, and the signal quality is better.

[0073] It should be noted that the plurality of memory chips in the memory module are dies, and the specifications of the plurality of memory chips can be the same or different. Exemplarily, the plurality of memory chips can all be dynamic random access memory (DRAM) chips. In some embodiments, the DRAM chip can be a DDR SDRAM (double data rate synchronous dynamic random access memory) chip, referred to as DDR for short; for example, DDR4, DDR5 or DDR6, etc.

[0074] As is understood by those skilled in the art, as the bit area is continuously reduced, the margin of the memory cell is also continuously reduced, and the probability of bit flip in the memory chip is increased. In order to ensure the reliability of data storage of the memory module, a plurality of memory chips in the memory module include data chips and ECC chips. It should be understood that ECC is an error correction technology that detects and corrects errors in data operations by adding additional bits to the data. The ECC technology can tolerate errors in the operation of the memory, and even if there is an error in the data, the memory controller will automatically correct the error based on the ECC, so that the system can continue to work normally and will not be interrupted due to errors. When data is written to the ECC chip, the error correction code corresponding to the written data is stored in the ECC chip. When reading data, the memory controller compares the stored error correction code with the error correction code generated when reading the data. If the read error correction code does not match the stored error correction code, the error correction code can be used to determine which bit is wrong, and then the bit is immediately corrected.

[0075] In addition, when the ECC technology is used, the ratio of the total bit width of the data chip to the total bit width of the ECC chip in the plurality of memory chips connected to one memory channel is usually 4:1, and in the case of a CPU bit width of 64 bits, the total bit width of the data chip is 64 bits and the total bit width of the ECC chip is 16 bits, and the error correction capability of the ECC technology is to correct bit errors of any 4 data queues (DQs). If the on-die error correction engine (ODECC) is included on the memory chip, the error correction capability can be improved to correct bit errors in 8 DQs by using on-die ECC and multi-level system ECC error correction, that is, if the bit width of the data chip is 16 bits, the half chipkill of the data chip can be achieved.

[0076] The memory module provided by the embodiments of the present application is further described below through specific embodiments.

[0077] Figure 1 is a schematic diagram of a memory module 1A according to an embodiment of the present application. As shown in Figure 1, the memory module 1A includes a memory chip group 11 and a memory chip group 12. The memory chip group 11 includes a first sub-group 110 and the memory chip group 12 includes a first sub-group 120. The first sub-group 110 and the first sub-group 120 are each stacked by six memory chips, four of which are data chips (shown by white fill in Figure 1) and two of which are ECC chips (shown by black fill in Figure 1). Each data chip has a bit width of 16 bits and each ECC chip has a bit width of 8 bits. The ratio of the number of data chips to the number of ECC chips is 2:1 and the ratio of the bit width of a single data chip to the bit width of a single ECC chip is 2:1. The first sub-group 110 is connected to one memory channel and the first sub-group 120 is connected to another memory channel. That is, one memory module 1A can be connected to two memory channels at the same time and different sub-groups in the memory module group are connected to different memory channels. Understandably, the total bit width of the four data chips in one first sub-group shown in Figure 1 is 64, which is aligned with the CPU bit width, and the total bit width of the two ECC chips in one first sub-group shown in Figure 1 is 16. The ratio of the total bit width of the data chips to the total bit width of the ECC chips is 4:1.

[0078] In some embodiments, the plurality of memory chips in the memory module 1A are stacked and packaged based on RDL technology and wire bonding. Illustratively, the pads in the center of the plurality of memory chips are rewired to the edges of the plurality of memory chips by RDL technology, and the pads of the plurality of memory chips are connected to the substrate leads by wire bonding to form a memory module. In some embodiments, when the number of stacked layers of memory chips in the memory module is greater than a preset value, the above-mentioned packaging method is used, wherein the value of the preset value can be determined according to actual needs. Illustratively, the preset value can be 3.

[0079] In some embodiments, the capacity of the data chips and the capacity of the ECC chips in the memory chip group are equal. For example, the capacity of the data chips and the capacity of the ECC chips are each 4 gigabytes (GB), 8 GB or 16 GB. It should be noted that the above examples of the capacity of the memory chips are only illustrative and the embodiments of the present application do not limit the capacity of the memory chips.

[0080] In some embodiments, the capacity ratio of the data chip to the ECC chip in the memory chip set is 2:1. For example, the capacity of the data chip is 16 GB and the capacity of the ECC chip is 8 GB; or for example, the capacity of the data chip is 8 GB and the capacity of the ECC chip is 4 GB. In the above optional manner, the bit width ratio of the data chip to the ECC chip is 2:1, that is, the number of data pins of the data chip to the number of data pins of the ECC chip is 2:1. By setting the capacity ratio of the data chip to the ECC chip to 2:1, the correspondence between the number of data pins and the capacity of the memory chip can be ensured, that is, the more the capacity of the memory chip, the more the number of data pins of the memory chip, and the more the number of data bits written or read from the memory chip at a time, thereby avoiding the waste of the capacity of the memory chip.

[0081] It should be noted that FIG. 1 is an example of the up-down positional relationship and the number of stacked layers of the plurality of memory chips in a memory chip set, and is not a front view, a side view or a top view of the memory module 1A, and does not limit the actual product form of the memory module 1A.

[0082] It should be noted that the sizes of the plurality of memory chips in the first sub-group can be the same or different. In some embodiments, the sizes of the plurality of memory chips in the first sub-group are the same, thereby shortening the length of the bonding wire and improving the signal quality. The embodiments of the present application do not limit the size of the memory chip in the same memory chip set.

[0083] It should be noted that the memory module 1A shown in FIG. 1 is described by taking an example of a memory module including two memory chip sets. In some embodiments, the memory module includes one memory chip set. In other embodiments, the memory module includes two or more memory chip sets. The embodiments of the present application do not limit the number of memory chip sets included in the memory module. Two or more memory chip sets can be stacked in a single column or multiple columns, which is not limited by the embodiments of the present application. Among them, a single column means that two or more memory chip sets are arranged in a single column in a top view; and multiple columns means that two or more memory chip sets are arranged in multiple columns in a top view.

[0084] It should be noted that the memory module 1A shown in FIG. 1 is described by taking an example of a memory chip set including a first sub-group, and in some embodiments, the memory chip set includes a plurality of first sub-groups. In the plurality of first sub-groups, the stacking order of the data chips and the memory chips can be the same or different. For example, in some embodiments, the stacking order of the data chips and the ECC chips in the plurality of first sub-groups is the same, for example, each first sub-group is stacked in the order of data chip, data chip, ECC chip, data chip, data chip, ECC chip from bottom to top. In the above embodiment, since the plurality of chips in the plurality of sub-groups with the same composition adopt the same stacking order, the order of the memory module package can be ensured, thereby facilitating reduction of processing errors of the memory module, thereby improving the yield of the memory module, and facilitating identification of the data chips and the ECC chips by the CPU. It should be noted that the stacking order of the memory chips shown in FIG. 1 is exemplary only, and the embodiments of the present application do not limit the stacking order. For another example, in another embodiment, the stacking order of the data chips and the ECC chips in at least two first sub-groups of the plurality of first sub-groups of the memory chip set is different.

[0085] The memory module 1A shown in FIG. 1 is described by taking an example of a first sub-group including a data chip with a bit width of 16 bits and an ECC chip with a bit width of 8 bits, and in some embodiments, the first sub-group includes a data chip with a bit width of 32 bits and an ECC chip with a bit width of 16. For example, FIG. 2 is a schematic diagram of a memory module 2A provided by an embodiment of the present application. As shown in FIG. 2, the memory module 2A includes a memory chip set 21, which includes a first sub-group 211 and a first sub-group 212, each of which is stacked by 3 memory chips, wherein 2 memory chips are data chips (exemplarily shown by white filling in FIG. 2), and 1 memory chip is an ECC chip (exemplarily shown by black filling in FIG. 2), each data chip has a bit width of 32 bits, and each ECC chip has a bit width of 16 bits. The number ratio of the data chips to the ECC chips is 2:1, and the bit width ratio of a single data chip to a single ECC chip is 2:1. The first sub-group 211 is connected to a memory channel, and the first sub-group 212 is connected to another memory channel, that is, one memory module 2A can be connected to two memory channels at the same time, and different sub-groups in the first memory module 2A are connected to different memory channels. Understandably, the total bit width of 2 data chips in one first sub-group shown in FIG. 2 is 64, which is aligned with the bit width of the CPU, and the total bit width of 1 ECC chip in one first sub-group shown in FIG. 2 is 16, and the total bit width ratio of the data chips to the ECC chips is 4:1.

[0086] Compared with the memory module 1A, the memory module 2A shown in FIG. 2 uses a memory chip with a larger bit width, the number of memory chips included in a first subgroup is reduced, and the number of stacked layers formed by stacking the memory chips in a first subgroup is also reduced. Therefore, with the same number of stacked layers, the memory module 2A can stack more first subgroups, and since a first subgroup is connected to a memory channel, the memory module 2A can be connected to more memory channels with the same number of stacked layers, that is, the same number of memory chips stacked in the same area can be connected to more memory channels, thereby increasing the number of memory channels per unit area, that is, increasing the memory channel density. In addition, since the number of memory channels connected to a memory module is increased, the efficiency of the processor accessing the memory module can be improved.

[0087] In some embodiments, the memory module 2A is packaged by wire bonding. In some embodiments, when the number of stacked layers of memory chips in the memory module is less than or equal to a preset value, the memory module is packaged by wire bonding. The value of the preset value can be determined according to actual needs. For example, the preset value can be 3. In the above embodiment, when the number of stacked layers of memory chips is small, wire bonding is directly used for packaging without using RDL technology, which can reduce the process difficulty of packaging and save costs.

[0088] In some embodiments, the capacity of the data chip and the capacity of the ECC chip in the memory chip group are equal. For example, the capacity of the data chip and the capacity of the ECC chip are both 2GB, 4GB, 8GB, or 16GB. It should be noted that the above examples of the capacity of the memory chip are only exemplary, and the embodiments of the present application do not limit the capacity of the memory chip.

[0089] In some embodiments, the capacity ratio of the data chip to the ECC chip in the memory chip group is 2:1. For example, the capacity of the data chip is 16GB and the capacity of the ECC chip is 8GB. For another example, the capacity of the data chip is 8GB and the capacity of the ECC chip is 4GB. For another example, the capacity of the data chip is 4GB and the capacity of the ECC chip is 2GB. In the above optional manner, the bit width ratio of the data chip to the ECC chip is 2:1, that is, the number of data pins of the data chip to the number of data pins of the ECC chip is 2:1. By setting the capacity ratio of the data chip to the ECC chip to 2:1, the correspondence between the number of data pins and the capacity of the memory chip can be ensured, that is, the more the data pins of the memory chip, the more the number of data bits written or read from the memory chip at a time, thereby avoiding waste of the capacity of the memory chip.

[0090] It should be noted that FIG. 2 is an example of the up-down positional relationship and the number of stacked layers of the plurality of memory chips in a memory chip set, and is not a front view, a side view or a top view of the memory module 2A, and does not limit the actual product form of the memory module 2A.

[0091] The memory module shown in FIG. 1 and FIG. 2 is described by taking the ratio of the bit width of a single data chip to the bit width of a single ECC grain as 2:1 as an example, and in some embodiments, the ratio of the bit width of a single data chip to the bit width of a single ECC grain is 4:1, for example, FIG. 3 is a schematic diagram of a memory module provided in an embodiment of the present application. As shown in FIG. 3, the memory module 3A includes a memory chip set 31 and a memory chip set 32. The memory chip set 31 includes a second sub-set 310, and the memory chip set 32 includes a second sub-set 320, and the second sub-set 310 and the second sub-set 320 are stacked by 4 memory chips, of which 2 memory chips are data chips (shown by white filling in FIG. 3 as an example), and 2 memory chips are ECC chips (shown by black filling in FIG. 3 as an example), and the bit width of each data chip is 32 bits, and the bit width of each ECC chip is 8 bits. The number ratio of the data chips to the ECC chips is 1:1, and the bit width ratio of a single data chip to a single ECC chip is 4:1. The second sub-set in the memory chip set 31 is connected with a memory channel, and the second sub-set in the memory chip set 32 is connected with another memory channel, that is, one memory module 3A can be connected with two memory channels at the same time, and different sub-sets in one memory module 3A are connected with different memory channels. Understandably, the total bit width of the 2 data chips in one second sub-set shown in FIG. 3 is 64, which is aligned with the CPU bit width, and the total bit width of the 2 ECC chips in one second sub-set shown in FIG. 3 is 16, and the total bit width ratio of the data chips to the ECC chips is 4:1.

[0092] In the above embodiment, compared with the memory module 1A, the memory module 3A shown in FIG. 3 uses memory chips with a bit width of 32 bits, and the number of memory chips connected with one memory channel is smaller, that is, the number of memory chips included in one sub-set is reduced, and therefore, the number of stacked layers of the memory chips in the memory module is smaller, and in the same packaging size, the packaging engineering difficulty is reduced, the length of the bonding wire is shortened, and the signal quality is better; and the capacity of the memory module is flexible and can be configured.

[0093] In some embodiments, the plurality of memory chips in the memory module 301 are packaged based on the RDL technology and the wire bonding mode.

[0094] In some embodiments, the capacity of the data chip and the capacity of the ECC chip in the memory chip set are equal. For example, the capacity of the data chip and the capacity of the ECC chip are 4GB, 8GB or 16GB. It should be noted that the above examples of the capacity of the memory chip are exemplary only, and the embodiments of the present application do not limit the capacity of the memory chip.

[0095] In some embodiments, the capacity ratio of the data chip and the ECC chip in the memory chip set is 4:1. For example, the capacity of the data chip is 16GB, and the capacity of the ECC chip is 4GB. In the above optional manner, the bit width ratio of the data chip and the ECC chip is 4:1, that is, the number of data pins of the data chip and the number of data pins of the ECC chip are in a ratio of 4:1. By setting the capacity ratio of the data chip and the ECC chip to 4:1, the correspondence between the number of data pins and the capacity of the memory chip can be ensured, that is, the more the data pins of the memory chip, the more the number of data bits written or read from the memory chip at a time, thereby avoiding the waste of the capacity of the memory chip.

[0096] It should be noted that FIG. 3 is an example of the up-down positional relationship and the number of stacked layers of the plurality of memory chips in a memory chip set, and is not a front view, a side view or a top view of the memory module 2A, and does not limit the actual product form of the memory module 2A.

[0097] It should be noted that the memory module 3A shown in FIG. 3 is described by taking an example of a memory module including two memory chip sets. In some embodiments, the memory module includes one memory chip set. In other embodiments, the memory module includes two or more memory chip sets, and the embodiments of the present application do not limit the number of memory chip sets included in the memory module. Two or more memory chip sets can be stacked in a single column or multiple columns, and the embodiments of the present application do not limit this. Among them, a single column means that two or more memory chip sets are arranged in a single column in a top view; and multiple columns means that two or more memory chip sets are arranged in multiple columns in a top view.

[0098] It should be noted that the memory module 3A shown in FIG. 3 is described by taking an example of the memory chip group including one second sub-group, and in some embodiments, the memory chip group includes a plurality of second sub-groups. In the plurality of second sub-groups, the stacking order of the data chip and the memory chip can be the same or different. For example, in some embodiments, the stacking order of the data chip and the ECC chip in the plurality of second sub-groups is the same, for example, each second sub-group is stacked in the order of data chip, ECC chip, data chip, ECC chip from bottom to top. In the above embodiment, since the plurality of chips in the plurality of sub-groups with the same composition adopt the same stacking order, the order of the memory module package can be ensured, thereby facilitating reduction of processing errors of the memory module, thereby improving the yield of the memory module, and facilitating identification of the data chip and the ECC chip by the CPU. It should be noted that the stacking order of the memory chip shown in FIG. 3 is exemplary, and the embodiments of the present application do not limit the stacking order. For another example, in another embodiment, the stacking order of the data chip and the ECC chip in at least two second sub-groups of the plurality of second sub-groups of the memory chip group is different.

[0099] It should be noted that the memory modules 1A, 2A and 3A are described by taking an example of the same memory module in which the different memory chip groups include the same number of sub-groups, and in some embodiments, the different memory chip groups in the same memory module can include different numbers of sub-groups.

[0100] It should be noted that the memory modules 1A, 2A and 3A are described by taking an example of the same memory module in which the different memory chip groups include the same number of sub-groups, and in some embodiments, the different memory chip groups in the same memory module can include different numbers of sub-groups.

[0101] It should be noted that the memory modules 1A, 2A and 3A are described by taking an example of the memory module in which one memory chip group includes only the first sub-group or only the second sub-group, and in some embodiments, in the case where the memory chip group includes a plurality of sub-groups, one memory chip group includes both the first sub-group and the second sub-group.

[0102] It should be noted that the memory modules 1A, 2A and 3A described above are described by taking an example in which the bit widths of the plurality of data chips in one sub-group and the plurality of ECC chips are equal, in some embodiments, the bit widths of the plurality of data chips in one sub-group can not be equal, and the bit widths of the plurality of ECC chips in one sub-group can not be equal, for example, in some embodiments, the memory chip set includes at least one third sub-group, each third sub-group is stacked by 3 data chips and 1 ECC chip, wherein the bit width of 1 data chip is 32 bits, the bit width of 2 data chips is 16 bits, and the bit width of 1 ECC chip is 16 bits.

[0103] In the above plurality of embodiments, compared with the related art, the use of a memory chip with a larger bit width reduces the number of memory chips connected to one memory channel, thereby reducing the number of CA lines of the memory channel, reducing the CA signal load of the processor, the number of memory chips connected to one memory channel is reduced by at least 40% compared with the related art, and the processor can directly drive the clock signal and the CA signal to access the memory module without the need for driving through the RCD relay, since the number of memory chips connected to one memory channel is significantly reduced and does not need to be configured with an RCD, the product cost can be reduced; and the stacked memory chip is used to package the memory module, compared with the related art in which each memory chip is flatly attached to the PCB to form a memory bank or is directly flatly attached to the motherboard, the size of the memory module is significantly reduced, the memory module can be installed closer to the processor, the memory channel connecting the processor and the memory module is shorter, the efficiency of the processor accessing the memory module is higher, and the signal quality is better. In addition, the memory module provided by the embodiments of the present application can be installed on the motherboard in a flat manner without the need for using a PCB and a connector (socket) for installation, which can improve the signal link environment and further reduce the cost.

[0104] The base area of the memory module provided in the embodiments of the present application is greater than 10 mm*10 mm and less than 20 mm*20 mm, compared with the related art, the size of the memory module is greatly reduced, and the number of memory channels is more under the same area, for example, taking the memory module accessed through two memory channels in the present application as an example, under the area occupied by 8 memory channels of the RDIMM in the related art, more than 100 memory channels can exist in the embodiments of the present application, and the bandwidth capacity density per unit area is significantly improved. Moreover, as shown in FIG. 4, which is a CA eye-mask simulation result diagram provided in the embodiments of the present application, the upper diagram in FIG. 4 is the simulation result of the RDIMM memory module in the related art, wherein the eye height is 164 mV, and the eye width is 272 ps; the lower diagram in FIG. 4 is the simulation result of the memory module provided in the embodiments of the present application, wherein the eye height is 195 mV, and the eye width is 288 ps. It can be seen that the two indexes of eye height and eye width in the simulation result of the memory module provided in the embodiments of the present application are higher than those in the simulation result of the related art, which indicates that the CA signal can be directly driven by the CPU without relying on the RCD in the present application, and the CA signal quality is better. Since it is not necessary to rely on the RCD, at least 10 dollars of cost is saved for each memory module.

[0105] Based on the above memory module, a computing device provided in the embodiments of the present application is further introduced.

[0106] The computing device provided in the embodiments of the present application can be applied to AI large model training, high-performance computing, graphics processing, data centers, virtual reality (VR) and augmented reality (AR), autonomous driving and professional image processing scenes, and the application scenarios of the computing device are not limited in the embodiments of the present application. Different configurations of the computing device can be selected according to the application scenarios. For example, in the image processing scene, if the trained AI large model is used to complete the processing and analysis of images and videos, the computing device used in this scene can include a GPU and at least one memory module group, and the total bandwidth of the at least one memory module group is 4*1024. For another example, in the AI large model training scene, a large-scale neural network is trained to complete image recognition, speech recognition or natural language processing, and the computing device used in this scene can include a TPU and at least one memory module group, and the total bandwidth of the at least one memory module group is 4*1024. It should be noted that the above description of the configuration of the computing device selected in different scenes is only exemplary, and the embodiments of the present application do not limit this.

[0107] FIG. 5 is a schematic diagram of a computing device 5B according to an embodiment of the present application. As shown in FIG. 5, the computing device 5B includes a plurality of first memory module groups 51-54, a processor 55, and a first substrate 56. The first substrate 56 has the first memory module groups 51-54 and the processor 55 mounted thereon. The plurality of memory modules in the first memory module group 51 and the first memory module group 52 are arranged in a column and mounted on one side of the processor 55 on the first substrate 56, and the plurality of memory modules in the first memory module group 53 and the first memory module group 54 are arranged in a column and mounted on the other side of the processor 55 on the first substrate 56.

[0108] It should be noted that the computing device 5B shown in FIG. 5 is described by way of example in which the computing device includes a plurality of memory module groups. In some embodiments, the computing device includes one memory module group, and the number of memory module groups in the computing device is not limited in the embodiments of the present application. In other embodiments, the computing device includes at least one memory module group and at least one HBM particle.

[0109] Each of the first memory module groups includes 8 memory modules, and each of the memory modules 1A, 2A, or 3A described above is independently accessed by two memory channels at the same time, and the bit width of each memory channel is 64 bits. Therefore, the bit width of one first memory module group is 8 x 2 x 64 = 1024 bits, and the bit width of one HBM particle is 1024 bits. Therefore, the bit width of one first memory module group shown in FIG. 5 is equal to the bit width of one HBM particle. Taking the specification of the memory chip in the memory module as an example, the bandwidth of one memory module in FIG. 5 is 102.4 GBps, the total bandwidth of one first memory module group is equal to the bandwidth of one HBM3 particle, and the total bandwidth of four first memory module groups is equal to the total bandwidth of four HBM3 particles. It should be noted that the above description of the bandwidth of the memory module group is exemplary only, and in some embodiments, the total bandwidth of the plurality of memory module groups is equal to the bandwidth of one HBM particle, or the bandwidth of one memory module group is equal to the bandwidth of one HBM particle, and the bandwidth of one memory module group is not limited in the embodiments of the present application. In addition, the number of memory modules in the first memory module group and the capacity of the single memory module can be configured such that the capacity of one memory module group is not less than the capacity of one HBM particle, or the capacity of the plurality of memory module groups is not less than the capacity of one HBM particle, and the specific configuration is not limited in the embodiments of the present application.

[0110] In the computing device shown in FIG. 5, the large bandwidth of the HBM grain is provided by arranging a plurality of memory modules, the stacking process of the memory chips in the memory modules is simple, and the cost of the memory modules is relatively low. Compared with the expensive 3D TSV stacking process of the HBM, the engineering difficulty can be reduced, the cost can be saved, the cost of the HBM with a capacity of 16 GB is about 320 US dollars, and the cost of the memory module with the same capacity is only 48 US dollars. A larger capacity can be obtained at a lower cost than the HBM, which can meet the demand for bandwidth and capacity in AI and large-scale computing, and ensure the supply of large-bandwidth memory.

[0111] The processor 55 can be a general-purpose CPU, a GPU, a switching module processor unit (SMPU), an NPU, a microprocessor, a TPU, or a GPGPU, and the like. The type of the processor is not limited in the embodiments of the present application. The processor 55 accesses the memory modules in the plurality of memory module groups through a plurality of memory interfaces, and the plurality of memory interfaces comply with the DDR protocol. The memory interface includes a DMC and a memory port network layer (DDR PHY). As shown in FIG. 6, FIG. 6 is a schematic diagram of a memory interface provided in the embodiments of the present application, and FIG. 6 includes 16 memory interfaces. One memory interface is used to access one subgroup in the memory module based on one memory channel (one subgroup is connected to one memory channel, and different subgroups are connected to different memory channels). FIG. 6 takes an example of one memory module including two subgroups. The processor accesses the memory module through two memory interfaces. The process in which the processor accesses the memory module through the memory interface includes: the processor executes a data read-write instruction, the processor generates a read-write request corresponding to the read-write instruction, and sends the read-write request to the memory management unit. The memory management unit first converts the virtual address carried in the read-write request into a physical address, and then sends the read-write request to the bus. The bus sends the data read-write request to the memory controller responsible for accessing the physical address. The memory controller reads data from the memory module corresponding to the physical address through the DDR PHY or writes data into the memory module corresponding to the physical address through the DDR PHY according to the data read-write request, so as to complete the access to the memory module. Since the processor 55 and the memory module communicate with each other based on the memory interface, compared with the HBM interface, the delay is smaller, the power consumption caused by the delay is smaller, the data read-write efficiency is improved, and the efficiency of random access is improved more obviously.

[0112] In some embodiments, the first substrate is a UHD. In the above embodiment, the UHD is used to mount the processor and the memory module, the UHD can meet the layout requirement of super multi-line in the large bit width scene, and the processor and the memory module can be connected through the UHD without using a packaging process such as CoWoS which is difficult and expensive to implement, thereby reducing the engineering difficulty and saving the cost. In some embodiments, the first substrate is of other types, and the embodiments of the present application do not limit the type of the first substrate.

[0113] In some embodiments, the memory modules in the first memory module group are mounted in a flat manner.

[0114] In some embodiments, the computing device further includes an integrated heat sink covering the plurality of first memory module groups and the accelerator, and the integrated heat sink dissipates heat through at least one of liquid cooling and air cooling. In the above embodiment, the processor and the memory module are cooled by the integrally formed heat sink, and the cooling effect is better. FIG. 7 is a mounting schematic diagram of a computing device 5B provided by an embodiment of the present application, as shown in FIG. 7, the computing device 5B is mounted to the PCB through a socket.

[0115] It should be noted that the computing device 5B shown in FIG. 5 includes four first memory module groups, which is only exemplary, and the number of first memory module groups can be configured according to the number of memory channels of the processor, and the embodiments of the present application do not limit this.

[0116] It should be noted that each memory module group shown in FIG. 5 includes eight memory modules, which is only exemplary, and the number of memory modules included in each first memory module group can be set according to actual needs. For example, in some embodiments, a memory module group includes 12 memory modules, and the total bandwidth of a memory module group is 1.2 TBps, and the bandwidth of an HBM3e particle is 1 TBps, so the total bandwidth of a memory module group is greater than that of an HBM3e particle. In addition, the number of memory modules included in the first memory module group can be configured according to the number of memory channel data of the processor, and the embodiments of the present application do not limit the number of first memory module groups included in the computing device.

[0117] It should be noted that the specification of the memory chip is DDR5-6400, which is only exemplary, and the embodiments of the present application do not limit the specification of the memory chip in the memory module. In some embodiments, the specification of the memory chip is DDR5 or DDR6 with higher read-write efficiency, and the total bandwidth of a memory module group is higher. With the evolution of DDR standards, higher read-write efficiency memory chips can be used, so that the memory module group can replace higher standard HBM particles.

[0118] It should be noted that the size of the memory module used by the computing device can be determined according to actual needs. For example, in some embodiments, a smaller computing device is required, and the size of the memory module 2A is smaller than that of the memory modules 1A and 3A. Therefore, the size of the memory module in the computing device uses the memory module 2A to meet the size requirements of the computing device. In other embodiments, there are no size requirements for the computing device. The bit width of 16 and the bit width of 8 in the memory module 1A do not need to be customized like the bit width of 32. Therefore, the size of the memory module in the computing device uses the memory module 1A to reduce the cost of the computing device.

[0119] It should be noted that the arrangement of the plurality of memory modules in each memory module group shown in FIG. 5 is only exemplary. In some embodiments, the plurality of memory modules in each first memory module group are arranged in multiple columns for installation, which is not limited in the embodiments of the present application.

[0120] It should be noted that the installation position of the plurality of first memory module groups on the first substrate shown in FIG. 5 is on both sides of the processor. In some embodiments, the installation position of the plurality of first memory module groups on the first substrate is on one side of the processor or around the processor. The embodiments of the present application do not limit the installation position of the first memory module group on the first substrate.

[0121] FIG. 8 is a schematic diagram of a computing device 8B according to an embodiment of the present application. As shown in FIG. 8, the computing device 8B includes eight second memory module groups 81-88, a processor 89, and a first substrate 810. The first substrate 810 is provided with the second memory module groups 81-88 and the processor 89. Each second memory module group includes four memory modules, and each second memory module group further includes a second substrate for installing the memory modules in the second memory module group. Each second memory module group is installed in a plug-in manner. The second memory module groups 81-84 are installed on one side of the processor 89 on the first substrate 810, and the second memory module groups 85 and 88 are installed on the other side of the processor 89 on the first substrate 810.

[0122] Each second memory module group includes 4 memory modules, and each memory module is independently accessed by two memory channels simultaneously, and the bit width of each memory channel is 64 bits. Therefore, the bit width of the two second memory module groups is 2*4*2*64=1024 bits, and the bit width of one HBM particle is 1024 bits. Therefore, the bit width of the two second memory module groups shown in FIG. 8 is equal to the bit width of one HBM particle. For example, the specification of the memory chip in the memory module is DDR5-6400, the bandwidth of one memory module in FIG. 8 is 102.4 GBps, the total bandwidth of the two second memory module groups is equal to 819.2 GBps, which is equal to the bandwidth of one HBM3 particle, and the total bandwidth of the eight second memory module groups is equal to the total bandwidth of four HBM3 particles.

[0123] The computing device 8B provided in FIG. 8 is compared with the computing device 5B. The second memory module group is pluggable. When the memory module is damaged, only the second memory module group in which the damaged memory module is located needs to be replaced, and the entire computing device does not need to be replaced, thereby improving the convenience of maintenance and reducing the maintenance cost.

[0124] In some embodiments, the second memory module group is mounted on the first substrate 86 through a socket, so as to ensure that the second memory module group is pluggable.

[0125] In some embodiments, the computing device further includes an integrated heat sink covering the at least one memory module group and the accelerator. In some embodiments, the integrated heat sink dissipates heat through at least one of liquid cooling and air cooling. In the above embodiment, the processor and the memory module are cooled by the integrally formed heat sink, and the cooling effect is better. FIG. 9 is a mounting schematic diagram of a computing device 5B according to an embodiment of the present application. As shown in FIG. 9, the computing device 8B is mounted on a PCB through a socket.

[0126] It should be noted that the above embodiment that each second memory module group includes 4 memory modules is only exemplary, and the number of memory modules included in each second memory module group can be set according to actual needs, and the present application does not limit this. In addition, the number of memory modules in the plurality of second memory module groups can be the same or different, and the present application does not limit this. The plurality of memory modules in one second memory module group can be arranged in one column and mounted on the second substrate, or can be arranged in multiple columns and mounted on the second substrate, and the present application does not limit this.

[0127] It should be noted that the second memory module group in FIG. 8 is realized by plugging the second memory module group into the first substrate, and in some embodiments, the memory modules in the second memory module group are directly plugged into the first substrate without being installed on the second substrate, so that the computing device can be repaired at the granularity of the memory module, and the convenience of repair and the cost of repair can be further improved compared with the computing device 5B which is repaired at the granularity of the second memory module group.

[0128] It should be noted that the installation positions of the second memory module groups on the first substrate 86 in FIG. 8 are on both sides of the processor, and in some embodiments, the installation positions of the second memory module groups on the first substrate are on one side of the processor or around the processor. The embodiments of the present application do not limit the installation positions of the second memory module groups on the first substrate.

[0129] It should be noted that the computing device 5B is described by taking an example of all the memory module groups being the first memory module groups, and the computing device 8B is described by taking an example of all the memory module groups being the second memory module groups, and in some embodiments, the memory module groups in the computing device include both the first memory module groups and the second memory module groups, and the embodiments of the present application do not limit this.

[0130] The embodiments of the present application provide a computing device, which includes a plurality of computing devices and a third substrate on which the plurality of computing devices are installed. The third substrate can be a PCB. The third substrate and the first substrate adopt a daughter-mother board structure, the first substrate is a daughter board, and the third substrate is a mother board.

[0131] The embodiments of the present application provide a computing device cluster, which includes at least one computing device, and each computing device includes a plurality of computing devices and a third substrate on which the plurality of computing devices are installed. The third substrate can be a PCB. The third substrate and the first substrate adopt a daughter-mother board structure, the first substrate is a daughter board, and the third substrate is a mother board.

[0132] The terms "first", "second", and the like, used in the specification, unless otherwise specified, are used to distinguish between similar elements and are not necessarily used to describe a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the descriptive terms "first", "second", etc., are to be interpreted, by those skilled in the art, as a structural or functional pertinence and not by their order or sequence. For example, a first memory module group can be termed a second memory module group, and similarly, a second memory module group can be termed a first memory module group, without departing from the scope of the various examples. The first memory module group and the second memory module group can both be a node memory module group, and in some cases, can be separate and distinct memory module groups.

[0133] The term "at least one", as used in this application is defined as one or more. The term "plurality", as used in this application, is defined as two or more. The terms "system" and "network" are often used interchangeably.

[0134] It will also be appreciated that the term "if' can be construed to mean "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be construed to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]", depending on the context.

[0135] The above description is merely illustrative of the application, and not restrictive. Any modifications or equivalent arrangements which are obvious to those skilled in the art should be included within the scope of the application. Accordingly, the scope of the application should be determined by the appended claims and equivalents thereof.

[0136] The above examples are merely used to illustrate the technical solutions of the present application, but not for limiting the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A computing device, comprising: The computing device comprises a processor, at least one memory module group and a first substrate; The first substrate is provided with the processor and the at least one memory module group; The memory module group comprises at least one memory module, the memory module comprises at least one memory chip group, the memory chip group comprises at least one sub-group, different sub-groups are connected with different memory channels, each sub-group is stacked by a plurality of memory chips, the bit width of the memory chips in each sub-group comprises a first bit width and a second bit width, the first bit width is N times of the second bit width, the second bit width is greater than or equal to 8 bits, N is a positive integer greater than 1; The processor accesses the memory module in the at least one memory module group through at least one memory interface, wherein different memory interfaces are used to access different memory modules, and the memory interface conforms to a double data rate synchronous dynamic random access memory (DDR) protocol.

2. The computing device of claim 1, wherein, The memory interface comprises a memory controller (DMC) and a memory port physical layer (DDR PHY).

3. The computing device of claim 1, wherein, The computing device comprises at least one first memory module group, and each first memory module group is installed in a flat manner.

4. The computing device of claim 1, wherein, The computing device comprises at least one second memory module group, and each second memory module group is installed in a pluggable manner.

5. The computing device of claim 4, wherein, Each second memory module group further comprises a second substrate, the memory module in the second memory module group is installed on the second substrate, and the second substrate is installed on the first substrate in a pluggable manner.

6. The computing device of any of claims 1 to 4, wherein, The installation position of the at least one memory module group on the first substrate is distributed on both sides of the accelerator, or surrounds the accelerator, or is distributed on one side of the accelerator.

7. The computing device of any of claims 1 to 6, wherein, The computing device further comprises an integrated heat sink, and the integrated heat sink covers the at least one memory module group and the accelerator.

8. The computing device of claim 7, wherein, The integrated heat sink dissipates heat by at least one of liquid cooling and air cooling.

9. The computing device of any of claims 1-8, wherein, The first substrate is an ultra-high density plate (UHD).

10. The computing device of any of claims 1 to 9, wherein, The total capacity of the at least one memory module group is not less than the capacity of one high bandwidth memory (HBM) particle, and the total bandwidth of the at least one memory module group is not less than the bandwidth of one HBM particle.

11. The computing device of any of claims 1 to 10, wherein, The processor is any one of a central processing unit (CPU), a graphics processing unit (GPU), a network processing unit (NPU), a tensor processing unit (TPU) and a general-purpose graphics processing unit (GPGPU).

12. The computing device of any of claims 1-11, wherein, Each sub-group comprises at least one data chip and at least one error correction code (ECC) chip.

13. The memory module of claim 12, wherein, The memory chip group comprises at least one first sub-group, the bit width ratio of a single data chip to a single ECC chip in the first sub-group is 2:1, and the number ratio of data chips to ECC chips in the first sub-group is 2:

1.

14. The memory module of claim 12, wherein, The memory chip group comprises at least one second sub-group, the bit width ratio of a single data chip to a single ECC chip in the second sub-group is 4:1, and the number ratio of data chips to ECC chips in the second sub-group is 1:

1.

15. The memory module of claim 12, wherein, The memory chip set comprises a plurality of sub-sets, the plurality of sub-sets comprising at least one first sub-set and at least one second sub-set, a bit width ratio of a single data chip and a single ECC chip in the first sub-set being 2:1, a quantity ratio of data chips and ECC chips in the first sub-set being 2:1, a bit width ratio of a single data chip and a single ECC chip in the second sub-set being 4:1, and a quantity ratio of data chips and ECC chips in the second sub-set being 1:

1.

16. The computing device of any of claims 1 to 15, wherein, The plurality of memory chips in each of the memory chip sets are stacked based on a re-wiring layer (RDL) technology and wire bonding.

17. The computing device of any of claims 1 to 15, wherein, The plurality of memory chips in each of the memory chip sets are stacked based on wire bonding.

18. The memory module of any one of claims 1 to 17, wherein, The memory module does not comprise a register clock driver (RCD).

19. A memory module bank, comprising: The memory module set comprises at least one memory module and a second substrate, the second substrate being used for mounting the at least one memory module, and each of the second memory module sets is mounted in a pluggable manner.

20. A computing device, comprising: The computing device comprises a plurality of computing apparatuses and a third substrate; The plurality of computing apparatuses are mounted on the third substrate.

21. A cluster of computing devices, characterized in that, The computing device cluster comprises at least one computing device, and each of the computing devices comprises a plurality of computing apparatuses and a third substrate; The plurality of computing apparatuses are mounted on the third substrate.

Images