Hybrid storage computing architecture and storage computing method

Abstract

The application relates to a hybrid storage computing architecture and a storage computing method. The method comprises a first storage chip, a second storage chip and a computing chip. The computing chip is connected with the first storage chip and the second storage chip respectively. The storage speed of the second storage chip is greater than that of the first storage chip, and the storage density of the first storage chip is greater than that of the second storage chip. The first storage chip and the second storage chip are used for storing related data of a large language model (LLM). The computing chip is used for obtaining the related data from the first storage chip and the second storage chip, and performing large language model training or large language model application calculation based on the related data to obtain a calculation result. The application can greatly improve the storage computing efficiency of data, the storage space of data in the large language model and the transmission speed of data from the storage unit to the computing unit.

Description

Technical Field

[0001] This application relates to the field of memory design technology, and in particular to a hybrid memory computing architecture and memory computing method. Background Technology

[0002] Rapid technological advancements are propelling artificial intelligence into a new era of high-speed development. However, with the exponential growth of AI model parameters, the demand for storage capacity and bandwidth on computing platforms is becoming increasingly urgent.

[0003] In traditional hardware architectures, computing chips handle data processing, dynamic random access memory (DRAM) is used for temporary data storage, and NAND flash memory or hard drives handle large-capacity data storage. Currently, the traditional von Neumann architecture separates computing chips from storage chips. This approach results in relatively low bandwidth, leading to lower data transmission efficiency and making it difficult to meet the computational demands of modern artificial intelligence models.

[0004] However, the aforementioned packaged chips suffer from low computational efficiency. Summary of the Invention

[0005] Therefore, it is necessary to provide a hybrid memory computing architecture and memory computing method that can improve the computing efficiency of packaged chips in response to the above-mentioned technical problems.

[0006] In a first aspect, this application provides a hybrid storage computing architecture, including: a first storage chip, a second storage chip, and a computing chip; the computing chip is connected to the first storage chip and the second storage chip respectively, the storage speed of the second storage chip is greater than the storage speed of the first storage chip, and the storage density of the first storage chip is greater than the storage density of the second storage chip;

[0007] The first and second storage chips are used to store relevant data of the Large Language Model (LLM).

[0008] The computing chip is used to acquire relevant data from the first and second memory chips, and to perform calculations based on the relevant data for training or application of large language models, and to obtain the calculation results.

[0009] In one embodiment, the first memory chip includes at least one memory cell array and a first peripheral circuit; the memory cell array is composed of at least one NAND memory cell arranged periodically.

[0010] NAND storage units are used to store the first data in the relevant data; the first data refers to the training data of the large language model.

[0011] The first peripheral circuit is used to process the first data to obtain the processed first data; the processing operations include write operations, read operations and erase operations;

[0012] The computing chip is used to perform calculations on the processed first data for large language model training or large language model application, and obtain the calculation results.

[0013] In one embodiment, the second memory chip includes at least one periodically arranged DRAM memory cell, the DRAM memory cell including a DRAM device and a second peripheral circuit.

[0014] DRAM devices are used to store the second data in the relevant data; the second data is intermediate data for training the large language model, or intermediate data for computation of the large language model.

[0015] The second peripheral circuit is used to process the second data to obtain the processed second data; the processing operations include write operations, read operations and erase operations;

[0016] The computing chip is used to perform calculations on the processed second data for large language model training or large language model application, and obtain the calculation results.

[0017] In one embodiment, the hybrid storage computing architecture further includes a cache chip, the first storage chip includes at least one NAND storage sub-chip, and the second storage chip includes at least one DRAM storage sub-chip;

[0018] Each NAND storage sub-chip is connected to the computing chip in sequence;

[0019] Each DRAM storage sub-chip and cache chip is connected to the computing chip in sequence, and each NAND storage sub-chip, each DRAM storage sub-chip and cache chip is located on the same side of the computing chip.

[0020] In one embodiment, the hybrid storage computing architecture described above also includes a NAND controller chip;

[0021] Each NAND storage sub-chip and NAND controller chip is connected to the computing chip in sequence.

[0022] In one embodiment, the hybrid storage computing architecture further includes a cache chip, the first storage chip includes at least one NAND storage sub-chip, and the second storage chip includes at least one DRAM storage sub-chip;

[0023] Each NAND storage sub-chip, each DRAM storage sub-chip, and the cache chip are sequentially connected to the computing chip.

[0024] In one embodiment, the hybrid storage computing architecture described above also includes a NAND controller chip;

[0025] Each NAND storage sub-chip, NAND controller chip, each DRAM storage sub-chip, and cache chip are sequentially connected to the computing chip.

[0026] In one embodiment, the first memory chip includes at least one of the following: a chip in which a NAND memory array and peripheral control circuitry are arranged side by side; a chip in which the peripheral control circuitry is arranged below the NAND memory array; and a chip in which the NAND memory array and peripheral control circuitry are respectively disposed on two semiconductor wafers and bonded together.

[0027] In one embodiment, the second memory chip includes a chip in which multiple DRAM layers are stacked and distributed based on through-silicon via (TSV) technology.

[0028] Secondly, this application also provides a storage computing method applied to the hybrid storage computing architecture described in the first aspect, comprising:

[0029] Acquire relevant data of the large language model stored in the first and second memory chips;

[0030] The computation results are obtained by performing calculations on the relevant data of the large language model for training or application.

[0031] The aforementioned hybrid storage computing architecture and method comprises a first storage chip, a second storage chip, and a computing chip. The computing chip is connected to both the first and second storage chips. The storage speed of the second storage chip is greater than that of the first storage chip, and the storage density of the first storage chip is greater than that of the second storage chip. The first and second storage chips are used to store relevant data for a large language model (LLM). The computing chip is used to retrieve relevant data from the first and second storage chips and perform calculations for training or applying the large language model based on the relevant data to obtain the calculation results. This application can significantly improve the efficiency of data storage and computing, the storage space of data in a large language model, and the transmission speed of data from the storage unit to the computing unit. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of a hybrid storage computing architecture in one embodiment;

[0034] Figure 2 This is a structural block diagram of a hybrid storage computing architecture in one embodiment;

[0035] Figure 3 This is a cross-sectional view of a hybrid storage computing architecture in one embodiment;

[0036] Figure 4 This is a schematic diagram of the first memory chip in one embodiment;

[0037] Figure 5 This is a schematic diagram of the first memory chip in another embodiment;

[0038] Figure 6 This is a schematic diagram of the second memory chip in one embodiment;

[0039] Figure 7 This is a schematic diagram of the first memory chip in another embodiment;

[0040] Figure 8 This is a schematic diagram of a first specific hybrid storage computing architecture in one embodiment;

[0041] Figure 9 This is a schematic diagram of a second specific hybrid storage computing architecture in one embodiment;

[0042] Figure 10 This is a schematic diagram of a third specific hybrid storage computing architecture in one embodiment;

[0043] Figure 11 This is a schematic diagram of a fourth specific hybrid storage computing architecture in one embodiment;

[0044] Figure 12 This is a schematic diagram of the architecture of a computing chip in one embodiment;

[0045] Figure 13 This is a process flow diagram of the manufacturing process of a hybrid storage computing architecture in one embodiment;

[0046] Figure 14 This is a flowchart illustrating the storage calculation method in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] Rapid technological advancements are propelling artificial intelligence into a new era of high-speed development. However, enabling intelligent operation and local data processing on devices such as smartphones and tablets to ensure user privacy and security has become a crucial development direction. Yet, with the exponential growth of AI model parameters, the demands on computing platform storage capacity and bandwidth are also increasing.

[0049] In traditional hardware architectures, computing chips handle data processing, Dynamic Random Access Memory (DRAM) is used for fast caching, and NAND flash memory or hard drives handle large-capacity data storage. This architecture, which separates computing and storage, results in limited data transmission bandwidth, with significant latency and power consumption during data transmission, making it unable to effectively support complex artificial intelligence computing tasks.

[0050] To address this challenge, the rapid development of advanced packaging technologies has spurred the emergence of various storage computing chip architectures. For example, AMD (Advanced Micro Devices, Inc.), a multinational semiconductor company in the United States, utilizes 3D V-cache technology, which integrates Static Random-Access Memory (SRAM) with a computing chip in three dimensions through hybrid bonding technology, effectively expanding the chip's storage capacity. However, due to the limitations of SRAM storage capacity, it still struggles to meet the data storage needs of edge devices. NVIDIA has also made breakthroughs in this area, employing TSMC's CoWoS technology to achieve 2.5D packaging of High Bandwidth Memory (HBM) and Graphics Processing Unit (GPU) via an adapter board. Its representative product, the H100, offers up to 80GB of storage capacity. However, the high cost, large area, and high power consumption of this technology pose challenges for its application in edge computing and edge AI.

[0051] Therefore, future research should focus on developing more efficient storage and computing chip architectures to meet the growing demands of artificial intelligence. This includes exploring new architectures and optimizing packaging technologies. Furthermore, integrated computing and storage solutions, such as those employing novel non-volatile memory (NVM) or hybrid storage architectures, may address this issue. These innovations will be key to driving the widespread adoption and application of smart terminal devices, ultimately achieving true intelligence and data security.

[0052] Current solutions can be broadly categorized as follows: The first is the traditional von Neumann architecture, which separates the computing chip from the memory chip. This approach has relatively low bandwidth, making it difficult to meet the needs of modern artificial intelligence models. The second involves three-dimensional packaging of SRAM and the computing chip using hybrid bonding technology. However, due to the limited storage density of SRAM, this solution cannot meet the storage capacity requirements of artificial intelligence models. The third approach uses advanced packaging technology to package DRAM and the computing chip together. However, DRAM storage capacity is also limited, so multi-layer DRAM stacking is typically used to form high-bandwidth memory (HBM) for packaging. However, this approach results in higher costs, larger area, and relatively higher power consumption.

[0053] To address the problems existing in the above-mentioned solutions, this application provides a hybrid storage computing architecture aimed at solving these problems. After introducing the background technology of the hybrid storage computing architecture provided in the embodiments of this application, the following focuses on the hybrid storage computing architecture proposed in this solution.

[0054] In one embodiment, such as Figure 1 As shown, a hybrid storage computing architecture 01 is provided. The hybrid storage computing architecture 01 includes a first storage chip 10, a second storage chip 20, and a computing chip 30. The computing chip 30 is connected to the first storage chip 10 and the second storage chip 20 respectively. The storage speed of the second storage chip 20 is greater than that of the first storage chip 10, and the storage density of the first storage chip 10 is greater than that of the second storage chip 20.

[0055] The first storage chip 10 and the second storage chip 20 are used to store data related to Large Language Models (LLMs).

[0056] The computing chip 30 is used to acquire relevant data from the first storage chip 10 and the second storage chip, and to perform calculations for large language model training or large language model application based on the relevant data, so as to obtain the calculation results.

[0057] The first memory chip can refer to a NAND memory chip, and the second memory chip can refer to a DRAM memory chip.

[0058] The data related to the Large Language Model (LLM) includes the training data for the large language model, the intermediate data during the training process, and the intermediate data during the computation process.

[0059] It should be noted that, see Figure 2The hybrid memory computing architecture provided in this application may include: fabricating a NAND flash memory array and its peripheral circuitry on a first semiconductor substrate, fabricating a DRAM memory array and its peripheral circuitry on a second semiconductor substrate, and fabricating computing circuitry on a third semiconductor substrate, wherein the three semiconductor substrates form a stacked architecture (see [reference]). Figure 3 ).

[0060] Optionally, the first and second semiconductors are bonded face-to-back to form an interface between them. Through-Silicon Via (TSV) technology is then used to guide signals from the second semiconductor to the back side. Bonding techniques include hybrid bonding, fusion bonding, transfer bonding, and eutectic bonding. Alternatively, the second and third semiconductors can be bonded face-to-face using a suitable bonding technique to form the interface. The third semiconductor, through bonding bumps and a TSV structure penetrating the second semiconductor, forms an electrical interconnect with the first semiconductor. Through the peripheral circuitry of the first semiconductor, voltage and / or current signals traveling from the bit lines to each target memory cell on the first semiconductor can be applied or sensed via bit lines, word lines, common-source lines, SSG lines, and DSG lines to perform read, erase, and write operations on the NAND flash memory cell array. The second semiconductor, through bonding bumps, forms an electrical interconnect with the first semiconductor. Through the peripheral circuitry of the second semiconductor, read, write, and erase operations are performed on the DRAM memory cell array.

[0061] It should be noted that the stacking order of the first memory chip, the second memory chip, and the computing chip can be adjusted, and the vertical relationship and the number of stacked chips can be changed. In this embodiment, there is no limitation on the number and connection relationship of each chip.

[0062] The hybrid storage computing architecture provided in this application embodiment, by setting the first storage chip, the second storage chip, and the computing chip in the same architecture, allows the second storage chip to efficiently store relevant data of the big data model, while the first storage chip provides sufficient storage space so that the computing chip can directly obtain relevant data for training the big data model from the first and second storage chips in the same architecture, and perform calculations for big data model training or big data model application based on the relevant data to obtain the calculation results. Compared with computing chips and storage chips being located in different architectures, this application can greatly improve the efficiency of data storage and computing.

[0063] In one embodiment, such as Figure 4 As shown, the first memory chip 10 includes at least one memory cell array 101 and a first peripheral circuit 102; the memory cell array 101 is composed of at least one NAND memory cell 1011 arranged periodically.

[0064] NAND storage cell 1011 is used to store the first data in the relevant data; the first data refers to the training data of the large language model;

[0065] The first peripheral circuit 102 is used to perform processing operations on the first data to obtain the processed first data; the processing operations include at least one of writing operations, reading operations, and erasing operations.

[0066] The computing chip 30 is used to perform calculations on the processed first data for large language model training or large language model application, and obtain the calculation results.

[0067] In the embodiments of this application, see Figure 5 The first memory chip 10 includes a NAND memory cell array and a first peripheral circuit. The NAND memory cell array includes at least one periodically arranged NAND memory cell. The NAND memory cell is a stack of memory, that is, stacked and interleaved conductive layers and dielectric layers. The NAND memory cell includes interconnects and gate transistors, wherein the interconnects include word lines, bit lines, DSG lines, common source lines, and SSG lines.

[0068] Figure 5 Only one NAND memory cell array is shown, but at least one NAND memory cell array can be fabricated. The first peripheral circuitry (also known as control and sensing circuitry) may include any suitable digital, analog, and / or mixed-signal circuitry used to facilitate the operation of the memory cell array. Examples include column decoders / bit line drivers, row decoders / word line drivers, sense amplifiers (SAs), logic control units, registers, etc.

[0069] It should be noted that NAND flash memory cells can be 2D or 3D structures; they can be CnA (CMOS Near Array) or CuA (CMOS Under Array); or the CMOS and array can be designed and manufactured separately on two independent semiconductor wafers and then bonded together. A single NAND flash chip can be used, or multiple chips can be used.

[0070] The first storage chip provided in this embodiment can efficiently store and transmit data, providing a foundation for efficient training of big data models based on the data in the first storage chip.

[0071] In one embodiment, such as Figure 6 As shown, the second memory chip 20 includes at least one periodically arranged DRAM memory cell 201, and the DRAM memory cell 201 includes a DRAM device 2011 and a second peripheral circuit 2012.

[0072] DRAM device 2011 is used to store the second data in the relevant data; the second data is intermediate data for training the large language model, or intermediate data for computation of the large language model.

[0073] The second peripheral circuit 2012 is used to perform processing operations on the second data to obtain the processed second data; the processing operations include at least one of write operations, read operations and erase operations.

[0074] The computing chip 30 is used to perform calculations on the processed second data for big data model training or big data model application, and obtain the calculation results.

[0075] In the embodiments of this application, see Figure 7 To improve interconnect bandwidth, a distributed scheme is adopted. The second memory chip 20 includes at least one periodically arranged DRAM memory cell 201, i.e., with a Bank as the smallest repeating unit. Each Bank includes DRAM devices 2011 with a certain storage capacity and corresponding second peripheral circuitry 2012. The second peripheral circuitry 2012 (also called control and sensing circuitry) may include any suitable digital, analog, and / or mixed-signal circuitry for facilitating the operation of the memory cell array. For example, it may include column decoders / bit line drivers, row decoders / word line drivers, sense amplifiers (SA), logic control units, registers, etc. The DRAM devices include interconnects and gating transistors, wherein the interconnects include word lines, bit lines, and common-source lines.

[0076] It should be noted that DRAM memory chips can be used individually or in multiples, connected together via TSVs, microbumps, or hybrid bonding. Furthermore, multiple DRAM chips can be configured with a single logic chip to form a high-bandwidth memory (HBM).

[0077] The second storage chip provided in this embodiment can realize large-capacity data storage and transmission, providing a foundation for efficient training of big data models based on the data in the second storage chip.

[0078] The following provides the specific structures of four hybrid storage computing architectures 01, including:

[0079] First, in one embodiment, see Figure 8 The hybrid storage computing architecture 01 also includes a cache chip, the first storage chip includes at least one NAND storage sub-chip, and the second storage chip includes at least one DRAM storage sub-chip.

[0080] Each NAND storage sub-chip is connected to the computing chip in sequence;

[0081] Each DRAM storage sub-chip and cache chip is connected to the computing chip in sequence, and each NAND storage sub-chip, each DRAM storage sub-chip and cache chip is located on the same side of the computing chip.

[0082] A cache chip is used to retrieve second data from the second storage chip 20 and vertically transmit the second data to the computing chip 30.

[0083] Among them, the cache chip can be a buffer die. The buffer die is a key component in high bandwidth memory (HBM) technology. Its connection method is as follows: the buffer die is connected to the host xPU (such as CPU or GPU) through a silicon interposer. Its stacking structure is that in HBM technology, multiple DRAM dies are stacked vertically and communicate with the buffer die through silicon vias (TSVs).

[0084] In this embodiment of the application, the above-mentioned hybrid storage computing architecture 01 further includes a connection chip 40. The computing chip 30 can obtain second data from the second storage chip 20 through the connection chip 40, and perform calculations for big data model training or big data model application based on the second data to obtain the calculation results.

[0085] The connectivity chip provided in this application provides a connection bridge between the computing chip and the second storage chip, thus providing the conditions for realizing an integrated storage and computing architecture of the computing chip and the second storage chip.

[0086] Second, in one embodiment, see Figure 9 ,exist Figure 8 Based on the hybrid storage computing architecture 01 shown, the hybrid storage computing architecture 01 also includes a NAND controller chip;

[0087] Each NAND storage sub-chip and NAND controller chip is connected to the computing chip in sequence.

[0088] It should be noted that the hybrid storage computing architecture also includes a NAND controller chip, which is connected to the computing chip 30 and the first storage chip 10 respectively. The NAND controller chip is used to manage the NAND storage cells in the first storage chip 10, send data read instructions to the first storage chip 10, and receive the computing data corresponding to the data read instructions.

[0089] Third, in one embodiment, see Figure 10It also provides a hybrid storage computing architecture that includes a cache chip, a first storage chip including at least one NAND storage sub-chip, and a second storage chip including at least one DRAM storage sub-chip;

[0090] Each NAND storage sub-chip, each DRAM storage sub-chip, and the cache chip are sequentially connected to the computing chip.

[0091] Fourth, in one embodiment, see [link to embodiment]. Figure 11 ,exist Figure 10 Based on the hybrid storage computing architecture 01 shown, the hybrid storage computing architecture 01 also includes a NAND controller chip;

[0092] Each NAND storage sub-chip, NAND controller chip, each DRAM storage sub-chip, and cache chip are sequentially connected to the computing chip.

[0093] The hybrid storage computing architecture provided in this application embodiment can realize the functions of large model data storage and inference with a single chip. In terms of data storage requirements, this includes large model parameters that need long-term storage and input encoded data, KV cache, and other data that need temporary storage. Since NAND flash has non-volatile characteristics and DRAM has high bandwidth characteristics, large model parameters that need long-term storage are stored in NAND flash, and data that needs temporary storage is stored in DRAM. In this way, the function of deploying and inferring large models with a single chip is realized.

[0094] In one embodiment, the specific contents of the first memory chip 10 can be further defined. The first memory chip 10 includes at least one of the following: a chip in which the NAND memory array and the peripheral control circuit are arranged side by side; a chip in which the peripheral control circuit is arranged below the NAND memory array; and a chip in which the NAND memory array and the peripheral control circuit are respectively arranged on two semiconductor wafers and the two semiconductor wafers are bonded together.

[0095] In one embodiment, the specific contents of the second memory chip 20 can also be defined, wherein the second memory chip 20 includes a chip in which multiple DRAM layers are stacked and distributed based on through silicon via (TSV) technology.

[0096] In one embodiment, such as Figure 12 As shown, the aforementioned computing chip 30 includes a computing core, cache, computing management unit, interface, and storage control unit;

[0097] The storage control unit is used to send data read commands to the first storage chip 10 and / or the second storage chip 20, and to receive the calculated data corresponding to the data read commands;

[0098] Cache 302 is used to receive and cache computation data;

[0099] The computing core 301 is used to process the computational data and obtain the computational results;

[0100] The computing management unit 303 is used to allocate computing resources to the computing core during the computing process of computing data;

[0101] Interface 304 is used to transmit calculation results to external devices.

[0102] In this embodiment, the computing chip 30 can adopt a GPU architecture, including a computing core (which may further include an arithmetic logic unit, a special function processing unit, a tensor calculation unit, registers, etc.), a cache, a computing management unit, an interface, and a storage control unit. For the computing core, the arithmetic logic unit performs arithmetic and logical operations; the special function processing unit performs trigonometric functions, square roots, and other special functions; the tensor calculation unit handles tensor calculations; and registers store variables and temporary data required during the calculation process. The cache can be an L2 cache, shared by all computing cores, used for storing data during the calculation process. The computing management unit is used for allocating and managing computing resources throughout the entire calculation process. The storage control unit is used to control the NAND flash memory and DRAM memory. The interface is used for communication with external devices.

[0103] The specific structure of the computing chip provided in this application embodiment provides conditions for realizing an integrated storage and computing architecture of the computing chip, the second storage chip, and the first storage chip.

[0104] In one embodiment, see Figure 13 It also provides the basic process flow for manufacturing hybrid storage computing architectures, including:

[0105] (1) On a first semiconductor substrate, a NAND memory cell array and a first peripheral circuit are fabricated separately. This can be achieved by first fabricating the NAND memory cell array by shielding the area of ​​the first peripheral circuit; then, by shielding the area of ​​the NAND memory cell array and fabricating the first peripheral circuit. A 3D NAND memory cell array and / or a 2D NAND memory cell array are fabricated. Taking NAND as an example, NAND memory cells can be organized into pages, and pages can be organized into blocks. Each NAND memory cell is connected to a separate bit line (BL). NAND memory cells at the same vertical position are connected by word lines (WL) through control gates. At the top of the NAND memory string is a gate transistor connected to the DSG line; at the bottom is a gate transistor connected to the SSG line. The first semiconductor structure may include one or more memory surfaces / blocks. In addition, on the first semiconductor substrate, the first peripheral circuit required to perform read / write / erase operations on the 3D NAND memory array is fabricated. The first peripheral circuit may include a column decoder / bit line driver, a row decoder / word line driver, a sense amplifier (SA), a control logic unit, registers, etc. The first peripheral circuit can be implemented using a process at a suitable technology node.

[0106] (2) Fabricate a DRAM memory cell array and a second peripheral circuit on the second semiconductor substrate. The two parts are fabricated separately. The method can be to first complete the fabrication of the DRAM memory cell array by blocking the area of ​​the second peripheral circuit; then block the area of ​​the DRAM memory cell array and fabricate the second peripheral circuit.

[0107] (3) The computing circuit is fabricated on the third semiconductor, which may include a computing core (including arithmetic logic unit, special function processing unit, tensor calculation unit, register, etc.), L2 cache, computing management unit, interface, memory control unit, etc. The computing circuit can be implemented using a process of a suitable technology node.

[0108] (4) Thinning the second semiconductor substrate. Thinning processes can be used to thin the silicon substrate by means of, but are not limited to, wafer grinding, dry etching, wet etching, CMP, any other suitable process or any combination thereof.

[0109] (5) The first semiconductor and the second semiconductor are arranged face to back to face using a suitable bonding technique to form an interface between the two semiconductors, such as hybrid bonding, fusion bonding, transfer bonding, eutectic bonding, etc.

[0110] (6) The second semiconductor and the third semiconductor are bonded together in a face-to-face orientation. A submicron or tens of micron through-contact (such as a TSV / ILV structure) is formed on the second semiconductor perpendicular to the structure of the second semiconductor and the third semiconductor, so that a direct, short-distance (e.g., submicron level) electrical connection is formed between the second semiconductor and the third semiconductor.

[0111] (7) Thinning the first or third semiconductor substrate. The thinning process can be performed by processes including but not limited to wafer grinding, dry etching, wet etching, CMP, any other suitable process or any combination thereof to thin the silicon substrate.

[0112] (8) Form pads to lead out interconnect layers.

[0113] Based on the same inventive concept, this application also provides a storage computing method for implementing the hybrid storage computing architecture described above. The solution provided by this method is similar to the implementation described in the above method; therefore, the specific limitations in one or more storage computing method embodiments provided below can be found in the limitations regarding the hybrid storage computing architecture described above, and will not be repeated here.

[0114] In one exemplary embodiment, such as Figure 14 As shown, a storage computing method is provided. Taking the application of this method to the storage computing chip structure provided in the above embodiments as an example, the method includes:

[0115] S201. Obtain relevant data of the large language model stored in the first and second memory chips.

[0116] In this embodiment of the application, when it is necessary to perform calculations on the relevant data in the large language model stored in the storage computing chip structure, the computing chip can first obtain the relevant data of the large language model stored in the first storage chip and the second storage chip.

[0117] S202. Perform calculations for training or applying the large language model based on relevant data from the large language model, and obtain the calculation results.

[0118] In this embodiment of the application, after obtaining the relevant data of the large language model stored in the first and second memory chips, the calculation of large language model training or application can be performed based on the relevant data of the large language model to obtain the calculation result.

[0119] The storage and computation method provided in this application provides a data foundation for the efficient computation of large language models.

[0120] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0122] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A hybrid storage computing architecture, characterized in that, The hybrid storage computing architecture includes: a first storage chip, a second storage chip, a computing chip, and a cache chip; the computing chip is connected to the first storage chip and the second storage chip respectively, the storage speed of the second storage chip is greater than the storage speed of the first storage chip, and the storage density of the first storage chip is greater than the storage density of the second storage chip; the first storage chip includes at least one storage cell array, a first peripheral circuit, and at least one NAND storage sub-chip; the storage cell array is composed of at least one NAND storage cell arranged periodically; the second storage chip includes at least one periodically arranged DRAM storage cell and at least one DRAM storage sub-chip, the DRAM storage cell including a DRAM device and a second peripheral circuit; each NAND storage sub-chip is sequentially connected to the computing chip; each DRAM storage sub-chip and the cache chip are sequentially connected to the computing chip, and each NAND storage sub-chip, each DRAM storage sub-chip, and the cache chip are disposed on the same side of the computing chip; The NAND storage unit is used to store the first data in the relevant data of the large language model LLM; the first data refers to the training data of the large language model. The DRAM device is used to store the second data in the related data; the second data is intermediate data for training the large language model, or intermediate data for calculating the large language model. The first peripheral circuit is used to process the first data to obtain processed first data; the processing operation includes a write operation, a read operation, and an erase operation. The second peripheral circuit is used to process the second data to obtain processed second data; the processing operations include write operations, read operations, and erase operations; The computing chip is used to perform large language model training or large language model application calculations on the processed first data to obtain calculation results, and to perform large language model training or large language model application calculations on the processed second data to obtain the calculation results.

2. The hybrid storage computing architecture according to claim 1, characterized in that, The NAND memory cell includes interconnects and gating transistors, and the interconnects include word lines, bit lines, DSG lines, common source lines, and SSG lines.

3. The hybrid storage computing architecture according to claim 1, characterized in that, The first peripheral circuit includes at least one of digital signal circuit, analog signal circuit, and mixed signal circuit.

4. The hybrid storage computing architecture according to claim 1, characterized in that, The second peripheral circuit includes at least one of digital signal circuit, analog signal circuit, and mixed signal circuit.

5. The hybrid storage computing architecture according to claim 1, characterized in that, The hybrid storage computing architecture also includes a NAND controller chip; Each of the NAND storage sub-chips and the NAND control chip are sequentially connected to the computing chip.

6. The hybrid storage computing architecture according to claim 5, characterized in that, The hybrid storage computing architecture also includes a NAND controller chip; Each of the NAND storage sub-chips, the NAND control chip, each of the DRAM storage sub-chips, and the cache chip are sequentially connected to the computing chip.

7. The hybrid storage computing architecture according to claim 1, characterized in that, The first memory chip includes at least one of the following: a chip in which a NAND memory array and peripheral control circuitry are arranged side by side; a chip in which the peripheral control circuitry is arranged below the NAND memory array; and a chip in which the NAND memory array and peripheral control circuitry are respectively disposed on two semiconductor wafers and bonded together.

8. The hybrid storage computing architecture according to claim 1, characterized in that, The second memory chip includes a chip in which multiple DRAM layers are stacked and distributed based on through-silicon via (TSV) technology.

9. A storage computing method, characterized in that, Applied to the storage computing chip structure as described in any one of claims 1-8, comprising: Acquire relevant data of the large language model stored in the first and second memory chips; The calculation results are obtained by performing calculations on the large language model training or application based on the relevant data of the large language model.

Images