Near memory computing chip and end side equipment
By stacking interface control chips, memory chips, and logic chips through 3D packaging technology, the limitations of computing resources and cost issues of edge devices are solved, achieving high-bandwidth data transmission and improved energy efficiency, which is suitable for accelerated applications in devices such as AI mobile phones and AIPCs.
Patent Information
- Application Number
- CN202520176114.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-27
- Publication Date
- 2026-02-27
- Estimated Expiration
- 2035-01-27
AI Technical Summary
Existing edge devices suffer from performance bottlenecks and increased energy consumption due to limited computing and storage resources. Existing near-memory computing architectures face storage density and cost issues when integrating computing logic on DRAM technology, and computing logic area is severely wasted under CMOS technology.
The interface control chip, memory chip and logic chip are stacked using a three-dimensional packaging process, separating the memory interface and control circuit. High bandwidth data transmission is achieved by using TSV, hybrid bonding or RDL technology, which reduces the waste of logic chip area and reduces costs.
It improves the performance and energy efficiency of large-scale models running on edge devices, reduces chip manufacturing costs, and is suitable for accelerated applications in devices such as AI smartphones and AIPCs.
Smart Images
Figure CN223956073U_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The utility model relates to artificial intelligence chip technical field, specifically to a near storage computing chip and end side equipment. BACKGROUND
[0002] In the prior art, with the rapid development of artificial intelligence (AI) technology, the application scenarios based on large pre-trained models (Large Pre-trained Models) are continuously expanding, such as natural language processing, image recognition, and multi-modal interaction. Due to its excellent performance, large models have been widely used in the cloud and end-side devices. However, with the continuous growth of model parameter size, migrating large models from the cloud to end-side devices (such as AI phones and AIPC) faces many challenges.
[0003] As shown in Figure 1 The computing resources and storage resources of end-side devices are relatively limited compared to cloud servers due to power consumption, volume, and real-time response requirements. Especially during the large model inference process, frequent memory access can cause a large amount of data to be transmitted between the processor and the memory. The current mainstream storage medium is DRAM, which is limited by data transmission bandwidth and access delay, and becomes an important bottleneck restricting the performance of large model inference. In addition, frequent data transmission can significantly increase system energy consumption, negatively affecting the endurance of mobile devices.
[0004] To solve the above problems, a common technical means is to integrate the computing unit directly into the memory system, such as embedding a processor unit or a computing engine inside the DRAM. This "near-memory computing" architecture effectively reduces the distance and frequency of data transmission, improving data processing efficiency. The core concept of memory-compute integration is to perform computing tasks on-site, i.e., directly completing calculations at the data storage location, thereby to some extent breaking through the bottleneck of the traditional "storage-compute" separation architecture, significantly improving the performance and energy efficiency of running large models on end-side devices. The design and optimization of memory-compute integration architecture for large model end-side acceleration have important research value and application prospects.
[0005] The existing near-memory architecture is as shown in Figure 2As shown, in edge device 1, computing logic is integrated on a DRAM chip, and part of the computation is performed within the DRAM chip. The SoC accesses the memory through the memory interface and uses the computing units within the SoC to complete the remaining computation. Because some operations are performed on the DRAM chip, the data transmission bandwidth requirement of the SoC is reduced. However, due to factors such as DRAM storage density, refresh time, and leakage current, the process technology used in DRAM is relatively outdated compared to computing chips. Integrating large-scale computing logic on DRAM technology severely impacts the original DRAM storage density, which limits the number of computing units that can be integrated and the wiring space for DRAM memory access, making it impossible to meet the high computing power and high bandwidth requirements of large models. In terms of energy efficiency, tightly coupling the computing logic with the DRAM memory unit in terms of process technology prevents the computing logic from leveraging its energy efficiency advantages. At the same time, the slower the evolution of DRAM technology lags behind CMOS technology, making this technology unsustainable.
[0006] like Figure 3 As shown, in edge device 2, DRAM chips and computing logic achieve high-bandwidth interconnection through a three-dimensional stacking configuration. Compared to device 1, which decouples the computing logic's process platform from the DRAM memory chip platform, the computing logic can utilize more advanced CMOS logic processes to achieve large-scale computing power integration. In this device, the near-memory computing architecture, due to its ability to integrate large-scale computing power, significantly reduces the SoC's computing pressure or allows the near-memory computing architecture to handle end-to-end applications of large models. This solves the performance bottleneck of large model inference caused by the low data transfer bandwidth of existing memory interfaces.
[0007] However, in edge device 2, the computing logic uses CMOS technology, which is affected by Moore's Law, resulting in a significant decrease in computing density as the technology evolves. With advanced CMOS technology, the area required for the computing logic is much smaller than that required by DRAM memory chips. However, due to the wafer-to-wafer packaging, the area of the DRAM memory chip must be equal to that of the computing logic, leading to significant area waste and cost losses. As the technology used becomes more advanced, these cost losses will also increase.
[0008] Therefore, it is necessary to improve existing edge devices and near-memory computing chips. Utility Model Content
[0009] In order to solve the above technical problems, the utility model provides a kind of near storage computing chip, including interface control chip, memory chip and logic chip sequentially stacked with three-dimensional packaging process;The interface control chip is configured to be connected with SoC system, and is connected with memory chip and logic chip, to provide the SoC system to realize the access to the memory chip and the control to the logic chip.
[0010] The near storage computing chip provided by the utility model integrates memory interface circuit and control circuit into an independent chip, so that the size of the memory chip can be reduced under fixed storage capacity, and the cost of using advanced technology for iteration of the logic chip is also reduced, thereby reducing the overall cost of the near storage computing chip.
[0011] Optionally, the interface control chip includes a high-speed interface circuit, a memory control circuit and a logic chip control circuit;The high-speed interface circuit is configured to be connected with the SoC system to interact information with the SoC system;The memory chip control circuit is connected with the memory chip and the high-speed interface circuit respectively, and can send control instructions to the memory chip based on the information transmitted by the SoC system through the high-speed interface circuit;The logic chip control circuit is connected with the logic chip and the high-speed interface circuit respectively, and can send control instructions to the logic chip based on the information transmitted by the SoC system through the high-speed interface circuit.
[0012] Optionally, the memory chip includes a plurality of memory groups, each of which is connected with the memory chip control circuit and can interact data with the memory chip control circuit in response to the instructions sent by the memory chip control circuit.
[0013] Optionally, the logic chip includes a controller, a computing engine and a memory group controller;The controller is connected with the logic chip control circuit, can receive the driving signal sent by the logic chip control circuit, and drives the computing engine;The computing engine is arranged one-to-one with the memory group controller, and is connected with the corresponding memory group controller;The memory group controller is connected with the corresponding memory group, and the computing engine can drive the memory group controller to control the corresponding memory group.
[0014] Optionally, the memory chip is designed as a memory group arrangement structure of dynamic random access memory with high density characteristics.
[0015] Optionally, the interface control chip is connected with the memory chip and / or the logic chip by RDL, TSV or hybrid bonding process.
[0016] Optionally, the storage chip and the logic chip are connected by RDL, TSV or hybrid bonding process.
[0017] Optionally, the interface control chip, the storage chip and the logic chip are packaged with the substrate by metal balls, the interface control chip, the storage chip and the logic chip each include a substrate, a circuit layer and a metal layer stacked in sequence, and the metal layer is arranged towards a side close to the substrate, the metal layer of the logic chip is electrically connected with the metal layer of the interface control chip and the storage chip respectively by RDL, TSV or hybrid bonding process, the metal layer of the interface control chip and the storage chip is electrically connected by RDL, TSV or hybrid bonding process, and the interface control chip is electrically connected with the substrate by metal balls.
[0018] Optionally, the interface control chip, the storage chip and the logic chip are packaged with the substrate by Wire Bonding, the interface control chip, the storage chip and the logic chip each include a substrate, a circuit layer and a metal layer stacked in sequence, the metal layer of the storage chip and the logic chip is arranged towards a side close to the substrate, and the metal layer of the interface control chip is arranged towards a side away from the substrate, the logic chip is electrically connected with the storage chip and the interface control chip by RDL, TSV or hybrid bonding process, the interface control chip and the storage chip are electrically connected by RDL, TSV or hybrid bonding process, and the logic chip and the interface control chip are electrically connected with the substrate by Wire Bonding.
[0019] To achieve the above-mentioned application purposes, the application provides an end-side device including the near-memory computing chip and the SoC system as described above, the SoC system is connected with the near-memory computing chip and can control the operation of the near-memory computing chip through the interface control chip.
[0020] The near-memory computing chip provided by the application stacks at least three chips by three-dimensional packaging, the architecture separates the memory interface from the control circuit, realizes high-bandwidth data transmission by using multi-layer stacking technology, reduces the area waste of the logic chip, realizes efficient interconnection between layers by using TSV, hybrid bonding or RDL technology, effectively reduces the energy efficiency problem caused by the coupling of the storage chip and the logic chip, reduces the area waste of the logic chip under advanced process, significantly improves the performance and energy efficiency of the end-side device running large models, reduces the chip manufacturing cost, and is suitable for large model acceleration application of AI mobile phones, AIPC and other end-side devices. BRIEF DESCRIPTION OF DRAWINGS
[0021] Figure 1is a structural schematic diagram of an end-side device of a memory-compute separation architecture in the prior art.
[0022] Figure 2 is a structural schematic diagram of another end-side device of a memory-compute separation architecture in the prior art.
[0023] Figure 3 is a structural schematic diagram of another end-side device of a memory-compute separation architecture in the prior art.
[0024] Figure 4 is a structural schematic diagram of a near-memory computing chip provided in an embodiment of the present application.
[0025] Figure 5 is a structural schematic diagram of a packaging structure of a near-memory computing chip provided in an embodiment of the present application.
[0026] Figure 6 is a structural schematic diagram of a packaging structure of a near-memory computing chip provided in an embodiment of the present application.
[0027] Figure 7 is a structural schematic diagram of an end-side device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.
[0029] As shown in Figure 4 The present embodiment provides a near-memory computing chip, which comprises an interface control chip 100, a memory chip 200 and a logic chip 300 stacked in sequence by a three-dimensional packaging process; the interface control chip 100 is configured to be connected with an SoC system and connected with the memory chip 200 and the logic chip 300, so that the SoC system can access the memory chip 200 and control the logic chip 300.
[0030] The near-memory computing chip provided by the present embodiment integrates the memory interface circuit and the control circuit into an independent chip, so that the size of the memory chip can be reduced under a fixed storage capacity, and the cost of using advanced process for iteration of the logic chip is also reduced, thereby reducing the cost of the near-memory computing chip as a whole.
[0031] Optionally, the interface control chip 100 comprises a high-speed interface circuit 110, a memory control circuit 120 and a logic chip control circuit 130; the high-speed interface circuit 110 is configured to connect with the SoC system to interact information with the SoC system; the memory chip control circuit 120 is connected with the memory chip 200 and the high-speed interface circuit 110 respectively, and can send control instructions to the memory chip 200 based on the information transmitted by the SoC system through the high-speed interface circuit 110; the logic chip control circuit 130 is connected with the logic chip 300 and the high-speed interface circuit 110 respectively, and can send control instructions to the logic chip 300 based on the information transmitted by the SoC system through the high-speed interface circuit 110.
[0032] Optionally, the memory chip 200 comprises a plurality of memory groups 210, each of which is connected with the memory chip control circuit 120 and can interact data with the memory chip control circuit 120 in response to the instructions sent by the memory chip control circuit 120. More specifically, each memory group 210 further comprises a plurality of memory blocks 211.
[0033] Optionally, the logic chip 300 comprises a controller 310, a computing engine 320 and a memory group controller 330; the controller 310 is connected with the logic chip control circuit 130 and can receive the driving signal sent by the logic chip control circuit 130 to drive the computing engine 320; the computing engine 320 and the memory group controller 330 are set one by one, the computing engine 320 is connected with the corresponding memory group controller 330, the memory group controller 330 is connected with the corresponding memory group 210, and the computing engine 320 can drive the memory group controller 330 to control the corresponding memory group 210.
[0034] In this embodiment, a separate computing engine 320 and a memory group controller 330 are arranged for each memory group 210, so that each memory group can execute the instructions of the logic chip 300 simultaneously or independently, realizing high-bandwidth data transmission.
[0035] Optionally, the memory chip 200 is designed as a memory group arrangement structure of a dynamic random access memory (DRAM) with high density characteristics.
[0036] Optionally, the logic chip 300 is designed by using a CMOS process.
[0037] Optionally, the interface control chip 100 is connected with the memory chip 200 and / or the logic chip 300 by using a RDL, TSV or hybrid bonding process.
[0038] Optionally, the memory chip 200 and the logic chip 300 are connected by using a RDL, TSV or hybrid bonding process.
[0039] Optionally, referring to Figure 5 , the interface control chip 100, the storage chip 200 and the logic chip 300 are packaged with the substrate in a metal ball mode, in which case, the interface control chip 100, the storage chip 200 and the logic chip 300 each include a substrate, a circuit layer and a metal layer stacked in sequence, the metal layer of the logic chip 300 is arranged towards the side close to the substrate, the metal layer of the interface control chip 100 and the metal layer of the storage chip 200 are electrically connected through a RDL, a TSV or a hybrid bonding process, the metal layer of the interface control chip 100 and the metal layer of the storage chip 200 are electrically connected through a RDL, a TSV or a hybrid bonding process, and the interface control chip 100 is electrically connected with the substrate through a metal ball.
[0040] Optionally, referring to Figure 6 , the interface control chip 100, the storage chip 200 and the logic chip 300 are packaged with the substrate in a Wire Bonding mode, in which case, the interface control chip 100, the storage chip 200 and the logic chip 300 each include a substrate, a circuit layer and a metal layer stacked in sequence, the metal layer of the storage chip 200 and the logic chip 300 is arranged towards the side close to the substrate, the metal layer of the interface control chip 100 is arranged towards the side away from the substrate, the logic chip 300 is electrically connected with the storage chip 200 and the interface control chip 100 through a RDL, a TSV or a hybrid bonding process, the interface control chip 100 and the storage chip 200 are electrically connected through a RDL, a TSV or a hybrid bonding process, and the logic chip 300 and the interface control chip 100 are electrically connected with the substrate through a Wire Bonding mode.
[0041] Optionally, referring to Figure 7 , the embodiment further provides an end-side device, which includes a SoC system and the near-memory computing chip provided by the embodiment, the SoC system is connected with the near-memory computing chip and can control the operation of the near-memory computing chip through the interface control chip 100.
[0042] Thus, the technical scheme of the present application has been described in conjunction with the drawings. However, it is easily understood by those skilled in the art that the protection scope of the present application is obviously not limited to the above-mentioned specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without deviating from the principles of the present application, and the technical schemes after the changes or replacements will all fall within the protection scope of the present application.
Claims
1. A near-memory computing chip, comprising: The interface control chip, the storage chip and the logic chip are sequentially stacked by a three-dimensional packaging process; the interface control chip is configured to be connected with an SoC system and connected with the storage chip and the logic chip, so that the SoC system realizes access to the storage chip and control over the logic chip.
2. The compute-near-memory chip of claim 1, wherein, The interface control chip includes a high-speed interface circuit, a memory control circuit and a logic chip control circuit; the high-speed interface circuit is configured to be connected with the SoC system to interact with the SoC system; the memory chip control circuit is connected with the storage chip and the high-speed interface circuit respectively, and can send a control instruction to the storage chip based on information transmitted by the SoC system through the high-speed interface circuit; the logic chip control circuit is connected with the logic chip and the high-speed interface circuit respectively, and can send a control instruction to the logic chip based on information transmitted by the SoC system through the high-speed interface circuit.
3. The compute-near-memory chip of claim 2, wherein, The storage chip includes a plurality of memory banks, each of which is connected with the memory chip control circuit and can interact with the memory chip control circuit based on an instruction sent by the memory chip control circuit.
4. The compute-near-memory chip of claim 3, wherein, The logic chip includes a controller, a computing engine and a memory bank controller; the controller is connected with the logic chip control circuit and can receive a driving signal sent by the logic chip control circuit to drive the computing engine; the computing engine is arranged in one-to-one correspondence with the memory bank controller and is connected with the corresponding memory bank controller; the memory bank controller is connected with the corresponding memory bank; and the computing engine can drive the memory bank controller to control the corresponding memory bank.
5. The compute-in-memory chip of any one of claims 1-4, wherein, The storage chip is designed to have a high-density dynamic random access memory bank arrangement structure.
6. The compute-in-memory chip of any one of claims 1-4, wherein, The interface control chip, the storage chip and / or the logic chip are connected by an RDL, TSV or hybrid bonding process.
7. The compute-in-memory chip of any one of claims 1-4, wherein, The storage chip and the logic chip are connected by an RDL, TSV or hybrid bonding process.
8. The compute-near-memory chip of any one of claims 1-4, wherein, The interface control chip, the storage chip and the logic chip are packaged with the substrate by using metal balls; the interface control chip, the storage chip and the logic chip each include a substrate, a circuit layer and a metal layer stacked in sequence, and the metal layer is arranged towards the side close to the substrate; the metal layer of the logic chip is electrically connected with the metal layer of the interface control chip and the storage chip by an RDL, TSV or hybrid bonding process; the metal layer of the interface control chip and the storage chip is electrically connected by an RDL, TSV or hybrid bonding process; and the interface control chip is electrically connected with the substrate by a metal ball.
9. The compute-in-memory chip of any one of claims 1-4, wherein, The interface control chip, the storage chip and the logic chip are packaged with the substrate in a Wire Bonding mode, the interface control chip, the storage chip and the logic chip each comprise a substrate, a circuit layer and a metal layer stacked in sequence, the metal layer of the storage chip and the logic chip is arranged towards a side close to the substrate, the metal layer of the interface control chip is arranged towards a side away from the substrate, the logic chip is electrically connected with the storage chip and the interface control chip in a RDL, TSV or hybrid bonding process, the interface control chip and the storage chip are electrically connected in a RDL, TSV or hybrid bonding process, and the logic chip and the interface control chip are electrically connected with the substrate in a Wire Bonding mode.
10. An end-side device characterized by comprising: The SoC system is connected with the near-memory computing chip and can control the operation of the near-memory computing chip through the interface control chip.