Artificial intelligence acceleration system based on vertical integration of nonvolatile memory and logic chip

Abstract

The invention discloses an artificial intelligence acceleration system based on vertical integration of a nonvolatile memory and a logic chip, and belongs to the field of chip integration, one layer or multiple layers of nonvolatile memory chips are vertically integrated on the logic chip, and the multiple layers of nonvolatile memory chips are vertically integrated through bumps, micro bumps or a hybrid bonding process. The multi-layer nonvolatile storage chip is used for storing model data, and the physical distance between the position where the model data is stored and the position where the model data is used in the logic chip is as short as possible. The error correction code (ECC) function in the invention can be put on the NVM and also can be put on the logic chip; when the ECC is placed on the NVM chip, the transmission path is long, and when the ECC is placed on the logic chip, the delay is shorter, and the bandwidth is higher.

Description

Technical Field

[0001] The present invention relates to the field of chip integration, and in particular to an artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chips. Background Art

[0002] When processing data-intensive tasks, the von Neumann architecture of traditional artificial intelligence computing needs to frequently move data between the processing unit and the storage unit, resulting in a storage wall. To solve this problem, near-memory computing can often be used. The artificial intelligence computing architecture can be vertically integrated. The vertical integration processes include wafer bumping, wafer micro-bumping (μ-Bumping) and hybrid bonding.

[0003] The vertical integration of traditional volatile memory and logic chips is widely used in artificial intelligence computing. Currently, there are HBM artificial intelligence acceleration system architectures based on TSV and silicon interposer technology, high-performance computing architectures based on three-dimensional vertically integrated SRAM Cache, and DRAM near-memory computing system architectures based on hybrid bonding. However, artificial intelligence computing systems based on volatile memory have prominent problems in edge applications, such as the need to frequently refresh data, high static power consumption, difficulty in achieving deep sleep mode, the need for external non-volatile memory, and high heat generation. It is difficult to achieve higher cost efficiency and energy efficiency, which does not match the needs of emerging applications such as edge computing and embodied intelligence.

[0004] On the other hand, the ECC function of the NVM chip can automatically correct the code errors that occur during data storage and transmission. The design goal of ECC is to reduce the bit error rate of the storage or communication system as much as possible under limited resources. Common ECC encoding and decoding and error correction algorithms can be divided into block codes, convolutional codes, turbo codes, etc. In memory applications, the ECC workflow includes key steps such as data encoding and writing, data reading, error detection, error correction, and result output. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the prior art and propose an artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chips.

[0006] The objective of the present invention is achieved through the following technical solutions: an artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chips, vertically integrating one or more layers of non-volatile memory chips on the logic chip, and vertical integration between the multiple layers of non-volatile memory chips is achieved through bumps, micro-bumps or hybrid bonding processes.

[0007] Furthermore, the multi-layer non-volatile memory chip is used to store the model data, and the physical distance between the location where the model data is stored and the location where the model data is used in the logic chip is as short as possible.

[0008] Furthermore, the model data in the multi-layer non-volatile memory chip is located directly above the computing module in the corresponding logic chip, so that the transmission path is the shortest.

[0009] Furthermore, for vertical integration of a layer of non-volatile memory chips, the electrodes on the top of the non-volatile memory chips are directly used to realize vertical integration, and the logic chip is connected to the packaging substrate through wire bonding during packaging.

[0010] Furthermore, a pad for wire bonding is reserved in the non-overlapping portion of the logic chip and the non-volatile memory chip; the pad is located in the area surrounding the non-volatile memory chip on the surface of the logic chip.

[0011] Furthermore, for vertically integrated multi-layer non-volatile memory chips, it is necessary to implement through silicon vias (TSVs) in the multi-layer non-volatile memory chips to meet the needs of multi-layer stacking. During packaging, the logic chip is connected to the packaging substrate through the through silicon vias (TSVs) inside the logic chip, or is connected to the packaging substrate through wire bonding.

[0012] Furthermore, the error correction code (ECC) function is set on the non-volatile memory chip, and the data distributed at different positions of the chip are concentrated in the ECC for error correction calculation, and then dispersed to the bumps or micro-bumps used in different vertical integration processes and transmitted to the logic chip.

[0013] Furthermore, the error correction code ECC function is set on the logic chip, and the data is directly transmitted to the logic chip through the bumps, micro-bumps or hybrid bonding channels used in the vertical integration process.

[0014] Furthermore, the specific process of chip packaging is as follows: (1) Wafer cutting: NVM and logic chips are cut separately for packaging; (2) Chip stacking: stack the NVM chip and logic chip together to ensure that the corresponding contact points are electrically connected; (3) Substrate manufacturing: design and manufacture of packaging substrates used to connect chips and external pins; (4) Wire bonding: Place the stacked chips on the substrate and connect the pads of the bottom chip to the corresponding pads on the substrate by wire bonding; (5) Final chip packaging: Based on the selected vertical integration process, the final chip packaging is completed on the basis of the vertical integration solution.

[0015] Furthermore, for vertical integration processes using bumping and μ-bump, it is necessary to fill insulating materials between the stacked chips to provide mechanical protection and ensure reliable electrical connections.

[0016] Beneficial effects of the present invention: (1) Significantly reduce the distance between non-volatile memory (NVM) and logical computing units, thereby effectively alleviating the storage wall bottleneck in artificial intelligence computing; (2) Using three-dimensional vertical integration technology to achieve a wider data transmission interface and provide a more flexible data flow implementation method, which can effectively reduce power consumption and heat generation; (3) Leverage NVM near-memory computing to achieve lower-cost, more energy-efficient AI acceleration solutions with deep sleep and instant wake-up capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 Schematic diagram of vertically integrating one or more layers of non-volatile memory (NVM) chips.

[0019] Figure 2 Schematic diagram of the relationship between model data location and calculation module location.

[0020] Figure 3 Schematic diagram of a vertically integrated non-volatile memory (NVM) chip package.

[0021] Figure 4 Schematic diagram of vertically integrated multi-layer non-volatile memory (NVM) chip packaging.

[0022] Figure 5 Schematic diagram of implementing ECC on the NVM chip.

[0023] Figure 6 Schematic diagram of implementing ECC on a logic chip. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with the accompanying drawings and implementation examples. It should be understood that the specific implementation examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0025] like Figure 1As shown, the present invention provides an artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip, which is a near-storage artificial intelligence computing architecture based on NVM, such as Figure 1 As shown, one or more layers of non-volatile memory (NVM) chips are vertically integrated on the logic chip through bumps, micro-bumps or hybrid bonding processes, and the multi-layer non-volatile memory (NVM) chips are vertically integrated through bumps, micro-bumps or hybrid bonding processes. The logic chip has an external I / O interface.

[0026] The storage devices used in non-volatile memory (NVM) chips can be various applicable NVM technologies such as RRAM, MRAM, PCRAM, FeRAM, and flash memory.

[0027] To ensure the efficiency of near-memory computing, NVM chips can be mainly used to store model data (data that needs to be read repeatedly but does not need to be rewritten during the calculation process); the architecture design should meet the following requirements: the location where the model data is stored in the NVM is as close as possible to the location where the model data is used in the logic chip below in terms of physical distance, thereby shortening the transmission path, such as Figure 2 As shown, the model data is located directly above the corresponding calculation module, so that the transmission path is the shortest.

[0028] The logic chip of the present invention includes AI computing functions, and vertically integrates a layer of non-volatile memory (NVM) chip, such as Figure 3 As shown, the electrodes on the top of NVM are directly used to achieve vertical integration, without the need to implement TSV in the NVM chip. During packaging, if the logic chip does not use TSV, its size must be significantly different from that of the NVM chip, so that pads for wire bonding are reserved in the non-overlapping part of the two; the pads are located in the area surrounding the NVM chip on the surface of the logic chip. For vertically integrated multi-layer non-volatile memory (NVM) chips, such as Figure 4 As shown, TSV needs to be implemented in the NVM chip to meet the needs of multi-layer stacking. During packaging, the connection between the logic chip and the packaging substrate can be achieved through wire bonding or through silicon vias (TSV) inside the logic chip. The pad design for wire bonding (wire bonding) is consistent with the vertical integration of a layer of non-volatile memory (NVM) chip.

[0029] The NVM chip has an error correction code (ECC) function, which can automatically correct the code errors that occur during data storage and transmission. The design goal of ECC is to reduce the bit error rate of the storage or communication system as much as possible under limited resources. The ECC function in the present invention can be placed on the NVM or on the logic chip, which depends on the scale of the artificial intelligence algorithm model. When the model scale is small, placing ECC on the NVM can simplify the design of the logic chip and reduce the development cycle and cost; when the model scale is large, placing ECC on the logic chip can achieve higher computing performance and efficiency. When ECC is placed on the NVM chip, such as Figure 5 As shown in Figure 1, data needs to be transmitted in a "scatter-gather-scatter-gather" mode. The data distributed in different locations on the chip must first be concentrated in the ECC module for error correction operations, and then dispersed to different bumps and μ-bumps and transmitted to the logic chip. The transmission path is long. When ECC is placed on the logic chip, Figure 6 As shown, the data only needs to be gathered once on the logic chip, and the original data is transmitted to the logic chip through nearby channels such as bump, μ-bump, and hybrid bonding. No additional aggregation-dispersion process is required, which reduces the delay and increases the bandwidth.

[0030] The specific process of chip packaging in the present invention is as follows: 1. Chip manufacturing: design and manufacture NVM chips and logic chips respectively; 2. Wafer cutting: Cut the NVM and logic chips separately for packaging; 3. Chip stacking: stack the NVM chip and logic chip together to ensure good electrical connection between corresponding contact points; 4. Insulation filling (optional): For vertical integration processes using bumping and μ-bump, it is necessary to fill insulation material between the stacked chips to provide mechanical protection and ensure reliable electrical connections; 5. Substrate manufacturing: design and manufacture packaging substrates used to connect chips and external pins; 6. Wire bonding: Place the stacked chips on the substrate and connect the pads of the bottom chip with the corresponding pads on the substrate by wire bonding; 7. Final chip packaging: According to the selected vertical integration process, the final chip packaging is completed on the basis of the vertical integration solution; 8. Verification test: Verify and test the packaged chips.

[0031] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modification and change made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. An artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip, characterized in that: One or more layers of non-volatile memory chips are vertically integrated on the logic chip, and the vertical integration between the multiple layers of non-volatile memory chips is achieved through bumps, micro-bumps or hybrid bonding processes.

2. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 1, characterized in that: The multi-layer non-volatile memory chip is used to store the model data, and the physical distance between the location where the model data is stored and the location where the model data is used in the logic chip is as short as possible.

3. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 2, characterized in that: The model data in the multi-layer non-volatile memory chip is located directly above the computing module in the corresponding logic chip, making the transmission path as short as possible.

4. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 1, characterized in that: For vertical integration of a layer of non-volatile memory chips, the electrodes on the top of the non-volatile memory chips are directly used to realize vertical integration, and the logic chip is connected to the packaging substrate through wire bonding during packaging.

5. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 4, characterized in that: The non-overlapping portion of the logic chip and the non-volatile memory chip reserves a pad for wire bonding; the pad is located in the area surrounding the non-volatile memory chip on the surface of the logic chip.

6. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 1, characterized in that: For vertically integrated multi-layer non-volatile memory chips, it is necessary to implement through silicon vias (TSVs) in the multi-layer non-volatile memory chips to meet the needs of multi-layer stacking. During packaging, the logic chip is connected to the packaging substrate through the through silicon vias (TSVs) inside the logic chip, or is connected to the packaging substrate through wire bonding.

7. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 1, characterized in that: The error correction code (ECC) function is set on the non-volatile memory chip, and the data distributed at different locations on the chip are concentrated in the ECC for error correction calculation, and then dispersed to the bumps or micro-bumps used in different vertical integration processes and transmitted to the logic chip.

8. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 1, characterized in that: The error correction code ECC function is set on the logic chip, and the data is directly transmitted to the logic chip through the bumps, micro-bumps or hybrid bonding channels used in the vertical integration process.

9. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 1, characterized in that: The specific process of chip packaging is as follows: (1) Wafer cutting: NVM and logic chips are cut separately for packaging; (2) Chip stacking: stack the NVM chip and logic chip together to ensure that the corresponding contact points are electrically connected; (3) Substrate manufacturing: design and manufacture of packaging substrates used to connect chips and external pins; (4) Wire bonding: Place the stacked chips on the substrate and connect the pads of the bottom chip to the corresponding pads on the substrate by wire bonding; (5) Final chip packaging: Based on the selected vertical integration process, the final chip packaging is completed on the basis of the vertical integration solution.

10. The artificial intelligence acceleration system based on vertical integration of non-volatile memory and logic chip according to claim 1, characterized in that: For vertical integration processes using bumping and μ-bump, insulating materials need to be filled between the stacked chips to provide mechanical protection and ensure reliable electrical connections.

Images