Neural computing circuit utilizing three-dimensional memory to store activation function lookup table

Abstract

A neural computing circuit (180) includes a multiplier (152) in a semiconductor substrate (0) and a three-dimensional memory (3D-M) array (196) stacked on the multiplier (152). The 3D-M array (196) stores a look-up table for at least one activation function. The 3D-M array (196) at least partially overlaps the multiplier (152), and an output of the multiplier (152) is coupled to an input of the 3D-M array (196). The 3D-M array (196) is preferably a three-dimensional read-only memory (3D- ROM), and more particularly a three-dimensional programmable read-only memory (3D-P).

Description

[0001] This application is a divisional application of Chinese patent application filed on March 21, 2017, with application number 201710171413.X. Technical Field

[0002] This invention relates to the field of integrated circuits, and more specifically, to neural network processors used in artificial intelligence (AI). Background Technology

[0003] Artificial intelligence will lead the next wave of computing. Artificial neural networks (or simply neural networks) provide a powerful tool for artificial intelligence. Figure 1 This is an example of a neural network. It contains an input layer 32, a hidden layer 34, and an output layer 36. The input layer 32 contains i neurons 22, whose input data are x1, ..., x2. i The input vector is 20. The output layer 36 contains k neurons 26, whose output data are y1, y2, ... y1. k This forms the output vector 30. Hidden layer 34 lies between input layer 32 and output layer 36. It contains j neurons 24, each neuron 24 coupled to the first neuron in input layer 32 and also coupled to the second neuron in output layer 36. The coupling strength between neurons is determined by the synaptic weight w. ij and w jk express.

[0004] US Patent 6,199,057 (Inventor: Tawel; Grant Date: March 6, 2001) discloses a neural network processor. For example... Figure 2A As shown, it contains a synaptic weight (W) s RAM 40X, one input neuron (N) in RAM 40Y and a neural processing unit (NPU) 50. During operation, synaptic weights 42 are transferred from W... s Read from RAM 40X, input data 46 from N in Read from RAM40Y; NPU 50 multiplies them, and the resulting multipliers are accumulated and then passed through an activation function to obtain the output data N. out 48. Note that in Tawel, W s The RAM 40X has a very small capacity, only 2kx16.

[0005] Currently, the scale of neural networks is increasing daily. Most neural networks contain billions to tens of billions of W... s Parameters. Clearly, these GB-level W... s Parameters cannot be stored in W sIn RAM 40X. To solve this problem, neural network computers using the traditional von Neumann architecture will use W s The parameters are stored in external RAM (main memory). However, the computational demands of neural networks are too high, requiring frequent readings and writes from main memory. s These frequent main memory accesses become a performance bottleneck. Due to these frequent main memory accesses, the performance of neural network processors drops by at least an order of magnitude.

[0006] To address the issue of frequent main memory access, Chen Yunji et al. proposed a machine learning supercomputer containing multiple neural network accelerator chips (see Chen Yunji et al., *DaDianNao: A Machine-Learning Supercomputer*, IEEE / ACM International Symposium on Micro-architecture, 5(1), pp. 609-622, 2014). If each neural network accelerator chip contains enough RAM, then all the RAM in the supercomputer can hold the entire neural network, thus eliminating the need for main memory. Figure 2B This is a chip layout diagram of the neural network accelerator 60. Each chip 60 contains 16 cores 70. These cores 70 are coupled to each other through a tree-like connection. At the center of the chip 60 is an eDRAM block 66 designed for input and output neurons. Because the input neuron data is larger than W... s The number of parameters is several orders of magnitude less, and the input neuron data is not W. s Parameters are transferred between chips.

[0007] The core 70 in the neural network accelerator 60 has the same architecture. Figure 2C A kernel architecture was disclosed. Each core 70 contains one NPU 50 and four eDRAM blocks 40. The NPU 50 performs neural computations and contains 256+32 16-bit multipliers and 256+32 16-bit adders. The eDRAM 40 stores W... s The parameters are as follows: its storage capacity is 2MB. Because the eDRAM 40 and NPU50 are located close to each other, the data transmission distance is short, thus saving transmission time and energy. Due to its large internal bandwidth and less external communication, this neural network accelerator is 450.65 times faster than a GPU. Furthermore, the supercomputer (64 nodes) based on this neural network accelerator consumes 150.31 times less energy.

[0008] Despite its many advantages, Chen Yunji's neural network accelerator still has room for improvement. Firstly, from a system perspective, since eDRAM 40 is a volatile memory, it can only temporarily store W... sParameters. Therefore, this neural network accelerator still requires external storage to permanently store synaptic weights. Before running, W s The parameters need to be uploaded to eDRAM 40, which takes time. Secondly, each neural network accelerator chip 60 has 32MB of eDRAM available for storing W. s Parameters. Although this capacity is much larger than Tawel's, it is still far below practical needs. A typical neural network contains billions of W's. s Parameters. To store them in eDRAM 40, hundreds of neural network accelerator chips 60 would be needed. For example, storing 1 billion 32-bit W... s The parameters require 125 neural network accelerator chips 60. This is too many for mobile devices. Therefore, the neural network accelerator 60 is not suitable for motion applications. Furthermore, the architecture of the neural network accelerator 60 is heavily weighted towards storage—in each core, eDRAM 40 occupies 80% of the area, while the NPU 50 occupies less than 10%. Therefore, the computing power per unit area (referring to the area per chip) is significantly limited.

[0009] The root cause of the aforementioned problem is that the integration between eDRAM 40 and NPU 50 is two-dimensional, both formed within the substrate. Two-dimensional integration presents a dilemma: while increasing computing power within the chip can be achieved by sacrificing storage capacity, the resulting additional external memory accesses negate this increase in computing power. This dilemma will persist as long as two-dimensional integration is used. To overcome these shortcomings, we hope to find a completely different integration method. Summary of the Invention

[0010] The main objective of this invention is to promote the advancement of neural networks.

[0011] Another objective of this invention is to improve the computational power per unit area of ​​a neural network processor.

[0012] Another objective of this invention is to increase the storage capacity per unit area of ​​a neural network processor.

[0013] Another object of the present invention is to provide a neural network processor that can be used in mobile devices.

[0014] To achieve these and other objectives, this invention proposes an integrated neural network processor containing a three-dimensional memory (3D-M) array: it not only performs neural computation but also internally stores the synaptic weights required for neural computation. An integrated neural network processor chip contains multiple computational units, each containing a neural computation circuit and at least one 3D-M array. The neural computation circuit performs neural computation, and the 3D-M array stores the synaptic weights. The 3D-M array is vertically stacked above the neural computation circuit. This vertical stacking is referred to as three-dimensional integration. The 3D-M array is electrically coupled to the neural computation circuit through multiple contact vias. These contact vias are collectively referred to as computational connections.

[0015] 3D integration significantly improves computing power per unit area. Because the 3D-M array is vertically stacked above the neural computing circuitry, the area of ​​the computing unit is approximately equal to the area of ​​the neural computing circuitry. This is much smaller than existing technologies (such as neural network accelerators). Existing technologies use 2D integration; the area of ​​core 70 (equivalent to a computing unit) is approximately equal to the sum of the areas of eDRAM 40 (equivalent to a 3D-M array) and NPU 50 (equivalent to a neural computing circuitry). Since the NPU 50 occupies less than 10% of the core 70 area, while eDRAM 40 occupies 80% of the core area, the area of ​​the computing unit is reduced by approximately 10 times compared to the existing core 70 when the memory storing synaptic weights is moved from the side to the top. Therefore, the number of computing units in an integrated neural network processor is approximately 10 times greater than the number of cores in existing technologies. The integrated neural network processor has approximately 10 times the computing power per unit area of ​​existing technologies, supporting ultra-large-scale parallel computing.

[0016] 3D integration can also significantly increase storage capacity per unit area. This is because the area of ​​a 3D-M storage cell is approximately 4 square meters. 2 The area of ​​eDRAM memory cells is >100F. 2 (F represents the feature size of the technology node, such as 14nm). Furthermore, 3D-M contains multiple storage layers (e.g., four layers), while eDRAM contains only one. Therefore, the storage capacity per unit area of ​​an integrated neural network processor is approximately 100 times greater than existing technologies. For example, the 3D-XPoint chip has a storage capacity of 128Gb. Correspondingly, an integrated neural network processor based on 3D-XPoint can store up to 16GB of synaptic weights, which is sufficient for most artificial intelligence applications. Since only one or a few chips are needed to store the synaptic weights of the entire neural network, integrated neural network processors are suitable for mobile applications.

[0017] Accordingly, the present invention proposes an integrated neural network processor (200), characterized in that it comprises: a semiconductor substrate (0) containing a plurality of transistors; a plurality of computing units (100aa-100mn) formed on the semiconductor substrate (0), each computing unit (100ij) comprising at least a three-dimensional storage (3D-M) array (170) and a neural computing circuit (180), wherein: the 3D-M array (170) is stacked above the neural computing circuit (180), the 3D-M array (170) stores at least one synaptic weight; the neural computing circuit (180) is located in the substrate (0), the neural computing circuit (180) performs neural computation using the synaptic weight; the 3D-M array (170) and the neural computing circuit (180) are electrically coupled through a plurality of contact channel holes (1av, 3av).

[0018] The present invention also proposes an integrated neural network processor (200), characterized in that it comprises: a semiconductor substrate (0) containing a plurality of transistors; a plurality of computing units (100aa-100mn) formed on the semiconductor substrate (0), each computing unit (100ij) comprising at least a three-dimensional storage (3D-M) array (170) and a neural computing circuit (180), wherein: the 3D-M array (170) is stacked above the neural computing circuit (180), the 3D-M array (170) stores at least one synaptic weight; the neural computing circuit (180) is located in the substrate (0), the neural computing circuit (180) comprises a multiplier (152), the synaptic weight being an input (142) of the multiplier (152); the 3D-M array (170) and the neural computing circuit (180) are electrically coupled through a plurality of contact channel holes (1av, 3av).

[0019] The present invention further proposes a multi-type three-dimensional memory (3D-M) module (300), characterized in that it comprises: a semiconductor substrate (0) containing a plurality of transistors; a first 3D-M array (196) stacked on the semiconductor substrate, the first 3D-M array (196) being electrically coupled to the substrate (0); and a second 3D-M array (170) stacked on the first 3D-M array (196), the first 3D-M array (196) being electrically coupled to the substrate (0); wherein the first and second 3D-M arrays (196, 170) are 3D-M of different types. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a neural network.

[0021] Figure 2A This is a circuit block diagram of a neural network processor (existing technology); Figure 2BThis is a chip layout diagram of a neural network accelerator (existing technology); Figure 2C This is the core architecture of the neural network accelerator.

[0022] Figure 3 This is a schematic diagram of an integrated neural network processor.

[0023] Figures 4A-4C These are the circuit block diagrams for the three types of storage units; Figure 5A This is a cross-sectional view of a computing unit containing a three-dimensional writable memory (3D-W) array; Figure 5B This is a cross-sectional view of a computing unit containing a three-dimensional printed memory (3D-P) array.

[0024] Figure 6 It is a perspective view of a computing unit.

[0025] Figures 7A-7C This is a layout diagram of the substrate circuit of three types of computing units.

[0026] Figure 8 This is a circuit block diagram of a neural computing circuit.

[0027] Figures 9A-9B These are circuit block diagrams for two computing components.

[0028] Figure 10A This is a circuit block diagram that uses a 3D-ROM LUT activation function circuit; Figure 10B This is a simplified cross-sectional view of a storage unit containing the activation function circuit.

[0029] Figure 11A This is a simplified cross-sectional view of the first type of multi-class 3D-M module; Figure 11B This is a detailed cross-sectional view of this embodiment; Figure 11C This is a simplified cross-sectional view of the second type of multi-class 3D-M module.

[0030] Figure 12A This is a simplified cross-sectional view of the third type of multi-class 3D-M module; Figure 12B This shows the projection of all 3D-M arrays onto the substrate in this embodiment.

[0031] Please note that these figures are schematic diagrams only and are not drawn to scale. For clarity and convenience, some dimensions and structures in the figures may be enlarged or reduced. In different embodiments, letter suffixes following the numbers indicate different instances of the same type of structure; the same numerical prefix indicates the same or similar structures.

[0032] In this specification, "memory" refers to any semiconductor-based information storage device that can store information permanently or temporarily. "Storage" refers to any storage device that permanently stores information. "In the substrate" means that the core functional components of a circuit (such as transistors) are formed in the substrate (e.g., on the substrate surface); the interconnects of these transistors may be formed above the substrate and not in contact with it. "On the substrate" means that the core functional components of a circuit (such as storage cells) are formed above the substrate, not in the substrate, and not in contact with it. In other published texts, "Neural Processing Unit (NPU)" is also referred to as "Neural Functional Unit (NFU)," etc., all of which have the same meaning; "Neural Network Processor" is also referred to as "Neural Processor," "Neural Network Accelerator," "Machine Learning Accelerator," etc., all of which have the same meaning. The symbol " / " indicates an "AND" or "OR" relationship. Detailed Implementation

[0033] Figure 3 This describes an integrated neural network processor chip 200 that not only performs neural computations but also stores the synaptic weights required for these computations. The integrated neural network processor chip 200 is formed in a semiconductor substrate 0 and contains m x n computing units 100aa-100mn. Each computing unit is electrically coupled to an input 110 and an output 120. Note that an integrated neural network processor chip 200 can contain tens of thousands of computing units 100aa-100mn. For example, an integrated neural network processor chip 200 with 128Gb of storage capacity contains 64,000 computing units. This large number of computing units ensures ultra-large-scale parallel neural computation.

[0034] Figures 4A-4C This is a circuit block diagram of three types of storage computing units 100ij. Each storage computing unit 100ij contains a neural computing circuit 180 and at least one 3D-M array 170 (or 170A-170D, 170W-170Z), which are electrically coupled to each other via storage computing connections 160 (or 160A-160D, 160W-160Z). The neural computing circuit 180 performs neural computation, and the 3D-M array 170 stores the synaptic weights required for neural computation. In these embodiments, one neural computing circuit 180 is coupled to a varying number of 3D-M arrays 170. Figure 4A The neural computing circuitry 180 is coupled to a 3D-M array 170. Figure 4B The neural computing circuit 180 is coupled to four 3D-M arrays 170A-170D; Figure 4C The neural computing circuitry 180 is coupled to eight memory arrays 170A-170D and 170W-170Z. From... Figures 7A-7CIt can be seen that the neural computing circuit 180 coupled with a large number of 3D-M arrays has strong computing capabilities.

[0035] Figures 5A-6 Different types of 3D-M are displayed. These 3D-Ms have been disclosed in previous patents, such as US Patent 5,835,396 (Inventor: Guo-Biao Zhang; Grant Date: November 10, 1998). A 3D-M chip contains multiple vertically stacked memory layers, each containing multiple 3D-M arrays. A 3D-M array is a collection of all memory cells in a memory layer that share at least one address line. Furthermore, a 3D-M chip contains multiple 3D-M modules. The top memory layer of each 3D-M module contains only one top 3D-M array, and the boundary of this top 3D-M array is the boundary of the 3D-M module.

[0036] 3D-M is divided into 3D-RAM (3D Random Access Memory) and 3D-ROM (3D Read-Only Memory). In this specification, RAM refers to any semiconductor memory for temporary information storage, including but not limited to registers, SRAM, and DRAM; ROM refers to any semiconductor memory for permanent information storage, which can be electrically programmed or non-electrically programmed. Most 3D-Ms are 3D-ROMs. 3D-ROMs are further divided into 3D-W and 3D-P.

[0037] Information stored in 3D-W is entered through electrical programming. Based on the number of times it can be programmed, 3D-W is further divided into three-dimensional one-time-programmable memory (3D-OTP) and three-dimensional multiple-time-programmable memory (3D-MTP). As the name suggests, 3D-OTP can only be written once, while 3D-MTP can be written multiple times (including reprogramming). A common type of 3D-MTP is 3D-XPoint. Other 3D-MTPs include memristors, resistive random access memory (RRAM), phase-change memory (PCM), programmable metallization cell (PMC), and conductive bridging random-access memory (CBRAM).

[0038] The information stored in 3D-P (3D Printed Image) is entered during the factory production process using a printing method (printing method). This information is permanently fixed and cannot be changed after leaving the factory. Printing methods can include photolithography, nano-imprinting, e-beam lithography, DUV scanning exposure, and laser programming. A common type of 3D-P is the 3D Mask Programmable Read-Only Memory (3D-MPROM), which uses photolithography to program data through a mask. Because it does not require electrical programming, 3D-P memory cells can be biased at a higher voltage during reading. Therefore, 3D-P has a faster read speed than 3D-W (3D Printed Image).

[0039] Figure 5A This is a cross-sectional view of a 3D-W-based storage unit 100ij. The storage unit 100ij contains a substrate circuit layer 0K formed in a substrate 0. Storage layer 16A is stacked on the substrate circuit 0K, and storage layer 16B is stacked on top of storage layer 16A. The substrate circuit layer 0K contains peripheral circuitry for storage layers 16A and 16B, including transistors 0t and interconnects 0M. Each storage layer (e.g., 16A) contains multiple first address lines (e.g., 2a, along the y-direction), multiple second address lines (e.g., 1a, along the x-direction), and multiple 3D-W memory cells (e.g., 1aa). Storage layers 16A and 16B are coupled to the substrate 0 through contact vias 1av and 3av, respectively. Since contact vias 1av and 3av electrically couple the 3D-M array 170 and the neural computing circuit 180, they are collectively referred to as storage connection 160.

[0040] The 3D-W memory cell 5aa contains a programming film 12 and a diode film 14. The programming film 12 can be an antifuse film (write-once, suitable for 3D-OTP) or another multi-programmable film (for 3D-MTP). The diode film 14 has the following generalized characteristics: its resistance is low at the read voltage; its resistance is high when the applied voltage is less than or opposite to the read voltage. The diode film can be a PiN diode or a metal oxide (such as TiO2) diode, etc.

[0041] Figure 5B This is a cross-sectional view of a 3D-P based computing unit 100ij. Aside from the different storage elements, it is similar to... Figure 5A Similarly, the 3D-P contains at least two types of memory cells 6aa and 7aa—a high-resistance memory cell 6aa and a low-resistance memory cell 7aa. The low-resistance memory cell 7aa contains a diode film 14, while the high-resistance memory cell 6aa contains a high-resistance film 13. The high-resistance film 13 is an insulating film, such as a silicon dioxide film. During the process flow, the high-resistance film 13 located at the low-resistance memory cell 7aa is physically removed.

[0042] Figure 6 The structure of the computing unit 100ij is shown from another angle. A 3D-M array 170 is stacked above a neural computing circuit 180, which is located in the substrate 0 and at least partially covered by the 3D-M array 170. They are electrically coupled through numerous contact vias 1av and 3av. Three-dimensional integration brings the 3D-M array 170 and the neural computing circuit 180 closer together. Due to the large number (at least several thousand) and short length (micrometer scale) of contact vias 1av and 3av, the computing connection 160 has a very large bandwidth. This bandwidth is far greater than that of existing technologies. Because existing technologies use two-dimensional integration, the eDRAM 40 and NPU 50 are arranged side-by-side in the substrate 0, with a limited number (at most several hundred) and relatively long (hundreds of micrometer scale) connections between them.

[0043] Figures 7A-7C Three types of storage and computing units 100ij were disclosed. Figure 7A The embodiments correspond to Figure 4A The central computing unit 100ij. The neural computing circuit 180 is coupled to a 3D-M array 170, which completely covers it. In this embodiment, the 3D-M array 170 contains four peripheral circuits, including X decoders 15, 15' and Y decoders (including readout circuitry) 17, 17', with the neural computing circuit 180 located between these four peripheral circuits. Because the 3D-M array 170 is located above the substrate circuit 0K, but not within the substrate circuit 0K, Figure 7A (And subsequent figures) only use dashed lines to represent the projection of the 3D-M array 170 onto the substrate 0.

[0044] In this embodiment, the neural computing circuit 180 is confined among four peripheral circuits, and its area cannot exceed the area of ​​the 3D-M array 170. Due to its small area, the functionality of the neural computing circuit 180 is limited. This embodiment is a simple neural network processor. Obviously, complex neural network processors require a larger substrate area under the 3D-M array 170. Figures 7B-7C Two complex neural computing circuits were disclosed.180

[0045] Figure 7B The embodiments correspond to Figure 4B The central computing unit 100ij. In this embodiment, a neural computing circuit 180 is coupled to four 3D-M arrays 170A-170D. Each 3D-M array (e.g., 170A) has only two peripheral circuits (e.g., X decoder 15A and Y decoder 17A). Below these four 3D-M arrays 170A-170D, the substrate circuitry OK can be freely arranged to form a complex neural computing circuit 180. Clearly, Figure 7BThe medium-complex neural computing circuit 180 can be Figure 7A It is four times larger than a simple neural computing circuit, and it can perform more complex neural computing functions.

[0046] Figure 7C The embodiments correspond to Figure 4C A central computing unit 100ij is included. In this embodiment, a neural computing circuit 180 is coupled to eight 3D-M arrays 170A-170D and 170W-170Z. These eight 3D-M arrays are divided into two groups, 150A and 150B. Each group (e.g., 150A) includes four 3D-M arrays (e.g., 170A-170D). Below the four 3D-M arrays 170A-170D in the first group 150A, a substrate circuit can be freely arranged to form a first neural computing circuit assembly A 180A. Similarly, below the four 3D-M arrays 170W-170Z in the second group 150B, a substrate circuit can also be freely arranged to form a second neural computing circuit assembly B 180B. The first neural computing circuit assembly 180A and the second neural computing circuit assembly 180B constitute the neural computing circuit 180. In this embodiment, gaps (such as G) are left between adjacent peripheral circuits (e.g., between adjacent X decoders 15A and 15C; between adjacent Y decoders 17A and 17B; between adjacent Y decoders 17C and 17D) to form wiring channels 182, 184, and 186, enabling communication between different neural computing circuit components 150A and 150B, or between different neural computing circuits. Clearly, Figure 7C The neural computing circuit 180 in the middle can be Figure 7A It is eight times larger than that of a machine, and it can perform more complex neural processing functions.

[0047] Three-dimensional integration significantly improves computing power per unit area. Since the 3D-M array 170 is vertically stacked above the neural computing circuit 180, the area of ​​the computing unit 100ij is approximately equal to the area of ​​the neural computing circuit 180. This is much smaller than existing technologies (such as neural network accelerators). Because existing technologies use two-dimensional integration, the area of ​​the core 70 (equivalent to a computing unit) is approximately equal to the sum of the areas of the eDRAM 40 (equivalent to the 3D-M array) and the NPU 50 (equivalent to the neural computing circuit). Since the NPU 50 occupies less than 10% of the core 70 area, while the eDRAM 40 occupies 80% of the core area, when the memory storing synaptic weights is moved from the side to the top, the area of ​​the computing unit is reduced by approximately 10 times compared to the core 70 in existing technologies. Therefore, the number of computing units in an integrated neural network processor is approximately 10 times greater than the number of cores in existing technologies. The computing power per unit area of ​​the integrated neural network processor is approximately 10 times that of existing technologies, supporting ultra-large-scale parallel computing.

[0048] 3D integration can also significantly increase storage capacity per unit area. This is because the area of ​​a 3D-M storage cell is approximately 4 square meters. 2 The area of ​​eDRAM memory cells is >100F. 2 (F represents the feature size of the technology node, such as 14nm). Furthermore, 3D-M contains multiple storage layers (e.g., four layers), while eDRAM contains only one. Therefore, the storage capacity per unit area of ​​an integrated neural network processor is approximately 100 times greater than existing technologies. For example, the 3D-XPoint chip has a storage capacity of 128Gb. Correspondingly, an integrated neural network processor based on 3D-XPoint can store up to 16GB of synaptic weights, which is sufficient for most artificial intelligence applications. Since only one or a few chips are needed to store the synaptic weights of the entire neural network, integrated neural network processors are suitable for mobile applications.

[0049] Figures 8-9B Details of a neural computing circuit 180 and its computing component 150 are disclosed. Figure 8 In one embodiment, the neural computing circuit 180 includes a synaptic weight (W). s RAM 140A, one input neuron (N) in RAM 140B and a computing unit 150. W s RAM 140A is a cache that temporarily stores synaptic weights 142 from the 3D-M array 170; N in RAM140B is also a cache, which temporarily stores input data 146 from input 110. Computation unit 150 performs neural computation and produces output data 148.

[0050] exist Figure 9A In one embodiment, the computation unit 150 includes a multiplier 152, an adder 154, a register 156, and an activation function circuit 190. The multiplier 152 multiplies the synaptic weights w... ij With input data x i Multiplication, adder 154 and register 156 pair the product (w ij ×x i The summation is performed, and the accumulated value is sent to the activation function circuit 190. The result is the output data y. j .

[0051] exist Figure 9B In the embodiments, Figure 9A The multiplier 152 in the original code is replaced by a multiply-adder (MAC) 152'. Of course, the multiply-adder 152' also contains a multiplier. s The RAM 140A not only outputs synaptic weights w ij (Through port 142w), it also outputs bias b. j(Through port 142b). Multiply-accumulate unit 152' multiplies input data x i Synaptic weight w ij and bias b j Perform bias multiplication operation (w) ij ×x i +b j ).

[0052] An activation function is a function whose output is controlled within a certain range (e.g., 0 to 1, or -1 to +1), including sigmoid, signum, threshold, piecewise linear, step, and tanh functions. Implementing activation functions in circuits is challenging. Tawel proposed an activation function circuit based on a lookup table (LUT). It contains a ROM that stores the LUT values. Similar to other existing technologies, the ROM storing the LUT is formed in the substrate. That is, the ROM is two-dimensionally integrated with other components (such as RAM40X, 40Y, and NPU 50). This two-dimensional integration faces the same challenges as other existing technologies: the additional ROM (for storing the LUT) increases the area of ​​the NPU 50, which reduces the computing power per unit area and also reduces the storage capacity per unit area (used to store synaptic weights).

[0053] Continuing to develop the spirit of this invention, the 3D-M array can not only be used to store synaptic weights, but also to store LUTs for activation functions. Since it is desirable to store activation functions for a long time, the 3D-M array is preferably a 3D-ROM. Figures 10A-10B An activation function circuit 190 based on a 3D-ROM LUT is disclosed. Figure 10A As can be seen from the circuit block diagram, the activation function circuit 190 includes a preprocessing circuit 192, an X decoder 194, a 3D-ROM array 196, and a Y decoder 198. The 3D-ROM array 196 stores the LUT of the activation function. The decoder 192 first converts the accumulated value 158 into an address of the 3D-ROM array 196, and then the X decoder 194 selects the corresponding memory cells in the 3D-ROM array. The data in these memory cells is read out by the Y decoder 198.

[0054] Figure 10BThis is a simplified cross-sectional view of a computing unit 100ij containing an activation function circuit 190. The computing unit 100ij includes a neural computing circuit 180 formed in a substrate 0, a first 3D-ROM array 196 stacked above the neural computing circuit 180, and a second 3D-ROM array 170 stacked above the first 3D-ROM array 196. The first 3D-ROM array 196 stores the LUT of the activation function, and the second 3D-ROM array 170 stores synaptic weights. For clarity, this simplified cross-sectional view (and subsequent simplified cross-sectional views) only shows the approximate boundaries of these circuit components (including the neural computing circuit, the first and second 3D-ROM arrays) with dashed lines; the details of these circuit components are not shown.

[0055] After implementing the activation function using the 3D-ROM array 196, the computation unit 150 becomes extremely simple—it only needs to implement addition and multiplication, but not the activation function. Therefore, the computation unit 150 based on the 3D-ROM LUT has a much smaller area than computation units that implement activation functions using other methods. Correspondingly, the neural computation circuit 180 can employ… Figure 7A The simple neural computing circuitry within it. Clearly, the smaller computing unit 150 delivers greater computational power per unit area. This is another advantage of using a 3D-ROM LUT.

[0056] exist Figure 10B In the 3D-ROM arrays 170 and 196, two types of data are stored: synaptic weights and LUTs of activation functions. These two types of data place different requirements on the 3D-ROM arrays. LUTs generally require higher read speeds and are more suitable for storage in 3D-P arrays; while synaptic weights generally need to be rewritten and are more suitable for storage in 3D-W arrays. Accordingly, this invention also proposes a multi-type 3D-M module. Different types of data are stored in different storage layers of the multi-type 3D-M module. As mentioned above, the top storage layer of each 3D-M module contains only one top 3D-M array, and the boundary of this top 3D-M array is the boundary of the 3D-M module.

[0057] Figure 11A This is a simplified cross-sectional view of the first type of multi-type 3D-M module 300. Its first 3D-ROM array 196 is a 3D-P array, and the second 3D-ROM array 170 is a 3D-W array. The second 3D-ROM array 170 is stacked on top of the first 3D-ROM array 196. Figure 11B This is a detailed cross-sectional view of this embodiment. The 3D-P array 196 in the first storage layer 16A contains two types of memory cells: a high-resistivity 3D-P memory cell 6aa and a low-resistivity 3D-P memory cell 7aa (see...). Figure 5BThe 3D-W array 170 in the second storage layer 16B contains only one type of storage cell: 3D-W storage cell 5aa (see...). Figure 5A ). Figure 11C This is a simplified cross-sectional view of the second type of multi-type 3D-M module 300. It contains three storage layers 16A-16C. The 3D-ROM array 196 in storage layer 16A is a 3D-P array, and the 3D-ROM arrays 170a and 170b in storage layers 16B and 16C are 3D-W arrays. Storage layers 16A, 16B, and 16C are stacked sequentially from bottom to top. For those familiar with this field, more storage layers or other combinations of 3D-P and 3D-W are also feasible.

[0058] Figures 12A-12B A third type of multi-class 3D-M module 300 is disclosed. This 3D-M module 300 contains two storage layers 16A and 16B. Storage layer 16B is the top storage layer, containing a 3D-W array 170. Storage layer 16A is an intermediate storage layer, containing at least two 3D-P arrays 196A and 196B arranged side-by-side. Figure 12A ).from Figure 12B As can be seen from the projection diagram, the 3D-W array 170 completely covers the 3D-P arrays 196A-196F. This embodiment uses the smaller 3D-P arrays 196A-196F because smaller arrays have faster speeds.

[0059] It should be understood that modifications to the form and details of the invention may be made without departing from the spirit and scope thereof, and such modifications do not preclude the application of the spirit of the invention. Therefore, the invention should not be limited in any way except in accordance with the spirit of the appended claims.

Claims

1. A neural network processor chip (200), characterized in that... It contains multiple computing units (100aa-100mn), each computing unit (100ij) contains a multiplier (152), a first three-dimensional storage 3D-M array (196), and a second three-dimensional storage 3D-M array (170), wherein: The multiplier (152) is located in a semiconductor substrate (0); The first three-dimensional storage 3D-M array (196) is located above the multiplier (152) and stores a lookup table of activation functions; The second three-dimensional storage 3D-M array (170) is located above the first three-dimensional storage 3D-M array (196) and stores at least one synaptic weight; The first three-dimensional storage 3D-M array (196), the second three-dimensional storage 3D-M array (170), and the multiplier (152) at least partially overlap; The storage unit (100ij) performs neural computation using the multiplier (152), the synaptic weights, and the lookup table of the activation function.

2. The neural network processor chip (200) according to claim 1, further characterized in that: The first three-dimensional storage 3D-M array (196) is a three-dimensional read-only memory 3D-ROM or a three-dimensional print memory 3D-P.

3. A neural network processor chip (200), characterized in that... It contains multiple computing units (100aa-100mn), each computing unit (100ij) contains a multiplier (152), a first three-dimensional storage 3D-M array (196A), and a second three-dimensional storage 3D-M array (170), wherein: The multiplier (152) is located in a semiconductor substrate (0); The first three-dimensional storage 3D-M array (196A) stores a lookup table for an activation function; The second three-dimensional storage 3D-M array (170) stores at least one synaptic weight; The first three-dimensional storage 3D-M array (196A) and the second three-dimensional storage 3D-M array (170) are located above the multiplier (152), and the first three-dimensional storage 3D-M array (196A) is smaller than the second three-dimensional storage 3D-M array (170); The first three-dimensional storage 3D-M array (196A), the second three-dimensional storage 3D-M array (170), and the multiplier (152) at least partially overlap; The storage unit (100ij) performs neural computation using the multiplier (152), the synaptic weights, and the lookup table of the activation function.

4. The neural network processor chip (200) according to claim 3, further characterized in that: The first three-dimensional storage 3D-M array (196A) is a three-dimensional read-only memory 3D-ROM or a three-dimensional print memory 3D-P.

5. A neural network processor chip (200), characterized in that... It contains multiple computing units (100aa-100mn), each computing unit (100ij) contains a multiplier (152), a 3D-P array for three-dimensional imprinted storage (196), and a 3D-W array for three-dimensional writable storage (170), wherein: The multiplier (152) is located in a semiconductor substrate (0); The three-dimensional imprint storage 3D-P array (196) stores a lookup table of an activation function; The three-dimensional writable storage 3D-W array (170) stores at least one synaptic weight; The three-dimensional imprint storage 3D-P array (196) and the three-dimensional writable storage 3D-W array (170) are located on top of the multiplier (152); The three-dimensional imprint storage 3D-P array (196), the three-dimensional writable storage 3D-W array (170), and the multiplier (152) at least partially overlap; The storage unit (100ij) performs neural computation using the multiplier (152), the synaptic weights, and the lookup table of the activation function.

6. The neural network processor chip (200) according to any one of claims 1-5, further characterized in that: The semiconductor substrate (0) contains an adder (154), a multiply-adder (152'), or a register (156).

Images