In-memory computing chip, operation method, manufacturing method and electronic device

Abstract

A memory-compute integrated chip, an operation method, a manufacturing method and an electronic device. The memory-compute integrated chip includes a substrate, a memory-compute device array, a data memory array and a control processing circuit, the memory-compute device array is configured to perform memory-compute integrated operation; the data memory array is configured to store data; the control processing circuit is configured to communicate with the data memory array and the memory-compute device array to read input data from the data memory array and provide the input data to the memory-compute device array, and receive processed output data from the memory-compute device array and provide the processed output data to the data memory array; the control processing circuit, the memory-compute device array and the data memory array are arranged on the substrate, and in a direction perpendicular to the substrate, are respectively provided in different structure layers and at least partially overlap. The memory-compute integrated chip has improved performance and energy efficiency.

Description

Technical Field

[0001] Embodiments of this disclosure relate to an in-memory computing chip, an operating method, a manufacturing method, and an electronic device. Background Technology

[0002] The rapid development of artificial intelligence in recent years is largely attributed to the advancements in deep learning technology. Deep learning has enabled significant progress in many AI (Artificial Intelligence) fields, and these substantial advancements and transformations have prompted the industry to rethink AI chip architecture and computing models. Currently, CPUs (Central Processing Units), GPUs (Graphics Processing Units), and ASICs (Application Specific Integrated Circuits) have all undergone architectural optimization for AI applications, achieving superior real-time system response speeds and power consumption. However, these chip architectures inevitably involve data exchange between memory and computing units. Due to the "memory wall" of the von Neumann architecture, problems exist such as large data transfer volumes, slow data transfer speeds in memory, and high energy consumption during data transfer. Therefore, CMOS (Complementary Metal Oxide Semiconductor) based processors face significant bottlenecks in performance and energy efficiency improvements. Summary of the Invention

[0003] At least one embodiment of this disclosure provides an in-memory computing chip, which includes a substrate, an in-memory computing device array, a data memory array, and a control processing circuit. The in-memory computing device array is configured to perform in-memory computing operations; the data memory array is configured to store data; the control processing circuit is configured to communicate with the data memory array and the in-memory computing device array to read input data from the data memory array and provide the input data to the in-memory computing device array, and to receive processed output data from the in-memory computing device array and provide the processed output data to the data memory array; wherein the control processing circuit, the in-memory computing device array, and the data memory array are disposed on the substrate and are respectively provided in different structural layers and at least partially overlap in a direction perpendicular to the substrate.

[0004] For example, in a memory computing chip provided in one embodiment of this disclosure, the control processing circuit is formed on the substrate; the data memory array is formed on the side of the control processing circuit away from the substrate; and the memory computing device array is formed on the side of the data memory array away from the control processing circuit.

[0005] For example, in an embodiment of the in-memory computing chip provided in this disclosure, the in-memory computing device array is divided into multiple blocks, and the data memory array is divided into multiple memory groups. The multiple blocks correspond to the multiple memory groups. The multiple blocks include a first block and a second block. The multiple memory groups include a first memory group corresponding to the first block and a second memory group corresponding to the second block. The first memory group is configured to store the output data processed by the first block and provide the output data processed by the first block to the second block as input data for the second block. The second memory group is configured to store the output data processed by the second block.

[0006] For example, in a memory computing chip provided in one embodiment of this disclosure, the control processing circuit includes a digital-to-analog converter and an analog-to-data converter. The digital-to-analog converter is configured to convert input data read from the data memory array into an analog signal to provide the analog signal to the memory computing device array. The analog-to-data converter is configured to convert processed analog output data received from the memory computing device array into digital output data to provide the digital output data to the data memory array.

[0007] For example, in a memory computing chip provided in one embodiment of this disclosure, the control processing circuit further includes a row decoder and a column decoder. The row decoder is configured to decode the row code of the data to be read based on the address data and send the row code to the data memory array to read the first read data corresponding to the row code from the data memory array. The column decoder is configured to decode the column code of the data to be read to select the second read data corresponding to the column code from the first read data.

[0008] For example, in a memory-computing chip provided in one embodiment of this disclosure, the control processing circuit further includes an inductive amplifier, a read queue memory, a data selector, a logic control unit, and a write driver. The inductive amplifier is configured to read the first read data from the data memory array; the read queue memory is configured to buffer the second read data; the data selector is configured to select the required data and send it to the digital-to-analog converter; the logic control unit is configured to generate a logic address and send it to the column decoder, so that the column decoder obtains the column code of the data to be written based on the logic address; and the write driver is configured to write the data to be written to the data memory array based on the column code of the data to be written.

[0009] For example, in a memory computing chip provided in one embodiment of this disclosure, the control processing circuit includes a shift accumulator, an activation unit, and a pooling unit. The shift accumulator is configured to perform an accumulation operation on the data output by the analog-to-digital converter; the activation unit is configured to calculate the activation function in the neural network algorithm; and the pooling unit is configured to calculate the pooling function in the neural network algorithm.

[0010] For example, in a memory computing chip provided in one embodiment of this disclosure, the data memory array is a dynamic random access memory array, which includes multiple dynamic random access memory cells arranged in an array; the memory computing device array is a memristor array, which includes multiple memristor cells arranged in an array.

[0011] At least one embodiment of this disclosure provides an operation method for an in-memory computing chip according to any embodiment of this disclosure. The operation method includes: reading input data from the data memory array and providing the input data to the in-memory computing device array; processing the input data in the in-memory computing device array to obtain processed output data; and storing the processed output data in the data memory array.

[0012] For example, in one embodiment of the present disclosure, the in-memory computing device array processes the input data, including: the in-memory computing device array performing vector-matrix multiplication on the input data to obtain processed output data; or the in-memory computing device array using stored weight data to perform vector-matrix multiplication on the input data to obtain neural network feature map data.

[0013] For example, in an embodiment of the present disclosure, the in-memory computing device array includes a first block and a second block, and the data storage array includes a first storage group corresponding to the first block and a second storage group corresponding to the second block. The in-memory computing device array processes the input data to obtain processed output data, including: the first block processes the input data to obtain processed output data; storing the processed output data in the data storage array includes: storing the processed output data in the first storage group; reading input data from the data storage array and providing the input data to the in-memory computing device array includes: reading the processed output data from the first storage group and providing the processed output data to the second block as input data for the second block.

[0014] At least one embodiment of this disclosure provides a method for fabricating an in-memory computing chip according to any embodiment of this disclosure. The method includes: providing the substrate; and layering the control processing circuit, the data memory array, and the in-memory computing device array on the substrate.

[0015] At least one embodiment of this disclosure provides an electronic device that includes a memory computing chip as described in any embodiment of this disclosure. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0017] Figure 1 A schematic diagram of a memristor-based in-memory computing chip is shown.

[0018] Figure 2 A schematic diagram of a memory computing chip provided in at least one embodiment of the present disclosure is shown;

[0019] Figure 3 Another schematic diagram of a memory computing chip provided in at least one embodiment of the present disclosure is shown;

[0020] Figure 4 A further schematic diagram of a memory computing chip provided in at least one embodiment of the present disclosure is shown;

[0021] Figure 5 This diagram illustrates the storage relationship between blocks and storage groups provided in at least one embodiment of the present disclosure.

[0022] Figure 6This illustration shows a data pipeline delay diagram provided by at least one embodiment of the present disclosure;

[0023] Figure 7 A flowchart of an operation method provided by at least one embodiment of the present disclosure is shown;

[0024] Figure 8 A flowchart of a manufacturing method provided by at least one embodiment of the present disclosure is shown. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0026] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0027] To overcome the storage wall problem in traditional architectures, one approach is to integrate storage and computing into a single module, which reduces the data transfer requirements during the computing process.

[0028] Memristors (such as resistive random access memory (RRAM), phase-change memory, and bridge memory) are non-volatile devices whose conductance can be adjusted by applying external stimuli. As a two-terminal device, memristors possess adjustable and non-volatile resistance, making them widely used in in-memory computing applications, such as for forward computation (inference) in artificial neural networks. According to Kirchhoff's current law and Ohm's law, arrays of memristors can be used to perform matrix-vector multiplication (multiply-accumulate) calculations in parallel, with both storage and computation occurring within the array's individual devices. Based on this computing architecture, in-memory computing can be achieved without requiring extensive data movement.

[0029] Figure 1 A schematic diagram of a memristor-based in-memory computing chip is shown. Figure 1 As shown, this in-memory computing chip based on a memristor (e.g., RRAM, Resistive Random Access Memory) array connects multiple tiles 110 via an on-chip network. Each tile 110 consists of a memristor array 111, a special function unit 112, a shift accumulator 113, a local cache 114, a digital-to-analog converter 115, an analog-to-digital converter 116, and a controller 117. The special function unit, for example, is used to implement specific computational functions. For instance, data from the previous level is written to the local cache 114 of the current tile via a bus. After sufficient data for computation is available, the controller 117 reads this data from the local cache 114, converts it into analog voltages via the digital-to-analog converter 115, and sends these analog voltage signals to the memristor array 111 for computation after the controller 117 issues a computation start signal. In the field of artificial intelligence, the memristor array can, for example, store the weight data of a neural network and perform computations on the input feature map data based on the weight data, outputting the processed feature map data.

[0030] The following description uses RRAM as an example of a memristor, but the embodiments disclosed herein are not limited to this.

[0031] Figure 1 In the in-memory computing chip shown, although the weight data is stored in the RRAM array, the input and output feature map data still need to be transferred between the RRAM array and off-chip memory. Since the amount of data in the input and output feature maps is enormous, the "memory wall" of the von Neumann architecture still exists for this in-memory computing chip.

[0032] At least one embodiment of this disclosure provides an in-memory computing chip, an operating method, a fabrication method, and an electronic device. The in-memory computing chip includes a substrate, an in-memory computing device array, a data memory array, and a control processing circuit. The in-memory computing device array is configured to perform in-memory computing operations; the data memory array is configured to store data; the control processing circuit is configured to communicate with the data memory array and the in-memory computing device array to read input data from the data memory array and provide the input data to the in-memory computing device array, and to receive processed output data from the in-memory computing device array and provide the processed output data to the data memory array. The control processing circuit, the in-memory computing device array, and the data memory array are disposed on the substrate and are respectively provided in different structural layers in a direction perpendicular to the substrate, and at least partially overlap.

[0033] This in-memory computing chip integrates an in-memory computing device array, a data memory array, and control processing circuitry onto a single chip, reducing or eliminating data transfer between the in-memory computing device array and off-chip memory. This further weakens or eliminates the memory wall, improving the performance and energy efficiency of the in-memory computing chip. Furthermore, the in-memory computing device array, data memory array, and control processing circuitry are located in three stacked structural layers. The monolithic three-dimensional integration of these three structural layers gives the chip advantages in both computational and storage density, effectively improving computing power.

[0034] Figure 2 A schematic diagram of a memory computing chip provided in at least one embodiment of the present disclosure is shown.

[0035] like Figure 2 As shown, the in-memory computing chip 200 includes a substrate 210, an in-memory computing device array 220, a data memory array 230, and a control processing circuit 240. The in-memory computing device array 220 is configured to perform in-memory computing operations, the data memory array 230 is configured to store data, and the control processing circuit 240 is configured to communicate with the data memory array 230 and the in-memory computing device array 220 to read input data from the data memory array 230 and provide the input data to the in-memory computing device array 220, and to receive processed output data from the in-memory computing device array 220 and provide the processed output data to the data memory array 230. The control processing circuit 240, the in-memory computing device array 220, and the data memory array 230 are disposed on the substrate 210, and in a direction perpendicular to the substrate 210, the control processing circuit 240, the in-memory computing device array 220, and the data memory array 230 are respectively provided in different structural layers and at least partially overlap.

[0036] For example, the substrate 210 may be made of, for example, silicon substrate, glass, plastic, quartz or other suitable materials, and the embodiments of this disclosure are not limited thereto.

[0037] For example, an in-memory computing device array 220, a data memory array 230, and a control processing circuit 240 are layered and formed on a substrate 210. The in-memory computing device array 220, the data memory array 230, and the control processing circuit 240 are located on different structural layers in a direction perpendicular to the substrate 210. The substrate 210, the in-memory computing device array 220, the data memory array 230, and the control processing circuit 240 are stacked vertically to form a three-dimensional chip. Each structural layer may include multiple structural layers. For example, the data memory array 230 includes transistors; therefore, the structural layer of the data memory array 230 includes at least an active semiconductor layer for forming transistors, multiple insulating layers (e.g., interlayer dielectric layers, planarization layers, etc.), and metal layers. For example, in a direction perpendicular to the substrate, the three structural layers at least partially overlap, and the different structural layers can be connected through multiple vias and traces.

[0038] For example, the data storage array is a dynamic random access memory (DRAM) array, which includes multiple dynamic random access memory cells (DRAM cells) arranged in an array.

[0039] For example, an in-memory computing device array can be a memristor array, which includes multiple memristors arranged in an array. Taking RRAM as an example, an in-memory computing device array can be a resistive random access memory (RRAM) array, which can include multiple resistive random access memory cells (RRAM cells) arranged in an array.

[0040] For example, in some embodiments, a control processing circuit 240 is formed on a substrate 210, a data memory array 230 is formed on the side of the control processing circuit 240 away from the substrate 210, and a memory computing device array 220 is formed on the side of the data memory array 230 away from the control processing circuit 240.

[0041] Figure 3 Another schematic diagram of a memory computing chip provided in at least one embodiment of the present disclosure is shown. Figure 4 A further schematic diagram of a memory computing chip provided in at least one embodiment of the present disclosure is shown.

[0042] like Figure 3 and Figure 4As shown, taking a DRAM+RRAM in-memory computing chip as an example, the chip can adopt monolithic 3D integration (M3D). The structure of the in-memory computing chip is as follows, forming a first construction layer (hereinafter referred to as the first layer), a second construction layer (hereinafter referred to as the second layer), and a third construction layer (hereinafter referred to as the third layer).

[0043] For example, the first layer can employ front-end technology to form a control processing circuit, which includes CMOS digital logic circuits and analog circuits. These CMOS digital logic circuits and analog circuits may include, for example, integrated sense amplifiers (SA), digital-to-analog converters, analog-to-digital converters, shift accumulators, controllers, etc.

[0044] As shown in the figure, the first layer also includes multiple structural layers for forming devices such as transistors. After the control processing circuit is formed, an insulating layer is formed to cover the surface, and vias for interlayer electrical connections are formed in the insulating layer. Then, for example, a planarized surface is obtained by chemical mechanical polishing, and the subsequent second layer process is performed on the planarized surface.

[0045] For example, the second layer can employ back-end processes to form a data memory array, such as a 2T0C 3D DRAM array based on Indium Gallium Zinc Oxide (IGZO) Channel-All-Around (CAA) transistors. Here, IGZO CAA FET represents a field-effect transistor based on an Indium Gallium Zinc Oxide (IGZO) Channel-All-Around (CAA) structure, which has high drive current (Ion) and extremely low off-state current (Ioff), exhibiting good thermal reliability and stability. 2T0C indicates that each DRAM cell includes two transistors and has no storage capacitor; in this embodiment, the DRAM cell can also be referred to as a 2T0C cell or a 2T0C memory cell. 3D indicates a three-dimensional structure formed by stacking two transistors.

[0046] like Figure 4 As shown, each 2T0C cell includes a CAA write transistor (Tw) and a CAA read transistor (Tr). The CAA write transistor (Tw) is stacked on top of the CAA read transistor (Tr). The source / drain (S / D) terminal of the top Tw is connected to the gate of the bottom Tr, and the charge is stored in the parasitic capacitance of the gate of Tr. Figure 4The diagram shows WWL for write word line, WBL for write bit line, RBL for read bit line, RWL for read word line, SL for source line, WL for word line, and BL for bit line. By directly connecting the two CAA devices, the size of each 2T0C memory cell (DRAM cell) can be reduced to 4F. 2 The CAA-based 2T0C 3D DRAM structure effectively reduces device area and supports multi-layer stacking, offering density advantages. Furthermore, the CAA-based 2T0C 3D DRAM boasts high storage density and a large data capacity, effectively reducing input / output data movement and thus minimizing power consumption for off-chip data transfer.

[0047] As shown in the figure, the second layer also includes multiple structural layers for forming devices such as transistors. After the data memory array is formed, an insulating layer is formed to cover the surface, and vias for interlayer electrical connections are formed in the insulating layer. Then, for example, a planarized surface is obtained by chemical mechanical polishing, and the subsequent third layer process is performed on the planarized surface.

[0048] For example, the third layer can also be fabricated using back-end processes to form a 1T1R array based on IGZO thin-film transistors (TFTs). Here, 1T1R indicates that each RRAM cell includes one transistor and one resistive random access memory (RRAM). In this embodiment, the RRAM cell can also be referred to as a 1T1R cell. The thin-film transistors in the third layer are, for example, planar, and the RRAM is located at the drain of the thin-film transistors. The low leakage current of the thin-film transistors reduces the static power consumption of the 1T1R array. IGZO back-end processes are compatible and can provide a large drive current for RRAM operation.

[0049] For example, RRAM arrays can be used to store data and perform computational operations, such as vector matrix multiplication (VMM) operations. When used in neural network operations, RRAM arrays can store the weight data of the neural network and perform vector matrix multiplication operations based on this weight data. DRAM arrays can also be used to store data; in neural network operations, DRAM arrays are used, for example, to store feature map data. For example, in a convolutional neural network comprising multiple convolutional layers, each convolutional layer uses a convolution kernel to perform a convolution operation on the feature map data output from the previous layer and outputs the resulting feature map data to the next layer. In this convolutional neural network scenario, the weight data stored in the RRAM array is, for example, convolution kernel data, and the vector matrix multiplication operation performed by the RRAM array is used, for example, to implement the convolution operation.

[0050] As shown in the figure, this third layer also includes multiple structural layers for forming devices such as transistors and RRAM. After the in-memory computing device array is formed, an insulating layer is formed to cover the surface, and then a planarized surface is obtained, for example, through chemical mechanical polishing.

[0051] For example, the control processing circuit of the first layer can read the first feature map data from the DRAM array of the second layer and transfer the first feature map data to the RRAM array of the third layer. The RRAM array uses the stored weight data to perform operations on the first feature map data to obtain the second feature map data, and the RRAM array sends the second feature map data to the control processing circuit of the first layer. The control processing circuit stores the second feature map data in the DRAM array of the second layer, thereby realizing the operation of one network layer of the neural network. Then, the control processing circuit reads the second feature map data from the DRAM array to complete the operation of the next network layer based on the second feature map data, and so on.

[0052] According to embodiments of the in-memory computing chip disclosed herein, an in-memory computing device array, a data memory array, and a control processing circuit are integrated on a single chip, reducing or eliminating data transfer between the in-memory computing device array and off-chip memory, further weakening or eliminating memory walls, and improving the performance and energy efficiency of the in-memory computing chip. Furthermore, the in-memory computing device array, data memory array, and control processing circuit are located in three stacked structural layers, and the three structural layers are integrated in three dimensions on a single chip, giving the chip advantages in both computing density and storage density, effectively improving computing power. In addition, at least one embodiment of the CAA-based 2T0C 3D DRAM has high storage density, effectively reducing the movement of input and output data, thereby reducing power consumption for off-chip data transfer.

[0053] For example, the control processing circuit may include devices such as digital-to-analog converters, analog-to-data converters, row decoders, column decoders, inductive amplifiers, read queue memory, data selectors, logic control units, and write drivers.

[0054] For example, the row decoder is configured to decode the row code of the data to be read based on the address data, and send the row code to the data memory array to read the first read data corresponding to the row code from the data memory array. Figure 3 As shown, the row decoder is, for example, the address latch and decoder 301 illustrated. The address latch and decoder 301 receives data from the address generator, decodes it through the decoding circuit to obtain the row code that the DRAM array needs to read, and sends the row code data to the word line of the DRAM array to control the reading of the corresponding row, so that the DRAM array outputs feature map data according to the information sent by the address latch and decoder circuit.

[0055] For example, a sense amplifier (SA) is configured to read the first readout data from the data memory array. Figure 3 As shown, the inductive amplifier 302 reads the feature map data of the corresponding row output by the DRAM array as the first readout data.

[0056] For example, the column decoder is configured to decode the column code of the data to be read, in order to select second read data corresponding to the column code from the first read data. Figure 3 As shown, the column decoder 303 is used to decode the column address and select the data corresponding to the column to be read / written from the data read from the inductive amplifier as the second read data.

[0057] For example, the read queue memory is configured to buffer the second read data, and the data selector is configured to select the desired data and send it to the digital-to-analog converter. Figure 3 As shown, the read FIFO and data MUX 304 include a read FIFO (First In First Out) queue and a data MUX (multiplexer). The read FIFO queue is a read queue memory, and the data MUX is a data selector. The read FIFO is used to buffer data to balance the speed difference between data reading and data use. The data selector is used to send the required data to the DAC (digital to analog converter) 306.

[0058] For example, a digital-to-analog converter is configured to convert input data read from a data memory array into an analog signal, so as to provide the analog signal to the in-memory computing device array. Figure 3As shown, the read driver 305 drives the DAC 306, causing the DAC (digital-to-analog converter) 306 to convert the feature map data into an analog signal and transmit the analog signal to the RRAM array through the BL (bit line) buffer. The BL buffer is used to drive the BL of the RRAM array. For example, the RRAM array performs arithmetic processing on the received data to obtain processed analog output data, and transmits the processed analog output data to the analog data converter in the control processing circuit.

[0059] For example, an analog-to-digital converter is configured to convert processed analog output data received from a memory-in-memory device array into digital output data, which is then provided to the data memory array. A logic control unit is configured to generate a logic address and send it to the column decoder, so that the column decoder obtains the column code of the data to be written based on the logic address. A write driver is configured to write the data to be written to the data memory array based on the column code. Figure 3 As shown, the write driver and I / O logic 308 includes a write driver and I / O logic. The I / O logic is a logic control unit. The ADC (analog to digital converter) 307 converts the analog signal of the calculation result into a digital signal. The logic address is generated by the I / O logic and given to the column decoder 303. The column decoder 303 decodes the column address and selects the column to be written. Then, the write driver writes the data to the DRAM array.

[0060] For example, as described above, the first layer integrates the address latch and decoding circuitry, sense amplifier (SA), and column decoder of the 2T0C 3D DRAM. The second layer integrates the data read from the 3D DRAM, which passes through a read FIFO and a data MUX to a digital-to-analog converter (DAC). The analog voltage is then stabilized by an analog bitline (BL) buffer and output to the 1T1R array BL in the third layer, where the BL buffer is a unity-gain amplifier (UGB) structure. Vector matrix multiplication (VMM) is implemented in the RRAM array, and current is output. This current is converted to an 8-bit digital output by an 8-bit ADC and written into the 3D DRAM through the 2T0C 3D DRAM write circuit.

[0061] For example, in some examples, the control processing circuit may further include a shift accumulator, an activation unit, and a pooling unit. The shift accumulator is configured to perform an accumulation operation on the data output from the analog-to-digital converter. For instance, the shift accumulator may include a summing circuit composed of a shift register and a full adder. The function of the shift accumulator includes accumulating partial results output by the ADC from each RRAM array. The activation unit is configured to calculate the activation function in the neural network algorithm, and the pooling unit is configured to calculate the pooling function in the neural network algorithm.

[0062] For example, an in-memory computing device array can be divided into multiple blocks, and the data storage array can be divided into multiple storage groups, with each block corresponding to a storage group. The multiple blocks include a first block and a second block, and the multiple storage groups include a first storage group corresponding to the first block and a second storage group corresponding to the second block. The first storage group is configured to store the output data processed by the first block and provide the processed output data to the second block as input data. The second storage group is configured to store the processed output data of the second block.

[0063] Figure 5 A schematic diagram illustrating the storage relationship between blocks and storage groups provided in at least one embodiment of this disclosure is shown. For example... Figure 5 As shown, a block of the RRAM array is, for example, a tile group. Each tile group includes multiple tiles. For example, the (i-1)th tile group (Tile group i-1) includes Tile 0 to Tile T (T is a positive integer), and the ith tile group (Tile group i) includes Tile T+1 to Tile 2T. Each tile includes multiple M×N arrays, and each M×N array includes M rows and N columns of 1T1R cells, where M and N are both positive integers.

[0064] For example, a memory bank in DRAM is a Rank, and each Rank includes multiple Banks. For instance, the z-th Rank (where z is a positive integer) includes Banks 0 to K (where K is a positive integer), and the (z+1)-th Rank includes Banks K+1 to 2K. Each Bank includes, for example, 8 basic array cells, each consisting of R rows, C columns, and 20C cells, where R and C are both positive integers. A single basic array cell can provide one bit at a time. If 8 bits need to be read or written at once, the 8 basic array cells can be combined, and one bit can be read from each of the 8 cells to obtain 8 bits of data. Multiple basic array cells within the same Bank can share a set of row and column addresses, thus requiring only one set of addresses to read the data stored at the corresponding addresses in the multiple basic array cells within the Bank. A Rank can comprise multiple Banks, with each basic array cell in each Bank being approximately 1k × 1k in size. The Rank uses the main word line (WL) to control rows, which is then split into local word lines controlling the Banks. During readout, data is first read from the sub-bit lines (BL) of the Banks and then from the global bit lines of the Rank. Because the load on the local word lines is reduced to the size of a Bank, the local word lines offer better drive capability for the array and reduce wire load. For example, the physical address encoding of a 2T0C 3D DRAM is (Rank z, Bank k, Row r, Column c), where z, k, r, and c are all positive integers. The IO bandwidth is 8-bit, meaning a column consists of 8 2T0C cells, and a Rank consists of K Banks.

[0065] For example, the first block is Tile group i-1, the second block is Tile group i, the first storage group is Rank z, and the second storage group is Rank z+1. Taking a neural network scenario as an example, Tile group i-1 calculates the first feature map data based on the input data, and the first feature map data output by Tile group i-1 is stored in Rank z. Rank z transmits the first feature map data to Tile group i as input data for Tile group i, so that Tile group can perform operations on the first feature map data to obtain the second feature map data. The second feature map data output by Tile group i is stored in Rank z+1, and so on. Rank z stores the output data of Tile group i-1, and at the same time, it also stores the input data of Tile group i. This allows direct utilization of data locality and the reusability of neural network feature maps, and reduces the length of data transmission lines.

[0066] Figure 6A schematic diagram illustrating data pipeline delays provided in at least one embodiment of this disclosure is shown. Figure 6 Part (1) shows the delay without pipeline, and part (2) shows the delay with pipeline, R1, R2, R z and R z+1 These represent the processing flow around a Rank. From the perspective of Rank storage in 3D DRAM, the data flow around Rank z includes the output data of Tile group i-1, which is also the input data of Tile group i. Therefore, the hardware resources for calculating and storing a Rank include DRAM arrays of Banks 0 to K and their decoders and inductive amplifiers, RRAM arrays of Tile 0 to T, read / write drivers and interfaces, DAC / ADC, etc. Correspondingly, a Rank operation includes reading DRAM, DA (analog-to-digital) conversion, matrix-vector multiplication performed by the RRAM array, AD (analog-to-digital) quantization, and writing to DRAM. As mentioned above, R represents a combination of decomposable operations, including five operations: reading DRAM, DAC, VMM, ADC, and writing to DRAM. Figure 6 As shown in part (1) of the diagram, without pipelining, R1 executes its five operations before R2 is executed. The chip's computation can be implemented using a data flow-driven mode. For example, once the data accumulated in Rank is sufficient to start the computation of the next tile group, the computation of the next tile group can be controlled. Therefore, pipelining can be used to shorten the computation latency. Since M3D has sufficient hardware resources and integrated DRAM can store intermediate results on-chip, pipelining can be implemented. That is, when R1 reads DRAM and performs DAC operations, R2 can simultaneously perform DRAM read operations, and so on, thereby significantly reducing the latency.

[0067] The in-memory computing chip according to embodiments of this disclosure further eliminates the "memory wall" of RRAM-based in-memory computing chips, improving their performance and energy efficiency. It effectively eliminates the transfer of neural network input / output feature map data between off-chip storage and RRAM computing units, and utilizes data reuse of the neural network under fixed weights, thereby improving the performance and energy efficiency of the in-memory computing chip. This chip architecture can be applied to artificial intelligence fields such as image recognition and speech recognition, and has the advantage of high energy efficiency.

[0068] At least one embodiment of this disclosure also provides a method for operating a memory computing chip for any of the above embodiments.

[0069] Figure 7 A flowchart of an operation method provided by at least one embodiment of the present disclosure is shown.

[0070] like Figure 7 As shown, the operation method may include steps S410 to S430.

[0071] Step S410: Read input data from the data memory array and provide the input data to the in-memory computing device array.

[0072] Step S420: The in-memory computing device array processes the input data to obtain processed output data.

[0073] Step S430: Store the processed output data into the data storage array.

[0074] For example, step S420 includes: the in-memory computing device array performing a vector-matrix multiplication operation on the input data to obtain processed output data; or the in-memory computing device array using stored weight data to perform a vector-matrix multiplication operation on the input data to obtain neural network feature map data.

[0075] For example, the in-memory computing device array includes a first block and a second block, and the data memory array includes a first memory group corresponding to the first block and a second memory group corresponding to the second block. In this case, step S420 includes: the first block processes the input data to obtain output data processed by the first block; step S430 includes: storing the output data processed by the first block into the first memory group; step S410 includes: reading the output data processed by the first block from the first memory group and providing the output data processed by the first block to the second block as input data for the second block.

[0076] The operation method can be found in the relevant description in the above embodiments regarding in-memory computing chips, and will not be repeated here.

[0077] At least one embodiment of this disclosure also provides a method for manufacturing a memory computing chip for any of the above embodiments.

[0078] Figure 8 A flowchart of a manufacturing method provided by at least one embodiment of the present disclosure is shown.

[0079] like Figure 8 As shown, the manufacturing method may include steps S510 to S530.

[0080] Step S510: Provide a substrate.

[0081] Step S520: A control processing circuit, a data memory array, and a memory computing device array are layered and formed on the substrate.

[0082] For example, step S520 may include forming a control processing circuit on a substrate, forming a data memory array on the side of the control processing circuit away from the substrate, and forming a memory computing device array on the side of the data memory array away from the control processing circuit.

[0083] For example, the substrate, control processing circuit, data memory array, and in-memory computing device array can be found in the relevant descriptions in the embodiments of the in-memory computing chip described above, and will not be repeated here. The chip fabrication process can employ conventional methods in the art.

[0084] At least one embodiment of this disclosure also provides an electronic device, which includes a memory computing chip. The memory computing chip can refer to the memory computing chip in any of the above embodiments, and will not be described again here.

[0085] For example, in some embodiments, the electronic device can be a server, mobile phone, tablet computer, laptop computer, digital photo frame, wearable electronic device, smart home device, etc.

[0086] For example, electronic devices may also include other components, such as other processing devices, including a central processing unit (CPU), a graphics processing unit (GPU), a direct memory access (DMA) controller, or other forms of processing units with data processing and / or program execution capabilities. For instance, under the control of the CPU, data requiring processing (e.g., neural network training or inference) is provided to the in-memory computing chip via DMA, and then the processed data is received from the in-memory computing chip. The CPU or GPU can then perform further processing on the processed data, including classification and recognition. Furthermore, electronic devices may include interconnecting devices such as buses to interconnect the various components included in the electronic device. Electronic devices may also include memory, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disks, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc.

[0087] It should be noted that the specific functions and technical effects of the electronic devices in the embodiments of this disclosure can be referred to the description of the in-memory computing chip above, and will not be repeated here.

[0088] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0089] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0090] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0091] The following points should be noted regarding this disclosure:

[0092] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0093] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0094] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. A memory computing chip, comprising: Substrate; In-memory computing device array, configured to perform in-memory computing operations; A data storage array configured to store data; as well as The control processing circuit is configured to communicate with the data memory array and the in-memory computing device array to read input data from the data memory array and provide the input data to the in-memory computing device array, and to receive processed output data from the in-memory computing device array and provide the processed output data to the data memory array. The control processing circuit, the in-memory computing device array, and the data memory array are disposed on the substrate and are provided in different structural layers in a direction perpendicular to the substrate, and at least partially overlap.

2. The chip according to claim 1, wherein, The control processing circuit is formed on the substrate. The data storage array is formed on the side of the control processing circuit away from the substrate. The in-memory computing device array is formed on the side of the data storage array away from the control processing circuit.

3. The chip according to claim 1, wherein, The in-memory computing device array is divided into multiple blocks, and the data storage array is divided into multiple storage groups, with the multiple blocks corresponding to the multiple storage groups; The plurality of blocks includes a first block and a second block, and the plurality of storage groups includes a first storage group corresponding to the first block and a second storage group corresponding to the second block. The first storage group is configured to store the output data after the first block is processed, and to provide the output data after the first block is processed to the second block as input data for the second block. The second storage group is configured to store the output data after the second block is processed.

4. The chip according to any one of claims 1-3, wherein, The control processing circuit includes: A digital-to-analog converter configured to convert input data read from the data memory array into an analog signal, so as to provide the analog signal to the in-memory computing device array; An analog-to-digital converter is configured to convert processed analog output data received from the in-memory computing device array into digital output data, so as to provide the digital output data to the data memory array.

5. The chip according to claim 4, wherein, The control processing circuit further includes: The row decoder is configured to decode the row code of the data to be read based on the address data, and send the row code to the data memory array to read the first read data corresponding to the row code from the data memory array; A column decoder is configured to decode the column encoding of the data to be read, so as to select a second read data corresponding to the column encoding from the first read data.

6. The chip according to claim 5, wherein, The control processing circuit further includes: An inductive amplifier configured to read the first readout data from the data memory array; A read queue storage is configured to buffer the second read data; A data selector is configured to select the desired data and send it to the digital-to-analog converter; A logic control unit is configured to generate a logical address and send it to the column decoder, so that the column decoder obtains the column encoding of the data to be written based on the logical address. The write driver is configured to write the data to be written to the data storage array based on the column encoding of the data to be written.

7. The chip according to claim 4, wherein, The control processing circuit includes: A shift accumulator is configured to perform an accumulation operation on the data output by the analog-to-data converter. The activation unit is configured to perform the calculation of the activation function in the neural network algorithm; Pooling units are configured to perform the calculation of pooling functions in neural network algorithms.

8. The chip according to any one of claims 1-3, wherein, The data storage array is a dynamic random access memory array, which includes multiple dynamic random access memory units arranged in an array. The in-memory computing device array is a memristor array, which includes multiple memristor units arranged in an array.

9. A method of operating a chip as described in any one of claims 1-8, comprising: Read input data from the data storage array and provide the input data to the in-memory computing device array; The in-memory computing device array processes the input data to obtain processed output data; The processed output data is stored in the data storage array.

10. The operating method according to claim 9, wherein, The in-memory computing device array processes the input data, including: The in-memory computing array performs vector-matrix multiplication on the input data to obtain processed output data; or The in-memory computing device array uses the stored weight data to perform vector-matrix multiplication on the input data to obtain neural network feature map data.

11. The operating method according to any one of claims 9-10, wherein, The in-memory computing device array includes a first block and a second block, and the data storage array includes a first storage group corresponding to the first block and a second storage group corresponding to the second block. The in-memory computing device array processes the input data to obtain processed output data, including: the first block processes the input data to obtain the output data processed by the first block; Storing the processed output data into the data storage array includes: storing the output data after the first block processing into the first storage group; Reading input data from the data storage array and providing the input data to the in-memory computing device array includes: reading the output data after the first block processing from the first storage group and providing the output data after the first block processing to the second block as input data for the second block.

12. A method for manufacturing a chip as described in any one of claims 1-8, comprising: Provide the substrate; The control processing circuit, the data memory array, and the in-memory computing device array are layered and formed on the substrate.

13. An electronic device comprising the chip as described in any one of claims 1-8.

Images