An in-memory computing circuit and apparatus based on transpose dram cell

Abstract

The application discloses an in-memory computing circuit and device based on a transpose DRAM unit, and belongs to the technical field of integrated circuits. The in-memory computing circuit comprises an N-row N-column DRAM unit to form a memory array, N transpose enabling switch groups and N sensitive amplifiers (SA) circuits. DRAM units on the same column share the same write word line (WWL) and column read word line (Col-RWL). DRAM units on the same row share the same write bit line (WBL) and column read bit line (Col-RBL). In the use mode of transposition, the Col-RWL is renamed as a row read bit line (Row-RBL), and the Col-RBL is renamed as a row read word line (Row-RWL). The in-memory computing device based on the DRAM unit comprises a controller, a peripheral computing circuit and the in-memory computing circuit, and is used for in-memory computing hardware implementation of two key network structures, i.e., a full connection layer and a convolution layer, in a deep neural network, fills the technical gap of realizing matrix in-place transposition based on a DRAM circuit array, and supports weight reuse and parallel computing in an on-chip training process.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of edge device in-memory computing, and more particularly to an in-memory computing circuit and device based on a transposed DRAM cell. BACKGROUND

[0002] In resource-limited mobile edge computing devices, in-memory computing (CIM) is a method for efficiently performing data-centric computing such as neural networks, which is widely used in face recognition, target detection, wearable health detection, and other scenarios. By embedding part of the operation logic into the memory architecture, the memory can perform part of the logic computation, thereby reducing data communication between the arithmetic logic unit and the memory, and reducing energy consumption and computation delay. In-memory computing can effectively solve the above storage wall problem and significantly improve the performance of deep neural networks (DNN).

[0003] In the hardware implementation of deep learning algorithms and models based on in-memory computing technology, in-memory computing with storage media such as static random-access memory (SRAM) has currently implemented on-chip training functions, but the six-transistor circuit structure of traditional SRAM has brought two important problems of large area overhead and the need for additional peripheral circuit support. Compared with non-volatile memory and SRAM, embedded dynamic random-access memory (DRAM) is a high-density, mature process, and compatible with complementary metal oxide semiconductor (CMOS) process storage technology, which has lower cost and can realize size reduction with process size reduction. The one-transistor / two-transistor / three-transistor structure of the traditional embedded DRAM cell has a great advantage in area compared with the SRAM cell under the same process node, and since it is also compatible with CMOS logic process, embedded DRAM is easier to realize large-scale on-chip integration. Based on the above advantages, DRAM stands out from many embedded memories, and has recently become a new research hotspot in the field of in-memory computing in the academic and industrial communities.

[0004] Although DRAM has the significant advantage of high density, in-memory computing based on DRAM cells currently cannot support the on-chip training process that requires weight reuse, resulting in a large hardware and energy overhead for training models at the edge device end. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides an in-memory computing circuit and device based on a transposed DRAM cell, which aims to utilize the symmetry of the transistors in the DRAM cell, to realize in-situ matrix transposition in the array by selecting the readout end of the DRAM cell, so as to construct an in-memory computing circuit and device capable of supporting on-chip training with weight reuse, thereby solving the technical problem that the existing DRAM memory circuit cannot support on-chip training with weight reuse, resulting in high resource consumption of the edge device.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, an in-memory computing circuit based on a transposed DRAM cell is provided, comprising:

[0007] a memory array comprising N rows and N columns of transposed dynamic random access memory (DRAM) cells; all DRAM cells on the same column share the same write word line (WWL) and column read word line (Col-RWL); all DRAM cells on the same row share the same write bit line (WBL) and column read bit line (Col-RBL); under the control of the N columns of WWL, 1-bit data of N rows and N columns is written into the N rows and N columns of DRAM cells through N rows of WBL; in a non-transposed normal readout mode, a readout control signal is input to the memory array through Col-RWL, or a group of calculation data is input, and the storage result of a column in the memory array is output through Col-RBL, or the calculation result of a column in the memory array is output; in a transposed readout mode, Col-RWL is renamed as a row read bit line (Row-RBL), and Col-RBL is renamed as a row read word line (Row-RWL); a readout control signal is input through Row-RWL, or another group of calculation data is input, and the storage result of a row in the memory array is output through Row-RBL, or the calculation result of a row in the memory array is output;

[0008] In one embodiment, the DRAM cell comprises a read transistor (TR) and at least one write transistor (TW);

[0009] N transpose enable switch groups, wherein the ith one of the transpose enable switch groups is provided with a first end connected with an ith row Col-RBL or Row-RWL, a second end connected with an ith column Col-RWL or Row-RWL, a control end of a transpose enable signal TEN input, a third end connected with an ith global read word line GRWL, and a fourth end connected with an ith global read bit line GRBL, 1≤i≤N; the N transpose enable switch groups select the input of the first end and the second end to the third end and the fourth end according to the input signal of the control end, input a read control signal through the GRWL, so that the GRBL outputs the storage result of a column or a row in the memory array, or outputs the calculation result of a column or a row in the memory array, and all the cells in the same row or column can output the storage or calculation result in parallel.

[0010] In one embodiment, the transpose enable switch group includes 4 NMOS tubes and 4 PMOS tubes.

[0011] N sensitive amplifiers SA circuits are connected with the GRBL of the N transpose enable switch groups one by one, for amplifying and outputting the difference between the input data and the read reference level, to obtain the voltage level of the storage data or the calculation result of the corresponding row or column of the memory array.

[0012] In one embodiment, the data storage of the DRAM cell includes a conventional readout mode, a transpose readout mode, a write mode and a standby mode.

[0013] On the basis of the above-mentioned circuit, according to another aspect of the present application, a transpose DRAM cell-based in-memory computing device is provided, comprising: a control circuit, a peripheral computing circuit and at least one transpose DRAM cell-based in-memory computing circuit;

[0014] The in-memory computing circuit is an in-memory computing array in the in-memory computing device, for calculating the bitwise multiplication of a group of weight data and another group of input data; under the control of the controller, a group of calculation data is stored to the corresponding DRAM cell through the write word line WWL and the write bit line WBL, the controller controls the readout mode of the DRAM cell, and executes the readout or calculation in the conventional or transpose mode; the data in the same row or column of the same Row-RWL or Col-RWL in the in-memory computing array can be read out in parallel, or another group of calculation data is input, to realize the parallel bitwise multiplication calculation with the stored calculation data; a plurality of in-memory computing circuits constitute an in-memory computing macro array, and all the in-memory computing arrays in the same in-memory computing macro array can calculate in parallel;

[0015] The peripheral computing circuit is connected with the output end of the SA circuit in the in-memory computing circuit, for performing further operation on the calculation result of the in-memory computing circuit;

[0016] The control circuit controls the writing of a set of weight data in the in-memory computing circuit, the input of another set of input data to the in-memory computing circuit, the overall operation of each module of the in-memory computing circuit, the operation of the peripheral computing circuit, and the output of the calculation result.

[0017] Based on the above in-memory computing device, according to another aspect of the present application, the present application also provides a full connection layer forward propagation and reverse propagation process calculation method based on the above in-memory computing device, which specifically comprises:

[0018] The full connection layer weight data mapping method based on the in-memory computing device, based on the in-memory computing device, the full connection layer weight data mapping method based on the in-memory computing device comprises: splitting the weight matrix W of H rows and J columns with a data bit width of K-bit into K H rows and J columns 1-bit matrixes according to bit positions, mapping each H rows and J columns 1-bit matrix according to the row and column position relationship in an H rows and J columns DRAM memory operator array, K H rows and J columns DRAM memory operator arrays form a DRAM memory macro array, and K memory operator arrays are calculated in parallel.

[0019] The full connection layer forward propagation process execution method based on the in-memory computing device, based on the in-memory computing device, the in-memory computing circuit of which stores weight data in the DRAM memory macro array of the in-memory computing circuit based on the full connection layer weight data mapping method; in the neural network training process, the execution mode selection and parallel calculation of the transpose weight matrix are performed when the weight matrix is propagated in the forward direction, so as to realize the forward propagation process of the full connection layer, which specifically comprises: splitting 1 input vector I of 1 row and J columns with a data bit width of K-bit according to bit positions, and inputting the input vector I into the DRAM memory macro array in a bit-serial manner, wherein J columns correspond to J Col-RWLs, the i-th Col-RWL corresponds to the i-th element of the input vector I, and 1≤i≤J. The bit-by-bit multiplication operation with the stored weight data is completed through H rows of Col-RBLs, and the multiplication calculation result is output through SA, and the peripheral computing circuit completes the shift addition and accumulation operation to obtain an output vector O of 1 row and 1 column; in this calculation mode, H units sharing the same Col-RWL can be calculated in parallel; and K memory operator arrays can also be calculated in parallel.

[0020] The full connection layer error calculation method based on the in-memory computing device is based on the in-memory computing device, and an error calculation is performed on a forward propagation output vector O of a full connection layer in a neural network training process to generate an input vector of a backward propagation process of the full connection layer, and specifically includes: quantizing the H-row 1-column output vector O in a peripheral computing circuit according to a bit width of K-bit, and performing a difference between the output vector prediction value O' with a bit width of K-bit to obtain an H-row 1-column error vector ΔO = O' - O with a bit width of K-bit as an input vector of the backward propagation.

[0021] The full connection layer backward propagation execution method based on the in-memory computing device is based on the in-memory computing device, and a weight data storage is stored in a DRAM storage calculation macro array of a storage calculation circuit of the in-memory computing device based on the full connection layer weight data mapping method; in a neural network training process, a mode selection and parallel calculation are performed on a transpose weight matrix when the weight matrix is required to be propagated backward to realize a backward propagation process of the full connection layer, and specifically includes: based on the error calculation method, an error vector ΔO after the forward propagation is calculated as an input vector of the backward propagation process; the input vector is input from a Col-RWL direction in the forward propagation process to a Row-RWL direction in the backward propagation process through a transpose enable switch group; an H-row 1-column error vector ΔO with a bit width of K-bit is input row by row through an H-row Row-RWL, wherein an i-th element of the error vector ΔO is input by an i-th Row-RWL, 1≤i≤H; J ΔO vectors are multiplied by bit through a J-column Row-RBL, and a multiplication calculation result is output through an SA, and a shift addition and accumulation operation is completed in a peripheral computing circuit to obtain an input error vector ΔI; in this calculation mode, J units sharing the same Row-RWL can be calculated in parallel; and K storage calculation subarrays can also be calculated in parallel.

[0022] In one embodiment, the in-memory computing device first performs a weight data mapping of the full connection layer according to the full connection layer data mapping method based on the in-memory computing device, then performs a forward propagation process of the full connection layer according to the full connection layer forward propagation process execution method based on the in-memory computing device, then performs a forward propagation error calculation process of the full connection layer according to the full connection layer error calculation method based on the in-memory computing device, and finally performs a backward propagation process of the full connection layer according to the full connection layer backward propagation execution method based on the in-memory computing device.

[0023] Based on the above in-memory computing device, according to another aspect of the present application, the present application also provides a full connection layer forward propagation and backward propagation process calculation method based on the above in-memory computing device, and specifically includes:

[0024] The method for mapping data of a convolution kernel matrix based on an in-memory computing device, based on the in-memory computing device, the method for mapping data of a convolution kernel matrix based on an in-memory computing device comprises: flattening elements in a convolution kernel matrix with a data bit width of K-bit and a size of M rows and M columns in a row direction from the top left to the bottom right of the convolution kernel matrix in sequence to M 2 rows and K columns of data with a data bit width of K-bit, then splitting the flattened data into K pieces of 1-bit data with M 2 rows and 1 column, mapping the split data in an M 2 row K column DRAM memory operator array; for P channels of the same convolution layer, mapping in the same way in P M 2 row K column DRAM memory operator arrays, P M 2 row K column DRAM memory operator arrays form a DRAM memory operator macro array, and P memory operator arrays are calculated in parallel.

[0025] The method for executing a forward propagation process of a convolution layer based on an in-memory computing device, based on the in-memory computing device, the in-memory computing circuit of which stores data of a convolution kernel matrix in a DRAM memory operator macro array of the in-memory computing circuit based on the method for mapping data of a convolution kernel matrix; in the process of training a neural network, the mode selection and parallel calculation are performed on the demand for transposition when the convolution kernel matrix is propagated forward to realize the forward propagation process of the convolution layer, specifically comprising: selecting an input matrix I with a data bit width of K-bit and M rows and M columns in a step of 1 from an input matrix with a data bit width of K-bit and Q rows and Q columns, wherein M≤Q, and performing the following calculations respectively; flattening the input matrix I in the same way as the convolution kernel matrix from the top left to the bottom right of the matrix in sequence to M 2 rows and K columns of data with a data bit width of K-bit, and inputting the M 2 rows and K columns of data with a data bit width of K-bit, and inputting the M 2 rows and K columns of data with a data bit width of K-bit, and inputting the M 2 rows and K columns of data with a data bit width of K-bit, and inputting the M 2; the K Col-RBLs complete the bit-by-bit multiplication operation of the weight data and the input data, and the multiplication calculation result is output through the SA, the peripheral calculation circuit completes the shift addition and accumulation operation, and an output value in the output matrix O is obtained; the above process is repeated to select different M rows and M columns of the input matrix I from the Q rows and Q columns of the input matrix with a data bit width of K-bit, and output values in different positions in the output matrix O are obtained, the row and column positions of the output values in the output matrix O are the same as the row and column positions of the M rows and M columns of the input matrix I in the Q rows and Q columns of the input matrix, and finally a Q-M+1 row and Q-M+1 column output matrix O is obtained; in this calculation mode, the K units sharing the same Col-RWL can be calculated in parallel; and the different channels in different memory-computing array can also be calculated in parallel.

[0026] The convolution layer error calculation method based on the in-memory computing device includes the following steps: quantizing the output matrix O with a size of Q-M+1 rows and Q-M+1 columns in the peripheral calculation circuit according to a bit width of K-bit, and performing a difference operation between the quantized output matrix O and a predicted value O' of the output matrix to obtain an error matrix ΔO with a bit width of K and a size of Q-M+1 rows and Q-M+1 columns, and then externally supplementing 0 to expand the error matrix ΔO into an error matrix with a size of R rows and R columns, where R=M+Q-1, as an input matrix of the back propagation process.

[0027] The convolution layer back propagation execution method based on the in-memory computing device includes the following steps: in the back propagation mode, the deployment mode of the convolution kernel matrix with a data bit width of K and a size of M rows and M columns in the DRAM memory-computing array is the same as that in the forward propagation, the error matrix ΔO with a data bit width of K and a size of M rows and M columns is selected from the error matrix with a size of R rows and R columns in a step of 1, the error matrix ΔO is flattened into M 2 column and K rows of data according to the flattening mode of the input matrix I, the M 2 -i column data of the error matrix ΔO after flattening corresponds to the i-th Col-RWL, 1≤i≤M 2 ; the K Col-RBLs complete the bit-by-bit multiplication operation of the weight data and the input data, and the multiplication calculation result is output through the SA, the peripheral calculation circuit completes the shift addition and accumulation operation, and an output value in the output matrix O is obtained; the above process is repeated to select different M rows and M columns of the input matrix I from the Q rows and Q columns of the input matrix with a data bit width of K-bit, and output values in different positions in the output matrix O are obtained, the row and column positions of the output values in the output matrix O are the same as the row and column positions of the M rows and M columns of the input matrix I in the Q rows and Q columns of the input matrix, and finally a Q-M+1 row and Q-M+1 column output matrix O is obtained; in this calculation mode, the K units sharing the same Col-RWL can be calculated in parallel; and the different channels in different memory-computing array can also be calculated in parallel.

[0028] In one of the embodiments, the in-memory computing device first performs the convolution kernel matrix data mapping of the convolution layer according to the convolution kernel matrix data mapping method based on the in-memory computing device, then performs the forward propagation process of the convolution layer according to the convolution layer forward propagation process execution method based on the in-memory computing device, secondly performs the forward propagation error calculation process of the convolution layer according to the convolution layer error calculation method based on the in-memory computing device, and finally performs the backward propagation process of the convolution layer according to the convolution layer backward propagation execution method based on the in-memory computing device.

[0029] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0030] (1) The in-memory computing circuit based on a transposed DRAM cell provided in the present application includes an N-row and N-column dynamic random access memory (DRAM) cell to form a memory array, N transposition enabling switch groups, and N sensitive amplifiers (SA) circuits. The present application fills the technical gap of implementing matrix in-place transposition based on a DRAM circuit array, utilizes the symmetry of a DRAM read transistor, and realizes in-place matrix transposition in the array by selecting a DRAM readout end, thereby supporting weight reuse of an in-memory computing system for on-chip training and inference.

[0031] (2) The in-memory computing device provided in the present application fully utilizes the parallel computing characteristics of the array, and the same row or column data in the array can be read out or calculated in parallel, and the arrays can be read out or calculated in parallel, thereby improving the parallel degree of the overall calculation. While achieving high accuracy and high density, the present application reduces the operation amount hardware overhead and improves the hardware utilization rate. Meanwhile, by utilizing the characteristics of the memory array itself, the present application completes the neural network mapping method based on the in-memory computing device while maintaining the storage function of the DRAM array.

[0032] (3) The in-memory computing device provided in the present application is used to implement full connection layer calculation of a neural network. The present application can realize mapping of the forward and backward propagation processes of the full connection layer in the neural network on the in-memory computing device, and avoids re-writing of the transposed weight matrix during the backward propagation calculation; the present application improves the weight array reuse rate and reduces the data movement amount.

[0033] (4) The in-memory computing device provided in the present application is used to implement convolution layer calculation of a neural network. The present application can realize circuit mapping of the forward and backward propagation processes of the convolution layer in the neural network, and avoids re-writing of the transposed weight matrix during the backward propagation calculation based on the principle that the weight matrix for the convolution layer backward propagation process is rotated by 180° compared with the forward propagation; the present application improves the weight array reuse rate and reduces the data movement amount. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1Circuit diagram of in-memory computing circuit based on DRAM cell for an embodiment of the present application.

[0035] Figure 2 In-memory computing device diagram for an embodiment of the present application.

[0036] Figure 3 Circuit diagram of DRAM cell for an embodiment of the present application.

[0037] Figure 4a and Figure 4b Circuit diagram of transpose enable switch group for an embodiment of the present application.

[0038] Figure 5 Workflow diagram for implementing deep neural network fully connected layer computation using in-memory computing device of the present application.

[0039] Figure 6 Process instance diagram for implementing deep neural network fully connected layer computation using in-memory computing device for an embodiment of the present application.

[0040] Figure 7 Workflow diagram for implementing deep neural network convolution layer computation using in-memory computing device of the present application.

[0041] Figure 8 Process instance diagram for implementing deep neural network convolution layer computation using in-memory computing device for an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as there is no conflict.

[0043] Embodiment 1

[0044] The present application provides an in-memory computing circuit based on transpose DRAM cell, comprising an N-row N-column memory array taking DRAM cell as a basic unit, a transpose enable switch group, and a readout circuit sensitive amplifier for each column of the memory array.

[0045] The memory array comprises N rows and N columns of transposed dynamic random access memory (DRAM) units; all DRAM units in the same column share the same write word line (WWL) and column read word line (Col-RWL); all DRAM units in the same row share the same write bit line (WBL) and column read bit line (Col-RBL); N rows and N columns of 1-bit data are written into the DRAM units under the control of the WWL through the WBL; in a conventional use mode, a read control signal is input through the Col-RWL, or another set of calculation data is input, and the storage result of a column in the memory array is output through the Col-RBL, or the calculation result of a column in the memory array is output; in a transposed use mode, the Col-RWL is renamed as a row read bit line (Row-RBL), the Col-RBL is renamed as a row read word line (Row-RWL), a read control signal is input through the Row-RWL, or another set of calculation data is input, and the storage result of a row in the memory array is output through the Row-RBL, or the calculation result of a row in the memory array is output.

[0046] N transposed enable switch groups, wherein the i-th transposed enable switch group is provided with a first end connected to the Col-RBL on the i-th row of DRAM units, a second end connected to the Col-RWL on the i-th column of DRAM units, a control end of a transposed enable signal TEN input, a third end connected to the i-th global read word line (GRWL), and a fourth end connected to the i-th global read bit line (GRBL), 1≤i≤N; the N transposed enable switch groups enable the third end and the fourth end according to the signal of the control end, input a read control signal through the GRWL connected to the third end, so that the GRBL connected to the fourth end outputs the storage data of a column or a row in the memory array, or outputs the multiplication result of a column or a row in the memory array.

[0047] N sensitive amplifier (SA) circuits, which are connected to the fourth end of the N transposed enable switch groups through the GRBL one by one, for amplifying and outputting the difference between the input data and the read reference level to obtain the voltage level of the storage data on the corresponding row or column of the memory array.

[0048] It should be noted that the use mode of the transposed enable switch group is not limited to the above description. If an input driving circuit is arranged outside the Col-RWL and Row-RWL by column and row respectively, and an SA circuit and a peripheral calculation circuit are arranged outside the Col-RBL and Row-RBL by column and row respectively, this circuit still holds.

[0049] As Figure 1As shown, an in-memory computing circuit based on a transpose DRAM cell is exemplified, with a memory array size of 4 rows and 4 columns, and both constituent transistors of the 2T DRAM cell being NMOS. The proposed in-array matrix transpose circuit is described in detail. It can be understood that the circuit system in this embodiment can include other array sizes composed of DRAM cells composed of CMOS transistors. The circuit includes: a memory array composed of DRAM cells 1, in which all cells on the same column share the same WWL and Col-RWL (Row-RBL), and all cells on the same row share the same WBL and Col-RBL (Row-RWL), and the sequence numbers of the signal lines follow the order from top to bottom and from left to right; a transpose enable switch 2 composed of four transmission gate switches, each group of Col-RWL (Row-RBL) and Col-RBL (Row-RWL) is equipped with one, which selects the corresponding Col-RWL and Col-RBL to GRWL and GRBL respectively in the normal mode, and selects Row-RWL and Row-RBL to GRWL and GRBL respectively in the transpose mode; a read reference cell 3 and a sense amplifier 4, each group of GRBL is equipped with one, and the sense amplifier SA is used to compare the content on the GRBL with the read reference cell REF, distinguish the storage content of the storage cell, and drive the SA output end to a stable voltage level corresponding to the storage node data or the calculation result.

[0050] Embodiment 2

[0051] The application also provides an in-memory computing device, comprising: a controller, at least one in-memory computing circuit based on a transpose DRAM cell, and a peripheral computing circuit; and a process for executing the working process of the in-memory computing circuit based on the transpose DRAM cell.

[0052] As Figure 2 shown, an in-memory computing device is exemplified, which includes a controller, two in-memory computing circuits based on a transpose DRAM cell, and a peripheral computing circuit.

[0053] Embodiment 3

[0054] The DRAM cell includes a read tube TR and at least one write tube TW; wherein the at least one write tube TW has a gate connected to the WWL of the column where the DRAM cell is located, and a drain connected to the WBL of the row where the DRAM cell is located; the read tube TR has a gate connected to the source of the write tube TW, a source connected to the Col-RWL (Row-RBL), and a drain connected to the Col-RBL (Row-RWL), so as to realize data storage by using the gate-source parasitic capacitance as a storage capacitor.

[0055] As Figure 3As shown, taking a 2T DRAM cell as an example, the DRAM cell includes a write transistor TW and a read transistor TR. The gate of the write transistor TW is connected to the WWL of the column where the DRAM cell is located, and its drain is connected to the WBL of the row where the DRAM cell is located. The gate of the read transistor TR is connected to the source of the write transistor TW, and its gate-source parasitic capacitance is used as the storage capacitance to realize data storage. For a memory array of N rows and N columns of DRAM cells, the Col-RBL (Row-RWL) and Col-RWL (Row-RBL) of the array are selected to the global read word line GRWL and the global read bit line GRBL through a transpose enable switch group. Readout control is performed using GRWL. GRBL reads out a column (normal mode) or a row (transposed mode) of the array through a Sensing Amplifier (SA) circuit group, or inputs the calculation result of a column (normal mode) or a row (transposed mode) of the array into the peripheral computing circuit.

[0056] Specifically, each DRAM cell includes a write transistor (TW) and a read transistor (TR). The gate of the write transistor is connected to the write word line (WWL), and the drain is connected to the write bit line (WBL). The source of the write transistor is connected to the gate of the read transistor, using the gate-source parasitic capacitance of the read transistor as the storage capacitance to achieve data storage. During operation, the drain of the read transistor is connected to the read word line Col-RWL in normal mode, which is also the read bit line ROW-RBL in transpose mode; the source of the read transistor is connected to the read bit line Col-RBL in normal mode, which is also the read word line ROW-RWL in transpose mode. The write transistor TW and the read transistor TR can be either NMOS or PMOS transistors.

[0057] Example 4

[0058] like Figure 3 As shown, each transpose enable switch group includes 4 NMOS transistors and 4 PMOS transistors. The connection is such that each NMOS transistor and each PMOS transistor forms a selector switch. The transpose enable switch group includes first, second, third, and fourth selectors. The selectors are connected such that the source of the NMOS transistor is connected to the source of the PMOS transistor, and the drain of the NMOS transistor is connected to the drain of the PMOS transistor. The transpose enable switch group is configured such that the sources of the first and second selectors are connected to GRWL, and the sources of the third and fourth selectors are connected to GRBL; the drains of the second and third selectors are connected to Col-RWL (also known as Row-RBL), and the drains of the first and fourth selectors are connected to RBL (also known as Row-RWL). The gates of the NMOS transistors of the first and third gating switches and the gates of the PMOS transistors of the second and fourth gating switches are connected to the TEN signal; the gates of the PMOS transistors of the first and third gating switches and the gates of the NMOS transistors of the second and fourth gating switches are connected to the inverted signal of the TEN signal. Figure 4a andFigure 4b Circuit diagram of the transposition enable switch group for the embodiment.

[0059] In the transposition enable switch group, TEN is used as the transposition enable signal. When the transposition enable signal TEN is set to high level representing signal 1, Row-RBL is connected to GRBL and ROW-RWL is connected to GRWL, which is the transposition mode. When the transposition enable signal TEN is set to low level representing signal 0, Col-RBL is connected to GRBL and Col-RWL is connected to GRWL, which is the normal mode.

[0060] Embodiment 5

[0061] The data storage of the DRAM unit includes the normal readout mode, which is the column readout mode with Col-RWL connected to GRWL and Col-RBL connected to GRBL. In the normal readout mode, the drain Col-RWL of the readout tube TR is set to low level and the source Col-RBL is set to high level. The readout tube TR is used to generate the source-drain current corresponding to the stored data.

[0062] The data storage of the DRAM unit includes the transposition readout mode, which is the row readout mode with ROW-RBL connected to GRBL and ROW-RWL connected to GRWL. In the transposition readout mode, the source Row-RWL of the readout tube TR is set to low level and the drain Row-RBL is set to high level. The readout tube TR generates the source-drain current related to the stored data, and the current direction is opposite to that in the normal readout mode.

[0063] The data storage of the DRAM unit includes the write-in mode, which is operated as follows: the gate of the write-in tube TW is set to the level that makes the write-in tube TW conductive, and the drain WBL is set to the level corresponding to the write-in data. The source and the drain of the readout tube TR are both set to high level.

[0064] The data storage of the DRAM unit includes the standby mode, which is operated as follows: the gate of the write-in tube TW is set to the level that makes the write-in tube TW non-conductive, and the drain WBL is set to high level. The source RBL and the drain RWL of the readout tube TR are both set to high level.

[0065] Combination Figure 3 , Figure 4a and Figure 4b Unit case diagram, for each operation of the DRAM unit is introduced one by one:

[0066] (1)Write mode: the process of storing data into the memory cell. The gate of the write transistor TW is set to a level that turns on the write transistor, in this embodiment, the power supply voltage VDD, the drain of the write transistor WBL is set to a level corresponding to the write data, in this embodiment, 1 / 0 corresponds to VDD / 0V respectively, the source and the drain of the read transistor TR are both set to a high level VDD;

[0067] (2)Standby mode: the process of keeping data without external operation. The gate of the write transistor TW is set to a level that turns off the write transistor, in this embodiment, 0V, the drain of the write transistor WBL is set to the power supply voltage VDD to slow down the leakage of the stored data 1, the source and the drain of the read transistor TR are both set to a high level VDD;

[0068] (3)Normal read mode: the column read mode in which Col-RWL is connected to GRWL and Col-RBL is connected to GRBL. The drain of the read transistor Col-RWL is set to a low level 0V, and the other ports remain the same as in the standby mode, and the read transistor generates a source-drain current related to the stored data;

[0069] (4)Transposed read mode: the row read mode in which Row-RBL is connected to GRBL and Row-RWL is connected to GRWL. The source of the read transistor RBL is set to a low level 0V, and the other ports remain the same as in the standby mode, and the read transistor generates a source-drain current related to the stored data, and the current direction in the cell is opposite to that in the normal read mode.

[0070] Preferably, for an array size of 2 rows and 2 columns, the write data is 1, 1 and 0, 0, and the read data is 1, 1 and 0, 0. Figure 1 The normal and transposed read operation processes of the specific circuit diagram are introduced as follows:

[0071] Firstly, according to the configuration of the write mode, WWL[1]~WWL[2] are sequentially set to a high level VDD to turn on, and WBL[1]~WBL[2] are sequentially configured to a voltage level corresponding to the write data: when WWL[1] is turned on, WBL[1] is configured to VDD and WBL[2] is configured to VDD, corresponding to data 1, 1; when WWL[2] is turned on, WBL[1] is configured to 0V and WBL[2] is configured to 0V, corresponding to data 0, 0.

[0072] Secondly, after the write is completed, according to the configuration of the normal read mode, TEN is configured to 0V, at this time, GRWL is sequentially connected with Col-RWL and GRBL is sequentially connected with Col-RBL; GRWL[1]~GRWL[2] are sequentially set to a low level 0V to generate a read current, and the SA outputs corresponding to GRBL[1] and GRBL[2] are respectively the contents of the memory cells on the same column: 1, 1 and 0, 0;

[0073] Third step, according to the configuration of the transpose read mode, TEN is configured as VDD, at this time GRWL is connected with Row-RWL in turn, and GRBL is connected with Row-RBL in turn; GRWL[1]~GRWL[2] are sequentially set to low voltage 0V to generate a read current, and the SA outputs corresponding to GRBL[1] and GRBL[2] are respectively the contents of the storage units on the same row: 1, 0 and 1, 0.

[0074] In combination with the true value table of each storage and calculation case in Table 1, the operation of the storage and calculation unit is introduced:

[0075] Case 1: the input is 1, and the weight is 1. In the write-in mode, the data 1 represented by the weight is written into the storage node, in the read-out mode, the corresponding RWL is set to 0, the normal read 1 operation is performed, and the calculation of 1-bit multiplication 1x1=1 is completed;

[0076] Case 2: the input is 0, and the weight is 1. In the write-in mode, the data 1 represented by the weight is written into the storage node, in the read-out mode, the corresponding RWL is set to VDD, no read current is generated and no read voltage is established, which is equivalent to reading 0, and the calculation of 1-bit multiplication 1x0=0 is completed;

[0077] Case 3: the input is 1, and the weight is 0. In the write-in mode, the data 0 represented by the weight is written into the storage node, in the read-out mode, the corresponding RWL is set to 0, the normal read 0 operation is performed, and the calculation of 1-bit multiplication 0x1=0 is completed;

[0078] Case 4: the input is 0, and the weight is 0. In the write-in mode, the data 0 represented by the weight is written into the storage node, in the read-out mode, the corresponding RWL is set to VDD, no read current is generated and no read voltage is established, which is equivalent to reading 0, and the calculation of 1-bit multiplication 0x0=0 is completed.

[0079] Table 1

[0080]

[0081] Embodiment 6

[0082] The application also provides an in-memory computing circuit based on a transpose DRAM unit, comprising: an in-array matrix transposition circuit based on a DRAM unit, data is stored in the form of a level to the corresponding DRAM unit through each write word line WWL and each write bit line WBL, and the read-out end of the DRAM unit is selected to perform in-array matrix transposition; the in-array matrix transposition comprises: performing in-array matrix transposition and parallel calculation on a weight matrix on demand in a neural network training process, so as to realize the forward and reverse propagation processes of a fully connected layer.

[0083] The training of a deep learning neural network is achieved through repeated iterations of two processes, forward inference and backpropagation. Backpropagation is a method used in deep neural networks to obtain the optimal model that minimizes the loss function, which uses the gradient descent algorithm to calculate the gradient of the objective function with respect to the weight vector layer by layer, and then optimizes the weights to output results that meet the expectations. The training process is usually completed in the cloud or server, but due to the requirements of precision and privacy on the edge device side, it is necessary to deploy part of the backpropagation calculation process on the edge device side with limited resources, and then re-tune the general DNN model trained in the cloud locally to adapt to changes in specific situations.

[0084] The deep neural network fully connected layer training process based on the in-memory computing device:

[0085] Step one: Based on the characteristics that the weight matrix in forward and backward propagation is the matrix itself and its transpose matrix respectively, a data mapping method for weight matrix reuse is proposed. Specifically, the H-row J-column weight matrix W with data bit width K is split into K H-row J-column 1-bit matrices, and each H-row J-column 1-bit matrix is mapped in an H-row J-column DRAM compute array according to the position relationship. K H-row J-column DRAM compute arrays form a DRAM compute macro array, and K compute arrays are calculated in parallel.

[0086] Step two: In the forward propagation mode, the J-row 1-column input vector I with data bit width K is split by bit, and input into the J-column corresponding J Col-RWL in the array in a bit-serial manner. The i-th Col-RWL corresponds to the i-th element of the input vector, 1≤i≤J. The bit multiplication is completed by H-row Col-RBL, and the multiplication result is output by SA. The peripheral calculation circuit completes the shift addition and accumulation operation. In this calculation mode, H cells sharing the same Col-RWL can be calculated in parallel; and in different compute arrays, each bit of the weight matrix needs to be multiplied by each bit of the corresponding input vector to obtain the calculation result, so different compute arrays can be input with the same input vector value, and therefore the cells storing different bits of the same weight element in different compute arrays can also be calculated in parallel.

[0087] Step three: In the forward propagation mode, the calculation result of the H-row J-column weight matrix with data bit width K and the J-row 1-column input vector with data bit width K is an H-row 1-column output vector O, which is quantized in the peripheral calculation circuit according to the bit width K, and the difference between the output vector and the predicted value O’ is obtained as an H-row 1-column error vector ΔO=O’-O, which is used as the input vector of backpropagation.

[0088] Step four: in the back propagation mode, the deployment of the weight matrix W with H rows and J columns and a data bit width of K in the DRAM computing operator array is the same as that in the forward propagation, the input weight is changed from the Col-RWL input direction to the Row-RWL input direction by enabling the switch group through transposition, and the organization method of the weight matrix does not need to be changed. The H-row 1-column error vector ΔO with a bit width of K is input bit by bit in rows through H rows of Row-RWL, and the i-th element of the error vector ΔO is input on the i-th Row-RWL, 1≤i≤H. Bitwise multiplication is completed through J columns of Row-RBL, and the multiplication result is output through SA, and the shift addition and accumulation operation is completed in the peripheral calculation circuit. In this calculation mode, the J cells sharing the same Row-RWL can calculate in parallel; and in different computing operator arrays, each bit of the weight matrix needs to perform bitwise multiplication with each bit of the corresponding error vector to obtain the calculation result, and the same error vector value is input in parallel in different computing operator arrays, so the cells storing different bits of the same weight element in different computing operator arrays can also calculate in parallel.

[0089] After one forward propagation, if in-place transposition cannot be achieved, the weight matrix W stored in the storage array needs to be re-stored as the transposed matrix of the weight matrix W in the neural network training to perform the back propagation process. Based on the in-memory computing device disclosed in the present application, between the forward propagation and the back propagation, the weight matrix does not need to be re-written into the DRAM computing array, but the back propagation process is directly started. The in-memory computing device of the deep neural network full connection layer avoids the data movement delay and power consumption for transposing the weight matrix.

[0090] As shown in Figure 5 and Figure 6 , the present application provides a data mapping method for a deep neural network full connection layer, based on the need for a transposed matrix in the back propagation algorithm. In the forward propagation and the back propagation process, the transposition enabling signals are TEN=0 and TEN=1, respectively. Preferably, the data mapping method for the full connection layer is described in detail by taking a weight bit width of 4-bit and a weight matrix size of 3 rows and 3 columns as an example. It can be understood that the mapping method in the embodiment can include other matrix sizes that can realize the function of the full connection layer.

[0091] The specific implementation steps of the full connection layer calculation according to the design method are introduced as follows:

[0092] Step 1): according to the writing method introduced in the writing mode, the 0th, 1st, 2nd and 3rd bits of the weight matrix W with a size of 3 rows and 3 columns are split into four 1-bit weight matrices with a size of 3 rows and 3 columns, and the four weight matrices are written into the computing operator arrays 0-3 according to the relative positions, respectively;

[0093] Step 2): In the forward propagation mode, the transposition enable signal TEN=0, input through Col-RWL, read out through Col-RBL, and the calculation formula is as follows:

[0094]

[0095] According to the principle of the storage and calculation unit, 4-bit width vectors I are input bit by bit through the corresponding Col-RWL, for example, I1 is input to the corresponding Col-RWL columns of W1,1, W2,1 and W3,1 corresponding to the calculation task, and is multiplied by the weight data stored in the unit, and each SA completes 1-bit multiplication in a working period;

[0096] The output results of all SAs in the storage and calculation subarray are accumulated in the peripheral calculation circuit according to the weight to obtain the calculation result, for example: O1=W1,1* I1+W1,2* I2+W1,3* I3;

[0097] Step 3): After the completion of one forward propagation process, the output result is subtracted from the target expected result to obtain the error value of this forward propagation, for example: ΔO1=O1_calculate-O1_expect;

[0098] Step 4): In the backward propagation mode, the transposition enable signal TEN=1, input through Row-RWL, and read out through Row-RBL, and the calculation formula is as follows:

[0099]

[0100] According to the principle of the storage and calculation unit, vectors are input bit by bit through the corresponding Row-RWL, for example, 4-bit width vectors ΔO1 are input to the corresponding Row-RWL rows of W1,1, W1,2 and W1,3 corresponding to the calculation task, and the input data is multiplied by the weight data stored in the unit, each SA completes 1-bit multiplication in a working period, and all SAs are calculated in parallel;

[0101] The output results of all SAs in the storage and calculation subarray are accumulated in the peripheral calculation circuit according to the weight to obtain the calculation result, for example: ΔI1=W1,1* ΔO1+W2,1* ΔO2+W3,1* ΔO3;

[0102] Example 7

[0103] The storage-in-computation circuit based on the transposed DRAM unit provided by the application further comprises: performing in-situ matrix transposition and parallel calculation in the array on demand on the weight matrix in the neural network training process, so as to realize the forward and backward propagation processes of the convolution layer.

[0104] The deep neural network convolution layer training process based on the storage-in-computation device:

[0105] Specifically, the in-memory computing process of the transpose DRAM circuit-based deep neural network convolution layer includes: constructing a transpose DRAM circuit array, and using the characteristics that the weight matrix in the forward and backward propagation of the convolution layer is the matrix itself and the matrix rotated by 180 degrees along the center point of the matrix itself respectively to realize the in-memory computing of the deep neural network convolution layer.

[0106] Step one: flattening the convolution kernel matrix with a data bit width of K and a size of M rows and M columns from the upper left to the lower right of the convolution kernel matrix in sequence into a row of data with a data bit width of K and M 2 rows and 1 column, then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M

[0107] Step two: in the forward propagation mode, selecting data from the input matrix I with a width of K and Q rows and Q columns in a step of 1 to obtain an input matrix I with a width of K and M rows and M columns, where M≤Q, and performing the following calculations respectively. Flattening the input matrix I in the same manner as the convolution kernel matrix from the upper left to the lower right of the matrix in sequence into a column of data with a data bit width of K and M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 row and 1 column data, and then splitting the convolution kernel matrix by bit into K M 2 . By K Col-RBL, the bit multiplication operation is completed, and the multiplication calculation result is output through SA. In this calculation mode, K units sharing the same Col-RWL can be calculated in parallel; and different storage operator arrays in different channels can also be calculated in parallel.

[0108] Step three: in the forward propagation mode, the calculation result of the M row and M column convolution kernel matrix with data bit width K and the Q row and Q column input matrix with data bit width K is the output matrix O, the calculation step is 1, and the output matrix size is Q-M+1 row and Q-M+1 column. It is quantized in the peripheral calculation circuit according to the bit width K, and the difference between the predicted value O' of the output matrix is obtained, which is the error matrix O' - O with Q-M+1 row and Q-M+1 column and bit width K. It is externally supplemented 0 and expanded to an error matrix with R rows and R columns, where R = M + Q - 1, as the input matrix of the backward propagation.

[0109] Step four: in the backward propagation mode, the deployment mode of the M row and M column convolution kernel matrix with data bit width K in the DRAM storage operator array is the same as that in the forward propagation, and the data with bit width K is selected from the error matrix with R rows and R columns in the form of step 1 to obtain the error matrix with M rows and M columns, and the i-th Col-RWL corresponds to the M 2 -i element of the error matrix ΔO, 1 ≤ i ≤ M 2 The bit multiplication operation is completed through K Col-RBL, and the multiplication calculation result is output through SA, and the shift addition and accumulation operation is completed in the peripheral calculation circuit. In this calculation mode, K units sharing the same Col-RWL can be calculated in parallel; and different channels in different storage operator arrays can also be calculated in parallel.

[0110] According to Figure 7 and Figure 8 , the specific implementation steps of the convolution layer calculation of the design method are as follows:

[0111] As shown in Figure 7 , the present application provides a data mapping method of a deep neural network convolution layer, which can be compatibly used in the above-mentioned matrix transposition circuit in the array based on the DRAM unit. In the backward propagation calculation process of the convolution layer, matrix transposition is not required, so the transposition enable signal TEN = 0; GRWL and Col-RWL, GRBL and Col-RBL are enabled by default. Preferably, taking the weight bit width 4-bit and the convolution kernel matrix size 3 row and 3 column as an example, the data mapping method of the convolution layer is described in detail. It can be understood that the mapping method in the embodiment can include other matrix sizes that realize the function of the convolution layer.

[0112] Preferably, the specific implementation steps of the design method are as follows:

[0113] Step (1): according to the writing method introduced in the writing mode, the convolution kernel matrix with size 3 row and 3 column W is flattened into a row in the order from the top left to the bottom right, and then the column data is split into four rows according to the 4-bit bit width, and written into the four rows of the DRAM storage operator array;

[0114] Repeat step (1), if there are two channels, write the second channel into another array of storage-operators in the same way;

[0115] Step (2): in the forward propagation mode, the calculation formula is as follows:

[0116]

[0117] According to the principle of storage-operators, input a 3*3 input matrix I with 4-bit bit width bit by bit through the corresponding Col-RWL, for example, input I1,1[0]~I1,1[3] bit by bit into the column where W1,1[0]~W1,1[3] are located, and multiply the weight data stored in the unit, the working period of each SA completes 1-bit multiplication, and 4 times of 1-bit multiplication are completed between 4 rows in parallel;

[0118] Accumulate all output results of SAs in the storage-operator array in the peripheral calculation circuit according to the weight to obtain the calculation result;

[0119] Step (3): after the completion of one forward propagation process, the output result is subtracted from the target expected result to obtain the error value of this forward propagation, for example, ΔO1,1=O1,1_calculate-O1,1_expect;

[0120] Step (4): in the backward propagation mode, the calculation formula is as follows:

[0121]

[0122] Wherein,

[0123]

[0124] According to the principle of convolution layer backward propagation, rotate the filled ΔO matrix by 180° to obtain the input matrix of the backward propagation process, according to the principle of storage-operators, input a 3*3 error matrix ΔO with 4-bit bit width bit by bit, for example, input ΔO5,5[0]~ΔO5,5[3] bit by bit into the column where W1,1[0]~W1,1[3] are located, and multiply the weight data stored in the unit, the working period of each SA completes 1-bit multiplication, and 4 times of 1-bit multiplication are completed between 4 rows in parallel; accumulate all output results of SAs in the storage-operator array in the peripheral calculation circuit according to the weight to obtain the calculation result.

[0125] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An in-memory computing circuit based on a transposed DRAM cell, comprising: include: The memory array comprises N rows and N columns of transposed dynamic random access memory (DRAM) cells; all DRAM cells in the same column share the same write word line (WWL) and column read word line (Col-RWL); all DRAM cells in the same row share the same write bit line (WBL) and column read bit line (Col-RBL); under the control of the N-column WWL, 1-bit data in the N rows and N columns is written to the N rows and N columns of DRAM cells through the N-column WBL; In the non-transposed conventional read mode, a read control signal is input to the memory array via Col-RWL, or a set of calculation data is input, and the storage result of one column in the memory array is output via Col-RBL, or the calculation result of one column in the memory array is output; in the transposed read mode, Col-RWL is renamed Row-RBL, and Col-RBL is renamed Row-RWL. A read control signal is input via Row-RWL, or another set of calculation data is input, and the storage result of one row in the memory array is output via Row-RBL, or the calculation result of one row in the memory array is output; There are N transpose enable switch groups, wherein the i-th transpose enable switch group has a first terminal connected to the i-th row Col-RBL (also known as Row-RWL), a second terminal connected to the i-th column Col-RWL (also known as Row-RBL), a control terminal for the transpose enable signal TEN input, a third terminal connected to the i-th global read word line GRWL, and a fourth terminal connected to the i-th global read bit line GRBL, 1≤i≤N; the N transpose enable switch groups select the inputs of the first and second terminals to the third and fourth terminals according to the input signal of the control terminal, and read the control signal through the GRWL input, so that the GRBL outputs the storage result of one column or one row in the memory array, or outputs the calculation result of one column or one row in the memory array. All cells on the same row or column can output the storage or calculation result in parallel; N sensitive amplifier SA circuits are connected one-to-one with N transpose enable switch groups GRBL, which are used to amplify and output the difference between the input data and the read reference level to obtain the voltage level of the data or calculation result stored in the corresponding row or column of the memory array.

2. An in-memory computing device based on a transposed DRAM cell, comprising: include: The control circuit, the peripheral computing circuit, and at least one in-memory computing circuit based on a transposed DRAM cell as described in claim 1; The in-memory computing circuit is an array of compute-and-memory units in the in-memory computing device, configured to compute a bitwise multiplication of a set of weight data and another set of input data; under the control of the controller, a set of computing data is stored in corresponding DRAM cells through a write word line WWL and a write bit line WBL, and the controller controls the readout mode of the DRAM cells to perform the readout or computation in a normal mode or a transposed mode; the data in a same row or column of the array of compute-and-memory units located in a same Row-RWL or Col-RWL can be read out in parallel or input with another set of computing data to realize a parallel bitwise multiplication computation with the stored computing data; a plurality of in-memory computing circuits form an array of compute-and-memory macros, and all the arrays of compute-and-memory units in a same array of compute-and-memory macros can perform parallel computation. The peripheral computing circuit is connected with the output end of the SA circuit in the in-memory computing circuit, and is configured to perform further operation on the computation result of the in-memory computing circuit. The control circuit controls the writing of a set of weight data in the in-memory computing circuit, the input of another set of input data into the in-memory computing circuit, the overall operation process of each module of the in-memory computing circuit, the operation process of the peripheral computing circuit, and the output of the computation result.

3. A weighted data mapping method for fully connected layers based on in-memory computing devices, characterized in that, The in-memory computing device according to claim 2, wherein a weight data mapping method of a full connection layer of the in-memory computing device comprises: a weight matrix W of H rows and J columns with a data bit width of K-bit is split by bit to K weight matrices of H rows and J columns with a data bit width of 1-bit, each weight matrix of H rows and J columns with a data bit width of 1-bit is mapped in a DRAM compute-and-memory unit array of H rows and J columns according to the row and column position relationship, and K DRAM compute-and-memory unit arrays of H rows and J columns form a DRAM compute-and-memory macro array, and parallel computation is performed among the K compute-and-memory unit arrays.

4. A forward propagation process execution method of a full connection layer based on the in-memory computing device, characterized in that, The in-memory computing device according to claim 2, wherein the in-memory computing circuit stores weight data in the DRAM compute-and-memory macro array of the in-memory computing circuit based on the data mapping method according to claim 3; In the neural network training process, the mode selection and parallel computation are performed on the demand for the transposed weight matrix when the weight matrix is propagated forward, so as to realize the forward propagation process of the full connection layer, and specifically comprising: 1 input vector I of 1 row and J columns with a data bit width of K-bit is split by bit and input into the DRAM compute-and-memory macro array in a bit-serial manner, wherein J columns correspond to J Col-RWLs, the i-th Col-RWL corresponds to the i-th element of the input vector I, and 1≤i≤J; a bitwise multiplication operation is performed on the stored weight data through H rows of Col-RBLs, and the multiplication computation result is output through the SA, and the peripheral computing circuit performs shift addition and accumulation operation to obtain an output vector O of 1 row and 1 column; in this computation mode, H cells sharing a same Col-RWL can perform parallel computation; and parallel computation can also be performed among K compute-and-memory unit arrays.

5. A full connection layer error calculation method based on the in-memory computing device, characterized in that, The in-memory computing device of claim 2, wherein the error calculation is performed on a forward propagation output vector O of a fully connected layer during a neural network training process to generate an input vector of a back propagation process of the fully connected layer, and the error calculation comprises: quantizing the H-row 1-column output vector O in the peripheral computing circuit according to a bit width of K-bit, and subtracting an output vector prediction value O' of the bit width of K-bit to obtain an H-row 1-column error vector ΔO = O' - O of the bit width of K-bit as an input vector of the back propagation.

6. A method for performing a back propagation of a fully connected layer based on an in-memory computing device, and the method comprises: The in-memory computing device of claim 2, wherein the in-memory computing circuit stores weight data in a DRAM computing macro array of the in-memory computing circuit based on the data mapping method of claim 3. The in-memory computing device of claim 2, wherein the error calculation is performed on a forward propagation output vector O of a fully connected layer during a neural network training process to generate an input vector of a back propagation process of the fully connected layer, and the error calculation comprises: The error calculation method of claim 5, wherein the error vector ΔO after the forward propagation is calculated as the input vector of the back propagation process. The input vector is input from the Col-RWL direction in the forward propagation process of claim 4 to the Row-RWL direction in the back propagation process by the transpose enable switch group; the H-row 1-column error vector ΔO of the bit width of K-bit is input row by row by the H-row Row-RWL, wherein the i-th element of the error vector ΔO is input by the i-th Row-RWL, 1≤i≤H; the J ΔO vectors are multiplied by bit by the J-column Row-RBL, and the multiplication result is output by the SA; the shift addition and accumulation operation is completed in the peripheral computing circuit to obtain the input error vector ΔI; in this calculation mode, the J units sharing the same Row-RWL can be calculated in parallel; and the K computing sub-arrays can also be calculated in parallel.

7. A method for mapping convolution kernel matrix data based on in-memory computing devices, characterized in that, The in-memory computing device of claim 2, wherein the data mapping method of a convolution kernel matrix based on the in-memory computing device comprises: The elements in the convolution kernel matrix with data bit width K-bit and size M rows and M columns are flattened in the order from the top left to the bottom right of the convolution kernel matrix in the row direction into M 2 row 1 column data, and then the flattened data is split into K M 2 row 1 column 1-bit data, and the split data is mapped in an M 2 row K column DRAM memory operator array; for P channels of the same convolution layer, the same way is mapped in P M 2 row K column DRAM memory operator array, P M 2 row K column DRAM memory operator array into a DRAM memory operator macro array, and P memory operator arrays are calculated in parallel.

8. A method for performing a forward propagation of a convolution layer based on an in-memory computing device, and the method comprises: The in-memory computing device of claim 2, wherein the in-memory computing circuit stores convolution kernel matrix data in a DRAM computing macro array of the in-memory computing circuit based on the data mapping method of claim 7. The in-memory computing device of claim 2, wherein the error calculation is performed on a forward propagation output vector O of a fully connected layer during a neural network training process to generate an input vector of a back propagation process of the fully connected layer, and the error calculation comprises: An input matrix I of M rows and M columns with a data bit width of K is selected from a Q-row and Q-column input matrix with a data bit width of K-bit in a step of 1, where M≤Q, and the following calculations are performed respectively; the input matrix I is flattened into a column of data with M 2 rows and 1 column and a data bit width of K-bit in a bit-serial manner from left to right and from top to bottom in the same manner as the convolution kernel matrix 2 M rows and K columns of the DRAM storage and computation operator array 2 M rows and K columns of the DRAM storage and computation operator array 2 Col-RWL, the i-th Col-RWL corresponds to the i-th element of the input matrix I, 1≤i≤M 2 ; the bit-by-bit multiplication of the weight data and the input data is completed through the K Col-RBLs, and the multiplication result is output through the SA; the peripheral calculation circuit completes the shift addition and accumulation operation to obtain an output value in the output matrix O; the above process is repeated to select different input matrices I of M rows and M columns from the Q-row and Q-column input matrix with a data bit width of K-bit in a step of 1, and output values at different positions in the output matrix O are obtained; the row and column positions of the output values in the output matrix O are the same as the row and column positions of the M-row and M-column input matrix I in the Q-row and Q-column input matrix; finally, an output matrix O of Q-M+1 rows and Q-M+1 columns is obtained; in this calculation mode, the K units sharing the same Col-RWL can be calculated in parallel; and the different storage and computation operator arrays in different channels can also be calculated in parallel.

9. A method for calculating an error of a convolution layer based on an in-memory computing device, and the method comprises: The in-memory computing device of claim 8, wherein the forward propagation process of the convolution layer based on the in-memory computing device obtains a Q-M+1-row Q-M+1-column output matrix O, and the error calculation is performed on the forward propagation output matrix O of the convolution layer during a neural network training process to generate an input matrix of a back propagation process of the convolution layer, and the error calculation comprises: The output matrix O with a size of Q-M+1 rows and Q-M+1 columns is quantized in the peripheral calculation circuit with a bit width of K-bit, and is subtracted from the predicted value O' of the output matrix to obtain an error matrix ΔO=O'-O with a bit width of K, Q-M+1 rows and Q-M+1 columns, which is externally supplemented with 0 and expanded into an error matrix with R rows and R columns, where R=M+Q-1, as an input matrix of the back propagation process.

10. The method for performing back propagation of a convolution layer based on an in-memory computing device, characterized in that, In the backward propagation mode, the mapping manner of the convolution kernel matrix with the size of M rows and M columns and the data bit width of K-bit in the DRAM computing operator array is the same as the forward propagation, the error matrix ΔO with the size of M rows and M columns and the data bit width of K-bit is selected from the error matrix with the size of R rows and R columns and the data bit width of K-bit in the data with the step of 1, ΔO is flattened into M 2 column K row data, the i-th Col-RWL corresponds to the M 2 -th column data of the flattened error matrix ΔO, 1≤i≤M 2 ; the bitwise multiplication operation of the error matrix ΔO and the convolution kernel matrix is completed through K Col-RBLs, and the multiplication calculation result is output through SA, the shift addition and accumulation operation is completed in the peripheral calculation circuit; in this calculation mode, the K units sharing the same Col-RWL can be calculated in parallel, and different channels in different computing operator arrays can also be calculated in parallel.

Images