In-memory calculation circuit based on BNN algorithm acceleration realization

Abstract

The invention relates to an in-memory calculation circuit based on BNN algorithm acceleration, belongs to the technical field of integrated circuits, and solves the problems of large circuit area, high energy consumption and low energy efficiency of storage calculation in the prior art. The circuit comprises the following steps: selecting a first algorithm network and a second algorithm network in a flexible RRAM storage array; inputting multiple rows of data into a first algorithm network and a multiply-accumulate module to obtain a multiply-accumulate result, and inputting the multiply-accumulate result into a shifter; the shifter shifts the multiply-accumulate result, outputs a shift result to the adder-subtractor, and outputs an addition calculation result to the comparator; the comparator carries out binarization processing on the sum calculation result, outputs the result to a second algorithm network, carries out operation again, outputs column accumulated current, and sends the column accumulated current to a maximum value comparator output module; and the maximum value comparator output module judges that the column where the maximum value of the column sum is located is a classification result. On the premise of keeping the original network precision, the required circuit area is greatly reduced, the energy consumption is reduced, and the energy efficiency of the whole storage and calculation circuit is improved.

Description

Technical Field

[0001] The present invention relates to the field of integrated circuit technology, and in particular to an in-memory computing circuit based on BNN algorithm acceleration. Background Art

[0002] With the rapid development of deep neural network technology in artificial intelligence, the current floating-point (real-valued) neural network has high memory consumption and low computing speed, which need to be solved urgently. Developers hope to apply neural network models to embedded devices or mobile scenarios to solve real-time problems and use them whenever they want. Binarized neural network (BNN) came into being.

[0003] Binarized neural network (BNN) refers to a neural network that uses only +1 and -1 to represent weights and activations. Compared with full-precision neural networks, it can use a very simple combination such as XNOR+popcount instead of float32 multiplication and accumulation to implement convolution operations, thereby saving a lot of memory and calculations, greatly facilitating the deployment of models on resource-constrained devices, and is very suitable for edge computing application scenarios. However, due to the limited amount of information that can be expressed by binary values, BNN has always been far lower than full-precision models in terms of model accuracy. At present, the research on BNN is generally divided into two aspects: how to improve the accuracy of BNN and how to effectively deploy it on low-power and resource-constrained platforms. The traditional von Neumann architecture is subject to the "storage wall bottleneck", with high power consumption and slow speed, while in-memory computing technology can achieve a more efficient computing system and reduce computing power consumption. Since a large number of multiplication and addition convolution operations are the core components of deep learning algorithms, in-memory computing and in-memory logic are very suitable for deep neural network applications of artificial intelligence such as BNN and big data technologies based on AI.

[0004] Currently, RRAM storage and computing arrays want to implement BNN algorithms mostly through XNOR-based operations, which require the use of 2T2R structured RRAM arrays and peripheral circuits. Therefore, in order to further improve energy efficiency without losing accuracy, an in-memory computing circuit based on BNN algorithm acceleration is urgently needed. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide an in-memory computing circuit based on BNN algorithm acceleration to solve the problems of large circuit area, high energy consumption and low memory computing energy efficiency in existing circuits.

[0006] On the one hand, an embodiment of the present invention provides an in-memory computing circuit based on BNN algorithm acceleration, the circuit comprising: a flexible RRAM storage array, a multiplication and accumulation module, a shifter, a comparator, and a maximum value comparison output module;

[0007] According to the BNN network structure and input data, two areas are selected in the flexible RRAM storage array as a first algorithm network and a second algorithm network;

[0008] Input multiple rows of data of each bit into the first algorithm network and the multiplication-accumulation module, the first algorithm network outputs the accumulated current of each column of the storage and calculation unit to the multiplication-accumulation module, the multiplication-accumulation module obtains the multiplication-accumulation result of each column based on the multiple rows of data and the accumulated current of each column, and inputs the multiplication-accumulation result of each column to the shifter;

[0009] The shifter shifts the multiplication and accumulation results of each column by the corresponding number of bits according to the bit positions of each row of input data, and outputs the shifted result to the adder-subtractor in the multiplication and accumulation module. The adder-subtractor performs sum calculation on the shifted results of different bit positions of each column, and outputs the sum calculation result of each column to the comparator;

[0010] The comparator performs binarization processing on the sum calculation results of each column to obtain a binarization result, outputs it to the second algorithm network, calculates and outputs the accumulated current of each column again in the second algorithm network, and sends it to the maximum comparator output module;

[0011] The maximum comparator output module determines the maximum value of the accumulated current in each column, and the column where the maximum value is located is the classification result.

[0012] Furthermore, the multiplication-accumulation module further includes: a popcount module and a multiplier;

[0013] The popcount module receives multiple lines of input and obtains the number of 1s in the multiple lines of input, and outputs the number of 1s to the multiplier;

[0014] The multiplier multiplies the number of 1s by a fixed value β and sends the product result to the adder-subtractor;

[0015] The adder-subtractor subtracts the product result from the digital signal to obtain the matrix multiplication and addition result, and sends it to the shifter.

[0016] Furthermore, the circuit further comprises: a first column multiplexer MUX, a first analog-to-digital converter ADC;

[0017] The first column multiplexer MUX sequentially selects each column storage and calculation unit of the first algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the first analog-to-digital converter ADC;

[0018] The first analog-to-digital converter ADC converts the accumulated current output by the first algorithm network into a digital signal and sends it to the adder-subtractor.

[0019] Furthermore, the circuit further comprises: a second column multiplexer MUX, a second analog-to-digital converter ADC;

[0020] The second column multiplexer MUX sequentially selects each column storage and calculation unit of the second algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the second analog-to-digital converter ADC;

[0021] The second analog-to-digital converter ADC converts the accumulated current output by the second algorithm network into a digital signal and sends it to the maximum value comparison output module.

[0022] Furthermore, the circuit further comprises: a first data buffer module, a second data buffer module and a third data buffer module; all of which are used for temporary storage and integration of data;

[0023] The first data buffer module receives the digital signal output by the first analog-to-digital converter ADC, buffers it, and then outputs it to the adder-subtractor;

[0024] The second data buffer module receives the shift result output by the shifter, buffers it, and then outputs it to the adder-subtractor;

[0025] The third data buffer module receives the binarization result output by the comparator, buffers it, and then outputs it to the second algorithm network.

[0026] Furthermore, the adder-subtractor implements addition operation or subtraction operation through the control signal sent by the FPGA.

[0027] Furthermore, according to the bit positions of the input multiple rows of data, the multiplication and addition result is shifted left by the corresponding number of bits in the shifter; for the multiplication and accumulation result corresponding to the i-th multiple rows of data, the multiplication and accumulation result is shifted left by i bits in the shifter; wherein 0≤i≤7.

[0028] Furthermore, a threshold is preset in the comparator. When the sum calculation result is greater than the threshold, the comparator outputs 1, and when the sum calculation result is less than the threshold, the comparator outputs 0.

[0029] Furthermore, the flexible RRAM storage and computing array includes n×m storage and computing units;

[0030] The word lines WL of each row of storage and calculation units are connected in sequence, the source lines SL of each column of storage and calculation units are connected in sequence, and the bit lines BL of each column of storage and calculation units are connected in sequence, and the bit lines BL are used to output the calculation results of the storage and calculation units in this column;

[0031] Each of the storage and computing units has a 1T1R structure, including a transistor and a memristor.

[0032] Further, the transistor is an indium gallium zinc oxide thin film transistor IGZO TFT, and the memristor is RRAM;

[0033] The gate of the indium gallium zinc oxide thin film transistor IGZO TFT is connected to the word line WL of the storage unit, the drain is connected to the bit line BL of the storage unit, the source is connected to one end of the memristor RRAM, and the other end of the RRAM is connected to the source line SL of the storage unit.

[0034] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0035] 1. This application uses a flexible RRAM storage and computing array with a 1T1R unit structure. The indium gallium zinc oxide thin-film transistor IGZO TFT and memristor in the 1T1R unit structure are both flexible devices. Compared with the traditional 2T2R structure, this application reduces the storage and computing array area by half without affecting the calculation accuracy, further improving energy efficiency. Its flexible characteristics are more suitable for application in creating electronic skin.

[0036] 2. Peripheral circuits such as analog-to-digital converter ADC, popcount module, multiplier and subtractor can fully calculate the entire BNN algorithm and can input multiple bits, which improves the calculation precision and accuracy of the hardware circuit implementation of the BNN algorithm.

[0037] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0039] Figure 1 The overall circuit architecture diagram provided by the present invention;

[0040] Figure 2 A data flow chart for implementing the BNN algorithm provided by the present invention;

[0041] Figure 3 The circuit structure diagram of the flexible RRAM storage and computing array;

[0042] Figure 4 The computational coding table for the RRAM storage array based on the 1T1R structure. DETAILED DESCRIPTION

[0043] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0044] A specific embodiment of the present invention discloses an in-memory computing circuit based on BNN algorithm acceleration implementation, such as Figure 1 The circuit includes: a flexible RRAM storage array, a multiplication and accumulation module, a shifter, a comparator and a maximum value comparison output module;

[0045] According to the BNN network structure and input data, two areas are selected in the flexible RRAM storage array as a first algorithm network and a second algorithm network;

[0046] Input multiple rows of data of each bit into the first algorithm network and the multiplication-accumulation module, the first algorithm network outputs the accumulated current of each column of the storage and calculation unit to the multiplication-accumulation module, the multiplication-accumulation module obtains the multiplication-accumulation result of each column based on the multiple rows of data and the accumulated current of each column, and inputs the multiplication-accumulation result of each column to the shifter;

[0047] The shifter shifts the multiplication and accumulation results of each column by the corresponding number of bits according to the bit positions of each row of input data, and outputs the shifted result to the adder-subtractor in the multiplication and accumulation module. The adder-subtractor performs sum calculation on the shifted results of different bit positions of each column, and outputs the sum calculation result of each column to the comparator;

[0048] The comparator performs binarization processing on the sum calculation results of each column to obtain a binarization result, outputs it to the second algorithm network, calculates and outputs the accumulated current of each column again in the second algorithm network, and sends it to the maximum comparator output module;

[0049] The maximum comparator output module determines the maximum value of the accumulated current in each column, and the column where the maximum value is located is the classification result.

[0050] Specifically, according to the network structure of the BNN algorithm and the amount of input data, two areas are selected in the flexible RRAM storage array as the first algorithm network and the second algorithm network.

[0051] Specifically, the rows and columns of the first algorithm network are determined according to the number of rows of input data and the size of the hidden layer, and then the size of the first algorithm network is determined. The rows and columns of the second algorithm network are determined according to the size of the hidden layer and the number of output data of the output layer, and then the size of the second algorithm network is determined.

[0052] Furthermore, the flexible RRAM storage and computing array includes n×m storage and computing units;

[0053] The word lines WL of each row of storage and calculation units are connected in sequence, the source lines SL of each column of storage and calculation units are connected in sequence, and the bit lines BL of each column of storage and calculation units are connected in sequence, and the bit lines BL are used to output the calculation results of the storage and calculation units in this column;

[0054] Each of the storage and computing units has a 1T1R structure, including a transistor and a memristor.

[0055] Preferably, Figure 3 As shown, the flexible RRAM storage array has a size of 64*64 and a total capacity of 4Kb; it cooperates with peripheral circuits such as word line decoder, bit line decoder, source line decoder to form a multiplication and addition calculation unit. The word line decoder, bit line decoder and source line decoder decode the address signal given by the external FPGA to cooperate with it to complete the read and write operations.

[0056] Specifically, the transistor is an indium gallium zinc oxide thin film transistor IGZO TFT, and the memristor is RRAM;

[0057] The gate of the indium gallium zinc oxide thin film transistor IGZO TFT is connected to the word line WL of the storage unit, the drain is connected to the bit line BL of the storage unit, the source is connected to one end of the memristor RRAM, and the other end of the RRAM is connected to the source line SL of the storage unit.

[0058] Specifically, the working modes of the flexible RRAM storage and computing array include a write weight mode and an inference working mode.

[0059] Specifically, when the bit line BL or the source line SL is connected to the write voltage Vw, the working mode of the flexible RRAM storage array is the write weight mode, and when the bit line BL is connected to the read voltage Vr, the working mode of the flexible RRAM storage array is the inference working mode.

[0060] Specifically, the input voltages of the word lines, bit lines and source lines connected to the array are controlled by the peripheral FPGA, that is, the word line WL is connected to a high level or a low level to control the on and off of the transistor, thereby selecting whether the current storage unit is enabled; when the word line WL is connected to a high level, the transistor is turned on, that is, the storage unit is enabled; when the word line WL is connected to a low level, the transistor is not turned on, that is, the storage unit is not enabled.

[0061] Specifically, in the write weight working mode, the connection voltages of the word line WL, the bit line BL and the source line SL are controlled to change the resistance state of the memristor RRAM, thereby achieving weight writing.

[0062] Specifically, when the word line WL is connected to a high level, the bit line BL is connected to a write voltage Vw, and the source line SL is grounded, the conductance state of the memristor RRAM is a low resistance state LRS, and its weight is assigned to be β+1;

[0063] When the word line WL is connected to a high level, the bit line BL is grounded, and the source line SL is connected to a write voltage Vw, the conductance state of the memristor RRAM is a high resistance state HRS, and its weight is assigned to β-1;

[0064] When the word line WL is connected to a low level, it indicates that the memristor RRAM is not selected, and the resistance state of the memristor RRAM remains unchanged.

[0065] In the write weight mode, the high and low resistance states of the RRAM are set by controlling the write voltage Vw of the storage and calculation unit and the on and off of the transistor, thereby completing the deployment of the weight in the RRAM array; the output of the source line SL and the bit line BL is grounded or connected to the write voltage, and the word line WL controls the on and off of the transistor to control the weight writing of the RRAM in the storage and calculation unit. For example, when the word line WL <0> When the source line SL is connected to the ground terminal and the bit line BL is connected to the write voltage, the RRAM is programmed to a low resistance state.

[0066] Specifically, in the write weight mode and the inference working mode, because the memristor RRAM is a non-volatile device, after the write weight mode completes the resistance state setting of the RRAM, the resistance state of the memristor RRAM is fixed and will not change unless a new programming voltage is applied to the word line WL, the bit line BL and the source line SL.

[0067] Specifically, the write weight mode adopts a half voltage method, the write voltage Vw is greater than the threshold voltage of the memristor RRAM, and the half write voltage Vw is less than the threshold voltage of the memristor RRAM to ensure that the state of the unselected memristor RRAM remains unchanged.

[0068] Specifically, in the inference working mode, the bit line BL is connected to the read voltage Vr and the source line SL is grounded. The input of the storage unit is determined according to whether the bit line WL is connected to a high level or a low level. The current value output by the storage unit is taken as the product of the input of the storage unit and the weight of the memristor RRAM. The currents output by each storage unit in each column of storage units are combined in the bit line corresponding to the storage unit in the column, so that the bit line output is the sum of the products of each unit in the column of storage units.

[0069] Specifically, Figure 4 As shown in FIG. 1 , when the word line WL is connected to a high level and the resistance state of the memristor RRAM is a high resistance state HRS, the input of the storage unit is 1 and the output is a low current I L , low current I L The value of is β-1;

[0070] When the word line WL is connected to a high level and the resistance state of the memristor RRAM is a low resistance state LRS, the input of the storage unit is 1 and the output current I H , high current I H The value of is β+1;

[0071] When the word line WL is connected to a low level, the indium gallium zinc oxide thin film transistor IGZO TFT is disconnected, the storage unit is not connected, the input of the storage unit is 0, and the output current of the storage unit is 0.

[0072] In the inference working mode, the entire RRAM array can realize the calculation function of multiplication and addition; the on and off of the TFT transistor is controlled by the word line WL to realize the input: when the word line WL is connected to a high level, the TFT transistor is turned on, that is, the storage unit is selected, indicating that the input is 1; when the word line WL is connected to a low level, the transistor is turned off, that is, the unit is not connected, indicating that the input is 0. The source line SL is connected to the ground, and the bit line BL is connected to the read voltage Vr to obtain the corresponding accumulated current, which is the multiplication and addition result. RRAM represents the weight distribution through the high and low resistance states after programming. When a bit line BL of the RRAM array is selected, a current I will be generated according to different high resistance states or low resistance states. H or I L For example, when the word line WL <0> When the RRAM is in a high impedance state, the storage unit generates a reference current I L , which means that the calculation result of the unit is β-1 (β>1), and so on for the calculation of a whole column of storage and calculation units. The accumulated current obtained at the output bit line BL is the multiplication and addition result of a column of storage and calculation units. In this way, only one TFT transistor and one RRAM can be used as a storage and calculation unit to form a storage and calculation array.

[0073] Specifically, the β>1, the present application is equivalent to a deformed binary calculation, the input of the word line WL is to control whether the transistor is turned on or not, and the high resistance state and low resistance state of the memristor RRAM are determined by controlling the voltage connected to the bit line BL and the source line SL in the write weight mode. Since the high resistance state and the low resistance state are actually positive values, the low resistance state and the high resistance state are encoded as β+1 and β-1 respectively. In the inference working mode, the output also changes accordingly, and the output is also a binary high current and a low current.

[0074] Furthermore, the multiplication-accumulation module further includes: a popcount module and a multiplier;

[0075] The popcount module receives multiple lines of input and obtains the number of 1s in the multiple lines of input, and outputs the number of 1s to the multiplier;

[0076] The multiplier multiplies the number of 1s by a fixed value β and sends the product result to the adder-subtractor;

[0077] The adder-subtractor subtracts the product result from the digital signal to obtain a matrix multiplication and addition result, and sends the result to the shifter;

[0078] Specifically, the function of the multiplier is to multiply the input by a fixed β value to solve the weight setting problem, where the β value is determined by the performance of the RRAM storage array.

[0079] Exemplarily, when the matrix multiplication is [1,0,1,0,1]*[+1,+1,-1,-1,-1], that is, multiple rows of input are [1,0,1,0,1], the weight of a column of the RRAM storage array is [+1,+1,-1,-1,-1], the multiplication and addition result is -1, and the corresponding multiple rows of input of the RRAM storage array are 10101. The resistance states of the input corresponding RRAM units are β+1, β+1, β-1, β-1, and β-1, respectively. Then, the accumulated current output by the RRAM storage array in this column is input to the multiplexer, and the multiplexer selects the accumulated current of the column of the storage array to be output to the analog-to-digital converter ADC. After analog-to-digital conversion, a current digital signal of 3β-1 is output to the adder-subtractor. The popcount module output is 3, and the multiplier output is 3β. Then the adder-subtractor performs a subtraction operation and outputs -1. It can be seen that this scheme is completely consistent with the original multiplication and addition result.

[0080] Furthermore, the circuit further comprises: a first column multiplexer MUX, a first analog-to-digital converter ADC;

[0081] The first column multiplexer MUX sequentially selects each column storage and calculation unit of the first algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the first analog-to-digital converter ADC;

[0082] The first analog-to-digital converter ADC converts the accumulated current output by the first algorithm network into a digital signal and sends it to the adder-subtractor.

[0083] Furthermore, the adder-subtractor implements addition operation or subtraction operation through the control signal sent by the FPGA.

[0084] Specifically, the adder-subtractor module adopts a carry-lookahead adder structure, which can perform addition and subtraction of two signed numbers. The addition or subtraction is controlled by the control signal given by the FPGA. Because the RRAM storage array weight encoding adopts all-positive number encoding during hardware implementation, it is actually greater than the weight value in the actual algorithm. Therefore, the multiplication and addition result from the RRAM array is subtracted from the output result of the popcount module to make a correction, and subtraction is performed at this time; this implementation method is to split the input into each bit input, so the results of each bit need to be added to get the calculation result of the complete input, and addition is performed at this time.

[0085] Furthermore, the circuit further comprises: a first data buffer module, a second data buffer module and a third data buffer module; all of which are used for temporary storage and integration of data;

[0086] The first data buffer module receives the digital signal output by the first analog-to-digital converter ADC, buffers it, and then outputs it to the adder-subtractor;

[0087] The second data buffer module receives the shift result output by the shifter, buffers it, and then outputs it to the adder-subtractor;

[0088] The third data buffer module receives the binarization result output by the comparator, buffers it, and then outputs it to the second algorithm network.

[0089] Furthermore, the circuit further comprises: a second column multiplexer MUX, a second analog-to-digital converter ADC;

[0090] The second column multiplexer MUX sequentially selects each column storage and calculation unit of the second algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the second analog-to-digital converter ADC;

[0091] The second analog-to-digital converter ADC converts the accumulated current output by the second algorithm network into a digital signal and sends it to the maximum value comparison output module.

[0092] Further, according to the bit positions of the input multiple rows of data, the multiplication and addition result is shifted to the left by the corresponding number of bits in the shifter; for the multiplication and accumulation result corresponding to the i-th multiple rows of data, the multiplication and accumulation result is shifted to the left by i bits in the shifter;

[0093] Among them, 0≤i≤7.

[0094] Specifically, the shifter is used to perform data shift of a signed number instead of a multiplication operation.

[0095] Furthermore, a threshold is preset in the comparator. When the sum calculation result is greater than the threshold, the comparator outputs 1, and when the sum calculation result is less than the threshold, the comparator outputs 0.

[0096] Specifically, the comparator implements normalization and binarization of the data for use by the next layer of the network.

[0097] The maximum value comparison output module can compare the maximum values ​​of the four output data of the last layer of the network and use them as the output of the classification result.

[0098] Exemplarily, if the hidden layer size in the BNN network is 14, the input data is 35 rows of 8-bit data, and the BNN network output is 4 data, the required algorithm network is two fully connected layers of 35*14 and 14*4, which are mapped to the circuit implementation as two RRAM arrays of 35*14 and 14*4. Therefore, according to the above BNN network structure and input data, a 35*14 area is selected in the flexible RRAM storage array as the first algorithm network, and a 14*4 area is selected as the second algorithm network; the first input of the flexible RRAM storage array is 35 rows*8bit input, and the weights are reused. Each of the 35 bits is written into the first algorithm network through the word line decoder eight times, that is, the input data is each bit of data of multiple rows (35 rows), and the multiplication and addition result is obtained after calculation through the first algorithm network, and the first column multiplexer MUX is connected to the first analog-to-digital converter ADC to obtain the digital-to-analog conversion result of the accumulated current of each column, and the analog-to-digital conversion result is stored in the data cache module 1, and the output of the data buffer module 1 is connected to the adder-subtractor. At the same time, when each bit of the 35 rows is input, it is connected to the popcount module, and the popcount module obtains the number of 1s in each bit of the multiple rows of input. The output of the popcount module is connected to the multiplier, in which the number of 1s is multiplied by a fixed β value, and the output of the multiplier is connected to the adder-subtractor. The FPGA first sets the adder-subtractor to subtraction, and the output of the data buffer module 1 is subtracted from the output of the multiplier to obtain the multiplication and accumulation results of each bit of the 35 rows. The multiplication and accumulation results of each column corresponding to each bit are input into the shifter, and the corresponding shift of each bit is performed, such as the 0th bit is shifted to the left by 0 bits, and the 7th bit is shifted to the left by 7 bits, and the shift results of each bit of each column are stored in the data buffer module 2. For example, if it is the 0th bit, the shifter input data is 00000010, which is shifted by 0 bits and becomes 00000010 after shifting; if it is the 3rd bit, the shifter input data is 00000010, which is shifted by 3 bits and becomes 00010000 after shifting. After the above operations are performed on all 8 bits, the shift results are stored in the data buffer module 2, and the shift results in the data buffer module 2 are input to the adder-subtractor. The FPGA sets the adder-subtractor to addition, and adds all the bit shift results of each column stored in the data buffer module 2 to obtain the sum of each column, and outputs the sum to the comparator.A normalization operation is performed in the comparator, that is, the sum of the addition results is compared with the set threshold value. When the addition result is greater than the threshold value, 1 is output, and when the addition result is less than the threshold value, 0 is output; that is, when the input addition result is 3 and the threshold value is 1, the output is 1; when the input addition result is 0 and the threshold value is 1, the output is 0; in this way, the binary input of the next layer is obtained, that is, the input data of the second algorithm network; the last layer is not binarized, and the classification result is directly output through the maximum value comparison output module. Specifically, if the accumulated current value of the first column is the largest, 0 is output, and so on, if the data in the fourth column is the largest, 3 is output.

[0099] Compared with the prior art, this embodiment provides an in-memory computing circuit based on the accelerated implementation of the BNN algorithm. This application uses a flexible RRAM memory computing array with a 1T1R unit structure. The indium gallium zinc oxide thin film transistor IGZO TFT and memristor in the 1T1R unit structure are both flexible structures. Compared with the traditional 2T2R structure, this application reduces the memory computing array area by half without affecting the calculation accuracy, further improving energy efficiency, and its flexible characteristics are more suitable for application in the creation of electronic skin. The design of peripheral circuits such as analog-to-digital converter ADC, popcount module, multiplier and subtractor realizes a complete BNN algorithm, and can realize multi-bit input, further improving the precision and accuracy of the hardware circuit implementation of the algorithm.

[0100] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0101] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An in-memory computing circuit based on BNN algorithm acceleration, characterized in that: The circuit includes: a flexible RRAM storage array, a multiplication and accumulation module, a shifter, a comparator and a maximum value comparison output module; According to the BNN network structure and input data, two areas are selected in the flexible RRAM storage array as a first algorithm network and a second algorithm network; Input multiple rows of data of each bit into the first algorithm network and the multiplication-accumulation module, the first algorithm network outputs the accumulated current of each column of the storage and calculation unit to the multiplication-accumulation module, the multiplication-accumulation module obtains the multiplication-accumulation result of each column based on the multiple rows of data and the accumulated current of each column, and inputs the multiplication-accumulation result of each column to the shifter; The shifter shifts the multiplication and accumulation results of each column by the corresponding number of bits according to the bit positions of each row of input data, and outputs the shifted result to the adder-subtractor in the multiplication and accumulation module. The adder-subtractor performs sum calculation on the shifted results of different bit positions of each column, and outputs the sum calculation result of each column to the comparator; The comparator performs binarization processing on the sum calculation results of each column to obtain a binarization result, outputs it to the second algorithm network, calculates and outputs the accumulated current of each column again in the second algorithm network, and sends it to the maximum comparator output module; The maximum comparator output module determines the maximum value of the accumulated current in each column, and the column where the maximum value is located is the classification result.

2. The in-memory computing circuit based on BNN algorithm acceleration according to claim 1, characterized in that: The multiplication-accumulation module further comprises: a popcount module and a multiplier; The popcount module receives multiple lines of input and obtains the number of 1s in the multiple lines of input, and outputs the number of 1s to the multiplier; The multiplier multiplies the number of 1s by a fixed value β and sends the product result to the adder-subtractor; The adder-subtractor subtracts the product result from the digital signal to obtain the matrix multiplication and addition result, and sends it to the shifter.

3. The in-memory computing circuit based on BNN algorithm acceleration implementation according to claim 2, characterized in that: The circuit further comprises: a first column multiplexer MUX and a first analog-to-digital converter ADC; The first column multiplexer MUX sequentially selects each column storage and calculation unit of the first algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the first analog-to-digital converter ADC; The first analog-to-digital converter ADC converts the accumulated current output by the first algorithm network into a digital signal and sends it to the adder-subtractor.

4. The in-memory computing circuit based on BNN algorithm acceleration according to claim 3, characterized in that: The circuit further comprises: a second column multiplexer MUX and a second analog-to-digital converter ADC; The second column multiplexer MUX sequentially selects each column storage and calculation unit of the second algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the second analog-to-digital converter ADC; The second analog-to-digital converter ADC converts the accumulated current output by the second algorithm network into a digital signal and sends it to the maximum comparator output module.

5. The in-memory computing circuit based on BNN algorithm acceleration implementation according to claim 4, characterized in that: The circuit further comprises: a first data buffer module, a second data buffer module and a third data buffer module; all of which are used for temporary storage and integration of data; The first data buffer module receives the digital signal output by the first analog-to-digital converter ADC, buffers it, and then outputs it to the adder-subtractor; The second data buffer module receives the shift result output by the shifter, buffers it, and then outputs it to the adder-subtractor; The third data buffer module receives the binarization result output by the comparator, buffers it, and then outputs it to the second algorithm network.

6. The in-memory computing circuit based on BNN algorithm acceleration implementation according to claim 5, characterized in that: The adder-subtractor realizes addition operation or subtraction operation through the control signal sent by the FPGA.

7. The in-memory computing circuit based on BNN algorithm acceleration according to claim 6, characterized in that: According to the bit positions of the input multiple rows of data, the multiplication and addition result is shifted to the left by the corresponding number of bits in the shifter; for the multiplication and accumulation result corresponding to the i-th multiple rows of data, the multiplication and accumulation result is shifted to the left by i bits in the shifter; wherein 0≤i≤7.

8. The in-memory computing circuit based on BNN algorithm acceleration implementation according to claim 7, characterized in that: A threshold is preset in the comparator. When the sum calculation result is greater than the threshold, the comparator outputs 1, and when the sum calculation result is less than the threshold, the comparator outputs 0.

9. The in-memory computing circuit based on BNN algorithm acceleration according to claim 1, characterized in that: The flexible RRAM storage and computing array includes n×m storage and computing units; The word lines WL of each row of storage and calculation units are connected in sequence, the source lines SL of each column of storage and calculation units are connected in sequence, and the bit lines BL of each column of storage and calculation units are connected in sequence, and the bit lines BL are used to output the calculation results of the storage and calculation units in this column; Each of the storage and computing units has a 1T1R structure, including a transistor and a memristor.

10. The in-memory computing circuit based on BNN algorithm acceleration implementation according to claim 9, characterized in that: The transistor is an indium gallium zinc oxide thin film transistor IGZO TFT, and the memristor is RRAM; The gate of the indium gallium zinc oxide thin film transistor IGZO TFT is connected to the word line WL of the storage unit, the drain is connected to the bit line BL of the storage unit, the source is connected to one end of the memristor RRAM, and the other end of the RRAM is connected to the source line SL of the storage unit.

Images