Chip for in-memory calculation of spiking neural network

Abstract

The invention relates to the field of artificial intelligence and the field of integrated circuits, in particular to a chip for in-memory calculation of a spiking neural network, which comprises a synaptic weight storage array for storing synaptic weights; the threshold storage array is used for storing a neuron threshold voltage; the membrane potential storage array is used for storing a neuron membrane potential, a bit line of the membrane potential storage array is connected with a bit line of the synaptic weight storage array, and logic calculation results of the synaptic weight and the neuron membrane potential are directly generated on the connected bit lines; the membrane potential accumulation unit obtains a membrane potential accumulation result through processing of an addition logic circuit; the comparison unit is used for opening read sub-lines of the membrane potential storage array and the threshold voltage storage array and multiplexing an addition logic circuit to realize subtraction calculation of a membrane potential accumulation result and a threshold voltage; and the transmitting unit is used for judging whether the neurons output pulses or not and resetting the membrane potential. The method can achieve the push-training integration of in-memory calculation, remarkably improves the SNN operation energy efficiency, and reduces the power consumption.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and integrated circuits, and specifically to a chip for in-memory computation of spiking neural networks. Background Technology

[0002] Vision is a crucial way for humans to acquire information from the outside world, and its applications are becoming increasingly widespread in fields such as autonomous driving, intelligent manufacturing, and medical diagnosis. Traditional vision processing systems face bottlenecks such as the "memory wall" and the "power wall," making it difficult to meet the energy efficiency and cost requirements of edge computing. To address these challenges, researchers have proposed a vision chip that integrates image sensors, processors, and memory onto a single chip. This tight coupling achieves in-memory computing, significantly reducing energy consumption and latency in data transfer. This innovation breaks through the limitations of the traditional von Neumann architecture, significantly improving the system's energy efficiency and real-time processing capabilities, bringing profound changes to the image vision industry.

[0003] However, the co-location and coupling of storage and computing units in existing processors, which prevents true integration, leads to high-energy-consuming data handling. Existing non-volatile memories increase process complexity and limit the application of high-recognition-rate deep spiking neural networks (SNNs). Summary of the Invention

[0004] The purpose of this invention is to provide a chip for in-memory computation of spiking neural networks, which can realize the integration of in-memory computation and training, significantly improving the energy efficiency of SNN operations and reducing power consumption.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A chip for in-memory computation of a spiking neural network, comprising: Synaptic weight storage array, used to store the synaptic weights corresponding to each neuron; Threshold storage array, used to store neuron threshold voltages; A membrane potential storage array is used to store neuronal membrane potentials. The bit lines of the membrane potential storage array are connected to the bit lines of the synaptic weight storage array. The logical calculation results of synaptic weights and neuronal membrane potentials are directly generated on the connected bit lines, and the logical calculation results are input into the membrane potential accumulation unit in parallel. The membrane potential accumulation unit is used to receive the logic calculation result, process it through the addition logic circuit to obtain the membrane potential accumulation result, and write the membrane potential accumulation result back to the membrane potential storage array for storage. The comparison unit is used to simultaneously open the read sub-lines of the membrane potential storage array and the threshold voltage storage array after obtaining the membrane potential accumulation result, and to use the addition logic circuit to realize the subtraction calculation between the membrane potential accumulation result and the threshold voltage. The transmitting unit determines whether a neuron outputs a pulse and resets the membrane potential based on the result of subtracting the accumulated membrane potential from the threshold voltage.

[0006] Furthermore, each first storage unit in the synaptic weight storage array is connected to an inverter to form a single-bit multiplication unit. The power supply terminal of the inverter is used as the error input port for the single-bit multiplication, and multiplied with the synaptic weights stored in the first storage unit to output the single-bit product result. The error is input bit by bit in a serial manner into a storage array composed of single-bit multiplication units. In the storage array, each single-bit multiplication unit works synchronously, and multiple single-bit multiplication units output multiple sets of single-bit product results in parallel, realizing bit-by-bit multiplication of multi-bit error and multi-bit weight. It also includes an adder tree circuit, which receives all single-bit product results and accumulates all received single-bit product results into a multi-bit accumulation result.

[0007] Furthermore, it also includes an attenuation unit, which reads back the accumulated membrane potential result stored in the membrane potential storage array, achieves attenuation through a shift operation, and writes the result back to the membrane potential storage array to complete the calculation of the current time step.

[0008] Furthermore, a single first storage cell in the synaptic weight storage array is a 6T-SRAM, a single second storage cell in the threshold storage array is a 10T-SRAM, and a single third storage cell in the membrane potential storage array is a 10T-SRAM. The 10T-SRAM is based on a standard 6T-SRAM cell with four additional transistors. These four additional transistors are used to implement read and write operations for storing data Q, as well as read and write operations for storing data QNOT.

[0009] Furthermore, it also includes a sign bit extension unit, which is coupled to the precharge circuit of the membrane potential storage array.

[0010] The present invention has the following unexpected beneficial effects: 1. In the chip for in-memory computation of spiking neural networks described in this invention, the bit lines of the membrane potential storage array are directly connected to the bit lines of the synaptic weight storage array. This eliminates the need to read synaptic weights from the weight storage array through a sensitive amplifier, and also eliminates the need to transfer membrane potentials from the membrane potential storage array to an external computing unit. Instead, the logical calculation results of synaptic weights and membrane potentials are directly generated on the connected bit lines. This completely avoids the energy consumption and latency caused by frequent data transfer between storage and computing units, reduces read / write energy consumption and processing latency, and significantly improves the energy efficiency of spiking neural network computation.

[0011] 2. The chip for in-memory computation of a spiking neural network described in this invention significantly simplifies the hardware structure and reduces chip area and control logic complexity through functional module reuse design. The comparison unit eliminates the need for a separate threshold comparison circuit; instead, it directly performs the subtraction calculation between the accumulated membrane potential and the threshold voltage by simultaneously opening the read word lines of both the membrane potential storage array and the threshold voltage storage array, reusing the addition logic circuit of the membrane potential accumulation unit. Essentially, it is a reverse application of addition logic. The emission unit directly determines whether a neuron outputs a pulse and whether the membrane potential is reset based on the result of the membrane potential-threshold subtraction calculation (determined by the sign bit), eliminating the need for a complex independent emission judgment circuit. This reduces the design of additional hardware modules, lowers chip area, simplifies control timing, and further reduces circuit power consumption.

[0012] 3. The hardware architecture of the chip for in-memory computation of spiking neural networks described in this invention not only supports forward inference but also reserves a compatible foundation for on-chip reverse learning. Through the collaboration of the weight storage array, membrane potential storage array, membrane potential accumulation, comparison unit, and emitter unit, it efficiently supports sparse and irregular forward inference operations of spiking neural networks, such as event-driven membrane potential accumulation and pulse determination. By connecting a single first storage unit in the synaptic weight storage array to an inverter to form a single-bit multiplication unit, and using the power supply terminal of the inverter as the error input port of the single-bit multiplication, multiplying it with the synaptic weights stored in the first storage unit, and outputting the single-bit product result, the accumulation of weights and errors can be calculated to achieve error backpropagation. There is no need to design two separate hardware sets for inference and training; a single architecture is compatible with integrated inference and training, improving the chip's adaptability to complex SNN tasks. Attached Figure Description

[0013] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention.

[0014] Figure 1 A schematic diagram of the structure of a chip for in-memory computation of a spiking neural network, as described in an embodiment of the present invention, is shown.

[0015] Figure 2 A schematic diagram of the chip architecture for in-memory computation of a spiking neural network, as described in an embodiment of the present invention, is shown.

[0016] Figure 3 The diagram illustrates a sparse, irregular forward inference in-memory computing architecture corresponding to a single neuron in an embodiment of the present invention.

[0017] Figure 4A schematic diagram of the architecture of the 10T-SRAM according to an embodiment of the present invention is shown.

[0018] Figure 5 This diagram illustrates the architecture of a single-bit multiplication circuit based on 6T-SRAM combined with an inverter array, as described in an embodiment of the present invention.

[0019] Figure 6 This diagram illustrates the architecture of multi-bit multiplication in-memory computation based on 6T-SRAM combined with an inverter array, as described in an embodiment of the present invention. Detailed Implementation

[0020] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0021] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0022] In one embodiment, the present invention provides a chip for in-memory computation of a spiking neural network, see [link to relevant documentation]. Figure 1 As shown, the chip 1 includes: Synaptic weight storage array 11 is used to store the synaptic weights corresponding to each neuron; Threshold storage array 12 is used to store neuron threshold voltages; The membrane potential storage array 13 is used to store neuronal membrane potentials. The bit lines of the membrane potential storage array 13 are connected to the bit lines of the synaptic weight storage array 11. The logical calculation results of synaptic weights and neuronal membrane potentials are directly generated on the connected bit lines, and the logical calculation results are input into the membrane potential accumulation unit 14 in parallel. The membrane potential accumulation unit 14 is used to receive the logic calculation result, process it through the addition logic circuit to obtain the membrane potential accumulation result, and write the membrane potential accumulation result back to the membrane potential storage array 13 for storage. The comparison unit 15 is used to simultaneously open the read sub-lines of the membrane potential storage array 13 and the threshold voltage storage array 12 after obtaining the membrane potential accumulation result, and to use the addition logic circuit to realize the subtraction calculation between the membrane potential accumulation result and the threshold voltage. The transmitting unit 16 determines whether the neuron outputs a pulse and resets the membrane potential based on the result of subtracting the accumulated membrane potential from the threshold voltage.

[0023] As a preferred embodiment of the present invention, see Figure 2 and Figure 5 As shown, a single first storage unit 111 in the synaptic weight storage array 11 is connected to an inverter to form a single-bit multiplication unit. The power supply terminal of the inverter is used as the error input port of the single-bit multiplication, and multiplied with the synaptic weight stored in the first storage unit 111 to output the single-bit product result. The error is input bit by bit in a serial manner into a storage array composed of single-bit multiplication units. In the storage array, each single-bit multiplication unit works synchronously, and multiple single-bit multiplication units output multiple sets of single-bit product results in parallel, realizing bit-by-bit multiplication of multi-bit error and multi-bit weight. It also includes an adder tree circuit, which receives all single-bit product results and accumulates all received single-bit product results into a multi-bit accumulation result.

[0024] For example, to achieve on-chip learning in a spiking neural network, it is necessary to calculate the product and summation of synaptic weights and errors to obtain the error result of the membrane potential of the current layer of neurons, thereby achieving error backpropagation. Therefore, we further extend the in-memory computing circuit based on SRAM for forward inference computation to implement the product operation of error and weights. First, we propose a standard 6T-SRAM combined with an inverter to implement single-bit multiplication, such as... Figure 5 The power supply terminal of the inverter is used as the second input port of the multiplication circuit, which is connected to the standard 6T-SRAM stored value Q to realize the function of multiplying the input and stored values.

[0025] The in-memory calculation method for the product of synaptic error and weight is as follows: Figure 6 As shown, the synaptic weights are 8 bits, and the error is also assumed to be 8 bits in precision. The product of the error and the synaptic weights is calculated using pure digital means. This can be achieved by serially inputting the error into the memory array, and then sending the product result into the adder circuit for accumulation. The adder tree circuit uses 14T-28T full adders with alternating data paths to form the adder tree circuit, which can perform a large number of parallel addition operations without requiring additional registers to store the input and partial sum.

[0026] Specifically, for the multiplication and accumulation of 8-bit synaptic weights and 8-bit errors, a process of serial input + parallel operation + addition tree accumulation is adopted: 1) Error serial input storage array, with an error precision of 8-bit, is input bit by bit in a serial manner into the storage array composed of single-bit multiplication units; the 8 bits of the synaptic weight have been pre-stored in the corresponding positions of the 6T-SRAM array, with each bit corresponding to a single-bit multiplication unit.

[0027] 2) Parallel execution of single-bit multiplication: In the memory array, each single-bit multiplication unit works synchronously. The i-th bit of the error (IN_i) and the i-th bit of the weight (Q_i) are multiplied within the single-bit multiplication unit. The eight single-bit multiplication units output eight sets of single-bit product results in parallel, realizing the bitwise multiplication of 8-bit error × 8-bit synaptic weight.

[0028] 3) The adder tree circuit completes the accumulation, sending all single-bit product results into the adder tree circuit. This circuit supports a large number of parallel addition operations, quickly accumulating multiple sets of single-bit product results into a complete multi-bit result, i.e., the final accumulated sum of error × synaptic weights. This method eliminates the need for additional registers to store input data or intermediate sums, simplifying the hardware structure and improving computational efficiency.

[0029] As a preferred embodiment of the present invention, see Figure 2 and Figure 3 As shown, the chip used for in-memory computation of the spiking neural network also includes an attenuation unit 17, which is used to read and write back the accumulated membrane potential result stored in the membrane potential storage array 12, achieve attenuation through shift operation, and write the result back to the membrane potential storage array to complete the calculation of the current time step.

[0030] As a preferred embodiment of the present invention, see Figure 3 As shown, a single first storage cell 111 in the synaptic weight storage array 11 is a 6T-SRAM, a single second storage cell 121 in the threshold storage array 12 is a 10T-SRAM, and a single third storage cell 131 in the membrane potential storage array 13 is a 10T-SRAM.

[0031] In "6T-SRAM", "T" stands for transistor, and 6T-SRAM refers to a standard SRAM storage cell composed of 6 transistors. SRAM is an abbreviation for Static Random-Access Memory, a type of semiconductor memory and a core component of the cache in computer systems.

[0032] For example, see Figure 5As shown, the 6T-SRAM memory cell consists of a cross-coupled inverter pair composed of transistors N1, N2, N3, N4, N5, and N6, used to stably store 1 bit of data Q, which can be understood as 1 bit of synaptic weight. Transistors N7 and N8 form an inverter. The input IN is a single bit of error, and Q is a single bit of synaptic weight; the output is a single bit of error multiplied by synaptic weight; OUT=1 only when IN=1 and Q=1; otherwise, OUT=0, which perfectly conforms to the mathematical logic of single-bit multiplication.

[0033] Specifically, based on the existing SRAM in-memory computing circuit (originally used for forward inference), an extension is made to add the function of multiplying error and synaptic weight on the same architecture, without the need to design additional independent multiplication hardware, thus maintaining the low power consumption characteristics of in-memory computing.

[0034] The 10T-SRAM memory cell is an improved SRAM memory cell composed of 10 transistors. It is an enhanced structure formed by adding 4 additional transistors to the standard 6T-SRAM cell. These four additional transistors respectively implement read and write operations for stored data Q, as well as read and write operations for stored data QNOT. This makes the data stored in the SRAM more stable, unaffected by data reads, avoiding interference from read operations and ensuring data accuracy during calculations.

[0035] For example, see Figure 4 As shown, the 10T-SRAM memory cell includes a memory core, a write operation branch, and a read operation branch. The memory core consists of two cross-coupled inverters. M1 and M2 form one set of inverters, and M3 and M4 form another set of inverters. They are cross-coupled to form a bistable circuit, which can stably store data Q and the inverted data Qnot.

[0036] The write operation branch includes M5, M6, write word line, write bit line, and write bit line NOT. When data needs to be written, the write word line is activated, and the levels of the write bit line and write bit line NOT (representing the data to be written) are transmitted through transistors M5 and M6 to write the data into the intermediate bistable memory core, thus completing the write operation.

[0037] The read operation branches include M7, M8, M9, and M 10 The read word line, read bit line, and read bit line NOT are used to read data. When data needs to be read, the read word line is activated, and the state Q of the intermediate storage core is controlled by M7, M8, M9, and M... 10 Transistors transmit data to the read line to complete the data reading; and because the read and write branches are separated, the read operation will not interfere with the data stored in the intermediate bistable core, thus ensuring the stability of the storage.

[0038] As a preferred embodiment of the present invention, see Figure 2 and Figure 3 As shown, the chip for in-memory computation of the spiking neural network also includes a sign bit extension unit 18, which is coupled to the precharge circuit of the membrane potential storage array 12.

[0039] For example, see Figure 3 The diagram shows a sparse, irregular forward inference in-memory computing architecture corresponding to a single neuron. When a pulse arrives, the presynaptic neuron corresponding to the current pulse is analyzed, and its corresponding weight is added to the membrane potential of the postsynaptic neuron. The forward inference computing circuit mainly consists of a synaptic weight storage array 11, a threshold storage array 12, a membrane potential storage array 13, and some logic computing circuits. Among them, the synaptic weight storage array 11 includes 784×6 first storage units, and the first storage unit is 6T-SRAM. "784" represents the number of synapses corresponding to a single neuron, and "6" represents the precision of a single synaptic weight, i.e., 6 bits.

[0040] The 6T-SRAM array stores 6-bit synaptic weights, and the bit lines are directly connected to the lower six read bit lines of the 10T SRAM; The 10T-SRAM array (two rows of 12-bit) adds four transistors to the 6T-SRAM to separate read and write operations and improve data stability. One row is used to store the opposite of the neuron threshold, and the other row is used to store the neuron membrane potential. Couple the sign bit extension unit 18 to the high six bits of the precharge circuitry of the 10T-SRAM array to reduce additional circuit overhead.

[0041] The forward inference computation process includes (single time step): 1) Accumulation of membrane potential.

[0042] Step 1: Precharge the bit lines of the 6T-SRAM array and the 10T-SRAM array; Step 2: Simultaneously open the word lines of the synaptic weight row (6T-SRAM) and membrane potential row (10T-SRAM) corresponding to the current pulse; Step 3: The bit line directly generates the NOR and AND logic results of the synaptic weights and neuronal membrane potentials, and sends them to the membrane potential accumulation unit; Step 4: The membrane potential accumulation unit outputs the accumulated result of the membrane potential and the current synaptic weight, and writes it directly back to the membrane potential row of the 10T-SRAM; Weights can be read out directly at the bit line of the storage array without the need for a sensitive amplifier, eliminating data transfer.

[0043] 2) Comparison of membrane potential and threshold Step 1: After accumulating the membrane potentials of the neuron during forward inference, simultaneously open the read lines of the membrane potential line and the line opposite to the threshold line of the 10T-SRAM; Step 2: Reuse the membrane potential accumulation unit to realize the accumulation of membrane potential and threshold, that is, membrane potential + the negative of threshold; Step 3: Send the sign bit of the calculation result to the emission determination unit to determine whether the neuron outputs a pulse and whether the membrane potential needs to be reset; Key advantages: No additional threshold comparison circuits and complex emission judgment circuits are required, simplifying the architecture.

[0044] 3) Membrane potential decay Step 1: The membrane potential decay unit 17 reads the membrane potential from the 10T-SRAM; Step 2: Achieve membrane potential decay through a shifting operation; Step 3: Write the decayed result back to the membrane potential line of the 10T-SRAM to complete the calculation of the current time step.

[0045] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A chip for in-memory computation of a spiking neural network, characterized in that, include: Synaptic weight storage array, used to store the synaptic weights corresponding to each neuron; Threshold storage array, used to store neuron threshold voltages; A membrane potential storage array is used to store neuronal membrane potentials. The bit lines of the membrane potential storage array are connected to the bit lines of the synaptic weight storage array. The logical calculation results of synaptic weights and neuronal membrane potentials are directly generated on the connected bit lines, and the logical calculation results are input into the membrane potential accumulation unit in parallel. The membrane potential accumulation unit is used to receive the logic calculation result, process it through the addition logic circuit to obtain the membrane potential accumulation result, and write the membrane potential accumulation result back to the membrane potential storage array for storage. The comparison unit is used to simultaneously open the read sub-lines of the membrane potential storage array and the threshold voltage storage array after obtaining the membrane potential accumulation result, and to use the addition logic circuit to realize the subtraction calculation between the membrane potential accumulation result and the threshold voltage. The transmitting unit determines whether a neuron outputs a pulse and resets the membrane potential based on the result of subtracting the accumulated membrane potential from the threshold voltage.

2. The chip for in-memory computation of a spiking neural network according to claim 1, characterized in that: A single first storage unit in the synaptic weight storage array is connected to an inverter to form a single-bit multiplication unit. The power supply terminal of the inverter is used as the error input port of the single-bit multiplication. The unit is multiplied with the synaptic weights stored in the first storage unit to output the single-bit product result. The error is input bit by bit in a serial manner into a storage array composed of single-bit multiplication units. In the storage array, each single-bit multiplication unit works synchronously, and multiple single-bit multiplication units output multiple sets of single-bit product results in parallel, realizing bit-by-bit multiplication of multi-bit error and multi-bit weight. It also includes an adder tree circuit, which receives all single-bit product results and accumulates all received single-bit product results into a multi-bit accumulation result.

3. The chip for in-memory computation of a spiking neural network according to claim 1, characterized in that: It also includes an attenuation unit, which reads and writes back the accumulated membrane potential result stored in the membrane potential storage array, attenuates it through a shift operation, and writes the result back to the membrane potential storage array to complete the calculation of the current time step.

4. The chip for in-memory computation of a spiking neural network according to claim 1, characterized in that: The first storage cell in the synaptic weight storage array is 6T-SRAM, the second storage cell in the threshold storage array is 10T-SRAM, and the third storage cell in the membrane potential storage array is 10T-SRAM. The 10T-SRAM is based on a standard 6T-SRAM cell with four additional transistors. These four additional transistors are used to implement read and write operations for storing data Q, as well as read and write operations for storing data QNOT.

5. The chip for in-memory computation of a spiking neural network according to claim 1, characterized in that: It also includes a sign bit extension unit, which is coupled to the precharge circuit of the membrane potential storage array.

Images