In-memory computing device, in-memory computing method, processing device, tile module and accelerator

Abstract

The application discloses a memory-computing integrated device, a memory-computing method, a processing device, a tile module and an accelerator, and relates to the technical field of electronic circuits, and comprises: each memory-computing integrated array realizes parallel computation within the array, the memory-computing integrated array can simultaneously perform multiplication operation on single-bit input pulses at each moment in the current cycle buffer and corresponding weights, and synchronously generate membrane potential increment values at each moment; and the parallel accumulation of the increment values by a summation tree forms an efficient computing link of "parallel multiplication + parallel accumulation". Compared with the delay caused by the step-by-step waiting of the row-by-row serial processing of a computing task, the parallel architecture greatly shortens the processing time of the "multiplication-accumulation" whole process in a single cycle under the premise of ensuring high computing precision, further improves the computing efficiency under unit energy consumption due to the reduction of repeated data scheduling and state switching overhead in serial computation, and finally realizes the double optimization of computing delay reduction and energy efficiency ratio.

Description

Technical Field

[0001] This application relates to the field of electronic circuit technology, and in particular to in-memory computing devices, in-memory computing methods, processing devices, tile modules, and accelerators. Background Technology

[0002] As a key model for neuromorphic computing, Spike Neural Networks (SNNs) have become an effective alternative to traditional Artificial Neural Networks (ANNs) due to their significant low-power characteristics. The core feature of SNNs lies in their use of discrete pulse sequences (i.e., binary events of 0 or 1) as input and output signals, a fundamental difference from Artificial Neural Networks (ANNs), which rely on continuous high-precision floating-point or fixed-point data for computation. Thanks to this sparse, binary information representation, SNNs do not need to perform the energy-intensive multiplication and accumulation operations found in ANNs when performing core computational operations. Instead, they accumulate pulse events across multiple time steps. This fundamental shift in computational paradigm effectively avoids the enormous energy overhead of high-precision multiplication operations, thus significantly reducing overall computational energy consumption.

[0003] In related technologies, to achieve fast inference mode, serial readout and multi-bit enable signals are used to add weights, and a shift accumulator is used to shift and accumulate multi-bit weights. However, with large array sizes, serial readout leads to increased computational latency, thus negating the advantages of Computation in Memory (CIM) parallel computing. Furthermore, column-parallel sensitive amplifiers introduce additional power consumption during each weight readout. Summary of the Invention

[0004] This application provides an in-memory computing device, an in-memory computing method, a processing device, a tile module, and an accelerator to at least solve the problem of slow speed in serial-based in-memory computing methods in related technologies.

[0005] This application provides an in-memory computing device, including: multiple in-memory computing arrays, an addition tree, and a control module. The in-memory computing arrays are used to perform multiplication calculations of the single-bit input pulse and weights at each moment of the current cycle's buffer to obtain the membrane potential increment value at each moment of the current cycle. The addition tree is used to accumulate the membrane potential increment values ​​at each moment of the current cycle to obtain the total membrane potential increment of the current cycle. The control module is used to sum the total membrane potential increment of the current cycle with the membrane potential of the previous cycle to obtain the membrane potential of the current cycle, and compare the membrane potential of the current cycle with a preset threshold. When the membrane potential of the current cycle is greater than the preset threshold, a pulse is emitted.

[0006] This application also provides a storage and calculation method, including: multiplying the single-bit input pulse and weight at each moment of the current cycle to obtain the membrane potential increment value at each moment of the current cycle; multiplying the single-bit input pulse and weight at each moment of the current cycle to obtain the membrane potential increment value at each moment of the current cycle; summing the total membrane potential increment of the current cycle with the membrane potential of the previous cycle to obtain the membrane potential of the current cycle; comparing the membrane potential of the current cycle with a preset threshold; and when the membrane potential of the current cycle is greater than the preset threshold, issuing a pulse generation signal, which is used to generate a pulse.

[0007] This application also provides a processing device that supports spiking neural networks, including: the above-mentioned in-memory computing device.

[0008] This application also provides a tile module, including: multiple or more processing devices supporting spiking neural networks.

[0009] This application also provides an in-memory computing accelerator that supports spiking neural networks, including: a tile array, wherein the tile array includes multiple or more tile modules.

[0010] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described computing methods when executing the computer program.

[0011] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described storage and computing methods.

[0012] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described storage methods.

[0013] Each in-memory computing array in this application implements parallel computing within the array. The in-memory computing array can simultaneously perform multiplication operations on the single-bit input pulse and corresponding weight at each moment in the current cycle cache, synchronously generating the membrane potential increment value at each moment. With the help of the addition tree, these increment values ​​are accumulated in parallel, forming a high-efficiency computing link of "parallel multiplication + parallel accumulation". Compared with the step-by-step waiting delay caused by sequential processing of computing tasks, this parallel architecture significantly shortens the processing time of the entire "multiplication-accumulation" process in a single cycle while ensuring high computing accuracy. At the same time, by reducing the overhead of repeated data scheduling and state switching in serial computing, it further improves the computing efficiency per unit of energy consumption, ultimately achieving a dual optimization of computing latency and energy efficiency ratio.

[0014] This application divides the input pulse sequence into time windows in a fast prediction mode. Within each window, the membrane potential state is directly predicted by multiplying and accumulating multi-bit inputs and weights, skipping redundant calculations. Then, the prediction result determines whether to perform conventional calculations. The core idea is to utilize the spatiotemporal sparsity of pulses in a spiking neural network to achieve fast membrane potential prediction. Attached Figure Description

[0015] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the neuron output pulse provided in an embodiment of this application;

[0017] Figure 2(a) shows the structure of the top layer of the chip provided in the embodiment of this application;

[0018] Figure 2(b) shows the structure of the tile module provided in the embodiment of this application;

[0019] Figure 2(c) shows the structure of the processing unit provided in the embodiment of this application;

[0020] Figure 2(d) shows the structure of the synaptic array in the related technology;

[0021] Figure 3 A diagram illustrating the composition of the in-memory computing device provided in the embodiments of this application;

[0022] Figure 4 This is a structural diagram of the in-memory computing unit provided in the embodiments of this application;

[0023] Figure 5 A diagram illustrating the composition of another in-memory computing device provided in this application embodiment;

[0024] Figure 6 This is a state transition diagram of the pulse accumulation circuit provided in the embodiments of this application;

[0025] Figure 7 A structural diagram of the logic operation circuit provided in the embodiments of this application;

[0026] Figure 8 A schematic diagram of the accumulator circuit provided in an embodiment of this application;

[0027] Figure 9 A schematic diagram of the pulse delivery circuit provided in an embodiment of this application;

[0028] Figure 10 A structural diagram of the input buffer circuit provided in an embodiment of this application;

[0029] Figure 11 This is a state transition diagram of the output buffer circuit provided in an embodiment of this application;

[0030] Figure 12 A comparison chart of the computational accuracy of the solutions provided in this application and related technologies for the purposes of this application embodiments;

[0031] Figure 13 A comparison chart of computational delays provided for embodiments of this application;

[0032] Figure 14 A comparison chart of energy efficiency ratios provided for embodiments of this application;

[0033] Figure 15 This is a composition diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0035] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0036] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] In the SNN computation scheme of related technologies, the presynaptic neuron of the SNN sequentially fires pulses to the postsynaptic neuron in time steps. When the postsynaptic neuron receives a pulse, the synaptic weight of that input is added to the membrane potential. When the membrane potential reaches a certain threshold, the postsynaptic neuron fires a pulse. The mathematical expression for this computation process is:

[0038] (1)

[0039] (2)

[0040] in,V i ( t )and V i ( t- 1) These represent the membrane potentials at the current time step and the previous time step, respectively; x i ( t )and w ij These are the current input value and the weight value, respectively. Indicates the membrane potential threshold; s i ( t () indicates the firing of a pulse. The calculation of the current time step depends on the current input and the membrane potential value of the previous time step. This single-time-step serial calculation mechanism leads to a long computation delay. In actual SNN computation, neuronal spiking signals have high spatiotemporal sparsity, such as... Figure 1 As shown, pulses are not generated at every moment within any given time period. The spatial and temporal sparsity generated by this characteristic can further reduce the computational power consumption of SNNs.

[0041] To accomplish the aforementioned calculations, a CIM architecture supporting a fast inference mechanism for sparse impulses has been proposed to achieve low-latency, high-energy-efficiency SNN computation, such as... Figures 2(a) to 2(d) As shown, the architecture employs a hierarchical organization. The top layer of the chip consists of a tile array, a global cache, and peripheral circuitry for neural functions (pooling, accumulation, and activation). Each tile contains several processing units, and each processing unit consists of multiple synaptic arrays. The synaptic array is the basic unit of this architecture, composed of a 6TSRAM array and peripheral circuitry. This architecture has two phases: fast speculation and regular computation. Input pulses are stored in an input buffer, and the read unit reads the input values ​​sequentially. In the fast speculation phase, the set calculation module accumulates the neuron input within a time window TW, generating a multi-bit input value. The CIM array weights are read sequentially row-by-row into the adder. The multi-bit input cached in the set calculation serves as an enable signal to perform weight addition. The shift register can cache a partial sum, which is finally accumulated to the membrane potential of the current neuron and compared with the threshold voltage. If the current membrane potential is less than the threshold voltage, the speculation is successful, and the TW step calculation is skipped, entering the next speculation cycle. This mode shortens the computation completed in TW time steps to [missing information]. If the prediction fails (i.e., the current membrane potential is greater than or equal to the threshold voltage), the system switches to the normal calculation mode and performs calculations in a single time step.

[0042] Figures 2(a) to 2(d)The illustrated architecture, in order to achieve a fast inference mode, employs serial readout and multi-bit enable signals to add weights, and uses a shift accumulator to shift and accumulate multi-bit weights. With large array sizes, the serial readout method leads to increased computational latency, thus negating the advantages of CIM parallel computing. Furthermore, the column-parallel sensitive amplifier introduces additional power consumption during each weight readout.

[0043] Based on the above, embodiments of this application provide a memory computing device, such as... Figure 3 As shown, it includes: multiple in-memory computing arrays, an addition tree, and a control module. Figure 3 Take a storage-computing array as an example.

[0044] Figure 3 In this process, the in-memory computing array is used to perform multiplication calculations of the single-bit input pulse and weight at each moment of the current cycle, so as to obtain the membrane potential increment value at each moment of the current cycle.

[0045] Specifically, the in-memory compute array serves as the basic computing unit of the device, employing a cross-array topology (such as a cross-array based on Static Random Access Memory (SRAM)). Each array contains several in-memory compute units (each unit can simultaneously store weight data and perform computations). Its core function is to perform the multiplication calculation of the single-bit input pulse and weight at each moment of the current cycle's cache. The specific process is as follows:

[0046] (1) Data preparation stage: After the previous cycle ends, the control module pre-writes the weight data to be processed into the storage unit of the in-memory computing array (the weight data adopts binary or multi-value quantization form to adapt to the calculation requirements of single-bit input), and at the same time converts the input signal to be processed in the current cycle into a single-bit pulse sequence (the "presence" or "absence" of the pulse sequence corresponds to "1" or "0" of the input data, and the pulse time is strictly synchronized with the calculation timing), and temporarily stores it in the input buffer area of ​​the array (such as SRAM cache unit).

[0047] (2) Multiplication calculation stage: At each time step of the current cycle (i.e., "each moment"), the input buffer outputs a single-bit pulse in the row direction of the in-memory computing array according to the timing. When the pulse signal flows through the array cross point, it performs in-situ multiplication with the weight data stored in the column direction through "current / voltage mapping" (for example, when the input pulse is "1", the cross point outputs the current value corresponding to the weight; when the input pulse is "0", the output current is 0). The calculation result of each time step is "the membrane potential increment value at each moment of the current cycle" (the membrane potential increment value is output in analog or digital form, depending on the array implementation process).

[0048] (3) Parallelism design: Multiple in-memory computing arrays can process input data from different channels simultaneously (such as channels of different feature maps in a neural network). The arrays are synchronized through a data bus, which further improves the parallelism of the overall multiplication calculation.

[0049] Figure 3 In this context, the addition tree is used to accumulate the membrane potential increment at each moment of the current cycle to obtain the total membrane potential increment for the current cycle.

[0050] Specifically, the addition tree module adopts a multi-level tree topology. Its core function is to rapidly accumulate the "membrane potential increment value at each moment of the current cycle" output by the in-memory computing array to obtain the "total membrane potential increment of the current cycle". Key features include:

[0051] (1) Timing Synchronous Accumulation: The input end of the adder tree is directly connected to the output end of multiple in-memory computing arrays, and the timing of each input channel is strictly aligned with the output of the in-memory computing array, ensuring that the membrane potential increment value (including the increment value of different array outputs) at each moment in the current cycle can be input into the adder tree without omission.

[0052] (2) Low latency computation: Through the tree-structured hierarchical design, the addition tree can decompose the accumulation process of N incremental values ​​into log2N level parallel operations (for example, the accumulation of 8 incremental values ​​can be completed in only 3 levels of operations). Compared with the traditional serial adder, it greatly reduces carry latency and adapts to the high-speed computing requirements of in-memory computing devices.

[0053] (3) Accuracy guarantee: The addition tree has a built-in accuracy control unit, which can compensate for the quantization error in the accumulation process (such as by using rounding or truncation optimization strategies) to ensure that the accuracy of the output "total membrane potential increment" meets the requirements of subsequent calculations (such as the input accuracy of neural network activation functions).

[0054] Figure 3 In this process, the control module is used to sum the total increment of the membrane potential in the current cycle with the membrane potential in the previous cycle to obtain the membrane potential in the current cycle, and compare the membrane potential in the current cycle with a preset threshold. When the membrane potential in the current cycle is greater than the preset threshold, a pulse is emitted.

[0055] Specifically, the control module, acting as the "brain" of the device, integrates a timing controller, an arithmetic logic unit, a threshold register, and a pulse generator. Its core function is to achieve iterative updates of the membrane potential and control of pulse output. The specific process is as follows:

[0056] (1) Membrane potential update stage: The control module first temporarily stores the "membrane potential of the previous cycle" in the internal register; when the adder tree outputs the "total increment of membrane potential in the current cycle", the control module immediately performs the summation operation of "total increment of membrane potential in the current cycle and membrane potential of the previous cycle" to obtain the "membrane potential of the current cycle", and stores the updated membrane potential back to the register to complete the iteration of membrane potential.

[0057] (2) Threshold comparison and pulse generation stage: The threshold register of the control module stores a "preset threshold" adapted to the application scenario (such as the activation threshold of neurons in a neural network, which can be dynamically adjusted through the software interface); after obtaining the "membrane potential of the current cycle", the comparator of the control module immediately compares it with the preset threshold:

[0058] If the membrane potential of the current cycle is greater than the preset threshold, the pulse generator is triggered to output a standard digital pulse (such as a pulse that is high for one clock cycle). This pulse can be used as the output signal of the device to drive subsequent circuits (such as the next layer of the neural network's memory array or activation function module).

[0059] If the membrane potential of the current cycle is less than or equal to the preset threshold, the pulse generator will not output a pulse, and the control module will enter the preparation state for the next cycle.

[0060] (3) The control module is also responsible for outputting synchronous clock signals to the in-memory computing array and the adder tree to ensure that the working timing of the three components is strictly matched (such as the multiplication calculation of the in-memory computing array, the accumulation of the adder tree, and the updating of the membrane potential are completed in an orderly manner within the same cycle) to avoid data conflicts or calculation errors.

[0061] In some alternative implementations, such as Figure 3 As shown, the in-memory computing array includes multiple in-memory computing units, wherein the in-memory computing unit is used to perform multiplication calculation of a single-bit input pulse and weight at a moment in the current cycle to obtain the membrane potential increment value at a moment in the current cycle.

[0062] Specifically, the in-memory computing unit is the "computing atom" of the in-memory computing array. Its core responsibility is to perform "multiplication calculation of a single-bit input pulse and weight at a certain moment in the current cycle" and output "the membrane potential increment value at a certain moment in the current cycle". This unit adopts a "storage-computation in-situ fusion" design, which includes a weight storage unit, a pulse input and synchronization unit, an in-situ multiplication operation unit, and an output buffer unit. The components work together to achieve high-precision, low-latency multiplication calculation.

[0063] In some optional implementations, the in-memory computing unit includes: a static random access memory and an XOR gate circuit, wherein the static random access memory is used to store the weight of a moment in the current cycle; and the XOR gate is used to multiply the single-bit input pulse of each moment in the current cycle with the weight.

[0064] Specifically, each SRAM corresponds to only "a fixed moment in the current cycle" (such as time T1, time T2, etc. in the current cycle). At the end of the previous cycle, the control module issues a "time-weight" mapping instruction. The SRAM address decoder writes the corresponding binary weight into the specified 6T storage unit according to the "time address" (such as address 0x001 corresponding to time T1). In the current cycle, the read / write enable terminal (WE) of this storage unit is locked in a "read-only" state to ensure that the weight remains stable during the calculation process at the current moment and to prevent weight tampering across time periods.

[0065] Specifically, the output of the SRAM is directly connected to one of the inputs of the XOR gate circuit (called the "weight input"), which converts the logic level output by the SRAM (such as 0V representing weight 0 and 1.8V representing weight) into the standard logic level adapted by the XOR gate, avoiding the operation error caused by level mismatch, while enhancing the signal driving capability and ensuring that the weight signal can be stably transmitted to the XOR gate.

[0066] In some alternative implementations, such as Figure 4 As shown, the static random access memory includes: a first MOS transistor M1, a second MOS transistor M2, a third MOS transistor M3, a fourth MOS transistor M4, a fifth MOS transistor M5, and a sixth MOS transistor M6. Among them, the first MOS transistor M1 and the second MOS transistor M2 are access control transistors, and the third MOS transistor M3, the fourth MOS transistor M4, the fifth MOS transistor M5, and the sixth MOS transistor M6 constitute a cross-coupled inverter.

[0067] Specifically, the first MOSFET M1 and the second MOSFET M2 serve as access control transistors, employing an enhancement-mode NMOS structure. Their core function is to respond to the control signal of the word line (WL), achieving signal isolation or conduction between the memory node and the bit line (BL / BLB), thus providing a controllable channel for the "write" and "read" of weights.

[0068] In the scenario of storing weights in an in-memory computing unit, six MOSFETs work together through "access control-storage feedback" to achieve the complete "write-store-read" process of weights:

[0069] (1) Writing stage: WL high level → M1 / M2 turn on → BL / BLB weight signal forces update weight value and weight inversion value → cross-coupled inverter locks the new state through positive feedback → WL low level → M1 / M2 turn off, weight stored.

[0070] (2) Storage stage: The cross-coupled inverters (M3-M6) maintain the stable level of the drain of M1 and the drain of M2 through positive feedback. M1 / M2 are turned off to isolate external interference, and only microampere-level leakage current compensation is required.

[0071] (3) Reading stage: BL / BLB precharge → WL high level → M1 / M2 turn on → The level difference between the drain of M1 and the drain of M2 is output through BL / BLB → The read amplifier amplifies to obtain the weight signal and completes the reading.

[0072] This collaborative mechanism ensures high-speed weight storage while achieving low power consumption and high stability, perfectly meeting the application requirements of the in-memory computing unit for "real-time weight storage and support for high-frequency read and write".

[0073] Optionally, the 6-transistor SRAM cell used in the static random access memory of this application can also be implemented using 8-transistor, 10-transistor, or 12-transistor structures to improve the reliability of storage and computation. Furthermore, this new architecture can be further applied to in-memory accelerators based on dynamic random access memory (DRAM), flash memory (NAND / NOR), phase-change memory (PCM), and ferroelectric memory (FeRAM) to further improve performance such as computational energy efficiency.

[0074] In some alternative implementations, such as Figure 4 As shown, the XOR gate circuit includes: a seventh MOSFET M7, an eighth MOSFET M8, a ninth MOSFET M9, and a tenth MOSFET M10. The first terminal of the seventh MOSFET M7 is connected to the adder tree, and its gate is connected to the gate of the eighth MOSFET M8. The gate of the seventh MOSFET M7 is also connected to the inverted input pulse value, and its second terminal is grounded. The first terminal of the eighth MOSFET M8 is connected to the inverted weight value, and its second terminal is connected to the first terminal of the ninth MOSFET M9. The second terminal of the ninth MOSFET M9 is connected to the first terminal of the tenth MOSFET M10, and its gate is connected to the static random access memory (SRAM). The second terminal of the tenth MOSFET M10 is grounded, and its gate is connected to the gate of the ninth MOSFET M9.

[0075] Specifically, the XOR gate circuit adopts a CMOS complementary logic structure (NMOS and PMOS transistors work together). The seventh MOS transistor M7 and the tenth MOS transistor M10 are enhancement-mode MOS transistors (responsible for ground path conduction control), while the eighth MOS transistor M8 and the ninth MOS transistor M9 are enhancement-mode PMOS transistors (responsible for power path conduction control). This four-transistor combination, through "common-gate control + complementary path design," realizes the logical operation of the "inverted input pulse value" and the "inverted weight value," ultimately outputting a level signal that can be mapped to a single-bit multiplication result, providing accurate input for the subsequent accumulation of the film potential increment in the adder tree.

[0076] In a practical application, the specific structure of an in-memory computing unit is as follows: Figure 4 As shown, in the underlying architecture of the in-memory computing device, the in-memory computing unit is the smallest carrier simulating the "weight storage-signal computation" function of biological synapses. Each synaptic unit is specifically responsible for a set of "single-bit input pulse-weight" multiplication operations. Its core consists of a "6-transistor SRAM unit (weight storage module)" and a "4-transistor XNOR gate (logic operation module)" (as shown in the figure). Figure 4 As shown, Figure 4 Typically, it includes a block diagram connecting a 6TSRAM module, an input inverter, an XNOR gate module, and a level-to-physical quantity conversion module. Both are integrated into the same circuit unit using CMOS technology, achieving "weight local storage and in-situ computation," which significantly reduces data transfer losses.

[0077] To implement the multiplication calculation of a single-bit input pulse and its weights, i.e.

[0078] (3)

[0079] in, IN_B It is a single-bit input pulse; Q_N As weight; IN · Q This represents the membrane potential increment at a given moment in the current cycle.

[0080] Optionally, in the SNN-based in-memory computing accelerator proposed in this application, the synaptic array stores binary weight data. By reorganizing multiple storage units in the same column and adding a multi-bit multiplier, the binary weight parameter SNN-IMC accelerator architecture can be extended to support multi-bit weight parameters.

[0081] In some alternative implementations, such as Figure 5 As shown, the control module includes: a pulse accumulation circuit, a logic operation circuit, a drive circuit, an accumulator circuit, and a pulse delivery circuit.

[0082] Figure 5In the process, the pulse accumulation circuit is used to read the input pulse data in speculative mode and accumulate it according to the time window. In normal mode, the accumulated multi-bit pulse value is serially output to the logic operation circuit bit by bit.

[0083] Optionally, the pulse accumulation circuit adopts a "register + timing controller" architecture. Its core function is to perform "time window-level accumulation" or "multi-bit serial output" on the input pulse data to adapt to the computational efficiency requirements of different modes. Its design details and mode adaptation logic are as follows:

[0084] (1) Directly receive the input pulse sequence and simultaneously receive the "mode control signal" issued by the pulse output circuit.

[0085] (2) Preset configurable time window parameters (e.g., 16 time steps as 1 window, and the window length is dynamically adjusted through the software interface). Each window corresponds to a set of continuous input pulse data, which is used for batch accumulation in speculative mode.

[0086] (3) The timing controller divides the pulse sequence according to the "time window" and triggers the internal adder to accumulate the single-bit pulses in the window (e.g., there are 8 "1" pulses in 16 time steps, so the accumulation result is 8, which is temporarily stored in the register array in 8-bit multi-bit data format).

[0087] (4) After receiving the “mode switching signal”, the timing controller outputs the multi-bit pulse value (e.g., 8 bits) accumulated in the speculative mode in a “bit serial” manner. The multi-bit data can be output sequentially from the highest bit to the lowest bit through the shift register. Each bit corresponds to the pulse value of 1 time step. The output timing is strictly synchronized with the computing clock of the in-memory array.

[0088] Figure 5 In the fast speculation mode, the logic operation module is used to accumulate positive weights, reset negative weights to 0, and output to the drive circuit.

[0089] Specifically, the logic operation module only starts full operation in fast speculation mode. In normal mode, it switches to "pass-through mode", which directly transmits the output data of the pulse accumulation circuit to the drive circuit without weight filtering (normal mode requires accurate calculation of all weights).

[0090] Figure 5 In this circuit, the driving circuit is based on the output of the logic operation module to drive the in-memory computing array.

[0091] Optionally, the core function of the driver circuit is to generate drive signals adapted to the in-memory computing array based on the output of the logic operation circuit, ensuring the stable operation of the array. Its design details and driving mechanism are as follows:

[0092] (1) Requirements for in-memory computing array: The synaptic units (6T SRAM + 4 XNOR gates) in the array require two types of drive signals: "weight read / write control signal" and "computation timing synchronization signal".

[0093] (2) Driver circuit output: Converts the accumulation result of the logic operation circuit into two types of signals:

[0094] Enable drive signal: When the accumulated result > 0, a high-level enable signal is output to activate the corresponding synapse unit of the in-memory computing array; when the accumulated result = 0, a 0V low-level signal is output to disable the unit to reduce power consumption.

[0095] Timing synchronization signal: The built-in clock divider divides the device's main clock into an array-adaptive synchronous clock to ensure that the drive signal is aligned with the multiplication operation timing of the synaptic unit.

[0096] Figure 5 In this circuit, the accumulator circuit is used to sum the total increment of the membrane potential in the current cycle with the membrane potential in the previous cycle to obtain the membrane potential in the current cycle.

[0097] Optionally, the core function of the accumulator circuit is to sum the "total increment of membrane potential in the current period" and the "membrane potential in the previous period" to complete the iterative update of the membrane potential. The accumulator circuit receives the "total increment of membrane potential in the current period" output by the addition tree and triggers the summation operation; it outputs the "membrane potential in the current period" to the pulse emission circuit for threshold comparison; and it receives the "reset signal" from the pulse emission circuit (such as when the speculation fails) to clear the membrane potential register and restart the membrane potential update.

[0098] Figure 5 In the process, the pulse emission circuit compares the membrane potential of the current cycle with a preset threshold. If the membrane potential is less than the preset threshold, the prediction is successful and the system enters the fast prediction mode of the next time window. If the membrane potential of the current cycle is greater than or equal to the preset threshold, the prediction fails and the system switches to the normal calculation mode. The input pulse is accumulated and calculated in single time steps, and it is determined whether to emit a pulse.

[0099] Specifically, the pulse delivery circuit, as the core decision-making unit for the dual-mode operation of the in-memory computing device, has the core task of dynamically scheduling the system to enter either "rapid prediction mode" or "normal calculation mode" based on the accurate comparison between the "current period membrane potential" and the "preset threshold". Specifically, the circuit first receives the "current period membrane potential" data transmitted by the accumulator circuit, and simultaneously retrieves the pre-configured "preset threshold" from the built-in threshold register. Then, the comparator performs the comparison operation between the two.

[0100] Specifically, when the comparison result determines that "the current period membrane potential is less than the preset threshold", the "prediction success" process is triggered. The circuit immediately generates a "prediction mode hold signal" through the mode control logic. After the comparison result stabilizes, it is synchronously transmitted to the pulse accumulation circuit and the logic operation circuit to ensure that the pulse accumulation circuit continues to maintain the "time window level batch accumulation" mode and the logic operation circuit maintains the "positive weight screening + parallel accumulation" function.

[0101] Specifically, if the comparison result determines that "the current period membrane potential is greater than or equal to the preset threshold", the "prediction failure" process is triggered and the circuit enters the emergency mode switching state: first, a "normal mode switching signal" is generated and simultaneously sent to the pulse accumulation circuit, logic operation circuit and drive circuit.

[0102] In some alternative implementations, the pulse accumulation circuit includes a multiplexer, wherein the multiplexer is used to read input pulse data and accumulate it according to a time window in speculative mode, and to output the accumulated multi-bit pulse value bit-by-bit serially to the logic operation circuit in normal mode.

[0103] Specifically, as the key switching unit for the pulse accumulation circuit to realize "batch accumulation in speculative mode" and "serial output in normal mode", the multiplexer adopts a high-speed CMOS 2-to-1 structure. Its core responsibility is to dynamically select different data channels based on the "mode control signal" (high level corresponds to speculative mode, low level corresponds to normal mode) issued by the control module, realize seamless switching between "real-time accumulation of input pulses" and "serial output of multiple bit values", and at the same time ensure the timing consistency and signal integrity of data transmission, so as to provide accurate data input for subsequent logic operation circuits.

[0104] Specifically, in speculative mode, the multiplexer responds to a high-level "mode control signal" and prioritizes the "real-time input pulse channel." At this time, it needs to work with the "window timing controller" of the pulse accumulation circuit to complete the batch accumulation of input pulses. The specific workflow is as follows: The window timing controller first outputs a "window enable signal" to the multiplexer through the "time window parameters" configured in the software; during the window enable period, the multiplexer keeps the "real-time input pulse channel" on, continuously sending the single-bit input pulse of each time step to the pulse accumulation circuit; the adder performs parallel accumulation of the pulses within the window. During this process, the multiplexer must strictly shield the "multi-bit accumulated value buffer channel" to avoid buffered data interfering with real-time accumulation; when the window timing controller detects a "window end signal" (i.e., completion of pulse reception for the preset time step), it immediately triggers the multiplexer to temporarily close the "real-time input pulse channel," and simultaneously latches the accumulation result of the adder into the register array of the "multi-bit accumulated value buffer channel," preparing data for possible subsequent normal mode switching.

[0105] In normal mode (triggered by the pulse output circuit after determining "speculation failure", the mode control signal turns low), the multiplexer quickly switches to the "multi-bit accumulated value buffer channel" and outputs the buffered multi-bit pulse value to the logic operation circuit in a "bit serial" manner.

[0106] Optionally, to meet the low power consumption and high reliability requirements of in-memory computing devices, the multiplexer also integrates low power consumption control and fault detection functions: In speculative mode, when the input pulse sequence is a continuous "0" (no valid pulse), the multiplexer automatically enters a "low power sleep state" and shuts down the drive circuit of the real-time input channel; during mode switching, a built-in "glitch suppression circuit" filters out level glitches at the moment of switching through delay matching to avoid erroneous data input to the logic operation circuit; at the same time, the multiplexer also periodically checks the signal integrity of the input channel. If a continuous level abnormality occurs in the real-time input pulse channel, it immediately outputs a "channel fault signal" to trigger fault isolation and redundant channel switching, ensuring the continuous and stable operation of the pulse accumulation circuit.

[0107] In summary, through dynamic channel selection, timing coordination, and performance optimization, the multiplexer not only realizes the core data scheduling function of the pulse accumulation circuit in dual modes, but also becomes a key hub connecting "input pulse acquisition" and "logic operation processing" through low power consumption, anti-interference, and fault detection design, directly affecting the computing efficiency and reliability of the in-memory computing device.

[0108] Optionally, the pulse accumulation circuit is used for fast speculation mode, and its state transitions are as follows: Figure 6 As shown, it mainly includes the following steps:

[0109] (1) IDLE: When the circuit detects that the reset signal is low, all registers are reset, the Data_done signal is pulled high, the Predict enable signal is pulled high, and the circuit transitions to State1 state when the reset signal is high.

[0110] The IDLE state is the initial state of the fast speculation mode. Its core function is to ensure that all hardware components of the circuit are in an initial ready state through a reset operation, preparing for subsequent speculation accumulation. Its specific implementation is closely related to the signal logic and the reset mechanism of the control module.

[0111] ① Reset Trigger and Hardware Initialization: The entry into IDLE state is triggered by a "reset signal". When the circuit detects that the reset signal is low, it immediately sends a "clear signal" to all core registers of the pulse accumulation circuit, including the window accumulation register array, the shift register (for subsequent serial output), and the counting register inside the timing controller (for time window counting), to ensure that the initial value of all registers is 0, avoiding accumulation errors caused by residual data from the previous cycle; at the same time, the real-time input pulse channel and multi-bit buffer channel of the multiplexer are turned off, so that the output is in a high-impedance state to prevent invalid signals from interfering with the logic operation circuit.

[0112] ② Ready Signal Generation and State Maintenance: After the reset operation is completed, the circuit automatically generates the "Data_done" signal and pulls it high; simultaneously, the "Predict Enable Signal" (predictive mode enable signal, active high) is also pulled high. This signal directly controls the channel selection logic of the multiplexer (preparing for channel switching in State 1) and activates the parameter loading function of the window timing controller. At this time, the circuit remains in the IDLE state until the reset signal is detected to change from low to high. That is, the high reset signal means that the system reset is released or the fault recovery is completed, and the state transition condition is met. The circuit transitions from the IDLE state to the State 1 state triggered by the rising edge of the global clock.

[0113] ③ Abnormal handling adaptation: If the reset signal is detected to be continuously low in IDLE state, the circuit will output a "reset abnormal alarm signal" to the control module, indicating that there may be a fault in the reset circuit. At the same time, the internal redundant reset mechanism will be activated to ensure that the register can still maintain the initial state and avoid the circuit from falling into abnormal standby.

[0114] (2) State 1: This state is the speculative mode. When the speculative mode signal Predict is high, the input pulse data in the input buffer circuit is read and accumulated according to the time window. At this time, the write completion signal Data_done is low. When Data_done is high, the state transitions to State 2.

[0115] Specifically, State 1 is the core computational stage of the fast prediction mode. During the high level of the Predict enable signal, it completes the core action of "input pulse reading - time window accumulation". Its hardware coordination revolves around the multiplexer, window timing controller and carry-save adder (CSA):

[0116] ① Mode Enable and Channel Selection: After entering State1, the Predict enable signal remains high, directly triggering the multiplexer to select the "real-time input pulse channel" (consistent with the multiplexer design described above), while simultaneously closing the signal path of the "multi-bit accumulated value buffer channel". At this time, the input terminal of the multiplexer only receives a single-bit input pulse sequence from the "input buffer circuit".

[0117] ② Time Window Configuration and Accumulation Execution: First, the preset "time window parameters" are read, and the "window count register" is started to count. Then, a "pulse read enable signal" is output to the input buffer circuit, allowing the input buffer to release single-bit pulses sequentially according to time steps, and the pulses within the window are accumulated in parallel. During this process, the "Data_done signal" remains at a low level (0V). This level is used to inform the control module that "the current stage is accumulation, and the data is not ready," to avoid the control module accidentally triggering the output action.

[0118] ③ Accumulation Completion and State Transition Trigger: When the window count value reaches the preset time window parameter, a "window end signal" is generated. This signal triggers the adder to stop accumulating and latches the final accumulated result into the dedicated register of the "multi-bit accumulated value buffer channel" to ensure that the data is not overwritten by subsequent pulses. On the other hand, it triggers the "Data_done signal" to change from low to high. The rising edge of this signal serves as the core trigger condition for State1→State2. After detecting the high level of Data_done, the circuit transitions to State2 on the next rising edge of the global clock. If "pulse loss" occurs during the accumulation process, the window period is automatically extended until the pulse of the complete window is received, avoiding incomplete accumulation due to signal delay.

[0119] (3) State2: At this time, the accumulated multi-bit pulse value is serially output to the logic operation circuit bit by bit. The Data_out signal is low level until the data output is completed, and then the Data_out signal is pulled high.

[0120] Specifically, State2 is the result output stage of the fast speculation mode. Its core task is to serially transmit the multi-bit pulse value accumulated in State1 to the logic operation circuit, as follows:

[0121] ① Output channel switching and signal initialization: After entering State2, the Predict enable signal remains high and outputs a "channel switching signal" to the multiplexer, causing the multiplexer to switch from the "real-time input pulse channel" to the "multi-bit accumulated value buffer channel". At this time, the input of the multiplexer receives the multi-bit accumulated value latched in State1; at the same time, the "Data_out signal" is pulled low (0V). This level is used to inform the logic operation circuit that "data is being output and sampling needs to be prepared" and triggers the shift register to enter the working state.

[0122] ② Bit-by-bit serial output with timing synchronization: The shift register first receives the multi-bit data from the buffer register, and then, driven by the "serial output clock," shifts and outputs the data sequentially in the order of "most significant bit (MSB) → least significant bit (LSB)." During this process, the window timing controller monitors the number of shifts in real time through the "shift counter register." When the number of shifts equals the number of bits of data, it determines that "data output is complete."

[0123] ③ Output Completion and Signal Feedback: After data output is complete, the shift register sends a "shift completion signal" to the window timing controller. The timing controller immediately triggers the "Data_out signal" to change from low to high. This high-level signal feeds back to the control module, informing it that "multi-bit value output is complete, and the next round of speculation or switching mode can begin." Simultaneously, it triggers a reset of the shift register and multiplexer, preparing for the next output. If the control module needs to continuously perform multi-window speculation, after Data_out goes high, State2 will automatically detect the "next window enable signal" (from the control module). If this signal is valid, it will jump back to State1 to begin accumulating a new window; if invalid, it will jump back to the IDLE state to wait for the next speculation instruction.

[0124] In some alternative implementations, such as Figure 7 As shown, the logic operation circuit includes: an XOR gate chip and a DQ flip-flop. The first terminal of the XOR gate chip is connected to the weight value, the second terminal of the XOR gate chip is connected to the reset pulse, and the third terminal of the XOR gate chip is connected to the first terminal of the DQ flip-flop. The second terminal of the DQ flip-flop is connected to the clock signal, the third terminal of the DQ flip-flop is connected to the control signal, and the fourth terminal of the DQ flip-flop is connected to the logic operation circuit.

[0125] Specifically, the logic operation circuit, as the key unit of "weight screening-data preprocessing" in the fast speculation mode, is composed of an XOR gate chip and an edge-triggered DQ flip-flop. The two are connected by signal cascading and timing synchronization to realize three core functions: "negative weight reset to 0", "input inversion to adapt XNOR multiplication" and "WEN control high impedance state".

[0126] Specifically, in fast speculation mode, to improve computational efficiency and avoid interference from negative weights on the speculation results, the input signal corresponding to the negative weight must be forcibly set to "0", retaining only the positive weight for the accumulation calculation of the membrane potential increment. Furthermore, considering that the Computation in Memory (CIM) unit uses XNOR gates to implement the multiplication operation between a single-bit input pulse and the weight, and the core operation logic of the XNOR gate requires the input signal to be complementary (i.e., the inverted value of the weight and the inverted value of the input pulse), the corresponding input signal of the CIM unit needs to be preprocessed by inverting. Based on this requirement, a dedicated logic operation module needs to be introduced. The core function of this module is limited to logic operations (excluding arithmetic operations), and it is only responsible for completing two types of logic processing: "resetting negative weights to 0" and "input signal inversion". Simultaneously, the WEN (write enable) signal serves as the module's output control signal. When WEN is high (logic "1"), the module output needs to switch to a high-impedance state to isolate invalid data and avoid interfering with the subsequent drive circuit and the normal operation of the CIM unit.

[0127] In some optional implementations, the accumulator circuit is also used to sample the total increment of the output membrane potential of the adder tree in the current cycle, perform bit expansion on the total increment of the membrane potential, shift the accumulated membrane potential obtained in the previous cycle one bit to the left, and add the bit-expanded total increment of the membrane potential to the shifted accumulated membrane potential obtained in the previous cycle to obtain the membrane potential of the current cycle.

[0128] Specifically, in addition to completing the basic membrane potential update function, the accumulator circuit also needs to achieve precise iteration of the membrane potential through refined operations of "sampling-bit expansion-shifting-summing" (especially to meet the weighted accumulation requirements of historical membrane potential in neural networks). Its core operations revolve around four major stages: "addition tree output sampling", "membrane potential total increment bit expansion", "last period membrane potential shift", and "double data summing".

[0129] Optionally, the schematic diagram of the accumulator circuit is as follows: Figure 8 As shown, this module, as the core unit for iterative updating of membrane potential in the in-memory computing device, has the core function of receiving the calculation results (total increment of membrane potential) output from the adder tree within the in-memory array, and realizing the shift accumulation of membrane potential through refined processing of "sampling-bit expansion-shifting-summing," ultimately generating the membrane potential for the current period. Its shift accumulation process is specifically implemented through the following three core operations:

[0130] (1) Sample the output result PSUM (i.e. the total increment of membrane potential) of the adder tree in the current cycle, and perform bit extension processing on PSUM to extend the bit width of PSUM to the same specification as the accumulated membrane potential value SUM in the previous cycle (e.g., when PSUM is 12 bits, extend it to 16 bits) to avoid high-bit data truncation due to bit width mismatch in subsequent summation operations, thus ensuring calculation accuracy.

[0131] (2) Read the accumulated membrane potential value SUM stored in the previous cycle and shift SUM one bit to the left through the built-in shift register. This operation is essentially a "×2 weighting" process for historical membrane potentials to highlight the cumulative effect of recent membrane potentials (to adapt to the timing characteristics of the spiking neural network). If data overflow is detected during the shifting process, it will be clamped to the maximum effective value through the saturation compensation mechanism to avoid abnormal value interference.

[0132] (3) The bit-expanded PSUM and the shifted SUM are added by the carry-lookahead adder. The result is temporarily stored in a register to ensure timing stability. In the next cycle, the final value Q (i.e. the membrane potential of the current cycle) is output through the synchronous sampling circuit, providing accurate membrane potential data support for the threshold comparison of the subsequent pulse firing circuit.

[0133] In some alternative implementations, the schematic diagram of the pulse delivery circuit is as follows: Figure 9 As shown, this circuit serves as the mode decision core of the in-memory computing device. Its main function is to receive and accumulate intermediate values ​​of the membrane potential, and then compare the accumulated result with a preset membrane potential threshold at high speed. The comparison result directly serves as the basis for switching between "fast prediction mode" and "conventional calculation mode". Its state transition process is clear and the timing is rigorous, specifically including the following steps:

[0134] (1) IDLE (Idle Reset) State: When the circuit detects that the reset signal is low (low level is valid), it immediately triggers all internal registers to perform a reset and clear operation; after the reset is completed, the TW_done signal (time window completion signal) is pulled high (indicating that the circuit is ready), and at the same time the Predict enable signal (predict mode enable signal) is pulled high (to prepare for entering the speculative mode); this state will continue until the circuit detects that the reset signal has switched to a high level. At this time, the state triggers the transition condition, and the circuit switches to State1 state synchronously.

[0135] (2) State1 (Speculation Mode Accumulation): During the period when the speculation mode enable signal (Predict) is kept high, the circuit will continuously read the intermediate value of the membrane potential output by the accumulator and accumulate the intermediate values ​​of all accumulators through the internal addition unit; during this process, the TW_done signal is kept low (indicating that the current time window is not completed); when the circuit detects that the calculation task of the current time window is completed (such as reaching the preset number of time steps), the TW_done signal is pulled high (indicating that the time window accumulation is completed), and the circuit switches to State2 synchronously.

[0136] (3) State2 (threshold comparison and mode judgment): After entering this state, the circuit first compares the accumulated membrane potential value obtained in the State1 stage with the preset membrane potential threshold at high speed: if the accumulated value is less than the preset threshold, the comparison result False=0 is output (indicating successful speculation), and the circuit is triggered to directly enter the fast speculation process of the next time window (no need to switch modes); if the accumulated value is greater than or equal to the preset threshold, the comparison result False=1 is output (indicating speculation failure). At this time, the mode switching condition is met, and the circuit switches to State3 (normal calculation mode).

[0137] (4) State3 (normal calculation mode): This state is the precise calculation mode. The circuit first reads the single-bit input pulse from the input buffer circuit step by step, and accumulates the result of the calculation of the pulse signal and weight of each time step in real time. After the accumulation of each time step is completed, the currently calculated membrane potential value is immediately compared with the preset membrane potential threshold. If the membrane potential value is less than the threshold, the current membrane potential is temporarily stored in a dedicated register (for the membrane potential iterative update of the next time step). If the membrane potential value is greater than or equal to the threshold, a standard digital pulse is generated by the internal pulse generator and the pulse is sent to the output buffer. After the calculation of all time steps within the preset TW (time window) is completed, the circuit pulls up the TW_done signal (indicating that the normal mode calculation is completed) to prepare for the next round of mode switching or process reset.

[0138] In some alternative implementations, such as Figure 5 As shown, the control module also includes an input buffer circuit and an output buffer circuit. The input buffer circuit is used to read and buffer the off-chip weights and input pulses for each clock cycle so that the pulse accumulation circuit can read them. The output buffer circuit is used to buffer the pulses output by the pulse delivery circuit.

[0139] In some alternative implementations, such as Figure 10 As shown, the input buffer circuit includes: an input multiplexer, a pair of ping-pong buffers, and an output multiplexer. The input terminal of the input multiplexer, which enables pipelined operation, is connected to the external weights and input pulses. The output terminal of the input multiplexer is connected to the input terminals of the pair of ping-pong buffers. The input and output terminals of the pair of ping-pong buffers are connected to the input terminal of the output multiplexer. The output multiplexer is used to output the external weights and input pulses of the buffer for each clock cycle.

[0140] The input buffer circuit serves as the core of off-chip data interaction and pipelined scheduling in the in-memory computing device, and its circuit schematic is shown below. Figure 10 As shown, the core function is to receive, temporarily store, and efficiently transmit off-chip weight data and input pulse signals. Through a pair of ping-pong buffer units, it achieves parallel pipelined operation of "data writing-data output" to avoid data transmission bottlenecks.

[0141] Specifically, when a low-level active reset signal is detected, all internal registers (including cache unit control registers, data latch registers, etc.) immediately perform a reset and clear operation to ensure that there is no residual data in the initial state of the cache unit; after the reset signal switches to a high level, the reset operation terminates, and the cache circuit enters an idle and ready state, waiting to receive external data and control instructions.

[0142] Specifically, each clock cycle, the input buffer circuit can precisely read one set of data (weighted data or input pulse signal, switched according to system scheduling instructions) from the external storage device. Simultaneously, the Ctrl signal serves as a buffer write strobe: when Ctrl is high, the currently read external data is written to one buffer; when Ctrl is low, the data is written to another buffer. Through this "alternating write" logic, the two buffer units can achieve parallel pipelined operation. For example, while one buffer receives new data, the other buffer can output stored data to the subsequent in-memory computing array (transmitting weights) or pulse accumulation circuit (transmitting input pulses), significantly improving the throughput of external data transmission to the internal device and avoiding computational stagnation due to data waiting.

[0143] In some alternative implementations, the state transition process of the output buffer circuit is as follows: Figure 11 As shown, this circuit serves as the core of pulse signal distribution in the in-memory computing device, responsible for achieving the orderly connection of "pulse temporary storage within the neural network layer - inter-layer transmission - off-chip output". Its state transition logic is clear and strictly synchronized with the timing of the preceding calculation module. The specific steps are as follows:

[0144] (1) IDLE (Idle Reset) State: When the circuit detects a low-level reset signal (active low), it immediately triggers all internal registers (including pulse buffer register, status control register, and output synchronization register) to perform a reset and clear operation. After the reset operation is completed, the Comp_done signal (calculation completed and output ready signal) is pulled high synchronously to indicate that the circuit has entered the ready state. This state will continue until the circuit detects that the reset signal has switched to a high level. At this time, the state transition condition is met, and the circuit synchronously switches to State1 state.

[0145] (2) State1 (Interlayer Pulse Output): After entering this state, the Comp_done signal changes from high level to low level (indicating that the circuit has entered the pulse output stage). The output buffer circuit outputs the temporarily stored pulse signal to the "input buffer circuit of the next layer of the neural network" clock cycle according to the "pulse output enable signal" of the preceding calculation module (such as the pulse output circuit), providing input data for the calculation of the next layer. During the entire output process, the circuit receives the "receive confirmation signal" of the input buffer circuit of the next layer in real time to ensure that each pulse is accurately received. When all pulse outputs of the current layer are completed and the "full receive confirmation" of the next layer is received, the circuit determines that "interlayer transmission and preparation for the next layer calculation are completed", and then pulls the Comp_done signal high, and the state is synchronously switched to State2.

[0146] (3) State2 (Off-chip result output): In this state, the output buffer circuit switches the signal output path and no longer transmits to the interlayer input buffer circuit. Instead, it converts the format of the temporarily stored final pulse result (such as the classification pulse signal of the neural network output layer) and outputs it to the off-chip result display or storage module. During the output process, the circuit monitors the transmission status through the "off-chip feedback signal" to ensure that the pulse signal is not lost or distorted. After all off-chip output tasks are completed, the circuit can maintain the State2 state and wait for the next round of off-chip output instructions according to the instructions of the control module, or pull the Comp_done signal low again and return to the IDLE state to wait for the next system reset or calculation cycle to start.

[0147] and Figures 2(a) to 2(d) Compared with the related technologies shown, this application can further reduce computational latency and improve computational energy efficiency without sacrificing computational accuracy. The comparison of computational accuracy, computational latency, and energy efficiency ratio of the solutions in this application and related technologies under different array sizes is as follows: Figure 12 , Figure 13 and Figure 14 As shown. Both this application and related technologies employ the same spiking neural network architecture and use the CIFAR-10 dataset as the standard dataset. Implemented using a 180nm CMOS process, the computational accuracy of this application is the same as that of related technologies. The computational latency of this application is significantly reduced for different array sizes. With a 512×512 array size, the computational latency of this application is only 56.3% of that of related technologies, and the energy efficiency is improved to 2.03 times that of related technologies.

[0148] Embodiments of this application provide a storage computing method, including:

[0149] The membrane potential increment value at each moment of the current cycle is obtained by multiplying the single-bit input pulse and weights at each moment of the current cycle. The total membrane potential increment of the current cycle is summed with the membrane potential of the previous cycle to obtain the membrane potential of the current cycle. The membrane potential of the current cycle is then compared with a preset threshold. When the membrane potential of the current cycle is greater than the preset threshold, a pulse generation signal is issued to generate a pulse.

[0150] In some optional implementations, the membrane potential of the current cycle is compared with a preset threshold. If the membrane potential is less than the preset threshold, the prediction is successful and the system enters the fast prediction mode of the next time window. If the membrane potential of the current cycle is greater than or equal to the preset threshold, the prediction fails and the system switches to the normal calculation mode. The input pulse is accumulated and calculated on a single time step basis, and it is determined whether to issue a pulse.

[0151] The in-memory computing method provided in this application relies on the in-memory computing device (including in-memory computing array, adder tree, accumulator circuit, pulse generation circuit, etc.) described above. Through the ordered steps of "data preprocessing - operation - incremental accumulation - membrane potential iteration - threshold decision - pulse generation", it achieves low-power, high-precision spiking neural network operation, which is particularly suitable for dual-mode scenarios of "fast speculation" and "conventional calculation". The specific steps are as follows:

[0152] (1) Relying on the synaptic unit (6T SRAM + 4 XNOR gates) of the in-memory computing array, the "multiplication calculation of single-bit input pulse and weight at each moment of the current cycle" is realized to generate the membrane potential increment value at the corresponding moment, and the difference between fast prediction and conventional mode needs to be adapted.

[0153] (2) The membrane potential increment value at each moment of the current cycle is accumulated through the addition tree module to obtain the total membrane potential increment of the current cycle.

[0154] (3) Using the accumulator circuit, the "total increment of membrane potential in the current cycle" is added to the "membrane potential in the previous cycle" to obtain the membrane potential in the current cycle.

[0155] (4) This step relies on the pulse generation circuit to determine whether to generate a pulse by comparing the membrane potential of the current cycle with the preset threshold, and at the same time realizes dual-mode switching.

[0156] (5) After the pulse is generated, the output buffer circuit uses the “Comp_done signal” to indicate that the current cycle operation is completed.

[0157] In summary, this in-memory computing method, through deep collaboration with in-memory computing hardware, not only realizes the core operations of "multiplication-accumulation-comparison-pulse", but also balances computing efficiency and accuracy through dual-mode design, making it fully adaptable to the needs of low-power, high-computing-power scenarios such as edge computing and neural network inference.

[0158] An embodiment of this application provides a processing device that supports spiking neural networks, including: the above-mentioned in-memory computing device.

[0159] Specifically, the structure of the processing device supporting the spiking neural network in this application is shown in Figure 2(c). The synaptic array in Figure 2(c) is the processing device, which includes the above-mentioned in-memory computing device.

[0160] In some optional implementations, the processing device supporting the spiking neural network further includes: a processing input buffer module and a processing output buffer module, wherein the processing input buffer module is connected to the in-memory computing array and is used to access off-chip weights and input pulses; the processing output buffer module is connected to the control module and is used to buffer the pulses output by the control module.

[0161] An embodiment of this application provides a tile module, comprising: multiple processing devices supporting spiking neural networks. Specifically, the structure of the tile module of this application is shown in Figure 2(b).

[0162] An embodiment of this application discloses an in-memory computing accelerator supporting spiking neural networks, comprising: a tile array, wherein the tile array includes multiple tile modules. Specifically, the structure of the in-memory computing accelerator of this application is shown in Figure 2(a).

[0163] Embodiments of this application also provide an electronic device, such as... Figure 15 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to perform the steps in any of the above-described storage method embodiments.

[0164] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described storage method embodiments when running.

[0165] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0166] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described storage and computing method embodiments.

[0167] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described storage method embodiments.

[0168] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0169] The foregoing has provided a detailed description of the in-memory computing device, in-memory computing method, processing device, tile module, and accelerator provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A storage-and-computing integrated device, characterized in that, include: Multiple in-memory computing arrays, addition trees, and control modules, among which, The in-memory computing array is used to simultaneously perform multiplication calculations on the single-bit input pulse and weight at each moment of the current cycle's cache, and synchronously obtain the membrane potential increment value at each moment of the current cycle. The addition tree is used to accumulate the membrane potential increment value at each moment of the current cycle to obtain the total membrane potential increment of the current cycle. The control module is used to sum the total increment of the membrane potential in the current cycle with the membrane potential in the previous cycle to obtain the membrane potential in the current cycle, and compare the membrane potential in the current cycle with a preset threshold. When the membrane potential in the current cycle is greater than the preset threshold, a pulse is emitted. The in-memory computing array includes multiple in-memory computing units, wherein the in-memory computing unit is used to perform multiplication calculation of a single-bit input pulse and weight at a moment in the current cycle to obtain the membrane potential increment value at a moment in the current cycle. The in-memory computing unit includes a static random access memory and an XOR gate circuit, wherein the static random access memory is used to store the weight of a moment in the current cycle; the XOR gate is used to multiply the single-bit input pulse of each moment in the current cycle with the weight.

2. The in-memory computing device according to claim 1, characterized in that, The static random access memory includes a first MOS transistor, a second MOS transistor, a third MOS transistor, a fourth MOS transistor, a fifth MOS transistor, and a sixth MOS transistor, wherein the first MOS transistor and the second MOS transistor are access control transistors, and the third MOS transistor, the fourth MOS transistor, the fifth MOS transistor, and the sixth MOS transistor constitute a cross-coupled inverter.

3. The in-memory computing device according to claim 1, characterized in that, The XOR gate circuit includes: a seventh MOSFET, an eighth MOSFET, a ninth MOSFET, and a tenth MOSFET, wherein, The first terminal of the seventh MOS transistor is connected to the adder tree, the gate of the seventh MOS transistor is connected to the gate of the eighth MOS transistor, the gate of the seventh MOS transistor is also connected to the inverted value of the input pulse, and the second terminal of the seventh MOS transistor is grounded. The first terminal of the eighth MOS transistor is connected to the reverse weight value, and the second terminal of the eighth MOS transistor is connected to the first terminal of the ninth MOS transistor. The second terminal of the ninth MOS transistor is connected to the first terminal of the tenth MOS transistor, and the gate of the ninth MOS transistor is connected to the static random access memory. The second terminal of the tenth MOS transistor is grounded, and the gate of the tenth MOS transistor is connected to the gate of the ninth MOS transistor.

4. The in-memory computing device according to claim 1, characterized in that, The control module includes: a pulse accumulation circuit, a logic operation circuit, a driver circuit, an accumulator circuit, and a pulse delivery circuit, wherein... The pulse accumulation circuit is used to read the input pulse data in speculative mode and accumulate it according to a time window, and in normal mode, it outputs the accumulated multi-bit pulse value to the logic operation circuit bit by bit in serial mode. The logic operation circuit is used to accumulate positive weights in fast speculation mode, reset negative weights to 0, and output to the driving circuit. The driving circuit drives the in-memory computing array based on the output of the logic operation circuit; The accumulator circuit is used to sum the total increment of the membrane potential in the current cycle with the membrane potential in the previous cycle to obtain the membrane potential in the current cycle. The pulse emission circuit is used to compare the membrane potential of the current cycle with a preset threshold. If the membrane potential is less than the preset threshold, the prediction is successful and the circuit enters the fast prediction mode of the next time window. If the membrane potential of the current cycle is greater than or equal to the preset threshold, the prediction fails and the circuit switches to the normal calculation mode. The input pulse is accumulated and calculated in single time steps, and it is determined whether to emit a pulse.

5. The in-memory computing device according to claim 4, characterized in that, The pulse accumulation circuit includes: a multiplexer, wherein... The multiplexer is used to read the input pulse data and accumulate it according to a time window in speculative mode, and to output the accumulated multi-bit pulse value to the logic operation circuit bit by bit in normal mode.

6. The in-memory computing device according to claim 4, characterized in that, The logic operation circuit includes: an XOR gate chip and a DQ flip-flop, wherein... The first terminal of the XOR gate chip is connected to the weight value, the second terminal of the XOR gate chip is connected to the reset pulse, and the third terminal of the XOR gate chip is connected to the first terminal of the DQ flip-flop. The second terminal of the DQ flip-flop is connected to a clock signal, the third terminal of the DQ flip-flop is connected to a control signal, and the fourth terminal of the DQ flip-flop is connected to the logic operation circuit.

7. The in-memory computing device according to claim 4, characterized in that, The accumulator circuit is also used to sample the total increment of the output membrane potential of the adder tree in the current cycle, perform bit expansion on the total increment of the membrane potential, shift the accumulated value membrane potential obtained in the previous cycle to the left by one bit, and add the bit-expanded total increment of the membrane potential to the shifted accumulated value membrane potential obtained in the previous cycle to obtain the membrane potential of the current cycle.

8. The in-memory computing device according to claim 4, characterized in that, The control module further includes: an input buffer circuit and an output buffer circuit, wherein... The input buffer circuit is used to read and buffer the off-chip weights and input pulses for each clock cycle so that the pulse accumulation circuit can read them. The output buffer circuit is used to buffer the pulses output by the pulse emission circuit.

9. The in-memory computing device according to claim 8, characterized in that, The input buffer circuit includes: an input multiplexer, a pair of ping-pong buffers, and an output multiplexer, wherein pipelined operation is implemented. The input terminal of the input multiplexer is connected to an external weight and an input pulse, and the output terminal of the input multiplexer is connected to the input terminal of the pair of ping-pong buffers; The input and output terminals of the pair of ping-pong buffers are connected to the input terminal of the output multiplexer; The output multiplexer is used to output the off-chip weights and input pulses for each clock cycle of the output buffer.

10. A storage and calculation method, characterized in that, Based on the in-memory computing device according to any one of claims 1-9, the method includes: The membrane potential increment value at each moment of the current cycle is obtained by multiplying the single-bit input pulse and weight at each moment of the current cycle's buffer. The membrane potential increment value at each moment of the current cycle is obtained by multiplying the single-bit input pulse at each moment of the current cycle with the weight. The membrane potential of the current cycle is summed with the membrane potential of the previous cycle to obtain the membrane potential of the current cycle. The membrane potential of the current cycle is then compared with a preset threshold. When the membrane potential of the current cycle is greater than the preset threshold, a pulse generation signal is issued to generate a pulse.

11. The storage and computation method according to claim 10, characterized in that, include: The membrane potential of the current cycle is compared with a preset threshold. If the membrane potential is less than the preset threshold, the prediction is successful and the system enters the fast prediction mode of the next time window. If the membrane potential of the current cycle is greater than or equal to the preset threshold, the prediction fails and the system switches to the normal calculation mode. The input pulse is accumulated and calculated in single time steps, and it is determined whether to fire a pulse.

12. A processing apparatus supporting a spiking neural network, characterized in that, include: The in-memory computing device according to any one of claims 1-9.

13. The processing apparatus supporting a spiking neural network according to claim 12, characterized in that, Also includes: The input buffer module and the output buffer module are used for processing. The processing input cache module is connected to the in-memory computing array, and the processing input cache module is used to access off-chip weights and input pulses; The processing output buffer module is connected to the control module, and the processing output buffer module is used to buffer the pulses output by the control module.

14. A tile module, characterized in that, include: The processing apparatus supporting a spiking neural network as described in any one of claims 12-13.

15. A memory computing accelerator supporting spiking neural networks, characterized in that, include: A tile array comprising a plurality of tile modules as described in claim 14.

16. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the memory computing method as described in any one of claims 10 to 11.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the storage method as described in any one of claims 10 to 11.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the storage method as described in any one of claims 10 to 11.

Images