Low-power-consumption digital in-memory calculation circuit based on dynamic row and column sparse configuration, memory and chip

Abstract

The invention belongs to the technical field of digital in-memory computing, and discloses a low-power-consumption digital in-memory computing circuit based on dynamic row and column sparse configuration, a memory and a chip, which are characterized in that in a column dimension, 2-bit configuration signals are introduced, each column is supported to be independently configured into three modes of normal computing, weight reuse or full idle, and in a row dimension, the 2-bit configuration signals are used for configuring the 2-bit configuration signals into three modes of normal computing, weight reuse or full idle; a dynamic sparse gating mechanism is integrated, an input zero value or an unmapped row can be detected in real time, then a word line driving signal is directly pulled down to prohibit data loading, a connection switch of an isolation local bit line (LBL) and a global bit line (GBL) is controlled, and invalid coupling power consumption caused by global signal fluctuation is prevented; according to the method, the array power consumption strictly depends on the actual effective calculation amount instead of the physical array size through a software-hardware combined dynamic configuration strategy, so that the dynamic and static power consumption is remarkably reduced while high-performance calculation is ensured.

Description

Technical Field

[0001] This invention belongs to the field of digital in-memory computing technology, and in particular relates to a low-power digital in-memory computing circuit, memory and chip based on dynamic row and column sparse configuration. Background Technology

[0002] Digital Domain In-Memory (DCIM) architecture is widely used in high-performance AI inference chips due to its high energy efficiency and parallelism. However, in practical applications, neural network algorithms generally exhibit significant data sparsity, meaning that the input activation values ​​or weights contain a large number of zero values, and the actual matrix size is often smaller than the physical array size when the operator is mapped to the in-memory computing array (Mapping Mismatch). Existing DCIM architectures face significant energy efficiency bottlenecks when handling such sparse matrices or partial mapping scenarios: on the one hand, unused rows or columns still generate clock flips, global bit line (GBL) precharges, or invalid memory cell read operations, resulting in huge unnecessary power consumption; on the other hand, in continuous computation scenarios where weights remain unchanged, existing circuits lack flexible reuse mechanisms, often repeatedly performing local loading of weights, causing unnecessary memory read power consumption. In summary, existing technologies have many shortcomings, and there is an urgent need for a circuit architecture that can flexibly and finely control the row and column operating modes according to the mapping size and data sparsity to minimize invalid flips and static power consumption. Summary of the Invention

[0003] The purpose of this invention is to provide a low-power digital in-memory computing circuit, memory, and chip based on dynamic row and column sparse configuration, so as to solve the problems existing in the prior art.

[0004] In a first aspect, the present invention provides a low-power digital in-memory computing circuit based on dynamic row-column sparse configuration, comprising:

[0005] A storage computing array, comprising multiple rows and columns of computing units for performing multiply-accumulate operations;

[0006] A column configuration control module is used to receive configuration signals and independently control the working mode of each column calculation unit.

[0007] A row sparsity control module is used to control the access permissions of each row calculation unit according to the input data status.

[0008] A global bit line (GBL) management circuit that dynamically controls the precharge operation of bit lines in response to column configuration and row sparse signals.

[0009] Optionally, the column configuration control module supports configuring the column unit to one of the following three operating modes via a 2-bit control signal:

[0010] First working mode (normal mode): Enables storage data reading, enables computation logic, and enables bit line precharging.

[0011] Second working mode (weighted reuse mode): Disallows reading stored data, retains data from the previous cycle, enables computation logic, and disables bit line precharging.

[0012] Third working mode (idle mode): Disallows reading stored data, sets the computation logic input to zero, and disables bit line precharging.

[0013] Optionally, in the third mode, the circuit forces the operands entering the dot product unit to be replaced with zero to block the logical flip of the addition tree.

[0014] Optionally, the row sparsity control module is configured to: for zero-input or unmapped rows, directly pull down the word line drive signal through control logic to prevent data from being loaded from the memory into the ALU buffer; disconnect the switch between the local bit line (Local BL) and the global bit line (Global BL) to completely eliminate the pre-charging and transmission power consumption of that row.

[0015] Optionally, when the size of the mapping matrix is ​​smaller than the physical size of the array, the columns not covered by the mapping are configured as the third mode (idle mode), and the read control signal corresponding to the row not covered by the mapping is disconnected from the bit line switch.

[0016] In a second aspect, the present invention provides a memory including a low-power digital in-memory computing circuit based on a dynamic row-column sparse configuration as described in the first aspect.

[0017] Thirdly, the present invention provides a chip including the memory described in the second aspect.

[0018] The technical effects of this invention are as follows:

[0019] This invention significantly reduces the dynamic power consumption of digital in-memory computing circuits by providing fine-grained column-based configuration modes and row-based gating mechanisms, achieving deep low-power management of the circuit at the computation, storage read, and interconnect transmission levels. Specifically, this invention eliminates redundant SRAM read power consumption through a "weighted reuse mode," and completely eliminates invalid arithmetic logic unit flipping and long-line transmission power consumption through a "fully idle mode" and "row-based gating." It can perfectly adapt to mapping mismatch scenarios where the matrix size is smaller than the array size, as well as scenarios with highly sparse inputs or weights. Its significant advantage lies in its ability to precisely disable the pre-charge operation of the global bit line (GBL) for inactive rows and columns, greatly saving the most energy-consuming interconnect charging and discharging overhead in the array. This invention is mainly applied in the field of high-performance, low-power integrated circuit design, and is particularly suitable for high-computing-power artificial intelligence inference chips with stringent energy efficiency requirements. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a schematic diagram of the column-based sparsity control mechanism in an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the row-based sparsity control mechanism in an embodiment of the present invention. Detailed Implementation

[0024] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0025] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0026] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.

[0027] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] Example 1

[0030] like Figure 1 - Figure 2 As shown, this embodiment provides a low-power digital in-memory computing circuit based on dynamic row and column sparse configuration. The main technical problem it solves is how to provide a flexible and fine-grained control architecture at the hardware level to minimize invalid flips and static power consumption caused by sparse data and non-full-size mapping. The specific architecture includes:

[0031] A storage computing array comprising multiple computing units arranged in a row-column pattern; the storage computing array is configured to perform multiply-accumulate operations;

[0032] The column configuration control module is configured to receive configuration signals and independently control the working mode of each column calculation unit according to the configuration signals.

[0033] The row sparsity control module is configured to control the access permissions of each row of computation units based on the state of the input data;

[0034] The global bitline management circuit is configured to dynamically control the precharge operation of the bitlines based on the control states of the column configuration control module and the row sparse control module.

[0035] This embodiment provides a fine-grained column-based configuration mode and a row-based gating mechanism to achieve deep low-power management of the circuit at the computation, storage retrieval, and interconnection transmission levels, thus solving the problem of power waste in digital in-memory computing circuits when processing sparse data and non-full-size matrix mapping.

[0036] This embodiment implements column-wise configuration and row-wise sparsity gating.

[0037] Multi-mode configuration by column: In this embodiment, a 2-bit configuration signal MODE_SEL[1:0] is introduced for each column calculation unit (DCIM COL) to configure each DCIM calculation unit into different working modes, and supports state switching, specifically including:

[0038] Normal computation mode (normal execution of storage read and multiply-accumulate operations, used for regular dense matrix multiplication calculations): Enable local storage cell read logic, load weights into ALU Buffer; enable multiplier and adder tree; enable GBL precharge circuit.

[0039] Weight Reuse Mode (Calculation Only / Weight Reuse, which performs calculations but does not read, suitable for scenarios in Convolutional Neural Networks (CNNs) where weights are shared or the same row of weights is reused multiple times): This mode prevents new data from being loaded into the ALU Buffer from local memory, reuses the weight data from the previous cycle latched in the ALU Buffer, and continues to perform multiplication and addition operations with the held weights on the input data. At the same time, the pre-charging of the column's Global Bit Line (GBL) is gated. This mode saves data transfer power (load power) from memory to buffer. The global bit line (GBL) of the column is not pre-charged and uses the existing level for transmission or holding, which greatly reduces interconnect power consumption.

[0040] Full Idle Mode (Full Power Gating mode, no computation or reading, suitable for scenarios where the column is not mapped (matrix width is less than array width) or the column weight is pruned to all 0): Disallows memory reads and GBL precharge; simultaneously masks input data (sets it to zero), forcibly replacing the input data sent to the Dot-product Unit with 0 (Input Masking), stopping the adder tree flipping. Because the input is 0, subsequent multipliers and adder tree logic gates do not flip, reducing computational power consumption to zero. GBL precharge and memory read operations are also disabled.

[0041] In this feasible embodiment, the row-wise sparsity control mechanism performs the following operations for zero-value (zero activation) or unmapped rows in the input vector:

[0042] Input detection and drive gating: When a row of input data is detected to be 0 or the row is not enabled, a row masking signal is generated.

[0043] Word line drive disabled: Forces the read control signal of this row to be pulled low, prohibits pulse excitation of the word line of the memory cell in this row, and the ALU Buffer does not update the data.

[0044] Layered bit line isolation: Controls the disconnection of the isolation switch between the local bit line (LBL) and the global bit line (GBL).

[0045] Through the above operations, even if the level of GBL fluctuates during other row operations, the LBL of this row will not be charged or discharged, thus achieving zero-power standby at the row level.

[0046] In practice, when the mapped matrix size is smaller than the physical size of the CIM array, the row and column control logic described above can be used to configure all unused rows and columns at the array edge to a shutdown state (rows disconnected switch, columns configured to 00), so that the actual power consumption is strictly determined by the effective computational load rather than by the physical area of ​​the array.

[0047] This embodiment is primarily applied to AI inference and edge computing scenarios requiring extreme energy efficiency, particularly for the hardware design of various sparse neural network accelerators. In practical applications, the computational load of neural networks often exhibits highly dynamic characteristics. For example, in convolutional layer computation, weight data often needs to remain constant over multiple computation cycles (Weight Stationary). The "weight reuse mode" provided in this embodiment can avoid repeatedly reading SRAM in such scenarios, minimizing the power consumption of each column. Furthermore, with the widespread adoption of model pruning techniques, the weight matrix contains a large number of zero values, or when the actual matrix size is smaller than the physical size of the hardware array (Mapping Mismatch), the "full idle mode" and "row-by-row gating mechanism" in this embodiment can configure unused or sparse regions at the array edges into a deep sleep state. This flexible control logic severs the connection between local and global bit lines, ensuring that chip power consumption is strictly determined by the actual effective computational load, rather than by the physical area of ​​the hardware. This allows for optimal energy efficiency under various complex AI inference loads, demonstrating broad market application prospects.

[0048] In summary, this embodiment eliminates redundant SRAM read power consumption through the "weight reuse mode"; and eliminates invalid arithmetic logic unit flipping and long-line transmission power consumption through the "full idle mode" and "row-by-row gating". This embodiment can perfectly adapt to scenarios where the matrix size is smaller than the array size (Mapping Mismatch) and the input / weights are highly sparse. For inactive columns / rows, it accurately shuts down the pre-charge operation of the global bit line (GBL), significantly saving the most energy-consuming interconnect charging and discharging overhead in the array.

[0049] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A low-power digital in-memory computing circuit based on dynamic row-column sparse configuration, characterized in that, include: A storage computing array comprising multiple computing units arranged in a row-column pattern; The storage computing array is configured to perform multiply-accumulate operations; The column configuration control module is configured to receive configuration signals and independently control the working mode of each column calculation unit according to the configuration signals. The row sparsity control module is configured to control the access permissions of each row of computation units based on the state of the input data; The global bitline management circuit is configured to dynamically control the precharge operation of the bitlines based on the control states of the column configuration control module and the row sparse control module.

2. The low-power digital in-memory computing circuit based on dynamic row and column sparse configuration according to claim 1, characterized in that, The configuration signal is a 2-bit column configuration signal.

3. The low-power digital in-memory computing circuit based on dynamic row and column sparse configuration according to claim 2, characterized in that, The column configuration control module controls the operating mode of each column calculation unit by introducing a 2-bit column configuration signal. The operating modes include: First operating mode: Activates stored data reading, calculation operations, and bit line precharge function through configuration signals; Second operating mode: Activate computation operations by configuring signals and disable data reading from storage and bit line precharging; The third operating mode: By configuring signals to disable stored data reading and bit line precharging, and setting the computation logic input to zero, the logic flipping of the addition tree is blocked.

4. The low-power digital in-memory computing circuit based on dynamic row and column sparse configuration according to claim 3, characterized in that, When the input data of any row of computing units is zero or the row of computing units is not mapped, the memory read control signal of the row of computing units is disabled by the row sparse control module, and the connection switch between the local bit line and the global bit line corresponding to the row of computing units is disconnected to eliminate pre-charging and data loading power consumption.

5. A low-power digital in-memory computing circuit based on dynamic row and column sparse configuration according to claim 4, characterized in that, When the size of the mapping matrix is ​​smaller than the physical size of the storage computing array, the column computing units not covered by the mapping are configured to the third working mode through the column configuration control module, and the memory read control signals and connection switches corresponding to the row computing units not covered by the mapping are disabled through the row sparsity control module.

6. A memory, characterized in that, The invention includes a low-power digital in-memory computing circuit based on a dynamic row-column sparse configuration, as described in any one of claims 1-5.

7. A chip, characterized in that, Includes the memory described in claim 6.

Images