Multi-precision in-memory computing circuit with balanced output bit width, memory and chip

Abstract

The invention belongs to the technical field of in-memory computing, and discloses a multi-precision in-memory computing circuit with balanced output bit width, a memory and a chip, which are characterized in that through the collaborative change of a broadcast mode of an input side and an accumulation mode of an output side, under different computing precisions, the multi-precision in-memory computing circuit with balanced output bit width is obtained; the external interface bandwidth (the number of ADCs / the number of adders) of the physical array is kept constant. According to the invention, the contradiction between high bandwidth demand in a low-precision mode and low bandwidth demand in a high-precision mode in a multi-precision in-memory computing architecture is solved, the idle waste of the output bandwidth during high-precision computing is avoided, the relatively balanced bit width of an output interface is realized, and the area cost of a chip can be greatly reduced.

Description

Technical Field

[0001] This invention belongs to the field of in-memory computing technology, and in particular relates to a multi-precision in-memory computing circuit, memory and chip with balanced output bit width. Background Technology

[0002] With the development of artificial intelligence (AI) and the Internet of Things (IoT), edge devices are placing increasingly higher demands on the energy efficiency of neural network computing. Compute-in-Memory (CIM) technology, by performing analog or digital multiply-accumulate operations (MAC) directly within the memory array, alleviates the "memory wall" bottleneck and has become a research hotspot.

[0003] In practical applications, different neural network layers or different application scenarios have different requirements for data precision (e.g., INT4, INT8, INT16). Existing technical solutions typically use "bit-level parallel weights" or "multi-column expansion" to support multiple precisions. For example, when designing chips, to be compatible with the high throughput of low precision (e.g., INT4), the output bus bandwidth is often configured according to the maximum output data volume of INT4. This leads to a bandwidth-precision mismatch problem. That is, when the system switches to high precision mode (e.g., INT16), because multiple columns are combined into a single result, the actual effective data output volume is greatly reduced, causing most of the system output bandwidth designed according to the maximum requirement to be idle, resulting in wasted area and power consumption. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-precision in-memory computing circuit, memory, and chip with balanced output bit width to solve the problems existing in the prior art.

[0005] In a first aspect, to achieve the above objective, the present invention provides a multi-precision in-memory computing circuit with a balanced output bit width, comprising:

[0006] A storage computing array comprising multiple columns of bit lines, consisting of multiple basic reconstruction groups, each basic reconstruction group comprising multiple columns of physical units, the physical units being used to store weight data and perform multiply-accumulate operations;

[0007] A configurable input module is connected to the input terminal of each physical unit and is used to send the input vector into each physical unit according to different mode control signals.

[0008] A general-purpose accumulation readout module is connected to the output terminal of each physical unit and is used to perform in-situ accumulation or shift accumulation on the calculation results of each physical unit to obtain an output stream.

[0009] Optionally, the configurable input module has a broadcast mode, which can selectively map input data to a single column or multiple columns. The broadcast mode includes: broadcasting 1 group of data when INT4 is active, broadcasting 2 groups of data when INT8 is active, and broadcasting 4 groups of data when INT16 is active.

[0010] Optionally, the general-purpose accumulator readout module includes a shifter, an adder, and a multiplexer (MUX).

[0011] Optionally, the configurable input module and the general-purpose accumulation readout module execute different configurations according to different mode control signals, specifically including:

[0012] Low-precision mode: The configurable input module disables the broadcast function according to the low-precision mode signal and sends different input vectors to different physical units independently. Each physical unit performs independent multiplication and addition operations. The general accumulation readout module adjusts to the in-situ accumulation mode according to the low-precision mode signal, adds the calculation results of each column of physical units, and obtains an output stream.

[0013] Medium Precision Mode: The configurable input module enables two-column broadcasting based on the medium precision mode signal, broadcasting the first input vector to the first and second physical units, and the second input vector to the third and fourth physical units. The weights are split into high 4 bits and low 4 bits, which are stored in the two adjacent physical units respectively. The general accumulation readout module shifts and accumulates the calculation results of the first and second physical units based on the medium precision mode signal to obtain the first shift accumulation result, and shifts and accumulates the calculation results of the third and fourth physical units to obtain the second shift accumulation result. The first and second shift accumulation results are then added in place to obtain an output stream.

[0014] High-precision mode; the configurable input module enables 4-column broadcasting based on the high-precision mode signal, sending the same input vector into four physical units simultaneously, and the weights are split into four groups; the general accumulation readout module is configured as a multi-level shift accumulation mode, and the output results of each column are shifted left by 0, 4, 8, and 12 bits respectively according to the weight's bit order before accumulation to obtain an output stream.

[0015] In a second aspect, the present invention provides a memory including a multi-precision in-memory computing circuit with balanced output bit width as described in the first aspect.

[0016] Thirdly, the present invention provides a chip including the memory described in the second aspect.

[0017] The technical effects of this invention are as follows:

[0018] This invention resolves the contradiction between the large bandwidth requirement in low-precision mode and the small bandwidth requirement in high-precision mode in multi-precision in-memory computing architectures, avoids the waste of idle output bandwidth during high-precision computing, achieves a more balanced output interface bit width, and can significantly reduce chip area cost. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This is a schematic diagram of the low-precision mode in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of the medium precision mode in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the high-precision mode in an embodiment of the present invention. Detailed Implementation

[0024] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0025] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0026] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.

[0027] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] Example 1

[0030] like Figure 1 - Figure 3 As shown, this embodiment provides a multi-precision in-memory computing circuit with balanced output bit width, including:

[0031] A storage computing array comprising multiple columns of bit lines, consisting of multiple basic reconstruction groups, each basic reconstruction group comprising multiple columns of physical units, the physical units being used to store weight data and perform multiply-accumulate operations;

[0032] A configurable input module is connected to the input terminal of each physical unit and is used to send the input vector into each physical unit according to different mode control signals.

[0033] A general-purpose accumulation readout module is connected to the output terminal of each physical unit and is used to perform in-situ accumulation or shift accumulation on the calculation results of each physical unit to obtain an output stream.

[0034] This embodiment achieves a constant external interface bandwidth (number of ADCs / number of adders) for the physical array under different computational accumulator levels by coordinating the broadcast mode on the input side and the accumulation mode on the output side. This differs from traditional solutions that only provide simple multi-precision support but have variable output bandwidth.

[0035] This embodiment proposes an in-memory computing circuit architecture and data processing method that supports multiple precision levels, including a strategy combining input broadcast column configuration and dynamic column aggregation. For low precision, spatial parallelism is used for independent computation, and multiple sets of results are directly accumulated at the output, reducing the output bit width (All-reduce). For high precision, a broadcast mechanism is used to distribute input data to multiple columns, and the results from these multiple columns are shifted and merged at the output, thereby ensuring that the data bandwidth output from the array block to subsequent circuits remains balanced regardless of the precision.

[0036] The device in this embodiment mainly comprises three parts: a reconfigurable input driving module, a storage computing array, and a general-purpose accumulation readout module.

[0037] The reconfigurable input driver module can, based on control signals, select to independently send different input vectors to different columns, or broadcast the same input vector to two, four, or more adjacent columns. The storage and computation array stores weight data and performs multiplication and addition operations.

[0038] The general-purpose accumulator readout module, connected to the array output, contains a configurable adder tree and shifters. It can select between in-place accumulation (for low-precision multi-channel merging) or shift-and-add (for high-precision bit-width restoration) depending on the precision mode.

[0039] In terms of specific working mode, this solution uses every 4 columns of physical units as a basic reconfiguration group, and achieves unified output of the following three modes through configuration.

[0040] Mode 1 is a low-precision mode (such as INT4-INT4), such as Figure 1 As shown, the input driver module disables broadcasting. Four different input vectors (In1, In2, In3, In4) are independently fed into four reconstruction groups. Each of the four columns undergoes an independent INT4 multiply-add operation, and the general accumulator is configured for in-situ accumulation mode. It directly adds the results of these four columns (because they represent different output channels, but in this architecture, to match bandwidth, we aggregate the local sums of the four channels). These four reconstruction groups produce a single aggregated output stream.

[0041] Mode 2 is a medium-precision input configuration (such as INT8-INT8), such as... Figure 2 As shown, the input driver module enables two-column broadcasting. Specifically, input vector In1 is broadcast to columns 1 and 2; input vector In2 is broadcast to columns 3 and 4. The 8-bit weight is split into high 4 bits and low 4 bits, stored in two adjacent columns respectively. The general accumulator is configured in mixed accumulation mode. The results of columns 1 and 2 are shifted and accumulated (O = O). c1 << 4 + O c2 The first two columns are combined into one INT8 result. The third and fourth columns are similarly combined into another INT8 result. Finally, these two INT8 results are added in place, so that the four columns ultimately maintain the bandwidth usage of one standard output stream.

[0042] Mode 3 is a high-precision input configuration (such as INT16-INT16), such as... Figure 3As shown, the input driver module enables broadcasting across four columns. That is, the same input vector In1 is simultaneously fed into columns 1, 2, 3, and 4, and the 16-bit weight is split into four groups of 4 bits, distributed across the four columns. The general accumulator is configured in multi-stage shift-accumulation mode. The output results of the four columns are shifted left by 0, 4, 8, and 12 bits respectively, based on the weight's significance, before being accumulated, executing O=O. C1 +(O C2 <<4)+(O C3 <<8)+(O C4 <<12), the 4 columns are reconstructed and combined into a single INT16 output result.

[0043] Core circuit logic (general-purpose accumulator design): This module contains multi-stage adders and multiplexers (MUX). When the control signal is INT4, the shift logic is bypassed (Shiftamount=0), executing O=O. C1 +O C2 +O C3 +O C4 When the control signal is INT16, the shift logic is activated, executing O=O. C1 +(O C2 <<4)+(O C3 <<8)+(O C4 <<12). This design avoids designing separate addition trees for different precisions, saving hardware overhead.

[0044] In this embodiment, regardless of whether the front-end calculation precision is 4-bit, 8-bit, or 16-bit, the data bandwidth (Output Width) from the memory array output to the digital logic section remains relatively consistent (e.g., one number is listed every four bits). This greatly simplifies the design complexity of subsequent digital processing circuits (such as buffers and activation units).

[0045] This embodiment only requires changing the broadcast configuration of the input driver and the shift control signal of the accumulator to dynamically switch the precision without changing the hardware connection.

[0046] In summary, this embodiment resolves the contradiction between the large bandwidth requirement in low-precision mode and the small bandwidth requirement in high-precision mode in multi-precision in-memory computing architecture, avoids the waste of idle output bandwidth during high-precision computing, achieves a more balanced output interface bit width, and can significantly reduce chip area cost.

[0047] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-precision in-memory computing circuit with balanced output bit width, characterized in that, include: A storage computing array, which is composed of multiple basic reconstruction groups, each of which includes multiple columns of physical units, the physical units being used to store weight data and perform multiply-accumulate operations; A configurable input module is connected to the input terminal of each physical unit and is used to send the input vector into each physical unit according to different mode control signals. A general-purpose accumulation readout module is connected to the output terminal of each physical unit and is used to perform in-situ accumulation or shift accumulation on the calculation results of each physical unit to obtain an output stream.

2. The multi-precision in-memory computing circuit with balanced output bit width according to claim 1, characterized in that, The configurable input module is configured with a broadcast mode, which is used to broadcast the same input vector to multiple adjacent columns of physical units.

3. A multi-precision in-memory computing circuit with balanced output bit width according to claim 2, characterized in that, The general-purpose accumulator readout module includes a shifter, a multi-stage adder, and a multiplexer.

4. A multi-precision in-memory computing circuit with balanced output bit width according to claim 3, characterized in that, The configurable input module and the general-purpose accumulation readout module execute different configurations according to different mode control signals, specifically including: Low-precision mode: The configurable input module disables the broadcast function according to the low-precision mode signal and sends different input vectors to different physical units independently. Each physical unit performs independent multiplication and addition operations. The general accumulation readout module adjusts to the in-situ accumulation mode according to the low-precision mode signal, adds the calculation results of each column of physical units, and obtains an output stream. Medium Precision Mode: The configurable input module enables two-column broadcasting based on the medium precision mode signal, broadcasting the first input vector to the first and second physical units, and the second input vector to the third and fourth physical units. The weights are split into high 4 bits and low 4 bits, which are stored in the two adjacent physical units respectively. The general accumulation readout module shifts and accumulates the calculation results of the first and second physical units based on the medium precision mode signal to obtain the first shift accumulation result, and shifts and accumulates the calculation results of the third and fourth physical units to obtain the second shift accumulation result. The first and second shift accumulation results are then added in place to obtain an output stream. High-precision mode; the configurable input module enables 4-column broadcasting based on the high-precision mode signal, sending the same input vector into four physical units simultaneously, and the weights are split into four groups; the general accumulation readout module is configured as a multi-level shift accumulation mode, and the output results of each column are shifted left by 0, 4, 8, and 12 bits respectively according to the weight's bit order before accumulation to obtain an output stream.

5. A memory, characterized in that, The invention includes a multi-precision in-memory computing circuit with a balanced output bit width, as described in any one of claims 1-4.

6. A chip, characterized in that, Includes the memory described in claim 5.

Images