A multi-bit signed in-memory computation weighting circuit

Abstract

This invention provides a multi-bit signed in-memory computation weighting circuit, comprising: a multi-bit in-memory computation unit, an analog domain weighting structure, and a mapping network; the multi-bit in-memory computation unit performs multiplication of input activation values ​​and weights through a combination of signed and non-signed in-memory computation units, generating a computation result; the analog domain weighting structure weights the computation result, performing multiplication and accumulation of input activation values ​​and weights in the analog domain based on charge conservation, and generating a corresponding digital result after an analog-to-digital conversion; based on the digital result, multiple switching operations are performed through a preset array structure and mapping network to complete the traversal and sliding of preset convolution kernels, realizing convolution operation on the entire feature map. This invention solves the problems of high complexity, high latency, and high power consumption in existing in-memory computations.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and in particular to a multi-bit signed memory in-memory weighted computing circuit. Background Technology

[0002] In the era of artificial intelligence, neural networks are widely deployed in edge devices for applications such as image classification, object detection, and natural language processing. The multiply-accumulate (MAC) operation, which involves input activation values ​​and weights, is the core operation for neural network inference, generating a large amount of data. In the traditional von Neumann architecture, instructions and data are first written to memory. Then, the control unit receives instructions via a control bus, sends the data from memory to the computing unit via a data bus, and finally writes the calculation result back to memory via the data bus. Because the storage unit and the computing unit are physically isolated, this method requires frequent data transfer between them, limiting the speed of data processing. Furthermore, the data transfer process also consumes significant power. To address the "memory wall" and "power wall" problems in the traditional von Neumann architecture, researchers have proposed the in-memory computing (CIM) architecture, which integrates the storage unit and the computing unit, reducing power consumption and latency.

[0003] Existing in-memory computing circuits lack efficient solutions for performing multi-bit signed number (MAC) calculations. One approach involves calculating the multiplication of a single-bit input activation value and a multi-bit privileged value in the analog domain, performing multiple analog-to-digital (AD) conversions, and finally shifting and adding the results in the digital domain to obtain the MAC result of the multi-bit activation value and the multi-bit privileged value. However, each MAC calculation requires multiple AD conversions. Another approach involves converting the multi-bit input activation value to an analog value using a digital-to-analog converter (DAC), performing the MAC calculation in the analog domain, and finally obtaining the digital output result through AD conversion. Both the multiple AD conversions in the first approach and the DAC conversion of the multi-bit digital input to the analog domain in the second approach result in significant latency and power consumption. Summary of the Invention

[0004] This invention provides a multi-bit signed in-memory computation weighting circuit to solve the problems of high complexity, high latency, and high power consumption in existing in-memory computations.

[0005] This invention provides a multi-bit signed in-memory computation weighting circuit, comprising: a multi-bit in-memory computation unit, an analog domain weighting structure, and a mapping network; The multi-bit in-memory computation unit completes the multiplication calculation of the input activation value and weight by combining the sign bit in-memory computation unit and the non-sign bit in-memory computation unit, and generates the calculation result; The calculation results are weighted by the analog domain weighting structure. Based on charge conservation, the multiplication and accumulation calculation of the input activation value and the weight is completed in the analog domain. After one analog-to-digital conversion, the corresponding digital result is generated. Based on the digital results, multiple switching operations are performed through a preset array structure and mapping network to complete the traversal and sliding of the preset convolution kernel, thereby realizing the convolution operation on the entire feature map.

[0006] According to the present invention, a multi-bit signed in-memory computation weighting circuit is provided, wherein the multi-bit in-memory computation unit includes a multi-bit in-memory computation unit with fixed weights. The fixed-weight multi-bit computation unit includes one in-memory computation unit and multiple non-sign-memory computation units. The unsigned activation value is serially input into a multi-bit memory computation unit with fixed weights. The weights are in original code form. The highest bit of the activation value is input first. The multiplication of four privileged values ​​and one bit of activation value is completed at once, i.e., 4w1a, and the calculation result is generated.

[0007] According to the present invention, a multi-bit signed in-memory computation weighting circuit is provided, wherein the multi-bit in-memory computation unit further includes a multi-bit in-memory computation unit with a fixed activation value; The multi-bit in-memory computation unit with a fixed activation value includes: multiple non-signed bit in-memory computation units and two switches sw0 and sw1; The signed weights are serially input into the multi-bit internal calculation unit with the activation value fixed. The most significant bit is input first, and the sign bit and the unsigned bit of the weights act simultaneously to complete the multiplication calculation of one privileged value and four-bit activation value, i.e., 1w4a, and generate the calculation result.

[0008] According to the present invention, a multi-bit signed in-memory computation weighting circuit is provided, wherein the signed bit in-memory computation unit includes a standard 6T-SRAM cell and two switches sw0 and sw1, wherein sw0 and sw1 are controlled by ia and iab; During the calculation phase, when the sign bit of the four privileged values ​​is equal to 1, the weight is negative, q=1, qb=0; when ia=1, switch sw0 is closed and switch sw1 is open, ia_sign=qb=0; when ia=0, switch sw0 is open and side switch sw1 is closed, ia_sign=vdd_2, that is, vdd / 2=450mV. After the input activation value ia interacts with the in-memory computation unit of the storage weight sign bit, the result ia_sign is obtained. ia_sign is then connected to the input in of the in-memory computation unit of the non-sign bit, i.e., ia_sign = in.

[0009] According to the present invention, a multi-bit signed in-memory computation weighting circuit is provided, wherein the unsigned bit in-memory computation unit includes a standard 6T-SRAM, two switches sw0 and sw1 and a capacitor, wherein sw0 and sw1 are composed of q and qb; During the reset phase, the upper plate of the capacitor is reset to vdd_2, i.e., vdd / 2 = 450mV, the voltage at node 4w1a is reset to vdd / 2, and the lower plate of the capacitor is clamped to a certain voltage value. When the voltage of the upper plate of the capacitor changes, due to charge coupling, the voltage value of the lower plate of the capacitor will also change accordingly, and thus the voltage value at node 4w1a will also change accordingly. During the calculation phase, when the non-sign bit of the weight is 0, q=0, qb=1, switch sw0 is open, switch sw1 is closed, and the voltage on the upper plate of the capacitor is still vdd / 2; when the non-sign bit of the weight is 1, q=1, qb=0, switch sw0 is closed, switch sw1 is open, and the voltage on the upper plate of the capacitor is equal to in, with a value of vdd, vdd / 2, or gnd.

[0010] According to the present invention, a multi-bit signed in-memory computation weighting circuit is provided, wherein the analog domain weighting structure includes: a C2C capacitor weighting array, an adjustable gain amplifier, and a switched proportional capacitor weighting array; The highest bit of the activation value a[3:0] is input first, multiplied by the four privileged values, and the resulting 4w1a voltage value is used to charge the capacitors 8C, 4C, 2C, and C in sequence. The results of the four 4W1A calculations are used to charge the switching proportional capacitor weighted array in sequence, thereby completing the weighting of the 4W4A calculation results. With sw0, sw1, sw2, and sw3 closed, k0 open and k1 closed, the lower plates of the capacitor are connected in parallel, allowing the charge to accumulate and be shared. Based on the principle of charge conservation, a calculation result of 4w4a is obtained for the lower plate of the capacitor.

[0011] According to the present invention, a multi-bit signed in-memory computation weighting circuit is provided, wherein the analog domain weighting structure further includes: a C2C capacitor weighting array, an adjustable gain amplifier, a switched proportional capacitor weighting array, and a charge sharing switch sw_share; The highest bit of the activation value a[3:0] is input first, multiplied by the four privileged values, and the resulting 4w1a voltage value is used to charge the capacitors 8C, 4C, 2C, and C in sequence. The results of the four 4W1A calculations are used to charge the switching proportional capacitor weighted array in sequence, thereby completing the weighting of the 4W4A calculation results. sw0, sw1, sw2, and sw3 are disconnected, sw_share is closed, and the upper plates of the capacitor are connected in parallel. Based on the principle of charge conservation, the calculated result of 4w4a is obtained on the upper plate of the capacitor.

[0012] According to the present invention, a multi-bit signed memory computation weighting circuit is provided, wherein the input activation value and weight are multiplied and accumulated in the analog domain based on charge conservation, and the four-bit activation value and four-bit privileged value are multiplied and accumulated in the analog domain, and the corresponding digital result is obtained after one analog-to-digital conversion.

[0013] According to the present invention, a multi-bit signed in-memory computation weighting circuit is provided, wherein the adjustable gain amplifier transfers the 4w1a node voltage from the C2C capacitor weighting array to the switched proportional capacitor weighting array, and amplifies the overall analog voltage range through the adjustable gain amplifier.

[0014] According to the present invention, a multi-bit signed memory in-memory computation weighting circuit is provided, wherein the digital result is used to perform multiple switching operations through a preset array structure and a mapping network to complete the traversal and sliding of a preset convolution kernel, thereby realizing the convolution operation on the entire feature map, including: Neural network mapping with fixed weights and neural network mapping with fixed activation values ​​are respectively constructed using corresponding array structures. The neural network mapping with fixed activation values ​​completes the traversal and sliding process of the convolution kernel through three switching operations, thereby realizing the convolution operation on the entire feature map.

[0015] This invention provides a multi-bit signed in-memory computation weighting circuit. Through a combination of signed and non-signed in-memory computation units, a multi-bit in-memory computation unit performs multiplication of input activation values ​​and weights, generating a computation result and realizing multi-bit signed number product calculation. The computation result is weighted by an analog domain weighting structure. Based on charge conservation, the multiplication and accumulation of input activation values ​​and weights are performed in the analog domain. After one analog-to-digital conversion, the corresponding digital result is generated. The analog domain weighting structure completes the MAC calculation of multi-bit activation values ​​and multi-bit privileged values ​​in the analog domain, obtaining the final digital output result through only one AD conversion, reducing latency and power consumption. Based on the digital result, multiple switching operations are performed through a preset array structure and mapping network to complete the traversal and sliding of preset convolution kernels, realizing convolution operations on the entire feature map. The circuit architecture supports various neural network mappings, including lightweight networks such as MobileNet, and has good applicability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a circuit diagram of a multi-bit signed memory in-memory weighted calculation circuit provided by the present invention.

[0018] Figure 2 This is a circuit diagram of the symbol memory computation unit provided by the present invention.

[0019] Figure 3 This is a circuit diagram of the non-signed bit memory computation unit provided by the present invention.

[0020] Figure 4 This is a circuit diagram of a multi-bit in-memory computing unit with fixed weights provided by the present invention.

[0021] Figure 5 This is a schematic diagram of the multiplication calculation of 4 privileged values ​​and 1 activation value and the connection of the C2C capacitor weighted array provided by the present invention.

[0022] Figure 6 This is a circuit diagram of a multi-bit in-memory computing unit with a fixed activation value provided by the present invention.

[0023] Figure 7 This is a schematic diagram of the multiplication calculation of 1-bit privileged value and 4-bit activation value and the connection of C2C capacitor weighted array provided by the present invention.

[0024] Figure 8 This is a circuit diagram of an analog domain weighted structure provided by the present invention.

[0025] Figure 9 The 4w1a provided by this invention <3> The calculation results are used to design an 8C charging circuit.

[0026] Figure 10 The 4w1a provided by this invention <2> The calculation results are used to design a 4C charging circuit diagram.

[0027] Figure 11 The 4w1a provided by this invention <1> The calculation results are used to design a 2C charging circuit diagram.

[0028] Figure 12 The 4w1a provided by this invention <0> The calculation results are used to charge circuit diagram C.

[0029] Figure 13 This is the analog-to-digital converter circuit diagram provided by the present invention.

[0030] Figure 14 This is a circuit diagram of an analog domain weighted structure for a multi-bit in-memory computation unit with fixed activation values, provided by the present invention.

[0031] Figure 15 This is another analog domain weighted structure circuit diagram provided by the present invention.

[0032] Figure 16 This is a schematic diagram of an array structure for a neural network mapping with fixed weights provided by the present invention.

[0033] Figure 17 This is a schematic diagram of a neural network mapping array structure with fixed activation values ​​provided by the present invention.

[0034] Figure 18 This is a schematic diagram of the switch switching when Ф1 is open and Ф2 and Ф3 are closed, provided by the present invention.

[0035] Figure 19 This is a schematic diagram of the switch switching when Ф2 is open and Ф1 and Ф3 are closed, provided by the present invention.

[0036] Figure 20 This is a schematic diagram of the switch switching when Ф3 is open and Ф1 and Ф2 are closed, provided by the present invention.

[0037] Figure 21 This is a circuit diagram of a multi-bit in-memory computing unit with fixed weights in a specific embodiment of the present invention.

[0038] Figure 22 This is a circuit diagram of an analog-to-digital converter provided in a specific embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0040] Existing in-memory computing circuits lack efficient solutions for performing multi-bit signed number MAC (multiplicative-accumulator) calculations. One solution involves calculating the multiplication of a single-bit input activation value and a multi-bit privileged value in the analog domain, performing multiple analog-to-digital (AD) conversions, and finally shifting and adding the results in the digital domain to obtain the MAC result of the multi-bit activation value and the multi-bit privileged value. This MAC calculation requires multiple AD conversions. Another solution involves converting the multi-bit input activation value to an analog value using a digital-to-analog converter (DAC), performing the MAC calculation in the analog domain, and finally obtaining the digital output result through AD conversion. Both the multiple AD conversions in the first solution and the DAC conversion of the multi-bit digital input to the analog domain in the second solution result in significant latency and power consumption.

[0041] The following is combined Figure 1The present invention describes a multi-bit signed in-memory computation weighting circuit, comprising: a multi-bit in-memory computation unit, an analog domain weighting structure, and a mapping network.

[0042] The multi-bit in-memory computation unit completes the multiplication of the input activation value and weight by combining the sign bit in-memory computation unit and the non-sign bit in-memory computation unit, and generates the computation result.

[0043] The calculation results are weighted by the analog domain weighting structure. Based on charge conservation, the multiplication and accumulation calculation of the input activation value and the weight is completed in the analog domain. After one analog-to-digital conversion, the corresponding digital result is generated.

[0044] Based on the digital results, multiple switching operations are performed through a preset array structure and mapping network to complete the traversal and sliding of the preset convolution kernel, thereby realizing the convolution operation on the entire feature map.

[0045] In this invention, the in-memory computation unit and the non-signed in-memory computation unit are first introduced.

[0046] Among them, reference Figure 2 One symbol-bit in-memory computing unit consists of one standard 6T-SRAM cell and two switches sw0 and sw1, which are controlled by ia and iab. The switches use, but are not limited to, a single MOS (NMOS or PMOS) switch or a CMOS transmission gate switch.

[0047] Calculation phase: When the sign bit of the privileged value is equal to 1, the weight is negative, q=1, qb=0; when ia=1, the left switch sw0 is closed and the right switch sw1 is open, ia_sign=qb=0; when ia=0, the left switch sw0 is open and the right switch sw1 is closed, ia_sign=vdd_2, that is, vdd / 2=450mV.

[0048] Referring to Table 1, the computation logic of the sign bit in-memory computation unit can be seen. After the input activation value ia interacts with the in-memory computation unit of the storage weight sign bit, the result ia_sign is obtained. ia_sign is connected to in of the unsigned bit in-memory computation unit (unsignCIM unit), i.e., ia_sign = in.

[0049] Table 1. Calculation Logic Table of the Computation Unit in the Symbolic Memory

[0050] refer to Figure 3One unsigned bit in-memory computing unit consists of a standard 6T-SRAM, two switches sw0 and sw1, and a capacitor. sw0 and sw1 are controlled by q and qb. The switches use, but are not limited to, a single MOS (NMOS or PMOS) switch or a CMOS transmission gate switch.

[0051] Reset Phase: The upper plate of the capacitor is reset to vdd_2, i.e., vdd / 2 = 450mV, the voltage at node 4w1a is reset to vdd / 2, and the lower plate of the capacitor is clamped to a certain voltage value. When the voltage of the upper plate of the capacitor changes, due to charge coupling, the voltage value of the lower plate of the capacitor will also change accordingly, and consequently, the voltage value at node 4w1a will also change accordingly.

[0052] Referring to Table 2, the calculation logic table of the non-sign bit in the memory calculation unit, during the calculation stage: when the non-sign bit of the weight is equal to 0, q=0, qb=1, the left switch sw0 is open, the right switch sw1 is closed, and the voltage on the upper plate of the capacitor is still vdd / 2; when the non-sign bit of the weight is 1, q=1, qb=0, the left switch sw0 is closed, the right switch sw1 is open, and the voltage on the upper plate of the capacitor is equal to in, and its value is vdd, vdd / 2 or gnd.

[0053] Table 2 Computational Logic Table of Non-Signed Memory Computation Units

[0054] The multi-bit in-memory computation unit includes a multi-bit in-memory computation unit with fixed weights and a multi-bit in-memory computation unit with fixed activation values.

[0055] The fixed-weight multi-bit computation unit includes one in-signed memory computation unit and multiple non-signed memory computation units.

[0056] The unsigned activation value is serially input into a multi-bit memory computation unit with fixed weights. The weights are in original code form. The highest bit of the activation value is input first. The multiplication of four privileged values ​​and one bit of activation value is completed at once, i.e., 4w1a, and the calculation result is generated.

[0057] A multi-bit in-memory computation unit with a fixed activation value; the multi-bit in-memory computation unit with a fixed activation value includes: multiple non-sign bit in-memory computation units and two switches sw0 and sw1.

[0058] The signed weights are serially input into the multi-bit internal calculation unit with the activation value fixed. The most significant bit is input first, and the sign bit and the unsigned bit of the weights act simultaneously to complete the multiplication calculation of one privileged value and four-bit activation value, i.e., 1w4a, and generate the calculation result.

[0059] In this invention, reference Figure 4 and Figure 5 A multi-bit internal computational unit with fixed weights, the weights being signed.

[0060] In traditional convolutional neural networks, the multi-bit privileged values ​​can be stored in such multi-bit in-memory computation units (CIM cells). Similarly, the pointwise convolution part of depthwise separable convolutions (mobilenets), with its fixed weights, can also be stored in such multi-bit in-memory computation units (CIM cells).

[0061] A multi-bit in-memory computation unit consists of one sign bit in-memory computation unit and three non-sign bit in-memory computation units, storing four privileged values. The highest bit (MSB) is the sign bit, and the last three bits are data bits. The data range represented by one multi-bit in-memory computation unit is -7 to +7 (1111 to 0111).

[0062] This is only one possible scenario. The number of unsigned CIM units can be determined according to actual needs. That is, one multi-bit CIM unit consists of one signed CIM unit and multiple unsigned CIM units. This embodiment is only for the purpose of more clearly illustrating the technical solution of the present invention.

[0063] The activation value (unsigned) is input serially, and the weights are in original code form. The most significant bit of the activation value is input first (MSB-first-in). The multiplication of 4 privileged values ​​and 1 activation value is completed in one step (4w1a). The result of this calculation is weighted and accumulated by the C2C capacitor weighting array that follows.

[0064] refer to Figure 6 and Figure 7 A multi-bit in-memory computation unit with fixed activation values, the depthwise convolution part of a depthwise separable convolution (mobilenet) has fixed activation values ​​and is stored in this multi-bit in-memory computation unit.

[0065] A multi-bit in-memory computation unit includes four unsigned in-memory computation units (unsigned CIM units), storing a 4-bit activation value (the activation value is unsigned), and two switches sw0 and sw1.

[0066] This is only one possible scenario. The number of unsigned CIM units can be determined according to actual needs. That is, one multi-bit CIM unit consists of multiple unsigned CIM units and two switches sw0 and sw1. This embodiment is only for the purpose of more clearly illustrating the technical solution of the present invention.

[0067] sw0 and sw1 are controlled by wi and wi~. The switches can be, but are not limited to, a single MOS (NMOS or PMOS) switch or a CMOS transmission gate switch, etc., and are all within the scope of protection of this patent.

[0068] The data range of the activation value represented by a single multi-bit in-memory computation unit is 0~15 (0000~1111). The weights (signed) are input serially, with the most significant bit input first (MSB-first-in). The sign bit and the unsigned bit of the weights act simultaneously. A multiplication calculation of 1 ratio of the privileged value and the 4-bit activation value is performed (1w4a). The result of 1w4a is weighted and accumulated through the subsequent C2C capacitor weighting array.

[0069] Current multiply-accumulate calculations involve performing single-bit multiplication or multi-bit multiplication in the analog domain, followed by multiple A / D conversions, and then shifting and adding in the digital domain to obtain the MAC result of the multi-bit activation value and the multi-bit privileged value. However, a single MAC calculation of the multi-bit activation value and the multi-bit privileged value requires multiple A / D conversions, which is slow and consumes a lot of power.

[0070] This invention proposes two analog domain weighted structures that can complete the MAC calculation of multi-bit activation values ​​and multi-bit privileged values ​​in only one AD conversion, which is fast, has low latency and low power consumption.

[0071] The analog domain weighted structure includes a C2C capacitor weighted array, an adjustable gain amplifier, and a switched proportional capacitor weighted array. The switches used are not limited to single MOS (NMOS or PMOS) switches or CMOS transmission gate switches, etc., and are all within the scope of this patent.

[0072] The number of capacitors in the C2C capacitor weighted array and the number of switching proportional capacitors can be set according to actual needs. This embodiment is only one possible case, and is only used to more clearly illustrate the technical solution of the present invention.

[0073] One type of analog domain weighting structure, reference Figure 8 Taking a multi-bit memory computing unit with fixed weights as an example, the highest bit (MSB) of the activation value a[3:0] is input first, multiplied by the privileged value of 4, and the resulting 4w1a voltage value should charge the capacitors 8C, 4C, 2C, and C in sequence.

[0074] refer to Figure 9 ,4w1a <3> (V3) When charging 8C, sw3 is closed, sw0, sw1, and sw2 are open, k0 is closed, and k1 is open. The charge Q3 = (V3 - Vcm) * 8C.

[0075] refer to Figure 10,4w1a <2> (V2) When charging to 4C, sw2 is closed, sw0, sw1, and sw3 are open, k0 is closed, and k1 is open. The charge Q2 = (V2 - Vcm) * 4C.

[0076] refer to Figure 11 ,4w1a <1> When charging to 2C using the calculation result (V1), sw1 is closed, sw0, sw2, and sw3 are open, k0 is closed, and k1 is open. The charge Q1 = (V1 - Vcm) * 2C.

[0077] refer to Figure 12 ,4w1a <0> When charging C according to the calculation result (V0), sw0 is closed, sw1, sw2, and sw3 are open, k0 is closed, and k1 is open. The amount of charge Q0 is equal to (V0 - Vcm) * C.

[0078] The results of the four 4W1A calculations are used to charge the weighted array of the proportional capacitors in sequence, thus completing the weighting of the 4W4A calculation results. When sw0, sw1, sw2, and sw3 are closed, k0 is open, and k1 is closed, the lower plates of the capacitors are connected in parallel, and the charge is accumulated and shared. Based on the principle of charge conservation, the 4W4A calculation result is obtained at the lower plate of the capacitor.

[0079] Qt = Q3 + Q2 + Q1 + Q0.

[0080] That is: (Vcm-Vt)*15C=(V3-Vcm)*8C+(V2- Vcm)*4C+(V1- Vcm)*2C+(V0- Vcm)*C.

[0081] Vt=V_4w4a.

[0082] △V = Vcm - Vt, where △V is the analog voltage value corresponding to the sum of the product of the actual 4-bit activation value and the 4-bit privileged value.

[0083] Thus, the MAC calculation for the 4-bit activation value and the 4-bit privileged value has been completed in the analog domain. The corresponding digital result can be obtained after one AD conversion. (Reference) Figure 13 and Figure 14 The analog domain weighting structure and operation process for multi-bit in-memory computation units with fixed activation values ​​are similar to those in the above embodiments.

[0084] Another analog domain weighted structure includes a C2C capacitor weighted array, an adjustable gain amplifier, a switched proportional capacitor weighted array, and a charge-sharing switch sw_share. The switch uses, but is not limited to, a single MOS (NMOS or PMOS) switch or a CMOS transmission gate switch.

[0085] refer to Figure 15The activation value a[3:0] highest bit (MSB) is input first, multiplied by the privilege value of 4, and the resulting 4w1a voltage value should charge the capacitors 8C, 4C, 2C, and C in sequence.

[0086] 4w1a <3> When charging 8C according to the calculation result of (V3), sw3 is closed, sw0, sw1, and sw2 are open, sw_share is open, and the amount of charge Q3 is equal to V3*8C.

[0087] 4w1a <2> When charging to 4C using the (V2) calculation result, sw2 is closed, sw0, sw1, and sw3 are open, sw_share is open, and the charge Q2 = V2 * 4C is generated.

[0088] 4w1a <1> When charging 2C using the (V1) calculation result, sw1 is closed, sw0, sw2, and sw3 are open, sw_share is open, and the charged amount Q1 = V1 * 2C.

[0089] 4w1a <0> When the (V0) calculation result charges C, sw0 is closed, sw1, sw2, and sw3 are open, sw_share is open, and the charged amount Q0 = V0 * C.

[0090] Finally, charge accumulation and sharing are performed. sw0, sw1, sw2, and sw3 are disconnected, sw_share is closed, and the upper plates of the capacitor are connected in parallel. Based on the principle of charge conservation, the calculated result of 4w4a is obtained on the upper plate of the capacitor.

[0091] Qt = Q3 + Q2 + Q1 + Q0.

[0092] That is: Vt*15C=V3*8C+V2*4C+V1*2C+V0*C.

[0093] Vt=V_4w4a.

[0094] Vt is the analog voltage value corresponding to the sum of the actual 4-bit activation value and the 4-bit privileged value.

[0095] Thus, the MAC calculation of the 4-bit activation value and the 4-bit privileged value has been completed in the analog domain, and the corresponding digital result can be obtained after one AD conversion.

[0096] In this invention, the adjustable gain amplifier transfers the 4w1a node voltage from the C2C capacitor weighted array to the switched proportional capacitor weighted array, and amplifies the overall analog voltage range through the adjustable gain amplifier.

[0097] Specifically, when the upper C2C capacitor-weighted array and the lower switched proportional capacitor-weighted array are directly cascaded, they will affect each other, causing changes in the capacitance value. This results in a deviation between the charged amount and the ideal situation, leading to inaccurate calculation results. The adjustable gain amplifier (amp) has a high input impedance and a low output impedance, effectively isolating the two capacitor-weighted arrays and preventing them from interfering with each other. This allows for better transfer of the 4W1A node voltage from the upper C2C capacitor-weighted array to the lower switched proportional capacitor-weighted array. When the data distribution range is small, the corresponding analog voltage value of the 4W1A node will be distributed within a small range around 450mV. The adjustable gain amplifier (amp) can amplify the overall analog voltage range, facilitating subsequent quantization and processing.

[0098] Neural network mapping with fixed weights and neural network mapping with fixed activation values ​​are respectively constructed using corresponding array structures. The neural network mapping with fixed activation values ​​completes the traversal and sliding process of the convolution kernel through three switching operations, thereby realizing the convolution operation on the entire feature map.

[0099] Specifically, refer to Figure 16 Considering a neural network mapping with fixed weights, the following array structure can be adopted: j multi-bit in-memory computation units with fixed weights constitute a group (referred to as a tile), equivalent to j channels of a multi-level privileged value; assuming the convolution kernel size is m*m, then m*m tiles are needed. To more clearly illustrate the technical solution of this invention, this embodiment adopts a 1*1*16 structure, that is, the convolution kernel size is 1*1 and the number of channels is 16, which can deploy the multi-level privileged value of pointwise convolution in the MobileNet network. i represents the pixel index. k represents the k-th convolution kernel, corresponding to the k-th channel of the output.

[0100] refer to Figure 17 Considering a neural network mapping with fixed activation values, the following array structure can be adopted: 3*16 multi-bit in-memory computation units with fixed weights and multiple sets of switches Ф1, Ф2, Ф3 constitute a tile. The feature map size is 3*16, the number of channels is 1, and multi-bit activation values ​​of depthwise convolution in the MobileNet network can be deployed. The convolution kernel size is 3*3.

[0101] The convolution kernel can be traversed and slid through using only three switch operations, achieving convolution operation on the entire feature map. Specifically: refer to Figure 18 Step 1: Ф1 is disconnected, Ф2 and Ф3 are closed: (Refer to...) Figure 19 Step 2: Ф2 is disconnected, Ф1 and Ф3 are closed; (Refer to...) Figure 20 Step 3: Ф3 is disconnected, Ф1 and Ф2 are closed.

[0102] In one specific embodiment, reference Figure 21 and Figure 22 Pointwise: Assuming there is only one column, the activation value is a=4'b1010(+10), and the weight is w=4'b0111(+7), so the MAC is 70.

[0103] To facilitate the explanation of the calculation process, this embodiment uses Vcm=450mV as an example, but it is not limited to 450mV. Vcm can be any reasonable supply voltage value.

[0104] Step 1: Activate the highest bit 'a' <3> =1 is input first, multiplied by the privileged value of 4, and the output states of the CIM cell storing the weights are: ia_sign=vdd, ia_w <2> =vdd,ia_w <1> =vdd,ia_w <0> =vdd, after being weighted by the C2C capacitor weighting array, the result of 4w1a should be 450+450*7 / 8mV, that is, 843.75mV.

[0105] V3=4w1a <3> When charging an 8C capacitor, the charge Q3 = (V3 - 450) * 8C.

[0106] Step 2: Activate the second highest digit 'a' <2> =0 input, multiplied by 4 privileged values, the output states of the stored weight CIM cell are: ia_sign=vdd / 2, ia_w <2> =vdd / 2, ia_w <1> =vdd / 2, ia_w <0> =vdd / 2, after being weighted by the C2C capacitor weighting array, the result of 4w1a should be vdd / 2=450mV.

[0107] V2=4w1a <2> When charging a 4C capacitor, the charge Q2 = (V2 - 450) * 4C.

[0108] Step 3: Activate value a <1> =1 input, multiplied by 4 privileged values, the output states of the CIM cell storing the weights are: ia_sign=vdd, ia_w <2> =vdd,ia_w <1> =vdd,ia_w <0> =vdd, after being weighted by the C2C capacitor weighting array, the result of 4w1a should be 450+450*7 / 8mV, that is, 843.75mV.

[0109] V1=4w1a <1> When charging a 2C capacitor, the charge Q1 = (V1 - 450) * 2C.

[0110] Step 4: Activate the least significant bit 'a' <1> =0 input, multiplied by 4 privileged values, the output states of the CIM cell storing the weights are: ia_sign=vdd / 2, ia_w <2> =vdd / 2, ia_w <1> =vdd / 2, ia_w <0> =vdd / 2, after being weighted by the C2C capacitor weighting array, the result of 4w1a should be vdd / 2=450mV.

[0111] V0=4w1a <0> When capacitor C is charged, the charge Q0 = (V0 - 450) * C.

[0112] Finally, the upper plates of the capacitor are connected in parallel to vdd / 2 = 450mV, and the lower plates of the capacitor, which were originally connected in parallel, are disconnected from vdd / 2 to accumulate and share charge.

[0113] Based on the principle of charge conservation, Q = Q3 + Q2 + Q1 + Q0, so we get the MAC result Vt = 4w4a.

[0114] (450-Vt)*15C=(V3-450)*8C+(V2-450)*4C+(V1-450)*2C+(V0-450)*C.

[0115] △V=450-Vt=262.5mV.

[0116] 1 LSB = (450 / 8) / 15 = 3.75mV, ΔV / 1LSB = 70, which corresponds to the initially preset MAC result.

[0117] In another specific embodiment, pointwise: assuming there is only 1 column: activation value a = 4'b1001(+9), weight w = 4'b1010(-2), MAC = -18.

[0118] Step 1: Activate the highest bit 'a' <3> =1 is input first, multiplied by the privileged value of 4, and the output states of the CIM cell storing the weights are: ia_sign=gnd, ia_w <2> =vdd / 2, ia_w <1> =gnd,ia_w <0> =vdd / 2, after weighting 4w1a by the C2C capacitor weighting array, the result of 4w1a should be 450-450*1 / 4mV, that is, 337.5mV.

[0119] V3=4w1a <3> When charging an 8C capacitor, the charge Q3 = (V3 - 450) * 8C.

[0120] Step 2: Activate the second highest digit 'a' <2> =0 input, multiplied by 4 privileged values, the output states of the CIM cell storing the weights are: ia_sign=vdd / 2, ia_w <2> =vdd / 2, ia_w <1> =vdd / 2, ia_w <0> =vdd / 2, after being weighted by the C2C capacitor weighting array, the result of 4w1a should be vdd / 2=450mV.

[0121] V2=4w1a <2> When charging a 4C capacitor, the charge Q2 = (V2 - 450) * 4C.

[0122] Step 3: Activate value a <1> =1 input, multiplied by 4 privileged values, the output states of the CIM cell storing the weights are: ia_sign=vdd, ia_w <2> =vdd,ia_w <1> =vdd,ia_w <0> =vdd, after being weighted by the C2C capacitor weighting array, the result of 4w1a should be vdd / 2=450mV.

[0123] V1=4w1a <1> When charging a 2C capacitor, the charge Q1 = (V1 - 450) * 2C.

[0124] Step 4: Activate the least significant bit 'a' <1> =0 input, multiplied by 4 privileged values, the output states of the CIM cell storing the weights are: ia_sign=vdd / 2, ia_w <2> =vdd / 2, ia_w <1> =vdd / 2, ia_w <0> =vdd / 2, after weighting 4w1a by the C2C capacitor weighting array, the result of 4w1a should be 450-450*1 / 4mV=337.5mV.

[0125] V0=4w1a <0> When capacitor C is charged, the charge Q0 = (V0 - 450) * C.

[0126] Finally, the upper plates of the capacitors are connected in parallel to vdd / 2 = 450mV, and the lower plates of the capacitors that were originally connected in parallel are disconnected from vdd / 2 to accumulate and share charge.

[0127] Based on the principle of charge conservation, Q = Q3 + Q2 + Q1 + Q0, so we get the MAC result Vt = 4w4a.

[0128] (450-Vt)*15C=(V3-450)*8C+(V2-450)*4C+(V1-450)*2C+(V0-450)*C.

[0129] △V=450-Vt=-67.5mV.

[0130] 1 LSB = (450 / 8) / 15 = 3.75mV, ΔV / 1LSB = -18, which corresponds to the initially preset MAC result.

[0131] In another specific embodiment, depthwise: assuming there is only 1 row and 1 column: input weight w=4'b1101(-5), stored activation value a=4'b1011(11), MAC=-55.

[0132] Step 1: w <3> =1, w <2> =1, the sign bit and non-sign bit of the weight act simultaneously, multiplied by the 4-bit activation value, and the output states of the CIM cell storing the activation value are: ia_sign=gnd, ia_w <3> =gnd,ia_w <2> =vdd / 2, ia_w <1> =gnd,ia_w <0> =gnd, after weighting 1w4a by the C2C capacitor weighting array, the result of 1w4a should be 450-450*11 / 16mV, that is, 140.625mV.

[0133] V2=1w4a <3> When charging a 4C capacitor, the charge Q2 = (V3 - 450) * 4C.

[0134] Step 2: w <3> =1, w <2> =0, the sign and unsign bits of the weight act simultaneously, multiplied by the 4-bit activation value, and the output states of the CIM cell storing the activation value are: ia_sign = vdd / 2, ia_w <3> =vdd / 2, ia_w <2> =vdd / 2, ia_w <1> =vdd / 2, ia_w <0> =gnd, after weighting 1w4a by the C2C capacitor weighting array, the result of 1w4a should be 450mV.

[0135] V1=1w4a <2> When charging a 2C capacitor, the charge Q1 = (V2 - 450) * 2C.

[0136] Step 3: w <3> =1, w <2> =1, the sign and unsign bits of the weight act simultaneously, multiplied by the 4-bit activation value. The output states of the CIM cell storing the activation value are: ia_sign=gnd, ia_w <3> =gnd,ia_w <2> =vdd / 2, ia_w <1> =gnd,ia_w <0> =gnd, after weighting 1w4a by the C2C capacitor weighting array, the result of 1w4a should be 450-450*1 / 4mV, that is, 140.625mV.

[0137] V0=1w4a <1> When capacitor C is charged, the charge Q0 = (V1 - 450) * C.

[0138] Finally, the upper plates of the capacitors are connected in parallel to vdd / 2 = 450mV, and the lower plates of the capacitors that were originally connected in parallel are disconnected from vdd / 2 to accumulate and share charge.

[0139] Based on the principle of charge conservation, Q = Q2 + Q1 + Q0, we obtain the MAC result Vt = 4w4a.

[0140] (450-Vt)*7C= (V2-450)*4C+(V1-450)*2C+(V0-450)*C.

[0141] △V=450-Vt=-1546.875 / 7mV.

[0142] 1 LSB = (450 / 16) / 7 = 225 / 56mV, ΔV / 1LSB = -55, which corresponds to the initially preset MAC result.

[0143] This invention provides a multi-bit signed in-memory computation weighting circuit. The multi-bit in-memory computation unit combines signed and non-signed in-memory computation units to perform multiplication of input activation values ​​and weights, generating a calculation result and realizing multi-bit signed number product calculation. The calculation result is weighted by an analog domain weighting structure. Based on charge conservation, the multiplication and accumulation of input activation values ​​and weights are performed in the analog domain. After one analog-to-digital conversion, the corresponding digital result is generated. The analog domain weighting structure completes the MAC calculation of multi-bit activation values ​​and multi-bit privileged values ​​in the analog domain, obtaining the final digital output result through only one AD conversion, reducing latency and power consumption. Based on the digital result, multiple switching operations are performed through a preset array structure and mapping network to complete the traversal and sliding of preset convolution kernels, realizing convolution operation on the entire feature map. The circuit architecture supports various neural network mappings, including lightweight networks such as MobileNet, and has good applicability.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0145] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-bit signed memory in-memory computation weighting circuit, characterized in that, include: Multi-bit in-memory computing units, analog domain weighted structures, and mapping networks; The multi-bit in-memory computation unit completes the multiplication calculation of the input activation value and weight by combining the sign bit in-memory computation unit and the non-sign bit in-memory computation unit, and generates the calculation result; The calculation results are weighted by the analog domain weighting structure. Based on charge conservation, the multiplication and accumulation calculation of the input activation value and the weight is completed in the analog domain. After one analog-to-digital conversion, the corresponding digital result is generated. Based on the digital results, multiple switching operations are performed through a preset array structure and mapping network to complete the traversal and sliding of the preset convolution kernel, thereby realizing the convolution operation on the entire feature map. The multi-bit in-memory computation unit includes a multi-bit in-memory computation unit with fixed weights; the multi-bit in-memory computation unit with fixed weights includes a sign bit in-memory computation unit and multiple unsigned bit in-memory computation units; the unsigned activation value is serially input into the multi-bit in-memory computation unit with fixed weights, the weights are in original code form, the highest bit of the activation value is input first, and the multiplication calculation of four privileged values ​​and one bit activation value, i.e., 4w1a, is completed in one step to generate the calculation result; The multi-bit in-memory computation unit further includes a multi-bit in-memory computation unit with a fixed activation value; the multi-bit in-memory computation unit with a fixed activation value includes: multiple unsigned bit in-memory computation units and two switches sw0 and sw1; the signed weights are serially input into the multi-bit in-memory computation unit with the fixed activation value, with the most significant bit input first, and the sign bit and unsigned bit of the weights act simultaneously to complete the multiplication calculation of a privileged value and a four-bit activation value, i.e., 1w4a, and generate the calculation result; The analog domain weighting structure includes: a C2C capacitor weighting array, an adjustable gain amplifier, and a switched proportional capacitor weighting array; the highest bit of the activation value a[3:0] is input first, multiplied by the four privileged values, and the resulting 4w1a voltage value is used to charge the capacitors 8C, 4C, 2C, and C in sequence; after the four 4w1a calculation results are used to charge the switched proportional capacitor weighting array in sequence, the weighting of the 4w4a calculation result is completed; switches sw0, sw1, sw2, and sw3 are closed, k0 is open, and k1 is closed, the lower plates of the capacitors are connected in parallel, and the charge is accumulated and shared. Based on the principle of charge conservation, the 4w4a calculation result is obtained on the lower plate of the capacitor.

2. The multi-bit signed memory in-memory weighted calculation circuit according to claim 1, characterized in that, The symbol-in-memory computation unit includes a standard 6T-SRAM cell and two switches sw0 and sw1, wherein sw0 and sw1 are controlled by input activation values ​​ia and iab; During the calculation phase, when the sign bit of the four privileged values ​​is equal to 1, the weight is negative, q=1, qb=0; when ia=1, switch sw0 is closed and switch sw1 is open, ia_sign=qb=0; when ia=0, switch sw0 is open and side switch sw1 is closed, ia_sign=vdd_2, that is, vdd / 2=450mV. After the input activation value ia interacts with the in-memory computation unit of the storage weight sign bit, the result ia_sign is obtained. ia_sign is then connected to the input in of the in-memory computation unit of the non-sign bit, i.e., ia_sign = in.

3. The multi-bit signed memory in-memory weighted calculation circuit according to claim 1, characterized in that, The non-signed bit memory computing unit includes a standard 6T-SRAM, two switches sw0 and sw1 and a capacitor, wherein sw0 and sw1 are controlled by q and qb; During the reset phase, the upper plate of the capacitor is reset to vdd_2, i.e., vdd / 2 = 450mV, the voltage at node 4w1a is reset to vdd / 2, and the lower plate of the capacitor is clamped to a certain voltage value. When the voltage of the upper plate of the capacitor changes, due to charge coupling, the voltage value of the lower plate of the capacitor will also change accordingly, and thus the voltage value at node 4w1a will also change accordingly. During the calculation phase, when the non-sign bit of the weight is 0, q=0, qb=1, switch sw0 is open, switch sw1 is closed, and the voltage on the upper plate of the capacitor is still vdd / 2; when the non-sign bit of the weight is 1, q=1, qb=0, switch sw0 is closed, switch sw1 is open, and the voltage on the upper plate of the capacitor is equal to in, with a value of vdd, vdd / 2, or gnd.

4. The multi-bit signed memory in-memory weighted calculation circuit according to claim 1, characterized in that, The analog domain weighting structure also includes: a C2C capacitor weighting array, an adjustable gain amplifier, a switched proportional capacitor weighting array, and a charge-sharing switch sw_share; The highest bit of the activation value a[3:0] is input first, multiplied by the four privileged values, and the resulting 4w1a voltage value is used to charge the capacitors 8C, 4C, 2C, and C in sequence. The results of the four 4W1A calculations are used to charge the switching proportional capacitor weighted array in sequence, thereby completing the weighting of the 4W4A calculation results. sw0, sw1, sw2, and sw3 are disconnected, sw_share is closed, and the upper plates of the capacitor are connected in parallel. Based on the principle of charge conservation, the calculated result of 4w4a is obtained on the upper plate of the capacitor.

5. The multi-bit signed memory in-memory weighted calculation circuit according to claim 1, characterized in that, The calculation based on charge conservation is performed in the analog domain to accumulate and multiply the input activation value and weight. The analog domain is used to calculate the product of the four-bit activation value and the four-bit privileged value. After one analog-to-digital conversion, the corresponding digital result is obtained.

6. The multi-bit signed memory in-memory computation weighting circuit according to claim 4 or 5, characterized in that, The adjustable gain amplifier transfers the 4w1a node voltage from the C2C capacitor weighted array to the switched proportional capacitor weighted array, and amplifies the overall analog voltage range through the adjustable gain amplifier.

7. The multi-bit signed memory in-memory weighted calculation circuit according to claim 1, characterized in that, The process involves multiple switching operations based on the digital results using a preset array structure and mapping network to complete the traversal and sliding of preset convolutional kernels, thereby achieving convolution operations on the entire feature map. This includes: Neural network mapping with fixed weights and neural network mapping with fixed activation values ​​are respectively constructed using corresponding array structures. The neural network mapping with fixed activation values ​​completes the traversal and sliding process of the convolution kernel through three switching operations, thereby realizing the convolution operation on the entire feature map.

Images