Pre-quantization circuit and pre-quantization method for computing-in-memory

Abstract

A pre-quantization circuit for computing-in-memory, including an analog multiply-accumulate circuit and an analog-to-digital converter. The analog multiply-accumulate circuit is configured to receive a set of weights from the data memory and allocate it on a computing memory, and receive an input feature from the register, and then perform multiplication and accumulation operations utilizing the set of weights and the input feature to generate an operation result in the form of an analog voltage / current. The analog-to-digital converter is configured to convert the analog voltage / current into its corresponding digital code. The analog multiply-accumulate circuit is further configured to determine the pre-quantization information of the analog-to-digital converter before performing multiplication and accumulation operations. The analog-to-digital converter is further configured to adjust the conversion bit number of the converter based on the pre-quantization information.

Description

[Technical Field]

[0001] This invention relates to a prequantization circuit and method, and more particularly to a prequantization circuit and method for memory-based operations. [Previous Technology]

[0002] In typical computer architectures (such as Von Neumann architecture), the computing units must first access and transfer data from memory to the computing units before performing calculations. Such architectures, like Von Neumann architecture, are efficient when the computation itself is complex. However, neural network computation, due to its unique characteristics, involves simple computations but a massive amount of data. If a Von Neumann architecture is used for neural network computation, the access and transfer of large amounts of data can become a bottleneck for power consumption and computational speed. Therefore, the so-called non-Von Neumann architecture emerged, aiming to blur the boundary between computing units and memory, becoming a dataflow-centric design architecture.

[0003] Computing-In-Memory (CIM) architecture is a type of non-Van Neumann architecture. Its concept is to integrate the computing unit and memory in the same circuit at the transistor level. This can significantly reduce the power consumption of data reading and moving and break through the bottleneck of computing speed. [Summary of the Invention]

[0004] The purpose of this invention is to provide a prequantization circuit and prequantization method for operations within memory. Before the analog multiply-accumulate circuit in the prequantization circuit performs multiply-accumulate operations, the software pre-calculates the weighted bit data configured in the memory with operational functions to obtain prequantization information, so as to pre-quantize the first few most significant bits (MSB) of the analog-to-digital converter, thereby determining the number of conversion bits that can be omitted by the analog-to-digital converter, so as to achieve the effect of omitting the number of conversion bits of the analog-to-digital converter without the need for detection circuit, saving power consumption and saving electricity.

[0005] One aspect of the present invention provides a pre-quantization circuit for in-memory computation, comprising an analog multiply-accumulate circuit and an analog-to-digital converter (ADC). The analog multiply-accumulate circuit is configured to receive a set of weights from data memory and allocate them to memory capable of performing computation, and to receive input features from a register. Then, it uses the computationally capable memory to perform a multiply-accumulate (MAC) operation on this set of weights and the input features to generate an analog voltage / current value corresponding to the computation result. The ADC is configured to convert the analog voltage / current into corresponding digital codes. The analog multiply-accumulate circuit is further configured to determine the pre-quantization information of the ADC before performing the multiply-accumulate operation. The ADC is configured to convert the analog voltage / current into corresponding digital codes.

[0006] In some embodiments, the analog multiply-accumulate circuit is further configured to determine whether the set of weights has been fully utilized after the analog-to-digital converter converts the analog voltage / current into the corresponding digital code.

[0007] In some embodiments, the analog multiply-accumulate circuit is further configured to load an updated set of weights from memory after determining that the set of weights has been fully used.

[0008] In some embodiments, the analog multiply-accumulate circuit is further configured to continue multiplying and accumulating using the set of weights and input features to generate an analog voltage / current value corresponding to the result of the operation when it is determined that the set of weights has not been fully used.

[0009] In some embodiments, the analog-to-digital converter is a successive-approximation register ADC (SAR ADC).

[0010] Another aspect of the present invention is to provide a prequantization method for computation within a memory, comprising the following steps: receiving a set of weights from a data memory through an analog multiply-accumulate circuit and allocating them to a memory capable of performing computation, and receiving input features through a temporary register; then performing a multiply-accumulate operation on the set of weights and the input features using a memory with computational capabilities to generate an analog voltage / current value corresponding to the computation result; and converting the analog voltage / current value into a corresponding digital code using an analog-to-digital converter; wherein the analog multiply-accumulate circuit is further configured to pre-calculate the weight bit data configured in the memory with computational capabilities to obtain prequantization information before performing the multiply-accumulate operation, and wherein the analog-to-digital converter is further configured to obtain the digital code of the first few bits based on the prequantization information, and convert the analog voltage / current into the digital code of the subsequent few bits.

[0011] In some embodiments, the analog multiply-accumulate circuit is further configured to determine whether the set of weights has been fully utilized after the analog-to-digital converter converts the analog voltage / current into the corresponding digital code.

[0012] In some embodiments, the analog multiply-accumulate circuit is further configured to load an updated set of weights from memory after determining that the set of weights has been fully used.

[0013] In some embodiments, the analog multiply-accumulate circuit is further configured to continue multiplying and accumulating using the set of weights and input features to generate analog voltage / current when it is determined that the set of weights has not been fully used.

[0014] In some embodiments, the analog-to-digital converter is a cyclic asymptotic temporary analog-to-digital converter.

Implementation Method

[0015] Embodiments of the present invention will be discussed in detail below. It will be understood that the embodiments provide many applicable concepts that can be implemented in a wide variety of specific situations. The embodiments discussed and disclosed are for illustrative purposes only and are not intended to limit the scope of the invention.

[0016] Neural network operations consist of many layers of mathematical operations. Taking the common convolutional neural network (CNN) as an example, the computational workload of the convolutional layer and the fully-connected layer accounts for a very high proportion of the overall computational workload (approximately over 90%). Both of these operations consist of multiply-accumulate (MAC) operations. In detail, multiply-accumulate is the operation of multiplying the input feature and the weight and then summing them. The prequantization circuit of computing-in-memory (CIM) is used to process multiply-accumulate operations, where the weights are loaded from memory (e.g., dynamic random-access memory, DRAM), while the input features are loaded from outside the prequantization circuit (e.g., temporary registers). It should be noted that the prequantization circuit for in-memory operations can be implemented in an analog or digital manner, but this disclosure only discusses the prequantization circuit for in-memory operations implemented in an analog manner.

[0017] Although it is known that using analog circuits to implement artificial intelligence accelerators will have better energy efficiency than digital circuits, since digital signals are used for transmission in the system of neural network models, if it is desired to use pre-quantization circuits that are implemented in memory in an analog manner, a common practice is to first convert the value to be processed into an analog signal through a digital-to-analog converter (DAC) (for example, converting a set of weights loaded from memory into a set of analog weights, and converting the input features loaded from the register into analog input features), perform the operation through analog operators, and then convert it into a digital signal through an analog-to-digital converter (ADC) for transmission. However, this may result in most of the power being consumed in the converter.

[0018] Figure 1 is a schematic diagram of the operation of a prequantization circuit 100 performing operations within a memory according to an embodiment of the present invention. As can be seen from Figure 1, the analog multiply-accumulate circuit 120 in the prequantization circuit 100 first loads a set of weights W from the memory (not shown in Figure 1) and an input feature F from the temporary register (not shown in Figure 1). Then, it performs multiplication-accumulation operations using this set of weights W and the input feature F to generate an analog voltage / current AMACV. It should be noted that, under normal circumstances, the weights W loaded by the analog multiply-accumulate circuit 120 are in digital form, while the input feature F loaded by the analog multiply-accumulate circuit 120 can be in digital form or analog form (e.g., analog voltage sensed by a sensor). The present invention does not limit this.

[0019] After the analog multiply-accumulate circuit 120 performs multiply-accumulate operations and generates an analog voltage / current AMACV, the analog voltage / current AMACV is then quantized into a corresponding digital code CODE by the analog-to-digital converter 130 (or quantizer), thus completing the so-called memory operation. It should be noted that the process of converting the analog voltage / current AMACV into the corresponding digital code CODE by the analog-to-digital converter 130 is the most power-consuming part of the pre-quantization circuit 100. In detail, the analog multiply-accumulate circuit 120 in the prequantization circuit 100 is implemented in an analog manner rather than a digital manner because the analog multiply-accumulate circuit 120 has the advantages of relatively high parallelism and low power consumption compared to the digital multiply-accumulate circuit. However, the disadvantage of the analog multiply-accumulate circuit 120 is that the generated multiply-accumulate voltage / current still needs to be converted from analog form to digital form through the analog-to-digital converter 130 for output. Therefore, how to reduce the power consumption of the analog-to-digital converter 130 becomes an important issue in the design of the prequantization circuit 100.

[0020] A common practice for reducing the power consumption of the prequantization circuit 100 is to first perform an analog judgment on the generated analog voltage / current AMACV after the analog multiply-accumulate circuit 120 performs multiplication and accumulation operations using weight W and input characteristics F. Specifically, for example, a detection circuit is designed for analog judgment. When the detection circuit determines that the value of analog voltage / current AMACV is less than a certain analog threshold, it can control the analog-to-digital converter 130 to omit the solution process of one or more most significant bits (MSBs). In other words, by reducing the operation of some circuits in the analog-to-digital converter 130, the power consumption of the analog-to-digital converter 130 is effectively reduced. Conversely, when the detection circuit determines that the value of analog voltage / current AMACV is greater than or equal to the certain analog threshold, it controls the analog-to-digital converter 130 to still perform the normal solution process.

[0021] However, it can be observed that the weights of the neural network are sparsity, that is, among all the weights trained by the neural network, the majority of the weights have a value of 0. Furthermore, in general, a set of weights W can be reused dozens, hundreds, or even thousands of times after being loaded into the pre-quantization circuit 100 in memory for operation, without having to be loaded again and again. In other words, during the operation, only different input features F need to be loaded into the pre-quantization circuit 100 in memory from the temporary register, and the entire set of weights W only needs to be loaded once. Only after this set of weights W has been fully used will a new set of updated weights be loaded into the pre-quantization circuit 100 in memory for operation, and so on.

[0022] In one embodiment of the present invention, the maximum value of analog voltage / current AMACV can be calculated by the following formula (1), which is expressed as: (1) where represents the maximum value of input feature F customized by the circuit designer, represents the pre-trained and known weight values ​​of the neural network, c is the number of non-zero weight values, and represents the sum of these weight values.

[0023] It is known that the common practice for reducing the power consumption of the pre-quantization circuit 100 is to perform multiplication and addition operations using weights W and input features F in the analog multiply-add circuit 120 before performing analog comparison judgment on the generated analog voltage / current AMACV value. This invention further incorporates the concept of pre-quantization based on this common practice. Given that most weights trained by the neural network have a value of 0, the step of performing analog comparison judgment can be moved from after the multiplication and addition operation to before it. In other words, the power consumption reduction method proposed in this invention through pre-quantization technology allows the analog multiply-add circuit 120 to obtain pre-quantization information via software before performing multiplication and addition operations using weights W and input features F. Specifically, the software calculates the number of non-zero weight values ​​and, based on this number, pre-determines the maximum value of the multiplication and addition operation result. This pre-quantizes the number of bits required for the analog-to-digital converter 130 to solve the value, thereby determining the omission of the solution process in the analog-to-digital converter 130. Compared to current methods for reducing power consumption, the advantage of moving the analog threshold judgment step from after the multiplication-addition operation to before the multiplication-addition operation is that: the current method requires an analog threshold judgment to be performed after each multiplication-addition operation, for example, through a detection circuit. Each judgment consumes power, and the time spent on each judgment also increases the solution time of the analog-to-digital converter 130, resulting in slower operation. In contrast, the method proposed in this invention calculates the number of non-zero weight values ​​in the weight bits to pre-quantize the number of bits that the analog-to-digital converter 130 needs to solve, thereby determining the solution process that can be omitted by the analog-to-digital converter 130 (because once a set of weights W is loaded into the pre-quantization circuit 100, it does not need to be loaded again for modification until it is used up). This method saves the cost of designing a detection circuit, avoids the power consumption caused by multiple analog threshold judgments, and does not increase the solution time of the analog-to-digital converter 130 due to the time spent on multiple analog threshold judgments.

[0024] In other words, the common practice of reducing the power consumption of the prequantization circuit 100 can be regarded as making an analog judgment based on the value of the analog voltage / current AMACV (i.e.), while the method proposed in this invention to reduce the power consumption of the prequantization circuit 100 only makes an analog judgment based on the weight W loaded by memory (i.e.).

[0025] In some embodiments, the analog-to-digital converter 130 in the prequantization circuit 100 is a successive-approximation register (SAR) analog-to-digital converter. It should be noted that although the present invention implements the proposed prequantization circuit 100 using a successive-approximation register analog-to-digital converter, other suitable analog-to-digital converters 130 can also be used to implement the proposed prequantization circuit 100, and the present invention does not limit this to such implementations.

[0026] As mentioned above, a set of weights W can typically be reused dozens, hundreds, or even thousands of times after being loaded into memory and processed by the prequantization circuit 100, without needing to be loaded repeatedly. Therefore, after the analog voltage / current AMACV is converted into the corresponding digital code CODE by the analog-to-digital converter 130, the analog multiply-accumulate circuit 120 will then determine whether the set of weights W has been fully used. In one embodiment of the present invention, when the analog multiply-accumulate circuit 120 determines that the set of weights W has been fully used, it will load an updated set of weights from memory to replace the original weights W. In another embodiment of the present invention, when the analog multiply-accumulate circuit 120 determines that the set of weights W has not been fully used, it will continue to use the set of weights W and the input feature F to perform multiplication and addition operations to generate the analog voltage / current AMACV.

[0027] Figure 2 is a flowchart of a prequantization method 200 for in-memory operations according to an embodiment of the present invention. The prequantization method 200 for in-memory operations can be performed by, for example, the prequantization circuit 100 of Figure 1, which includes an analog multiply-accumulate circuit 120 and an analog-to-digital converter 130, or other similar circuits. As shown in Figure 2, the prequantization method 200 includes steps S210 to S220. The following paragraphs describe the implementation method of each step in the prequantization method 200 in conjunction with the content of Figures 1-2.

[0028] Step S210: Receive a set of weights from memory and input features from temporary register through analog multiply-accumulate circuit. Then, perform multiply-accumulate operation using this set of weights and input features through analog multiply-accumulate circuit to generate analog voltage / current.

[0029] Step S220: Convert analog voltage / current into corresponding digital code using an analog-to-digital converter, wherein the analog multiply-accumulate circuit is further configured to determine the pre-quantization information of the analog-to-digital converter before performing multiply-accumulate operations, and wherein the analog-to-digital converter is further configured to convert analog voltage / current into corresponding digital code based on the pre-quantization information.

[0030] The explanation of steps S210 to S220 above can be found in the operation of each component (including but not limited to analog multiply-accumulate circuit 120 and analog-to-digital converter 130) in the prequantization circuit 100 for operation in memory as shown in Figure 1, and will not be repeated here.

[0031] In summary, the prequantization circuit and prequantization method of the present invention obtain prequantization information through software before the analog multiply-accumulate circuit in the prequantization circuit performs multiply-accumulate operation, thereby prequantizing the number of bits of the value to be solved by the analog digital converter, thereby determining the solution process that can be omitted by the analog digital converter, so as to achieve the effects of saving detection circuit design cost, saving power consumption, and saving power.

[0032] Although the present invention has been disclosed above by way of embodiments, it is not intended to limit the scope of the present invention. Anyone with ordinary knowledge in the art can make various changes, substitutions and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims. [Simplified Explanation of the Diagram]

[0033] To make the above and other objects, features, advantages and embodiments of the present invention more apparent and understandable, the accompanying drawings are described as follows: Figure 1 is a schematic diagram of the operation of a prequantization circuit for in-memory computation according to an embodiment of the present invention; and Figure 2 is a flowchart of a prequantization method for in-memory computation according to an embodiment of the present invention. [Biomaterial Storage]

[0035] Domestic storage information (please note in order of storage institution, date, and number): None. International storage information (please note in order of storage country, institution, date, and number): None.

Claims

1. A pre-quantization circuit for in-memory operations, comprising: an analog multiply-accumulate circuit configured to receive a set of weights from a memory and an input feature from a register, and then perform a multiply-accumulate (MAC) operation using the set of weights and the input feature to generate an analog voltage / current; and an analog-to-digital converter (ADC) configured to convert the analog voltage / current into a corresponding digital code, wherein the analog multiply-accumulate circuit is further configured to determine pre-quantization information of the ADC before performing the multiply-accumulate operation, and wherein the ADC is further configured to convert the analog voltage / current into the corresponding digital code based on the pre-quantization information.

2. The prequantization circuit as described in claim 1, wherein the analog multiply-accumulate circuit is further configured to determine whether the set of weights has been fully utilized after the analog-to-digital converter converts the analog voltage / current into the corresponding digital code.

3. The prequantization circuit as described in claim 2, wherein the analog multiply-accumulate circuit is further configured to load an updated set of weights from the memory after determining that the set of weights has been fully used.

4. The prequantization circuit as described in claim 2, wherein the analog multiply-accumulate circuit is further configured to continue using the set of weights and the input feature to generate the analog voltage / current when it is determined that the set of weights has not been fully used.

5. The prequantization circuit as described in claim 1, wherein the analog-to-digital converter is a successive-approximation register ADC (SAR ADC).

6. A prequantization method for operations within a memory, comprising: receiving a set of weights from a memory and receiving an input feature from a register via an analog multiply-accumulate circuit; then performing a multiply-accumulate operation using the set of weights and the input feature via the analog multiply-accumulate circuit to generate an analog voltage / current; and converting the analog voltage / current into a corresponding digital code using an analog-to-digital converter, wherein the analog multiply-accumulate circuit is further configured to determine prequantization information of the analog-to-digital converter before performing the multiply-accumulate operation, and wherein the analog-to-digital converter is further configured to convert the analog voltage / current into the corresponding digital code according to the prequantization information.

7. The prequantization method as described in claim 6, wherein the analog multiply-accumulate circuit is further configured to determine whether the set of weights has been fully utilized after the analog-to-digital converter converts the analog voltage / current into the corresponding digital code.

8. The prequantization method as described in claim 7, wherein the analog multiply-accumulate circuit is further configured to load a set of updated weights from the memory after determining that the set of weights has been fully used.

9. The prequantization method as described in claim 7, wherein the analog multiply-accumulate circuit is further configured to continue using the set of weights and the input feature to generate the analog voltage / current when it is determined that the set of weights has not been fully used.

10. The prequantization method as described in claim 6, wherein the analog-to-digital converter is a cyclic asymptotic temporary analog-to-digital converter.

Images