Convolution operation device

Abstract

The convolution operation device includes a multiply-accumulate operation circuit and a post-processing circuit. The multiply-accumulate operation circuit performs a convolution operation based on a first feature data and a weighting coefficient to generate initial operation data and generates a completion signal. The post-processing circuit retrieves a first shift bit from a memory based on the completion signal, performs a one-bit shift on the initial operation data based on the first shift bit to generate a shifted operation data, and performs a first truncation operation on the shifted operation data according to a preset numerical range to generate a first operation data, wherein the number of bits of the first operation data is less than the number of bits of the initial operation data.

Description

[Technical Field]

[0001] This case relates to convolution operation devices, and in particular to convolution operation devices that can reduce the cost of circuit implementation. [Previous Technology]

[0002] Convolutional processing units are commonly used to implement neural networks for various types of recognition applications. In existing applications, the processing circuitry within the convolutional processing unit needs to process the results of multiply-accumulate operations to produce the final output. Since the bit count of the results generated by multiply-accumulate operations is typically very high, the processing circuitry needs to support a large input bit width to fully receive and process the result. This significantly increases the overall cost of the processing circuitry. [Summary of the Invention]

[0003] In some embodiments, one of the objectives of this invention is to provide a convolution operation device that can reduce the circuit implementation cost, thereby improving the shortcomings of the prior art.

[0004] In some embodiments, the convolution operation device includes a multiply-accumulate operation circuit and a post-processing circuit. The multiply-accumulate operation circuit performs a convolution operation based on a first feature data and a weighting coefficient to generate an initial operation data and generates a completion signal. The post-processing circuit retrieves a first shift bit from a memory based on the completion signal, performs a one-bit shift on the initial operation data based on the first shift bit to generate a shifted operation data, and performs a first truncation operation on the shifted operation data based on a preset numerical range to generate a first operation data, wherein the number of bits of the first operation data is less than the number of bits of the initial operation data.

[0005] Regarding the features, implementation and effects of this case, the preferred embodiments are described in detail below with reference to the drawings.

Implementation Method

[0007] All terms used herein have their common meanings. The definitions of the above terms in commonly used dictionaries, and any examples of the use of any term discussed herein, are merely illustrative and should not limit the scope or meaning of this application. Similarly, this application is not limited to the various embodiments shown in this specification.

[0008] The terms "coupled" or "connected" as used herein can refer to two or more components making direct physical or electrical contact with each other, or making indirect physical or electrical contact with each other, or two or more components operating or moving together. As used herein, the term "circuit" can refer to a device in which at least one transistor and / or at least one active or passive component are connected in a certain manner to process signals.

[0009] Figure 1 is a schematic diagram of a convolution operation device 100 according to some embodiments of the present invention. The convolution operation device 100 includes a memory 110, a multiply accumulate (MAC) operation circuit 120, a memory 130, and a post-processing circuit 140. In some embodiments, each of the memory 110 and the memory 130 may be, but is not limited to, a static random access memory.

[0010] The multiply-accumulate circuit 120 performs a convolution operation based on the feature data IB and the weight coefficients KB to generate initial operation data AB, stores the initial operation data AB in memory 110, and generates a completion signal SS accordingly. In some embodiments, the feature data IB may be input feature data, such as, but not limited to, image data, voice data, etc. In some embodiments, the feature data IB and the weight coefficients KB may also be stored in memory 110. Memory 130 may store a plurality of pre-configured shift values ​​OBS, a plurality of bias values ​​BV, a plurality of scale values ​​SV, and a plurality of shift values ​​VBS. The above-mentioned parameters are provided to the post-processing circuit 140 to process the initial operation data AB. The relevant configuration of the above parameters will be described later. It is understood that memory 110 and memory 130 may be combined into one, which stores the aforementioned feature data IB, weight coefficients KB, initial operation data AB, a plurality of shift values ​​OBS, a plurality of bias values ​​BV, a plurality of scale values ​​SV, and a plurality of shift values ​​VBS.

[0011] The post-processing circuit 140 obtains the initial operation data AB from the memory 110 according to the completion signal SS, and obtains the corresponding shift value OBS, bias value BV, scaling value SV, and shift value VBS from the memory 130. Next, the post-processing circuit 140 performs bit shifting on the initial operation data AB to generate shifted operation data (e.g., shifted operation data SAB in FIG2), and performs a clipping operation on the shifted operation data according to a preset value range PR to generate first operation data (e.g., operation data D1 in FIG2). Through the above-mentioned clipping operation, it can be ensured that the number of bits in the first operation data is less than the number of bits in each of the initial operation data AB and the shifted operation data. For example, the number of bits in each of the initial operation data AB and the shifted operation data can be 48, while the number of bits in the first operation data is 16. The bit value mentioned above is for illustrative purposes only, and this invention is not limited thereto. A detailed explanation of the clipping operation will be given later with reference to FIG5. In different embodiments, information about the preset numerical range PR can be stored in memory 130 or in a temporary register (not shown) in post-processing circuit 140.

[0012] Further, the post-processing circuit 140 can add the bias value BV and the first operational data to generate the second operational data (e.g., operational data D2 in FIG2), and multiply the scaling value SV and the second operational data to generate the third operational data (e.g., operational data D3 in FIG2). Finally, the post-processing circuit 140 can shift the third operational data according to the shift number VBS to generate the fourth operational data (e.g., operational data D4 in FIG2), and perform a truncation operation on the fourth operational data according to a preset numerical range PR to generate the output data DO. In some embodiments, the post-processing circuit 140 can store the output data DO in a memory (not shown). The configuration and related operations of the post-processing circuit 140 will be described later with reference to FIG2 and FIG3.

[0013] Figure 2 is a schematic diagram of the post-processing circuit 140 of Figure 1 according to some embodiments of the present invention, and Figure 3 is an operation flowchart of the post-processing circuit 140 of Figure 2 according to some embodiments of the present invention. The post-processing circuit 140 includes a shifter 210, a truncator 220, an adder 230, a multiplier 240, a shifter 250, and a truncator 260. In some embodiments, each of the above-mentioned units in the post-processing circuit 140 may be implemented by a register circuit, a digital circuit, and / or a logic circuit. For ease of explanation, the operation of the above-mentioned units will be described in conjunction with Figure 3.

[0014] In operation S310, shifter 210 retrieves the initial operational data AB from memory 110 and the shift amount OBS from memory 130 according to the completion signal SS, and performs bit shifting on the initial operational data AB according to the shift amount OBS to generate the shifted operational data SAB. For example, the initial operational data AB may contain 48 bits, and the value of the shift amount OBS is m. Then shifter 210 can shift the 48 bits in the initial operational data AB to the right by m bits, and output these bits after the right shift is completed as the shifted operational data SAB, where the value m may be a non-negative integer (including 0).

[0015] In operation S320, the truncation unit 220 performs a truncation operation on the shifted operational data SAB according to a preset numerical range PR to generate operational data D1. In some embodiments, the preset numerical range PR can be determined according to the quantization scenario bandwidth of the actual application. For example, if the number of bits of the initial operational data AB is 16, the preset numerical range PR can be set to [-32767, 32767] according to this number of bits, with an upper limit of 32767 and a lower limit of -32767. If the value of the shifted operational data SAB is within this preset numerical range PR, the truncation unit 220 can output the shifted operational data SAB as the first sub-data. Alternatively, if the value of the shifted operational data SAB exceeds the upper limit of the preset numerical range PR (e.g., 32767) or is lower than the lower limit (e.g., -32767), the truncation unit 220 can set the corresponding value of the upper limit and the lower limit as the first sub-data. For example, if the value of the shifted operational data SAB exceeds 32767, the truncator 220 can set the aforementioned first sub-data to the data corresponding to the upper limit value of 32767. On the other hand, if the value of the shifted operational data SAB is lower than -32767, the truncator 220 can set the aforementioned first sub-data to the data corresponding to the lower limit value of -32767. Through the above operations, unnecessary error overflow can be avoided in subsequent calculations, thereby improving the accuracy of the overall calculation. Further, the truncator 220 can output a portion of the bits in the first sub-data as operational data D1. The details here will be explained later with reference to FIG5. In some embodiments, the truncator 220 may be implemented by, but is not limited to, a comparator circuit and a selector circuit.

[0016] In operation S330, adder 230 obtains the bias value BV from memory 130 and adds the bias value BV to the operand data D1 to produce operand data D2. In operation S340, multiplier 240 obtains the scaling value SV from memory 130 and multiplies the scaling value SV to the operand data D2 to produce operand data D3. In operation S350, shifter 250 obtains the shift amount VBS from memory 130 and performs bit shifting on operand data D3 according to the shift amount VBS to produce operand data D4. Similar to operation S310, shifter 250 can right-shift multiple bits in operand data D3 according to the shift amount VBS and output the shifted multiple bits as operand data D4.

[0017] In operation S360, truncation unit 260 can perform a truncation operation on the operational data D4 according to a preset numerical range PR to generate output data DO. Similarly, in operation S320, truncation unit 260 can perform a truncation operation on the operational data D4 according to a preset numerical range PR to generate a second sub-data, and output a portion of the bits in this second sub-data as output data DO.

[0018] In some related technologies, the convolution operation device directly convolves the input feature map data. Under this condition, the data size that the memory in the convolution operation device can store and the input bit width of the post-processing circuit (e.g., a multiplier) must be able to support input feature map data with a high number of bits. For example, if the number of bits in the input feature map data is 48, then the data size that the memory can store and the input bit width of the multiplier must each be at least 48 bits. Similarly, in these related technologies, other circuits in the post-processing circuit also need to be configured as operation circuits capable of processing high-bit-count data. This results in an excessively large overall circuit area, leading to excessively high overall circuit costs.

[0019] Compared to the aforementioned related technologies, in some embodiments of this case, by configuring the corresponding shift bit OBS according to the corresponding convolution operator in advance during the offline stage, the convolution operation device 100 can perform bit shifting and truncation operations on the initial operation data AB based on this shift bit OBS and a preset numerical range PR (which corresponds to the quantization range of the application scenario) to generate operation data D1 when it receives the initial operation data AB. This allows the number of bits in the operation data D1 to be less than the number of bits in the initial operation data AB. In this way, the complexity of other circuits in the post-processing circuit 140 and the required bit width can be reduced, thereby reducing the overall circuit area and cost. For example, if the number of bits in the initial operation data AB is 48, and the number of bits in the operation data D1 after bit shifting and truncation is 16, the input bit width of the adder 230 can be reduced to 16 bits, which is significantly lower than the number of bits in the initial operation data AB.

[0020] It should be understood that, for clarity and simplicity, Figure 2 only shows the post-processing circuit 140 corresponding to a single channel. In practical applications, the convolution operation device 100 may include multiple post-processing circuits 140 in parallel, each corresponding to multiple channels in the neural network, and each channel can be used with corresponding initial operation data AB to generate corresponding output data DO. In some embodiments, the post-processing circuit 140 corresponding to each channel can retrieve the shift value OBS, bias value BV, scaling value SV, and shift value VBS corresponding to that channel from memory 130. In some embodiments, the set of circuits corresponding to each channel is an operator in the neural network executed by the post-processing circuit 140. Therefore, in some embodiments, the shift value OBS, bias value BV, scaling value SV, and shift value VBS used by each channel can be generated in advance in an offline stage by the neural network executed by the post-processing circuit 140 and the sample data set.

[0021] Figure 4 is an operation flowchart for generating the shift bit OBS of Figure 1 or Figure 2 according to some embodiments of the present invention. In some embodiments, the multiple processes in Figure 4 may be implemented in an offline stage (e.g., may be, but not limited to, the circuit design stage or the circuit measurement stage) and may be executed by a machine or computer that executes wafer design tools or circuit simulation software.

[0022] In operation S410, a simulation test network is established by simulating all operators in the neural network executed by the post-processing circuit 140 using functions modeled with a computer-readable instruction set. In some embodiments, the instruction set may be built using C language, but this invention is not limited thereto. In operation S420, a sample dataset is input to the simulation test network to obtain the value range of all initial operation data AB corresponding to the sample dataset. In some embodiments, the sample dataset is pre-established sample data. For example, if the neural network executed by the post-processing circuit 140 is applied to image recognition for vehicles, the sample dataset may be multiple pre-established image data, the content of which may be common scenarios related to vehicle image recognition. The above-mentioned types of sample datasets are merely examples, and this invention is not limited thereto. In operation S430, multiple shift bits OBS are determined based on the value range of each initial operation data AB. In operation S440, the shifted OBS are updated to the simulation test network, and operations S420 to S430 are performed again to adjust the shifted OBS according to the output data generated by the simulation test network. In this way, by repeating the above operations, multiple shifted OBS that can provide accurate image recognition results (equivalent to accurate output data DO) can be found.

[0023] Figure 5 is a schematic diagram of operation S430 that generates Figure 4 according to some embodiments of this case. In operation S430, the value range of all initial operational data AB corresponding to the sample data set can be obtained. The highest bit of the initial operational data AB is a plurality of extended bits of the sign bit. Specifically, if bit 501 is the sign bit, then all bits above bit 501 are extended bits of the sign bit. Taking the largest initial operational data AB as an example, bit 501 is a logic value of 0, which means that this initial operational data AB is a positive value, and all extended bits above bit 501 are logic values ​​of 0. Similarly, taking the smallest initial operational data AB as an example, bit 501 is a logic value of 1, which means that this initial operational data AB is a negative value, and all extended bits above bit 501 are logic values ​​of 1. Furthermore, the lowest bit portion of the initial computation data AB is set to all bits between the next bit of bit 502 and the lowest bit, which is the number of bits that can be deleted. That is, the number of bits in this lowest bit portion is the shift number OBS.

[0024] In other words, the shifted data OBS found through the above training process ensures that all bits between bits 501 and 502 are valid data DV in the initial operational data AB, which effectively reduces the total number of bits and provides valid content with sufficient accuracy. Therefore, the valid data DV shown in FIG5 is equivalent to the shifted operational data SAB generated in operation S310 of FIG3. In operation S320, as mentioned above, the post-processing circuit 140 can process the shifted operational data SAB according to the preset value range PR to generate the first sub-data, and output the corresponding part of the bits in this first sub-data (equivalent to the valid data DV or a part thereof in FIG5) as operational data D1. That is, the post-processing circuit 140 can delete the highest and lowest part of the shifted operational data SAB by truncation operation to obtain the shifted operational data SAB. The post-processing circuit 140 can perform operation S350 in a similar manner to generate operational data D4. Incidentally, in some embodiments, the actual number of bits of valid data DV may be less than the preset number of data bits (e.g., the aforementioned 16). Under this condition, the shift bit OBS can be set to 0.

[0025] Similarly, in some embodiments, during the offline phase, the bias value BV can be obtained through the following operations: representing the original bias value of the neural network as a fixed-point number; bit-shifting the original bias value according to the shifting bit OBS to generate the bias value BV; if the bias value BV is within a preset value range (e.g., the aforementioned preset value range PR), keeping the bias value BV unchanged; if the bias value BV exceeds the upper limit of the preset value range, setting the bias value BV to the upper limit; and if the bias value BV is lower than the lower limit of the preset value range, setting the bias value BV to the lower limit. Similarly, after obtaining the value range of all initial computational data AB, the above operations can be further used to configure multiple corresponding bias values ​​BV according to multiple shifting bit OBS.

[0026] Similarly, in some embodiments, during the offline phase, the scaling value SV and the shift value VBS can be obtained by the following operations: estimating the original scaling value and the original shift value through the mathematical model of the original neural network; setting the original scaling value to the scaling value SV; and subtracting the shift value OBS from the original shift value to generate the shift value VBS.

[0027] The above-described operations related to generating the shift value OBS, bias value BV, scaling value SV, and shift value VBS are merely illustrative. Various operations that can perform truncation operations on the input data of the post-processing circuit, and their offline generation methods, are all within the scope of this application. For example, in the examples above, bit shifting is exemplified by right shifting, but this application is not limited to this. In other embodiments, depending on actual application requirements, the aforementioned bit shifting can also be left shifting.

[0028] In summary, the convolution operation device provided in some embodiments of this case can pre-configure multiple operation operations in the offline stage to perform shift and truncation operations on the input of the post-processing circuit. In this way, while ensuring the accuracy of the output data, the required space of memory and the input bit width required by subsequent circuits in the processing circuit can be further reduced, thereby reducing the overall circuit cost.

[0029] Although the embodiments of this case are described above, these embodiments are not intended to limit this case. Those skilled in the art can make variations to the technical features of this case based on the express or implied content of this case. All such variations may fall within the scope of patent protection sought in this case. In other words, the scope of patent protection in this case shall be determined by the scope of the patent application in this specification. [Simplified Explanation of the Diagram]

[0006] [Figure 1] is a schematic diagram of a convolution operation device according to some embodiments of the present invention; [Figure 2] is a schematic diagram of the post-processing circuit of Figure 1 according to some embodiments of the present invention; [Figure 3] is an operation flowchart of the post-processing circuit of Figure 2 according to some embodiments of the present invention; [Figure 4] is an operation flowchart of generating the shift number of Figure 1 or Figure 2 according to some embodiments of the present invention; and [Figure 5] is a schematic diagram of an operation of generating Figure 4 according to some embodiments of the present invention.

Claims

1. A convolution operation device, comprising: a multiply-accumulate operation circuit, which performs a convolution operation based on a first feature data and a weight coefficient to generate an initial operation data and generates a completion signal; and a post-processing circuit, which retrieves a first shift bit from a memory based on the completion signal, shifts the initial operation data to the right by the first shift bit to generate a shifted operation data, and performs a first truncation operation on the shifted operation data based on a preset numerical range to generate a first operation data, wherein the number of bits of the first operation data is less than the number of bits of the initial operation data, and the first shift bit is related to the value range of the initial operation data.

2. The convolution operation apparatus of claim 1, wherein the post-processing circuit performs the first truncation operation according to the preset numerical range to generate a sub-data and outputs a corresponding portion of the sub-data as the first operation data.

3. The convolution operation apparatus of claim 2, wherein if the shifted operation data exceeds an upper limit of the preset numerical range or falls below a lower limit of the preset numerical range, the post-processing circuit sets the sub-data as the data corresponding to the upper limit or the lower limit.

4. The convolution operation device as described in claim 2, wherein if the shifted operation data is within the preset value range, the post-processing circuit outputs the shifted operation data as the sub-data.

5. The convolution operation apparatus of claim 2, wherein the post-processing circuit deletes a highest-order bit and a lowest-order bit from the sub-data to obtain the corresponding bit, wherein the highest-order bit is a plurality of extended bits corresponding to a sign bit in the sub-data, and the number of bits of the lowest-order bit is the first shift number.

6. The convolution operation apparatus of claim 1, wherein the first shift is generated by a neural network and a sample dataset executed by the post-processing circuitry in an offline stage.

7. The convolution operation apparatus of claim 1, wherein the post-processing circuit further obtains a bias value from the memory according to the completion signal, and adds the bias value to the first operation data to generate a second operation data.

8. The convolution operation apparatus of claim 7, wherein the post-processing circuit includes an adder that adds the bias value to the first operation data to generate the second operation data, and the input bit width of the adder is lower than the number of bits of the initial operation data.

9. The convolution operation apparatus of claim 7, wherein the post-processing circuit further obtains a scaling value from the memory based on the completion signal, and multiplies the scaling value by the second operation data to generate a third operation data.

10. The convolution operation apparatus of claim 9, wherein the post-processing circuit further obtains a second shift number from the memory according to the completion signal, shifts the third operation data according to the second shift number to generate a fourth operation data, and performs a second truncation operation on the fourth operation data according to the preset numerical range to generate an output data.

Images