Convolution operation device
By performing bit shift and cutoff operations in the post-processing circuit of the convolution operation device, the problem that processing circuits in the prior art need to support high bit width is solved, and the effect of reducing circuit costs and ensuring output accuracy is achieved.
Patent Information
- Application Number
- CN202411169933.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-13
AI Technical Summary
In the existing convolutional computing device, the processing circuit needs to support a large input bit width to receive the multiplication and accumulation operation result, resulting in an increase in overall cost.
By configuring the shift number and a preset numerical range in the offline stage, the post-processing circuit performs bit shift and cut-off operations on the initial operation data, reducing the number of bits of the operation data, reducing the complexity of the post-processing circuit and the required bit width.
Effectively reduces the overall circuit area and cost while ensuring the accuracy of output data.
Smart Images
Figure CN119128349A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of convolution operation, and in particular to a convolution operation device. Background Art
[0002] Convolution operation devices are often used to implement neural networks to achieve various types of recognition applications. In existing applications, the processing circuit in the convolution operation device needs to process the operation results generated by the multiplication and addition operation circuit to generate the final output. Since the number of bits of the operation results generated by the multiplication and addition operation circuit is usually very high, the processing circuit needs to support a larger input bit width to fully receive and process the operation results. This will significantly increase the overall cost of the processing circuit. Summary of the invention
[0003] One of the purposes of the present application is to provide a convolution operation device that can reduce the cost of circuit implementation, so as to improve the deficiencies of the prior art.
[0004] In some embodiments, the convolution operation device includes a multiplication-accumulation operation circuit and a post-processing circuit. The multiplication-accumulation operation circuit performs a convolution operation according to a first feature data and a weight coefficient to generate an initial operation data, and generates a completion signal. The post-processing circuit obtains a first shift number from a memory according to the completion signal, performs a bit shift on the initial operation data according to the first shift number to generate a shifted operation data, and performs a first truncation operation on the shifted operation data according to a preset value range to generate a first operation data, wherein the number of bits of the first operation data is less than the number of bits of the initial operation data.
[0005] The features, implementation and effects of the present application are described in detail below with reference to the accompanying drawings for a preferred embodiment. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Figure 1 A schematic diagram of a convolution operation device is drawn according to some embodiments of the present application;
[0007] Figure 2 According to some embodiments of the present application, Figure 1 A schematic diagram of a post-processing circuit;
[0008] Figure 3 According to some embodiments of the present application, Figure 2 Operation flow chart of the post-processing circuit;
[0009] Figure 4 To generate a drawing according to some embodiments of the present application Figure 1 or Figure 2 An operation flow chart of the number of shifts; and
[0010] Figure 5 To generate a drawing according to some embodiments of the present application Figure 4 A schematic diagram of an operation.
[0011] Reference numerals:
[0012] 100: convolution operation device;
[0013] 110, 130: memory;
[0014] 120: multiplication and accumulation circuit;
[0015] 140: post-processing circuit;
[0016] 210, 250: shifter;
[0017] 220, 260: Interceptor;
[0018] 230: adder;
[0019] 240: multiplier;
[0020] 501, 502: bits;
[0021] AB: initial operation data;
[0022] BV: bias value;
[0023] D1, D2, D3, D4: operation data;
[0024] DO: output data;
[0025] DV: valid data;
[0026] IB: characteristic data;
[0027] KB: weight coefficient;
[0028] OBS: number of shifts;
[0029] PR: preset value range;
[0030] S310, S320, S330, S340, S350, S360: operation;
[0031] S410, S420, S430, S440: operation;
[0032] SAB: Calculation data after shifting;
[0033] SS: completion signal;
[0034] SV: scaling value;
[0035] VBS: Number of shifts. DETAILED DESCRIPTION
[0036] All words used herein have their ordinary meanings. The definitions of the above words in commonly used dictionaries, including any use examples of the words discussed herein in the content of this application are only examples and should not limit the scope and meaning of this application. Similarly, this application is not limited to the various embodiments shown in this specification.
[0037] As used herein, "coupled" or "connected" may refer to two or more elements making direct physical or electrical contact with each other, or making indirect physical or electrical contact with each other, or may refer to two or more elements operating or acting on each other. As used herein, the term "circuit" may refer to a device composed of at least one transistor and / or at least one active and passive element connected in a certain manner to process signals.
[0038] Figure 1 A schematic diagram of a convolution operation device 100 is drawn according to some embodiments of the present application. The convolution operation device 100 includes a memory 110, a multiply accumulate (MAC) circuit 120, a memory 130, and a post-processing circuit 140. In some embodiments, each of the memory 110 and the memory 130 may be, but is not limited to, a static random access memory.
[0039] The multiplication and accumulation operation circuit 120 performs a convolution operation according to the feature data IB and the weight coefficient KB to generate the initial operation data AB, and stores the initial operation data AB in the memory 110 and generates a completion signal SS accordingly. In some embodiments, the feature data IB may be input feature data, such as, but not limited to, image data, voice data, etc. In some embodiments, the feature data IB and the weight coefficient KB may also be stored in the memory 110. The memory 130 may store a plurality of pre-configured shift numbers OBS, a plurality of bias values BV, a plurality of scale values SV, and a plurality of shift numbers VBS. The above-mentioned plurality of parameters are provided to the post-processing circuit 140 to process the initial operation data AB. The configuration method of the above-mentioned parameters will be described later. It is understandable that the memory 110 and the memory 130 may be combined into one, which stores the aforementioned feature data IB, the weight coefficient KB, the initial operation data AB, a plurality of shift numbers OBS, a plurality of bias values BV, a plurality of scale values SV, and a plurality of shift numbers VBS.
[0040] The post-processing circuit 140 obtains the initial operation data AB from the memory 110 according to the completion signal SS, and obtains the corresponding shift number OBS, offset value BV, scaling value SV and shift number VBS from the memory 130. Then, the post-processing circuit 140 can perform bit shifting on the initial operation data AB to generate shifted operation data (e.g. Figure 2 , and performs a clip operation on the shifted operation data according to a preset value range PR to generate a first operation data (for example, Figure 2 The operation data D1 in the first operation data is obtained by the above-mentioned truncation operation, so that the number of bits of the first operation data is less than the number of bits of each of the initial operation data AB and the shifted operation data. For example, the number of bits of each of the initial operation data AB and the shifted operation data can be 48, and the number of bits of the first operation data is 16. The above-mentioned bit number values are for example only, and the present application is not limited thereto. The detailed description of the truncation operation will be referred to later. Figure 5 In different embodiments, the information of the preset value range PR may be stored in the memory 130 , or may be stored in a register (not shown) in the post-processing circuit 140 .
[0041] Furthermore, the post-processing circuit 140 may add the offset value BV to the first operation data to generate the second operation data (eg Figure 2 The operation data D2 in the figure is multiplied by the scaling value SV and the second operation data to generate the third operation data (for example, Figure 2 Finally, the post-processing circuit 140 may shift the third operation data according to the shift number VBS to generate fourth operation data (for example, Figure 2 The post-processing circuit 140 may store the output data DO in a memory (not shown). The configuration and related operations of the post-processing circuit 140 will be described later. Figure 2 and Figure 3 illustrate.
[0042] Figure 2 According to some embodiments of the present application, Figure 1 A schematic diagram of a post-processing circuit 140 is shown, and Figure 3 According to some embodiments of the present application, Figure 2The operation flow chart of the post-processing circuit 140 is shown in FIG. The post-processing circuit 140 includes a shifter 210, a truncation device 220, an adder 230, a multiplier 240, a shifter 250, and a truncation device 260. In some embodiments, each of the above-mentioned multiple units in the post-processing circuit 140 can be implemented by a register circuit, a digital circuit, and / or a logic circuit. For the convenience of description, the operations of the above-mentioned multiple units will be described in conjunction with FIG. Figure 3 Explain together.
[0043] In operation S310, the shifter 210 obtains the initial operation data AB from the memory 110 and the shift number OBS from the memory 130 according to the completion signal SS, and performs bit shifting on the initial operation data AB according to the shift number OBS to generate shifted operation data SAB. For example, the initial operation data AB may include 48 bits, and the value of the shift number OBS is m, then the shifter 210 may shift the 48 bits of the initial operation data AB to the right by m bits, and output the plurality of bits after the right shift is completed as the shifted operation data SAB, wherein the value m may be a non-negative integer (including 0).
[0044] In operation S320, the truncation device 220 performs a truncation operation on the shifted operation data SAB according to a preset value range PR to generate operation data D1. In some embodiments, the preset value range PR may be determined according to the quantization scene bandwidth of the actual application. For example, if the number of bits of the initial operation data AB is 16, the preset value range PR may be set to (-32767, 32767) according to the number of bits, with an upper limit of 32767 and a lower limit of -32767. If the value of the shifted operation data SAB is within the preset value range PR, the truncation device 220 may output the shifted operation data SAB as the first sub-data. Alternatively, if the value of the shifted operation data SAB exceeds the upper limit of the preset value range PR (e.g., 32767) or is lower than the lower limit (e.g., -32767), the truncation device 220 may set the corresponding one of the upper limit and the lower limit as the first sub-data. For example, if the value of the shifted operation data SAB exceeds 32767, the truncation device 220 may set the aforementioned first sub-data to the data corresponding to the upper limit value 32767. On the other hand, if the value of the shifted operation data SAB is lower than -32767, the truncation device 220 may set the aforementioned first sub-data to the data corresponding to the lower limit value -32767. Through the above operation, unnecessary error overflows in subsequent operations can be avoided to improve the accuracy of the overall operation. Furthermore, the truncation device 220 may output some bits in the first sub-data as the operation data D1. The content here will be referred to later. Figure 5 In some embodiments, the bit truncation device 220 may be implemented by, but not limited to, a comparator circuit and a selector circuit.
[0045] In operation S330, the adder 230 obtains the bias value BV from the memory 130, and adds the bias value BV to the operation data D1 to generate the operation data D2. In operation S340, the multiplier 240 obtains the scaling value SV from the memory 130, and multiplies the scaling value SV and the operation data D2 to generate the operation data D3. In operation S350, the shifter 250 obtains the shift number VBS from the memory 130, and performs bit shifting on the operation data D3 according to the shift number VBS to generate the operation data D4. Similar to operation S310, the shifter 250 can right-shift a plurality of bits in the operation data D3 according to the shift number VBS, and output the shifted plurality of bits as the operation data D4.
[0046] In operation S360, the truncation device 260 may perform a truncation operation on the operation data D4 according to the preset value range PR to generate output data DO. Similar to operation S320, the truncation device 260 may perform a truncation operation on the operation data D4 according to the preset value range PR to generate second sub-data, and output some bits of the second sub-data as output data DO.
[0047] In some related technologies, the convolution operation device directly convolves the input feature map data. Under this condition, the data size that can be stored in the memory in the convolution operation device and the input bit width of the post-processing circuit (such as a multiplier) must be able to support input feature map data with a higher bit number. For example, if the number of bits of the input feature map data is 48, the data size that can be stored in the memory and the input bit width of the multiplier must each be at least 48 bits. Similarly, in the multiple related technologies, other circuits in the post-processing circuit also need to be set as operation circuits that can process high-bit data. In this way, the overall circuit area will be too large, resulting in excessively high overall circuit costs.
[0048] Compared with the above-mentioned related art, in some embodiments of the present application, by configuring the corresponding shift number OBS according to the corresponding convolution operator in advance in the offline stage, the convolution operation device 100 can perform bit shift and truncation operations on the initial operation data AB according to the shift number OBS and the preset value range PR (which corresponds to the quantization range of the usage scenario) to generate operation data D1 when receiving the initial operation data AB, so that the number of bits of the operation data D1 can be less than the number of bits of the initial operation data AB. In this way, the complexity and required bit width of other circuits in the post-processing circuit 140 can be reduced, thereby reducing the overall circuit area and cost. For example, if the number of bits of the initial operation data AB is 48, the number of bits of the operation data D1 after the bit shift and truncation operation is 16, and the input bit width of the adder 230 can be reduced to 16 bits accordingly, which can be significantly lower than the number of bits of the initial operation data AB.
[0049] It should be understood that for the sake of clarity and brevity, Figure 2 Only the post-processing circuit 140 corresponding to a single channel is shown. In practical applications, the convolution processing device 100 may include a plurality of post-processing circuits 140 in parallel, which correspond to a plurality of channels in the neural network respectively, and each channel may be used for the corresponding initial operation data AB and generate the corresponding output data DO accordingly. In some embodiments, the post-processing circuit 140 corresponding to each channel may fetch the shift number OBS, the bias value BV, the scaling value SV and the shift number VBS corresponding to the channel from the memory 130. In some embodiments, the group of circuits corresponding to each channel is used to execute an operator in the neural network executed by the post-processing circuit 140. Therefore, in some embodiments, the shift number OBS, the bias value BV, the scaling value SV and the shift number VBS used by each channel may be generated in advance in an offline stage by the neural network executed by the post-processing circuit 140 and the sample data set.
[0050] Figure 4 To generate a drawing according to some embodiments of the present application Figure 1 or Figure 2 In some embodiments, Figure 4 The multiple processes in the method may be implemented in an offline stage (eg, but not limited to, a circuit design stage or a circuit measurement stage), and may be executed by a machine or a computer that executes a chip design tool or circuit simulation software.
[0051] In operation S410, all operators in the neural network executed by the post-processing circuit 140 are simulated by using a function modeled by a computer-readable instruction set to establish a simulation test network. In some embodiments, the instruction set can be established by C language, but the present application is not limited thereto. In operation S420, a sample data set is input to the simulation test network to obtain the value range of all initial operation data AB corresponding to the sample data set. In some embodiments, the sample data set is a sample data established in advance. For example, if the neural network executed by the post-processing circuit 140 is applied to image recognition for vehicles, the sample data set can be a plurality of image data established in advance, and its content can be a common scene related to vehicle image recognition. The above types of sample data sets are only examples, and the present application is not limited thereto. In operation S430, multiple shift numbers OBS are confirmed according to the value range of each initial operation data AB. In operation S440, the multiple shift numbers OBS are updated to the simulation test network, and operations S420 to S430 are performed again to adjust the multiple shift numbers OBS according to the output data generated by the simulation test network. In this way, multiple shift numbers OBS that can provide accurate image recognition results (equivalent to accurate output data DO) can be found by repeatedly performing the above multiple operations.
[0052] Figure 5 To generate a drawing according to some embodiments of the present application Figure 4 FIG4 is a schematic diagram of operation S430 of FIG4. In operation S430, the value range of all initial operation data AB corresponding to the sample data set can be obtained. The highest part of the bits of the initial operation data AB is a plurality of extended bits of the sign bit. Specifically, if bit 501 is a sign bit, all bits higher than bit 501 are extended bits of the sign bit. Taking the largest initial operation data AB as an example, bit 501 is a logical value of 0, indicating that the initial operation data AB is a positive value, and the extended bits higher than bit 501 are all logical values 0. Similarly, taking the smallest initial operation data AB as an example, bit 501 is a logical value of 1, indicating that the initial operation data AB is a negative value, and the extended bits higher than bit 501 are all logical values 1. Furthermore, the lowest part of the bits of the initial operation data AB is set to all bits between the next bit of bit 502 and the lowest bit, which is the number of bits that can be deleted. That is, the number of bits of the lowest part of the bits is the shift number OBS.
[0053] In other words, the shift number OBS found through the above training process can make all bits between bit 501 and bit 502 valid data DV in the initial operation data AB, which can effectively reduce the total number of bits and provide valid content with sufficient accuracy. Figure 5 The effective data DV shown is equivalent to the aforementioned Figure 3In operation S320, as described above, the post-processing circuit 140 may process the shifted operation data SAB according to the preset value range PR to generate the first sub-data, and convert the corresponding part of the bits (equivalent to Figure 5 The valid data DV or part thereof in the operation data SAB is output as the operation data D1. That is, the post-processing circuit 140 can delete the highest part bits and the lowest part bits in the operation data SAB by truncation operation, so as to obtain the shifted operation data SAB. The post-processing circuit 140 can perform operation S350 in a similar manner to generate the operation data D4. Incidentally, in some embodiments, in the actual execution process, the actual number of bits of the valid data DV may also be less than the preset number of data bits (for example, the aforementioned 16). Under this condition, the shift number OBS can be set to 0.
[0054] Similarly, in some embodiments, in the offline stage, the bias value BV can be obtained by the following operations: the original bias value of the neural network is expressed in the form of a fixed-point number; the original bias value is bit-shifted according to the shift number OBS to generate the bias value BV; if the bias value BV is within a preset numerical range (for example, the preset numerical range PR mentioned above), the bias value BV is kept unchanged; if the bias value BV exceeds the upper limit of the preset numerical range, the bias value BV is set to the upper limit; and if the bias value BV is lower than the lower limit of the preset numerical range, the bias value BV is set to the lower limit. Similarly, after obtaining the numerical range of all initial operation data AB, the above operations can be further used to configure multiple corresponding bias values BV according to multiple shift numbers OBS.
[0055] Similarly, in some embodiments, in the offline stage, the scaling value SV and the shift number VBS can be obtained by the following operations: estimating the original scaling value and the original shift number through the mathematical model of the original neural network; setting the original scaling value to the scaling value SV; and subtracting the shift number OBS from the original shift number to generate the shift number VBS.
[0056] The above operations for generating the shift number OBS, the offset value BV, the scaling value SV, and the shift number VBS are only used as examples, and various operations for performing bit truncation operations on the input data of the post-processing circuit and their offline generation methods are all within the scope of this application. For example, in each of the above examples, the bit shift is taken as an example of right shift, but this application is not limited to this. In other embodiments, according to actual application requirements, the above bit shift can also be a left shift.
[0057] In summary, the convolution operation device provided in some embodiments of the present application can pre-configure multiple operation operations in the offline stage to perform shift and truncation operations on the input of the post-processing circuit. In this way, the required space of the memory and the input bit width required by the subsequent circuits in the processing circuit can be further reduced while ensuring the accuracy of the output data, thereby reducing the overall circuit cost.
[0058] Although the embodiments of the present application are described above, these embodiments are not intended to limit the present application. Technicians in this technical field may change the technical features of the present application based on the explicit or implicit contents of the present application. All these changes may fall within the scope of patent protection sought by the present application. The scope of patent protection of the present application shall be based on the scope of the patent application of this specification.
Claims
1. A convolution operation device, characterized in that: include: A multiplication-accumulation operation circuit performs a convolution operation according to a first characteristic data and a weight coefficient to generate an initial operation data, and generates a completion signal; as well as A post-processing circuit obtains a first shift number from a memory according to the completion signal, performs a bit shift on the initial operation data according to the first shift number to generate a shifted operation data, and performs a first truncation operation on the shifted operation data according to a preset value range to generate a first operation data, wherein the number of bits of the first operation data is less than the number of bits of the initial operation data.
2. The convolution operation device according to claim 1, characterized in that: The post-processing circuit performs the first truncation operation according to the preset value range to generate a sub-data and outputs a corresponding part of bits of the sub-data as the first operation data.
3. The convolution operation device according to claim 2, characterized in that: If the shifted operation data exceeds an upper limit of the preset value range or is lower than a lower limit of the preset value range, the post-processing circuit sets the sub-data to data corresponding to the upper limit or the lower limit.
4. The convolution operation device according to claim 2, characterized in that: If the shifted operation data is within the preset value range, the post-processing circuit outputs the shifted operation data as the sub-data.
5. The convolution operation device according to claim 2, characterized in that: The post-processing circuit deletes a highest portion of bits and a lowest portion of bits in the sub-data to obtain the corresponding portion of bits, wherein the highest portion of bits is a plurality of extended bits corresponding to a sign bit in the sub-data, and the number of bits of the lowest portion of bits is the first shift number.
6. The convolution operation device according to claim 1, characterized in that: The first shift amount is generated in an offline stage by a neural network executed by the post-processing circuit and a sample data set.
7. The convolution operation device according to claim 1, characterized in that: The post-processing circuit also obtains an offset value from the memory according to the completion signal, and adds the offset value to the first operation data to generate a second operation data.
8. The convolution operation device according to claim 7, characterized in that: The post-processing circuit includes an adder, which adds the offset value and the first operation data to generate the second operation data, and the input bit width of the adder is lower than the number of bits of the initial operation data.
9. The convolution operation device according to claim 7, characterized in that: The post-processing circuit further obtains a scaling value from the memory according to the completion signal, and multiplies the scaling value and the second operation data to generate a third operation data.
10. The convolution operation device according to claim 9, characterized in that: The post-processing circuit also obtains a second shift number from the memory according to the completion signal, shifts the third operation data according to the second shift number to generate a fourth operation data, and performs a second truncation operation on the fourth operation data according to the preset value range to generate an output data.