A method for realizing fast convolution without multiplication based on FPGA
By employing shifters and adders on an FPGA for data conversion and fundamental numerical solutions, and combining this with the Winograd algorithm, the problem of high DSP resource consumption and low throughput in FPGA structures is solved by replacing multiplication with shift-add operations, thus achieving efficient convolution operations.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-21
- Publication Date
- 2026-03-24
AI Technical Summary
Existing fast convolution implementation methods for FPGA architecture suffer from problems such as high DSP resource consumption, computational complexity, and low throughput.
We employ shifters and adders for data conversion and fundamental numerical solutions, and combine the Winograd algorithm to replace multiplication with shift-add operations. We design an FPGA-based method for fast convolution without multiplication, and utilize the idea of binary decomposition to reduce multiplication operations.
Without increasing DSP resource consumption, it significantly improves throughput and power efficiency while reducing LUT resource consumption.
Smart Images
Figure CN116225365B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of convolution technology, and particularly to a fast convolution implementation method without multiplication based on FPGA. BACKGROUND
[0002] Most of the existing fast convolution implementations for FPGA structure adopt traditional convolution, FFT and Winograd algorithm. The traditional convolution consumes a large number of resources and has low throughput, so the acceleration effect of convolution is not obvious enough. The FFT algorithm will bring a certain precision loss in the process of mutual transformation between spatial domain and frequency domain, and this method has obvious acceleration effect for large size convolution kernel, and almost no improvement when the size of the convolution kernel is small. The Winograd algorithm increases the number of additions to reduce the number of multiplications, thereby speeding up the convolution process. However, the extraction of public operation is only to the multiplication level, and the decomposition is not thorough enough. The above methods all contain a large number of multiplication operations, which will consume a large amount of DSPs resources, so that the large-scale deployment on FPGA is limited.
[0003] Therefore, the present application designs a new efficient fast convolution algorithm for FPGA structure to solve the problems of large DSPs consumption, complex operation and low throughput of the existing method. SUMMARY
[0004] (I) Technical problems solved
[0005] In view of the deficiencies in the prior art, the present application provides a fast convolution implementation method without multiplication based on FPGA, which solves the problems existing in the background art.
[0006] (II) Technical solutions
[0007] To achieve the above object, the present application provides the following technical solutions: a fast convolution implementation method without multiplication based on FPGA, comprising the following steps:
[0008] S1, system level structure design: using a shifter and an adder to make a 6*6 image data pass through a data conversion stage in each clock cycle, which is composed of simple constant multiplication and addition; then performing basic numerical solution on the converted data, which only needs to be completed by shifting and addition, and the solved result data is delivered to 64 parallel PE convolution calculation engines, each PE receives the solved data and also receives the corresponding converted convolution kernel binary decomposition parameters; then the convolution output result between the input image block and each convolution kernel is stored in the buffer at the output, and then the accumulation operation is performed to calculate the convolution result in the corresponding dimension;
[0009] S2, F(4x4, 3x3) convolution processing unit design: at present, Winograd algorithm has become a high efficient algorithm for realizing convolution acceleration on FPGA, after comprehensively considering transformation complexity, precision loss and memory bandwidth, etc., m=4 and r=3 are selected as parameters in F(mxm, rxr), and the design of the convolution processing engine unit is carried out under the parameters;
[0010] The data input size is (m+r-1)x(m+r-1)=6x6, and the convolution kernel size is rxr=3x3; the output m x m=4x4 of each convolution result needs to go through three stages, which are data transfer stage, shift addition stage and inverse transformation stage;
[0011] After conversion, the size of the input data is 6x6, and then the corresponding four basic values of each data are calculated; the size of the convolution kernel is 3x3, which is converted to 6x6, and then the corresponding binary form is processed according to the algorithm, so that each four value bits contain at most two 1s, and the decomposition results are recorded in the form of parameters; the number of parameters of each data after binary decomposition is assumed to be x; at this time, the original element-by-element multiplication can be represented by the following operation: according to the corresponding element decomposition parameters x, the data is selected from the basic value obtained by itself for shift addition; now only shift addition operation can complete the corresponding multiplication operation;
[0012] The function completed by one PE convolution processing engine includes shift addition and inverse transformation operation; the convolution kernel is determined before the inference stage, and the convolution and the corresponding conversion matrix are also determined under the condition of m, r=4, 3, so the conversion of the convolution kernel and the corresponding binary parameter decomposition can be calculated in advance;
[0013] S3, fixed-point number multiplication circuit design: when calculating the multiplication of two INT16 fixed-point numbers A and B, the basic values 3A, 5A and 9A of A can be calculated first, then the corresponding data is selected from the basic values of A for shift addition according to the binary decomposition result of B, and the result of AxB can be obtained;
[0014] The basic value can be calculated by using a shifter and an adder; then four basic values are selected for different bit shifts, the results after shift are added and processed, and finally the correction is added to obtain the multiplication result.
[0015] (Three) beneficial effects
[0016] The application provides a fast convolution implementation method without multiplication based on FPGA, which has the following beneficial effects:
[0017] This invention, based on the idea of binary decomposition, proposes an algorithm that replaces multiplication with shift-addition, resulting in a highly regular operational structure that is easy to implement in hardware. By combining this method with the Winograd algorithm, a system-level architecture for fast convolution is designed and implemented. Compared with existing technologies, our design achieves significant improvements in throughput and power efficiency, while requiring no DSP resources and reducing LUT resource consumption. Attached Figure Description
[0018] Figure 1 A system-level architecture diagram;
[0019] Figure 2 This is a schematic diagram of a fast convolution processing engine;
[0020] Figure 3 The circuit structure diagram for basic numerical solution;
[0021] Figure 4 The image shows the result obtained by shifting and adding numerical values.
[0022] Figure 5 The circuit diagram for performing 16-bit fixed-point multiplication using shift addition. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] This invention provides a technical solution: a fast convolution implementation method based on FPGA without multiplication, comprising the following steps:
[0025] S1. System-level architecture design: Convolutional kernel transformation and its fixed-point binary form decomposition stage can be computed offline before the forward pass of the CNN; however, the transformation of the input data needs to be performed online because it depends on the different input image data. Figure 1 The system-level implementation architecture diagram of the design is given.
[0026] In each clock cycle, a 6×6 image data block is processed through a data transformation stage using shifters and adders. This stage consists of simple constant multiplication and addition. The transformed data is then subjected to basic numerical solutions, which can be completed with only shifting and addition. The resulting data is fed into 64 parallelized PE convolutional computation engines. Each PE receives the solved data, as well as the parameters of its corresponding transformed convolutional kernel (binary decomposition). The convolution output between the input image block and each kernel is then stored in an output buffer. Finally, an accumulation operation is performed to calculate the convolution result in the corresponding dimension.
[0027] S2, F(4×4, 3×3) Convolution Processing Unit Design: Currently, the Winograd algorithm has become an efficient algorithm for accelerating convolution on FPGAs. Considering factors such as transformation complexity, accuracy loss, and memory bandwidth, m=4 and r=3 are selected as parameters in F(m×m, r×r). The convolution processing engine unit is designed under these parameters, with the specific structure as follows: Figure 2 As shown;
[0028] The data input size is (m+r-1)×(m+r-1)=6×6, and the convolution kernel size is r×r=3×3; the output of each convolution result m×m=4×4 goes through three stages, namely the data transfer stage, the shift and addition stage, and the inverse transformation stage.
[0029] The input data is transformed to a size of 6×6, and then the four basic values corresponding to each data point are calculated. The convolution kernel size is 3×3, which is transformed to a size of 6×6. Then, the corresponding binary form is processed according to the proposed algorithm so that each four bits contain at most two 1s. The decomposition result is recorded as parameters, and the number of parameters after binary decomposition of each data point is assumed to be x. At this point, the original element-wise multiplication can be represented as follows: based on the relevant parameters x decomposed from the corresponding element, select data from the basic values obtained by itself and perform shifting and addition. Now, the corresponding multiplication operation can be completed by simply performing shifting and addition operations.
[0030] The dashed box contains the functions performed by a PE convolutional processing engine, including shift addition and inverse transformation operations. The convolution kernel is determined before the inference stage. When m,r=4,3, the convolution and the corresponding transformation matrix are also determined. Therefore, the transformation of the convolution kernel and the corresponding binary parameter decomposition can be calculated in advance.
[0031] S3. Fixed-point number multiplication circuit design: When multiplying two INT16 fixed-point numbers A and B, the basic values of A (3A, 5A, 9A) can be calculated first. Then, based on the binary decomposition result of B, the corresponding data from the basic values of A are selected, shifted, and added to obtain the result A × B. The circuit structure for solving the basic values of A is as follows: Figure 3 As shown;
[0032] from Figure 3 As can be seen, solving for the basic values only requires shifters and adders; then, four values are selected from the basic values and shifted a different number of times, the shifted results are summed, and finally, a correction is added to obtain the multiplied result. The specific circuit implementation is as follows: Figure 4 As shown;
[0033] At this point, the overall circuit structure design diagram for INT16 fixed-point multiplication can be provided, such as... Figure 5 As shown;
[0034] from Figure 5 As can be seen from this, the basic numerical solution is first performed on the data, then the required data is selected from the basic numerical solution, shifted and added, and finally a correction is added to obtain the multiplication result.
[0035] The above first presents the system-level hardware circuit design for completing the convolution operation, then introduces the architecture design of the convolution engine computing unit, and finally designs the lowest-level shift-add circuit based on our proposed idea to complete the multiplication of fixed-point numbers.
[0036] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0037] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for implementing fast convolution without multiplication based on FPGA, characterized in that, Includes the following steps: S1. System-level architecture design: In each clock cycle, a 6×6 image data block is processed through a data transformation stage using shifters and adders. The transformed data is then subjected to basic numerical solutions, and the results are fed into 64 parallelized PE convolutional computation engines. Each PE receives the solved data, as well as the parameters derived from the binary decomposition of its corresponding transformed convolutional kernel. The convolution output between the input image block and each convolutional kernel is then stored in a buffer at the output. Finally, an accumulation operation is performed to calculate the convolution result in the corresponding dimension. S2, F(4×4,3×3) Convolution Processing Unit Design: The most efficient algorithm for accelerating convolution on FPGA is the Winograd algorithm. In F(m×m,r×r), m=4 and r=3 are selected as parameters, and the convolution processing engine unit is designed under these parameters. The data input size is (m+r-1)×(m+r-1)=6×6, and the convolution kernel size is r×r=3×3; the output of each convolution result m×m=4×4 goes through three stages, namely the data transfer stage, the shift and addition stage, and the inverse transformation stage. The input data is transformed to a size of 6×6, and then the four basic values corresponding to each data point are calculated. The convolution kernel size is 3×3, which is transformed to a size of 6×6. Then, the corresponding binary form is processed according to the proposed algorithm so that each four bits contain at most two 1s. The decomposition result is recorded as parameters, and the number of parameters after binary decomposition of each data point is assumed to be x. At this point, the original element-wise multiplication can be represented as follows: based on the relevant parameters x decomposed from the corresponding element, select data from the basic values obtained by itself and perform shifting and addition. Now, the corresponding multiplication operation can be completed by simply performing shifting and addition operations. The functions performed by a PE convolution processing engine include shift addition and inverse transformation operations; the convolution kernel is determined before the inference stage, and the convolution and the corresponding transformation matrix are also determined when m,r=4,3. Therefore, the transformation of the convolution kernel and the corresponding binary parameter decomposition can be calculated in advance. S3. Fixed-point number multiplication circuit design: When calculating the multiplication of two INT16 fixed-point numbers A and B, you can first calculate the basic values of A, such as 3A, 5A, and 9A. Then, based on the binary decomposition result of B, select the corresponding data from the basic values of A, shift and add them to get the result of A×B. Based on the binary decomposition method, we perform binary coefficient decomposition on fixed-point numbers and then extract the common coefficients between different multipliers. We integrate the common operation into the hardware structure and implement it at the hardware level. This hardware operation structure is extremely regular. Solving for basic values only requires shifters and adders; Then, select four values from the basic values and perform different numbers of shifts. Sum the shifted results and add a correction to get the multiplied result.
2. The method for implementing fast convolution without multiplication based on FPGA according to claim 1, characterized in that: In S1, a 6×6 image data is processed through a data conversion stage, which consists of simple constant multiplication and addition. The basic numerical solution of the converted data can be completed simply by shifting and adding.