Frequency domain convolution operation acceleration system for 8-bit frequency domain convolutional neural network

Abstract

The application provides a frequency domain convolution operation acceleration system for an 8bit frequency domain convolution neural network, relates to the technical field of frequency domain convolution operation acceleration design, and solves the problems of slow convolution calculation of the current frequency domain convolution neural network, difficulty in deployment to a hardware platform with limited resources, influence on product market time and design cost, and the like, obtains a pair of 8bit multiplication data, packs the obtained pair of 8bit multiplication data, performs multiplication calculation on the packed pair of 8bit multiplication data, outputs a frequency domain convolution multiplication result, accumulates the frequency domain convolution multiplication result, outputs a frequency domain convolution accumulation result, restores the frequency domain convolution accumulation result, and obtains a final frequency domain convolution operation complex output result; the application improves the operation speed of the frequency domain convolution, facilitates deployment of the frequency domain convolution to a hardware platform, improves the efficiency of processing digital signals in an FPGA accelerator, accelerates product market time and reduces product design cost.

Description

Acceleration system for frequency domain convolution operations in 8-bit frequency domain convolutional neural networks Technical Field

[0001] This invention relates to the technical field of frequency domain convolution operation acceleration design, and in particular to a frequency domain convolution operation acceleration system for 8-bit frequency domain convolutional neural networks. Background Technology

[0002] Convolutional neural networks (CNNs) have demonstrated excellent performance in many computer vision tasks, such as image recognition, and possess classification and recognition capabilities. They are also widely used in the design of numerous products, including intelligent recognition sensors and autonomous vehicles. However, the most advanced CNNs currently available generally have deep network layers and high computational complexity, making them difficult to deploy on resource-constrained hardware platforms such as FPGAs and ASICs. The computational speed of CNNs affects product time-to-market and design costs. Therefore, the design of hardware accelerators for CNNs has become one of the current research hotspots in the industry. Many fast algorithms for accelerating convolutional computation in CNNs have been proposed, such as the Fast Fourier Transform (FFT) algorithm, which can transform traditional spatial domain convolution into frequency domain convolution. This allows the use of the simpler element-wise multiplication (EWMM) operation to replace the traditional complex spatial convolution operation, effectively saving computational resources. The CNN transformed by the FFT algorithm is also known as a frequency domain convolutional neural network.

[0003] Currently, FPGA-based convolutional neural network accelerators have received widespread attention and research in the field of CNN hardware accelerators due to their ability to significantly reduce time-to-market and design costs. Existing literature discloses an FPGA-based method for accelerating convolutional computation in the frequency domain of convolutional neural networks. First, it uses FFT and overlapping addition to map the frequency domain algorithm onto a highly parallel 2D convolutional network based on overlapping addition on the FPGA. Then, it proposes parallel processing of the CPU and FPGA in shared memory, reducing inter-layer data remapping latency. Finally, it uses the data parallelism and task parallelism of the overlapping addition-based 2D convolutional network to measure the overall system acceleration. However, this method uses high-precision data, resulting in high memory consumption on the FPGA. This reduces the efficiency of the FPGA accelerator in processing digital signals in the frequency domain convolution computation, leading to slow frequency domain convolution operation speeds and making it difficult to apply on resource-constrained hardware platforms, thus impacting time-to-market and design costs. Summary of the Invention

[0004] To address the issues of slow convolution computation in current frequency-domain convolutional neural networks, which makes them difficult to deploy on resource-constrained hardware platforms and impacts product time-to-market and design costs, this invention proposes a frequency-domain convolution computation acceleration system for 8-bit frequency-domain convolutional neural networks. This system improves the computation speed of frequency-domain convolution, facilitates its deployment on hardware platforms, enhances the efficiency of digital signal processing in FPGA accelerators, accelerates product time-to-market, and reduces product design costs.

[0005] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows:

[0006] A frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network, the system comprising:

[0007] The data acquisition module is used to acquire a pair of 8-bit multiplication expressions.

[0008] The data packaging module is used to package the acquired pair of 8-bit multiplication data and transmit the packaged pair of 8-bit multiplication data to the complex number calculation module.

[0009] The complex number calculation module is used to perform frequency domain convolution complex multiplication calculation on two 8-bit data in a packaged pair of 8-bit multiplication expressions simultaneously, and output the frequency domain convolution complex multiplication calculation result to the accumulation module;

[0010] The accumulation module is used to store and accumulate the results of frequency domain convolution multiplication, and output the frequency domain convolution accumulation result;

[0011] The data recovery module is used to recover the frequency domain convolution summation result and obtain the final complex number output result of the frequency domain convolution operation.

[0012] In this technical solution, a pair of 8-bit multiplication expressions is first obtained from the frequency domain convolution operation. This pair of 8-bit multiplication expressions is then packaged into the complex number calculation module, avoiding the significant drawbacks of high memory consumption and bandwidth in the frequency domain convolution operation. Then, in the complex number calculation module, this pair of 8-bit multiplication expressions is simultaneously subjected to complex multiplication calculation, accelerating the frequency domain convolution operation speed. The frequency domain convolution multiplication result is output, and the frequency domain convolution multiplication result is stored and accumulated to obtain the frequency domain convolution accumulation result. Finally, by restoring the frequency domain convolution accumulation result, the final complex output result of the frequency domain convolution operation is obtained, completing a series of accelerations for the frequency domain convolution operation, effectively improving the efficiency of digital signal processing and increasing the speed of frequency domain convolution operation.

[0013] Preferably, the data acquisition module obtains a pair of 8-bit multiplication data from the frequency domain convolution operations of different output channels in the 8-bit frequency domain convolutional neural network.

[0014] Preferably, the data packaging module packages a pair of 8-bit multiplication data into a 27-bit wide data, and transmits it to the complex number calculation module for storage and calculation.

[0015] Preferably, the complex number calculation module includes a first register A, a second register B, a pre-adder / subtractor, a multiplier, a third register C, an adder, a fourth register D, and a fifth register E. The first register A and the second register B operate in parallel. The output of the data packing module is connected to the inputs of the first register A and the second register B, respectively. The outputs of the first register A and the second register B are connected to the inputs of the adder / subtractor. The outputs of the adder / subtractor and the third register C are connected to the input of the multiplier. The outputs of the multiplier and the fourth register D are connected to the input of the adder. The output of the adder is connected to the input of the fifth register E. The output of the fifth register E is connected to the input of the accumulation module. At the input end, the 27-bit wide data output from the data packing module is stored in the first register A and the second register B respectively. The 27-bit wide data stored in the first register A and the second register B is input into the adder / subtractor for addition and subtraction operations, and the result of the addition and subtraction operation is output. The result of the addition and subtraction operation and the multiplication number stored in the third register C are input into the multiplier for multiplication operation, and the result of the multiplication calculation is output. The result of the multiplication calculation and the correction value stored in the fourth register D are added by the adder to correct the result of the multiplication calculation output by the multiplier. The result of the frequency domain convolution multiplication is output and stored in the fifth register E. The fifth register E outputs the stored frequency domain convolution multiplication result to the accumulation module.

[0016] Preferably, the first register A and the second register B each have a bit width of 27 bits, the third register C has a bit width of 18 bits, and the fourth register D and the fifth register E each have a bit width of 48 bits.

[0017] Preferably, the pair of 8-bit multiplication data includes an 8-bit low-order term data and an 8-bit high-order term data. The 8-bit low-order term data is located in the lower 8 bits of the first register A and the second register B, respectively. The 8-bit high-order term data is located in the upper 18 to 25 bits of the first register A and the second register B, respectively. The highest bit of the first register A and the second register B is filled with the sign bit of the high-order term, and the other bits in the middle of the first register A and the second register B are filled with zeros.

[0018] Preferably, the multiplication result includes the low-order term multiplication result and the high-order term multiplication result, and the fourth register D corrects the offset value of the low-order term multiplication result.

[0019] Preferably, the frequency domain convolution multiplication result includes a low-bit complex multiplication result and a high-bit complex multiplication result, which are stored in the fifth register E.

[0020] Preferably, the accumulation module includes two parallel separation registers and an accumulation submodule. The output of the fifth register E is connected to the input of each separation register, and the output of each separation register is connected to the input of the accumulation submodule. The separation registers separate the low-bit complex multiplication results and the high-bit complex multiplication results stored in the fifth register E, and input the separated frequency domain convolution multiplication results into the accumulation submodule. The accumulation submodule performs separate accumulation operations on the low-bit complex multiplication results and the high-bit complex multiplication results respectively.

[0021] Preferably, the frequency domain convolution accumulation result output by the accumulation module is one of the components of the complex number output result of the frequency domain convolution operation.

[0022] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0023] This invention proposes a frequency-domain convolution operation acceleration system for 8-bit frequency-domain convolutional neural networks. First, a pair of 8-bit multiplication expressions is obtained from the frequency-domain convolution operation. This pair of 8-bit multiplication expressions is then packaged into a complex number calculation module, avoiding the significant high memory consumption and bandwidth drawbacks of frequency-domain convolution operations. Then, complex multiplication is performed on this pair of 8-bit multiplication expressions in the complex number calculation module, outputting the frequency-domain convolution multiplication result. This result is then accumulated to obtain the frequency-domain convolution accumulation result. Finally, by recovering the frequency-domain convolution accumulation result, the final complex output result of the frequency-domain convolution operation is obtained. This improves the operation speed of frequency-domain convolution, facilitates its deployment on hardware platforms, enhances the efficiency of digital signal processing in FPGA accelerators, accelerates product time-to-market, and reduces product design costs. Attached Figure Description

[0024] Figure 1 shows a structural diagram of a frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network proposed in an embodiment of the present invention;

[0025] Figure 2 shows a schematic diagram of the frequency domain convolution process proposed in the embodiment of the present invention;

[0026] Figure 3 shows a schematic diagram of the data packaging module proposed in this embodiment of the invention;

[0027] Figure 4 shows a structural block diagram of the data recovery module proposed in an embodiment of the present invention;

[0028] Figure 5 shows a block diagram of a frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network proposed in an embodiment of the present invention;

[0029] Figure 6 illustrates the algorithm for determining the output value of the fourth register D proposed in this embodiment of the invention. Detailed Implementation

[0030] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0031] To better illustrate this embodiment, some parts of the accompanying drawings may be omitted, enlarged, or reduced, and do not represent actual dimensions. The descriptions of directions such as "up" and "down" are not intended to limit this patent.

[0032] It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings;

[0033] The positional relationships depicted in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0034] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0035] Example 1

[0036] As shown in Figure 1, this embodiment proposes a frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network. The system includes:

[0037] The data acquisition module is used to acquire a pair of 8-bit multiplication expressions.

[0038] In the data acquisition module, the data acquisition module obtains a pair of 8-bit multiplication data from the frequency domain convolution operations of different output channels in the 8-bit frequency domain convolutional neural network. For the convenience of the explanation of this embodiment, referring to Figure 2, the frequency domain convolution of a single input channel in the 8-bit frequency domain convolutional neural network is selected, and there exists an input feature map I and a first convolution kernel F1 and a second convolution kernel F2. Let A = a r +j×a i Let W1 be the complex activation value at a certain position on I. r1 +j×w i1 W2 = w r2 +j×w i2 Let A and W1 be the complex weights on F1 and F2, respectively, located at the same position as A. Let j be the imaginary part. Then, when the input feature map I is convolved in the frequency domain with the first convolution kernel F1 and the second convolution kernel F2, there are two complex multiplications, A×W1 and A×W2, where each complex multiplication contains three real multiplications. Taking A×W1 as an example, A×W1=(a r +j×a i)×(w r1 +j×w i1 After simplification, the real part of A×W1 is a. r ×(w r1 -w i1 )+w i1 ×(a r -a i ), where the imaginary part is a. i ×(w r1 +w i1 )+w i1 ×(a r -a i Therefore, the real multiplication involved in A×W1 is: a r ×(w r1 -w i1 ), a i ×(w r1 +w i1 ), w i1 ×(a r -a i Similarly, the real number multiplication included in A×W2 is: a r ×(w r2 -w i2 ), a i ×(w r2 +w i2 ), w i2 ×(a r -a i Therefore, a r ×(w r1 -w i1 ) corresponds to a r ×(w r2 -w i2 ), a i ×(w r1 +w i1 ) corresponds to a i ×(w r2 +w i2 ), w i1 ×(a r -a i ) Corresponding to w i2 ×(a r -a i This data acquisition module obtains a r ×(w r1 -w i1 ) and a r ×(w r2 -w i2 For example, both are 8-bit multiplication expressions, meaning the data acquisition module obtains a pair of 8-bit multiplication expressions from the frequency domain convolution operation;

[0039] The data packaging module is used to package the acquired pair of 8-bit multiplication data and transmit the packaged pair of 8-bit multiplication data to the complex number calculation module.

[0040] In the data packaging module, see Figure 3, by observing a r ×(w r1 -w i1 ) and a r ×(w r2 -w i2 It can be observed that they share a common multiplier a. r Their other multipliers are (w r1 -w i1 ) and (w r2 -w i2 All of these require subtraction. Because a r ×(w r1 -w i1 ) and a r ×(w r2 -w i2 Since both are 8-bit multiplication expressions, the acquired pair of 8-bit multiplication expressions can be packaged into a 27-bit wide data and transmitted to the complex number calculation module. This allows the pair of multiplication expressions to be perfectly packaged into the complex number calculation module for operation.

[0041] The complex number calculation module is used to perform frequency domain convolution complex multiplication calculation on two 8-bit data in a packaged pair of 8-bit multiplication expressions simultaneously, and output the frequency domain convolution complex multiplication calculation result to the accumulation module;

[0042] In the complex number calculation module, the complex number calculation module includes a first register A, a second register B, a pre-adder / subtractor, a multiplier, a third register C, an adder, a fourth register D, and a fifth register E. The first register A and the second register B are in parallel. The output of the data packing module is connected to the input of the first register A and the second register B respectively. The outputs of the first register A and the second register B are connected to the input of the adder / subtractor. The outputs of the adder / subtractor and the third register C are connected to the input of the multiplier. The outputs of the multiplier and the fourth register D are connected to the input of the adder. The output of the adder is connected to the input of the fifth register E. The output of the fifth register E is connected to the input of the accumulation module. The specific calculation steps are as follows: First, the data packing module stores w... r1 and w r2 Packed into the first register A, the result after packing is {w r1 [7], w r1[7:0],10'b0,w r2 [7:0]}, it can be seen that w at this time r1 Located in the higher bit of register A, w r2 Located in the lower bit of the first register A; similarly, w i1 and w i2 Packed into the second register B, the result after packing is {w i1 [7], w i1 [7:0],10'b0,w i2 [7:0]}; Then, after the data packaging module finishes execution, it will enter the complex number calculation module, at which point the common multiplier a r It will be stored in the third register C, and the calculation of (AB) × C will begin. In fact, a will be executed simultaneously. r ×(w r1 -w i1 ) and a r ×(w r2 -w i2 In the two multiplication expressions, it is worth noting that the result of (AB)×C also needs to be added to D, the purpose of which is to eliminate the result of the multiplication of the lower-order terms. r ×(w r2 -w i2 Due to the existing bias, the complex number calculation module performs complex number multiplication on a pair of packaged 8-bit multiplication expressions at the same time, meaning it can calculate two 8-bit multiplication expressions simultaneously.

[0043] The accumulation module is used to store and accumulate the results of frequency domain convolution multiplication, and output the frequency domain convolution accumulation result;

[0044] Before the accumulation operation, first perform the two frequency domain convolution multiplication results a in the fifth register E. r ×(w r1 -w i1 ) and a r ×(w r2 -w i2 The results are separated and stored in two separate registers with a bit width higher than the fifth register E. This is to prevent the accumulation result from overflowing. Then, the a values ​​of different output channels are processed separately. r ×(w r1 -w i1 ) and a r ×(w r2 -w i2 The results are summed to obtain the frequency domain convolution summation result; similarly, the remaining multiplication expressions in A×W1 and A×W2 can be packaged, calculated and summed in the same way as before, which will not be repeated here.

[0045] The data recovery module is used to recover the frequency domain convolution summation result and obtain the final complex output result of the frequency domain convolution operation, thereby accelerating the frequency domain convolution operation of the 8-bit frequency domain convolutional neural network.

[0046] In the data recovery module, since the frequency domain convolution summation result output by the accumulation module is not the result of complex multiplication, but rather a part of the data that makes up the complex multiplication result, after the six multiplication expressions in A×W1 and A×W2 are calculated, each frequency domain convolution summation result needs to be input into the data recovery module for recovery to obtain the final complex output results A×W1 and A×W2 of the frequency domain convolution operation. This completes the acceleration of the frequency domain convolution operation of the 8-bit frequency domain convolutional neural network. See Figure 4. Taking the recovery of A×W1 as an example, only a needs to be calculated. r ×(w r1 -w i1 )+w i1 ×(a r -a i ) and a i ×(w r1 +w i1 )+w i1 ×(a r -a i By doing so, we can obtain the real and imaginary parts of A×W1.

[0047] In this embodiment, a pair of 8-bit multiplication expressions is first obtained from the frequency domain convolution operation. This pair of 8-bit multiplication expressions is then packaged into the complex number calculation module, avoiding the significant drawbacks of high memory consumption and bandwidth in the frequency domain convolution operation. Then, in the complex number calculation module, complex multiplication is performed on this pair of 8-bit multiplication expressions, outputting the frequency domain convolution multiplication result. This result is then accumulated to obtain the frequency domain convolution accumulation result. Finally, by restoring the frequency domain convolution accumulation result, the final complex output result of the frequency domain convolution operation is obtained. This improves the operation speed of frequency domain convolution, facilitates its deployment on hardware platforms, enhances the efficiency of digital signal processing in FPGA accelerators, accelerates product time-to-market, and reduces product design costs.

[0048] Example 2

[0049] Referring to Figures 1 and 5, the complex number calculation module includes a first register A, a second register B, a pre-adder / subtractor, a multiplier, a third register C, an adder, a fourth register D, and a fifth register E.

[0050] The calculation expression in the complex number calculation module is:

[0051] E = (A + / - B) × C + D

[0052] The first register A and the second register B are connected in parallel. The output of the data packing module is connected to the input of the first register A and the second register B, respectively. The outputs of the first register A and the second register B are connected to the input of the adder / subtractor. The outputs of the adder / subtractor and the third register C are connected to the input of the multiplier. The outputs of the multiplier and the fourth register D are connected to the input of the adder. The output of the adder is connected to the input of the fifth register E. The output of the fifth register E is connected to the input of the accumulation module. The 27-bit wide data output by the data packing module is input to the first register A. Registers A and B store 27-bit wide data. The input adder and subtractor performs addition and subtraction operations on the data stored in registers A and B, and outputs the 27-bit wide data addition and subtraction result. The addition and subtraction result is multiplied by the multiplier stored in register C, and the multiplication result is output. The multiplication result is added by the correction value stored in register D, and the multiplication result output by the multiplier is corrected. The frequency domain convolution multiplication result is then output to register E and stored in register E. Register E outputs the frequency domain convolution multiplication result to the accumulation module.

[0053] The first register A and the second register B are both 27 bits wide, the third register C is 18 bits wide and stores an 8-bit multiplication number, and the fourth register D and the fifth register E are both 48 bits wide.

[0054] Specifically, let's take the multiplication of a real number a in A×W1 as an example. r ×(w r1 -w i1 A real number multiplication a in A×W2 r ×(w r2 -w i2 For example, a r ×(w r1 -w i1 ) and a r ×(w r2 -w i2 Input the complex number calculation module, since the signed number w r2 and w i2 The lower bits of the data are stored in the first register A and the fourth register B, respectively. Therefore, the complex number calculation module will treat them as unsigned numbers for operation. After the complex number calculation module completes (AB)×C, the result of the multiplication of the lower bits may be incorrect. An offset value needs to be added to restore the correct result, that is, the output value of the fourth register D needs to be added. Regarding the value of the output value of the fourth register D, this embodiment will explain the algorithm described in conjunction with Figure 6. Referring to Figure 6, a diagram of the first register A, the second register B, and the third register C is given, where A... m Bm Indicates the higher-order terms, which in this example refer to w respectively. r1 w i1 A1 and B1 represent the lower-order terms, which in this example refer to w respectively. r2 w i2 When the operation between register A and register B is "+": if the least significant bits of both A and B are less than 0, then D = - (C << 9); if one least significant bit of A and B is less than 0 and the other is greater than or equal to 0, then D = - (C << 8); otherwise, D = 0. When the operation between registers A and B is "-", there are three similar cases: if the least significant bits of both A and B are less than 0, then D = 0; if one least significant bit of A and B is less than 0 and the other is greater than or equal to 0, it needs to be further divided into two cases, i.e., when A... l <0 and B l When A ≥ 0, D = -(C << 8), when A l ≥0 and B l When <0, D = C << 8; otherwise, D = 0.

[0055] In this embodiment, the data packaging module packages a pair of 8-bit multiplication expressions into a data set wider than 8 bits and transmits it to the complex number calculation module for storage and computation. The complex number calculation module processes the packaged pair of 8-bit multiplication expressions using the wide-bit registers A, B, C, D, and E respectively. First, addition and subtraction operations are performed in registers A and B. Then, complex multiplication is performed in register C. Next, the offset value of the complex multiplication result is corrected using the value retrieval algorithm of register D to ensure the accuracy of the complex multiplication result. Finally, the frequency domain convolution multiplication result is output to register E for storage. The above-mentioned series of accelerated calculations using wide-bit registers simplifies the complex calculation steps of the original complex multiplication, improves the speed of frequency domain convolution operations, accelerates product launch time, and reduces product design costs.

[0056] Example 3

[0057] In the data acquisition module, a pair of 8-bit multiplication expressions includes an 8-bit low-order term and an 8-bit high-order term. The 8-bit low-order term is located in the lower 8 bits of the first register A and the second register B, respectively. The 8-bit high-order term is located in the upper 18 to 25 bits of the first register A and the second register B, respectively. The highest bit of the first register A and the second register B is filled with the sign bit of the high-order term, and the other bits in the middle of the first register A and the second register B are filled with zeros. The multiplication result includes the low-order term multiplication result and the high-order term multiplication result. The fourth register D corrects the offset value of the low-order term multiplication result. The frequency domain convolution multiplication result includes the low-order term multiplication result and the high-order term multiplication result. The low-bit and high-bit complex multiplication results are stored in the fifth register E. The accumulation module has two parallel separation registers and an accumulation submodule. The output of the fifth register E is connected to the input of each separation register, and the output of each separation register is connected to the input of the accumulation submodule. The separation registers separate the low-bit and high-bit complex multiplication results stored in the fifth register E, and input the separated frequency domain convolution multiplication results into the accumulation submodule. The accumulation submodule performs separate accumulation operations on the low-bit and high-bit complex multiplication results respectively.

[0058] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network, characterized in that, The system includes: a data acquisition module for acquiring a pair of 8-bit multiplication expressions; a data packaging module for packaging the acquired pair of 8-bit multiplication expressions and transmitting the packaged pair of 8-bit multiplication expressions to a complex number calculation module; the data packaging module packages the pair of 8-bit multiplication expressions into a 27-bit wide data set and transmits it to the complex number calculation module for storage and calculation; the complex number calculation module performs frequency domain convolution complex multiplication calculations on the two 8-bit data sets in the packaged pair of 8-bit multiplication expressions simultaneously, and outputs the frequency domain convolution complex multiplication calculation result to an accumulation module; the complex number calculation module includes a first register A, a second register B, an adder / subtractor, a multiplier, a third register C, an adder, a fourth register D, and a fifth register E, and the calculation expression in the complex number calculation module is: The first register A and the second register B operate in parallel. The output of the data packing module is connected to the inputs of the first register A and the second register B, respectively. The outputs of the first register A and the second register B are connected to the inputs of the adder and subtractor. The outputs of the adder and subtractor and the third register C are connected to the input of the multiplier. The outputs of the multiplier and the fourth register D are connected to the input of the adder. The output of the adder is connected to the input of the fifth register E. The output of the fifth register E is connected to the input of the accumulation module. The 27-bit wide data output by the data packing module is stored in the first register A and the second register B, respectively. The 27-bit wide data stored in the first register A and the second register B is then input to the adder and subtractor for addition and subtraction operations, and the result of the addition and subtraction operation is output as a 27-bit wide data. The subtraction result and the multiplication number stored in the third register C are input into the multiplier for multiplication, and the multiplication result is output. The multiplication result and the correction value stored in the fourth register D are added by the adder to correct the multiplication result output by the multiplier. The frequency domain convolution multiplication result is then output to the fifth register E for storage. The fifth register E outputs the stored frequency domain convolution multiplication result to the accumulation module. The first register A and the second register B are both 27 bits wide, the third register C is 18 bits wide, and the fourth register D and the fifth register E are both 48 bits wide. The accumulation module is used to store and accumulate the frequency domain convolution multiplication result and output the frequency domain convolution accumulation result. The data recovery module is used to recover the frequency domain convolution accumulation result to obtain the final complex number output result of the frequency domain convolution operation.

2. The frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network according to claim 1, characterized in that, The data acquisition module obtains a pair of 8-bit multiplication data from the frequency domain convolution operations of different output channels in the 8-bit frequency domain convolutional neural network.

3. The frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network according to claim 1, characterized in that, The pair of 8-bit multiplication expressions includes an 8-bit low-order term data and an 8-bit high-order term data. The 8-bit low-order term data is located in the lower 8 bits of the first register A and the second register B, respectively. The 8-bit high-order term data is located in the upper 18 to 25 bits of the first register A and the second register B, respectively. The highest bit of the first register A and the second register B is filled with the sign bit of the high-order term, and the other bits in the middle of the first register A and the second register B are filled with zeros.

4. The frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network according to claim 3, characterized in that, The multiplication result includes the low-order term multiplication result and the high-order term multiplication result, and the fourth register D corrects the offset value of the low-order term multiplication result.

5. The frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network according to claim 4, characterized in that, The frequency domain convolution multiplication result includes a low-bit complex multiplication result and a high-bit complex multiplication result, which are stored in the fifth register E.

6. The frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network according to claim 5, characterized in that, The accumulation module has two parallel decoupling registers and an accumulation submodule. The output of the fifth register E is connected to the input of each decoupling register, and the output of each decoupling register is connected to the input of the accumulation submodule. The decoupling registers separate the low-bit complex multiplication results and the high-bit complex multiplication results stored in the fifth register E, and input the separated frequency domain convolution multiplication results to the accumulation submodule. The accumulation submodule performs separate accumulation operations on the low-bit complex multiplication results and the high-bit complex multiplication results respectively.

7. The frequency domain convolution operation acceleration system for an 8-bit frequency domain convolutional neural network according to any one of claims 1-6, characterized in that, The frequency domain convolution summation result output by the accumulation module is one of the components of the complex number output result of the frequency domain convolution operation.

Images