Calculation acceleration method based on cooperation of fast Fourier transform and neural network reasoning

Abstract

The invention belongs to the field of edge computing acceleration, and relates to a fast Fourier transform and neural network reasoning collaborative computing acceleration method, which comprises the following steps of: deploying a computing method which is based on butterfly computing merging and a tensor mapping strategy and is mixed with DFT (Discrete Fourier Transform) and FFT (Fast Fourier Transform) in an operator deployment level; on the interface integration level, a bus interface of a register access path in a tensor accelerator control path is subjected to lightweight reconstruction, so that the bus interface is adaptive to an edge computing platform; in an operation level, a user-defined instruction is introduced into a tensor calculation unit, and an accelerator is enabled to independently complete a whole-process task of FFT signal processing and NN intelligent identification. According to the method, an operation instruction set is expanded on hardware, and delay overhead caused by returning a large amount of intermediate data to a processor is avoided, so that efficient execution of FFT and neural network tasks on unified hardware is guaranteed, and multiple requirements of high real-time performance, high precision and low power consumption are met.

Description

Technical Field

[0001] This invention belongs to the field of edge computing acceleration and relates to a computational acceleration method that combines Fast Fourier Transform and neural network inference. Background Technology

[0002] With the rapid development of edge AI technology, the manufacturing industry is continuously enhancing the automation level of production equipment. In applications such as predictive maintenance, intelligent recognition, and decision optimization, the raw data collected by sensors often suffers from low quality due to unavoidable noise and random disturbances in industrial environments. Therefore, signal processing methods based on FFT are widely used for data analysis and preprocessing to efficiently filter out noise and extract key information, thereby providing higher-quality data input for subsequent intelligent reasoning and improving the adaptability and stability of algorithms in real industrial environments. Furthermore, the requirements for high real-time performance and high-efficiency computing are also core issues that urgently need to be addressed. With the dramatic increase in data volume in intelligent manufacturing, traditional computing methods and processing frameworks can no longer meet the demands for high-precision processing and real-time response under large-scale data.

[0003] Against this backdrop, artificial intelligence algorithms, represented by neural networks, are widely used in image processing, fault detection and identification, and other fields due to their powerful feature extraction and pattern recognition capabilities. However, these neural network algorithms are typically computationally intensive and require processing massive amounts of data simultaneously, leading to severe computational latency issues in industrial applications. Therefore, accelerating these computational processes, especially through hardware-software co-processing, has become one of the key technologies for improving the performance of Industry 4.0 systems. This acceleration strategy can not only significantly improve the computational speed of high-precision and complex tasks but also maintain low latency when processing large-scale data, providing strong technical support for real-time data analysis and decision-making.

[0004] However, for the two-stage computation process of "FFT signal processing + NN intelligent recognition," traditional industrial systems often use dedicated DSPs or CPUs to perform FFT calculations in the signal processing section, while the intelligent recognition section uses GPUs, FPGAs, or NPUs for model inference. This heterogeneous computing approach suffers from problems such as complex structure, large data transmission latency, and high power consumption, making it difficult to deploy in resource-constrained industrial edge devices. If a solution is to accelerate both computational tasks on the same device, the characteristics of the two types of tasks must be addressed. Mainstream FFT acceleration architectures focus on butterfly computation, while most neural network accelerators focus on convolutional tasks and lack native support for other tasks. Deploying a typical "FFT signal processing + NN intelligent recognition" pipeline still requires significant external computing resources, which severely impacts system consistency and edge-side response performance. Meanwhile, to reduce edge power consumption and model complexity, accelerator designs that have emerged in recent years are gradually evolving towards "software-hardware co-optimization": on the one hand, at the algorithm level, model pruning, structural compression, and quantization simplification are used to reduce parameter size and floating-point overhead; on the other hand, at the hardware architecture level, dedicated modules for computing tasks, low-precision computing units, and customizable programming instruction sets are provided to efficiently support the execution of lightweight algorithms. Especially in edge computing scenarios, the adoption of low-bit-width quantization models such as INT8 has become the mainstream trend for reducing power consumption and latency, but this also poses new challenges to hardware accelerators in terms of operator support, instruction and corresponding data path design, and precision control mechanisms.

[0005] In summary, given that most current industrial edge devices lack efficient hardware support for unified processing of heterogeneous computing tasks, researching a unified acceleration engine architecture capable of simultaneously supporting FFT and neural network computations has become a key breakthrough for achieving low-power, high-real-time industrial edge intelligence. How to efficiently complete tasks such as butterfly computation, matrix reconstruction, and quantization inference under resource constraints is the core problem this invention aims to solve. Summary of the Invention

[0006] To address the aforementioned technical problems in the existing technology, this invention proposes a computational acceleration method that combines Fast Fourier Transform (FFT) with neural network inference, the specific technical solution of which is as follows: A computation acceleration method that combines Fast Fourier Transform (FFT) and Neural Network Inference is proposed. At the operator deployment level, a computation method that integrates Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT) based on butterfly computation merging and tensor mapping strategies is deployed. At the interface integration level, the bus interface of the register access path in the tensor accelerator control path is refactored in a lightweight manner to adapt it to edge computing platforms. At the computation level, custom instructions are introduced into the tensor computation unit to enable the accelerator to independently complete the entire process of FFT signal processing and NN intelligent recognition.

[0007] Furthermore, in the computational method of hybrid DFT discrete Fourier transform and FFT fast Fourier transform, by analyzing the operational rules of butterfly units in FFT and the periodic characteristics between the rotation factors, cross-level butterfly computation merging is realized, thereby mapping the multi-level computation process into a single-level tensor operation.

[0008] Furthermore, after each level of tensor operation is completed, since the position of the input data in the frequency domain changes, a data rearrangement operation is performed according to the positional order of the signal flow graph, tensor transpose is performed, and the computationally related points are gathered to complete the data alignment for the next stage.

[0009] Furthermore, the merged computational structure is reorganized into a fixed-template rotation factor matrix. Combined with the batch processing characteristics of tensor channels, multiple data blocks with the same weight in the same layer are aggregated to achieve parallel butterfly block computation of data.

[0010] Furthermore, the bus interface is reconfigured to the APB protocol.

[0011] Furthermore, the APB protocol establishes a state machine, which consists of IDLE, SETUP, and ENABLE. The IDLE, SETUP, and ENABLE functions use the PSEL and PENABLE core handshake signals for synchronization control.

[0012] Furthermore, the custom instructions are integrated into the programmable ALU module. The custom instructions include transpose instructions, multiplication instructions with rounding, and logical right shift instructions with rounding. The multiplication instructions with rounding and the logical right shift instructions with rounding are used to support low-bit-width quantization processes, and the transpose instructions are used to support internal data rearrangement.

[0013] Furthermore, the multiplication instruction with rounding and the logical right shift instruction with rounding are calculated according to the process shown in equations (1) to (4) below, where the meanings of each parameter are as follows: out orig The result is the calculation of the height and width of the intermediate layer. multiplier The multiplier used in the quantization process. rounding_half These are intermediate boundary values ​​in the rounding process. saturation_num To be the maximum possible result of multiplication with rounding. out rmul The output result is the result of multiplication with rounding. out rshr The output result is a right shift with rounding logic. out req For the requantized output result calculated using multiplication with rounding and logical right shift, where out zp To quantize the zero point, int_minand int_max These represent the maximum and minimum values ​​of the range of quantized integers. clip The function is used for pruning, that is, to limit the previous calculation results to the range of quantized integer representation; among them, the multiplication calculation process with rounding is as shown in Equation (1), which is to round to the nearest integer after fixed-point multiplication in order to control the propagation of precision error to the previous layer; the logical right shift calculation process with rounding is as shown in Equation (2), which is used to map the multiplication result to the target bit width range by fixed-point scaling. (1); (2); (3); (4).

[0014] Furthermore, the tensor transpose is used for data rearrangement between FFT computation stages and dimensional transformations such as Reshape in neural network structures.

[0015] Furthermore, the tensor transpose instruction is equipped with two sets of register buffers of the same size during execution. One set is used for transposing and writing the current data, and the other set is used for outputting the transposed result. The input and output are executed in parallel through the alternation of reading and writing.

[0016] Beneficial effects: At the interface integration level, this invention lightweights the bus protocol interface of the register access path, making it suitable for power-sensitive and resource-limited edge computing platforms. At the operator deployment and operation level, it not only maps the typical butterfly computation structure of FFT operations by transforming the step-by-step butterfly operations into tensor computations, but also expands the operation instruction set in hardware for tensor transpose operations and the quantization process of fixed-point format data on the acceleration engine. This avoids the latency overhead caused by a large amount of intermediate data returning to the processor, thereby ensuring that FFT and neural network tasks are executed efficiently on unified hardware, meeting the multiple requirements of strong real-time performance, high precision, and low power consumption. Attached Figure Description

[0017] Figure 1 This is an overall design block diagram of an accelerator for a computation acceleration method that combines Fast Fourier Transform and neural network inference according to an embodiment of the present invention. Figure 2 This is a schematic diagram of merging multi-level butterfly calculations into a rotation factor matrix according to an embodiment of the present invention; Figure 3 This is a detailed flowchart of the radix 8-512 point hybrid DFT + FFT calculation according to an embodiment of the present invention; Figure 4 This is a basic state diagram of the APB bus protocol implementation in an embodiment of the present invention; Figure 5 This is a schematic diagram of the ALU module's extended custom instruction set according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0019] like Figure 1 As shown in this embodiment, a computational acceleration method combining Fast Fourier Transform (FFT) and Neural Network (NN) inference is proposed. Specifically, at the operator deployment level, a hybrid DFT (Discrete Fourier Transform) + FFT (Fast Fourier Transform) computation method based on butterfly computation merging and tensor mapping strategies is proposed to adapt to the hardware characteristics of the accelerator and improve the utilization of computing resources. At the interface integration level, the bus interface of the register access path in the traditional tensor accelerator control path is lightweightly reconstructed to achieve low-power, low-latency register access, making it suitable for power-sensitive, resource-limited edge computing platforms. At the computation level, a series of custom instructions are introduced into the computation unit, including multiplication instructions with rounding and logical right shift instructions to support low-bit-width quantization processes, and transpose instructions to support internal data rearrangement, enabling the accelerator to independently complete "FFT signal processing + NN (Neural Network)". The "intelligent recognition" full-process task solves the problem that some parts of heterogeneous computing tasks must fall back to CPU execution, making it easier to realize the pipeline of the whole process and improve execution efficiency.

[0020] Specifically, in the design of hybrid DFT + FFT computation methods, traditional FFT computation algorithms are based on hierarchical butterfly operations. Each level of operation requires independent complex multiplication and addition as well as data rearrangement, resulting in a strong dependence on controller scheduling for the entire computation process and making it difficult to directly map onto the matrix multiplication and addition path of the tensor structure. Therefore, this invention restructures the algorithm by analyzing the operational rules of butterfly units in FFT and the periodic characteristics between the rotation factor, achieving cross-level butterfly merging. This maps the multi-level computation process into a single-level tensor operation, such as... Figure 2 As shown. The merged computational structure is reorganized into a fixed-template rotation factor matrix. Combined with the batch processing characteristics of tensor channels, multiple data blocks with the same weight in the same layer can be aggregated, enabling parallel butterfly block computation of data and improving the utilization rate of the tensor processing path. After each level of tensor computation is completed, because the position of the input data in the frequency domain changes, a data rearrangement operation needs to be performed according to the positional order of the signal flow graph. This involves tensor transposition, gathering computationally relevant points to complete the data alignment for the next stage. The specific process is as follows: Figure 3 As shown. This phased and progressively deepening data processing strategy provides the necessary frequency domain information for the final FFT calculation result. While maintaining the same accuracy as the traditional FFT, it optimizes the parallelism of the calculation process, makes full use of the existing pipeline and computation control execution model in the tensor structure, is more compatible with hardware units, and improves module utilization and computational throughput.

[0021] The lightweight design of the bus interface is another key aspect of this invention at the system adaptation level. Traditional tensor accelerators mostly use high-performance, high-bandwidth bus protocols to access internal register configuration areas. Although such protocols have good timing control capabilities, their handshake process is complex and state transition paths are long, making them overly redundant for register accesses that only require simple read / write operations. This increases control logic resource overhead and introduces unnecessary latency and power consumption burdens. Therefore, this invention refactors its control bus design to the APB (Advanced Peripheral Bus) protocol to achieve a more streamlined and energy-efficient control interface structure. Specifically, based on the protocol specifications, the following is established: Figure 4 The state machine shown consists of IDLE, SETUP, and ENABLE. It uses core handshake signals such as PSEL and PENABLE for synchronous control. The control path of this APB bus is completely decoupled from the main data path. It will not block the main computing task when configuring the register, thus enhancing the practicality of deploying the accelerator in low-power edge devices.

[0022] Meanwhile, this invention introduces three custom instructions into the computing unit and integrates them into a programmable tensor ALU module. The entire instruction extension process includes reserving dedicated opcode space in the instruction configuration register, expanding support paths through the instruction decoder, and realizing the decoding and execution of the new instructions. In terms of hardware implementation, after receiving a valid extended instruction, the ALU's internal decoding module dispatches it to the newly added combinational logic computing unit according to the predefined functional module mapping logic, such as... Figure 5 As shown, this ensures that the instruction execution flow remains structurally consistent with the original basic arithmetic instruction set. This mechanism allows new functions to be naturally integrated into the main execution path without altering the original tensor pipeline structure.

[0023] For low-bit-width quantization models commonly found in industrial edge neural networks, to reduce accuracy loss, it is often necessary to calculate high-bit-width results in intermediate layers and then map the results back to low-bit-width fixed-point numbers. To directly implement intermediate quantization operations through hardware computation, this invention introduces relevant custom instructions, which perform calculations according to the processes shown in equations (1) to (4). The meanings of the parameters in the equations are as follows: out orig The result is the calculation of the height and width of the intermediate layer. multiplier The multiplier used in the quantization process. rounding_half These are intermediate boundary values ​​in the rounding process. saturation_num To be the maximum possible result of multiplication with rounding. out rmul The output result is the result of multiplication with rounding. out rshr The output result is a right shift with rounding logic. out req Requantization for calculations using multiplication with rounding and logical right shift ( requantization Output results, where out zp To quantize the zero point, int_min and int_max These represent the maximum and minimum values ​​of the range of quantized integers. clip The function is used for pruning, that is, to restrict the previous calculation results to the range of quantized integer representation. The multiplication calculation process with rounding is as shown in Equation (1), which is essentially rounding to the nearest integer after fixed-point multiplication to control the propagation of precision error to the previous layer. The logical right shift calculation process with rounding is as shown in Equation (2), which is used to map the multiplication result to the target bit width range by fixed-point scaling, and is a commonly used scaling method in quantized inference.

[0024] (1); (2); (3); (4); Another new custom instruction is tensor transpose, which is used for data rearrangement between FFT calculation stages and dimensional transformations such as Reshape in neural network structures.

[0025] To achieve efficient execution of transpose instructions, this invention introduces a double-buffered structure, which is equipped with two sets of register buffers of the same size during execution. One set is used for transposing and writing the current data, and the other set is used for outputting the transposed result. By alternating between reading and writing, the input and output are executed in parallel without interrupting the data path.

[0026] The three custom instructions described above connect the entire hardware path of the tensor structure data flow, unifying all operations of the "FFT signal processing + NN intelligent recognition" task within the computing acceleration engine, avoiding interruptions to the pipeline or rollback to CPU processing, and fundamentally improving overall execution efficiency and architectural consistency.

[0027] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A computational acceleration method combining Fast Fourier Transform and neural network inference, characterized in that, At the operator deployment level, a computational method combining Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT) based on butterfly computation merging and tensor mapping strategies is deployed. At the interface integration level, the bus interface of the register access path in the tensor accelerator control path is refactored in a lightweight manner to adapt it to the edge computing platform. At the computation level, custom instructions are introduced into the tensor computation unit to enable the accelerator to independently complete the entire process of FFT signal processing and NN intelligent recognition.

2. The computational acceleration method as described in claim 1, characterized in that, In the computational method of hybrid DFT (Discrete Fourier Transform) and FFT (Fast Fourier Transform), by analyzing the operational rules of butterfly units in FFT and the periodic characteristics between the rotation factors, cross-level butterfly computations are merged, thereby mapping the multi-level computation process into a single-level tensor operation.

3. The computational acceleration method as described in claim 2, characterized in that, After each level of tensor operation is completed, since the position of the input data in the frequency domain changes, a data rearrangement operation is performed according to the positional order of the signal flow graph. The tensor is transposed, and the computationally relevant points are gathered to complete the data alignment for the next stage.

4. The computational acceleration method as described in claim 3, characterized in that, The merged computational structure is reorganized into a fixed-template rotation factor matrix. Combined with the batch processing characteristics of tensor channels, multiple data blocks with the same weight in the same layer are aggregated to achieve parallel butterfly block computation of data.

5. The computational acceleration method as described in claim 1, characterized in that, The bus interface has been reconfigured to use the APB protocol.

6. The computational acceleration method as described in claim 5, characterized in that, The APB protocol establishes a state machine, which consists of IDLE, SETUP, and ENABLE. The IDLE, SETUP, and ENABLE functions use the PSEL and PENABLE core handshake signals for synchronization control.

7. The computational acceleration method as described in claim 1, characterized in that, The custom instructions are integrated into the programmable ALU module. The custom instructions include transpose instructions, multiplication instructions with rounding, and logical right shift instructions with rounding. The multiplication instructions with rounding and the logical right shift instructions with rounding are used to support low-bit-width quantization processes, and the transpose instructions are used to support internal data rearrangement.

8. The computational acceleration method as described in claim 7, characterized in that, The multiplication instruction with rounding is performed after fixed-point multiplication, rounding to the nearest integer to control the propagation of precision error to the previous layer; The logical right shift instruction with rounding is used to perform fixed-point scaling mapping of the multiplication result of the multiplication instruction with rounding to the target bit width range.

9. The computational acceleration method as described in claim 7, characterized in that, The tensor transpose is used for data rearrangement between FFT computation stages and for dimensional transformations such as Reshape in neural network structures.

10. The computational acceleration method as described in claim 9, characterized in that, The tensor transpose instruction is equipped with two sets of register buffers of the same size during execution. One set is used for transposing and writing the current data, and the other set is used for outputting the transposed result. The input and output are executed in parallel by alternating between reading and writing.

Images