A fast fourier transform and neural network inference collaborative computing acceleration method

Abstract

The application belongs to the field of edge computing acceleration, and relates to a computing acceleration method for cooperation of fast Fourier transform and neural network inference, wherein, at the operator deployment level, a computing method mixed with DFT discrete Fourier transform and FFT fast Fourier transform based on butterfly computing merging and tensor mapping strategy is deployed; at the interface integration level, a bus interface of a register access path in a tensor accelerator control path is reconstructed in a light weight manner to adapt to an edge computing platform; and at the operation level, a self-defined instruction is introduced into a tensor computing unit to enable the accelerator to independently complete a whole process task of FFT signal processing and NN intelligent identification. The application expands the operation instruction set on the hardware, avoids delay overhead caused by a large amount of intermediate data returned to the processor, and thus guarantees efficient execution of the FFT and the neural network task on the unified hardware, and meets multiple requirements of strong real-time performance, high precision and low power consumption.

Description

Technical Field

[0001] This invention belongs to the field of edge computing acceleration and relates to a computational acceleration method that combines Fast Fourier Transform and neural network inference. Background Technology

[0002] With the rapid development of edge AI technology, the manufacturing industry is continuously enhancing the automation level of production equipment. In applications such as predictive maintenance, intelligent recognition, and decision optimization, the raw data collected by sensors often suffers from low quality due to unavoidable noise and random disturbances in industrial environments. Therefore, signal processing methods based on FFT are widely used for data analysis and preprocessing to efficiently filter out noise and extract key information, thereby providing higher-quality data input for subsequent intelligent reasoning and improving the adaptability and stability of algorithms in real industrial environments. Furthermore, the requirements for high real-time performance and high-efficiency computing are also core issues that urgently need to be addressed. With the dramatic increase in data volume in intelligent manufacturing, traditional computing methods and processing frameworks can no longer meet the demands for high-precision processing and real-time response under large-scale data.

[0003] Against this backdrop, artificial intelligence algorithms, represented by neural networks, are widely used in image processing, fault detection and identification, and other fields due to their powerful feature extraction and pattern recognition capabilities. However, these neural network algorithms are typically computationally intensive and require processing massive amounts of data simultaneously, leading to severe computational latency issues in industrial applications. Therefore, accelerating these computational processes, especially through hardware-software co-processing, has become one of the key technologies for improving the performance of Industry 4.0 systems. This acceleration strategy can not only significantly improve the computational speed of high-precision and complex tasks but also maintain low latency when processing large-scale data, providing strong technical support for real-time data analysis and decision-making.

[0004] However, for the two-stage computation process of "FFT signal processing + NN intelligent recognition," traditional industrial systems often use dedicated DSPs or CPUs to perform FFT calculations in the signal processing section, while the intelligent recognition section uses GPUs, FPGAs, or NPUs for model inference. This heterogeneous computing approach suffers from problems such as complex structure, large data transmission latency, and high power consumption, making it difficult to deploy in resource-constrained industrial edge devices. If a solution is to accelerate both computational tasks on the same device, the characteristics of the two types of tasks must be addressed. Mainstream FFT acceleration architectures focus on butterfly computation, while most neural network accelerators focus on convolutional tasks and lack native support for other tasks. Deploying a typical "FFT signal processing + NN intelligent recognition" pipeline still requires significant external computing resources, which severely impacts system consistency and edge-side response performance. Meanwhile, to reduce edge power consumption and model complexity, accelerator designs that have emerged in recent years are gradually evolving towards "software-hardware co-optimization": on the one hand, at the algorithm level, model pruning, structural compression, and quantization simplification are used to reduce parameter size and floating-point overhead; on the other hand, at the hardware architecture level, dedicated modules for computing tasks, low-precision computing units, and customizable programming instruction sets are provided to efficiently support the execution of lightweight algorithms. Especially in edge computing scenarios, the adoption of low-bit-width quantization models such as INT8 has become the mainstream trend for reducing power consumption and latency, but this also poses new challenges to hardware accelerators in terms of operator support, instruction and corresponding data path design, and precision control mechanisms.

[0005] In summary, given that most current industrial edge devices lack efficient hardware support for unified processing of heterogeneous computing tasks, researching a unified acceleration engine architecture capable of simultaneously supporting FFT and neural network computations has become a key breakthrough for achieving low-power, high-real-time industrial edge intelligence. How to efficiently complete tasks such as butterfly computation, matrix reconstruction, and quantization inference under resource constraints is the core problem this invention aims to solve. Summary of the Invention

[0006] To address the aforementioned technical problems in the existing technology, this invention proposes a computational acceleration method that combines Fast Fourier Transform (FFT) with neural network inference, the specific technical solution of which is as follows:

[0007] A computation acceleration method that combines Fast Fourier Transform (FFT) and Neural Network Inference is proposed. At the operator deployment level, a computation method that integrates Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT) based on butterfly computation merging and tensor mapping strategies is deployed. At the interface integration level, the bus interface of the register access path in the tensor accelerator control path is refactored in a lightweight manner to adapt it to edge computing platforms. At the computation level, custom instructions are introduced into the tensor computation unit to enable the accelerator to independently complete the entire process of FFT signal processing and NN intelligent recognition.

[0008] Furthermore, in the computational method of hybrid DFT discrete Fourier transform and FFT fast Fourier transform, by analyzing the operational rules of butterfly units in FFT and the periodic characteristics between the rotation factors, cross-level butterfly computation merging is realized, thereby mapping the multi-level computation process into a single-level tensor operation.

[0009] Furthermore, after each level of tensor operation is completed, since the position of the input data in the frequency domain changes, a data rearrangement operation is performed according to the positional order of the signal flow graph, tensor transpose is performed, and the computationally related points are gathered to complete the data alignment for the next stage.

[0010] Furthermore, the merged computational structure is reorganized into a fixed-template rotation factor matrix. Combined with the batch processing characteristics of tensor channels, multiple data blocks with the same weight in the same layer are aggregated to achieve parallel butterfly block computation of data.

[0011] Furthermore, the bus interface is reconfigured to the APB protocol.

[0012] Furthermore, the APB protocol establishes a state machine, which consists of IDLE, SETUP, and ENABLE. The IDLE, SETUP, and ENABLE functions use the PSEL and PENABLE core handshake signals for synchronization control.

[0013] Furthermore, the custom instructions are integrated into the programmable ALU module. The custom instructions include transpose instructions, multiplication instructions with rounding, and logical right shift instructions with rounding. The multiplication instructions with rounding and the logical right shift instructions with rounding are used to support low-bit-width quantization processes, and the transpose instructions are used to support internal data rearrangement.

[0014] Furthermore, the multiplication instruction with rounding and the logical right shift instruction with rounding are calculated according to the process shown in equations (1) to (4) below, where the meanings of each parameter are as follows: out orig The result is the calculation of the height and width of the intermediate layer. multiplier The multiplier used in the quantization process. rounding_half These are intermediate boundary values ​​in the rounding process. saturation_numTo be the maximum possible result of multiplication with rounding. out rmul The output result is the result of multiplication with rounding. out rshr The output result is a right shift with rounding logic. out req For the requantized output result calculated using multiplication with rounding and logical right shift, where out zp To quantize the zero point, int_min and int_max These represent the maximum and minimum values ​​of the range of quantized integers. clip The function is used for pruning, that is, to limit the previous calculation results to the range of quantized integer representation; among them, the multiplication calculation process with rounding is as shown in Equation (1), which is to round to the nearest integer after fixed-point multiplication in order to control the propagation of precision error to the previous layer; the logical right shift calculation process with rounding is as shown in Equation (2), which is used to map the multiplication result to the target bit width range by fixed-point scaling.

[0015] (1);

[0016] (2);

[0017] (3);

[0018] (4).

[0019] Furthermore, the tensor transpose is used for data rearrangement between FFT computation stages and dimensional transformations such as Reshape in neural network structures.

[0020] Furthermore, the tensor transpose instruction is equipped with two sets of register buffers of the same size during execution. One set is used for transposing and writing the current data, and the other set is used for outputting the transposed result. The input and output are executed in parallel through the alternation of reading and writing.

[0021] Beneficial effects: At the interface integration level, this invention lightweights the bus protocol interface of the register access path, making it suitable for power-sensitive and resource-limited edge computing platforms. At the operator deployment and operation level, it not only maps the typical butterfly computation structure of FFT operations by transforming the step-by-step butterfly operations into tensor computations, but also expands the operation instruction set in hardware for tensor transpose operations and the quantization process of fixed-point format data on the acceleration engine. This avoids the latency overhead caused by a large amount of intermediate data returning to the processor, thereby ensuring that FFT and neural network tasks are executed efficiently on unified hardware, meeting the multiple requirements of strong real-time performance, high precision, and low power consumption. Attached Figure Description

[0022] Figure 1 This is an overall design block diagram of an accelerator for a computation acceleration method that combines Fast Fourier Transform and neural network inference according to an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of merging multi-level butterfly calculations into a rotation factor matrix according to an embodiment of the present invention;

[0024] Figure 3 This is a detailed flowchart of the radix 8-512 point hybrid DFT + FFT calculation according to an embodiment of the present invention;

[0025] Figure 4 This is a basic state diagram of the APB bus protocol implementation in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of the ALU module's extended custom instruction set according to an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0028] like Figure 1 As shown in this embodiment, a computational acceleration method combining Fast Fourier Transform (FFT) and Neural Network (NN) inference is proposed. Specifically, at the operator deployment level, a hybrid DFT (Discrete Fourier Transform) + FFT (Fast Fourier Transform) computation method based on butterfly computation merging and tensor mapping strategies is proposed to adapt to the hardware characteristics of the accelerator and improve the utilization of computing resources. At the interface integration level, the bus interface of the register access path in the traditional tensor accelerator control path is lightweightly reconstructed to achieve low-power, low-latency register access, making it suitable for power-sensitive, resource-limited edge computing platforms. At the computation level, a series of custom instructions are introduced into the computation unit, including multiplication instructions with rounding and logical right shift instructions to support low-bit-width quantization processes, and transpose instructions to support internal data rearrangement, enabling the accelerator to independently complete "FFT signal processing + NN (Neural Network)". The "intelligent recognition" full-process task solves the problem that some parts of heterogeneous computing tasks must fall back to CPU execution, making it easier to realize the pipeline of the whole process and improve execution efficiency.

[0029] Specifically, in the design of hybrid DFT + FFT computation methods, traditional FFT computation algorithms are based on hierarchical butterfly operations. Each level of operation requires independent complex multiplication and addition as well as data rearrangement, resulting in a strong dependence on controller scheduling for the entire computation process and making it difficult to directly map onto the matrix multiplication and addition path of the tensor structure. Therefore, this invention restructures the algorithm by analyzing the operational rules of butterfly units in FFT and the periodic characteristics between the rotation factor, achieving cross-level butterfly merging. This maps the multi-level computation process into a single-level tensor operation, such as... Figure 2 As shown. The merged computational structure is reorganized into a fixed-template rotation factor matrix. Combined with the batch processing characteristics of tensor channels, multiple data blocks with the same weight in the same layer can be aggregated, enabling parallel butterfly block computation of data and improving the utilization rate of the tensor processing path. After each level of tensor computation is completed, because the position of the input data in the frequency domain changes, a data rearrangement operation needs to be performed according to the positional order of the signal flow graph. This involves tensor transposition, gathering computationally relevant points to complete the data alignment for the next stage. The specific process is as follows: Figure 3 As shown. This phased and progressively deepening data processing strategy provides the necessary frequency domain information for the final FFT calculation result. While maintaining the same accuracy as the traditional FFT, it optimizes the parallelism of the calculation process, makes full use of the existing pipeline and computation control execution model in the tensor structure, is more compatible with hardware units, and improves module utilization and computational throughput.

[0030] The lightweight design of the bus interface is another key aspect of this invention at the system adaptation level. Traditional tensor accelerators mostly use high-performance, high-bandwidth bus protocols to access internal register configuration areas. Although such protocols have good timing control capabilities, their handshake process is complex and state transition paths are long, making them overly redundant for register accesses that only require simple read / write operations. This increases control logic resource overhead and introduces unnecessary latency and power consumption burdens. Therefore, this invention refactors its control bus design to the APB (Advanced Peripheral Bus) protocol to achieve a more streamlined and energy-efficient control interface structure. Specifically, based on the protocol specifications, the following is established: Figure 4 The state machine shown consists of IDLE, SETUP, and ENABLE. It uses core handshake signals such as PSEL and PENABLE for synchronous control. The control path of this APB bus is completely decoupled from the main data path. It will not block the main computing task when configuring the register, thus enhancing the practicality of deploying the accelerator in low-power edge devices.

[0031] Meanwhile, this invention introduces three custom instructions into the computing unit and integrates them into a programmable tensor ALU module. The entire instruction extension process includes reserving dedicated opcode space in the instruction configuration register, expanding support paths through the instruction decoder, and realizing the decoding and execution of the new instructions. In terms of hardware implementation, after receiving a valid extended instruction, the ALU's internal decoding module dispatches it to the newly added combinational logic computing unit according to the predefined functional module mapping logic, such as... Figure 5 As shown, this ensures that the instruction execution flow remains structurally consistent with the original basic arithmetic instruction set. This mechanism allows new functions to be naturally integrated into the main execution path without altering the original tensor pipeline structure.

[0032] For low-bit-width quantization models commonly found in industrial edge neural networks, to reduce accuracy loss, it is often necessary to calculate high-bit-width results in intermediate layers and then map the results back to low-bit-width fixed-point numbers. To directly implement intermediate quantization operations through hardware computation, this invention introduces relevant custom instructions, which perform calculations according to the processes shown in equations (1) to (4). The meanings of the parameters in the equations are as follows: out orig The result is the calculation of the height and width of the intermediate layer. multiplier The multiplier used in the quantization process. rounding_half These are intermediate boundary values ​​in the rounding process. saturation_num To be the maximum possible result of multiplication with rounding. out rmul The output result is the result of multiplication with rounding. out rshr The output result is a right shift with rounding logic. out req Requantization for calculations using multiplication with rounding and logical right shift ( requantization Output results, where out zp To quantize the zero point, int_min and int_max These represent the maximum and minimum values ​​of the range of quantized integers. clip The function is used for pruning, that is, to restrict the previous calculation results to the range of quantized integer representation. The multiplication calculation process with rounding is as shown in Equation (1), which is essentially rounding to the nearest integer after fixed-point multiplication to control the propagation of precision error to the previous layer. The logical right shift calculation process with rounding is as shown in Equation (2), which is used to map the multiplication result to the target bit width range by fixed-point scaling, and is a commonly used scaling method in quantized inference.

[0033] (1);

[0034] (2);

[0035] (3);

[0036] (4);

[0037] Another new custom instruction is tensor transpose, which is used for data rearrangement between FFT calculation stages and dimensional transformations such as Reshape in neural network structures.

[0038] To achieve efficient execution of transpose instructions, this invention introduces a double-buffered structure, which is equipped with two sets of register buffers of the same size during execution. One set is used for transposing and writing the current data, and the other set is used for outputting the transposed result. By alternating between reading and writing, the input and output are executed in parallel without interrupting the data path.

[0039] The three custom instructions described above connect the entire hardware path of the tensor structure data flow, unifying all operations of the "FFT signal processing + NN intelligent recognition" task within the computing acceleration engine, avoiding interruptions to the pipeline or rollback to CPU processing, and fundamentally improving overall execution efficiency and architectural consistency.

[0040] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for computing acceleration of fast Fourier transform in coordination with neural network inference, characterized in that, At the operator deployment level, a calculation method of mixed DFT discrete Fourier transform and FFT fast Fourier transform based on butterfly calculation merging and tensor mapping strategy is deployed; at the interface integration level, the bus interface of the register access path in the tensor accelerator control path is reconstructed in a lightweight manner to adapt to the edge computing platform; and at the operation level, a custom instruction is introduced in the tensor calculation unit to enable the accelerator to independently complete the whole process task of FFT signal processing and NN intelligent recognition. In the calculation method of mixed DFT discrete Fourier transform and FFT fast Fourier transform, the periodic characteristics between the operation law of the butterfly unit in the FFT and the rotation factor are analyzed, cross-level butterfly calculation merging is realized, and the multi-level calculation process is mapped to a single-level tensor operation. The merged operation structure is reorganized into a fixed template rotation factor matrix, and the same group of data blocks with the same weight in the same layer are gathered to realize parallel butterfly block operation of data, in cooperation with the batch processing characteristics of the tensor channel. After each level of tensor operation, the position of the input data in the frequency domain changes, and then the data rearrangement operation is performed according to the bit sequence law of the signal flow diagram to perform tensor transposition and collect the calculation-related points to complete the data alignment of the next stage.

2. The method of claim 1, wherein, The bus interface is reconstructed as an APB protocol.

3. The method of claim 2, wherein, The APB protocol establishes a state machine composed of IDLE, SETUP and ENABLE, and the IDLE, SETUP and ENABLE are synchronously controlled by using PSEL and PENABLE core handshake signals.

4. The method of claim 1, wherein, The custom instruction is integrated into a programmable ALU module, and the custom instruction includes a transpose instruction, a multiplication instruction with rounding and a logical right shift instruction with rounding, the multiplication instruction with rounding and the logical right shift instruction with rounding are used to support the low-bit-width quantization process, and the transpose instruction is used to support internal data rearrangement.

5. The method of claim 4, wherein, The multiplication instruction with rounding is a nearest integer rounding operation after fixed-point multiplication to control the precision error transmission to the previous layer. The logical right shift instruction with rounding is used to map the multiplication result of the multiplication instruction with rounding to the target bit width interval by fixed-point scaling.

6. The method of claim 4, wherein, The transpose instruction is used for data rearrangement between FFT calculation stages and dimension transformation using Reshape in the neural network structure.

7. The method of claim 6, wherein, The transpose instruction is equipped with two groups of register buffers with the same size during execution, one group is used for transpose writing of current data, and the other group is used for output of transposed results, and the input and output are executed in parallel through the read-write alternation mode.

Images