Mixing precision tensor calculation unit and method for edge calculation

Abstract

The invention relates to the technical field of artificial intelligence hardware acceleration, in particular to a mixed precision tensor calculation unit and method for edge calculation. The reconfigurable computing architecture based on the systolic array is designed for the contradiction between edge side hardware resource limitation and model reasoning high-performance requirements. The system comprises an AXI bus interface, a global control module, a data formatting module and a multi-Bank parallel cache structure. Mixed precision calculation is supported, and when core matrix multiplication and addition operation is processed, a half-precision floating-point number is adopted as an input operand, and a single-precision floating-point number is adopted as an accumulation and output operand, so that the data throughput is maximized on the premise of ensuring the calculation precision. Through the optimized two-dimensional mesh topology systolic array and data flow control strategy, on-chip storage access conflicts can be effectively reduced, transmission delay is reduced, the balance of high computing power, high energy efficiency and low resource occupation is achieved, and the method is suitable for complex load prediction model reasoning in the fields of intelligent power grids and the like.

Description

Technical Field

[0001] This invention relates to the fields of integrated circuit design and artificial intelligence hardware acceleration technology, and in particular to an FPGA mixed-precision tensor computation unit for edge computing load prediction. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning models are increasingly being used in edge computing scenarios. Particularly in the field of smart grids, load forecasting models are crucial for the stable operation of the power grid. However, edge devices typically face strict power consumption and hardware resource constraints. Existing general-purpose processors are inefficient at handling large-scale matrix operations and cannot meet real-time requirements; while graphics processing units (GPUs), although powerful, consume too much power and are unsuitable for deployment on resource-constrained edge nodes.

[0003] With the widespread application of edge computing in areas such as smart grid load forecasting, the performance requirements for hardware accelerators are increasing. Traditional matrix computing solutions are often based on fixed-precision architectures. For example, general-purpose processors that only support FP32, while offering high precision, have low energy efficiency and cannot meet the power consumption constraints of edge devices. While fixed-point accelerators that only support INT8 are fast, they are prone to accuracy degradation when processing prediction models sensitive to numerical ranges due to precision loss. Furthermore, existing FPGA accelerator designs often face the memory wall bottleneck when handling large-scale matrix operations, meaning the access bandwidth of on-chip caches cannot match the throughput demands of the computing array, leading to idle computing units. While single low-precision computation is fast and consumes few resources, it can significantly reduce accuracy when processing precision-sensitive load forecasting models. Conversely, while single high-precision computation ensures accuracy, it consumes excessive DSP and logic resources, and the computational throughput cannot meet high-performance requirements. Moreover, existing technologies struggle to simultaneously achieve dynamic switching between multiple data precisions and maintain high computing power within limited FPGA resources.

[0004] For example, Chinese patent application CN202510799685.9 discloses a mixed-precision matrix multiplication and addition unit and calculation method. This method uses a format partitioning module to uniformly convert input data of different precisions (such as FP32, FP16, INT8, etc.) into an intermediate floating-point format (IFFs0), and utilizes an adder tree module and an accumulator module to complete the calculation, thus achieving universal compatibility with multiple data precisions. However, this scheme still suffers from complex hardware logic. It relies on general intermediate format conversion and mantissa segmentation, which introduces additional format conversion overhead and resource consumption when targeting specific edge computing scenarios (such as those requiring only FP16 / FP32 mixed precision). Furthermore, the "adder tree" structure has low pipeline efficiency and area efficiency when processing large-scale dense matrices.

[0005] For example, Chinese patent application CN202310738051.3 discloses a pulsating array architecture for accelerating sparse matrix computation with hybrid precision. This scheme uses a dynamic data matching and driving module (DMD) to index and drive the compressed sparse data stream, and uses a data priority transmission rule to reduce the latency of sparse matrix computation. However, it suffers from redundant control logic. This architecture is highly dependent on the index matching mechanism of sparse data, and the processing unit (PE) integrates complex matching and handshaking logic. When applied to scenarios such as smart grid load forecasting, which are dominated by dense matrix operations, these control logics designed for sparsity will waste hardware resources and make it difficult to fully utilize on-chip storage bandwidth to achieve high-density parallel computing.

[0006] In summary, existing technical solutions still suffer from insufficient architectural adaptability when dealing with specific scenarios such as smart grid load forecasting in edge computing environments. On the one hand, general-purpose mixed-precision computing solutions sacrifice hardware efficiency in pursuit of excessive compatibility, resulting in excessive resource consumption when handling intensive computations of specific precisions (such as FP16 / FP32). On the other hand, architectures designed for sparse matrices cannot efficiently serve load forecasting models that are mainly based on dense matrices, and fail to effectively solve on-chip memory access conflicts and bandwidth bottlenecks under high-concurrency computing, resulting in the memory wall problem.

[0007] Therefore, it is urgent to design a computing architecture that is suitable for the dense matrix operation characteristics of the edge side, so as to achieve a balance between high computing power density and high energy efficiency while reducing the overhead of format conversion. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing hardware architectures in balancing high precision and high performance in edge computing scenarios. It designs a reconfigurable computing architecture based on a pulsating array and provides a mixed-precision tensor computing unit and method for edge computing.

[0009] The objective of this invention can be achieved through the following technical solutions: As a first aspect of the present invention, a mixed-precision tensor computation unit for edge computing is provided, comprising: The AXI bus interface is used to communicate with the external main processor and memory, and to receive computation instructions and tensor data. The global control unit receives and decodes the computing instructions issued by the main processor and generates global control signals; The on-chip cache system includes an input cache, a weight cache, and an output cache, which are used to store the data of the input matrix A, the weight matrix B, and the calculated result matrix D, respectively. The input cache and the weight cache adopt a multi-bank interleaved storage structure, with each bank corresponding to a row or column of the systolic array. A pulsating array is composed of multiple processing units connected in a two-dimensional mesh topology with defined rows and columns; The processing unit in the first column on the left side of the pulsating array receives the input matrix A data corresponding to the Bank in the input buffer along the row direction and passes it to the right according to the clock beat; the processing unit in the first row above the pulsating array receives the weight matrix B data corresponding to the Bank in the weight buffer along the column direction and passes it down according to the clock beat; the processing unit performs a matrix multiplication and addition operation by multiplying the input matrix A and the weight matrix B and then adding it to the bias / accumulation matrix C to obtain the result matrix D, wherein the data format of the input matrix A and the weight matrix B is half-precision floating-point number, and the data format of the bias / accumulation matrix C and the result matrix D is single-precision floating-point number.

[0010] As a preferred technical solution, the computing unit further includes a data formatting unit connected between the AXI bus interface and the on-chip cache; The data formatting unit converts the one-dimensional data stream input from the external memory into a two-dimensional block data input, and converts the input data and weight data in single-precision floating-point format into half-precision floating-point format; When the dimensions of the matrix to be calculated do not meet the requirements of the rows and columns corresponding to the pulsating array, the data formatting unit pads the matrix to be calculated with zeros. When the dimension of the matrix to be calculated exceeds the number of rows and columns corresponding to the systolic array, the data formatting unit adopts a block strategy to cut the matrix to be calculated into multiple sub-blocks corresponding to the dimensions of the systolic array, and schedules each sub-block to enter the array for iterative calculation in turn. During the data output phase, the data formatting unit receives single-precision floating-point data from the output buffer, performs address alignment and bit width adaptation, and then writes it back to external memory via the AXI bus interface.

[0011] As a preferred technical solution, an input cache and weight cache with a multi-bank interleaved storage structure are adopted, as follows: The number of banks in the input buffer corresponds to the column dimension of the systolic array. The input matrix A to be calculated is stored sequentially into the corresponding physical bank address segment in the input buffer according to the column dimension. The number of banks in the weight cache corresponds to the row dimension of the systolic array. The weight matrix B to be calculated is stored sequentially into the corresponding physical bank address segment in the input cache according to the row dimension. When the systolic array starts computing, the global control unit simultaneously reads the data of the corresponding row / column from each Bank and distributes it to the systolic array.

[0012] As a preferred technical solution, the processing unit includes two data input channels, a half-precision floating-point multiplier, a single-precision floating-point adder, and a single-precision floating-point accumulator register. The data input channels are used to receive input matrix A data from the adjacent processing unit on the left and weight matrix B data from the adjacent processing unit above; the processing unit is also equipped with a pipeline register, which, while sending the input data into the multiplier, also passes it to the next processing unit on the right and below respectively. The multiplier multiplies the input matrix A (input data format of half-precision floating-point numbers) with the weight matrix B, and then converts the multiplication result into single-precision floating-point number format without loss before sending it to the adder. The accumulator register stores a bias / accumulator matrix C in single-precision floating-point format. The adder adds the multiplication result to the current value of the bias / accumulator matrix C stored in the accumulator register and writes the updated sum back to the accumulator register. After the systolic array completes a full matrix multiplication calculation, the accumulated result in the processing unit is passed down along the column direction through the output channel, and the result matrix D output from the bottom of the systolic array is sent to the output buffer.

[0013] As a preferred technical solution, when the computation task is a pure matrix multiplication, the initial value of the bias / accumulation matrix C stored in the processing unit's accumulation register is set to zero; When the computation task is matrix multiplication and addition, the bias term vector is broadcast and filled into the bias / accumulation matrix C stored in the processing unit's accumulation register as the initial accumulation value.

[0014] As a preferred technical solution, the global control unit integrates a performance counter to monitor the utilization rate of the logic units inside the FPGA and the read / write conflict rate of the on-chip cache in real time. When the logic unit temperature is too high or the bus is congested, the global control unit dynamically adjusts the interval of instruction issuance, that is, adjusts the fill rate of the data pipeline.

[0015] As a second aspect of the present invention, a mixed-precision tensor computation method for edge computing is provided, the computation method employing the computation unit described above, and the steps include: Receive and parse the calculation instructions to obtain the dimension and storage address of the target matrix; The input matrix A and weight matrix B in half-precision floating-point format, as well as the bias / accumulation matrix C in single-precision floating-point format, are read from external memory via the AXI bus interface. The input matrix A and weight matrix B are rearranged according to the column / row interleaving rules and stored in the corresponding banks of the input buffer and weight buffer. The read bias / accumulation matrix C is preloaded into the accumulation register of the systolic array as the initial value. The systolic array starts systolic computation. In each clock cycle, multiple banks of the input buffer output a column of data of the input matrix A in parallel and enter the left boundary of the systolic array. Multiple banks of the weight buffer output a row of data of the weight matrix B in parallel and enter the upper boundary of the systolic array. The input matrix A, in clock-beat half-precision floating-point format, is passed to the right along the row direction in the pulsating array, and the weight matrix B data is passed down along the column direction in the pulsating array, forming a pulsating data stream. The input matrix A and the weight matrix B data enter the processing unit through two input channels to perform multiplication operations. After the multiplication result is converted into a single-precision floating-point number format, it is accumulated with the value of the bias / accumulation matrix C in single-precision floating-point number format to obtain the accumulation result in single-precision floating-point number format and written back to the accumulation register. Once all calculation cycles are completed, the value stored in the processing unit's accumulator register is used as the final result matrix D, which is then transferred to the output buffer via the output channel and written back to external memory.

[0016] As a preferred technical solution, if the matrix dimension of the matrix to be calculated exceeds the size of the systolic array, the matrix to be calculated is logically divided into multiple sub-blocks that match the dimension of the systolic array. The data of each sub-block is sequentially scheduled into the pulsating array for calculation. After the calculation of the current sub-block is completed, the intermediate result is temporarily stored in the accumulator register or written out and temporarily stored in the output buffer. The data of the next sub-block is scheduled to continue to enter the array and is accumulated with the intermediate result of the previous sub-block.

[0017] As a preferred technical solution, the global control unit determines the calculation type: If the computation task only involves matrix multiplication, then the accumulator registers of all execution units in the systolic array are reset to zero; If the computation task involves addition with intermediate results of bias or historical accumulated values, the bias term vector or intermediate results of historical accumulated values ​​are broadcast to the accumulation register of the corresponding execution unit as initial values.

[0018] As a preferred technical solution, the processing unit adopts a bit-width-based resource mapping strategy, receiving a half-precision floating-point format element of the input matrix A from the row direction and a half-precision floating-point format element of the weight matrix B from the vertical direction. The mapping logic of the processing unit directly maps these two half-precision floating-point format operands to the multiplier input port of a single DSP slice, and completes a full-precision mantissa multiplication operation within one clock cycle. The intermediate product results are expanded and aligned inside the processing unit and sent to the single-precision floating-point adder. The adder reads the current accumulated value in single-precision floating-point format stored in the accumulator register, performs the addition operation, and writes the updated accumulated result in single-precision floating-point format back to the accumulator register.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1) This invention designs a systolic array architecture based on a multi-bank caching mechanism. The systolic array adopts a two-dimensional mesh topology. Processing units pass input data to the next adjacent processing unit through data routing logic, forming a systolic data stream. Data flows only between adjacent processing units in the array. This local connectivity significantly reduces wiring congestion within the FPGA, which is beneficial for increasing the system's operating clock frequency. The input and weight caches adopt a multi-bank interleaved storage structure. Each bank corresponds to a row or column of the systolic array, providing the tensor data required for computation in parallel within a single clock cycle. This avoids data access conflicts, eliminates memory access bottlenecks, and ensures that the systolic array pipeline is always fully loaded. This invention achieves efficient matrix parallel computation through an optimized two-dimensional mesh topology systolic array and data flow control strategy, effectively meeting the real-time inference requirements of complex models at the edge.

[0020] 2) In order to optimize resources and performance, this invention decouples the precision of input operands and accumulation operands. When processing the core matrix multiplication and addition operations, mixed precision calculation is adopted. Half-precision floating-point numbers are used as input operands for input matrix A and weight matrix B to reduce memory bandwidth usage and improve parallelism. Single-precision floating-point numbers are used as accumulation and output operands to ensure the numerical stability and accuracy of the calculation process, prevent overflow or underflow, and maximize data throughput while ensuring calculation accuracy. Attached Figure Description

[0021] Figure 1 This is a block diagram of the overall hardware architecture of the hybrid precision tensor computing unit for edge computing provided in this embodiment of the invention.

[0022] Figure 2 This is a schematic diagram of the input feature map cache and weight cache structure using a multi-bank interleaving strategy in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of the internal logic structure of a single processing unit (PE) within a hybrid precision pulsating array in an embodiment of the present invention.

[0024] Figure 4 This is a flowchart illustrating the mixed-precision tensor calculation method provided in this embodiment of the invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0026] Example 1 This invention proposes a hybrid precision tensor computation unit for edge computing. Addressing the conflict between limited hardware resources at the edge and the high-performance requirements of model inference, a reconfigurable computation architecture based on a systolic array is designed. By decoupling the precision of input operands from that of accumulated operands, and combining this with a systolic array architecture based on a multi-bank storage mechanism, a balance between high performance and high precision is achieved.

[0027] like Figure 1 As shown, the tensor computation unit provided in this embodiment mainly consists of the following core modules: an AXI bus interface module, a global control unit, a data formatting unit, an on-chip cache system, and a core mixed-precision systolic array.

[0028] Regarding system interfaces and control logic: The AXI bus interface is used for communication between the mixed-precision tensor computation unit and external memory and the main processor, receiving computation instructions and tensor data.

[0029] In some embodiments, the system communicates with external memory or the main processor via an AXI bus interface. This interface module is configured with a direct memory access controller for efficiently moving large-scale tensor data under the scheduling of the global control unit.

[0030] The global control unit is the command center of the entire system, containing an instruction decoder and a state machine. It is responsible for parsing computational instructions from the main processor, including matrix dimensions, base addresses, and start signals, and generating the enable signals, read / write addresses, and handshake signals required by each submodule. This unit employs a synchronous timing design to ensure that all control signals are strictly aligned with the system clock, guaranteeing the stability of the large-scale array operation.

[0031] Furthermore, during system operation, the global control unit also performs resource monitoring. The global control unit integrates performance counters to monitor the utilization rate of the DSP units within the FPGA and the read / write conflict rate of the on-chip BRAM in real time. When excessively high logic unit temperatures or bus congestion are detected, the control unit dynamically adjusts the instruction issuance interval, i.e., adjusts the data pipeline fill rate, optimizing overall energy efficiency while ensuring the system does not experience thermal throttling.

[0032] Regarding data formatting and preprocessing: The data formatting unit is connected between the AXI bus interface and the on-chip cache. It is used to preprocess the input data and convert it into a format that conforms to the input of the systolic array. Since external memory typically stores a one-dimensional data stream with consecutive addresses, while the systolic array requires two-dimensional block data input, this unit is used to perform data rearrangement, padding, and format conversion.

[0033] The data formatting unit also includes down-precision conversion logic for processing FP32 to FP16, and alignment logic for processing FP32 output data.

[0034] In some embodiments, when the dimension of the matrix to be calculated is less than 32×32, the data formatting unit automatically performs zero-padding; when the dimension exceeds 32×32, the global control unit, in conjunction with this module, adopts a block-based strategy to divide the large matrix into multiple 32×32 sub-blocks for iterative calculation. Furthermore, this unit is also responsible for converting externally input FP32 data streams to FP16 format when necessary to adapt to the input requirements of the systolic array.

[0035] Meanwhile, during the data output phase, this unit is also responsible for receiving FP32 data from the output buffer, aligning its address and adapting its bit width so that it can be efficiently written back to external memory via the AXI bus.

[0036] Regarding the multi-bank design of on-chip cache systems: The on-chip caching system includes: an input feature map cache and a weight cache, used to store the data of the input matrix A and the weight matrix B, respectively; and an output cache, used to store the data of the calculated output matrix D.

[0037] To address the data access conflict issue under high-parallelism computing, this embodiment employs specific architectural optimizations for the input feature map cache and weight cache. For example... Figure 2 As shown, the input feature map cache is not a single storage block. Both the input feature map cache and the weight cache adopt a multi-bank interleaved storage structure, physically divided into 32 independent storage banks, and employ an interleaved storage strategy: the matrix's first... i The row data is stored in the first row. i Each Bank corresponds to a row or column of the systolic array, used to provide the tensor data required for computation in parallel within a single clock cycle, avoiding data access conflicts.

[0038] When a systolic array starts computation, the system often needs to provide 32 different row data elements to the first column of the array simultaneously within one clock cycle. If a single-unit memory is used, this requires 32 clock cycles to complete the read, severely slowing down the computation speed. However, with the multi-bank structure of this embodiment, the global control unit can simultaneously activate the read ports of 32 banks, reading 32 data elements in parallel within one clock cycle, completely eliminating the memory access bottleneck and ensuring that the pipeline of the systolic array is always at full load.

[0039] Regarding the hybrid precision pulsating array architecture: The core computing engine in this embodiment is a 32-row by 32-column systolic array containing 1024 processing units. The array adopts a two-dimensional mesh topology, and data flows only between adjacent processing elements (PEs) in the array. This local connectivity feature greatly reduces wiring congestion inside the FPGA and helps to improve the system's operating clock frequency.

[0040] The array is specifically configured to perform general matrix multiplication and addition operations such as A×B+C=D. To achieve optimal performance with the limited resources of the FPGA, the system employs a mixed-precision strategy: the input data for matrices A and B are in FP16 format. In this embodiment, FP16 specifically refers to a half-precision floating-point format conforming to the IEEE 754-2008 standard (1 sign bit, 5 exponent bits, 10 mantissa bits). This format reduces storage bandwidth and cache usage by half compared to FP32, while utilizing the half-precision multiplication capabilities of the FPGA's internal DSP slices, achieving double the multiplication throughput.

[0041] The bias / accumulation matrix C and the result matrix D use FP32 format (i.e., IEEE 754 single-precision floating-point numbers). This ensures that the values ​​have sufficient dynamic range and precision when performing a large number of accumulation operations, avoiding the overflow or quantization error problems common in fixed-point operations, thereby guaranteeing the inference accuracy of the load forecasting model.

[0042] Regarding the internal logic of the processing unit: like Figure 3As shown, each processing unit in the array is meticulously designed to support synchronous sequential circuit design. Internally, each processing unit includes: two input channels, responsible for receiving FP16 feature map data and weight data from the left or top PE respectively; two FP16 input registers, used to temporarily store data from the two input channels; data routing logic, configuring the two data input channels to pass data to the next adjacent processing unit while simultaneously feeding it into the multiplier, forming a pulsating data stream; one FP16 multiplier for performing the multiplication of matrix A and matrix B; one FP32 adder for performing the addition of the multiplication result and the accumulator value; one FP32 accumulator register for storing the initial value of matrix C and the intermediate accumulated result D; and one output channel responsible for passing or outputting the calculated FP32 result to the next level.

[0043] The internal data calculation logic for each PE is as follows: First, the data input and routing logic. Input channels In_A and In_B are used to receive matrix A data from the left-adjacent PE and matrix B data from the upper-adjacent PE, respectively. These two input channels are configured with pipeline registers, which simultaneously feed data into the multiplier and pass it to the next PE on the right and below, respectively, thus forming a pulsating data stream.

[0044] Second, mixed-precision arithmetic logic. It includes an FP16 multiplier and an FP32 adder. The multiplier multiplies the two input FP16 data, and the result is losslessly converted to FP32 format before being fed into the adder.

[0045] Third, the accumulation and storage logic. It includes an FP32 accumulator register to store intermediate accumulation results or preloaded initial values ​​of matrix C. The adder adds the multiplication result to the current value in the accumulator register and writes the updated sum back to the register.

[0046] Fourth, the result output logic. After a complete matrix multiplication calculation is completed, the FP32 accumulation result in the PE is passed down the column direction through the output channel and finally sent to the output buffer through the output interface at the bottom of the array.

[0047] Example 2 As another embodiment of the present invention, this embodiment also provides an implementation of a specific computational process based on the above architecture. The mixed-precision tensor computation method provided in this application has the following process: Figure 4 As shown, the specific execution steps are as follows: Step S1: Instruction Parsing and Array Initialization. The global control unit receives calculation instructions from the main processor, parses the matrix dimension information (M, K, N) and the base address of each matrix in external memory. Based on the parsing results, the control unit resets the state machine of the systolic array and configures the routing logic within each PE.

[0048] Step S2: Data Preloading and Multi-Bank Rearrangement. The DMA controller initiates data transfer. If the computation task includes bias or historical accumulated values, the global control unit determines the computation type: if it is a pure matrix multiplication (A×B), it controls the resetting of the accumulation registers of all PEs in the array to zero; if it is a matrix multiplication plus (A×B+bias), it broadcasts the bias vector to fill the corresponding row's PE accumulation register as the initial value. Simultaneously, the FP16 data of matrix A and matrix B are read from external memory and rearranged by the data formatting unit according to the "row / column interleaving" rule, and written to the corresponding banks of the input buffer and weight buffer respectively, ensuring no conflicts during subsequent reads.

[0049] As a specific application scenario example, suppose the matrix to be calculated has a dimension of 64×64, exceeding the size of the physical array of 32×32. In this case, the global control unit will automatically divide the matrix logically into four 32×32 sub-blocks (Block(0,0),(0,1),(1,0),(1,1)). The control unit first schedules the data of Block(0,0) into the array for calculation. After the calculation is completed, the intermediate result is temporarily stored in the accumulator register or written to the cache. Subsequently, the data of Block(0,1) continues to enter the array and is accumulated with the previous intermediate result. Through this block-based iterative mechanism, this embodiment can handle matrix operations of arbitrary dimensions without being limited by the size of the physical hardware.

[0050] Step S3: Initiate pulse calculation. The global control unit sends a start signal. At the rising edge of each clock cycle, the 32 banks of the input buffer output a column of matrix A data in parallel into the left boundary of the array, and the 32 banks of the weight buffer output a row of matrix B data in parallel into the upper boundary of the array.

[0051] Step S4: Mixed-precision parallel multiplication and addition with data transfer. This is the core computation step. Each processing unit in the array captures data according to the clock tick and performs the following micro-operations: Specifically, to fully utilize the hardware characteristics of the DSP slices inside the FPGA, the PE adopts a bit-width-based resource mapping strategy. The PE receives one FP16 element of matrix A from the horizontal direction and one FP16 element of matrix B from the vertical direction. The mapping logic inside the PE directly maps these two 16-bit operands to the multiplier input port of a single DSP slice. In this embodiment, the 27×18 bit-width interface of the DSP48E2 is used to process B_FP16, completing a full-precision mantissa multiplication operation within one clock cycle. Subsequently, the intermediate product is expanded and aligned inside the PE and sent to the FP32 adder. The adder reads the 32-bit current value C_FP32 stored in the accumulator register, performs the operation P=A_FP16×B_FP16+C_FP32, and writes the updated FP32 result back to the accumulator register.

[0052] Simultaneously with the computation, data flow occurs: the PE (Programmer) passes the currently used input data and A_FP16 and B_FP16 to the adjacent PEs on the right and below, respectively. Due to its pulsating nature, the data flows across the array like a wave, maximizing the reuse of input data and reducing redundant memory accesses.

[0053] Step S5: Result Output and Write-back. Once all computation cycles are complete, meaning all data from matrices A and B has flowed through the array, the global control unit determines the computation is finished and issues an output command. At this point, the accumulator registers within each PE store the final computation result D, or, for block-based iterative computation, the intermediate result of the current sub-block. If it is the final result, the PE shifts the FP32 format result data downwards along the column direction through the output channel and stores it in the output buffer. Finally, the DMA writes the subsequent data blocks back to external memory, completing a full inference computation task. If it is an intermediate result, the system can choose to temporarily store it in the output buffer for the next round of accumulation, or write it back to external memory.

[0054] In summary, this embodiment significantly improves the matrix operation efficiency of FPGA edge computing nodes without sacrificing model prediction accuracy through a hybrid precision design and multi-bank storage optimization at the hardware level. The combination of a 32×32 array size and FP16 / FP32 precision perfectly meets the needs of dense matrix operations in current power load forecasting models, solving the pain point of the inability to balance computing power and resources in existing technologies.

[0055] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0056] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An edge computing oriented mixed-precision tensor computing unit, characterized in that, Comprise: AXI bus interface for communicating with external host processor and memory, receiving computing instructions and tensor data; Global control unit receiving and decoding computing instructions issued by the host processor and generating global control signals; On-chip cache system including input cache, weight cache and output cache for storing input matrix A, weight matrix B and computed result matrix D data; the input cache and weight cache use multi-Bank interleaved storage structure inside, each Bank corresponds to a row or a column of systolic array; Systolic array composed of multiple processing units connected in a two-dimensional mesh topology with a set of rows and columns; The first column of processing units on the left side of the systolic array receives input matrix A data from the corresponding Bank in the input cache along the row direction and passes it to the right according to the clock beat; the first row of processing units above the systolic array receives weight matrix B data from the corresponding Bank in the weight cache along the column direction and passes it down according to the clock beat; the matrix multiplication and addition operation of the result matrix D is obtained by executing the multiplication of the input matrix A and the weight matrix B and then adding the bias / accumulation matrix C, wherein the data format of the input matrix A and the weight matrix B is half-precision floating-point number, and the data format of the bias / accumulation matrix C and the result matrix D is single-precision floating-point number.

2. The mixed-precision tensor computing unit for edge computing of claim 1, wherein, The computing unit also includes a data formatting unit connected between the AXI bus interface and the on-chip cache; The data formatting unit converts the one-dimensional data stream input from the external memory into two-dimensional block data input, and converts the input data and weight data in single-precision floating-point number format into half-precision floating-point number format; When the dimension of the matrix to be calculated does not meet the corresponding row and column of the systolic array, the data formatting unit pads the matrix to be calculated with zeros to fill up; When the dimension of the matrix to be calculated exceeds the corresponding row and column of the systolic array, the data formatting unit uses a block strategy to cut the matrix to be calculated into multiple sub-blocks corresponding to the dimension of the systolic array, and schedules each sub-block to enter the array in turn for iterative calculation; In the data output stage, the data formatting unit receives single-precision floating-point number data from the output cache, and performs address alignment and bit width adaptation, and then writes back to the external memory through the AXI bus interface.

3. The mixed-precision tensor computing unit for edge computing of claim 1, wherein, The input cache and weight cache using multi-Bank interleaved storage structure are as follows: The number of Banks in the input cache corresponds to the column dimension of the systolic array, and the input matrix A to be calculated is stored in the corresponding physical Bank address segment of the input cache according to the column dimension; The number of Banks in the weight cache corresponds to the row dimension of the systolic array, and the weight matrix B to be calculated is stored in the corresponding physical Bank address segment of the input cache according to the row dimension; When the systolic array starts computing, the global control unit reads the corresponding row / column data from each Bank and distributes it to the systolic array.

4. The mixed-precision tensor computing unit for edge computing of claim 1, wherein, The processing unit includes two data input channels, a half-precision floating-point multiplier, a single-precision floating-point adder, and a single-precision floating-point accumulation register; The data input channels are respectively used for receiving input matrix A data from a left neighboring processing unit and weight matrix B data from an upper neighboring processing unit; the processing unit is further configured with a pipeline register, which is used for transmitting the input data to the next processing unit on the right and below while sending the input data to the multiplier; The multiplier is used for multiplying the input matrix A data and the weight matrix B data, and transmitting the multiplication result converted into single-precision floating point number format to the adder; The bias / accumulation matrix C stored in the accumulation register is in single-precision floating point number format, the adder is used for adding the multiplication result and the current value of the bias / accumulation matrix C stored in the accumulation register, and writing the updated sum back to the accumulation register; After the systolic array completes a complete matrix multiplication calculation, the accumulation result in the processing unit is transmitted along the column direction to the next processing unit below through the output channel, and the result matrix D output from the bottom of the systolic array is transmitted to the output buffer.

5. The mixed-precision tensor computing unit for edge computing of claim 4, wherein, When the calculation task is pure matrix multiplication, the initial value of the bias / accumulation matrix C stored in the accumulation register of the processing unit is set to zero; When the calculation task is matrix multiplication and addition, the bias item vector is broadcasted and filled into the bias / accumulation matrix C stored in the accumulation register of the processing unit as the initial accumulation value.

6. The mixed-precision tensor computing unit for edge computing of claim 1, wherein, The global control unit is internally integrated with a performance counter, which is used for monitoring the utilization rate of the internal logic unit of the FPGA and the read-write conflict rate of the on-chip cache in real time, and dynamically adjusting the interval of instruction transmission, i.e. adjusting the filling rate of the data pipeline, when the logic unit temperature is too high or the bus is congested.

7. An edge computing oriented mixed-precision tensor computation method, characterized in that, The calculation method adopts the calculation unit according to any one of claims 1-6, and the steps comprise: receiving and analyzing the calculation instruction to obtain the target matrix dimension and the storage address; reading the input matrix A and the weight matrix B in half-precision floating point number format and the bias / accumulation matrix C in single-precision floating point number format from the external memory through the AXI bus interface; rearranging the input matrix A and the weight matrix B according to the column / row interleaving rule and storing them in the input buffer and the weight buffer respectively, and preloading the read bias / accumulation matrix C into the accumulation register of the systolic array as the initial value; starting the systolic calculation of the systolic array, and in each clock cycle, a plurality of Banks of the input buffer output a column of input matrix A data into the left boundary of the systolic array in parallel, and a plurality of Banks of the weight buffer output a row of weight matrix B data into the upper boundary of the systolic array in parallel; the input matrix A in half-precision floating point number format is transmitted in the row direction to the right in the systolic array according to the clock beat, and the weight matrix B data is transmitted in the column direction to the bottom in the systolic array, forming a systolic data stream; the input matrix A and the weight matrix B data enter the processing unit through two input channels to perform multiplication operation, the multiplication result is converted into single-precision floating point number format, and then added with the value of the bias / accumulation matrix C in single-precision floating point number format to obtain the accumulation result in single-precision floating point number format and write it back to the accumulation register; When all the calculation beats are completed, the value stored in the processing unit accumulation register is taken as the final result matrix D and transmitted to the output buffer through the output channel and written back to the external memory.

8. The mixed-precision tensor computation method for edge computing according to claim 7, wherein, If the matrix dimension of the matrix to be calculated exceeds the size of the systolic array, the matrix to be calculated is logically divided into multiple sub-blocks matching the dimension of the systolic array. The data of each sub-block is sequentially scheduled to enter the systolic array for calculation, and the intermediate result is temporarily saved in the accumulation register or written out to the output buffer after the current sub-block calculation is completed; the data of the next sub-block is scheduled to continue entering the array and accumulated with the intermediate result of the previous sub-block.

9. The edge computing oriented mixed precision tensor computing method according to claim 8, characterized in that, The global control unit determines the calculation type: If the calculation task only contains matrix multiplication, reset the accumulation registers of all execution units in the systolic array to zero; If the calculation task contains addition operation with bias or historical accumulation value intermediate result, the bias vector or historical accumulation value intermediate result is broadcast filled into the accumulation registers of the execution units in the corresponding row as the initial value.

10. The mixed-precision tensor computation method for edge computing according to claim 7, wherein, The processing unit adopts a resource mapping strategy based on bit width, receives one half-precision floating point format element of the input matrix A from the row direction and one half-precision floating point format element of the weight matrix B from the vertical direction, and the mapping logic of the processing unit directly maps the two half-precision floating point format operands to the multiplier input port of a single DSP slice, completes one full-precision mantissa multiplication operation in one clock cycle. The intermediate product result is expanded and aligned inside the processing unit and sent to the single-precision floating point adder, the adder reads the current accumulation value in single-precision floating point format stored in the accumulation register, performs addition operation, and writes the updated single-precision floating point format accumulation result back to the accumulation register.

Images