A method, device and FPGA accelerator for FPGA acceleration

Abstract

The application discloses a FPGA acceleration method and device and FPGA accelerator, and relates to the technical field of integrated circuit design. Two independent storage arrays are used to separately store even column weights and odd column weights, full set of weight data does not need to be copied, on-chip storage space is greatly saved, and device capacity over-limit problem is avoided. Even column weights and odd column weights are read in parallel, two-way independent multiplication and accumulation arrays are used to synchronously execute multiplication and accumulation operations of the even column weights and the odd column weights, the whole matrix vector multiplication operation round is halved, and reasoning delay is significantly reduced. The two-way operation shares input and hidden vectors read from a vector register, does not need to additionally copy a vector cache, and further saves on-chip storage resources. Finally, the results of the two-way independent operations are combined, calculation logic is not damaged, and original matrix vector multiplication mathematical semantics are strictly maintained. Meanwhile, the two-way calculation is completely parallel, and data supply is continuously sufficient, so that the problems of insufficient data supply and frequent data flow interruption in single-way serial operation are solved.

Description

Technical Field

[0001] This application relates to the field of integrated circuit design technology, and in particular to a method, apparatus and FPGA accelerator for FPGA acceleration. Background Technology

[0002] Gated recurrent units (GRUs) are widely used in edge-based time-series and reinforcement learning real-time inference scenarios, and their operations involve a large number of matrix-vector multiplication operations. Current field-programmable gate array (FPGA) neural network accelerators generally adopt a serial multiply-accumulate architecture that reads matrix weights column by column, relying on the entire on-chip memory of the FPGA to store all model weights, and completing the computation flow through general scheduling logic.

[0003] In existing technologies, this serial multiply-accumulate architecture requires numerous computation rounds when processing high-dimensional matrices, and general-purpose hardware lacks a dedicated GRU data stream. Improving efficiency by adding computation units is limited by insufficient bandwidth for weight reading, while increasing bandwidth by copying all matrix weights would consume a large amount of FPGA storage space. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method, apparatus, and FPGA accelerator for FPGA acceleration, thereby improving computational efficiency without significantly increasing the on-chip storage footprint of the FPGA.

[0005] This application discloses a method for FPGA acceleration, the method comprising: Parse the matrix-vector multiplication instruction to obtain the matrix dimension, the starting position of the weights, and the vector register address; The number of weights in even-numbered columns and the number of weights in odd-numbered columns in the weight matrix to be read are determined based on the matrix dimensions. Based on the weight starting position, the number of weights in the even-numbered columns, and the number of weights in the odd-numbered columns, the weights in the even-numbered columns and the weights in the odd-numbered columns are read in parallel; the storage array used to store the weights in the even-numbered columns and the storage array used to store the weights in the odd-numbered columns are independent of each other; The input vector and hidden vector are read based on the vector register address; The first and second operations are performed simultaneously to obtain the results of the first and second operations. Both the first and second operations are multiplication-accumulation operations. The multiplication-accumulation array used for the first operation and the multiplication-accumulation array used for the second operation are independent of each other. The elements participating in the first operation include the input vector, the hidden vector, and the even-numbered column weights. The elements participating in the second operation include the input vector, the hidden vector, and the odd-numbered column weights. Combine the results of the first and second operations to obtain the matrix-vector multiplication result.

[0006] Optionally, the matrix-vector multiplication instruction includes bias data and performs a first operation, including: The bias data is used as an initial value and fed into the multiply-accumulate array for performing the first operation.

[0007] Optionally, if the matrix dimension is odd, the last value of the weight of the odd-numbered column in the storage array is invalid, and a second operation is performed, including: In the second operation, the invalid values ​​are excluded from the operation.

[0008] Optionally, after obtaining the matrix-vector multiplication result, the method further includes: Convert the matrix-vector multiplication result into an output strategy; Select the output strategy.

[0009] Optionally, the matrix-vector multiplication instruction includes a nonlinear activation function, and after obtaining the matrix-vector multiplication result, the method further includes: The nonlinear activation function is applied to the result of the matrix-vector multiplication.

[0010] Based on the above-mentioned FPGA acceleration method, this application also discloses an FPGA acceleration apparatus, including: a parsing unit, a determination unit, a weight reading unit, a vector reading unit, a calculation unit, and a merging unit; The parsing unit is used to parse the matrix-vector multiplication instruction to obtain the matrix dimension, weight starting position, and vector register address; The determining unit is used to determine the number of even-numbered column weights and the number of odd-numbered column weights in the weight matrix to be read based on the matrix dimension. The weight reading unit is used to read the weights of the even-numbered columns and the weights of the odd-numbered columns in parallel based on the weight starting position, the number of weights in the even-numbered columns, and the number of weights in the odd-numbered columns; the storage array used to store the weights of the even-numbered columns and the storage array used to store the weights of the odd-numbered columns are independent of each other; The vector reading unit is used to read the input vector and the hidden vector based on the vector register address; The computation unit is used to simultaneously perform a first operation and a second operation to obtain a first operation result and a second operation result; both the first operation and the second operation are multiplication-accumulation operations, and the multiplication-accumulation array used for the first operation and the multiplication-accumulation array used for the second operation are independent of each other; the elements participating in the first operation include the input vector, the hidden vector, and the even-numbered column weights, and the elements participating in the second operation include the input vector, the hidden vector, and the odd-numbered column weights; The merging unit is used to merge the first operation result and the second operation result to obtain the matrix-vector multiplication result.

[0011] Optionally, the matrix-vector multiplication instruction includes bias data, and the operation unit includes: The first operation subunit is used to send the bias data as an initial value into the multiply-accumulate array used to perform the first operation.

[0012] Optionally, if the matrix dimension is odd, the last value of the weight of the odd-numbered column in the storage array is invalid. The operation unit includes: A masking subunit is used to mask the participation of the invalid value in the second operation.

[0013] Optionally, the device further includes: A conversion unit is used to convert the matrix-vector multiplication result into an output strategy; The selection unit is used to select the output strategy.

[0014] Optionally, the matrix-vector multiplication instruction includes a nonlinear activation function, and the apparatus further includes: An execution unit is used to execute the nonlinear activation function on the result of the matrix-vector multiplication.

[0015] Based on the above-mentioned FPGA acceleration method, this application also discloses an FPGA accelerator, characterized in that, for implementing the above-mentioned method, the FPGA accelerator includes: two independent memory arrays, a main controller, an address generation module, and a processor computing core; One of the two storage arrays is used to store the weights of even-numbered columns, and the other is used to store the weights of odd-numbered columns; The main controller is used to read and parse matrix-vector multiplication instructions, and to drive matrix-vector multiplication operations; The address generation module is used to generate the starting position of the weight; The processor computing core includes a vector register, a data router, two independent multiply-accumulate arrays, and a summation circuit. The vector register is used to store the input vector and the hidden vector; The data router is used to read the input vector and the hidden vector, and to send the input vector, the hidden vector, the even column weights and the odd column weights into the corresponding multiply-accumulate array; One of the two multiply-accumulate arrays is used for the first operation, and the other multiply-accumulate array is used for the second operation; The summing circuit is used to combine the results of the first operation and the second operation to obtain the matrix-vector multiplication result.

[0016] Optionally, data transmission in the FPGA accelerator is performed via a dedicated on-chip data stream.

[0017] Optionally, the FPGA accelerator further includes: an action value sampling module and an output result buffering module; The action value sampling module is used to obtain an output strategy based on the matrix-vector multiplication result, and to select the output strategy. The output result caching module is used to store the output strategy.

[0018] Optionally, the processor computing core further includes: an activation function coefficient generation module; The activation function coefficient generation module is used to generate the coefficients corresponding to the nonlinear activation function in the matrix-vector multiplication instruction.

[0019] This application discloses an FPGA acceleration method, apparatus, and FPGA accelerator. The method involves parsing instructions to obtain the matrix dimension, weight positions, and vector addresses, and determining the number of weights in even-numbered and odd-numbered columns based on the matrix dimension. The even-numbered and odd-numbered column weights are read in parallel based on their starting positions, the number of even-numbered and odd-numbered column weights. Two independent storage arrays are used to separately store the even-numbered and odd-numbered column weights, eliminating the need to copy the entire set of weight data, significantly saving on-chip storage space on the FPGA, and avoiding device capacity overrun issues. Two independent multiply-accumulate arrays are used to synchronously execute the multiply-accumulate operations of the even-numbered and odd-numbered columns, halving the overall matrix-vector multiplication rounds and significantly reducing inference latency. Furthermore, the two operations share the input and hidden vectors read from the vector register, eliminating the need for additional vector cache copying, further saving on-chip storage resources. Finally, the results of the two independent operations are merged without disrupting the computational logic, strictly maintaining the original mathematical semantics of matrix-vector multiplication. Simultaneously, the two computations are fully parallel with a continuous and sufficient data supply, solving the problems of insufficient data supply and frequent data stream interruptions in single-channel serial operations, fully leveraging the low-latency advantages of FPGA hardware parallelism. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating an FPGA acceleration method disclosed in an embodiment of this application; Figure 2 This is a flowchart illustrating another FPGA acceleration method disclosed in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an FPGA acceleration device disclosed in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an FPGA accelerator disclosed in an embodiment of this application; Figure 5 This is a schematic diagram of FPGA accelerator resource usage as disclosed in an embodiment of this application. Detailed Implementation

[0022] Existing FPGA neural network accelerators use a column-by-column serial processing approach when handling matrix-vector multiplication operations with GRUs. This architecture accumulates significant inference latency in time-series, small-batch edge inference scenarios, making it difficult to meet real-time decision-making requirements. Furthermore, general-purpose computing hardware lacks a dedicated data flow path adapted to GRUs, resulting in substantial intermediate data transfer overhead during computation and making the pipeline prone to waiting and interruptions. Even simply adding more multiply-accumulate units cannot achieve continuous parallel computation because weights can only be read column by column and vectors are supplied from a single source.

[0023] To achieve dual-path parallel weight reading, the traditional method requires completely copying the entire weight storage, which consumes a significant amount of the FPGA's limited on-chip block random access memory (BRAM), potentially exceeding the chip's hardware capacity. Furthermore, simply splitting the parallel computation lacks standardized result merging logic, which can easily corrupt the original mathematical semantics of the GRU, leading to model calculation errors.

[0024] To address the aforementioned shortcomings, this application's method parses microinstructions to obtain the matrix dimension, weight positions, and vector register addresses, thereby defining the read scale for odd and even columns. Two independent on-chip memory arrays are used to store the weights of even and odd columns respectively, providing dual-path read bandwidth without copying all weights. Simultaneously, the same set of vector registers is reused to read the input and hidden vectors. Two independent multiply-accumulate arrays are used to synchronously perform two-way parallel multiply-accumulate (MAC) operations. Finally, a parity-summing circuit merges some of the output results from the two operations, preserving the complete GRU computation logic throughout. This significantly reduces the number of computation rounds and inference latency while effectively controlling on-chip memory resource usage.

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Example 1: This application discloses a method for FPGA acceleration.

[0027] For details, please refer to Figure 1 The FPGA acceleration method disclosed in this embodiment includes the following steps: Step 101: Parse the matrix-vector multiplication instruction to obtain the matrix dimension, weight starting position, and vector register address.

[0028] In this embodiment, this step is the first control pre-stage of the entire GRU inference hardware pipeline. It provides all the basic configuration parameters for the main controller, address decoding module, and processor computing core, determining the subsequent weight read range, vector cache read / write targets, and the loop iterations of the multiply-accumulate array. Traditional column-wise matrix-vector multiplication hardware lacks a dedicated instruction parsing process, resulting in scattered parameter distribution logic and issues such as address mismatch, misjudgment of computation iterations, and vector read / write conflicts, causing pipeline interruptions and errors in calculation results. This step uniformly parses instructions to extract three types of parameters, providing a unified control basis for weight storage addressing, vector data scheduling, and parallel computation of the odd / even array, ensuring the orderly progress of the entire on-chip dedicated data flow and eliminating parameter synchronization deviations among multiple modules.

[0029] As a feasible approach, microinstructions for matrix-vector multiplication or matrix-vector multiplication with nonlinear activation functions can be read and parsed. Specifically, the instruction queue can be implemented and stored in on-chip BRAM, with each microinstruction containing multiple control fields. After parsing, the matrix dimension of the weight matrix to be computed, the starting storage address of the weights in the on-chip BRAM array, and the vector register addresses of the input vector and hidden vector participating in the current operation can be accurately separated. The nonlinear activation function can be a rectified linear unit (ReLU).

[0030] In this embodiment, the parsed matrix dimensions can be used to calculate the total number of even-numbered and odd-numbered column groups, determine the parallel processing rounds of the two sets of multiply-accumulate arrays, and generate boundary masking control signals for odd-numbered input dimensions. The parsed weight starting positions can be used to coordinate with the even-numbered and odd-numbered column data layout to synchronously generate even-numbered and odd-numbered column BRAM parallel access addresses, enabling synchronous reading of the two weights in subsequent steps. The parsed vector register addresses can be used to control the reading of input vectors and hidden vectors, as well as the writing back of matrix-vector multiplication results, preventing vector data from different gate calculations from overwriting each other.

[0031] Step 102: Determine the number of weights in even-numbered columns and the number of weights in odd-numbered columns in the weight matrix to be read based on the matrix dimensions.

[0032] In this embodiment, this step follows the matrix dimension parameters parsed in step 101, completing the weight matrix grouping quantization statistics. It is a prerequisite for subsequent parallel reading of dual BRAMs and synchronous scheduling of two sets of multiply-accumulate arrays. Traditional single-path column-by-column operation architecture does not need to distinguish between the number of even and odd columns, and cannot achieve two-path parallel data streams. If the data scale of even and odd columns is not accurately divided according to the matrix dimension in advance, problems such as mismatch in the operation rounds of the two sets of multiply-accumulate arrays and invalid odd column data participating in the operation under the odd dimension will occur, which will destroy the original mathematical semantics of GRU. This step divides the number of weights into two types through dimension matching calculation, ensuring load balancing of dual-path computing and compliance of data reading logic.

[0033] As a feasible approach, the matrix dimension values ​​are obtained, and the number of weights in even-numbered columns and odd-numbered columns are counted according to the even / odd distribution rules of the weight matrix column numbers. For example, the weight matrix columns are divided according to the rule that 0, 2, 4... are even columns, and 1, 3, 5... are odd columns. When the total input dimension of the matrix is ​​even, the number of weights in even-numbered columns is equal to the number of weights in odd-numbered columns. When the total input dimension of the matrix is ​​odd, the number of weights in even-numbered columns is one more group than the number of weights in odd-numbered columns. The last group only contains valid weights from even-numbered columns, and the corresponding odd-numbered columns are invalid data.

[0034] In the method of this embodiment, the determined number of even-numbered column weights and odd-numbered column weights can be used to divide the number of even-numbered and odd-numbered column processing groups required for the overall operation. In subsequent steps, the number of parallel processing rounds for the entire set of matrix-vector multiplication can be determined based on the two types of weights.

[0035] Step 103: Based on the weight starting position, the number of weights in the even column, and the number of weights in the odd column, read the weights in the even column and the weights in the odd column in parallel.

[0036] In this embodiment, this step implements synchronous parallel reading of dual-path weights, which is the core step in accelerating parallel matrix-vector multiplication. Traditional single-path weight reading architectures rely solely on a single on-chip storage partition to serially output weight data, failing to continuously supply parallel data to two multiply-accumulate arrays, easily leading to idle computing arrays and data pipeline interruptions. This step uses two independent BRAM storage areas to store even-numbered and odd-numbered weights respectively. Without needing to copy the complete weight matrix or significantly increase on-chip storage, it synchronously reads the two weights, continuously providing a weight data stream with matching bandwidth for the two multiply-accumulate arrays in subsequent steps, eliminating the problem of idle parallel computing power from the data supply perspective.

[0037] In the method of this embodiment, two BRAM storage arrays can be set up. The storage array for storing even-numbered column weights and the storage array for storing odd-numbered column weights are independent of each other, and their hardware resources are completely isolated. In this process, only the original weight matrix is ​​split by column, without copying and storing the complete weights. This can strictly control the BRAM resource usage and avoid the storage overflow risk caused by copying the entire set of weights.

[0038] In the method of this embodiment, based on the parsed weight starting position and the determined number of even-numbered column weights and odd-numbered column weights, the same set of logical address timing shared by two independent storage arrays is generated, and the even-numbered column weights and odd-numbered column weights under the corresponding group are read synchronously. Data is read out in parallel group by group according to the group order until the reading operation of all even-numbered column and odd-numbered column weights is completed.

[0039] During the reading process, the weight data read independently from the two channels can be forwarded to the odd-column multiply-accumulate array and the even-column multiply-accumulate array respectively, so that the two sets of computing arrays can synchronously obtain the corresponding weight data and continuously carry out multiply-accumulate operations.

[0040] When the matrix dimension is odd, that is, the number of odd column weights is less than the number of even column weights, the end of the storage array storing the odd column weights can be filled with invalid data (such as 0) to keep the number of odd column weights equal to the number of even column weights, so as not to disrupt the overall parallel pipeline timing due to the inconsistent read lengths of the two paths.

[0041] In this embodiment, the information of the two independent BRAM storage arrays can be shown in the following table: Table 1 BRAM Memory Array Information Table

[0042] With 512 512 weighted data and 1 Taking the biased data of 512 as an example, its column-based splitting logic is shown in the table below: Table 2. Weighted and Biased Data Splitting Table

[0043] Each BRAM can provide a bit width of 2048 bits. The bias data and the weight data of even columns are stored in the BRAM storage array used to store the weights of even columns, while the weights of odd columns are stored in the BRAM storage array used to store the weights of odd columns. The parameters of different dimensions are stacked according to the column block depth in the way the bias data and weights are interleaved.

[0044] Step 104: Read the input vector and hidden vector based on the vector register address.

[0045] In this embodiment, this step can be performed synchronously with step 103. It uses the vector register address parsed in step 101 to read vector data, providing complete vector operation operands for the two-way multiply-accumulate array. Traditional hardware architectures often suffer from fragmented scheduling of vector read / write and weight read, resulting in repeated trips to off-chip storage for vector data, leading to significant data transfer overhead and pipeline waiting time bubbles. This step reuses the on-chip vector register file dual-read capability, reading the input vector and hidden vector required for the current calculation based on the address obtained from instruction parsing. The vector data resides on-chip, eliminating the need for frequent access to off-chip storage and significantly reducing data interaction latency.

[0046] As a feasible approach, the input vector and hidden vector corresponding to the current GRU gated computation are read based on the vector register address. These two types of vectors are used to align with the weight data read in parallel with step 103, and then perform multiplication-accumulation operations. The vector reading process proceeds synchronously with the weight reading sequence. For each set of even-numbered and odd-numbered weight columns, the corresponding vector fragment is read synchronously, ensuring that the vectors fed into the two sets of multiplication-accumulation arrays are matched with the weight data in time, preventing data supply lag.

[0047] Step 105: Perform the first and second operations simultaneously to obtain the results of the first and second operations.

[0048] Traditional single-array serial computation can only perform multiplication and addition column-by-column, resulting in numerous iterations and high inference latency when processing large-dimensional matrices. Furthermore, it lacks supporting logic for bias initialization and masking invalid data in odd-dimensional dimensions. In the method of this embodiment, the multiply-accumulate array used for the first operation and the multiply-accumulate array used for the second operation are independent of each other. These two physically isolated arrays execute operations synchronously without causing timing blockages. This maintains balanced computational load across the two paths while strictly aligning the original GRU mathematical computation semantics, avoiding computational distortion.

[0049] As a feasible approach, the input vector, hidden vector, and even-numbered column weights are fed into a multiply-accumulate array for the first operation, while the same array is fed into a multiply-accumulate array for the second operation. The first and second operations are performed simultaneously. Specifically, when the matrix-vector multiplication instruction includes bias data, this bias data can be used as an initial value and fed into the multiply-accumulate array for the first operation. In other words, before the even-numbered column multiply-accumulate array starts its multiply-accumulate loop, the bias data is loaded into the array's internal accumulator register as an initial value. The results of each subsequent multiplication of the even-numbered column weights with the vector are then sequentially added to this initial value, thus completely reproducing the standard mathematical operation flow of matrix-vector multiplication with bias.

[0050] As a feasible solution, when the matrix dimension is odd, invalid values ​​can be used to fill the storage depth at the end of the storage partition of the odd column. Furthermore, the hardware can automatically close the calculation path corresponding to the odd column when the second operation reaches the end through built-in boundary shielding control logic, thereby blocking the invalid value from entering the odd column multiplication and accumulation array and preventing the invalid value from participating in the accumulation and causing the calculation result to be wrong.

[0051] In this embodiment, after the two multiply-accumulate arrays continuously and synchronously complete their operations, they output independent first and second operation results, respectively. The two sets of operation results are then combined to obtain the complete matrix-vector multiplication result. The parallel operation mechanism of this embodiment does not require waiting for a single-path computation to complete before starting another path, thus halving the number of processing rounds for the main matrix-vector multiplication and significantly reducing the total inference latency caused by the superposition of multiple gated branches in the GRU.

[0052] Step 106: Combine the first operation result and the second operation result to obtain the matrix-vector multiplication result.

[0053] In this embodiment, this step follows the first and second operation results, completing data merging, post-processing activation, policy generation, and action sampling. It is the final stage of a single-round matrix-vector multiplication calculation and connects to the subsequent reinforcement learning action output chain. As an feasible solution, the two independent first and second operation results can first be added and merged using an even-odd summation circuit. This merges the accumulated values ​​of the even and odd columns into a complete and unified matrix-vector multiplication result, restoring the complete output vector and ensuring that the final value after parallel splitting calculation is equivalent to the column-by-column serial operation result.

[0054] As a feasible approach, when the matrix-vector multiplication instruction contains nonlinear activation functions such as ReLU, Sigmoid, or Tanh, the nonlinear activation function can be further applied to the complete matrix-vector multiplication result.

[0055] As a feasible approach, the complete matrix-vector multiplication result can be converted into an output policy. This output policy can be temporarily stored and isolated in an output result buffer, serving as the original output policy usable for reinforcement learning decisions. This isolates the computational data streams of the current time step from the next time step, preventing interference between time-series data. Subsequently, the output policy can be selected. Specifically, the output policy vector can be read through a comparator tree structure, and the maximum value is selected through multi-order numerical comparisons, automatically outputting the optimal action index. This allows action selection to be completed directly on the hardware side, without the need for secondary computation by an external processor, adapting to real-time decision-making and low-latency edge inference scenarios in reinforcement learning.

[0056] In the method of this embodiment, the entire merging, activation, and strategy sampling process is subject to unified timing control. Specifically, the completion flag signal can be set to block the early start of the next micro-instruction, preventing intermediate data that has not been merged or activated from participating in the next round of calculation, thus ensuring the stable and orderly operation of the entire inference pipeline.

[0057] The method described in this embodiment effectively reduces the processing time of matrix-vector multiplication operations. By relying on a parallel architecture that splits even and odd columns, the number of processing rounds for the main matrix-vector multiplication operation can be halved. Taking a scenario with an input dimension of 512 as an example, the traditional column-by-column approach requires 512 rounds of computation, while the method in this embodiment only requires 256 rounds. For the six groups of 512×512 matrix-vector multiplication operations within the GRU, the total number of executions in the main loop can be reduced from 3072 to 1536. This fundamentally reduces the basic loop latency of GRU temporal inference, adapting to edge real-time scenarios such as GRU and reinforcement learning policy networks that rely solely on small-batch time-step inference and cannot utilize large-batch data to mask computational latency. It effectively solves the core bottleneck of excessively high end-to-end inference latency in traditional column-by-column hardware structures.

[0058] The method described in this embodiment achieves double the read bandwidth and releases parallel computing power while properly controlling the overhead of on-chip storage resources on the FPGA. Unlike the inefficient design of achieving dual-path reads by completely copying the weight ROM, the method described in this embodiment only rearranges the even and odd columns of the original weight matrix and stores them in two independent BRAM arrays. It does not require copying the entire set of weights and bias data, and will not cause the problem of a significant increase in on-chip storage resource usage or exceeding the capacity of the target FPGA device. It stably provides dual-path parallel weight read capability under the constraint of limited on-chip BRAM resources, taking into account the parallel acceleration effect and hardware resource utilization, and reducing the pressure of device selection when implementing engineering.

[0059] Meanwhile, the hardware data flow design makes full use of existing on-chip storage resources. For the input vector supply chain, the method described in this embodiment reuses the native dual-read capability of the vector register file, eliminating the need to add or copy the entire set of vector register files to match the data requirements of the two-way multiply-accumulate array. This further saves on distributed storage and register-related hardware resource consumption. The entire data supply chain relies on the original storage unit to complete parallel scheduling, resulting in significant advantages in lightweight hardware architecture.

[0060] Furthermore, for special scenarios where the input dimension is odd, the method described in this embodiment is equipped with a complete boundary shielding control logic, which can automatically identify invalid values ​​padded at the end of the weight storage array and block them from participating in multiplication and accumulation operations, avoiding invalid zero values ​​from being superimposed on the calculation results. Regardless of whether the input dimension is odd or even, the final vector result after merging the two parallel operations can be consistent with the original column-by-column serial calculation result, fully preserving the original mathematical calculation semantics of the GRU model. The parallel acceleration process does not change the model output logic, and there is no risk of computational distortion or inference result deviation, ensuring that the algorithm accuracy is not affected by hardware parallelization.

[0061] The parallel computing architecture described in this embodiment employs a hierarchical data flow with independent operation of two multiply-accumulate arrays and unified merging at the end. This splits large-scale multiply-accumulate operations into two independent hardware paths to distribute the computational pressure, reducing the instantaneous load on a single computing array. Simultaneously, a complete completion control signal is set in the matrix-vector multiplication result merging stage to strictly control the lifecycle of instruction execution. This prevents the next microinstruction from starting prematurely, avoids intermediate data that has not been merged or activated from flowing into the next round of operation, eliminates pipeline data disorder and result overwriting issues, and ensures the stability and controllability of the multi-round continuous GRU time-series inference process.

[0062] For matrix-vector multiplication operations with nonlinear activation requirements, the method described in this embodiment uniformly places the nonlinear activation operation on the complete vector after the odd and even results are merged, without performing separate activation processing on the two split partial sums. It strictly follows the GRU standard calculation process, and the hardware parallel optimization does not disrupt the operation order of GRU internal gate control and state fusion, ensuring a high degree of consistency in the model inference logic.

[0063] Furthermore, the method described in this embodiment relies on a multi-level comparator tree to directly filter the maximum value of the output policy vector, and outputs the action index natively inside the FPGA hardware. There is no need to transmit the policy vector to an external processor for action judgment, which further reduces off-chip data interaction. It is particularly suitable for edge inference scenarios such as real-time decision-making in reinforcement learning. The entire acceleration device forms a full hardware closed loop from matrix operation to action output, further compressing end-to-end decision latency.

[0064] The method described in this embodiment has good engineering adaptability, high component standardization, and can be synthesized, implemented, and verified on board using mainstream FPGA development tools. Resource consumption is balanced and controllable, and there will be no excessive occupation of a single resource. After synthesis, there is sufficient timing margin, which can stably run at the set working clock frequency. It can be adapted to different models and series of FPGA devices, and the cost of implementation and modification is low. At the same time, the computing power scheduling and data layout logic of the whole architecture are standardized, which makes it easy to adapt parameters according to different input dimensions and different scales of GRU models, and has a wider range of general adaptability.

[0065] Example 2: This application discloses another method for FPGA acceleration; please refer to [link / reference]. Figure 2 This embodiment describes the entire process of FPGA acceleration.

[0066] Step 201: Store the weight and bias data in two independent BRAM arrays on the FPGA chip. The bias data and even-numbered column weight data are stored in the first BRAM, and the odd-numbered column weight data are stored in the second BRAM.

[0067] Step 202: Parse the matrix-vector multiplication instruction to obtain the matrix dimension, the starting position of the weight storage, and the vector register address.

[0068] Step 203: Determine the number of weights in even-numbered columns and odd-numbered columns in the weight matrix to be read based on the matrix dimensions. Execute steps 204 and 205 simultaneously.

[0069] Step 204: Based on the starting position of the weights, the number of even-numbered column weights, and the number of odd-numbered column weights, read the even-numbered column weights and bias data from the first BRAM, and simultaneously read the odd-numbered column weights from the second BRAM. Proceed to step 206.

[0070] Step 205: Read the input vector and hidden vector based on the vector register address. Execute steps 207 and 208 simultaneously.

[0071] Step 206: Input the bias data as the initial value into the first multiply-accumulate array.

[0072] Step 207: Input vector, hidden vector and even column weights are fed into the first multiply-accumulate array and the first operation is performed.

[0073] Step 208: Simultaneously feed the input vector, hidden vector, and odd column weights into the second multiply-accumulate array and perform the second operation.

[0074] Step 209: Combine the results of the first and second operations to obtain the matrix-vector multiplication result.

[0075] Step 210: Apply a nonlinear activation function to the matrix-vector multiplication result.

[0076] Step 211: Convert the matrix-vector multiplication result after performing the nonlinear activation function into an output policy, and select an output policy.

[0077] Based on the FPGA acceleration method disclosed in the above embodiments, this embodiment correspondingly discloses an FPGA acceleration apparatus. Please refer to... Figure 3 The FPGA acceleration device includes: a parsing unit 301, a determining unit 302, a weight reading unit 303, a vector reading unit 304, a calculation unit 305, and a merging unit 306. The parsing unit 301 is used to parse the matrix-vector multiplication instruction to obtain the matrix dimension, weight starting position and vector register address; The determining unit 302 is used to determine the number of even-numbered column weights and the number of odd-numbered column weights in the weight matrix to be read based on the matrix dimension. The weight reading unit 303 is used to read the weights of the even-numbered columns and the weights of the odd-numbered columns in parallel based on the weight starting position, the number of weights in the even-numbered columns, and the number of weights in the odd-numbered columns; the storage array used to store the weights of the even-numbered columns and the storage array used to store the weights of the odd-numbered columns are independent of each other; The vector reading unit 304 is used to read the input vector and the hidden vector based on the vector register address; The operation unit 305 is used to simultaneously perform a first operation and a second operation to obtain a first operation result and a second operation result; both the first operation and the second operation are multiplication-accumulation operations, and the multiplication-accumulation array used for the first operation and the multiplication-accumulation array used for the second operation are independent of each other; the elements participating in the first operation include the input vector, the hidden vector and the even-numbered column weights, and the elements participating in the second operation include the input vector, the hidden vector and the odd-numbered column weights; The merging unit 306 is used to merge the first operation result and the second operation result to obtain the matrix-vector multiplication result.

[0078] Optionally, the matrix-vector multiplication instruction includes bias data, and the operation unit 305 includes: The first operation subunit is used to send the bias data as an initial value into the multiply-accumulate array used to perform the first operation.

[0079] Optionally, when the matrix dimension is odd, the last value of the weight of the odd-numbered column in the storage array is an invalid value. The operation unit 305 includes: A masking subunit is used to mask the participation of the invalid value in the second operation.

[0080] Optionally, the device further includes: A conversion unit is used to convert the matrix-vector multiplication result into an output strategy; The selection unit is used to select the output strategy.

[0081] Optionally, the matrix-vector multiplication instruction includes a nonlinear activation function, and the apparatus further includes: An execution unit is used to execute the nonlinear activation function on the result of the matrix-vector multiplication.

[0082] Based on the FPGA acceleration method disclosed in the above embodiments, this embodiment discloses an FPGA accelerator for implementing the above method. Please refer to... Figure 4 The FPGA accelerator includes: two independent storage arrays 401-1 and 401-2, a main controller 402, an address generation module 403, and a processor computing core 404; One of the two storage arrays is used to store the weights of even-numbered columns, and the other is used to store the weights of odd-numbered columns; The main controller 402 is used to read and parse matrix-vector multiplication instructions, and to drive matrix-vector multiplication operations; The address generation module 403 is used to generate the weight starting position; The processor computing core 404 includes a vector register, a data router, two independent multiply-accumulate arrays, and a summing circuit. The vector register is used to store the input vector and the hidden vector; The data router is used to read the input vector and the hidden vector, and to send the input vector, the hidden vector, the even column weights and the odd column weights into the corresponding multiply-accumulate array; One of the two multiply-accumulate arrays is used for the first operation, and the other multiply-accumulate array is used for the second operation; The summing circuit is used to combine the results of the first operation and the second operation to obtain the matrix-vector multiplication result.

[0083] Optionally, data transmission in the FPGA accelerator is performed via a dedicated on-chip data stream.

[0084] Optionally, the FPGA accelerator further includes: an action value sampling module and an output result buffering module; The action value sampling module is used to obtain an output strategy based on the matrix-vector multiplication result, and to select the output strategy. The output result caching module is used to store the output strategy.

[0085] Optionally, the processor computing core further includes: an activation function coefficient generation module; The activation function coefficient generation module is used to generate the coefficients corresponding to the nonlinear activation function in the matrix-vector multiplication instruction.

[0086] In this embodiment, Vivado 2025.1 can be used as the FPGA development tool. The target device can be a Virtex-7 series XC7VX690T-2FFG1761C or an FPGA with equivalent BRAM / Digital Signal Processing (DSP) unit resources. The implementation process includes importing the Register Transfer Level (RTL), Intellectual Property (IP) cores, importing initialization files, adding constraints, synthesis, implementation, generating bitstreams, and on-board verification.

[0087] The main Xilinx IPs and key configurations relied upon in this embodiment are shown in the table below. The parameters in the table are from the XCI configuration file in the current Vivado project and are used to illustrate the actual engineering implementation of this embodiment. When converting to other FPGA tools later, the IPs can be selected according to the actual situation.

[0088] Table 3 Configuration Table

[0089] The weight and instruction initialization files match the data layout of this embodiment. For the odd-even column parallel version, the instruction queue uses the corresponding microinstruction file, and the weight BRAM uses two initialization files: one for even columns and one for odd columns. The even column file stores the weights of even-numbered input columns (0, 2, 4, etc.), and the odd column file stores the weights of odd-numbered input columns (1, 3, 5, etc.). When the input dimension is odd, zero padding can be used at the end of the odd column file, and boundary masking further prevents invalid data from participating in the calculation.

[0090] The corresponding operating procedures are as follows: S1: Import the RTL source code and confirm that the top-level instance contains the main controller, data router, weight / bias BRAM array, vector register, odd multiply-accumulate array, even multiply-accumulate array, odd-even summation circuit, action value sampling unit, and output result buffer.

[0091] S2: Configure the BRAM IP to bind the weight initialization files for even and odd columns to two weight / bias storage partitions respectively.

[0092] S3: Configure the Floating Point FMA IP to make the input, output, and accumulator register bit widths of the odd-multiply-accumulate array and the even-multiply-accumulate array consistent with the BF16 or target calculation format in the RTL.

[0093] S4: Configure the instruction queue initialization file to ensure that the matrix dimension, weight base address, vector source address, and vector destination address correspond to the on-chip layout of the weights / biases.

[0094] S5: Add the XDC constraint file to bind the clock, reset, input vector interface, output valid, debug signals, and board physical pins. Perform synthesis and implementation, and check the reports for DSP units, BRAM, look-up tables (LUTs), timing margins, etc., to confirm that the second set of fused multiply-add (FMA) units and the odd BRAM have not been incorrectly optimized.

[0095] S6: Generate a bitstream and download it to the FPGA development board. Verify that the hardware results are consistent with the software model results through the output interface of the Integrated Logic Analyzer (ILA) or Light Emitting Diode (LED).

[0096] The synthesis report and implementation report in the above process can serve as supplementary evidence of the technical effects of this embodiment. For example, when both odd-numbered and even-numbered multiplication and accumulation arrays are fully involved in the calculation and result merging, the DSP usage and on-chip signal waveforms should reflect the dual-array structure. If a certain result is not merged, the relevant logic may be optimized away by the synthesizer. Therefore, it is necessary to combine the resource report and ILA waveform to confirm the integrity of the odd-even summation logic.

[0097] In the FPGA accelerator of this embodiment, the on-chip resource utilization rate after placement and routing is as follows: Figure 5 As shown, Figure 5 The horizontal axis represents the percentage of resource utilization, and the vertical axis represents the various hardware resource types, from top to bottom: LUT, LUTRAM (Landed Random Access Memory implemented with lookup tables), Flip Flop (FF), BRAM, DSP unit, Input / Output (IO) pins, Global Buffer (BUFG), and Mixed Mode Clock Manager (MMCM). It can be seen that the accelerator in this embodiment has high BRAM and LUT resource utilization, surplus DSP computing resources, and low clock and IO resource utilization. Figure 5 It is used to characterize the actual consumption of each hardware resource under the parallel architecture of odd and even column splitting, and provides a basis for device selection and debugging IP configuration.

[0098] The embodiments in this specification are described in a progressive manner. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant details can be found in the method section.

[0099] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0100] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0101] The features described in the embodiments of this specification can be substituted for or combined with each other, so that those skilled in the art can implement or use this application.

[0102] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for FPGA acceleration, characterized in that, include: Parse the matrix-vector multiplication instruction to obtain the matrix dimension, the starting position of the weights, and the vector register address; The number of weights in even-numbered columns and the number of weights in odd-numbered columns in the weight matrix to be read are determined based on the matrix dimensions. Based on the weight starting position, the number of weights in the even-numbered columns, and the number of weights in the odd-numbered columns, the weights in the even-numbered columns and the weights in the odd-numbered columns are read in parallel. The storage array used to store the weights of the even-numbered columns is independent of the storage array used to store the weights of the odd-numbered columns; The input vector and hidden vector are read based on the vector register address; The first and second operations are performed simultaneously to obtain the results of the first and second operations. Both the first and second operations are multiplication-accumulation operations. The multiplication-accumulation array used for the first operation and the multiplication-accumulation array used for the second operation are independent of each other. The elements participating in the first operation include the input vector, the hidden vector, and the even-numbered column weights. The elements participating in the second operation include the input vector, the hidden vector, and the odd-numbered column weights. Combine the results of the first and second operations to obtain the matrix-vector multiplication result.

2. The method according to claim 1, characterized in that, The matrix-vector multiplication instruction contains bias data and performs the first operation, including: The bias data is used as an initial value and fed into the multiply-accumulate array for performing the first operation.

3. The method according to claim 1, characterized in that, When the matrix dimension is odd, the last value of the weight of the odd-numbered column in the storage array is invalid, and a second operation is performed, including: In the second operation, the invalid values ​​are excluded from the operation.

4. The method according to any one of claims 1-3, characterized in that, After obtaining the matrix-vector multiplication result, the method further includes: Convert the matrix-vector multiplication result into an output strategy; Select the output strategy.

5. The method according to claim 4, characterized in that, The matrix-vector multiplication instruction includes a nonlinear activation function. After obtaining the matrix-vector multiplication result, the method further includes: The nonlinear activation function is applied to the result of the matrix-vector multiplication.

6. An FPGA-accelerated device, characterized in that, include: The system comprises a parsing unit, a determination unit, a weight reading unit, a vector reading unit, a computation unit, and a merging unit. The parsing unit is used to parse the matrix-vector multiplication instruction to obtain the matrix dimension, weight starting position, and vector register address; The determining unit is used to determine the number of even-numbered column weights and the number of odd-numbered column weights in the weight matrix to be read based on the matrix dimension. The weight reading unit is used to read the weights of the even-numbered columns and the odd-numbered columns in parallel based on the weight starting position, the number of weights in the even-numbered columns, and the number of weights in the odd-numbered columns; The storage array used to store the weights of the even-numbered columns is independent of the storage array used to store the weights of the odd-numbered columns; The vector reading unit is used to read the input vector and the hidden vector based on the vector register address; The computation unit is used to simultaneously perform a first operation and a second operation to obtain a first operation result and a second operation result; both the first operation and the second operation are multiplication-accumulation operations, and the multiplication-accumulation array used for the first operation and the multiplication-accumulation array used for the second operation are independent of each other; the elements participating in the first operation include the input vector, the hidden vector, and the even-numbered column weights, and the elements participating in the second operation include the input vector, the hidden vector, and the odd-numbered column weights; The merging unit is used to merge the first operation result and the second operation result to obtain the matrix-vector multiplication result.

7. An FPGA accelerator, characterized in that, For implementing the method of any one of claims 1-5, the FPGA accelerator comprises: two independent memory arrays, a main controller, an address generation module, and a processor computing core; One of the two storage arrays is used to store the weights of even-numbered columns, and the other is used to store the weights of odd-numbered columns; The main controller is used to read and parse matrix-vector multiplication instructions, and to drive matrix-vector multiplication operations; The address generation module is used to generate the starting position of the weight; The processor computing core includes a vector register, a data router, two independent multiply-accumulate arrays, and a summation circuit. The vector register is used to store the input vector and the hidden vector; The data router is used to read the input vector and the hidden vector, and to send the input vector, the hidden vector, the even column weights and the odd column weights into the corresponding multiply-accumulate array; One of the two multiply-accumulate arrays is used for the first operation, and the other multiply-accumulate array is used for the second operation; The summing circuit is used to combine the results of the first operation and the second operation to obtain the matrix-vector multiplication result.

8. The FPGA accelerator according to claim 7, characterized in that, Data transmission in the FPGA accelerator is performed via a dedicated on-chip data stream.

9. The FPGA accelerator according to claim 8, characterized in that, The FPGA accelerator also includes: an action value sampling module and an output result buffering module; The action value sampling module is used to obtain an output strategy based on the matrix-vector multiplication result, and to select the output strategy. The output result caching module is used to store the output strategy.

10. The FPGA accelerator according to claim 8 or 9, characterized in that, The processor computing core further includes: an activation function coefficient generation module; The activation function coefficient generation module is used to generate the coefficients corresponding to the nonlinear activation function in the matrix-vector multiplication instruction.

Images