A neural network processor constructed based on a field programmable gate array

Abstract

This application discloses a neural network processor based on a field-programmable gate array (FPGA), including a main controller, an on-chip block memory array, a vector register file, and a general-purpose computing module connected to the main controller, the on-chip block memory array, and the vector register file, respectively, for performing all computational operations required for model inference; an action value sampling unit for sampling the maximum value index of the output policy vector to obtain the action index; and an output result cache for temporarily storing the inference result of the current time step and maintaining the output until the input of the next time step is valid. By eliminating the dependency on off-chip memory access through a fully on-chip memory architecture, and combining a simplified instruction set customized for recurrent neural network operators with a parallel computing pipeline, this effectively solves the problems of high latency and low efficiency when processing small batches of sequence data in general-purpose architectures, achieving millisecond-level end-to-end inference latency, and is suitable for edge reinforcement learning decision-making scenarios.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a neural network processor based on a field-programmable gate array (FPGA). Background Technology

[0002] Currently, reinforcement learning decision models based on recurrent neural networks are widely used in edge intelligence scenarios such as robot control and autonomous driving. These applications typically require processing single frames or small batches of real-time sequence data and have stringent requirements for end-to-end inference latency down to the millisecond level. In terms of hardware deployment, existing technologies mainly use general-purpose processors such as CPUs and GPUs or general-purpose deep learning accelerators based on FPGAs to accelerate model inference.

[0003] However, general-purpose architectures exhibit significant efficiency bottlenecks when processing small batches of sequential data. GPU architectures, optimized for parallel computation on large datasets to achieve high throughput, rely on massive data parallelism to mask memory access latency. However, when processing single frames or small batches of data specific to reinforcement learning tasks, the utilization of computing units drops sharply, and frequent data transfers between the host and device introduce significant latency overhead. CPUs, limited by their serial processing mechanism and complex cache layers, are slow when performing large-scale matrix multiplication operations, making it difficult to meet the throughput requirements of high-frequency decision-making. Therefore, existing general-purpose architectures cannot meet the millisecond-level real-time response requirements of edge decision-making systems. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a neural network processor based on a field-programmable gate array (FPGA).

[0005] The embodiments of this application disclose the following technical solutions: The first aspect of this application provides a neural network processor based on a field-programmable gate array (FPGA), comprising: The main controller is used to parse instructions and control the data flow; The on-chip block memory array, composed of on-chip block memory of the field-programmable gate array (FPGA), is used to store the weight matrix and bias parameters of the neural network model. The vector register file, consisting of distributed memory on the FPGA chip, is used to store intermediate result vectors during the computation process. A general computing module is connected to the main controller, the on-chip block memory array, and the vector register file, respectively. It is used to read parameters from the on-chip block memory array according to the instructions of the main controller, read intermediate result vectors from the vector register file, perform the calculation operations required for model inference, write the calculated intermediate result vectors back to the vector register file, and send the final output policy vector to the action value sampling unit. An action value sampling unit, connected to the general computing module, is used to sample the maximum value index of the output policy vector to obtain the action index; The output result cache is connected to the action value sampling unit and is used to store the action index and strategy vector of the current time step.

[0006] In one possible implementation, the main controller adopts a custom instruction set architecture with fixed-length encoding. The instruction fields of the custom instruction set include opcode, output selection, matrix dimension definition, vector dimension definition, on-chip block memory base address, and vector cache source or destination address.

[0007] In one possible implementation, the computational operations include matrix multiplication, vector-level operations, and nonlinear activation function operations.

[0008] In one possible implementation, the general computing module includes: multiple parallel multiply-accumulate computing units for performing matrix multiplication and vector dot product; and at least one nonlinear activation function computing unit for performing nonlinear activation function operations.

[0009] In one possible implementation, in the on-chip block memory array, the weight matrix of the same neural network layer is stored interleaved with the corresponding bias vector after being divided into rows, and the parameters of different layers are arranged sequentially in a deep stack manner, so that the controller can perform sequential addressing and reading based on the layer index and the row index.

[0010] In one possible implementation, the vector register file consists of multiple distributed memory instances, the address width and address depth of which are dynamically adjusted according to the length of the intermediate result vector to be cached.

[0011] In one possible implementation, the action value sampling unit consists of multiple sets of programmable comparator trees, which are dynamically configured according to the number of action heads output by the model.

[0012] In one possible implementation, the comparison data format of the action value sampling unit includes FP32 or BF16.

[0013] In one possible implementation, the general computing module continuously performs calculations on the reset gate, update gate, and candidate hidden state of the gated loop unit in the same pipeline.

[0014] In one possible implementation, the on-chip block memory array consists of multiple on-chip block memory instances, employing a unified address space and interleaved storage method, stacking the weight matrix and bias vector according to row block depth, and supporting the reading of multiple rows of data at one time.

[0015] Compared with the prior art, this application has the following beneficial effects: By setting up an on-chip block memory array and a vector register stack to statically store all weight parameters of the neural network model and intermediate result vectors during the calculation process, complete on-chip data residency is achieved, completely eliminating data exchange with off-chip memory and avoiding latency caused by frequent data transfers in GPU architecture. Simultaneously, addressing the slow speed of CPUs performing large-scale matrix multiplications due to their serial processing mechanism and complex cache levels, this application sets up a general-purpose computing module directly connected to the main controller and storage unit. This module can execute all computational operations required for model inference in parallel, working with the main controller to achieve precise and efficient on-chip data flow scheduling, ensuring that the computing unit maintains high utilization when processing single frames or small batches of data. Based on this, through the collaboration of the action value sampling unit and the output result cache, a complete end-to-end hardware inference link from input to output is constructed, enabling the entire NPU to independently complete all computational tasks on-chip, thereby reducing end-to-end inference latency to the millisecond level and meeting the real-time requirements of edge reinforcement learning decision systems. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A neural network processor architecture based on a field-programmable gate array (FPGA) is provided as an embodiment of this application. Figure 2 This is a schematic diagram illustrating the data flow and module collaboration relationships within the NPU, provided in an embodiment of this application. Figure 3 This describes the specific mapping rules between the weight matrix and the bias vector in the FPGA on-chip BRAM array in this application. Figure 4 This is a schematic diagram of the specific network structure of the reinforcement learning agent model implemented in the embodiments of this application; Figure 5 This is a complete flowchart illustrating the implementation of an NPU on an FPGA according to an embodiment of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0019] To facilitate understanding of the technical solutions provided in the embodiments of this application, the technical terms involved in the embodiments of this application will be explained below.

[0020] An FPGA (Field-Programmable Gate Array) is a semiconductor device that contains programmable logic blocks and programmable interconnect resources. Unlike application-specific integrated circuits (ASICs), FPGAs allow users to reconfigure their internal circuitry after manufacturing to meet design requirements, making them ideal for prototyping hardware accelerators and deploying small batches of high-performance computing.

[0021] An NPU (Neural Processing Unit) is a microprocessor or dedicated circuit designed specifically to accelerate computational tasks involving artificial neural networks. In this application, the NPU specifically refers to a logic circuit system implemented on an FPGA that provides hardware acceleration for recurrent neural network algorithms.

[0022] GRU (Gated Recurrent Unit) is a variant of the Recurrent Neural Network (RNN) architecture. It addresses the vanishing gradient problem in standard RNNs by introducing update and reset gates. Compared to LSTM, GRU has fewer parameters, higher computational efficiency, and is suitable for resource-constrained hardware implementations.

[0023] LSTM (Long Short-Term Memory) is a special type of RNN architecture that can learn long-term dependencies. It includes an input gate, a forget gate, and an output gate.

[0024] MAC (Multiply-Accumulate) is the most fundamental operation in digital signal processing and deep learning, which calculates the product of two numbers and adds it to an accumulator. The throughput of the MAC unit typically determines the upper limit of the NPU's performance in processing matrix multiplication.

[0025] ReLU (Rectified Linear Unit) is a commonly used non-linear activation function in neural networks. It can effectively alleviate gradient vanishing problems and accelerate network convergence. It is often used for the output activation of fully connected layers.

[0026] The sigmoid function (Sigmoid activation function) is a mathematical function that maps real numbers to the interval (0,1). In GRU networks, the sigmoid function is mainly used to generate gating signals, controlling the retention and discarding of information.

[0027] The tangent function (Hyperbolic Tangent) is a hyperbolic function that maps real numbers to the interval (-1, 1). In GRU networks, it is typically used to compute candidate hidden states to normalize the data range.

[0028] Argmax (Argument of the Maximum) is a mathematical operation used to find the independent variable or index corresponding to the maximum value in a function or vector. In the final output layer of classification or reinforcement learning decision models, Argmax is used to select the action class with the highest probability from the probability distribution vector.

[0029] An ISA (Instruction Set Architecture) is an interface specification that connects software and hardware, defining the types of operations, instruction formats, and addressing modes that a processor can execute. This application relates to a custom ISA that includes specific instructions optimized for GRU operations (such as vector Hadamard product, matrix multiplication, etc.).

[0030] DDR (Double Data Rate SDRAM) is a general-purpose off-chip high-capacity memory technology. In this application, it specifically refers to the external storage device commonly used in the prior art, but which is avoided in this design to reduce latency.

[0031] BRAM (Block Random Access Memory) is a dedicated memory block resource integrated inside an FPGA chip, featuring dual-port read / write and low latency.

[0032] Distributed RAM is a small-capacity memory built using FPGA logic units. Compared to BRAM, it is more flexible and suitable for building small, multi-port register files.

[0033] BF16 (Brain Floating Point 16) is a 16-bit floating-point format optimized for machine learning. It sacrifices mantissa precision to retain the same 8-bit exponent range as FP32. BF16 is more resource-efficient in hardware implementation than standard floating-point numbers, while maintaining high numerical stability in deep learning training and inference.

[0034] ILA (Integrated Logic Analyzer) is an in-circuit debugging tool. ILA allows users to perform in-system debugging on an FPGA by capturing waveforms of digital signals inside the FPGA in real time.

[0035] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0036] As mentioned earlier, firstly, general-purpose architectures are inefficient and have high latency when processing small batches of sequence data. GPUs rely on large batches of data to mask memory access latency, resulting in low utilization of computational units when processing single frames or small batches of real-time sequences required for reinforcement learning. Furthermore, frequent data transfers between the host and device lead to significant latency. CPUs, due to their serial processing mechanism and complex cache layers, are slow when performing large-scale matrix multiplications, failing to meet the millisecond-level real-time response requirements of edge computing. Secondly, existing FPGA accelerators are mostly optimized for CNNs, with core units primarily supporting matrix multiplication and addition operations. However, recurrent networks such as GRU / LSTM also include a large number of Hadamard products, vector addition and subtraction, and nonlinear activation functions (Sigmoid / Tanh). Due to the lack of dedicated hardware paths or instructions for these vector operators, intermediate results require frequent read / write operations to off-chip memory or intervention from external processors, disrupting the hardware pipeline and causing additional data transfer power consumption and wasted computation cycles. Thirdly, to support multiple network models, existing NPU designs often employ complex instruction set architectures and general scheduling logic. Implementing complex instruction decoders and dynamic schedulers on FPGAs consumes significant logic resources and encroaches on the area of ​​core arithmetic units. Meanwhile, for operators like GRU that are highly time-dependent, general scheduling strategies struggle to precisely control operations in each clock cycle, easily leading to pipeline bubbles. Fourth, limited by the on-chip storage capacity of FPGAs, existing accelerators mostly employ an architecture of off-chip storage, on-chip cache, and on-chip computation, with model weights stored in off-chip DDR. During inference, the hardware needs to frequently read weight data via an external bus, and the data transfer latency often exceeds the computation latency, resulting in a typical memory-constrained design that severely restricts system performance.

[0037] To address the aforementioned issues, this application provides an on-chip NPU for recurrent neural network reinforcement learning decision models implemented entirely on an FPGA. This NPU constructs a storage architecture entirely based on on-chip BRAM and distributed RAM within the FPGA, residing all model weight parameters and intermediate result vectors on-chip, fundamentally eliminating reliance on off-chip DDR access and overcoming the memory wall bottleneck. Simultaneously, by employing a custom-defined, reduced instruction set with fixed-length encoding and designing a general-purpose computing module including parallel multiply-accumulate units and dedicated nonlinear activation function calculation units, it directly supports Hadamard product, vector operations, and GRU-specific operators such as Sigmoid / Tanh at the hardware level. This allows gate updates and state fusion to be completed continuously within the same depth pipeline, avoiding pipeline interruptions and data transfer overhead caused by insufficient operator support, and significantly saving FPGA logic resources due to the simplified instruction set. Ultimately, this solution achieves non-blocking, high-speed data flow on-chip, reducing end-to-end inference latency to milliseconds or even microseconds, meeting the high real-time performance and low power consumption requirements of edge reinforcement learning decision systems.

[0038] It should be noted that the neural network processor method, system, product, device, and medium based on field-programmable gate arrays (FPGAs) provided in this application can be applied to artificial intelligence, integrated circuit design, reinforcement learning, domain-custom integrated circuits, or FPGAs. The above are merely examples and do not limit the application areas of the neural network processor method, system, product, device, and medium based on FPGAs provided in this application. Furthermore, the embodiments of this application may not limit the executing entity of the neural network processor based on FPGAs. For example, the neural network processor method based on FPGAs in the embodiments of this application can be applied to data processing devices such as terminal devices or servers. The terminal device can be an electronic device such as a computer or a personal digital assistant (PDA). The server can be a standalone server, a cloud server, or a cluster server composed of multiple servers.

[0039] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0040] The following embodiment illustrates a dedicated neural network processor (NPU) architecture based on an FPGA provided in this application. (See attached image.) Figure 1 , Figure 1 This application provides an embodiment of a neural network processor architecture based on a field-programmable gate array (FPGA), designed to efficiently accelerate reinforcement learning decision models based on recurrent neural networks (especially GRUs). This NPU is built entirely using on-chip FPGA resources, such as BRAM, distributed RAM, and DSP units, without relying on off-chip DDR memory, thus eliminating the high latency and bandwidth bottlenecks caused by external memory access in traditional architectures.

[0041] The main controller is the brain of the NPU. As the starting point of the entire NPU, the main controller generates control signals by parsing fixed-length encoded instructions. It sends computation instructions to the general-purpose computing modules and address and read / write control signals to the on-chip block memory array and vector register file, thereby initiating the entire data flow process. A custom fixed-length instruction set architecture (ISA) is used, where each instruction includes an opcode, output selection, matrix / vector dimension, BRAM base address pointing to the weight storage location, and source / destination address in the vector register file. This streamlined instruction design allows the controller to efficiently parse and schedule the data flow, control the inference order of each layer of the model, and issue instructions to the data router to drive the general-purpose computing modules to complete the computation of different operators. Because the instruction format is fixed and the decoding logic is simple, FPGA logic resources are saved while ensuring efficient pipeline operation. The opcode indicates the type of operation, such as matrix multiplication or activation function.

[0042] The general-purpose computing module is the computational core of the NPU, responsible for performing all mathematical operations in model inference. In other words, the general-purpose computing module is the data flow aggregation point. It simultaneously retrieves weight parameters from the on-chip block memory array, obtains input vectors or intermediate results from the vector register file, and performs all computational operations such as matrix multiplication, vector operations, and nonlinear activations within the module. New intermediate results generated by the computation are written back to the vector register file, while the final output vector (policy vector) is sent to the action value sampling unit.

[0043] It contains 512 parallel multiply-accumulate MAC units, capable of simultaneously handling large-scale matrix multiplication and vector dot products, which is crucial for processing the multiplication of weight matrices and input vectors in GRU. Furthermore, the module integrates nonlinear activation function computation units, such as ReLU, and directly supports vector-level operations and activation functions like Sigmoid and Tanh through hardware, as well as nonlinear activation functions such as Hadamard product and vector addition and subtraction. Traditional accelerators typically only optimize MAC operations, lacking sufficient support for gating operations unique to GRU, often requiring multiple read / write operations to off-chip memory or CPU intervention, leading to pipeline interruptions, such as update and reset gates. This design, by hardware-enabling these operators, allows the entire gating update and state fusion process of GRU to be completed continuously in the same pipeline without interruption, significantly improving computational efficiency. The computational precision can be selected as FP32 or BF16. The BF16 format maintains the same exponent range as FP32 while reducing mantissa bits, saving hardware resources and making it suitable for edge deployment.

[0044] The BRAM array serves as the weight repository for the NPU, entirely composed of on-chip block RAM (BRAM) on the FPGA. It statically stores all weight matrices and bias parameters of the neural network. The array consists of 32 parallel BRAM instances using a unified address mapping, supporting the simultaneous reading of 32 rows of data and achieving extremely high memory access bandwidth. To optimize memory access efficiency, weights and biases are stored using a block-interleaved storage method: weight matrices of the same layer are divided into rows and interleaved with their corresponding biases; parameters from different layers are arranged in a depth-stack manner. This layout ensures strong addressing regularity, allowing the controller to quickly calculate the base address based on layer and row indices. This enables sequential and continuous parameter reading, avoids complex address calculations, and ensures close coordination between data supply and computation units.

[0045] The Vector Register File (VRF) is the intermediate result storage area of ​​the NPU, constructed from the FPGA's distributed memory. It stores intermediate vectors generated during operator computation, such as gating signals and candidate hidden states. It uses the same address mapping logic as the BRAM array, facilitating unified management. The VRF contains 32 memory instances, each with an address width and depth that can be dynamically adjusted according to model requirements. For example, if the model's intermediate vector length is 128, it can be configured as 128×32-bit, achieving flexible resource utilization. This design avoids writing intermediate results back to off-chip memory; all data flows within the on-chip cache, significantly reducing data transfer overhead.

[0046] In practical applications, the on-chip block memory array serves as a static parameter source. Based on the address issued by the main controller, it reads the weight matrix and bias parameters required by the current computation layer and directly sends them to the general-purpose computing module. The vector register file serves as a dynamic data temporary storage. On the one hand, it receives and stores the intermediate result vectors (such as gating states) produced by the general-purpose computing module in the previous calculation step. On the other hand, in subsequent calculation steps, it reads these intermediate results again and feeds them back into the general-purpose computing module for the next round of computation.

[0047] The action value sampling unit is responsible for converting the policy vector output by the model into specific discrete actions. It consists of multiple sets of programmable comparator trees, which can be dynamically configured according to the number of model output heads (e.g., multiple policy heads) to perform parallel argmax operations to generate action indices. Subsequently, the policy vector and action indices are pushed together into the output result buffer. The comparators support FP32 or BF16 formats and are easily extended to other precisions. Through hardware-level parallel comparison, multiple actions can be sampled within a single clock cycle, meeting the real-time decision-making requirements of reinforcement learning.

[0048] The output result buffer temporarily stores the inference results of the current time step, including the complete policy vector and the action index obtained by the action value sampling unit. After the calculation of the current time step is completed, it sends a valid representation result signal to the external system and maintains the stability of the output data. This state is maintained until the input vector of the next time step is valid, waiting for the main controller to start the next round of inference. This achieves asynchronous handshake with the external system, ensuring reliable data transmission.

[0049] All weights and intermediate results are stored on the FPGA chip, completely eliminating off-chip DDR access latency and solving the memory wall problem. For operators specific to recurrent neural networks such as GRU, vector operations and nonlinear activations are directly supported at the hardware level, achieving a seamless deep pipeline and significantly improving computational efficiency. The combination of 512 parallel MAC units and 32-way parallel BRAM reads, supplemented by a reduced instruction set, enables end-to-end inference latency to reach millisecond or even microsecond levels, with power consumption at only 10W, making it ideal for high real-time reinforcement learning applications at the edge. Through a configurable vector register file and programmable comparator tree, this architecture can adapt to reinforcement learning models of different sizes and with different numbers of output heads, exhibiting good versatility and scalability.

[0050] See Figure 2 , Figure 2 This is a schematic diagram of the data flow and module collaboration relationship within the NPU provided in this application embodiment. The entire system takes the main controller as the scheduling core, drives the data router through micro-instructions, and coordinates the orderly work of various functional units.

[0051] All weights and bias parameters are uniformly stored in a BRAM array consisting of 32 rows of BRAM. After being addressed by an address decoder, the data is sent to the general-purpose computing module. The core of this module is a large-scale multiply-accumulate unit array; the diagram exemplifies this by showing 31 parallel units responsible for performing core operations such as matrix multiplication. Intermediate results during the calculation process are temporarily stored in a vector register file constructed from LUT RAM. Its 32-row structure corresponds to the BRAM array, facilitating address mapping and management. Simultaneously, the sig / tanh coefficient generator provides hardware support for the nonlinear activation function.

[0052] After computation, the data flows to the action value sampling unit, where the comparator tree, under the scheduling of the sampling controller, performs the Argmax maximum value index operation on the policy vector. Finally, the inference result, including the policy vector and action index, is stored in the output result cache, managed by the output control unit, and output externally through the policy vector cache. The entire process achieves a non-blocking flow of data from on-chip storage to the computation unit and then to the result output, completely avoiding dependence on external DDR access.

[0053] This application communicates directly with external systems to receive input vectors at different time steps. After running a complete end-to-end inference process on-chip using an agent model based on a major variant of RNN, it outputs the inference result for the current time step. This approach does not rely on complex processing systems or CPU cores, is scalable to bi-vector operations and other matrix-vector multiplication operators, and has some general applicability to reinforcement learning models with similar structures to those described in the embodiments. By fully utilizing on-chip storage and computing resources, cleverly balancing memory access and computational overhead, and rationally arranging parallelism, this application achieves end-to-end inference latency of 1ms and power consumption of 10W for a 3M parameter model through a deep multi-stage pipeline, significantly improving the feasibility of deploying reinforcement learning decision-making models on the edge.

[0054] like Figure 3This document outlines the specific mapping rules between the weight matrix and bias vector in the on-chip BRAM array of the FPGA. As shown in the figure, for a 512×512 weight matrix and a 512-bit bias vector, this design uses interleaved storage and row-block depth stacking to evenly distribute them across 32 BRAM instances. Specifically, the weight matrix is ​​divided into 32 row blocks, Group 0 to Group 31, each containing 16 consecutive rows of data. For example, Group 0 contains row 0 to row 15, Group 1 contains row 16 to row 31, and so on. Within each BRAM instance, an interleaved storage format of "bias + weight row" is used: the first storage unit of each row stores the bias value for that row, and the subsequent 512 storage units store the 512 weight values ​​for that row. For example, BRAM 0 stores B0, W0, and W0, respectively. 511 (row0), then store B 16 W 16 ,0~W 16 , 511 (row16), and B 32 W 32 ,0~W 32 , 511 (row32), until the end of the row block covered by the BRAM.

[0055] Thirty-two BRAM instances operate in parallel, collectively storing the complete 512×512 weight matrix and 512-dimensional bias vector: BRAM 0 stores rows 0, 16, 32..., BRAM 1 stores rows 1, 17, 33..., and so on, with BRAM 31 storing rows 15, 31, 47... This mapping method allows the controller to quickly calculate the physical location of specific parameters in the BRAM based only on the base address, layer index, and row index of the current computing layer. It achieves highly systematic and easily hardware-implemented sequential addressing, while simultaneously supporting the parallel reading of 32 rows of data from 32 BRAMs, providing high-bandwidth parameter supply for general-purpose computing modules.

[0056] See Figure 4 , Figure 4 This is a schematic diagram of the specific network structure of the reinforcement learning agent model implemented in the embodiments of this application. As shown in the figure, the model adopts the classic encoder-aggregator-decoder architecture, which is specifically designed to handle reinforcement learning decision-making tasks based on recurrent neural networks. The encoder part consists of multiple cascaded linear layers (fully connected layers), which are responsible for feature extraction and dimensionality transformation of the original input vector, mapping the high-dimensional input into a latent space representation suitable for temporal processing.

[0057] The aggregator consists of two core components: a gated recurrent unit (GRU), which can be replaced with other RNN variants such as LSTM in the embodiment, and a fully connected layer. The GRU is responsible for temporal modeling of the feature sequence output by the encoder. It captures long-term dependencies in the sequence through its internal gating mechanism of update gates and reset gates, and outputs a hidden state vector. This hidden state vector is then passed through a fully connected layer for further nonlinear transformation, providing rich state representations for subsequent decisions.

[0058] The decoder consists of multiple parallel policy heads and a max-sampling layer. Each policy head is typically a linear layer responsible for mapping the hidden state vectors output by the aggregator to policy vectors of a specific action dimension, i.e., probability distributions. Multiple parallel policy heads allow the model to output multiple action heads simultaneously, for example, in continuous control tasks with multiple action dimensions, enabling multi-task decision-making. Finally, the policy vectors output by each policy head are fed into the max-sampling layer Argmax, which performs a maximum index operation on each policy vector to obtain the final discrete action index value as the model's output.

[0059] The entire model starts with the input vector at the current time step, and through end-to-end forward inference via the encoder, aggregator, and decoder, it finally outputs the policy vector and corresponding sampling action index for the current time step, providing real-time decision-making support for the reinforcement learning agent. This model structure is precisely the core acceleration target of the NPU architecture in this application.

[0060] See Figure 5 , Figure 5 This is a complete flowchart of the NPU implementation on an FPGA according to an embodiment of this application, which is divided into two stages: preparation and testing.

[0061] In the preparation phase, the model needs to be analyzed first: run the model parameter processing script to export the network structure, weight matrix, and bias parameters from the trained reinforcement learning agent model. Then, use the Vivado 2025.1 development tool to call the Xilinx IP core and adjust the IP settings according to design requirements, such as clock management, block memory, and floating-point units, to implement the functions of each NPU module. After completing the IP configuration, initialization is performed: import the exported weights and bias parameters into the BRAM initialization file (.mem or .coe) so that the FPGA can automatically load the model parameters after power-on. Finally, edit the pin constraint file (XDC) to bind the I / O ports in the design to the actual physical pins on the Virtex-7 series XC7VX690T-2FFG1761C development board, ensuring correct signal input and output.

[0062] After completing the preparation work, the testing process begins. First, synthesis is performed, converting the RTL code into a gate-level netlist and generating a circuit schematic for the designer to check the logic's correctness. After synthesis, the ILA (Integrated Logic Analyzer) IP core is inserted, and the internal signals to be observed (such as state machine states, intermediate calculation results, output results, etc.) are selected for subsequent debugging. Next, implementation is performed, including place and route operations, mapping the netlist to the actual logic resources of the FPGA, and generating a timing report for the designer to evaluate whether the timing meets the requirements. After implementation, a bitstream file (.bit) is generated, containing complete FPGA configuration information. Finally, the bitstream file is downloaded to the target development board for off-board verification. By observing the signal waveforms captured by the ILA or the LED indicator status on the development board, it is determined whether the hardware inference results are consistent with the software simulation results, thereby verifying the correctness of the NPU design.

[0063] This embodiment uses the Virtex-7 series XC7VX690T-2FFG1761C FPGA as the target platform and relies on the Vivado 2025.1 toolchain to complete the entire process from model analysis to hardware verification through the above process.

[0064] The specific IP configuration and implementation process in the Vivado 2025.1 development environment in this embodiment is as follows: In terms of IP core calling and parameter settings, this design mainly uses four types of Xilinx IP cores. ClockingWizard is used to generate the system clock: using the MMCM primitive, it takes a 200MHz board-level clock as input, multiplies / divides it, and outputs a 100MHz operating clock, providing a stable timing foundation for the entire NPU. Block Memory Generator is used to build the BRAM array: configured as a single-port ROM mode with a Native interface, a data width of 66 bits, and a depth of 256. The enable port is controlled via the ENA pin, and the Primitives output register is enabled to optimize timing; simultaneously, the initialization file is loaded, pre-storing the model weight parameters in the ROM. The Xilinx Floating Point IP core implements the floating-point arithmetic unit: the operation mode is FusedMultiply-Add, supporting addition / subtraction and multiply-add operations simultaneously; the input precision is Single precision (8-bit exponent, 24-bit mantissa); the DSP Slice usage strategy is set to medium utilization to balance resources and performance; the interface uses NonBlocking flow control, with a latency of 16 clock cycles, completing one operation per cycle, and enabling the ACLKEN and ARESETn control signals. The Distributed Memory Generator is used to construct the vector register file: configured as a dual-port RAM with a depth of 256 and a data width of 16, with both input and dual-port addresses using non-registered methods, output options enabled by registers and sharing a clock, and pipeline stages set to 0 to achieve low-latency intermediate result caching.

[0065] During the initialization and constraint phase, the model weight parameters are loaded into the BRAM in .mem format, and the model instructions are loaded into the instruction memory in .coe format. Subsequently, the I / O ports in the design are bound to the actual physical pins of the Virtex-7 development board through the XDC constraint file.

[0066] After completing the above preparations, proceed to the onboard testing process. First, perform synthesis to convert the RTL code into a gate-level netlist and optimize the logic. Check the synthesis report to avoid optimizing out critical logic. After synthesis, open the schematic, insert the ILA core, and select the signals to be observed, such as state machines or intermediate calculation results. Save the file; the constraint file will update automatically. Next, perform implementation placement and routing, generating a timing report for timing convergence analysis. Finally, generate a bitstream file and download it to the development board. Verify the consistency between the hardware inference results and the software model using ILA waveforms or onboard LED indicators.

[0067] This application also provides corresponding devices and computer storage media for implementing the neural network processor solution based on field-programmable gate arrays provided in this application.

[0068] The device includes a memory and a processor. The memory is used to store instructions or code, and the processor is used to execute the instructions or code to enable the device to perform the neural network processor method based on field-programmable gate arrays as described in any embodiment of this application.

[0069] The computer storage medium stores code, and when the code is executed, the device running the code implements the neural network processor method based on field-programmable gate arrays as described in any embodiment of this application.

[0070] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0071] It should be understood that in this application, "at least one" refers to one or more items, and "more" refers to two or more items. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one" or similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, "at least one" of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0072] It should be understood that the terms center, longitudinal, transverse, up, down, front, back, left, right, vertical, horizontal, top, bottom, inside, outside, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0073] It should be noted that, unless otherwise explicitly specified and limited, the terms installation, connection, and linking should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0074] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the statement "including a…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0076] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A neural network processor based on a field-programmable gate array (FPGA), characterized in that, include: The main controller is used to parse instructions and control the data flow; The on-chip block memory array, composed of on-chip block memory of the field-programmable gate array (FPGA), is used to store the weight matrix and bias parameters of the neural network model. The vector register file, consisting of distributed memory on the FPGA chip, is used to store intermediate result vectors during the computation process. A general computing module is connected to the main controller, the on-chip block memory array, and the vector register file, respectively. It is used to read parameters from the on-chip block memory array according to the instructions of the main controller, read intermediate result vectors from the vector register file, perform the calculation operations required for model inference, write the calculated intermediate result vectors back to the vector register file, and send the final output policy vector to the action value sampling unit. An action value sampling unit, connected to the general computing module, is used to sample the maximum value index of the output policy vector to obtain the action index; The output result cache is connected to the action value sampling unit and is used to store the action index and strategy vector of the current time step.

2. The neural network processor according to claim 1, characterized in that, The main controller adopts a custom instruction set architecture with fixed-length encoding. The instruction fields of the custom instruction set include opcode, output selection, matrix dimension definition, vector dimension definition, on-chip block memory base address, and vector cache source or destination address.

3. The neural network processor according to claim 1, characterized in that, The computational operations include matrix multiplication, vector-level operations, and nonlinear activation function operations.

4. The neural network processor according to claim 3, characterized in that, The general computing module includes: multiple parallel multiply-accumulate computing units for performing matrix multiplication and vector dot product; and at least one nonlinear activation function computing unit for performing nonlinear activation function operations.

5. The neural network processor according to claim 1, characterized in that, In the on-chip block memory array, the weight matrix of the same neural network layer is stored interleaved with the corresponding bias vector after being divided into rows. The parameters of different layers are arranged sequentially in a deep stack manner so that the controller can achieve sequential addressing and reading based on the layer index and the row index.

6. The neural network processor according to claim 1, characterized in that, The vector register file consists of multiple distributed memory instances, and the address width and address depth of the distributed memory instances are dynamically adjusted according to the length of the intermediate result vector to be cached.

7. The neural network processor according to claim 1, characterized in that, The action value sampling unit consists of multiple programmable comparator trees, which are dynamically configured according to the number of action heads output by the model.

8. The neural network processor according to claim 7, characterized in that, The comparison data format of the action value sampling unit includes FP32 or BF16.

9. The neural network processor according to claim 1, characterized in that, The general computing module continuously performs calculations on the reset gate, update gate, and candidate hidden state of the gated loop unit in the same pipeline.

10. The neural network processor according to claim 1, characterized in that, The on-chip block memory array consists of multiple on-chip block memory instances, adopts a unified address space and interleaved storage method, and stacks the weight matrix and bias vector according to the row block depth, supporting the reading of multiple rows of data at one time.

Images