FPGA Neural Network Accelerator With Custom Instructions and DMA Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing FPGA-based AI inference solutions are inflexible and resource-intensive due to fixed AI accelerator IP cores that do not support all neural network layers or operations, leading to inefficiencies in performance and time-to-market, especially with rapid advancements in AI model topologies and layers.
Innovation Solution
A neural network accelerator architecture in an FPGA that utilizes a custom instruction interface and AXI master interface for DMA, allowing seamless integration between software and hardware accelerators, with a scalable design that enables configuration of specific layer accelerators and efficient resource utilization through a command control block, response control block, and AXI control block.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed AI accelerator IP cores are used in FPGA, then hardware acceleration performance is improved, but flexibility and adaptability to new AI models deteriorate
Solution Approach 1:
The patent implements a dynamic layer accelerator architecture where the FPGA fabric can be reconfigured to support different neural network layer types. The system uses a base accelerator design that can be dynamically extended or modified through partial reconfiguration, allowing the same hardware platform to adapt to new AI models and layer operations as they are developed, while maintaining high inference performance through hardware acceleration.
Solution Approach 2:
The patent creates a universal accelerator core that can handle multiple types of neural network layers through a standardized interface and configurable computation units. The design incorporates generic tensor operation engines that can be programmed to perform different layer operations (convolution, fully connected, pooling, etc.) by loading different weight sets and configuration parameters, making the hardware applicable to various AI models without requiring dedicated custom hardware for each layer type.
2Adaptability or versatility
If all possible layer accelerators are implemented in FPGA, then adaptability to various neural network models is improved, but resource consumption and device complexity worsen
Solution Approach 1:
The patent divides the accelerator into modular functional blocks including separate computation units, memory interfaces, and control logic that can be independently configured. This segmentation allows the system to instantiate only the specific layer accelerator modules needed for a given AI model, rather than implementing all possible layer types simultaneously. Unused modules can be disabled or removed from the configuration, optimizing resource utilization while maintaining the capability to support various neural network layers.
3Productivity
If custom AI accelerator IP cores are used, then inference performance is improved, but development time and time-to-market worsen due to long design cycles
Solution Approach 1:
The patent employs a pre-validated base accelerator IP core design that has been previously tested and optimized for high-performance neural network inference. This preliminary preparation of a robust core architecture allows rapid deployment of new AI models by simply configuring the existing core with model-specific parameters and weights, rather than undergoing lengthy design and validation cycles for each new implementation, thus reducing time-to-market while maintaining performance.
Data Source
AI summary
The present invention relates to neural network accelerator (103) in a field programmable gate array (FPGA) which is based on custom instruction interface of an embedded processor in said FPGA and AXI master interface for DMA purposes, wherein said neural network accelerator (103) comprises of a command control block (301), at least one neural network layer accelerator (303), a response control block (305) and an AXI control block (307). The amount of neural network layer accelerators (103) that can be implemented can be configured easily (such as adding a new type of layer accelerator (303) to said neural network layer accelerator (103)) in said FPGA, which makes said invention flexible and scalable.