Binary Convolutional Neural Network Filters for FPGA Implementation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep neural networks (DNNs) face high computational costs due to precision requirements, leading to significant performance drops when reducing data representation precision, making it difficult to implement them on low-cost and high-speed devices like FPGAs.
Innovation Solution
The method approximates real-valued activation functions and filters using binary activations and filters, converting MAC operations into binary operations, allowing for high precision low bit convolutional neural networks (CNNs) that maintain performance while reducing computational costs, enabling implementation on FPGAs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If precision requirements are reduced to lower computational costs, then device implementation becomes feasible, but performance drops significantly
Solution Approach 1:
The patent changes the precision parameter from conventional high precision (32-bit floating point) to low precision (binary 1-bit) representation. This parameter change enables implementation on resource-constrained devices like FPGAs while maintaining acceptable performance through the binary neural network architecture and quantization techniques described in the patent.
Solution Approach 2:
The patent uses binary filters and binary activations instead of high-precision floating-point representations. These binary components are computationally cheaper and can be implemented using simple logic operations rather than complex arithmetic units, making the system feasible for deployment on low-cost embedded devices.
2Measurement precision
If high precision operations are performed, then accuracy is maintained, but computational cost increases significantly
Solution Approach 1:
The patent replaces complex mechanical arithmetic operations (multiplication and accumulation of floating-point numbers) with simpler logical operations (XNOR and population count operations on binary data). This substitution dramatically reduces computational cost while maintaining the essential functionality of convolution operations in neural networks.
Solution Approach 2:
The patent changes the data representation parameter from high-precision floating-point to low-precision binary format. This parameter change reduces the computational complexity of each operation from O(1) floating-point arithmetic to O(1) logical operations, significantly lowering energy consumption and computational cost.
3Quantity of substance
If data representation precision is reduced, then device resource requirements decrease, but performance drops
Solution Approach 1:
The patent changes the data representation parameter from conventional high-precision formats to low-precision binary format, reducing the quantity of data required to represent filters and activations. This parameter change enables efficient storage and processing on devices with limited memory resources while maintaining performance through the binary neural network framework.
Data Source
AI summary
Described herein are systems, methods, and computer-readable media for generating and training a high precision low bit convolutional neural network (CNN). A filter of each convolutional layer of the CNN is approximated using one or more binary filters and a real-valued activation function is approximated using a linear combination of binary activations. More specifically, a non-1×1 filter (e.g., a k×k filter, where k>1) is approximated using a scaled binary filter and a 1×1 filter is approximated using a linear combination of binary filters. Thus, a different strategy is employed for approximating different weights (e.g., 1×1 filter vs. a non-1×1 filter). In this manner, convolutions performed in convolutional layer(s) of the high precision low bit CNN become binary convolutions that yield a lower computational cost while still maintaining a high performance (e.g., a high accuracy).