Binarized Neural Network Processing Unit Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural networks, particularly convolutional neural networks, require a large number of floating-point parameters, leading to high memory demands and energy consumption, making them inefficient for small computing systems, and existing studies on low precision networks like binarized neural networks (BNNs) lack practical hardware implementations.
Innovation Solution
A hardware architecture for implementing a binarized neural network (BNN) is proposed, featuring a streaming architecture with a neural processing unit that includes blocks for convolution, batch-normalization, and max-pooling, utilizing XNOR logic gates and pop-count operations to reduce memory and power consumption, and incorporating a line-buffer and memory to store binarized parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional neural networks with floating-point parameters are used, then high accuracy can be achieved, but memory usage and power consumption increase significantly
Solution Approach 1:
The patent applies parameter changes by transitioning from floating-point parameters to binarized parameters (0 and 1). This fundamental change in parameter representation reduces memory bandwidth requirements and enables the use of simpler logic operations (XNOR and pop-count) instead of complex floating-point arithmetic, thereby significantly reducing power consumption while maintaining acceptable accuracy through techniques like batch normalization and knowledge distillation
Solution Approach 2:
The patent substitutes complex mechanical/computational systems with simpler alternatives. Specifically, it replaces floating-point multiplication and accumulation units with XNOR logic gates and pop-count circuits. This substitution leverages the properties of binarized data to use less complex hardware operations that consume fewer transistors and less power, while still achieving the necessary computational functionality for neural network inference
2Measurement precision
If conventional neural networks with floating-point parameters are used, then high accuracy can be achieved, but memory usage increases significantly
Solution Approach 1:
The patent changes the parameter representation from floating-point to binary format. This parameter transformation reduces the bit-width required to store and transmit weights and activations, thereby reducing memory bandwidth requirements and on-chip buffer sizes. The binarized parameters can be stored more compactly and transferred between memory and processing units with lower bandwidth requirements
Solution Approach 2:
The patent segments the neural network computation into distinct phases that can be processed efficiently with binarized data. By dividing the computation into convolution phases (using XNOR) and accumulation phases (using pop-count), the system can optimize memory usage for each phase separately, reducing peak memory bandwidth requirements compared to conventional approaches that process all data at full precision
3Use of energy by moving object
If binarized neural networks are implemented, then memory usage and power consumption are reduced, but hardware implementation complexity increases
Solution Approach 1:
The patent replaces complex floating-point arithmetic hardware with simpler logic-based hardware. By substituting multiply-accumulate units with XNOR gates followed by pop-count circuits, the design reduces transistor count and circuit complexity. The XNOR operation can be implemented with standard logic gates, and the pop-count can be achieved through parallel addition trees, both of which are simpler than floating-point units
Solution Approach 2:
The patent creates a universal binarized neural network processing architecture that can handle different network configurations and layers using the same basic building blocks. The XNOR-popcount pipeline can be replicated and configured for different convolution, fully-connected, and pooling operations, reducing the need for specialized hardware for each operation type and simplifying overall system design
4Quantity of substance
If binarized neural networks are implemented, then memory usage is reduced, but hardware implementation complexity increases
Solution Approach 1:
The patent changes data representation from floating-point to binary, which fundamentally alters memory requirements. Binarized weights and activations can be stored in compact binary formats rather than standard floating-point formats, reducing memory bandwidth requirements and on-chip buffer sizes. This parameter transformation enables more efficient memory hierarchies and reduces the need for large on-chip memory structures
Data Source
AI summary
A neural processing unit of a binarized neural network (BNN) as a hardware accelerator is provided, for the purpose of reducing hardware resource demand and electricity consumption while maintaining acceptable output precision. The neural processing unit may include: a first block configured to perform convolution by using a binarized feature map with a binarized weight; and a second block configured to perform batch-normalization on an output of the first block. A register having a particular size may be disposed between the first block and the second block. Each of the first block and the second block may include one or more processing engines. The one or more processing engines may be connected in a form of pipeline.