Slim Neural Network Model With Binary XNOR Convolution Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face challenges with increased model size and operation time due to deepening and widening for high performance, leading to inefficiencies in processing large datasets.
Innovation Solution
A neural network structure is designed to be slim and lightweight, utilizing a basic block comprising a batch normalization layer, quantization layer, convolution layer, and activation layer, driven by operations including sign function, bitwise convolution, and residual connections, with components like a comparator, XNOR gate, and MAC operator for efficient data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a convolutional neural network is deepened and widened to be trained with more data and achieve high performance, then the performance of the neural network is improved, but the size of the model and operation time increase
Solution Approach 1:
The patent applies parameter changes by transitioning from standard floating-point arithmetic to binary arithmetic (0 and 1) through quantization layers. This changes the representation parameters of weights and activations to reduce model size while maintaining computational functionality. The binary quantization reduces the number of bits required to store and process data, directly addressing the model size issue while preserving the network's ability to learn from data.
Solution Approach 2:
The patent replaces traditional mechanical arithmetic operations with bitwise operations. Instead of using conventional multiply-and-accumulate operations, the system uses XOR, XNOR, and other bitwise operations to perform convolution. This substitution reduces computational complexity and operation time while maintaining the essential function of the neural network layers.
2Reliability
If a convolutional neural network is deepened and widened to be trained with more data and achieve high performance, then the performance of the neural network is improved, but the operation time increases
Solution Approach 1:
The patent replaces traditional mechanical arithmetic operations with bitwise operations. Instead of using conventional multiply-and-accumulate operations, the system uses XOR, XNOR, and other bitwise operations to perform convolution. This substitution reduces computational complexity and operation time while maintaining the essential function of the neural network layers.
Solution Approach 2:
The patent applies parameter changes by transitioning from standard floating-point arithmetic to binary arithmetic (0 and 1) through quantization layers. This changes the representation parameters of weights and activations to reduce model size while maintaining computational functionality. The binary quantization reduces the number of bits required to store and process data, directly addressing the model size issue while preserving the network's ability to learn from data.
3Device complexity
If a neural network structure is designed to be slim and lightweight, then the model size is reduced, but the computational operations become more complex
Solution Approach 1:
The patent replaces traditional mechanical arithmetic operations with bitwise operations. Instead of using conventional multiply-and-accumulate operations, the system uses XOR, XNOR, and other bitwise operations to perform convolution. This substitution reduces computational complexity and operation time while maintaining the essential function of the neural network layers.
Solution Approach 2:
The patent merges multiple operations into unified computational blocks. The quantization layer combines normalization and quantization operations, while the convolution layer integrates bitwise convolution with activation functions. This merging reduces the number of separate computational steps and simplifies the overall architecture, making the slim network easier to implement despite using complex bitwise operations.
Data Source
AI summary
A device includes: a comparator; an exclusive-NOR (XNOR) gate; an accumulator; and a multiplication and accumulation (MAC) operator, wherein a basic block of a neural network comprises a first batch normalization layer, a quantization layer, a convolution layer, an active layer, and a second batch normalization layer, and wherein the basic block is driven by the device by a combination of a first batch normalization operation, a sign function operation, a bitwise convolution operation, an activation function operation, a second batch normalization operation, and a residual connection operation.