Binary Convolution Compression for Low-Power CPU Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require significant computing resources and high throughput, making them unsuitable for devices with low power consumption and computation capability, such as embedded devices, due to their large number of model parameters and high computation costs.
Innovation Solution
The method involves binarizing and bit-packing input data and convolution kernels of a convolution layer along the channel direction to compress them, dividing the compressed data into blocks, and performing convolutional computations on these blocks, which reduces computation costs and improves speed by using unsigned integer operations, suitable for CPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks with large number of model parameters are used, then classification accuracy and detection performance are improved, but computing resource consumption and power consumption increase significantly
Solution Approach 1:
The patent changes the parameter representation from high-precision floating-point numbers to low-precision binary values (±1). By binarizing the weight parameters and input data, the network maintains its computational functionality while dramatically reducing the precision requirements, which enables efficient implementation on low-power devices without significantly compromising classification accuracy.
Solution Approach 2:
The patent extracts and removes unnecessary computational complexity by eliminating multiplication and addition operations. Instead of performing full floating-point arithmetic, the method extracts only the essential binary comparison operations, replacing complex mathematical computations with simple threshold-based decisions that consume minimal power.
2Measurement precision
If deep neural networks with large number of model parameters are used, then detection performance is improved, but computation capability requirements increase
Solution Approach 1:
The patent transforms the computational parameters from complex floating-point multiplications to simple binary comparisons. By representing weights and inputs as binary values (±1), the network performs element-wise comparisons instead of multiplications, dramatically simplifying the computational capability requirements while preserving detection performance.
Solution Approach 2:
The patent substitutes complex mechanical computation (floating-point multiplication and addition units) with simpler logical operations (binary comparison and accumulation). This replacement eliminates the need for sophisticated arithmetic logic units, enabling deployment on devices with limited computation capability such as embedded systems and mobile processors.
3Measurement precision
If traditional convolution operations are performed, then accurate feature extraction is achieved, but computation time and memory usage increase
Solution Approach 1:
The patent changes the data representation parameters to binary format, which enables parallel processing of multiple data elements simultaneously. By packing binary values into compact representations and performing vectorized operations, the method reduces computation time while maintaining feature extraction accuracy through preserved spatial and channel relationships.
Solution Approach 2:
The patent merges multiple computational operations into unified binary comparison and accumulation steps. By combining the weight application, bias addition, and activation functions into streamlined operations that work directly with binary values, the method reduces the total number of computational steps required for feature extraction while maintaining accuracy.
Data Source
AI summary
The embodiments of this application provide a method and device for optimizing neural network. The method includes: binarizing and bit-packing input data of a convolution layer along a channel direction, and obtaining compressed input data; binarizing and bit-packing respectively each convolution kernel of the convolution layer along the channel direction, and obtaining each corresponding compressed convolution kernel; dividing the compressed input data sequentially in a convolutional computation order into blocks of the compressed input data with the same size of each compressed convolution kernel, wherein the data input to one time convolutional computation form a data block; and, taking a convolutional computation on each block of the compressed input data and each compressed convolution kernel sequentially, obtaining each convolutional result data, and obtaining multiple output data of the convolution layer according to each convolutional result data.