Binary Weight Convolutional Neural Network Processing System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning technologies, particularly convolutional neural networks, face challenges with high resource occupation, slow operational speed, and high energy consumption, making them difficult to apply in embedded devices like mobile phones and embedded electronic devices due to low energy efficiency.
Innovation Solution
A processing system and method for a binary weight convolutional neural network that reduces weight data bit width to single bits, replacing multiplication and addition operations with basic addition and subtraction, using a processor with a storage-control-calculation structure, including a convolution unit, accumulator, and adder, to improve operational speed and energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If binary weight values (1 or -1) are used instead of multi-bit weights, then parameter capacity is reduced and operational speed is increased, but calculation accuracy may be degraded
Solution Approach 1:
The patent changes the parameter of weight representation from multi-bit floating-point or integer values to binary values (1 or -1). This parameter change reduces the computational complexity from multiplication to simple addition/subtraction operations, thereby increasing operational speed while maintaining acceptable calculation accuracy through the binary constraint
2Adaptability or versatility
If deep neural network structures with large numbers of data nodes are used, then model capability is improved, but resource occupation increases and energy consumption rises
Solution Approach 1:
The patent extracts and removes the multiplication operation from the neural network computation, retaining only addition and subtraction operations. This extraction eliminates the most energy-intensive computational step while preserving the fundamental neural network structure and its capability to process complex patterns through binary weight convolutions
3Adaptability or versatility
If conventional processors (CPU or GPU) are used for deep learning, then general-purpose computing capability is maintained, but energy efficiency is low and operational speed is bottlenecked
Solution Approach 1:
The patent substitutes the conventional mechanical/computational approach of multiplication-based neural network operations with a simplified addition/subtraction-based binary operation system. This substitution fundamentally changes the computational mechanism from general-purpose floating-point arithmetic to specialized binary arithmetic, achieving superior energy efficiency and speed for neural network workloads
Data Source
AI summary
The present invention provides a processing system for a binary weight convolutional neural network. The system comprises: at least one storage unit for storing data and instructions; at least one control unit for acquiring the instructions stored in the storage unit and sending out a control signal; and, at least one calculation unit for acquiring, from the storage unit, node values of a layer in a convolutional neural network and corresponding binary weight value data and obtaining node values of a next layer by performing addition and subtraction operations. With the system of the present invention, the data bit width during the calculation process of a convolutional neural network is reduced, the convolutional operation speed is improved, and the storage capacity and operational energy consumption are reduced.