Neural Network Computing Using Shift Registers for Powers of 2
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) experience low computing speed due to a large number of layers and connection weights, which require numerous floating-point multiplication operations, leading to inefficient processing.
Innovation Solution
Implementing a neural network computing method that uses shift operations instead of multiplication operations for connection weights that are powers of 2, leveraging shift registers to increase processing speed and reduce storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point multiplication operations are used for connection weights, then computing accuracy is maintained, but computing speed decreases
Solution Approach 1:
The patent changes the parameter representation of connection weights from general floating-point numbers to powers of 2 (exponential form). This parameter transformation allows multiplication operations to be replaced with shift operations, significantly improving computing speed while maintaining sufficient accuracy for neural network computations.
2Reliability
If more connection weights are used to increase network complexity, then model performance improves, but computing time increases
Solution Approach 1:
By representing connection weights in exponential form (powers of 2), the patent enables parallel processing and optimization of multiplication operations. This allows complex models with more connection weights to be computed more efficiently, reducing the time penalty associated with increased model complexity.
3Measurement precision
If 32-bit or 64-bit floating-point numbers are used for connection weights, then precision is maintained, but storage space increases
Solution Approach 1:
The patent transforms connection weight representation from standard floating-point format (32-bit or 64-bit) to exponential form where weights are expressed as powers of 2. This parameter change significantly reduces the storage space required while maintaining the precision needed for neural network computations, as exponential representation requires fewer bits to encode the same level of precision.
Data Source
AI summary
A method including receiving, by a processor, a computing instruction for a neural network, wherein the computing instruction for the neural network includes a computing rule for the neural network and a connection weight of the neural network, and the connection weight is a power of 2; and inputting, for a multiplication operation in the computing rule for the neural network, a source operand corresponding to the multiplication operation to a shift register, and performing a shift operation based on a connection weight corresponding to the multiplication operation, wherein the shift register outputs a target result operand as a result of the multiplication operation. The neural network uses a shift operation, and a neural network computing speed is increased.