Neural Network Operation Apparatus Dynamic Accumulator Scaling

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing neural network hardware, such as feedback interactive neural networks (FINN), faces scalability limitations and performance issues due to non-optimized accumulators, which are affected by the size of the accumulator and precision requirements.

Innovation Solution

A neural network operation apparatus and method that determines a scaling constant based on the neural network structure, performs scaling and rounding operations, and adjusts the accumulator size dynamically to optimize neural network operations, using bit shifting and multiplexing to select appropriate scale values based on the type of neural network.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the accumulator size is increased to improve precision for neural network operations, then the measurement precision is improved, but the device complexity and area requirements increase

Engineering Contradiction:
ImproveprecisionVSAvoidarea requirements
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies parameter changes by dynamically adjusting the accumulator size based on the specific neural network operation requirements. Different accumulator sizes are selected depending on the precision needs of different layers and operations, rather than using a fixed large accumulator for all operations. This resolves the contradiction by matching the accumulator size precisely to the actual precision requirements, avoiding unnecessary area consumption while maintaining adequate precision.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent implements dynamics by making the accumulator size configurable and adaptable rather than fixed. The system can dynamically select appropriate accumulator sizes based on the neural network architecture, operation type, and precision requirements. This dynamic adjustment allows the system to optimize between precision and area requirements in real-time, resolving the trade-off between these two parameters.

Inventive Principle:
Principle #15Dynamics

2Measurement precision

If the accumulator size is increased to improve precision, then the measurement precision is improved, but the productivity decreases due to reduced computational efficiency

Engineering Contradiction:
ImproveprecisionVSAvoidcomputational efficiency
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent uses parameter changes to optimize the balance between precision and computational efficiency. By selecting appropriate accumulator sizes tailored to specific neural network operations, the system avoids the overhead of unnecessarily large accumulators while maintaining sufficient precision. This targeted parameter adjustment improves productivity by reducing the computational burden associated with oversized accumulators.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent applies partial action by using accumulators that are sufficiently large for the specific task at hand rather than consistently using maximum size accumulators. This partial approach provides adequate precision for each individual operation while avoiding the excessive computational cost of uniformly large accumulators across all operations, thereby improving overall productivity.

Inventive Principle:
Principle #16Partial or excessive action

3Device complexity

If fixed-size accumulators are used for all neural network operations, then the device complexity is reduced, but the adaptability to different neural network structures decreases

Engineering Contradiction:
Improveaccumulator configurationVSAvoidscalability
Core Design Contradiction:
Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent implements dynamics by transitioning from fixed-size accumulators to dynamically configurable accumulators. The system can adapt the accumulator size based on the specific neural network architecture and operation requirements, enabling better scalability and adaptability to different neural network structures while maintaining manageable device complexity through software-based configuration rather than hardware complexity.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent applies universality by creating a configurable accumulator system that can serve multiple neural network architectures and operation types through a single adaptable hardware design. Rather than requiring separate fixed-size accumulators for different operations, the system uses one configurable accumulator that can be adapted to various needs, thereby improving versatility without proportionally increasing device complexity.

Inventive Principle:
Principle #6Universality (Multi-functionality)

4Measurement precision

If larger accumulators are used to maintain precision in feedback interactive neural networks, then the measurement precision is improved, but the use of energy increases

Engineering Contradiction:
ImproveprecisionVSAvoidpower consumption
Core Design Contradiction:
Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent applies parameter changes by dynamically adjusting accumulator sizes based on the specific precision requirements of different neural network operations. Rather than consistently using large accumulators that consume more energy, the system selects appropriate accumulator sizes tailored to each operation's precision needs, thereby reducing overall power consumption while maintaining necessary precision where required.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent uses partial action by applying high-precision accumulators only when and where necessary rather than uniformly across all operations. This selective approach maintains precision for critical operations while avoiding the excessive energy consumption associated with continuously operating large accumulators, thus optimizing the balance between precision and power consumption.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS20240143274A1Neural network operation apparatus and method
Publication Date: 2024.05.02 SAMSUNG ELECTRONICS CO LTD
  • US20240143274A1 patent drawing
  • US20240143274A1 patent drawing
  • US20240143274A1 patent drawing

AI summary

A neural network operation apparatus and method are disclosed. A neural network operation apparatus includes a receiver that receives data for a neural network operation, and a processor that performs a scaling operation by multiplying the data by a constant, performs a rounding operation by truncating bits forming a result of the scaling operation, performs a scaling back operation based on a result of the rounding operation, and generates a neural network operation result by accumulating results of the scaling back operation.