On-Chip Buffer Memory for Neural Network Energy Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network devices require significant energy consumption due to frequent memory access for processing large amounts of complex data, which is particularly challenging for low-power devices like mobile and IoT devices.
Innovation Solution
The implementation of a neural network device with an on-chip buffer memory, a computational circuit, and a controller that stores and reuses feature maps within the on-chip buffer memory, reducing the need for external memory access and minimizing energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network device processes large amounts of complex data through frequent memory access, then computation performance is improved, but energy consumption increases
Solution Approach 1:
The patent implements an on-chip buffer memory that is integrated within the neural network device, creating a nested memory structure. This buffer memory is positioned between the computational units and external memory, allowing intermediate feature maps to be stored locally during neural network operations, thereby reducing the frequency of external memory access and lowering energy consumption while maintaining computation performance
Solution Approach 2:
The patent pre-loads input data and weight values into the on-chip buffer memory before computation begins. By preparing these data elements in advance in the local buffer, the system minimizes the need for frequent external memory access during the actual computation phase, thus improving energy efficiency while maintaining high computation performance
2Loss of energy
If on-chip buffer memory is used to store feature maps, then external memory access is reduced, but device complexity increases
Solution Approach 1:
The on-chip buffer memory is designed to serve multiple functions: storing input data, storing weight values, and storing intermediate feature maps during neural network operations. By making this single memory component multi-functional, the patent reduces the need for multiple separate memory structures, thereby limiting the increase in device complexity while achieving energy savings through reduced external memory access
3Device complexity
If single port buffer memory is used, then device complexity is reduced, but data access speed decreases
Solution Approach 1:
The patent divides the buffer memory into multiple regions or banks, each capable of independent access through the single port. This segmentation allows different data elements (input data, weight values, feature maps) to be accessed in different time slots or by different computational units, thereby maintaining data access speed despite using a single port, while keeping device complexity low
Data Source
AI summary
A neural network device including an on-chip buffer memory that stores an input feature map of a first layer of a neural network, a computational circuit that receives the input feature map of the first layer through a single port of the on-chip buffer memory and performs a neural network operation on the input feature map of the first layer to output an output feature map of the first layer corresponding to the input feature map of the first layer, and a controller that transmits the output feature map of the first layer to the on-chip buffer memory through the single port to store the output feature map of the first layer and the input feature map of the first layer together in the on-chip buffer memory.