Neural Network Internal Storage for Memory Access Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning applications, particularly convolutional neural networks (CNNs), face challenges in energy efficiency and processing power constraints, leading to high demands on resources when deployed on embedded devices due to frequent reads from external memory, which is time-consuming and resource-intensive.
Innovation Solution
Incorporating an internal storage system within the neural network that uses write and read transformer units to optimize data storage and retrieval patterns, allowing for batch processing and rearrangement of data to reduce the number of external memory accesses, thereby enhancing efficiency and reducing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning applications are deployed on embedded devices, then recognition and classification performance is improved, but energy consumption and processing power requirements increase significantly
Solution Approach 1:
The system segments memory into external memory and internal storage, with the internal storage acting as a local cache for frequently accessed feature data. This segmentation allows the neural network to process data with reduced external memory accesses, thereby lowering energy consumption while maintaining recognition and classification performance.
Solution Approach 2:
The internal storage pre-loads and stores feature data that will be needed for subsequent processing steps. By having data readily available in internal storage before processing, the system avoids repeated high-energy external memory accesses during neural network operations, thus reducing overall energy consumption.
2Productivity
If feature data is frequently read from external memory, then data availability for processing is improved, but processing time and resource utilization increase
Solution Approach 1:
The internal storage acts as an intermediary between external memory and the neural network processing units. It buffers feature data, allowing the neural network to access data quickly from internal storage rather than repeatedly accessing external memory, thus reducing processing time while maintaining data availability.
Solution Approach 2:
Feature data is pre-loaded into internal storage before processing operations begin. This preliminary action ensures that data is immediately available when needed during processing, eliminating wait times and reducing overall processing time without compromising data availability.
3Quantity of substance
If feature data is stored in external memory, then storage capacity is improved, but access efficiency and energy consumption worsen
Solution Approach 1:
The storage system is segmented into external memory for bulk storage capacity and internal storage for frequent access operations. This segmentation allows the system to maintain large storage capacity while achieving high access efficiency for feature data through the internal storage buffer, reducing the number of slow external memory accesses.
Solution Approach 2:
The internal storage provides localized, high-speed access to feature data that is actively being processed, while external memory provides bulk storage capacity. This local quality optimization ensures that frequently accessed data has high access efficiency, while overall storage capacity remains large through external memory.
Data Source
AI summary
A neural network includes an internal storage unit. The internal storage unit stores feature data received from a memory external to the neural network. The internal storage unit reads the feature data to a hardware accelerator of the neural network. The internal storage unit adapts a storage pattern of the feature data and a read pattern of the feature data to enhance the efficiency of the hardware accelerator.