Hardware Accelerator Memory Synchronization for Neural Network Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning hardware accelerators face inefficiencies in processing large amounts of data due to increased computational complexity and power consumption, particularly in neural network operations, where unnecessary data movements and sequential processing lead to high power usage and slow calculation times.
Innovation Solution
The implementation of a hardware accelerator system with a resource optimizer and read/write synchronizer that intelligently allocates memory resources, reuses data, and synchronizes read/write operations to reduce power consumption and execution time, utilizing distributed memory and reconfigurable computing circuits to optimize data processing in neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is stored and retrieved from memory to complete arithmetic computations, then computational accuracy is improved, but execution time increases and power consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-loading input data, weight data, and intermediate results into on-chip memory before computation begins. This allows the hardware accelerator to perform arithmetic operations using data already in fast memory, eliminating the need to retrieve data from slower off-chip memory during computation, thus reducing execution time while maintaining computational accuracy.
Solution Approach 2:
The patent introduces an intermediary component - the on-chip memory - that acts as a buffer between the computation units and off-chip memory. This intermediary allows frequent read/write operations to occur in fast memory without requiring constant access to slow external memory, thereby reducing execution time while preserving data accuracy for computations.
2Measurement precision
If data is stored and retrieved from memory to complete arithmetic computations, then computational accuracy is improved, but power consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-loading all necessary data into on-chip memory before computation begins. This allows the hardware accelerator to perform arithmetic operations using data already in fast memory, eliminating the need to retrieve data from slower off-chip memory during computation, thus reducing power consumption while maintaining computational accuracy.
Solution Approach 2:
The patent introduces an intermediary component - the on-chip memory - that acts as a buffer between the computation units and off-chip memory. This intermediary allows frequent read/write operations to occur in fast memory without requiring constant access to slow external memory, thereby reducing power consumption while preserving data accuracy for computations.
3Productivity
If hardware resources are increased to process large amounts of data, then processing speed is improved, but hardware cost increases
Solution Approach 1:
The patent applies segmentation by dividing the memory system into two segments: on-chip memory for frequent data access during computation and off-chip memory for storing large amounts of data. This segmentation allows the system to process large datasets efficiently using a moderate amount of expensive on-chip memory, rather than requiring a large amount of on-chip memory for all data, thus improving processing speed without proportionally increasing hardware cost.
Solution Approach 2:
The patent applies local quality by providing fast on-chip memory specifically for data that is actively being processed, while using slower off-chip memory for data that is not currently needed. This localized high-speed memory allocation optimizes processing speed for active computations without requiring the entire system to use expensive high-speed memory, thereby improving productivity without proportionally increasing hardware cost.
Data Source
AI summary
Low-power systems and methods increase computational efficiency in neural network processing by allowing hardware accelerators to perform processing steps on large amounts of data at reduced execution times without significantly increasing hardware cost. In various embodiments, this is accomplished by accessing locations in a source memory coupled to a hardware accelerator and using a resource optimizer that based on storage availability and network parameters determines target locations in a number of distributed memory elements. The target storage locations are selected according to one or more memory access metrics to reduce power consumption. A read/write synchronizer then schedules simultaneous read and write operations to reduce idle time and further increase computational efficiency.