DNN Neuron Buffer Data Reuse for Low-Bandwidth AI Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial intelligence/machine learning hardware, particularly deep neural networks (DNNs), face challenges in balancing computational performance and power consumption, especially in devices where low power consumption is critical, such as smartphones and power-over-Ethernet security cameras, due to inefficient memory and processing management.
Innovation Solution
The implementation of an optimized DNN module that enhances data sharing and distribution across memory components, utilizing a neuron buffer with a double buffer memory structure to minimize data reads and maximize processing efficiency, thereby reducing power consumption and improving overall performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNNs, then processing speed is improved, but power consumption increases making them unsuitable for battery-powered devices
Solution Approach 1:
The patent segments the processing system into specialized DNN processing units with dedicated memory buffers, separating the high-performance computing function from general-purpose processors. This segmentation allows DNN operations to be handled by optimized hardware that achieves high processing speed while consuming less power than general-purpose GPUs.
Solution Approach 2:
The patent introduces a new dimension of optimization by implementing double buffer memory structures and data reuse mechanisms at the hardware level. This adds temporal and spatial efficiency dimensions to the processing system, enabling sustained high performance with reduced power consumption through optimized data flow and minimal re-reading from external memory.
2Productivity
If data is frequently read from external memory to maintain high processing throughput, then processing performance is improved, but bandwidth usage increases and power consumption rises
Solution Approach 1:
The patent implements preliminary action by pre-loading data into double buffer memory structures before processing begins. The first buffer holds input data ready for processing, while the second buffer is pre-loaded with subsequent data. This preliminary preparation eliminates the need for frequent external memory reads during processing, reducing bandwidth usage while maintaining high throughput.
Solution Approach 2:
The double buffer mechanism enables continuous useful action by ensuring that processing units always have data available in local memory. While one buffer is being processed, the other is being loaded or prepared, creating a continuous data supply that maintains high throughput without requiring simultaneous high-bandwidth external memory access.
3Reliability
If current memory management practices are used to balance shared resources, then system stability is maintained, but processing performance and power consumption optimization is insufficient
Solution Approach 1:
The patent introduces double buffer memory structures as intermediary components between external memory and processing units. These buffers act as mediators that manage data flow, ensuring stable and reliable data supply to processing units while optimizing for performance and power efficiency. The intermediary buffers absorb memory access variability and provide steady data flow.
Solution Approach 2:
The patent applies parameter changes by implementing specific buffer size configurations and data reuse strategies tailored to DNN workloads. By adjusting memory organization parameters, buffer capacities, and data retention policies, the system achieves optimized processing performance and power consumption while maintaining stability through controlled memory management.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An exemplary artificial intelligence/machine learning hardware computing environment having an exemplary DNN module cooperating with one or more memory components can perform data sharing and distribution as well reuse of a buffer data to reduce the number of memory component read/writes thereby enhancing overall hardware performance and reducing power consumption. Illustratively, data from a cooperating memory component is read according to a selected operation of the exemplary hardware and written to corresponding other memory component for use by one or more processing elements (e.g., neurons). The data is read in such a manner to optimize the engagement of the one or more processing elements for each processing cycle as well as to reuse data previously stored in the one or more cooperating memory components. Operatively, the written data is copied to a shadow memory buffer prior to being consumed by the processing elements.