Neuron Buffer Data Reuse for Low-Power DNN Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial intelligence/machine learning hardware, specifically deep neural networks (DNNs), face challenges in optimizing data sharing and processing across memory components, leading to suboptimal performance and high power consumption, particularly in battery-powered devices and low-power applications.
Innovation Solution
The implementation of an optimized DNN module that enhances data sharing and distribution across cooperating memory components, utilizing a neuron buffer with a double buffer memory structure to minimize memory reads and maximize processing efficiency, thereby reducing power consumption and improving processing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNNs, then processing speed is improved, but power consumption increases making them unsuitable for battery-powered devices
Solution Approach 1:
The patent segments the memory system into multiple specialized buffers (neuron buffers, weight buffers, activation buffers) that are distributed across different processing elements. This segmentation allows each buffer to be optimized for specific data types and access patterns, reducing the need for high-power general-purpose memory access while maintaining processing speed.
Solution Approach 2:
The patent introduces a new dimensional organization of memory resources by creating a hierarchical buffer structure that extends beyond traditional single-level memory. This multi-dimensional memory architecture (with buffers at different levels and locations) reduces access latency and power consumption by placing frequently accessed data closer to processing elements without requiring high-speed interconnects.
2Device complexity
If current memory management practices are used in DNN modules, then system simplicity is maintained, but data sharing efficiency between memory components is suboptimal leading to reduced processing performance
Solution Approach 1:
The patent creates universal buffer structures that can serve multiple functions: neuron buffers can store both input data and intermediate results, weight buffers can serve multiple neurons, and activation buffers can handle different activation functions. This multi-functionality improves data sharing efficiency without significantly increasing system complexity, as the same buffer infrastructure serves multiple purposes.
Solution Approach 2:
The patent introduces buffer management units as intermediary components that coordinate data sharing between different buffers and processing elements. These intermediaries optimize data flow and reduce redundant memory accesses without requiring complex reconfiguration of the underlying memory architecture, thus improving performance with minimal complexity increase.
3Productivity
If data is frequently read from memory components to maximize processing operations, then processing throughput is improved, but power consumption increases
Solution Approach 1:
The patent implements preliminary data loading into specialized buffers before processing operations begin. Neuron buffers pre-load input data, weight buffers pre-load parameter data, and activation buffers pre-prepare activation functions. This preliminary action reduces the need for frequent memory reads during processing, thereby maintaining high throughput while reducing power consumption from repeated memory access operations.
Solution Approach 2:
The patent ensures continuous utilization of buffered data by coordinating multiple processing elements to operate on the same data simultaneously. Once data is loaded into buffers, multiple neurons and processing units continuously operate on this data without idle periods or repeated memory accesses, maximizing the utility of each byte read from external memory and reducing overall power consumption.
Data Source
AI summary
An exemplary artificial intelligence/machine learning hardware computing environment having an exemplary DNN module cooperating with one or more memory components can perform data sharing and distribution as well reuse of a buffer data to reduce the number of memory component read/writes thereby enhancing overall hardware performance and reducing power consumption. Illustratively, data from a cooperating memory component is read according to a selected operation of the exemplary hardware and written to corresponding other memory component for use by one or more processing elements (e.g., neurons). The data is read in such a manner to optimize the engagement of the one or more processing elements for each processing cycle as well as to reuse data previously stored in the one or more cooperating memory components. Operatively, the written data is copied to a shadow memory buffer prior to being consumed by the processing elements.