Ear-Worn Neural Network Chip With Tiled Memory for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional ear-worn devices face challenges in implementing neural networks for audio enhancement due to high power consumption and latency issues, which are exacerbated by increasing clock frequency or processor count, leading to reduced battery life and discomfort.
Innovation Solution
A neural network chip with collocated memory and processing circuitry tiles, optimized for power efficiency and reduced latency, utilizing a tile array architecture that minimizes data movement and power consumption through shared buses and parallel operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If clock frequency or processor count is increased to reduce latency, then processing speed is improved, but power consumption increases
Solution Approach 1:
The processor is divided into multiple independent tiles (e.g., 16 tiles) that can operate in parallel. Each tile contains its own processing units and memory, allowing the system to process audio data through multiple pathways simultaneously, reducing latency without requiring a single high-frequency processor that would consume excessive power.
Solution Approach 2:
The patent transitions from a single-processor architecture to a multi-tile parallel architecture, adding the dimension of spatial distribution. By organizing processors in a 2D tile array with row and column interconnects, the system achieves parallel processing across multiple dimensions, reducing latency through concurrent operations while maintaining power efficiency at the individual tile level.
2Productivity
If more processors are added to handle neural network computations, then processing capability is improved, but device complexity increases
Solution Approach 1:
Each tile in the array is designed with identical, universal functionality containing processing units, memory, and interconnect interfaces. This uniform tile design allows any tile to perform any neural network computation, simplifying the overall system architecture while enabling scalable processing capability through parallel instantiation of the same modular unit.
Solution Approach 2:
The patent employs homogeneous tiles with identical internal structures and interfaces throughout the array. This homogeneity simplifies control logic, memory management, and data routing compared to heterogeneous processor designs, reducing device complexity while maintaining high processing capability through parallel execution across identical units.
3Reliability
If memory capacity is increased to store more neural network weights, then model accuracy is improved, but chip area increases
Solution Approach 1:
The patent merges memory and processing units within each tile, creating tightly coupled compute nodes. Each tile contains local memory that stores weights and activation data needed for its processing operations, eliminating the need for large centralized memory and reducing overall chip area while providing sufficient capacity for neural network models through distributed storage across multiple tiles.
Solution Approach 2:
The patent distributes memory capacity across the 2D tile array rather than using a single large memory block. By organizing memory in a distributed manner across multiple tiles arranged in rows and columns, the system achieves large total memory capacity while maintaining compact chip area through spatial distribution and efficient use of the two-dimensional substrate.
Data Source
AI summary
A hearing aid may include a neural network chip having tiles arranged in an array, each tile including memory, 16-128 multiplier-accumulator circuits (MACs), and routing circuitry. The memory of each tile may be configured to store a portion of elements of a matrix A comprising weights of a recurrent neural network. Each tile may be configured to receive and store elements of an activation vector X, and all tiles in a column of the array may be configured to receive the same elements of X. The plurality of tiles may be configured to perform a matrix-vector multiplication A*X by performing multiply-and-accumulate sub-operations in parallel among the plurality of tiles. The routing circuitry from the tiles in each respective row of tiles may be configured to combine results of the multiply-and-accumulate sub-operations.