ROM Multiply-Adder Neural Network Cells With Reduced Routing Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network implementations face inefficiencies due to separate components like GPUs and NPUs, which introduce high power consumption and latency issues, particularly in real-time and power-sensitive applications such as mobile devices and IoT applications, and existing methods and edge computing, and lack of larger and more powerful models, and lack of efficient deployment of larger and more powerful models, and lack of efficient deployment of larger and more powerful models.
Innovation Solution
Embedding neural networks on silicon through integrated read-only memory (ROM) multiply-adders, which reduces routing overhead and improves efficiency by integrating memories, adders, and eliminates the need for extensive fabrics, addressing the limitations of current methodologies that utilize separate memories, multipliers, and introduces integrated cells.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If separate memories and multipliers are used in neural network implementations, then computational functionality is achieved, but routing overhead and device complexity increase
Solution Approach 1:
The patent combines separate memories and multipliers into integrated read-only memory multiply-adder units. Each integrated unit contains both storage elements (for weight matrices) and computational elements (multipliers and adders) in a unified structure, eliminating the need for extensive routing between separate components and reducing overall device complexity.
Solution Approach 2:
The integrated read-only memory multiply-adder units serve multiple functions simultaneously: they store weight matrices in their memory portions and perform matrix-vector multiplication operations in their computational portions. This multi-functionality reduces the number of separate components needed and simplifies the overall architecture.
2Area of stationary object
If extensive fabrics are used to connect separate components, then computational operations are enabled, but area consumption increases
Solution Approach 1:
By merging memory and computational units into integrated structures, the patent eliminates the need for extensive interconnect fabrics that would be required to connect separate components. The computational elements are directly integrated with their corresponding weight storage, dramatically reducing area consumption while maintaining computational efficiency.
3Use of energy by moving object
If separate memories, multipliers, and adders are used, then neural network computations are performed, but power consumption increases
Solution Approach 1:
The patent integrates memory, multipliers, and adders into unified read-only memory multiply-adder units. This consolidation reduces the number of separate components and their associated power consumption, while also reducing routing overhead. The integrated units perform computations using locally stored weights without requiring power-intensive data movement between separate components.
Data Source
AI summary
An integrated circuit (IC) device may implement a neural network model. The IC device may include integrated cells for performing matrix multiplication (MatMul) operations in the model. An integrated cell may include a sequential read-only memory (ROM) cell, multipliers, and adder. The sequential ROM cell may store weights. The multiplier may multiply the weights with activations. The adders may sum the products. The integrated cells may also include counters, which control weight fetching from sequential ROM cells to the multipliers, or multiplexers, which select and distribute appropriate activations to multipliers. The integrated cells may execute a MatMul operation through multiple clock cycles. The MatMul operation may be decomposed based on sizes of the weight matrix or activation matrix and features of the integrated cell array. The integrated cells may perform a part of the MatMul operation in each clock cycle. The integrated cells may be coupled with add units.