ML Accelerator Architecture With Wide SRAM and Multicast Dataflow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning technologies rely on general-purpose CPUs and GPUs, which are inefficient for computationally intensive tasks like deep learning due to their lack of specialized hardware acceleration for tensor contractions, addition of random noise, and element-wise non-linear operations, and existing hardware accelerators are either algorithm-specific or neuromorphic, failing to provide a general-purpose solution.
Innovation Solution
A hardware architecture that includes multiple banks of SRAM, operational units, and a multicast network to perform tensor contractions, addition of random noise, and element-wise non-linear operations efficiently, without being specific to a particular network architecture or algorithm, using a configurable instruction set for machine learning operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general purpose CPUs or GPUs are used for machine learning, then versatility across different algorithms is achieved, but processing speed and computational efficiency deteriorate
Solution Approach 1:
The patent segments the machine learning computation into distinct functional units: tensor contraction units, additive noise units, and element-wise non-linear operation units. Each unit is specialized for a specific operation type, allowing parallel processing while maintaining algorithm versatility through configurable parameters and memory interfaces.
Solution Approach 2:
The hardware architecture implements universal functionality through configurable computational units that can perform multiple operations. The tensor contraction units, for example, can handle different tensor dimensions and operation types by reconfiguring data flow paths, enabling the same hardware to serve various machine learning algorithms without dedicated specialization for each.
2Productivity
If hardware is specialized for specific network architectures, then processing efficiency is improved, but adaptability to other algorithms deteriorates
Solution Approach 1:
The patent implements dynamic reconfiguration capabilities where the hardware can adapt its data flow paths and operational modes based on the specific algorithm being executed. Memory banks and computational units can be dynamically allocated and configured to match the requirements of different network architectures, maintaining high efficiency across diverse algorithms rather than being fixed for a single architecture.
3Measurement precision
If more computational resources are allocated for deep learning, then model accuracy and capability are improved, but computational cost and resource consumption worsen
Solution Approach 1:
The patent employs parameter changes by implementing support for lower precision fixed-point operations alongside standard floating-point operations. This allows the system to maintain model accuracy where high precision is necessary while using lower precision for operations where it suffices, thereby reducing overall computational cost and resource consumption without significantly compromising model performance.
Data Source
AI summary
A machine learning hardware accelerator architecture and associated techniques are disclosed. The architecture features multiple memory banks of very wide SRAM that may be concurrently accessed by a large number of parallel operational units. Each operational unit supports an instruction set specific to machine learning, including optimizations for performing tensor operations and convolutions. Optimized addressing, an optimized shift reader and variations on a multicast network that permutes and copies data and associates with an operational unit that support those operations are also disclosed.