Neural Hardware Accelerator for Parallel Tensor Computations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network systems face inefficiencies in energy consumption and computation speed, particularly in convolutional operations, due to limitations in parallel processing and memory access in traditional architectures.
Innovation Solution
The implementation of a hardware accelerator with multiple neural cores and a network interconnect, where each neural core has local memory storing a portion of the weight tensor, allowing for parallel and distributed tensor computations, optimizing energy efficiency and speed by maximizing direct communication and aligning tensor dimensions with network locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If traditional neural network architectures are used, then device complexity is reduced, but energy efficiency and computation speed deteriorate due to limitations in parallel processing and memory access
Solution Approach 1:
The system divides the neural network computation into multiple neural cores, each handling a portion of the weight tensor and input data tensor. This segmentation enables parallel processing across multiple cores, improving energy efficiency and computation speed while distributing the overall device complexity across modular units.
Solution Approach 2:
The patent introduces a distributed memory architecture where weight tensors are stored across multiple neural core memories rather than a single centralized memory. This dimensional change in data organization enables parallel access patterns and reduces memory access latency, improving energy efficiency without proportionally increasing device complexity.
2Speed
If centralized memory architecture is used, then device complexity is reduced, but computation speed deteriorates due to memory access latency
Solution Approach 1:
The memory architecture is segmented into local memories at each neural core, allowing each core to access its assigned weight tensor portion locally without contending for centralized memory resources. This segmentation reduces memory access latency and improves computation speed while distributing memory complexity across multiple simple local units.
Solution Approach 2:
The patent introduces a distributed memory system as an intermediary between the neural cores and the weight tensor storage. Each neural core has direct access to its local memory portion, acting as an intermediary that reduces the access path and latency compared to a centralized memory architecture, thereby improving computation speed.
3Productivity
If sequential processing is used, then device complexity is reduced, but productivity deteriorates due to limited parallel processing capability
Solution Approach 1:
The neural network computation is segmented into independent tasks that can be executed in parallel across multiple neural cores. Each core processes a portion of the input data tensor with its assigned weight tensor portion, enabling simultaneous computation and improving productivity while keeping individual core complexity low.
Solution Approach 2:
The patent merges multiple simple neural core units into a distributed system that collectively performs complex neural network computations. By combining multiple identical or similar core designs, the system achieves high parallel processing capability and productivity while avoiding the complexity of designing a single highly complex processor.
Data Source
AI summary
Networks and encodings therefor are provided that are adapted to provide increased energy efficiency and speed for convolutional operations. In various embodiments, a neural network comprises a plurality of neural cores. Each of the plurality of neural cores comprises a memory. A network interconnects the plurality of neural cores. The memory of each of the plurality of neural cores comprises at least a portion of a weight tensor. The weight tensor comprising a plurality of weights. Each neural core is adapted to retrieve locally or receive a portion of an input image, apply the portion of the weight tensor thereto, and store locally or send a result therefrom via the network to other of the plurality of neural cores.