Multicore Neural Network Hardware Tile Splitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in resource-constrained environments where power consumption and processing capabilities are limited, requiring efficient hardware implementations that also support various configurations and real-time processing demands.
Innovation Solution
A multicore hardware implementation of DNNs is proposed, where input data is split into tiles or passes based on threshold sizes, allowing for depth-first evaluation across multiple cores, reducing memory requirements and enabling efficient processing whether the input tensor is large or small, by utilizing on-chip and off-chip memory effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If DNN hardware implementation uses larger memory to store entire tensors, then processing capability is improved, but silicon area and power consumption increase
Solution Approach 1:
The patent divides the neural network computation into multiple layer groups that can be processed independently. Each layer group processes a subset of layers, allowing the system to work with smaller data tiles rather than entire tensors, thus reducing memory requirements while maintaining processing capability
Solution Approach 2:
The patent introduces the dimension of time by processing layer groups sequentially rather than simultaneously. This temporal dimension allows the system to reuse the same hardware resources for different layer groups at different time steps, reducing the spatial memory requirements
2Productivity
If DNN hardware is designed for high processing power, then real-time processing capability is improved, but power consumption increases
Solution Approach 1:
By segmenting the computation into layer groups, the patent enables the hardware to process smaller data tiles through the same layer group multiple times. This reuse of hardware resources reduces overall power consumption while maintaining real-time processing capability
Solution Approach 2:
The patent ensures continuous utilization of hardware resources by pipelining the processing of multiple data tiles through the same layer group. The hardware remains actively processing useful work throughout, improving efficiency and reducing power consumption per unit of computation
3Adaptability or versatility
If DNN hardware supports multiple configurations, then adaptability is improved, but device complexity increases
Solution Approach 1:
The patent designs a universal layer group structure that can handle different types of neural network layers (convolutional, fully connected, etc.) through a unified processing framework. This multi-functional design allows the same hardware to support multiple DNN configurations without requiring separate specialized circuits for each layer type
Data Source
AI summary
A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. If a size of the input data in a first dimension is greater than a threshold, the hardware implementation splits the input data for the first layer group into at least a first tile and a second tile, along the first dimension. If the size of the input data in the first dimension is not greater than the threshold, the hardware implementation splits the evaluation of the first layer group into at least a first pass and a second pass, along a dimension other than the first dimension.