Heterogeneous Processor Integrating CNN and RNN Operators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network dedicated processors cannot simultaneously optimize convolutional neural networks (CNN) and recurrent neural networks (RNN) due to their different operational architectures, leading to reduced overall operation efficiency and increased power consumption.
Innovation Solution
A heterogeneous processor architecture that integrates CNN and RNN operators into a single high-performance, low-power chip, utilizing a hybrid segmentation scheme, dynamic fixed-point representation, and weight clustering to minimize external memory access and share resources like memory, controllers, and interfaces, thereby optimizing energy and area efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single processor architecture is used to process both CNN and RNN, then device complexity is reduced, but operation efficiency deteriorates because CNN and RNN have greatly different operational characteristics that cannot be simultaneously optimized
Solution Approach 1:
The processor is segmented into distinct CNN processing units and RNN processing units, each optimized for their specific operational characteristics. The CNN processor handles convolutional layers with spatial data, while the RNN processor handles recurrent layers with sequential data, allowing each segment to be optimized independently for maximum efficiency.
Solution Approach 2:
The processor achieves multi-functionality by integrating both CNN and RNN processing capabilities into a single system. Shared resources including memory, data buses, and control units are designed to support both processing modes, enabling the single processor to efficiently handle diverse neural network workloads without requiring separate dedicated processors.
2Productivity
If separate single-chip processors are used for CNN and RNN, then operation efficiency is maximized for each network type, but device complexity and area increase
Solution Approach 1:
The CNN processor and RNN processor are merged into a single heterogeneous processing system with shared resources. Common infrastructure including memory modules, data buses, and control logic are consolidated, reducing overall device complexity and area while maintaining the operational efficiency benefits of specialized processing units through resource sharing and coordinated operation.
3Productivity
If external memory access is increased for CNN operations, then processing capability is improved, but power consumption increases
Solution Approach 1:
Input data is pre-processed and segmented into appropriate formats before being transferred to the CNN processor. The hybrid segmentation scheme organizes data in advance to maximize utilization of on-chip memory buffers, reducing the frequency and volume of external memory accesses during actual processing operations and thereby lowering power consumption.
4Measurement precision
If precision of activation data is increased, then computational accuracy is improved, but the number of data bits increases leading to higher memory bandwidth requirements and power consumption
Solution Approach 1:
The processor dynamically adjusts the precision of activation data based on the specific computational requirements of different layers and operations. Variable precision formats allow the system to use higher precision when needed for accuracy-critical operations while using lower precision for operations where it is sufficient, optimizing the trade-off between computational accuracy and data bit quantity.
Data Source
AI summary
A heterogeneous processor architecture for integrating a convolutional neural network (CNN) and a recurrent neural network (RNN) into a single high-performance, low-power chip in a neural network processor architecture, the heterogeneous processor architecture includes: an on-chip integrated circuit including a CNN operator for processing the CNN, an RNN operator for processing the RNN, an operation controller for performing control, a memory for storing data which is to be used by the operators, an interface for externally exchanging data, and a data bus through which data moves between constituent elements, wherein a fully-connected layer constituting the CNN performs data processing by sharing the RNN operator.