Convolutional Neural Network Forward Propagation With On-Chip Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks face performance bottlenecks due to the limitations of general-purpose processors and GPUs, including low operational performance and high memory bandwidth requirements, which hinder efficient multilayer neural network operations.
Innovation Solution
A neural network acceleration processor with a direct memory access unit, master and slave computation modules, and an interconnection unit is employed to optimize forward propagation by segmenting input data, convoluting with convolution kernels, and merging intermediate results, utilizing on-chip caching to reduce memory access and improve computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a general-purpose processor is used to implement convolutional neural networks, then the system has high flexibility and ease of operation, but the operational performance is low and cannot meet performance requirements for multilayer neural network operations
Solution Approach 1:
The processor is divided into multiple specialized computation units (first computation unit, second computation unit, third computation unit, fourth computation unit) that can simultaneously process different portions of input data. Each unit is dedicated to specific computational tasks, enabling parallel processing and significantly improving operational performance while maintaining system flexibility through modular architecture.
Solution Approach 2:
The invention transitions from a single general-purpose processing dimension to a multi-dimensional parallel processing architecture. Multiple computation units operate simultaneously on different data streams, adding temporal and spatial dimensions to the processing capability, thereby achieving high performance for multilayer neural network operations while preserving flexibility.
2Productivity
If multiple general-purpose processors execute concurrently to improve performance, then the operational performance increases, but the intercommunication among them becomes a performance bottleneck
Solution Approach 1:
Multiple computation units are merged into a single integrated processor architecture with shared resources and unified control. This eliminates the need for complex intercommunication between separate processors while maintaining parallel processing capabilities, thereby improving performance without introducing communication bottlenecks.
Solution Approach 2:
The invention introduces an interconnection unit that acts as an intermediary between computation units and memory systems. This mediator efficiently manages data flow and communication between processing elements, eliminating the performance bottleneck associated with direct intercommunication between multiple independent processors.
3Device complexity
If a graphics processing unit (GPU) is used with small on-chip caching, then the device complexity is reduced, but model data must be repeatedly moved from off-chip causing huge power consumption and memory bandwidth bottleneck
Solution Approach 1:
The processor incorporates sufficient on-chip caching memory to pre-load and store model data (weight values) and input data before computation. This preliminary action ensures that data is readily available during processing, eliminating the need for repeated off-chip memory accesses and significantly reducing power consumption and memory bandwidth requirements.
Solution Approach 2:
The invention creates on-chip copies of model data and input data in caching memory, allowing multiple computation units to access local copies simultaneously. This eliminates the need for frequent off-chip data retrieval, reducing both power consumption and memory bandwidth bottlenecks while maintaining relatively simple device complexity.
Data Source
AI summary
Aspects for forward propagation of a convolutional artificial neural network are described herein. The aspects may include a direct memory access unit configured to receive input data from a storage device and a master computation module configured to select one or more portions of the input data based on a predetermined convolution window. Further, the aspects may include one or more slave computation modules respectively configured to convolute a convolution kernel with one of the one or more portions of the input data to generate a slave output value. Further still, the aspects may include an interconnection unit configured to combine the one or more slave output values into one or more intermediate result vectors, wherein the master computation module is further configured to merge the one or more intermediate result vectors into a merged intermediate vector.