DMA-Based CNN Backpropagation With Modular Kernel Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural networks face performance bottlenecks due to the limitations of general-purpose processors and GPUs, including inefficient intercommunication among multiple processors and high power consumption from off-chip data movement.
Innovation Solution
An apparatus and method for backpropagation in convolutional neural networks utilizing a direct memory access unit and computation modules to process input data efficiently, including a master computation module and slave computation modules to update convolution kernels based on kernel gradients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple general-purpose processors execute concurrently, then computational capability is improved, but intercommunication becomes a performance bottleneck
Solution Approach 1:
The patent merges multiple computation functions into a single integrated neural network processor that combines computation modules, memory units, and communication interfaces in one device. This eliminates the need for complex intercommunication between multiple general-purpose processors while maintaining high computational capability through parallel processing within the unified architecture.
Solution Approach 2:
The processor is divided into multiple computation modules that can operate independently and concurrently. Each module handles specific computational tasks, allowing parallel processing without requiring complex intercommunication between separate processors. The segmentation is implemented through independent memory access units and modular computation cores within a single chip.
2Quantity of substance
If GPU uses off-chip memory for model data, then storage capacity is improved, but off-chip bandwidth becomes a performance bottleneck causing high power consumption
Solution Approach 1:
The patent implements a hierarchical memory structure where small on-chip cache memory units are nested within each computation module, and these are nested within larger off-chip memory. This nested architecture allows frequent small data accesses to occur on-chip without requiring large data transfers, reducing power consumption while maintaining high storage capacity through the hierarchical structure.
Solution Approach 2:
The patent introduces intermediate cache memory units between the computation modules and off-chip memory. These intermediaries act as buffers that store frequently accessed model data locally, reducing the frequency of high-power off-chip memory accesses while still providing access to large storage capacities through the cache hierarchy.
Data Source
AI summary
Aspects for backpropagation of a convolutional neural network are described herein. The aspects may include a direct memory access unit configured to receive input data from a storage device and a master computation module configured to select one or more portions of the input data based on a predetermined convolution window. Further, the aspects may include one or more slave computation modules respectively configured to convolute one of the one or more portions of the input data with one of one or more previously calculated first data gradients to generate a kernel gradient, wherein the master computation module is further configured to update a prestored convolution kernel based on the kernel gradient.


