Backpropagation in Fully Connected Layers Using DMA and Master-Slave Modules

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies for convolutional artificial neural networks face performance bottlenecks due to the operational limitations of general-purpose processors and the high power consumption associated with off-chip bandwidth in graphics processing units (GPUs).

Innovation Solution

The proposed solution involves an apparatus and method for backpropagation in a fully connected layer of a convolutional artificial neural network, utilizing a direct memory access unit and master/slave computation modules to efficiently process input data and gradients, thereby generating weight gradient vectors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple general-purpose processors execute concurrently to improve performance, then operational performance is improved, but intercommunication overhead becomes a performance bottleneck

Engineering Contradiction:

Improveoperational performanceVSAvoidintercommunication overhead

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system segments computation into master and slave processor roles, where master processors handle coordination and slave processors handle computation. This segmentation reduces intercommunication overhead by minimizing coordination requirements while maintaining parallel processing capabilities for improved operational performance.

Inventive Principle:

Principle #1Segmentation

2Device complexity

If GPU uses small on-chip caching to reduce cost, then device complexity is reduced, but off-chip bandwidth becomes a main performance bottleneck causing huge power consumption

Engineering Contradiction:

Improveon-chip cachingVSAvoidpower consumption

Core Design Contradiction:

Device complexityVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary action by pre-fetching and caching weight values in on-chip memory before they are needed for computation. This preliminary caching action reduces subsequent off-chip memory access requirements, thereby reducing power consumption while maintaining simple device architecture.

Inventive Principle:

Principle #10Preliminary action

3Device complexity

If GPU uses small on-chip caching, then device complexity is reduced, but off-chip bandwidth becomes a main performance bottleneck

Engineering Contradiction:

Improveon-chip cachingVSAvoidperformance

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system introduces on-chip memory as an intermediary between the simple GPU architecture and external memory. This intermediary cache stores frequently accessed weight values, reducing direct off-chip memory access and improving performance without significantly increasing device complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12205036B2Apparatus and methods for training in fully connected layers of convolutional networks

Publication Date: 2025.01.21 CAMBRICON TECH CO LTD

AI summary

Aspects for backpropagation in a fully connect layer of a convolutional neural network are described herein. The aspects may include a direct memory access unit configured to receive input data and one or more first data gradients from a storage device. The aspects may further include a master computation module configured to transmit the input data and the one or more first data gradients to one or more slave computation modules. The slave computation modules are respectively configured to multiply one of the one or more first data gradients with the input data to generate a default weight gradient vector.