DMA-Based CNN Backpropagation With Modular Kernel Updates

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing convolutional neural networks face performance bottlenecks due to the limitations of general-purpose processors and GPUs, including inefficient intercommunication among multiple processors and high power consumption from off-chip data movement.

Innovation Solution

An apparatus and method for backpropagation in convolutional neural networks utilizing a direct memory access unit and computation modules to process input data efficiently, including a master computation module and slave computation modules to update convolution kernels based on kernel gradients.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple general-purpose processors execute concurrently, then computational capability is improved, but intercommunication becomes a performance bottleneck

Engineering Contradiction:
Improvecomputational capabilityVSAvoidintercommunication complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent merges multiple computation functions into a single integrated neural network processor that combines computation modules, memory units, and communication interfaces in one device. This eliminates the need for complex intercommunication between multiple general-purpose processors while maintaining high computational capability through parallel processing within the unified architecture.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The processor is divided into multiple computation modules that can operate independently and concurrently. Each module handles specific computational tasks, allowing parallel processing without requiring complex intercommunication between separate processors. The segmentation is implemented through independent memory access units and modular computation cores within a single chip.

Inventive Principle:
Principle #1Segmentation

2Quantity of substance

If GPU uses off-chip memory for model data, then storage capacity is improved, but off-chip bandwidth becomes a performance bottleneck causing high power consumption

Engineering Contradiction:
Improvestorage capacityVSAvoidpower consumption
Core Design Contradiction:
Quantity of substanceVSUse of energy by moving object

Solution Approach 1:

The patent implements a hierarchical memory structure where small on-chip cache memory units are nested within each computation module, and these are nested within larger off-chip memory. This nested architecture allows frequent small data accesses to occur on-chip without requiring large data transfers, reducing power consumption while maintaining high storage capacity through the hierarchical structure.

Inventive Principle:
Principle #7Nested doll (Nesting)

Solution Approach 2:

The patent introduces intermediate cache memory units between the computation modules and off-chip memory. These intermediaries act as buffers that store frequently accessed model data locally, reducing the frequency of high-power off-chip memory accesses while still providing access to large storage capacities through the cache hierarchy.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS12423582B2Apparatus and methods for training in convolutional neural networks
Publication Date: 2025.09.23 CAMBRICON TECH CO LTD
  • US12423582B2 patent drawing
  • US12423582B2 patent drawing
  • US12423582B2 patent drawing

AI summary

Aspects for backpropagation of a convolutional neural network are described herein. The aspects may include a direct memory access unit configured to receive input data from a storage device and a master computation module configured to select one or more portions of the input data based on a predetermined convolution window. Further, the aspects may include one or more slave computation modules respectively configured to convolute one of the one or more portions of the input data with one of one or more previously calculated first data gradients to generate a kernel gradient, wherein the master computation module is further configured to update a prestored convolution kernel based on the kernel gradient.