CNN Global Pool Feature Reduces DRAM Bandwidth

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Convolutional neural networks (CNNs) processing images in a block-based manner lack global information, leading to incomplete image blocks and high DRAM bandwidth requirements for transmitting entire image data, which is costly and inefficient.

Innovation Solution

Implementing a dual-process approach within the CNN system, where a global branch generates parameters representing global features of the input image, which are then used by a main trunk process to ensure each image block includes global information, reducing the need for transmitting entire images and thus minimizing DRAM bandwidth requirements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If block-based CNN operation is used to process image blocks independently, then processing efficiency is improved, but global information is lost

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidglobal information

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

A global pool feature is introduced as an intermediary that aggregates global information from all image blocks and makes it available to each block processing unit. This mediator enables independent block processing while preserving global context through the shared pool feature.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The CNN processing is segmented into independent block processing units that can operate in parallel, while a separate global pooling mechanism collects and distributes global features. This segmentation enables both parallel processing efficiency and global information availability.

Inventive Principle:

Principle #1Segmentation

2Loss of information

If entire input image data is transmitted for processing, then global information is preserved, but DRAM bandwidth consumption increases

Engineering Contradiction:

Improveglobal informationVSAvoidDRAM bandwidth

Core Design Contradiction:

Loss of informationVSQuantity of substance

Solution Approach 1:

Global information is extracted from the entire input image through a pooling operation that creates a compact global pool feature. This extracted representation preserves essential global context while occupying minimal memory bandwidth compared to transmitting the full image data.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of transmitting the entire image data, a copied and compressed representation (global pool feature) is created and shared across all processing units. This copy contains the essential global information needed for each block processing operation.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS20230029335A1System and method of convolutional neural network

Publication Date: 2023.01.26 TAIWAN SEMICONDUCTOR MANUFACTURING CO LTD

AI summary

A method the following operations: downscaling an input image to generate a scaled image; performing, to the scaled image, a first convolutional neural networks (CNN) modeling process with first non-local operations, to generate global parameters; and performing, to the input image, a second CNN modeling process with second non-local operations that are performed with the global parameters, to generate an output image corresponding to the input image. A system is also disclosed herein.