CNN Accelerator Internal Buffering for Pooling Data Reuse

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Convolutional neural networks face high energy consumption due to frequent accesses to external memory for read/write operations during pooling operations across layers, leading to inefficient memory usage and increased power consumption.

Innovation Solution

A convolutional neural network accelerator is designed with Processing Elements (PEs) arranged in a 2D array, featuring internal buffers for data storage and operation units that perform convolution, non-linear, and normalization operations, with a pooling unit to reduce external memory accesses by maximizing data reuse and storing intermediate results internally before transferring them to external memory when reaching a predetermined size.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If pooling operations are performed across layers with frequent external memory accesses, then data processing completeness is improved, but energy consumption increases

Engineering Contradiction:
Improvedata processing completenessVSAvoidenergy consumption
Core Design Contradiction:
ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent merges multiple pooling operations across different layers into a single unified pooling operation. Instead of performing pooling separately for each layer (which would require multiple external memory accesses), the accelerator performs convolution and pooling operations jointly across multiple layers, consolidating data transfers and reducing the frequency of external memory accesses while maintaining complete data processing.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent implements preliminary action by loading and storing intermediate results of pooling operations in internal buffers before final external memory access is needed. The accelerator performs pooling operations on intermediate data and stores results in internal buffers, allowing subsequent operations to access this pre-processed data without requiring additional external memory accesses, thus reducing overall energy consumption.

Inventive Principle:
Principle #10Preliminary action

2Measurement precision

If intermediate results are stored in external memory after each layer processing, then data accuracy is improved, but memory access frequency increases

Engineering Contradiction:
Improvedata accuracyVSAvoidmemory access frequency
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent merges the storage operations for intermediate results across multiple layers into a single consolidated external memory access. Instead of writing intermediate results to external memory after each layer (which would require frequent accesses), the accelerator consolidates these writes into fewer batched operations, reducing memory access frequency while maintaining data accuracy through proper buffer management.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces internal buffers as intermediary storage between processing elements and external memory. These buffers act as mediators that temporarily hold intermediate results, allowing processing elements to access data quickly without requiring immediate external memory access. The buffers manage data transfer to external memory in optimized batches, reducing access frequency while preserving data integrity.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS11120330B2Accelerator in convolutional neural network and method for operating the same
Publication Date: 2021.09.14 SAMSUNG ELECTRONICS CO LTD
  • US11120330B2 patent drawing
  • US11120330B2 patent drawing
  • US11120330B2 patent drawing

AI summary

The present disclosure relates to a communication method and system for converging a 5th-Generation (5G) communication system for supporting higher data rates beyond a 4th-Generation (4G) system with a technology for Internet of Things (IoT). The present disclosure may be applied to intelligent services based on the 5G communication technology and the IoT-related technology, such as smart home, smart building, smart city, smart car, connected car, health care, digital education, smart retail, security and safety services. A Processing Element (PE) implemented in an accelerator in a convolutional neural network, which includes a first buffer configured to transfer input data to one other PE, and a second buffer configured to transmit to an outside output data that is processed on the basis of the input data; and an operation unit configured to generate output data.