CNN Memory Partitioning Strategy for Bandwidth Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing CNN implementations on smaller, less powerful devices face limitations due to limited hardware computing power and memory constraints, leading to inefficient memory management and increased processing time, as they struggle to optimize data partitioning and memory usage for convolutional layer calculations.

Innovation Solution

The implementation of a memory replacement policy that prioritizes data types and determines optimal partition sizes for input feature maps, kernel data, and output feature maps to minimize computation cost, using an exhaustive search method to identify the combination with the lowest computation cost for convolutional layer calculations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of energy

If CNN data is kept loaded in internal memory to reduce memory bandwidth requirements, then memory efficiency improves, but internal memory size requirements increase which is not feasible in embedded systems

Engineering Contradiction:

Improvememory bandwidth consumptionVSAvoidinternal memory size

Core Design Contradiction:

Loss of energyVSVolume of stationary object

Solution Approach 1:

The patent segments CNN data into three distinct types (input feature map data, kernel data, output feature map data) and further divides each type into multiple partitions. This segmentation allows the system to manage large datasets in smaller chunks that can fit within limited internal memory, while systematically reloading partitions as needed to minimize memory bandwidth consumption.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements dynamic memory management through a replacement policy that adaptively determines which data partitions to keep in internal memory and which to move to external memory based on usage patterns. The system dynamically adjusts the number of partitions loaded into internal memory and the timing of data transfers, optimizing the balance between memory bandwidth consumption and internal memory utilization.

Inventive Principle:

Principle #15Dynamics

2Volume of stationary object

If data is reloaded from external memory multiple times, then internal memory capacity requirements decrease, but processing time increases

Engineering Contradiction:

Improveinternal memory capacityVSAvoidprocessing time

Core Design Contradiction:

Volume of stationary objectVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-loading multiple partitions of each data type into internal memory before they are needed for computation. The system proactively manages data transfers between external and internal memory based on predicted usage patterns, reducing the frequency of urgent data reloads during processing and minimizing processing time delays.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent ensures continuity of useful action by maintaining an optimal number of data partitions continuously loaded in internal memory, allowing computation to proceed without interruption. The replacement policy ensures that as one partition is being used, another is being pre-loaded, minimizing idle time and maintaining continuous processing flow.

Inventive Principle:

Principle #20Continuity of useful action

3Productivity

If more data is loaded into internal memory, then computation speed improves, but hardware resource costs increase

Engineering Contradiction:

Improvecomputation speedVSAvoidhardware resource requirements

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent applies local quality by optimizing the memory residency of specific data partitions based on their individual access patterns and computational importance. Instead of uniformly loading all data, the system selectively maintains high-priority partitions in internal memory while allowing lower-priority partitions to reside in external memory, optimizing computation speed while minimizing hardware resource requirements.

Inventive Principle:

Principle #3Local quality

4Ease of operation

If traditional arbitrary partitioning methods are used, then implementation simplicity is maintained, but optimization effectiveness decreases

Engineering Contradiction:

Improveimplementation simplicityVSAvoidmemory optimization effectiveness

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent changes key parameters of data partitioning by considering the multi-dimensional characteristics of CNN data (height, width, depth/channels) rather than using arbitrary partitions. The replacement policy adjusts partition sizes and numbers based on specific data dimensions and access patterns, significantly improving memory optimization effectiveness while maintaining reasonable implementation complexity through systematic rules.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11010308B2Optimizing data partitioning and replacement strategy for convolutional neural networks

Publication Date: 2021.05.18 LG ELECTRONICS INC

AI summary

Embodiments of the present disclosure include method for optimizing an internal memory for calculation of a convolutional layer of a convolutional neural network (CNN), the method including determining a computation cost of calculating the convolutional layer using each combination of a memory management scheme of a plurality of memory management schemes and data partition sizes of input feature map (IFM) data, kernel data, and output feature map (OFM) data to be loaded in the internal memory; identifying one combination of a memory management scheme and data partition sizes having a lowest computation cost for the convolutional layer; and implementing the CNN to use the one combination for calculation of the convolutional layer.