CNN Feature Map Caching with Distance-Based Flushing

Solution Overview

Problem

Current FIFO and LRU flushing algorithms in convolutional neural networks lead to increased DRAM access times and bandwidth power consumption by flushing tensors that are needed in the future, resulting in inefficient use of DRAM resources.

Innovation Solution

A feature map caching method that analyzes the convolutional neural network connection list to determine the distance between layer operation steps and future usage, allowing for optimized flushing and write-back operations to reduce DRAM access times and power consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If FIFO or LRU flushing algorithm is used, then cache management is simple, but DRAM access times increase and bandwidth power consumption increases

Engineering Contradiction:

Improvecache management simplicityVSAvoidDRAM access times

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The system performs preliminary analysis of the CNN connection list to identify future layer operations and their required tensors. By pre-calculating the distance between current and future operations, the system determines which tensors to retain in cache before they are needed, preventing premature flushing and reducing DRAM access times.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system continuously monitors the CNN operation sequence and uses this feedback to dynamically adjust cache flushing decisions. By analyzing the connection list and tracking which tensors will be used in future operations, the system adapts its cache management strategy to retain necessary tensors, thereby reducing unnecessary DRAM accesses and power consumption.

Inventive Principle:

Principle #23Feedback

2Ease of operation

If FIFO or LRU flushing algorithm is used, then cache management is simple, but DRAM bandwidth power consumption increases

Engineering Contradiction:

Improvecache management simplicityVSAvoidDRAM bandwidth power consumption

Core Design Contradiction:

Ease of operationVSUse of energy by stationary object

Solution Approach 1:

The system performs preliminary analysis of the CNN connection list to identify future layer operations and their required tensors. By pre-calculating the distance between current and future operations, the system determines which tensors to retain in cache before they are needed, preventing premature flushing and reducing DRAM access times.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system continuously monitors the CNN operation sequence and uses this feedback to dynamically adjust cache flushing decisions. By analyzing the connection list and tracking which tensors will be used in future operations, the system adapts its cache management strategy to retain necessary tensors, thereby reducing unnecessary DRAM accesses and power consumption.

Inventive Principle:

Principle #23Feedback

3Loss of time

If distance-based flushing strategy is used, then DRAM access times are reduced, but system complexity increases

Engineering Contradiction:

ImproveDRAM access timesVSAvoidcache management complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The system performs preliminary analysis of the CNN connection list to identify future layer operations and their required tensors. By pre-calculating the distance between current and future operations, the system determines which tensors to retain in cache before they are needed, preventing premature flushing and reducing DRAM access times.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system automatically analyzes the CNN connection list and makes autonomous cache flushing decisions based on calculated distances to future operations. The lookup controller self-manages the cache by identifying which tensors will be needed soon and retaining them, eliminating the need for complex external control mechanisms while reducing DRAM access times.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS11782839B2Feature map caching method of convolutional neural network and system thereof

Publication Date: 2023.10.10 NEUCHIPS CORP

AI summary

A feature map caching method of a convolutional neural network includes a connection analyzing step and a plurality of layer operation steps. The connection analyzing step is for analyzing a network to establish a convolutional neural network connection list. The convolutional neural network connection list includes a plurality of tensors and a plurality of layer operation coefficients. Each of the layer operation coefficients includes a step index, at least one input operand label and an output operand label. The step index as a processing order for the layer operation step. At least one of the layer operation steps is for flushing at least one of the tensors in a cache according to a distance between the at least one of the layer operation steps and a future layer operation step of the layer operation steps. The distance is calculated according to the convolutional neural network connection list.