Neural Network Convolution Order Optimization for Cache Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks face inefficiencies due to the limited capacity of on-chip caches, leading to high interlayer feature map data access overhead and idle computation units, as they rely on off-chip memory for data transfer, which is slow and resource-intensive.
Innovation Solution
The proposed method adjusts the convolution order of neural network layers based on the size of feature maps, allowing the next layer to access feature maps from the on-chip cache instead of off-chip memory, thereby reducing data access overhead and idle time by determining the optimal size and order for convolution computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If feature maps are stored in off-chip memory to overcome cache capacity limitations, then storage capacity is sufficient, but data access speed decreases and computation units become idle
Solution Approach 1:
The patent applies preliminary action by determining the convolution computation order before execution, analyzing feature map sizes in advance to plan which layers can sequentially access data from on-chip cache. This advance planning enables the system to maximize cache utilization and minimize off-chip memory access, resolving the contradiction between storage capacity and data access speed.
2Speed
If feature maps are divided into small chunks to fit on-chip cache, then data access speed improves, but computation overhead increases due to multiple load-store operations
Solution Approach 1:
The patent applies dynamics by dynamically determining the convolution computation order based on actual feature map sizes and cache capacity. Instead of fixed chunking, the system adaptively plans computation sequences to maximize continuous cache utilization, reducing the need for repeated load-store operations while maintaining fast data access.
3Ease of operation
If convolution computation order is fixed, then implementation is simple, but on-chip cache cannot be fully utilized leading to idle computation units
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the convolution computation order based on feature map size parameters and cache capacity. This optimization transforms the fixed computation order into a variable one that adapts to different neural network configurations, fully utilizing on-chip cache resources and eliminating idle computation units while maintaining implementation feasibility.
Data Source
AI summary
The present application discloses a neural network computation method includes determining the size of the first feature map obtained when the processor computes the present layer of the neural network before performing convolution computation on the next layer of the neural network; determining a convolution computation order of the next layer according to the size of the first feature map and the size of the second feature map for a convolution supported by the next layer; performing convolution computation instructions from the next layer based on the convolution computation order. Exemplary embodiments in the present disclosure decrease the interlayer feature map data access overhead and reduce the idle time of a computation unit by leaving out the storage of the first feature map and the loading process of the second feature map.