CNN Layer Processing Modes for Lower External Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN inference processing technologies fail to optimize the usage amount of external memory bandwidth due to fixed processing patterns that do not account for the optimal pattern for each CNN layer, leading to inefficient memory usage.
Innovation Solution
A convolutional neural network inference processing device that sets a processing mode for each CNN layer to minimize external memory bandwidth by employing three modes: fixing input data, output data, or kernel sliding methods, depending on the layer's requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a fixed processing pattern is applied to all CNN layers, then the device complexity is reduced and operation is simplified, but the external memory bandwidth usage cannot be minimized for each specific layer
Solution Approach 1:
The patent applies dynamics by making the processing pattern adjustable and adaptive rather than fixed. The determination unit dynamically selects from multiple processing modes (first mode with fixed input data, second mode with fixed output data, third mode with fixed kernel) based on the specific characteristics of each CNN layer, allowing the system to optimize memory bandwidth usage for different layer types while maintaining operational simplicity through automated selection.
2Quantity of substance
If multiple processing modes are implemented for different CNN layers, then the external memory bandwidth usage is minimized, but the device complexity increases due to multiple convolution calculators and mode selection logic
Solution Approach 1:
The determination unit implements self-service by automatically analyzing the characteristics of each CNN layer and selecting the appropriate processing mode without requiring external intervention or complex manual configuration. The system serves itself by making intelligent decisions about resource allocation and processing patterns, reducing the burden on external control systems while achieving optimized memory bandwidth usage.
Solution Approach 2:
The patent employs feedback mechanisms where the determination unit continuously monitors the characteristics of processed layers and adjusts the processing mode selection accordingly. This feedback loop allows the system to learn from previous processing experiences and optimize future memory bandwidth usage patterns, balancing the increased device complexity with improved overall system efficiency.
3Productivity
If batch processing with sliding processing range is used, then the processing efficiency is improved, but the external memory bandwidth usage increases due to repeated data access
Solution Approach 1:
The patent applies local quality by tailoring the processing mode to the specific characteristics of each CNN layer rather than using a uniform approach. Different layers receive different processing modes (first mode for layers benefiting from fixed input data, second mode for layers with fixed output data, third mode for kernel-fixed operations) based on their local requirements, thereby optimizing memory bandwidth usage while maintaining high processing efficiency through batch processing with sliding ranges.
Data Source
AI summary
A determination unit (33) determines a processing mode for minimizing a usage amount of an external memory band for each layer of a CNN among processing modes that are based on a sliding method in a case in which entire input data is processed by causing the plurality of convolution calculators to perform batch processing on a processing range in the input data that is input to each layer of the CNN by sliding the processing range, the processing modes including a first mode in which the input data is fixed and a kernel is slid, a second mode in which output data is fixed and the input data and the kernel are slid in a channel direction, and a third mode in which the kernel is fixed and the input data are slid in the vertical and horizontal directions. The setting unit (34) sets the determined processing mode in a calculation unit 35. The calculation unit (35) performs convolution calculation on the input data based on the set processing mode.