Convolution Data Dividing for Mobile CNN Memory Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning CNN models, especially large and complex ones like ResNet, face challenges in mobile and embedded devices due to insufficient memory and response delays, making it difficult to perform convolution operations efficiently with limited storage and computing resources.
Innovation Solution
A data dividing method for convolution operations that determines restriction conditions, calculates optimal dividing sizes for each dimension, generates candidate sub-data blocks, combines them, and evaluates their cost function to select the most efficient data blocks for processing, minimizing memory access and improving reusability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large and complex CNN model (e.g., ResNet with 152 layers) is used to improve classification accuracy, then the model performance is improved, but the memory requirements and response time worsen, making it difficult to apply on mobile and embedded devices
Solution Approach 1:
The patent divides the CNN computation into multiple segments by partitioning data into blocks and processing them in stages. The convolution operation is segmented into multiple computation stages where intermediate results are stored in on-chip memory, allowing large models to be processed in chunks rather than requiring all data to be loaded simultaneously into memory.
Solution Approach 2:
The patent introduces a new dimension of data organization by dividing data along multiple dimensions (spatial blocks, channel groups, etc.) and processing them in a multi-stage pipeline. This dimensional decomposition allows the system to handle large models by distributing computation across multiple processing stages rather than being constrained by single-dimension memory limitations.
2Reliability
If a large and complex CNN model is used to improve classification accuracy, then the model performance is improved, but the response time worsens due to computation-consuming operations
Solution Approach 1:
The patent performs preliminary data preparation by dividing input data and weight data into blocks before computation. Candidate data blocks are pre-identified and organized in memory, so that when convolution operations need to be performed, the data is already in the optimal format and location, reducing computation time and improving response speed.
Solution Approach 2:
The patent implements a continuous multi-stage computation pipeline where data flows through multiple processing stages without interruption. Intermediate results are continuously generated and passed to the next stage, maximizing hardware utilization and reducing idle time, thereby improving overall processing efficiency and reducing response time.
3Productivity
If data is divided into multiple blocks to reduce memory access, then data reusability is improved, but the complexity of data management and division increases
Solution Approach 1:
The patent systematically varies division parameters (block size, number of stages, channel grouping) to find optimal configurations for different CNN models and hardware platforms. By establishing parameter relationships and optimization rules, the system manages the complexity of data division through parameterization rather than ad-hoc decisions, making the complex data management process more controllable and adaptable.
Data Source
AI summary
A data dividing method applied to a computing device that performs a convolution operation based on an input image data and a weight data is provided. The method includes: determining a restriction condition in connection with the performing of the convolution operation by the computing device; determining a set of candidate data blocks for the input image data and a set of candidate data blocks for the weight data according to the restriction condition; generating an evaluation result by evaluating, according to candidate data blocks in the set of candidate data blocks for the input image data and the set of candidate data blocks for the weight data, an amount of data load of the computing device in accessing both an external memory and an internal memory of the computing device; and determining a method of dividing the input image data and the weight data according to the evaluation result.