CNN Operation Grouping for SRAM Capacity Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning frameworks like Convolutional Neural Networks (CNNs) face challenges in power consumption and cost due to limited Static Random-Access Memory (SRAM) capacity in embedded devices, particularly when processing high-resolution data that exceeds the memory limits, leading to reduced computing efficiency.
Innovation Solution
The method involves splitting data operations into smaller parts to fit within the memory unit capacity, allowing input and output data to be processed entirely within the memory unit by grouping operations and storing intermediate results efficiently, thereby avoiding the need for external memory storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the capacity of SRAM is reduced to lower cost, then the cost of the embedded device is reduced, but the data amount that can be processed exceeds the memory capacity
Solution Approach 1:
The patent divides the data operation into multiple split operations, each processing a subset of data that fits within the SRAM capacity. This segmentation allows the system to handle large datasets using limited on-chip memory by processing data in manageable chunks rather than attempting to load the entire dataset at once.
Solution Approach 2:
The patent introduces a time dimension to memory utilization by processing data sequentially in multiple passes. Instead of requiring all data to be available simultaneously in memory, the system processes data in temporal segments, loading only the necessary portions into SRAM at each processing stage and utilizing external memory for data storage across time steps.
2Measurement precision
If the input data amount is increased to ensure output accuracy, then the output accuracy is maintained, but the data amount exceeds the memory unit capacity
Solution Approach 1:
The patent segments the data operation into multiple smaller operations, each processing a portion of the input data that fits within SRAM capacity. This allows the system to maintain accuracy by processing complete data through multiple sequential operations rather than attempting to process the entire large dataset simultaneously in limited memory.
Solution Approach 2:
The patent maintains continuous processing by sequentially executing multiple split operations that collectively process the entire input dataset. Each operation continues the data processing flow from the previous operation, ensuring that the complete input data is processed through the neural network operations to maintain output accuracy while managing memory constraints through temporal continuity.
3Quantity of substance
If data is stored in external memory storage, then the data amount can exceed memory capacity, but the computing efficiency is reduced
Solution Approach 1:
The patent segments data operations to load only the necessary data portions into SRAM for processing, rather than loading the entire external memory content. This selective loading approach minimizes data transfer requirements while maintaining the ability to process large datasets, thereby improving computing efficiency by reducing the overhead of external memory access.
Solution Approach 2:
The patent performs preliminary identification and selection of data portions that need to be processed, organizing them into split operations that can be efficiently loaded into SRAM. This preliminary organization reduces the complexity of subsequent processing and minimizes external memory access during the actual computation, improving overall computing efficiency.
4Quantity of substance
If operations are split into multiple split operations, then the data amount fits within memory capacity, but the processing time increases
Solution Approach 1:
The patent segments operations to fit data within SRAM capacity, which enables faster processing by eliminating external memory access during computation. Although multiple operations are required, each operation benefits from working entirely within on-chip memory, significantly reducing the time penalty compared to the alternative of loading entire large datasets into external memory and processing them there.
Data Source
AI summary
Systems and methods of data processing are provided. The method comprises receiving an input data to be processed by a series of operations, identifying a first operation from the series of operations, selecting at least one second operation from the series of operations to be grouped with the first operation based at least in part on an amount of an input data and an output data of the grouped operations and the capacity of the memory unit, and processing a portion of the input data of the grouped operations. An efficiency of the series of data operations can be improved by ensuring the input data and output data of any data operations are both stored in the memory unit.