Tensor Slicing for CNN Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) require significant memory, memory bandwidth, and computational power to efficiently process large digital images for image recognition and classification, leading to performance bottlenecks and energy inefficiencies.
Innovation Solution
Implementing a cone of dependency and cone of influence based processing method, where only the necessary data is kept in a local cache memory, reducing main memory accesses and optimizing cache memory usage by dividing images into sections and processing them in parallel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CNN processing methods are used to process large digital images, then image recognition and classification accuracy is improved, but memory usage and computational overhead increase significantly
Solution Approach 1:
The patent divides the input image into multiple smaller patches or tiles that are processed independently through the CNN. This segmentation allows the system to process large images by handling smaller data portions simultaneously, reducing peak memory requirements while maintaining overall processing accuracy through subsequent merging of results.
Solution Approach 2:
The patent introduces a temporal dimension by processing image patches at different time steps or stages. Instead of processing the entire image simultaneously, the system processes patches sequentially or in batches across multiple time steps, transforming a spatial memory problem into a temporal processing sequence that reduces memory pressure.
2Measurement precision
If conventional CNN processing methods are used to process large digital images, then image recognition and classification accuracy is improved, but computational power requirements and energy consumption increase
Solution Approach 1:
By segmenting the image into smaller patches, the patent reduces the computational workload per processing step. Each patch requires fewer computational operations compared to processing the entire large image at once, thereby reducing total energy consumption while maintaining recognition accuracy through comprehensive patch processing.
Solution Approach 2:
The patent employs periodic processing where image patches are processed in repeated cycles or batches. This periodic action allows the system to process large images through multiple smaller computational steps, reducing peak energy consumption compared to single-pass processing of entire large images.
3Reliability
If conventional CNN processing methods are used to process large digital images, then processing completeness is improved, but processing time increases due to memory bandwidth limitations
Solution Approach 1:
The patent segments the image processing task into independent patch processing units that can be handled concurrently. This segmentation enables parallel processing of multiple image regions simultaneously, reducing total processing time while maintaining completeness through systematic coverage of all image areas.
Solution Approach 2:
The patent performs preliminary processing of image patches before final assembly. By pre-processing patches independently and preparing them in advance, the system reduces the time required for subsequent processing steps and memory operations, overall accelerating the complete processing pipeline.
4Device complexity
If the entire image is processed simultaneously, then processing simplicity is maintained, but cache memory utilization is inefficient leading to increased main memory accesses
Solution Approach 1:
The patent segments the image into cache-friendly patches that fit within available cache memory. This segmentation allows efficient utilization of cache memory for processing smaller data portions, reducing the need to access slower main memory while maintaining processing simplicity through automated patch management.
Solution Approach 2:
The patent applies different processing strategies to different regions of the image based on their characteristics. By analyzing local properties of image regions, the system optimizes cache utilization for each region, processing data locally to minimize main memory accesses while maintaining overall processing simplicity.
Data Source
AI summary
A system and method of performing tensor operations with a multi-step operation processing system in a memory-efficient manner. The method includes the stages of dividing an N-dimensional tensor into a set of tensor slices. The tensor slices consist of one or more consecutive rows. The tensor slices may further be segmented. The tensor slice segments, along with the dependency data, form based on the tensor dependencies are used for an tensor operation computation to generate a first result. Each processed slice segment is fused into a result slice by removing extra data used in the computation. This process is repeated for each slice to be processed and combined into a final processed tensor result.