Image Processing Apparatus Memory Access Pattern Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In hierarchical neural networks, the varying memory access patterns across layers lead to inefficient memory access, resulting in increased clock cycles for processing, which reduces the utilization efficiency of calculating devices due to differing processing target regions for feature data.
Innovation Solution
An image processing apparatus with a storage control unit that divides feature images into two-dimensional blocks and assigns them to memories in a predetermined order, allowing for efficient memory access patterns to be determined and applied, enabling blocks to be read in one cycle regardless of layer differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If feature images are stored in memories with varying access patterns across hierarchical layers, then the system can handle different convolution operations (e.g., 3×3 vs 1×1), but the number of clock cycles required for memory access increases, reducing calculation device utilization efficiency
Solution Approach 1:
The feature image is divided into multiple two-dimensional blocks that are distributed across multiple memories. This segmentation allows different blocks to be accessed in parallel from different memory locations, enabling the system to maintain high calculation device utilization efficiency while supporting various convolution operations with different kernel sizes.
Solution Approach 2:
The patent introduces a spatial distribution dimension by mapping feature image blocks to different memory locations based on their position in the feature image. This dimensional transformation from sequential memory access to parallel spatial access across multiple memories enables simultaneous access to multiple feature blocks, reducing the clock cycle time and improving calculation device utilization efficiency.
2Adaptability or versatility
If multiple accesses to the same memory are required for different hierarchical layers, then the system can process various operations, but the processing cannot be completed efficiently due to sequential access constraints
Solution Approach 1:
The feature image is segmented into multiple two-dimensional blocks distributed across multiple memories. Each block can be independently accessed from its respective memory location, enabling parallel access patterns that reduce the total time required to gather all necessary feature data for different hierarchical layer operations.
Solution Approach 2:
The storage control unit pre-determines and pre-arranges the memory access patterns for different hierarchical layers based on the convolution operation parameters. This preliminary planning allows the readout unit to efficiently access the required blocks without sequential delays, reducing the clock cycles needed for memory access across different layers.
3Device complexity
If feature data is arranged at different addresses in the same memory, then storage flexibility is improved, but data cannot be acquired all at once due to single-port memory constraints
Solution Approach 1:
Instead of storing all feature data at different addresses in a single memory, the system segments the feature image into blocks distributed across multiple memories. Each memory location holds a specific subset of feature blocks, allowing parallel acquisition from multiple memories simultaneously, thereby maintaining storage flexibility while dramatically improving data acquisition speed.
Solution Approach 2:
The patent transitions from a single-memory address space to a multi-memory distributed address space. This dimensional expansion allows simultaneous access to multiple feature blocks from different memory locations in parallel, overcoming the single-port memory constraint and enabling high-speed data acquisition while maintaining flexible storage arrangements.
Data Source
AI summary
An apparatus includes a storage control unit that divides, into two-dimensional blocks, a feature image of a layer and stores the respective blocks in any one of a predetermined number of memories, a unit that determines a pattern for reading blocks from the memories based on information relating to an operation on the feature image, and a unit that reads blocks from the memories in accordance with the pattern. The storage control unit assigns repeatedly, for the two-dimensional blocks, the memories in a predetermined order from a head row/column along a row/column. In a second or a subsequent row/column, a memory, which the assignment is started, is shifted by a constant number from a previous row/column in the predetermined order.