Data Temporary Storage Apparatus for Convolution Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning CNN models, such as MobileNet, face memory capacity and response delay issues due to large-scale convolution operations, which are exacerbated by the inefficient use of memory bandwidth in depthwise convolution methods, leading to increased hardware resource demands and storage costs.
Innovation Solution
A data temporary storage apparatus and method that temporarily stores input data in a first storage unit to multiple second storage units, using a moving unit to fetch and store data in an alternate and sequential manner, optimizing data distribution across multiple storage units to enhance convolution operation efficiency without requiring additional hardware resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If depthwise convolution is used to reduce computation and model parameters, then the amount of computation and model parameters are significantly reduced, but the bandwidth stress upon memory is tremendously aggravated
Solution Approach 1:
The patent divides the feature data into multiple blocks and distributes them across multiple storage units (e.g., storage units 0-5 for different blocks). This segmentation allows parallel access to different feature blocks during convolution operations, reducing the concentration of bandwidth stress on a single memory interface while maintaining the computational efficiency of depthwise convolution.
Solution Approach 2:
The patent introduces a spatial dimension by distributing feature blocks across multiple storage units instead of storing all features in a single linear memory space. This multi-dimensional storage organization enables simultaneous access to multiple feature blocks through parallel read operations, effectively reducing bandwidth stress during depthwise convolution operations.
2Measurement precision
If large-scale CNN models are implemented to improve classification accuracy, then classification accuracy is improved, but memory capacities become insufficient and response delays increase
Solution Approach 1:
The patent segments large-scale feature data into multiple manageable blocks distributed across several storage units. This allows the system to handle large CNN models by processing features in distributed blocks rather than requiring all features to be stored in a single memory location, effectively overcoming memory capacity limitations.
Solution Approach 2:
The patent creates a universal storage architecture where multiple storage units can be dynamically allocated and accessed based on the specific convolution operation requirements. This multi-functional storage system can adapt to different CNN model sizes and configurations, supporting both small and large-scale models with the same hardware infrastructure.
3Quantity of substance
If feature data is read from memory and used once then discarded in depthwise convolution, then computation amount is reduced, but utilization rate of temporary storage units deteriorates
Solution Approach 1:
The patent performs preliminary organization of feature data into multiple blocks before convolution operations begin. By pre-distributing feature blocks across storage units in an optimized manner, the system prepares the data layout to enable efficient parallel access during convolution, improving temporary storage utilization without increasing computation.
Solution Approach 2:
The patent enables continuous utilization of temporary storage units by maintaining feature blocks in distributed storage during the entire convolution process. Instead of reading and discarding data in discrete operations, the system maintains continuous access to distributed feature blocks across multiple storage units, keeping the storage infrastructure continuously productive throughout the computation.
Data Source
AI summary
A data temporary storage apparatus includes a moving unit coupled to a first storage unit and multiple second storage units. The moving unit receives a moving instruction having contents including a read address, a destination address and a predetermined moving rule. The moving unit further executes the moving instruction to fetch input data by row from the first storage unit according to the read address, and to temporarily stores one after another in an alternate and sequential manner the data in each row to each of the second storage units indicated by the destination address. The data moving, data reading and convolution approaches of the present invention implement in parallel data moving and a convolution operation, achieving a ping-pong operation of convolution units and enhancing convolution efficiency, while reducing memory costs since configuring two data storage spaces in a memory is not necessary.