Feature Re-Extraction for Low-SRAM Neural Network Skip Connections
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Skip connections in neural networks improve generalization performance but require significant SRAM storage space, leading to increased costs, which is undesirable for resource-constrained devices.
Innovation Solution
Implement a re-extraction process to propagate feature quantity vectors through a bypass path, avoiding the need to store them in cache memory, thus enabling skip connections without expanding SRAM usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If skip connections are implemented to improve generalization performance, then generalization performance is improved, but SRAM storage space increases leading to higher costs
Solution Approach 1:
The patent extracts the feature quantity vectors from the forward propagation path and stores them in main memory instead of cache memory. This allows the skip connection functionality to be maintained while reducing the SRAM storage space requirement from cache memory to only the minimum necessary for active computation, with historical feature vectors stored in cheaper main memory.
Solution Approach 2:
The patent introduces main memory as an intermediary storage layer between the processing units and the skip connection mechanism. This intermediary allows feature vectors to be retained across different processing stages without requiring large cache memory, effectively decoupling the skip connection functionality from high-cost SRAM resources.
2Adaptability or versatility
If feature quantity vectors are retained in cache memory to enable skip connections, then skip connection functionality is achieved, but circuit area and costs increase
Solution Approach 1:
The patent extracts the storage function for feature quantity vectors from the cache memory system and relocates it to main memory. This separation allows the cache memory to be minimized to only what is necessary for active computation, while the bulk storage requirement for skip connections is satisfied by main memory, thereby reducing overall circuit area.
Solution Approach 2:
The patent changes the storage dimension hierarchy by moving feature vector storage from the fast but expensive cache memory dimension to the slower but cheaper main memory dimension. This dimensional transition in the memory hierarchy allows skip connection functionality to be maintained while reducing the footprint in the critical cache memory dimension.
3Quantity of substance
If SRAM storage space is not increased to reduce costs, then costs are reduced, but storage space for processing results becomes insufficient
Solution Approach 1:
The patent introduces main memory as an intermediary storage layer that supplements the limited SRAM cache memory. This intermediary provides additional storage capacity for feature quantity vectors needed by skip connections, ensuring storage sufficiency without requiring an increase in expensive SRAM resources.
Solution Approach 2:
The patent segments the storage requirement into two parts: a small portion in SRAM cache memory for actively computed feature vectors, and a larger portion in main memory for historical feature vectors needed by skip connections. This segmentation allows the system to meet total storage requirements while keeping SRAM usage minimal.
Data Source
AI summary
An information processing apparatus includes at least one memory storing a plurality of convolution layers and a processor connected to the at least one memory. The processor propagates output data based on a feature quantity vector extracted from input data from a preceding stage side at each convolution layer to a subsequent stage side; concatenates a forward propagation path with a bypass path that bypasses the forward propagation path; performs processing of extracting the feature quantity vector from the input data at each convolution layer; in the processing of extracting the feature quantity vector, performs, as re-extraction processing, processing of re-extracting the feature quantity vectors included in convolution layers up to a convolution layer where bypassing through the bypass path starts; and in a case where the re-extraction processing is performed, concatenates an output result from the forward propagation path with a result of the re-extraction processing.