Sequential Neural Network Layer Processing for Edge Memory Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Constrained edge devices lack sufficient memory to implement deep neural networks due to limited resources, making it difficult to perform efficient inference tasks such as Acoustic Scene Classification.
Innovation Solution
A method that involves sequentially obtaining and storing parameters of each neural network layer, processing input data with the currently stored layer, and discarding previous layer parameters to free memory, allowing for the use of neural networks with an arbitrary number of layers on devices with limited memory without the need for dedicated hardware like GPUs or TPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks with multiple layers are implemented, then feature extraction capability is improved, but memory requirements increase beyond what constrained edge devices can provide
Solution Approach 1:
The patent segments the deep neural network into multiple layers that are processed sequentially. Each layer's parameters are loaded, processed, and then discarded before loading the next layer's parameters. This segmentation allows the system to handle deep network architectures without requiring sufficient memory to hold all parameters simultaneously.
Solution Approach 2:
The patent extracts only the necessary parameters for the current layer being processed from the complete neural network model. By taking out and processing one layer at a time rather than loading the entire network, the memory footprint is dramatically reduced to fit constrained edge devices.
2Quantity of substance
If sequential layer processing with parameter replacement is used, then memory usage is reduced, but processing time increases due to repeated data loading
Solution Approach 1:
The patent performs preliminary actions by pre-processing input data before the sequential layer processing begins. This includes any necessary data transformations or feature extractions that can be done in advance, reducing the need to reprocess data during the sequential layer execution and mitigating time losses.
3Quantity of substance
If layer parameters are replaced sequentially, then memory capacity requirements are reduced, but system complexity increases due to parameter management overhead
Solution Approach 1:
The patent introduces an intermediary data storage arrangement that manages the loading and discarding of layer parameters. This intermediary structure simplifies the parameter management process by providing a standardized interface between the sequential layer processing and the underlying memory system, reducing the complexity burden on the main processing logic.
Data Source
Figure 1~2
Figure 3a~3c
Figure 4
AI summary
The present disclosure relates to a method for performing inference on input data (201) using a neural network (1) and a processing device (1000) employing the aforementioned method. The method comprises obtaining (S11) and storing input data (201), obtaining (S21) parameter data (210a) indicating the parameters of the first layer (110) and storing (S22) the parameter data (210a) in a parameter data storage location (21) and processing (S13) the input data (210) to form first layer output data (202). The method further comprises storing (S14) the first layer output data (202), obtaining (S23) parameter data (210b) of the second layer (120) and storing (S24) the second layer parameter data (210b) by replacing the first layer parameter data (210a) with the second layer parameter data (210b), processing (S25) the first layer output data (202) to form second layer output data (203) and storing the second layer output data (203).