Neural Network Intermediate Feature Storage for Edge Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity and size of neural networks for processing time-series data, such as video content, lead to high computational demands, necessitating more efficient processing at edge devices and storage solutions that balance accuracy and speed without relying solely on increased computational power.
Innovation Solution
Implementing a system where intermediate layer data from a primary neural network is stored in a fast-access storage device and used as input for secondary neural networks, optimizing data storage and processing by utilizing different types of storage devices for read/write operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks are increased in size and complexity to improve output accuracy, then output accuracy is improved, but computational demand increases
Solution Approach 1:
The patent extracts intermediate feature data from the primary neural network's hidden layers and stores it separately in fast-access storage. This allows secondary neural networks to reuse these extracted features without requiring the primary network to recompute them, thereby reducing overall computational demand while maintaining accuracy
Solution Approach 2:
The patent performs preliminary computation by generating and storing intermediate layer data from the primary neural network before it is needed by secondary networks. This pre-computed data is stored in fast-access storage, eliminating the need for redundant computations and reducing peak computational power requirements
2Productivity
If computational power is increased to process more video data, then processing speed is improved, but energy consumption increases
Solution Approach 1:
The patent merges the computational workflows of multiple secondary neural networks by having them share common intermediate features from the primary network. This consolidation eliminates redundant computations across different analysis tasks, improving processing speed while reducing the total energy consumption of the system
Solution Approach 2:
The patent creates copies of intermediate layer data and stores them in fast-access storage for rapid retrieval by multiple secondary networks. This copying approach allows parallel processing without requiring multiple full computations, thereby increasing processing speed while minimizing additional energy consumption
3Power
If video data is transmitted to distant locations for computation, then processing capability is improved, but transmission overhead increases
Solution Approach 1:
The patent segments the video processing workflow into a primary network that extracts intermediate features and secondary networks that perform specialized analysis. The intermediate features are stored locally in fast-access storage at the edge device, enabling localized processing without requiring constant data transmission to distant centralized systems
4Speed
If fast-access storage is used for intermediate layer data, then processing speed is improved, but storage cost increases
Solution Approach 1:
The patent applies local quality by using fast-access storage specifically for intermediate layer data that requires rapid access by multiple secondary networks, while relying on standard storage for the original video data. This selective approach optimizes processing speed for critical operations without incurring fast-storage costs for all data
Data Source
AI summary
Systems and methods described herein provide for the use and storage of intermediate layer data within a neural network processing system. A first neural network, such as an object detection neural network may receive and process raw video image data to generate output utilized for metadata creation. Secondary neural networks may be configured to accept input data from one or more intermediate layers of the first neural network instead of the raw video image data. In this way, the initial data processed by the intermediate layers of the first neural network can be stored and utilized as a shortcut for processing additional features or attributes within the video image data. This alleviates the need to process video image data multiple times in different neural networks. The intermediate layer data can be stored in a different and often cheaper storage system and recalled faster and with fewer resources for future use.