Partitioned Neural Networks for Real-Time Multi-Sensor Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual reality and augmented reality systems utilizing multiple sensors face computational challenges due to the complexity of neural networks, particularly when real-time determinations are required, as the bandwidth and latency for transmitting sensor data to an aggregator computing device are not suitable for real-time applications.
Innovation Solution
A method for partitioning neural networks between sensor computing devices and an aggregator computing device, optimizing the neural network architecture by training a set of candidate networks with varying layer sizes, compression block locations, and selecting the optimal partition based on task performance and hardware constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex neural networks are used for real-time sensor data processing, then determination accuracy is improved, but computational latency increases
Solution Approach 1:
The neural network is divided into multiple stages with different complexity levels. Early stages use simpler networks for quick initial processing, while later stages apply more complex networks for refined determination, allowing real-time processing without sacrificing accuracy
Solution Approach 2:
The system applies neural networks selectively based on confidence thresholds. When simple networks achieve sufficient determination accuracy, complex networks are skipped, reducing latency while maintaining accuracy when needed
2Measurement precision
If complex neural networks with many layers are used, then determination accuracy is improved, but computational resources required increase
Solution Approach 1:
The computational workload is segmented across multiple devices in the distributed system. Each device executes only the portion of the neural network assigned to it, reducing individual device complexity requirements while enabling complex determinations through collaboration
Solution Approach 2:
The system distributes neural network execution across the spatial dimension of multiple devices rather than concentrating all computational resources in a single device, reducing the computational burden on any individual device
3Measurement precision
If sensor data is transmitted to an aggregator computing device for processing, then determination accuracy is improved, but bandwidth requirements increase
Solution Approach 1:
The system extracts and processes only the essential sensor data features locally at each device before transmission to the aggregator. This reduces the volume of data transmitted while retaining the critical information needed for accurate determinations
Solution Approach 2:
Data processing is segmented between local devices and the aggregator. Local devices perform preliminary processing and feature extraction, transmitting only processed results rather than raw sensor data, reducing bandwidth requirements
4Measurement precision
If sensor data is transmitted to an aggregator computing device for processing, then determination accuracy is improved, but latency increases
Solution Approach 1:
Sensor data undergoes preliminary processing, filtering, and feature extraction at local devices before transmission. This preliminary action reduces data volume and prepares data for faster transmission and more efficient aggregator processing, reducing overall latency
Data Source
AI summary
Methods, systems, and media for partitioning neural networks are provided. In some embodiments, a method comprises obtaining a training set. The method comprises training a plurality of neural networks using the training set, wherein neural networks differ based on dimensions of one or more layers of the neural networks and a location of a compression block positioned between a first set of layers of a neural network and a second set of layers of the neural network. The method comprises selecting a neural network based on hardware constraints of a system on which the neural network is to be implemented, wherein the first set of layers of the selected neural network are executed by one or more sensor devices of the system, and wherein the second set of layers of the selected neural network are executed by an aggregator computing device of the system.