Split DNN Feature Switching for Variable-Bandwidth Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Deep Neural Networks (DNNs) face challenges in executing computations on mobile devices with low processing power, often requiring data transfer to cloud servers, which is bandwidth-intensive, time-consuming, and raises data privacy concerns. Additionally, existing methods for distributed AI do not adapt well to varying network bandwidths and device resources.
Innovation Solution
The implementation of a Dynamic Switchable Feature Size Network (DyFsNet) that splits DNNs among devices, using switchable bottleneck subnetworks to compress and expand feature sizes dynamically based on available bandwidth and device resources, allowing for flexible adaptation of compression factors during inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If DNN computations are transferred to cloud servers, then processing capability is improved, but bandwidth consumption increases and data privacy concerns arise
Solution Approach 1:
The DNN model is segmented into multiple parts that can be distributed across different devices. Each device executes a portion of the DNN computations locally, reducing the need to transfer all data to a centralized cloud server while still leveraging distributed computing power.
Solution Approach 2:
The patent introduces a new dimension of feature representation by dynamically changing the dimensionality of intermediate features at split points. This allows the system to adapt feature sizes to match available network bandwidth, enabling distributed computation without proportionally increasing bandwidth consumption.
2Quantity of substance
If feature size is compressed at split points, then communication bandwidth requirements are reduced, but prediction accuracy may deteriorate
Solution Approach 1:
The patent implements dynamic feature size adaptation where the dimensionality of intermediate features is adjusted based on available network bandwidth and device resources. This dynamic approach allows the system to optimize the trade-off between communication efficiency and model accuracy by adapting feature sizes to current operating conditions rather than using a fixed compression ratio.
Solution Approach 2:
The system changes the parameter of feature dimensionality at split points between different parts of the DNN. By controlling which nodes are connected and adjusting the number of feature dimensions, the system can compress features to reduce bandwidth requirements while maintaining sufficient accuracy for the distributed computation task.
3Power
If DNN is split across multiple devices, then processing load is distributed, but coordination complexity increases
Solution Approach 1:
The DNN model is divided into distinct segments that can be independently deployed on different devices. Each device receives a specific part of the model through the receiver and executes it locally, simplifying the coordination requirements compared to distributing individual computations or data elements.
Solution Approach 2:
The patent introduces intermediate feature representations at split points that serve as mediators between different device segments. These intermediate features with adaptive dimensionality act as standardized interfaces that simplify the coordination between distributed devices, as each device only needs to handle feature transmission and reception at defined split points rather than complex multi-device synchronization.
4Productivity
If compression factor is increased, then communication throughput requirements are reduced, but loss of information increases
Solution Approach 1:
The system dynamically adjusts the compression factor based on available network bandwidth and device resources. When bandwidth is abundant, higher compression factors can be used; when bandwidth is limited, lower compression factors preserve more information. This dynamic adaptation optimizes the trade-off between communication throughput and information preservation.
Solution Approach 2:
The patent changes the parameter of feature dimensionality at split points to control the compression level. By adjusting which nodes are connected and the number of feature dimensions, the system can control the compression factor to match communication requirements while minimizing information loss necessary for maintaining model accuracy.
Data Source
AI summary
The proposed approach deals with efficient transmission for distributed AI with a provision to switch among multiple bandwidths. During the distributed inference at edge devices, each device needs to load part of the AI model only once, but the input/output features communicated between them can be flexibly configured depending on the available transmission bandwidth by enabling/disabling connection between nodes in the Dynamic feature size Switch (DySw). When some nodes are connected or disconnected in order to achieve the desired compression factor, other parameters of the DNN remain the same. That is, the same DNN model is used for different compression factors, and no new DNN model needs to be downloaded to adapt to the compression factor or the network bandwidth.