Deep Neural Network Skip Connections for Dynamic Layer Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks on mobile devices face challenges in energy efficiency and memory usage, leading to high latency and suboptimal performance due to the need to execute all layers, which is not feasible with limited energy budgets.
Innovation Solution
The method involves selectively activating only the necessary layers of a deep neural network through bridging connections, allowing for partial or sequential execution, which dynamically adjusts the network's depth based on energy requirements, thereby reducing unnecessary computations and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If all layers of the deep neural network are executed sequentially, then energy consumption is reduced, but processing time increases
Solution Approach 1:
The patent applies dynamic execution by allowing the deep neural network to adaptively select the number of layers to execute based on input characteristics and energy availability. The system dynamically determines whether to execute all layers or only a subset, enabling flexible trade-off between energy consumption and processing time rather than fixed sequential execution.
Solution Approach 2:
The deep neural network is segmented into multiple executable layers that can be selectively activated. The system divides the network into a first deep neural network with fewer layers and a second deep neural network with additional layers, allowing partial execution of the full network based on energy budget and accuracy requirements.
2Use of energy by moving object
If multiple deep neural networks are optimized for specific energy budgets, then energy efficiency improves, but memory usage increases
Solution Approach 1:
The patent implements a universal deep neural network architecture that can function at multiple energy budgets by selectively executing different numbers of layers. Instead of maintaining separate optimized networks for different energy levels, a single network structure serves multiple purposes by dynamic layer selection, reducing memory requirements while maintaining energy efficiency.
3Measurement precision
If the deep neural network executes all layers, then processing accuracy improves, but latency increases
Solution Approach 1:
The system applies partial action by executing only the necessary number of layers required to meet accuracy thresholds. Rather than always executing all layers, the network dynamically determines the minimum sufficient execution depth, performing partial processing when full execution is unnecessary, thus reducing latency while maintaining required accuracy.
4Use of energy by moving object
If bridging connections are used to enable selective layer execution, then energy efficiency improves, but device complexity increases
Solution Approach 1:
Bridging connections serve as intermediary elements that enable selective execution paths through the deep neural network. These connections act as mediators between different layer subsets, allowing the system to route inputs through appropriate layer combinations based on energy budget without requiring complete architectural redesign, thus managing complexity while enabling efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method (30) for operating a deep neural network (10) comprising at least one skip connection (13a, 13b), having the steps of: selecting a first path (10a) through the deep neural network (10) along the specifiable sequence using the skip connection (13a, 13b); propagating an input variable (x) along the first path (10a); and checking whether the output variable corresponds to a specifiable criterion, wherein if the specifiable criterion is not satisfied, another path (10b, 10c) through the deep neural network (10) is selected which is at least one layer longer than the first path (10a), and in response thereto the input variable (x) is propagated along the second path (10b, 10c) while reusing the intermediate results of the first path. The invention also relates to a computer program, to a device for carrying out the method (30), and to a machine-readable storage element on which the computer program is stored.