Neural Network Path Switching Under Real-Time Inference Deadlines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks with fixed execution paths struggle to adapt to changing time constraints in real-time inference tasks, such as object detection in autonomous vehicles, leading to inconsistent inference times and reduced responsiveness to dynamic environments.
Innovation Solution
A data processing method and apparatus that dynamically select execution paths for neural networks based on remaining time, using operators like skip and switch operators to adjust the execution of subnets and region proposals, allowing for adaptive inference within time constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fixed execution path is used in neural network inference, then the inference time is consistent and predictable, but the system cannot adapt to changing time constraints in real-time tasks
Solution Approach 1:
The patent implements dynamic execution path selection by introducing operators that can switch between different execution paths based on real-time conditions. The system dynamically adjusts the inference process by selecting from multiple possible execution paths (including skipping certain layers or subnets) depending on the remaining time and deadline requirements, making the previously static neural network execution adaptable to varying time constraints.
2Adaptability or versatility
If multiple execution paths are introduced to handle time constraints, then adaptability improves, but the device complexity increases
Solution Approach 1:
The patent segments the neural network into multiple executable units with different computational costs by introducing special operators (skip operators and switch operators). These operators divide the network execution into alternative paths, allowing the system to select appropriate segments based on time constraints. This segmentation enables adaptability without requiring complete redesign of the entire network architecture.
Solution Approach 2:
The patent introduces universal operators (skip operators and switch operators) that can be integrated into various neural network architectures to provide multiple execution paths. These operators serve multiple functions: they can skip subnets, select between different execution paths, and adapt to different time constraints, making the system multi-functional without requiring separate specialized systems for each scenario.
3Productivity
If dynamic path selection is implemented, then real-time inference capability improves, but the computational overhead increases
Solution Approach 1:
The patent performs preliminary actions by pre-defining multiple execution paths and their associated computational costs during network design. The skip operators and switch operators are pre-configured with knowledge of which paths to take under different time conditions, eliminating the need for complex real-time calculations to determine execution paths. This preliminary preparation reduces runtime overhead while maintaining real-time inference capability.
Data Source
AI summary
A data processing method and apparatus using a neural network, and an electronic device including the data processing apparatus. The data processing method includes identifying an operator that selects one of a plurality of execution paths for a portion of the neural network while sequentially executing layers included in the neural network, selecting a specific execution path, from among the plurality of execution paths, based on a remaining time that is left for an inference of the neural network, and obtaining a result of the inference of the neural network through the specific execution path.