Multi-Path Neural Network Inference Under Real-Time Deadlines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks with fixed execution paths struggle to adapt to changing time constraints in real-time inference tasks, such as object detection in autonomous vehicles, leading to inconsistent inference times and reduced responsiveness to dynamic environments.
Innovation Solution
The implementation of a multi-path neural network with operators like skip and switch operators that dynamically select execution paths based on remaining time, allowing for adaptive inference by skipping or executing subnets and adjusting the number of region proposals according to time constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fixed execution path is used in the neural network, then the inference time is consistent, but the system cannot adapt to changing time constraints in real-time tasks
Solution Approach 1:
The neural network is transformed from a static fixed-path structure to a dynamic multi-path structure where execution paths can be selected based on real-time conditions. The system dynamically adjusts the execution path through operators like skip and switch operators that respond to time constraints, enabling adaptation to varying inference time requirements while maintaining manageable complexity through structured path selection mechanisms.
2Productivity
If the neural network executes all layers sequentially, then complete processing is ensured, but the inference time increases and real-time performance is reduced
Solution Approach 1:
The neural network is segmented into multiple executable paths with different depths and processing intensities. Instead of forcing sequential execution of all layers, the network is divided into segments that can be selectively executed based on time constraints. This segmentation allows the system to complete essential processing faster when time is limited while maintaining the option to execute more complete processing when time permits.
Solution Approach 2:
The skip operator enables the neural network to skip certain layers or subnetworks when time constraints prevent complete execution. This allows the inference process to rush through essential layers to meet real-time deadlines, sacrificing some processing completeness for speed when necessary, while still maintaining the ability to execute fully when time allows.
3Measurement precision
If more subnets are executed to improve accuracy, then the inference quality increases, but the execution time exceeds the deadline
Solution Approach 1:
The system changes the execution parameter (number of subnets processed) dynamically based on time constraints. Instead of fixing the number of subnets to always maximize accuracy, the system adjusts this parameter in real-time, selecting fewer subnets when time is limited and more subnets when time is abundant. This parameter adaptation resolves the contradiction between accuracy and time by making both variables flexible rather than fixed.
Data Source
AI summary
A data processing method and apparatus using a neural network, and an electronic device including the data processing apparatus. The data processing method includes identifying an operator that selects one of a plurality of execution paths for a portion of the neural network while sequentially executing layers included in the neural network, selecting a specific execution path, from among the plurality of execution paths, based on a remaining time that is left for an inference of the neural network, and obtaining a result of the inference of the neural network through the specific execution path.