Semantic Segmentation Early Exits for Low-Latency Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current semantic segmentation techniques are computation- and memory-intensive, failing to meet real-time requirements on resource-constrained devices like smartphones, leading to high latency and impacting user experience and safety in applications such as autonomous vehicles.
Innovation Solution
A two-staged training process for Multi-Exit Semantic Segmentation (MESS) networks with parametrised early exits, combining end-to-end and frozen-backbone training, and positive filtering distillation to optimize feature extraction and early exit accuracy, allowing deployment on diverse devices without retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models with numerous layers and dilated convolutions are used to maintain high-resolution feature maps and large receptive field, then semantic segmentation accuracy is improved, but inference latency increases significantly
Solution Approach 1:
The network is divided into multiple stages with progressive feature extraction. Early stages extract basic features with smaller computational cost, while later stages refine features with higher computational cost. This staged approach allows the system to achieve high accuracy through progressive refinement while enabling early exit options to reduce latency for simpler cases.
Solution Approach 2:
The system implements dynamic computation paths where the network can exit at different stages based on input complexity and accuracy requirements. For simple inputs, the network exits early with lower computational cost; for complex inputs, it proceeds through all stages to achieve maximum accuracy. This dynamic adaptation resolves the contradiction by making latency variable rather than fixed.
2Measurement precision
If computation-intensive deep learning models are deployed on resource-constrained devices, then semantic segmentation accuracy can be maintained, but real-time performance requirements are not met
Solution Approach 1:
The network performs partial computation when sufficient accuracy is achieved at intermediate stages. Instead of always executing the full computation pipeline, the system stops processing early when the accumulated features reach the accuracy threshold required for the application, thereby reducing computational load on resource-constrained devices while maintaining adequate performance.
Solution Approach 2:
The system changes computational parameters dynamically based on input characteristics and device capabilities. For resource-constrained devices, the network adjusts the depth of processing, the resolution of feature maps, and the complexity of operations to match available computational resources, thereby achieving real-time performance while preserving essential accuracy.
3Measurement precision
If feature-volume downsampling is replaced with dilated convolutions of increasing rate to preserve spatial information, then semantic segmentation quality is improved, but workload concentration deeper in the network increases
Solution Approach 1:
The computational workload is segmented across multiple stages with early exit points. Each stage performs a portion of the feature extraction and processing, allowing the system to distribute workload more evenly rather than concentrating it deeply in the network. This segmentation enables intermediate stages to handle simpler inputs independently.
Solution Approach 2:
The network introduces an additional dimension of computational depth through multiple stages and layers. By expanding the network in the depth dimension with staged processing and early exit options, the system can preserve spatial information through progressive feature extraction while distributing computational workload across multiple manageable stages rather than concentrating it in a single deep section.
Data Source
AI summary
Broadly speaking, the present techniques generally relate to a method for training a machine learning, ML, model to perform semantic image segmentation, and to a computer-implemented method and apparatus for performing semantic image segmentation using a trained machine learning, ML, model. The training method enables a semantic image segmentation ML model that is able to make predictions faster, without significant loss in accuracy. The training method also enables the ML model to be implemented on apparatus with different hardware specifications, i.e. different computational power and memory, for example.