Content-Aware Approximate Object Detection with Contention Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object detection systems on mobile devices face challenges in maintaining low latency and accuracy due to resource contention from concurrent applications, and are not effectively adapted to video content characteristics, violating latency requirements and reducing performance unpredictably.
Innovation Solution
A system that dynamically adjusts object detection and tracking parameters based on video content and resource contention, using a scheduler to select the most accurate and efficient configuration at runtime, incorporating a content-aware feature extractor and contention sensor to predict latency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If object detection is performed on all video frames, then detection accuracy is improved, but computational cost and latency increase excessively
Solution Approach 1:
The system performs object detection periodically at selected frames rather than continuously on all frames. A scheduler determines which frames should undergo full detection based on temporal patterns and resource availability, while intermediate frames use lighter tracking or prediction methods, thereby reducing overall computational cost while maintaining acceptable detection accuracy.
Solution Approach 2:
Instead of applying full detection to every frame, the system applies partial detection actions selectively. The scheduler identifies frames where full detection is necessary based on content changes, resource contention levels, and latency requirements, performing complete detection only when needed while using simplified methods for other frames.
2Adaptability or versatility
If resource contention from concurrent applications is not managed, then system versatility is improved, but object detection latency becomes unpredictable and violates requirements
Solution Approach 1:
The system implements a contention sensor that continuously monitors resource availability and system load. This feedback mechanism allows the scheduler to adaptively adjust detection frequency and configuration based on current resource contention levels, ensuring latency requirements are met even when multiple applications are running concurrently.
Solution Approach 2:
The object detection system transitions from a static, fixed-frequency approach to a dynamic scheduling mechanism. The scheduler continuously adjusts detection parameters based on real-time resource contention feedback, content characteristics, and latency requirements, enabling the system to maintain performance under varying system conditions and concurrent workloads.
3Device complexity
If detection parameters are fixed, then device complexity is reduced, but performance cannot adapt to varying video content and resource conditions
Solution Approach 1:
The system implements self-service through automated scheduler and contention sensor components that dynamically configure detection parameters without manual intervention. The scheduler automatically selects appropriate detection configurations based on content analysis and resource conditions, enabling the system to adapt to varying requirements while maintaining simple operation for the user.
Solution Approach 2:
The system dynamically changes detection parameters such as detection frequency, model complexity, and processing resolution based on video content characteristics and resource availability. The scheduler adjusts these parameters in real-time to optimize performance for different scenarios, from high-accuracy requirements to resource-constrained conditions, without requiring complex manual configuration.
Data Source
AI summary
System and methods for content-and contention-aware object detection are provided. A system may receive video information and perform object detection and object tracking based on an execution configuration. The system may approximate an optimized execution configuration. To approximate the optimized execution configuration, the system may identify, based on the video information, a plurality of content features. The system may further measure a contention level of a computer resource or multiple resources. The system may approximate, based on the content features and the utilization metric, latency metrics, for a plurality of execution configuration sets, respectively. The system may also approximate, based on the content features, accuracy metrics for the execution configuration sets, respectively. The system may select the optimized execution configuration set in response to satisfaction of a performance criterion. The system may perform object detection and object tracking based on the optimized execution configuration set.