Computer Vision Branch Reconfiguration for Accuracy-Latency Control

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing video object detection systems on mobile devices face challenges in maintaining consistent accuracy and latency performance due to insufficient feature extraction, inaccurate prediction, and high reconfiguration overhead, particularly when dealing with dynamically changing content and resource availability.

Innovation Solution

A cost and content-aware reconfiguration system using a scheduler that incorporates a cost-benefit analyzer and a content-aware accuracy prediction model to select the optimal execution branch configuration based on light-weight and heavy-weight features, minimizing reconfiguration costs and improving accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If dynamic reconfiguration of execution branches is implemented to adapt to changing video content and resource availability, then accuracy and latency performance can be optimized, but reconfiguration overhead increases and system stability deteriorates

Engineering Contradiction:

Improveadaptability to changing video content and resource availabilityVSAvoidconsistency of accuracy and latency performance

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system pre-extracts and stores light-weight features (such as frame size, number of objects, object motion) and heavy-weight features (such as color histograms, texture analysis) before reconfiguration is needed. These pre-computed features are stored in feature stores and can be immediately accessed during scheduling decisions, eliminating the need for real-time feature extraction and reducing reconfiguration overhead while maintaining adaptive performance

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts the execution branch configuration based on real-time analysis of video content features and available computational resources. The scheduler continuously monitors feature extraction costs, prediction accuracy, and resource availability to make optimal reconfiguration decisions, allowing the system to adapt to changing conditions while managing reconfiguration overhead through intelligent scheduling

Inventive Principle:

Principle #15Dynamics

2Adaptability or versatility

If multiple execution branches are used to provide adaptation to deep models, then accuracy and latency can be optimized, but switching overhead increases and device complexity increases

Engineering Contradiction:

Improvemodel adaptation capabilityVSAvoidcomplexity of multi-model switching

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The execution system is segmented into multiple independent execution branches, each optimized for specific accuracy-latency tradeoffs. Each branch can be independently configured and switched between based on current needs. The scheduler manages these segmented branches by selecting appropriate configurations based on pre-computed features and predicted performance, reducing the complexity of managing multiple models through systematic organization

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of managing completely different models, the system achieves adaptation by changing parameters such as input size, number of regions of interest, and other configuration settings within a unified execution framework. This parameter-based adaptation reduces switching overhead and complexity compared to loading entirely different models, while still providing the necessary flexibility to optimize accuracy and latency based on content and resource conditions

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If heavy-weight features are extracted to improve prediction accuracy, then accuracy prediction improves, but computational cost and latency increase

Engineering Contradiction:

Improveaccuracy prediction precisionVSAvoidcomputational cost

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system extracts and stores both light-weight features (minimal computational cost) and heavy-weight features (high computational cost but high accuracy) in advance. During scheduling, the system can choose to use only light-weight features when computational resources are constrained, or combine both types when accuracy is critical and resources are available. This partial usage strategy allows the system to achieve high accuracy prediction when needed while avoiding unnecessary computational cost in resource-constrained scenarios

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20260057673A1System and methods for cost and content aware reconfiguration of computer vision

Publication Date: 2026.02.26 PURDUE RES FOUND

AI summary

A system may receive video information. The system may extract light weight features from the video information. The system may select a combination of light-weight features and heavy weight feature types, where the light-weight features are extracted from the video information. The system may forecast, based on a combination of the light-weight features and the heavy weight feature types, accuracy and latency metrics for performing the object detection and tracking using a plurality of candidate branch configurations, respectively. The system may select a branch configuration from the plurality of candidate branch configurations in response to satisfaction of an optimization criterion. The system may perform object detection and tracking based on the selected branch configuration. Performing object detection and tracking may include extracting heavy weight features according to the branch configuration.