Computer Vision Model Selection for Fog-Edge Hybrid Deployment

Overview of Technical Issues:

The inference execution module faces insufficient capability to balance accuracy and resource constraints across fog-edge layers—complex models exceed edge device computational and memory capacity causing deployment failure, while lightweight models provide inadequate detection accuracy; the goal is to establish systematic model selection criteria that optimize the trade-off between inference accuracy, latency, bandwidth consumption, and distributed computational resource utilization.

Solution directions generated for this problem

Problem Direction 1 :

ImproveInference accuracy level
VS
ConstraintModel computational complexity

Inspiration 1 : Cross-domain reference

Application Principle: #1 Segmentation
Cross-domain applicability Assess applicability
Intent identification for agent matching by assistant systems
Innovative Solution Refine solution

Hierarchical model partitioning with layer-wise edge-fog distribution for accuracy-complexity balance

Partition neural network into independent functional modules distributed across fog-edge hierarchy
How to solve :
  • Decompose detection model into three independent segments: preprocessing module (edge, 150MB), feature extraction backbone (fog, 800MB), detection head (edge, 200MB)
  • edge handles input normalization and output post-processing within 512MB limit, fog executes computationally intensive convolutional layers
  • Establish inter-layer communication protocol using compressed feature tensors (quantized to int8, 4:1 compression ratio) transmitted via MQTT with latency budget allocation: edge preprocessing ≤20ms, fog-edge transmission ≤15ms, fog computation ≤50ms, total pipeline ≤100ms
  • Implement dynamic load balancing where fog node monitors edge device CPU utilization every 500ms—when edge load exceeds 75%, offload detection head to fog backup instance, maintaining accuracy while preventing device overload
Expected Effect : Accuracy matches full model (mAP ≥0.85), edge memory usage ≤480MB, total latency ≤100ms, fog utilization +40%
Risk Control :
  • network transmission jitter causing pipeline stalls
  • feature tensor compression introducing accuracy degradation beyond 3%
  • synchronization overhead between distributed segments

Problem Direction 2 :

ImproveInference accuracy level
VS
ConstraintEdge device memory capacity

Inspiration 1 : Cross-domain reference

Application Principle: #35 Parameter changes
Cross-domain applicability Assess applicability
Integrated circuit for controlling sequence assigning method and reference signal generating method
Innovative Solution Refine solution

Adaptive precision quantization with dynamic bit-width allocation for memory-constrained inference

Dynamic precision allocation across layers
How to solve :
  • Implement layer-wise adaptive quantization—assign higher bit-width (int16/float16) to accuracy-critical layers (first 3 conv layers, final classification layer) and lower bit-width (int4/int8) to intermediate layers
  • profile each layer's sensitivity via gradient-based importance scoring, allocating precision budget proportionally
  • Deploy mixed-precision inference engine with runtime bit-width switching—load int4 weights (0.25× memory), dynamically upcast to int8/int16 during computation for sensitive operations, then downcast outputs
  • total model footprint reduced from 1.8GB to 480MB while maintaining 96-98% of float32 accuracy
  • Establish calibration-driven quantization pipeline—use 500-1000 representative samples to compute per-layer activation ranges, apply percentile clipping (99.9th) to minimize outlier impact, generate lookup tables for non-uniform quantization bins
  • validate accuracy degradation <2% before deployment, adjust bit allocation iteratively until memory constraint satisfied
Expected Effect : Memory usage 73% reduction, accuracy retention 96-98%, latency decrease 40%
Risk Control :
  • layer sensitivity profiling accuracy insufficient
  • calibration dataset representativeness inadequate
  • runtime bit-width switching overhead

Problem Direction 3 :

ImproveInference accuracy level
VS
ConstraintInference latency

Inspiration 1 : Cross-domain reference

Application Principle: #10 Preliminary action
Cross-domain applicability Assess applicability
Method for reporting capability information and dual mode user equipment adapted thereto
Innovative Solution Refine solution

Pre-computed feature cache architecture for fog-edge inference acceleration

Cache-based inference with pre-staged features
How to solve :
  • Deploy background feature pre-computation module on fog nodes during idle cycles—extract and cache static scene representations (200-300MB per scene) updated every 30 seconds, achieving 95% scene coverage
  • Edge devices load pre-cached feature maps from fog storage (10ms retrieval latency) and execute only dynamic object detection layers (40-50ms inference)—total pipeline latency reduced from 150ms full-model to 60ms cached-feature mode
  • Implement adaptive cache invalidation logic—monitor scene change rate via lightweight motion detector (5ms overhead)
  • trigger full re-computation when pixel difference exceeds 15% threshold, otherwise reuse cached features for 20-50 consecutive frames
Expected Effect : Latency reduced 60%, accuracy maintained at 96-98% of full complex model, cache hit rate 85-92%
Risk Control :
  • scene change detection false negatives
  • cache synchronization delays across fog-edge
  • memory overhead for feature storage

Problem Direction 4 :

ImproveModel selection decision precision
VS
ConstraintDistributed resource utilization efficiency

Inspiration 1 : Cross-domain reference

Application Principle: #23 Feedback
Cross-domain applicability Assess applicability
MAC multiplexing and tfc selection procedure for enhanced uplink, WTRU and base station
Innovative Solution Refine solution

Closed-loop adaptive model orchestration with real-time performance feedback

Deploy closed-loop orchestration with real-time feedback
How to solve :
  • Install lightweight monitoring agents (CPU <5%, memory <50MB overhead) on each fog-edge device reporting accuracy, latency, memory usage, bandwidth consumption every 1 second to central orchestrator
  • Implement multi-objective scoring function S = 0.4×(accuracy/target) + 0.3×(threshold/latency) + 0.2×(capacity/memory) + 0.1×(limit/bandwidth), normalized to 0-100 scale, recalculated every 5 seconds based on live feedback
  • Execute dynamic model reallocation — when device score drops below 60 or utilization exceeds 85%, orchestrator migrates model layers to underutilized nodes (utilization <50%) within 2 seconds, maintaining score ≥70 across all devices
Expected Effect : Resource utilization 40%→78%, selection precision ±3%, reallocation latency <2s
Risk Control :
  • network partition disrupts feedback loop
  • scoring weight calibration requires domain tuning
  • migration overhead during peak load

Problem Direction 5 :

ImproveDistributed resource utilization efficiency
VS
ConstraintModel computational complexity

Inspiration 1 : Cross-domain reference

Application Principle: #1 Segmentation
Cross-domain applicability Assess applicability
Edge datapath using inter-process transports for higher network layers
Innovative Solution Refine solution

Hierarchical model partitioning with layer-wise distributed execution across fog-edge infrastructure

Partition neural network into independent computational stages distributed across fog-edge layers
How to solve :
  • Decompose detection model into 4 independent execution stages: preprocessing (edge, 150MB), feature extraction (edge, 400MB), detection head (fog, 600MB), post-processing (fog, 200MB)—each stage fits device memory limits while full model capability is preserved
  • Implement stage-wise execution protocol with shared memory transport between layers—edge completes stage 1-2 within 40ms, streams intermediate tensors (compressed to 5MB via int8 quantization) to fog via gRPC, fog executes stage 3-4 in 35ms, total latency 75ms meets real-time threshold
  • Deploy resource-aware stage scheduler monitoring CPU utilization (target 65-75%), memory occupancy (≤80% capacity), and network bandwidth (≤10Mbps)—dynamically reassigns stages when edge load exceeds 80% or fog idle below 50%, maintaining balanced utilization across infrastructure
Expected Effect : Resource utilization +65% (from 40% to 75%), per-device complexity -50%, latency within 80ms threshold, accuracy maintained at 92%
Risk Control :
  • inter-stage communication latency variance
  • tensor serialization overhead accumulation
  • stage migration causing temporary service interruption

Problem Direction 6 :

ImproveModel selection decision precision
VS
ConstraintModel selection decision precision

Inspiration 1 : Cross-domain reference

Application Principle: #24 Intermediary
Cross-domain applicability Assess applicability
Method for independently controlling average optical power and extinction ratio and driving circuit
Innovative Solution Refine solution

Adaptive multi-tier model profiling with proxy metric intermediary for fog-edge selection

Establish proxy metric intermediary layer
How to solve :
  • Deploy lightweight proxy metric layer that maps easily-measured architecture features (FLOPs count, parameter count, layer depth) to performance outcomes—initial screening completes in <5 seconds per model variant, reducing measurement overhead by 95%
  • Implement two-stage profiling protocol: Stage-1 uses proxy metrics (FLOPs for latency estimation ±15%, parameters for memory ±10%) to filter top-5 candidates from 50+ variants in 30 seconds
  • Stage-2 performs precise on-device benchmarking only on finalists, measuring actual accuracy/latency/memory within ±3% tolerance over 10 minutes
  • Build calibration database storing 200+ pre-profiled model-device pairs as reference intermediaries—new model selection queries this database for nearest-neighbor performance prediction (cosine similarity ≥0.85), delivering selection decision in 2 seconds with 92% accuracy match to full measurement
Expected Effect : Selection time: 30min→2min; measurement precision: ±3%; decision accuracy: 92%
Risk Control :
  • proxy metric prediction drift over time
  • database coverage gaps for novel architectures
  • calibration overhead for new device types
Patsnap Eureka Solution