Dynamic Load Balancer for Real-Time Video Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video analytics applications face challenges in processing video frames in real-time due to varying requirements for resolution and format across processing elements, leading to delays and potential frame dropping, especially when computing resources are limited.
Innovation Solution
A dynamic load balancer distributes work operations across available hardware accelerators like GPUs, VICs, and DPUs, based on current load, compute requirements, and deadlines, to ensure efficient processing and prevent bottlenecks, without requiring manual tuning or configuration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video frames are processed through multiple processing elements with specific resolution and format requirements, then deep learning analytics can be performed, but processing delays are introduced and frames may be dropped
Solution Approach 1:
The patent implements dynamic load balancing that continuously monitors processing element utilization and adapts workload distribution in real-time. The system dynamically assigns video frames to different processing elements based on current load conditions, transforming static resource allocation into adaptive dynamic management to prevent processing delays and frame dropping
Solution Approach 2:
The patent introduces a load balancer as an intermediary component between video frame sources and processing elements. This mediator monitors processing element utilization and intelligently routes frames to appropriate processors, optimizing the flow through the pipeline and preventing bottlenecks that cause delays and frame loss
2Productivity
If multiple processing elements operate in parallel with different resolution and format requirements, then video analytics throughput increases, but system complexity increases
Solution Approach 1:
The patent implements a universal load balancer that can handle multiple video formats, resolutions, and processing requirements through a single centralized component. This universal interface simplifies the system architecture by providing a common entry point that routes to specialized processing elements, managing complexity centrally rather than distributing it throughout the system
3Ease of operation
If processing elements are assigned fixed workloads, then resource allocation is simple, but bottlenecks occur when some processors are overloaded while others are underutilized
Solution Approach 1:
The patent implements a feedback mechanism where the load balancer continuously monitors processing element utilization metrics and uses this information to dynamically adjust workload distribution. The system measures actual processing speeds and frame completion rates, then feeds this information back to optimize frame assignment, ensuring high utilization of all processing elements while maintaining simplicity in resource management
Data Source
AI summary
Apparatuses, systems, and techniques to balance processing load between a plurality of hardware accelerators. In at least one embodiment, operations performed on batches of frames of a video (e.g., as part of a video analytics pipeline) are distributed by a load balancer between a first hardware accelerator and a second hardware accelerator.