Dynamic Throttling of Video Processing Using Machine Learning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Live video systems face performance degradation in frame rate, resolution, and bit rate due to insufficient compute and storage resources when processing video streams for object detection, classification, and tracking, leading to reduced image quality and dropped frames.

Innovation Solution

Implementing workload-based dynamic throttling by sharing system resources between video processing and streaming, monitoring performance metrics, and adjusting parameters such as frame rate, resolution, and computation accuracy to maintain quality of service (QoS) and prevent uncontrolled performance degradation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If system resources are allocated exclusively to video processing, then processing performance is improved, but streaming quality degrades

Engineering Contradiction:

Improvevideo processing performanceVSAvoidstreaming quality

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The system dynamically adjusts resource allocation between video processing and streaming based on real-time workload conditions. A workload-based dynamic throttling mechanism modifies processing parameters such as frame rate, resolution, and computation accuracy to maintain quality of service while adapting to varying system loads, preventing both resource exhaustion and uncontrolled performance degradation

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes operational parameters including frame rate, resolution, and computation accuracy based on workload conditions. By adjusting these parameters dynamically, the system maintains acceptable performance levels for both processing and streaming under varying resource availability, resolving the contradiction between processing productivity and streaming quality

Inventive Principle:

Principle #35Parameter changes

2Manufacturing precision

If system resources are allocated exclusively to video streaming, then streaming quality is improved, but processing performance degrades

Engineering Contradiction:

Improvestreaming qualityVSAvoidvideo processing performance

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The system implements dynamic resource allocation that responds to workload conditions by adjusting the balance between streaming and processing. When processing demand increases, the system dynamically throttles processing functions to preserve streaming quality, while allowing processing to scale when resources are available

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system monitors system workload and performance metrics in real-time, using this feedback to dynamically adjust resource allocation. This closed-loop control ensures that streaming quality is maintained while processing performance adapts to available resources, preventing either function from degrading uncontrollably

Inventive Principle:

Principle #23Feedback

3Productivity

If video processing workload increases, then processing capability is improved, but system performance degradation occurs

Engineering Contradiction:

Improveprocessing capabilityVSAvoidsystem performance

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system applies preliminary anti-action by implementing dynamic throttling before system performance degradation becomes severe. By proactively adjusting processing parameters based on workload monitoring, the system prevents frame drops and maintains reliability while allowing processing capability to increase up to sustainable limits

Inventive Principle:

Principle #9Preliminary anti-action

Solution Approach 2:

The system provides beforehand cushioning by maintaining a buffer of acceptable performance levels through dynamic parameter adjustment. When workload increases, the system gradually adjusts frame rate, resolution, and accuracy parameters to absorb the increased load while maintaining system stability and preventing catastrophic performance failure

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

Data Source

PatentUS11689750B2Workload-based dynamic throttling of video processing functions using machine learning

Publication Date: 2023.06.27 NVIDIA CORP

AI summary

Embodiments of the present disclosure relate to workload-based dynamic throttling of video processing functions. Systems and methods are disclosed that dynamically throttle video processing and/or streaming based on a workload. Live video is captured from one or more sources (e.g., cameras) and stored. The video is then provided to a video processing engine and a video streaming engine. The video processing engine may perform one or more operations such as object detection, object tracking, and object classification to produce characterization data (e.g., bounding boxes, object trajectories, alerts, object labels, object counts, boundary crossings, intersection highlighting, etc.). System resource usage and performance of the video processing and streaming are monitored to produce workload data (e.g., metrics). Based on the policies and the workload data, the video streaming and/or processing is dynamically reconfigured by adjusting parameters provided to the video streaming and processing engines.