Video Stream Processing with Task-Graph Scheduler
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video analytics systems face challenges in efficiently processing and storing high-definition video streams from multiple cameras due to computational demands, resource requirements, and compliance with data privacy and storage regulations, leading to complex and costly management.
Innovation Solution
A computer-implemented method utilizing deep convolutional neural networks for video processing, combined with a task-graph scheduler and message-queuing middleware, to efficiently process and store video streams by generating lightweight data records, allowing parallel processing and decoupling video stream reception from context detection, thereby reducing resource usage and improving system resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video streams from multiple HD cameras are stored and processed for video analytics, then data extraction capabilities and analysis possibilities are improved, but storage capacity requirements and computational demands increase significantly
Solution Approach 1:
The patent segments video processing into multiple independent detection stages (object detection, attribute detection, context detection) that can be executed selectively. This allows the system to process only relevant portions of video data based on specific analysis needs, reducing overall storage and computational requirements while maintaining versatile data extraction capabilities.
Solution Approach 2:
The patent extracts only the essential video frames and detection results needed for specific analytics applications, rather than storing and processing all video data. By taking out only the relevant information (object detections, attributes, and context), the system achieves versatile analysis with significantly reduced storage requirements.
2Adaptability or versatility
If all video streams are stored continuously to meet future data extraction needs, then flexibility for future applications is improved, but resource requirements and processing complexity increase
Solution Approach 1:
The patent performs preliminary object detection and attribute extraction on video frames before storing them. This preliminary action creates a structured format with pre-processed information that can be efficiently queried and analyzed for various future applications without requiring re-processing of raw video data, thus reducing future processing complexity.
Solution Approach 2:
The patent changes the parameter representation from raw video pixels to structured detection results (object bounding boxes, attributes, confidence scores). This parameter transformation enables flexible future analysis with reduced computational complexity, as the transformed data is already in an analysis-ready format.
3Power
If video streams are transmitted online for remote analysis, then centralized processing capabilities are improved, but bandwidth consumption and transmission time increase
Solution Approach 1:
The patent extracts only the essential detection results and relevant video frames for transmission, rather than sending complete high-definition video streams. This extraction of critical information maintains centralized processing capability while dramatically reducing transmission time and bandwidth consumption.
Solution Approach 2:
The patent creates simplified copies of video data in the form of detection results and key frame extracts that contain the essential information needed for analysis. These lightweight copies can be transmitted efficiently while preserving the core analytical value, reducing transmission time without sacrificing processing capability.
4Measurement precision
If deep convolutional neural networks are deployed for multiple detection tasks, then detection accuracy and analysis quality are improved, but computational demand and processing time increase
Solution Approach 1:
The patent segments the detection pipeline into multiple specialized neural networks, each optimized for specific detection tasks (object detection, attribute detection, context detection). This segmentation allows parallel execution of independent detectors and selective activation based on analysis needs, improving overall processing speed while maintaining high detection accuracy through specialized models.
Solution Approach 2:
The patent implements partial action by allowing the system to activate only the necessary detection stages based on the specific analysis requirement. For example, if only object detection is needed, attribute and context detection can be skipped, maintaining high accuracy for the required task while improving processing speed by avoiding unnecessary computational steps.
Data Source
AI summary
A computer-implemented method of processing a video stream comprises:feeding the at least one video stream to at least two video processing detectors, each comprising a respective deep convolutional neural network trained to detect one or more of the object or a property of the object;for each of said at least two video processing detectors, receiving a respective stream of object lightweight data records from the respective video processing detector, the stream of object lightweight data records generated based on the respective detection, and representing a property of the object; andfeeding at least one stream of object lightweight data records to at least one context detector configured to perform a secondary detection based on object lightweight data of the stream of object lightweight data records,wherein the detections of the at least two video processing detectors are scheduled by a task-graph scheduler.