Non-object-based ROI Detection Neural Network for Video Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Object-based region-of-interest detection for variable bit rate encoding is computationally intensive, consuming significant processing bandwidth and power, particularly due to object classification and sorting processes in video processing systems.
Innovation Solution
Implementing a non-object-based region-of-interest detection neural network within an AI accelerator, which generates candidate non-object-based region-of-interest blocks and a threshold selection module to identify selected blocks, reducing the need for central processing unit involvement and enabling differential encoding based on a region-of-interest map.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If object-based region-of-interest detection with classification network is used, then region-of-interest identification accuracy is improved, but processing bandwidth consumption and power consumption increase significantly
Solution Approach 1:
The patent segments the region-of-interest detection process into two independent networks: a non-object-based detection network that processes video frames to identify candidate regions, and an object classification network that only processes the candidate regions identified by the first network. This segmentation reduces the overall computational load and power consumption while maintaining detection accuracy.
Solution Approach 2:
The patent extracts and removes the object classification step from the main video processing pipeline, making it an optional post-processing stage. The non-object-based detection network operates independently to identify candidate regions without requiring object classification, thereby eliminating the computational overhead of classification for all video frames.
2Measurement precision
If sorting and non-maximum suppression modules are implemented on central processing unit, then region-of-interest detection accuracy is improved, but communication bandwidth consumption between AI accelerator and CPU increases
Solution Approach 1:
The patent introduces a region-of-interest map as an intermediary data structure that consolidates detection results from the AI accelerator before transferring to the CPU. This map contains only the essential region-of-interest information needed for subsequent processing, reducing the volume of data that needs to be communicated between the AI accelerator and CPU.
3Use of energy by moving object
If non-object-based region-of-interest detection is used, then processing bandwidth consumption is reduced, but region-of-interest identification capability is simplified
Solution Approach 1:
The patent implements a dynamic processing pipeline where the object classification network is conditionally activated based on the candidate regions identified by the non-object-based detection network. This allows the system to adapt its processing capability dynamically, using full object classification only when and where needed, rather than applying it uniformly to all video frames.
Data Source
AI summary
A video processing unit can include a non-object-based region-of-interest detection neural network, a threshold selection module and a region-of-interest map generator. The non-object-based region-of-interest detection neural network can be configured to receive a video frame and generate a plurality of candidate non-object-based region-of-interest blocks. The threshold selection module can be configured to receive the plurality of candidate non-object-based region-of-interest blocks and identify a plurality of selected region-of-interest blocks based on a predetermined threshold. The region-of-interest map generator can be configured to receive the selected non-object-based region-of-interest blocks and generate a region-of-interest map.