Video Frame Encoding Using Region of Interest Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision systems face challenges in maintaining object recognition accuracy due to bandwidth consumption and visual artifacts introduced by video compression, especially when transmitting raw video data to the cloud for processing.
Innovation Solution
The system employs region of interest (ROI) maps to selectively adjust encoding parameters, prioritizing higher fidelity in areas containing significant objects and lower fidelity in non-ROI areas, using an edge processor to generate ROI maps based on expected object regions, heat maps, and prior detections, and transmitting these to the cloud for efficient object detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If video compression is applied to reduce bandwidth consumption, then transmission efficiency is improved, but visual artifacts and loss of details are introduced which affect object detection accuracy
Solution Approach 1:
The patent applies different compression quality levels to different spatial regions of the video frame. Regions containing objects of interest are encoded with higher quality (lower compression) to preserve visual details, while background regions are encoded with lower quality (higher compression) to reduce bandwidth. This is achieved by generating ROI maps that identify object locations and using these maps to adjust compression parameters locally.
Solution Approach 2:
The video frame is segmented into multiple regions based on the presence and importance of objects. The frame is divided into regions of interest (containing objects) and non-interest regions (background). Different compression strategies are applied to each segment, allowing the system to optimize the trade-off between bandwidth consumption and detection accuracy for each region independently.
2Measurement precision
If uniform high quality encoding is applied to all video regions, then object detection accuracy is maintained, but bandwidth consumption increases significantly
Solution Approach 1:
Instead of applying uniform high quality encoding across the entire video frame, the system applies high quality encoding only to local regions containing objects of interest. The compression quality is adjusted locally based on the ROI map, which identifies where objects are present. This allows the system to maintain detection accuracy in critical regions while significantly reducing overall bandwidth consumption in non-critical background regions.
3Productivity
If video compression is applied before object detection, then transmission efficiency is improved, but visual artifacts affect the accuracy of computer vision algorithms
Solution Approach 1:
The system performs preliminary object location estimation using ROI maps before the actual object detection algorithm processes the compressed video. The ROI maps are generated in advance based on expected object regions, heat maps, and prior detections. This preliminary action allows the compression process to be optimized beforehand, ensuring that regions likely to contain objects are preserved with higher quality, thereby mitigating the negative effects of compression on subsequent detection accuracy.
Solution Approach 2:
The compression quality is adjusted locally based on the preliminary ROI analysis. Regions identified as containing objects of interest are encoded with higher quality to minimize visual artifacts, while background regions use lower quality encoding. This local quality adjustment ensures that the compressed video maintains sufficient fidelity for accurate object detection in critical regions while achieving overall transmission efficiency.
Data Source
AI summary
An example apparatus for encoding video frames includes a receiver to receive video frames and a heat map from a camera and expected object regions from a video database. The apparatus also includes a region of interest (ROI) map generator to detect a region of interest in a video frame based on the expected object regions. The ROI map generator can also detect a region of interest in the video frame based on the heat map. The ROI map generator can then generate an ROI map based on the detected regions of interest. The apparatus further includes a parameter adjuster to adjust an encoding parameter based on the ROI map. The apparatus also further includes a video encoder to encode the video frame using the adjusted encoding parameter.