Hybrid Pixel and Compressed Domain Video Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed visual analytics systems face challenges with high network bandwidth consumption and increased computation time due to the need to decompress videos before analysis, as well as high I/O bandwidth requirements for transmitting decompressed data between decompression and analysis engines.
Innovation Solution
A hybrid video analytics framework that processes video streams in both pixel and compressed domains, primarily using compressed-domain analytics while intermittently using pixel-domain analytics for select frames, reducing the need for full decompression and minimizing data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If video is compressed before transmission, then network bandwidth consumption is reduced, but computation time increases due to decompression requirements
Solution Approach 1:
The patent performs only partial decompression of video frames - specifically decompressing only the luminance (Y) component while keeping chrominance (Cb, Cr) components in compressed form. This partial action approach reduces the computation time compared to full decompression while still providing sufficient visual information for analytics operations, thus resolving the contradiction between bandwidth savings and computation time.
2Measurement precision
If video is decompressed before analysis, then analysis accuracy is improved, but I/O bandwidth requirements increase for transmitting decompressed data
Solution Approach 1:
The patent applies different quality levels to different components of the video data - the luminance component is decompressed with high quality for accurate analytics, while the chrominance components remain in compressed form with lower quality. This local quality differentiation allows the system to maintain analysis accuracy where it matters most (luminance) while reducing overall I/O bandwidth requirements by keeping chrominance compressed.
3Measurement precision
If full decompression is performed, then processing accuracy is improved, but computation and memory access increase
Solution Approach 1:
The patent performs partial decompression only of the luminance component rather than full decompression of all video components. This partial action maintains sufficient processing accuracy for analytics operations while reducing computation and memory access requirements, thereby improving overall processing throughput and resolving the contradiction between accuracy and productivity.
Data Source
AI summary
In one embodiment, an apparatus comprises processing circuitry to: receive, via a communication interface, a compressed video stream captured by a camera, wherein the compressed video stream comprises: a first compressed frame; and a second compressed frame, wherein the second compressed frame is compressed based at least in part on the first compressed frame, and wherein the second compressed frame comprises a plurality of motion vectors; decompress the first compressed frame into a first decompressed frame; perform pixel-domain object detection to detect an object at a first position in the first decompressed frame; and perform compressed-domain object detection to detect the object at a second position in the second compressed frame, wherein the object is detected at the second position in the second compressed frame based on: the first position of the object in the first decompressed frame; and the plurality of motion vectors from the second compressed frame.