Video Query Latency Reduction via Compressed CNN Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional convolutional neural networks (CNNs) used for video analytics are expensive and slow, leading to high query latency and cost when identifying objects in large video datasets, as they require significant computational resources and are inefficient in processing redundant frames.
Innovation Solution
Implementing a system that uses compressed and specialized CNNs for ingest-time indexing of video frames, clustering similar objects, and leveraging a ground truth CNN at query time to reduce latency and cost, while maintaining high accuracy through judicious selection of system and model parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CNNs are used for video analytics queries, then classification accuracy is improved, but query latency and cost increase significantly
Solution Approach 1:
The system segments video data processing into two distinct phases: an ingest phase that creates compressed representations and indices of video frames, and a query phase that leverages these pre-computed structures. This segmentation allows the expensive ground truth CNN to be applied only to relevant segments (candidate frames) rather than all frames, reducing query latency while maintaining accuracy.
Solution Approach 2:
The system performs preliminary actions during the ingest phase by pre-computing compressed representations, feature vectors, and indices of video frames using lighter CNN models. This preliminary processing creates a structured database that enables rapid querying without requiring the full computational power of ground truth CNNs during query execution, thus reducing query latency.
2Measurement precision
If conventional CNNs are used for video analytics queries, then classification accuracy is improved, but computational cost increases significantly
Solution Approach 1:
The system applies partial action by using ground truth CNNs only on a subset of candidate frames identified through cheaper filtering mechanisms (compressed CNNs and indexing structures). Instead of applying expensive ground truth CNNs to all video frames, the system performs partial processing on relevant frames only, significantly reducing computational cost while maintaining classification accuracy for queried objects.
Solution Approach 2:
The system introduces compressed CNN representations and indexing structures as intermediaries between the video data and ground truth CNN classification. These intermediaries pre-process and filter video frames, identifying candidate frames that are then passed to ground truth CNNs. This intermediary layer reduces the number of frames requiring expensive ground truth CNN processing, lowering overall computational cost.
3Loss of time
If more GPU resources are allocated for parallel processing, then query latency is reduced, but system complexity and cost increase
Solution Approach 1:
The system segments the processing workload into ingest-time preprocessing (creating compressed representations and indices) and query-time retrieval (leveraging pre-computed structures). This segmentation reduces query-time complexity by shifting computational burden to ingest time, allowing queries to execute faster without requiring proportional increases in GPU resources during querying.
Solution Approach 2:
The system changes the temporal distribution of computational parameters by performing expensive operations during ingest rather than during queries. By transforming the system from query-time heavy processing to ingest-time heavy processing, the system achieves low query latency without requiring large numbers of GPUs during query execution, thus avoiding increased system complexity.
Data Source
AI summary
A method can include classifying, using a compressed and specialized convolutional neural network (CNN), an object of a video frame into classes, clustering the object based on a distance of a feature vector of the object to a feature vector of a centroid object of the cluster, storing top-k classes, a centroid identification, and a cluster identification, in response to receiving a query for objects of class X from a specific video stream, retrieving image data for each centroid of each cluster that includes the class X as one of the top-k classes, classifying, using a ground truth CNN (GT-CNN), the retrieved image data for each centroid, and for each centroid determined to be classified as a member of the class X providing image data for each object in each cluster associated with the centroid.