Video Classifier Frame Selection via Similarity Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time video understanding systems face challenges in balancing processing power, power consumption, latency, and accuracy, as existing solutions often submit unnecessary video frames to classifiers, wasting computational resources and sacrificing accuracy.
Innovation Solution
A method that calculates a similarity score between current video frames and previously identified frames containing features of interest, only submitting frames with a similarity score below a threshold to the classifier, thereby reducing unnecessary computations and conserving resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If every video frame is submitted to the classifier, then detection accuracy is improved, but computational resource usage increases
Solution Approach 1:
The system performs preliminary action by computing similarity scores between current video frames and previously classified frames before submitting to the classifier. This preliminary similarity check identifies frames that are likely to contain the same features of interest, allowing the system to skip classification for redundant frames while maintaining detection accuracy for novel frames.
Solution Approach 2:
The system extracts and utilizes previously classified frames as reference data. By comparing current frames against this extracted historical data using similarity metrics, the system identifies and filters out redundant classification requests, keeping only those frames that differ sufficiently from previous frames to warrant full classification processing.
2Loss of energy
If computational resources are reduced, then energy consumption decreases, but detection accuracy deteriorates
Solution Approach 1:
The system implements feedback by using the output of previous classifications (classified frames with identified features) to inform future classification decisions. The similarity comparison mechanism creates a feedback loop where past classification results directly influence whether future frames require classification, enabling energy-efficient operation without sacrificing accuracy through intelligent frame selection.
3Speed
If frame submission frequency is increased, then feature detection speed is improved, but computational load increases
Solution Approach 1:
The system applies partial action by submitting only a subset of video frames to the classifier rather than every frame. The similarity threshold mechanism determines which frames warrant full classification processing, performing exactly enough classification actions needed to maintain detection speed while avoiding excessive computational load from processing redundant frames.
Data Source
AI summary
Disclosed in some examples are real-time video classifier systems, methods, devices, and machine readable mediums that determine whether or not to submit a current video frame to a classifier based upon a similarity score that measures a similarity between the current video frame and previous video frames found by the classifier to contain the feature of interest. For example, a similarity score may be calculated for a current video frame and a previous video frame found by the classifier to contain a feature of interest. The current video frame may be submitted to the classifier only if the similarity score is below a threshold similarity score. That is, only video frames that are sufficiently different from previous images that were found as containing the feature of interest are presented to the classifier.