Video Scene Detection Using Optimal Sequence Grouping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video scene detection models face limitations in accuracy due to the reliance on deterministic algorithms and the scarcity of annotated data, which hampers the development of fully functional machine learning models, particularly in effectively grouping video shots into scenes using keyframe similarity and quantization methods.
Innovation Solution
The integration of optimal sequence grouping with supervised machine learning to enhance scene division, utilizing a neural network that combines visual and audio features through a learning pipeline, where the neural network is trained with optimal sequence grouping to improve the robustness and accuracy of scene division.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deterministic algorithms are used for video scene detection, then the model can operate without annotated data, but the accuracy of scene detection is limited
Solution Approach 1:
The patent combines deterministic scene detection algorithms with supervised machine learning models into a unified system. The deterministic algorithm provides initial scene boundaries without requiring annotated data, while the supervised learning model refines these boundaries using available annotations, thereby achieving both operational capability and high accuracy.
Solution Approach 2:
The deterministic algorithm performs preliminary scene detection to generate initial scene boundaries before the supervised learning model processes the data. This preliminary action provides a foundation that reduces the amount of annotated data needed for training while maintaining accuracy.
2Measurement precision
If supervised machine learning is used to improve scene detection accuracy, then more annotated data is required, but annotated data is scarce
Solution Approach 1:
The patent uses the deterministic algorithm to perform the majority of scene detection work, requiring only partial supervision from annotated data. The supervised learning model is trained on a smaller subset of annotated data to refine specific aspects of scene boundaries rather than learning from scratch, thereby achieving high accuracy with limited annotated data.
Solution Approach 2:
The deterministic algorithm acts as an intermediary that transforms the supervised learning problem into a refinement task. Instead of requiring the model to learn scene boundaries from scratch with大量 annotated data, the deterministic algorithm provides preliminary boundaries that the model only needs to refine, reducing the annotated data requirement.
3Productivity
If keyframe similarity and quantization methods are used for grouping shots, then the process is computationally efficient, but the grouping accuracy into scenes is insufficient
Solution Approach 1:
The patent segments the video processing into two stages: first, the deterministic algorithm efficiently groups shots using keyframe similarity and quantization methods; second, the supervised learning model refines these groups to achieve accurate scene boundaries. This segmentation maintains computational efficiency while improving grouping accuracy.
Solution Approach 2:
The deterministic grouping method performs preliminary shot organization based on keyframe similarity, creating an efficient initial structure. The supervised learning model then applies preliminary actions to refine this structure into accurate scene boundaries, combining efficiency with precision.
Data Source
AI summary
A video scene detection machine learning model is provided. A computer device receives feature vectors corresponding to audio and video components of a video. The computing device provides the feature vectors as input to a trained neural network. The computing device receives from the trained neural network, a plurality of output feature vectors that correspond to shots of the video. The computing device applies optimal sequence grouping to the output feature vectors. The computing device further trains the trained neural network based, at least in part, on the applied optimal sequence grouping.