Video Highlight Extraction via Modular Feature Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic video clipping technologies for online classes lack interpretability and require frequent retraining, leading to high costs and workload due to their low adaptability to changing requirements.
Innovation Solution
A video highlight extraction method that analyzes teacher-student interaction features, including facial expressions, eye movements, gestures, speech recognition, and timeliness, to determine highlights through a weighted sum of feature priorities, allowing for easy adjustment of parameters without retraining the model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If end-to-end video clipping method through neural network is used, then automatic clipping is achieved, but interpretability is low and model needs retraining when requirements change
Solution Approach 1:
The patent segments the video clipping task into multiple independent modules: scene change detection, speaker recognition, keyword extraction, and highlight segmentation. Each module processes specific features separately (visual, audio, text) and combines results through weighted scoring. This modular architecture allows individual modules to be adjusted or retrained independently without affecting the entire system, resolving the contradiction between automation and adaptability.
Solution Approach 2:
The patent employs adjustable parameters including weight coefficients for different features (visual, audio, text), threshold values for highlight identification, and time interval parameters for speech segments. These parameters can be modified according to different requirements without changing the core algorithm structure. The system calculates highlight scores using configurable weight combinations, enabling flexible adaptation to different educational video types and requirements.
2Extent of automation
If end-to-end clipping model is used, then automatic clipping is achieved, but retraining workload is huge and cost is high
Solution Approach 1:
The system divides the clipping model into separate feature extraction modules (visual feature extraction, audio feature extraction, text feature extraction) and a scoring combination module. Each module can be trained and optimized independently. When requirements change, only the relevant module needs retraining, not the entire end-to-end model, significantly reducing retraining workload and cost.
Solution Approach 2:
The patent introduces an intermediary scoring combination module that integrates results from different feature extraction modules through weighted summation. This intermediary layer acts as a mediator that can be adjusted by changing weight parameters rather than retraining the entire model. The combination module processes outputs from visual, audio, and text modules independently, allowing flexible adjustment of importance weights without affecting the underlying extraction modules.
3Manufacturing precision
If manual clipping is used, then quality can be controlled, but time consumption is high
Solution Approach 1:
The system performs self-service by automatically analyzing video content through multiple feature extraction modules and independently determining highlights based on calculated scores. The automated process includes automatic scene change detection, speaker identification, keyword extraction, and highlight segmentation without requiring manual intervention. This self-service capability maintains consistent quality while dramatically reducing the time investment compared to manual clipping.
Solution Approach 2:
The patent replaces the mechanical manual clipping process with an automated information processing system that uses computational algorithms to analyze visual, audio, and text features. Instead of manual review and selection, the system uses machine-based feature extraction and scoring mechanisms to automatically identify and segment highlights, substituting human labor with automated computational processes that maintain quality standards.
Data Source
AI summary
The present disclosure relates to a video highlight extraction method and system, and a storage medium. The method includes: obtaining a to-be-processed online class video and a teacher-student interaction feature and dividing the to-be-processed online class video into a plurality of target videos; respectively analysis on pictures corresponding to all frames of a target video, to obtain a visual feature set of a student and a visual feature set of a teacher in the pictures corresponding to the frames; determining timeliness of student feedback; performing speech recognition on the speech segment corresponding to the student and the speech segment corresponding to the teacher and extracting a key word, to determine fluency of language of the teacher, fluency of language of the student, and correctness of teaching knowledge; and determining a highlight in the to-be-processed online class video according to priorities of the target videos.