Video Hot Clip Extraction via Frame Filtering and Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting hot clips from videos during multiple speed play or skip play are inaccurate, leading to missed content or repeated viewing, and result in wastage of memory resources and bandwidth due to inefficient processing.
Innovation Solution
A method involving two optimization processes: filtering out repetitive image frames using a pre-trained depth learning model and scoring frames based on user viewing information to select and stitch relevant frames, resulting in a concise and accurate hot clip.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If multiple speed play or skip play is used to quickly watch videos, then time efficiency is improved, but accuracy of watching hot clips deteriorates
Solution Approach 1:
The system performs preliminary actions by pre-processing the video to extract keyframes and generate a simplified video representation before the user watches. It pre-identifies potential hot clips and prepares them for accurate retrieval, so that when the user watches at different speeds or skips around, the system can still accurately identify and present the correct hot clip content without requiring the user to watch at normal speed.
Solution Approach 2:
The system creates a simplified copy or representation of the video containing only the essential hot clip information extracted through keyframe analysis and user behavior modeling. This copy preserves the critical content while removing redundant portions, allowing users to access hot clips accurately even when viewing the original video through speed play or skip play operations.
2Device complexity
If traditional hot clip extraction methods are used, then simplicity of the system is maintained, but accuracy of hot clip extraction deteriorates
Solution Approach 1:
The system segments the video into discrete keyframes at regular intervals, then further segments these keyframes into potential hot clips based on user viewing behavior analysis. This segmentation allows the complex task of accurate hot clip extraction to be broken down into manageable steps: extracting keyframes, analyzing user behavior patterns, identifying hot clips, and generating the final simplified video representation.
Solution Approach 2:
The system introduces an intermediary processing layer between the original video and the final hot clip output. This intermediary layer includes the keyframe extraction module, user behavior analysis module, and hot clip identification module, which collectively transform the raw video into an accurate simplified representation without requiring the entire system to be overly complex.
3Loss of information
If redundant frames are included in hot clips, then completeness of content is improved, but memory resources and bandwidth are wasted
Solution Approach 1:
The system extracts only the essential information from the video by identifying and retaining only those keyframes that users actually watched and interacted with. It removes redundant frames and portions of the video that do not contain valuable hot clip content, thereby taking out only the necessary information while discarding the rest to save memory resources and bandwidth.
Solution Approach 2:
The system changes the parameter of video representation from the original full-resolution video to a simplified version containing only extracted keyframes and hot clip segments. By adjusting the density and quality parameters of the video representation based on user behavior analysis, the system maintains content completeness for important segments while reducing the overall data size to minimize resource consumption.
Data Source
Figure 1~2
Figure 3
AI summary
The present disclosure provides a method and apparatus of extracting a hot clip in a video, an electronic device, and a computer-readable storage medium, which relates to a field of multimedia processing technology. The method of extracting the hot clip in the video of the present disclosure is implemented to include: acquiring a video to be processed; filtering image frames with repeated content in the video to be processed, so as to obtain a simplified clip of the video to be processed; scoring each image frame in the simplified clip according to user viewing information associated with the video to be processed; and stitching image frames with scores meeting a preset condition sequentially to obtain a stitching result, and determining the stitching result as a hot clip of the video to be processed. The present disclosure may improve accuracy of extracting the hot clip and reduce a size of the hot clip, so that a waste of user memory resources and bandwidth may be reduced.