Video Summary System Using Neural Network Key Frame Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face inefficiencies in selecting relevant information from lengthy video clips in video search engines, as existing methods either require watching substantial portions of videos or result in loss of relevant information due to compression techniques.
Innovation Solution
A computer-implemented method and system that generates a video summary by selecting key frames from a video clip, adjusting weighting factors to reduce dimensionality, and using a convolutional neural network to create a one-dimensional representation, allowing for accurate selection and reduced browsing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If frame deletion strategies are used for video compression, then browsing time is reduced, but relevant information is lost
Solution Approach 1:
The patent introduces a neural network model as an intermediary between the original video and the user. This model automatically selects key frames that capture relevant information, acting as a mediator that filters out unnecessary content while preserving important details. The selected frames serve as an intermediate representation that maintains information quality while reducing browsing time.
Solution Approach 2:
The patent replaces manual frame-by-frame browsing (mechanical user action) with an automated neural network system. The model automatically identifies and selects key frames based on their informational value, substituting the mechanical process of user scrolling and selection with an intelligent automated system that preserves relevant information while reducing time loss.
2Loss of information
If users replay large portions of video clips to find relevant information, then information completeness is improved, but time consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-selecting key frames from video clips before user interaction. The neural network model processes the entire video in advance, identifying and marking the most informative frames. When users search, they immediately access these pre-identified key frames without needing to replay large portions of the video, thus maintaining information completeness while drastically reducing time consumption.
Solution Approach 2:
The patent extracts only the essential key frames from the full video content. Instead of requiring users to view complete video clips, the system extracts and presents only the most relevant frames that contain critical information. This extraction process maintains information completeness by focusing on the most important content while eliminating redundant portions that would consume user time.
3Quantity of substance
If traditional video compression is used, then storage efficiency is improved, but quality of relevant information deteriorates
Solution Approach 1:
The patent extracts only the essential key frames needed to represent the video content, rather than compressing the entire video. This extraction approach achieves storage efficiency by storing only a small subset of critical frames while maintaining the quality of relevant information. The neural network ensures that the extracted frames preserve the most important visual and contextual details.
Solution Approach 2:
The patent changes the parameter of information representation from continuous video frames to discrete key frames selected by a neural network. This parameter change allows for more efficient storage while maintaining information quality, as the model selects frames based on their informational value rather than uniformly sampling or compressing all frames. The weighting factors and selection criteria ensure that quality is preserved in the reduced set.
Data Source
AI summary
A method is provided for receiving a selection of a key frame in a video and including the key frame within a first plurality of frames from the video. The method includes adjusting at least one weighting factor for a value entry in the first plurality of frames to obtain a second plurality of frames having at least one reduced dimensionality and obtaining a one-dimensional representation of the first plurality of frames with a plurality of weighted entry values from the second plurality of frames. Further, the method includes correlating two entry values in the one-dimensional representation to obtain a selector, storing the at least one weighting factor as a network entry in a process network when the selector picks the key frame from the first plurality of frames, and applying the process network to a third plurality of frames in the video to form a summary of the video.