Relevance-Sorted Video Summary Using Spectral Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video summarization methods for surveillance cameras generate either too long or confusing summaries, and automatic detection of activities of interest in video archives remains inefficient, with human performance still outperforming automated systems in accuracy and speed.
Innovation Solution
A method and system for generating video summaries sorted by relevance, using unsupervised spectral clustering based on appearance and motion features, which clusters similar activities and assigns play times to ensure more relevant activities are displayed first, reducing summary length and improving clarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional video summarization methods are used to generate video summaries from surveillance footage, then the summary includes comprehensive coverage of activities, but the summary length becomes too long and becomes confusing to browse
Solution Approach 1:
The patent segments video activities into distinct clusters based on similarity metrics (spatial, temporal, appearance, motion). By dividing the continuous video stream into discrete activity clusters, the system can selectively summarize only representative segments from each cluster, dramatically reducing overall summary length while preserving comprehensive coverage of activity types.
Solution Approach 2:
The patent extracts only the most relevant and representative activity segments from each cluster to include in the final summary. By taking out only essential representative clips rather than including all activities, the system achieves comprehensive coverage of activity categories with minimal summary duration.
2Measurement precision
If traditional video summarization methods display activities in chronological order, then the summary maintains temporal accuracy, but relevant activities are buried deeper in the summary making browsing inefficient
Solution Approach 1:
The patent dynamically reorders summary segments based on their relevance to user-defined queries or interests. Instead of fixed chronological ordering, the system adapts the display sequence by calculating relevance scores for each activity cluster and arranging them accordingly, allowing users to see most important activities first while temporal relationships are preserved within each cluster.
Solution Approach 2:
The patent changes the ordering parameter from purely temporal to a composite relevance metric that incorporates temporal information, spatial relationships, appearance similarity, and motion characteristics. This parameter transformation enables relevant activities to surface at the beginning of the summary while maintaining accurate temporal representation of events within each cluster.
3Extent of automation
If automatic video analysis methods are used to detect activities of interest, then the system provides automated detection capability, but the detection accuracy remains inferior to human observers
Solution Approach 1:
The patent introduces an intermediary clustering layer between raw video detection and final activity identification. By clustering similar detected activities and selecting representative segments, the system refines automated detection output, effectively filtering out false positives and improving overall detection accuracy to approach human-level performance while maintaining full automation.
Data Source
AI summary
Method and system for producing relevance sorted video summary are provided herein. The method may include: obtaining a source video containing a plurality of source objects; receiving features descriptive of at least some of the source objects; clustering the source objects into clusters, each cluster including source objects that are similar in respect to one of the features or a combination of the features; obtaining relevance level of the clustered source objects, respectively; generating synopsis objects by sampling respective clustered source objects; and generating a synopsis video having an overall play time shorter than the overall play time of the source video by determining a play time for each of the synopsis objects based at least partially on the respective relevance level.