Media Segment Identification via Audio-Visual Cross-Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for identifying media segments in digital media content streams are computationally expensive and prone to errors due to alterations in broadcast streams, such as resizing or color filtering, especially in real-time high-definition television broadcasts.
Innovation Solution
A system and method that combines audio and video analysis to produce a unique identifier for media segments by reducing images to a manageable size and comparing corresponding pixels, using audio landmarks to determine landmark-linked-image-comparisons, which can be efficiently matched against a database of previously identified segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pixel by pixel comparison is used to match images, then measurement precision is improved, but computational cost increases significantly
Solution Approach 1:
The patent segments the image comparison process by dividing images into blocks and further into sub-blocks. Instead of comparing all pixels globally, the method performs hierarchical segmentation where coarse-level block comparisons first identify potential matches, followed by finer sub-block comparisons only in candidate regions. This segmentation reduces the overall computational burden while maintaining matching precision through multi-resolution analysis.
Solution Approach 2:
The patent applies local quality by using different comparison strategies for different regions. The method performs exhaustive pixel-by-pixel comparison only in candidate regions identified by coarser methods, while using block-based or feature-based comparison in other areas. This localized application of high-precision comparison maintains accuracy where needed while reducing computational cost in less critical regions.
2Measurement precision
If pixel by pixel comparison is used to match images, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent reduces device complexity through segmentation by organizing the comparison system into hierarchical levels. The first level performs block-based comparison using simplified metrics, the second level performs sub-block comparison in candidate regions, and the third level performs detailed pixel comparison only where necessary. This segmented architecture reduces memory requirements and processing complexity compared to a single-stage exhaustive comparison system.
Solution Approach 2:
The patent extracts and processes only the necessary portions of images for detailed comparison. Instead of loading and comparing entire high-resolution images, the method extracts candidate regions based on coarser analysis, then performs detailed pixel-by-pixel comparison only on these extracted sub-regions. This extraction approach reduces memory usage and processing complexity while maintaining matching precision.
3Adaptability or versatility
If traditional image comparison methods are used, then reliability is reduced due to alterations, but adaptability is improved
Solution Approach 1:
The patent applies local quality by using different comparison approaches for different image regions and characteristics. The method identifies stable regions that are less susceptible to broadcast alterations and performs more rigorous comparison on these regions, while using more tolerant comparison methods on regions prone to compression or filtering artifacts. This region-specific approach maintains reliability while adapting to various broadcast alterations.
Solution Approach 2:
The patent performs preliminary actions by conducting coarse-level block and sub-block comparisons before detailed pixel comparison. This preliminary analysis identifies candidate regions that are likely matches, allowing the system to focus detailed comparison resources on these candidates while filtering out obvious non-matches early. This preliminary filtering action improves both reliability and efficiency in the presence of broadcast alterations.
4Speed
If real-time processing is implemented for high definition television, then speed is improved, but computational cost increases
Solution Approach 1:
The patent enables real-time processing by segmenting the computational workload into hierarchical stages with progressively finer resolution. The first stage performs rapid block-based comparison at low computational cost, the second stage performs sub-block comparison on identified candidates, and the third stage performs detailed pixel comparison only on final candidates. This segmentation allows the system to process high definition television streams in real-time by distributing computational effort across multiple resolution levels.
Solution Approach 2:
The patent applies partial action by performing detailed pixel-by-pixel comparison only on candidate regions identified by coarser methods, rather than exhaustive comparison of entire images. The method performs block comparison on all image blocks, sub-block comparison on selected candidate blocks, and full pixel comparison only on final candidate pairs. This partial application of high-precision comparison achieves real-time performance while maintaining identification accuracy.
Data Source
AI summary
A system and method for identifying media segments using audio augmented image cross-comparisoning is disclosed, in which a media segment identifying system analyzes both audio and video content, producing a unique identifier to compare with previously identified media segments in a media segment database. The characteristic landmark-linked-image-comparisones are constructed by first identifying pairs of audio landmarks separated by a characteristic, or landmark, time. Digital images associated the audio landmarks are then image comparisoned, with the combination providing a characteristic landmark-linked-image-comparison. The audio landmarks are audio peaks that exceed predetermined thresholds. A landmark-time is the time between adjacent pairs of audio peaks. The pair of images associated with the audio peaks are reduced in pixel size and converted to gray scale. Corresponding pixels are compared to form a numeric comparison. One image may be mirrored before comparison to reduce the possibility of null comparisones.