Streaming Video Analysis With Audio-Visual Join Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video streaming systems face issues with the detection of digitally joined ancillary content, such as ads, which can lead to improper ad beacon triggering, transcoding failures, and inefficient resource utilization, as well as the storage of duplicate video content, resulting in poor user experience and resource waste.
Innovation Solution
Implementing methods to analyze streaming video frames for characteristics indicative of joined ancillary content, using techniques like audio and visual analysis, optical character recognition (OCR), and deep learning models to detect and prevent the transmission of such content, and fingerprinting to identify and delete duplicate content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video streaming systems transmit ancillary content without detection, then content delivery is simple and fast, but ad beacon triggering becomes inaccurate and transcoding failures occur
Solution Approach 1:
The system performs preliminary analysis of ancillary content before transmission to detect joined content patterns. By analyzing video frames and audio tracks in advance using machine learning models, the system identifies digitally joined ads and prevents them from being transmitted, ensuring accurate ad beacon triggering without requiring complex real-time intervention mechanisms.
Solution Approach 2:
An intermediary analysis system is introduced between the content source and the streaming delivery system. This intermediary component uses machine learning models to detect joined ancillary content patterns, acting as a filter that prevents problematic content from entering the delivery pipeline, thereby resolving the contradiction between reliability and complexity.
2Productivity
If duplicate video content is stored in the system, then content availability is improved, but resource utilization becomes inefficient and user experience deteriorates
Solution Approach 1:
The system implements feedback mechanisms that continuously monitor transmitted ancillary content for duplication. By analyzing content characteristics and comparing them against stored fingerprints, the system identifies duplicate content and prevents its transmission or storage, creating a feedback loop that maintains efficient resource utilization while ensuring content availability through intelligent filtering rather than brute-force storage.
Data Source
AI summary
An aspect of the disclosure is related to methods and systems configured to detect an item of ancillary video content, the item of ancillary video content comprising video frames and an audio track. A first portion of the item of ancillary video content comprising video frames and an audio track. A presence of a first feature in a video frame and/or a second feature within the audio track are detected within the first portion of the item of ancillary video content, wherein the first feature and/or the second features are indicative of a joining of two separate items of ancillary content. At least partly in response to detecting the presence of the first feature in the video frame and/or the second feature, streaming of the first item of ancillary video content to one or more client devices is inhibited.