Video Audio Layer Segmentation for Restricted Content Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large websites face challenges in identifying and removing restricted video content, such as copyrighted music, which often results in false positives and requires users to re-upload videos without the restricted portions, leading to potential penalties and loss of context.
Innovation Solution
A method that identifies restricted audio elements in videos and mutes them while using a speech synthesizer to generate audible speech from the corresponding speech elements, allowing users to understand the content without hearing the restricted audio, and provides text overlays for additional context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional technology flags and prohibits video upload upon detection of restricted information, then restricted audio elements are removed, but the video loses context and users face penalties
Solution Approach 1:
The audio track is segmented into multiple audio layers, where restricted audio elements (e.g., copyrighted music) are identified and isolated in specific layers. The system then mutes only the layers containing restricted content while preserving playback of layers with legitimate speech or commentary, thereby removing harmful restricted information without losing the contextual value of the video.
2Reliability
If users are required to re-upload videos without restricted portions, then restricted content is removed, but computational intensity and processing time increase
Solution Approach 1:
The system performs preliminary analysis of the audio track during the initial upload process, identifying restricted audio elements and creating a muted version of the video with restricted content removed. This preliminary action allows the video to be approved for publication without requiring the user to re-upload, significantly improving processing efficiency while ensuring content compliance.
3Reliability
If the entire audio track is muted to remove restricted content, then restricted audio elements are eliminated, but the audio experience becomes unrealistic and less engaging
Solution Approach 1:
The audio track is divided into multiple layers, allowing selective muting of only those layers containing restricted content. Layers with legitimate speech, commentary, or other non-restricted audio elements remain audible, preserving the realism and engagement of the audio experience while effectively removing restricted elements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A video is provided to viewers using a web-based platform without restricted audio, such as a copyrighted soundtrack. To do so, a video comprising at least two audio layers is received. The audio layers can include separate and distinct audio layers or a mix of audio from separate sources. A restricted audio element is identified in a first audio layer and a speech element is identified in a second audio layer. A stitched text string can be generated by performing speech-to-text on both audio layers and removing the text corresponding to the restricted audio element of the second audio layer. When playing back the video, a portion of the video is muted based on the restricted audio element. A voice synthesizer is employed to generate audible sound during the muted portion using the stitched text string.