Audio Segment Classification for Selective Dialogue Enhancement

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing media processing systems face challenges in enhancing audio streams without distorting non-speech segments, leading to a poor user experience.

Innovation Solution

A computing system dynamically and selectively enhances certain segments of an audio stream by programmatically evaluating each segment to determine if it is primarily speech or music, applying dialogue enhancement only to speech segments, and using a crossfading multiplexer for seamless transitions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If dialogue enhancement is applied throughout the entire audio stream, then speech segments are enhanced and become clearer, but non-speech segments (music, background noise) are distorted and user experience deteriorates

Engineering Contradiction:

Improvespeech clarityVSAvoiddistortion of non-speech content

Core Design Contradiction:

Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The audio stream is divided into discrete segments that are individually analyzed and processed. The system evaluates each segment to determine whether it contains primarily speech or non-speech content, then applies enhancement only to speech segments. This segmentation approach allows selective processing that enhances speech clarity while preserving non-speech content without distortion.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different processing qualities are applied to different parts of the audio stream based on content type. Speech segments receive dialogue enhancement processing to improve clarity, while non-speech segments bypass enhancement to maintain their original quality. This local quality approach ensures that enhancement is applied only where beneficial, avoiding the distortion of music and background noise.

Inventive Principle:

Principle #3Local quality

2Object-affected harmful factors

If dialogue enhancement is applied selectively based on speech detection, then non-speech content is preserved without distortion, but the complexity of the processing system increases

Engineering Contradiction:

Improvepreservation of non-speech contentVSAvoidprocessing system complexity

Core Design Contradiction:

Object-affected harmful factorsVSDevice complexity

Solution Approach 1:

The system performs preliminary evaluation of each audio segment to classify it as speech or non-speech content before applying enhancement. By pre-evaluating and categorizing segments, the system avoids the complexity of real-time conditional processing during enhancement, as the decision logic is established in advance through waveform attribute comparison against predefined thresholds.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces complex machine learning-based speech detection systems with a simpler waveform attribute analysis approach. By using basic signal processing metrics and comparison against predefined speech waveform characteristics, the system achieves speech detection functionality with reduced computational complexity and simpler system architecture.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If the system dynamically switches between enhanced and non-enhanced audio output, then the appropriate processing is applied to each segment, but seamless transitions require additional processing complexity

Engineering Contradiction:

Improvedynamic processing adaptationVSAvoidtransition processing complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system dynamically adapts its processing approach by evaluating each audio segment and selectively applying enhancement based on speech content detection. This dynamic behavior allows the system to optimize audio quality for each segment while maintaining overall coherence. The crossfading multiplexer implements smooth transitions between enhanced and non-enhanced segments, reducing the perceptual impact of processing changes.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS12300259B2Automatic classification of audio content as either primarily speech or primarily non-speech, to facilitate dynamic application of dialogue enhancement

Publication Date: 2025.05.13 ROKU INC

AI summary

A method for dynamically controlling enhancement of an audio stream is provided, where the audio stream defines a sequence of audio segments over time. Each audio segment defines a waveform having a plurality of waveform attributes. For each audio segment of the sequence of audio segments, the method includes: (i) determining a set of waveform-attribute values of the audio segment's waveform attributes, (ii) computing a first distance between the determined set of waveform-attribute values and a first predefined set of waveform-attribute values representative of speech, and computing a second distance between the determined set of waveform-attribute values and a second predefined set of waveform-attribute values representative of music, (iii) using the computed first and second distances as a basis to classify the audio segment as primarily speech or rather primarily music, and (iv) controlling, based on the classifying, whether or not to enhance the audio segment for output.