Perception-Based Multimedia Processing Using Psychophysical Clustering

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing multimedia processing algorithms often fail to optimize user experience as they rely on predefined content categories that do not directly link to human perceptions, leading to inconsistent processing results across different types of content.

Innovation Solution

A method and system for perception-based multimedia processing that automatically determines user perception by clustering multimedia data into perceptual and data clusters, using psychophysical testing and statistical techniques to model correlations between lower-level features and user experiences, allowing for dynamic and content-specific signal processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If multimedia content is classified into predefined categories (speech, music, movie), then algorithm configuration can be dynamically adapted, but user experience optimization is limited because categories do not directly link to human perceptions

Engineering Contradiction:

Improvedynamic adaptation of algorithm configurationVSAvoiduser experience optimization

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent changes the parameter basis from predefined content categories to perceptual attributes derived from psychophysical testing. By modeling correlations between lower-level features (dynamic range, cross-correlation, bandwidth) and user perceptions, the system dynamically adjusts algorithm parameters based on perceived audio characteristics rather than categorical labels, directly linking processing to human perception.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces the mechanical classification system (categorical labeling) with a perceptual modeling system based on psychophysical testing and statistical clustering. This substitution allows the system to operate in perceptual space rather than categorical space, creating a more direct link between algorithm parameters and user experience.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Manufacturing precision

If different parameter values are used for different content categories, then processing can be optimized for each category, but perceptually similar content (speech and music) may be processed differently leading to negative impact on user experience

Engineering Contradiction:

Improveprocessing optimization for content categoriesVSAvoiduser experience consistency

Core Design Contradiction:

Manufacturing precisionVSReliability

Solution Approach 1:

The patent applies local quality by adjusting processing parameters based on local perceptual characteristics of the audio signal rather than global categorical labels. By analyzing specific perceptual attributes (dynamic range, spectral properties, temporal correlations) of the current audio segment, the system applies appropriate processing parameters locally, ensuring that perceptually similar content receives similar processing regardless of category.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent makes the processing system dynamic by continuously analyzing perceptual attributes and adjusting parameters in real-time based on the current audio characteristics. Rather than applying fixed parameters based on category classification, the system dynamically adapts to the perceptual properties of each audio segment, ensuring consistency for perceptually similar content.

Inventive Principle:

Principle #15Dynamics

3Reliability

If psychophysical testing and statistical clustering are used to generate perceptual clusters, then direct link to user experience is achieved, but system complexity increases

Engineering Contradiction:

Improvelink to user experienceVSAvoidprocessing system complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by pre-conducting psychophysical testing and statistical clustering during an offline training phase to generate perceptual clusters and establish correlations between lower-level features and user perceptions. This preprocessing creates lookup tables and models that can be efficiently applied during real-time processing, reducing online computational complexity while maintaining the direct link to user experience.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces perceptual clusters as intermediaries between raw audio features and user experience. By pre-establishing the relationship between lower-level features and perceptual categories through psychophysical testing, the system creates a mediating layer that simplifies real-time processing while maintaining accuracy in linking to user experience.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10748555B2Perception based multimedia processing

Publication Date: 2020.08.18 DOLBY LABORATORIES LICENSING CORP

AI summary

Example embodiments disclosed herein relate to perception based multimedia processing. There is provided a method for processing multimedia data, the method includes automatically determining user perception on a segment of the multimedia data based on a plurality of clusters, the plurality of clusters obtained in association with predefined user perceptions and processing the segment of the multimedia data at least in part based on determined user perception on the segment. Corresponding system and computer program products are disclosed as well.