Video Special Effects via Music Feature Extraction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Users, especially those unfamiliar with video shooting functions, find it challenging to create varied videos as they can only maintain a single video effect throughout the shooting process without easy options to change it in real-time.

Innovation Solution

A method and apparatus that automatically apply special effects to videos by extracting features from background music and video content, determining the type and parameters of the effects based on their combination, and adding them to the video, including temporal and spatial information for precise application.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a single video effect is maintained throughout shooting, then the operation is simple, but the video variety and user satisfaction are limited

Engineering Contradiction:

Improvevideo varietyVSAvoidoperation simplicity

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The system automatically analyzes video content and background music features to select and apply appropriate special effects without requiring manual user intervention. The processor extracts features from both the video and background music, determines suitable effects based on these features, and applies them automatically, allowing the system to serve itself in the effect selection process.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system changes the parameters of video effects dynamically based on extracted features from background music and video content. By analyzing acoustic features, temporal information, and spatial information, the system adjusts effect parameters in real-time to match the content being captured, enabling video variety while maintaining simple operation.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If the user directly changes effects at needed points in time, then the video can be customized, but the operation becomes complex for unfamiliar users

Engineering Contradiction:

Improveeffect customizationVSAvoiduser familiarity

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The system performs automatic effect selection and application by analyzing video and music features, eliminating the need for users to manually change effects. The processor autonomously determines appropriate effects based on extracted features and applies them at suitable temporal and spatial positions without requiring user knowledge or intervention.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The manual mechanical process of selecting and changing effects is replaced by an automated system that uses feature extraction and analysis. Instead of users manually navigating effect options, the system substitutes this mechanical interaction with automated processing based on video and music feature analysis.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Manufacturing precision

If multiple special effects are applied dynamically, then the video quality improves, but the processing complexity increases

Engineering Contradiction:

Improvevideo qualityVSAvoidprocessing complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The processing system is segmented into distinct functional modules: feature extraction from video, feature extraction from background music, effect determination based on combined features, and effect application. This segmentation allows complex processing to be divided into manageable steps, reducing overall system complexity while maintaining high video quality through multiple dynamic effects.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system employs a universal feature extraction and analysis mechanism that handles both video and background music inputs through common processing pathways. The same processor and feature extraction algorithms are used for both input types, reducing complexity by avoiding separate specialized systems while still enabling multiple dynamic effects to be applied based on combined analysis.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables dynamic and varied video effects based on the combination of background music and video features, allowing for real-time adjustments and enhancing user experience by automatically applying background music and special effects, even during live video recording.

Implementation Method 1

extracting an acoustic feature of a frequency domain from the background music by converting the time domain of the background music to the frequency domain of the background music through a fast Fourier transform (FFT)

Methodology Applied

Scientific EffectFast Fourier transform:

Data Source

PatentUS11563902B2Method and apparatus for providing special effects to video

Publication Date: 2023.01.24 KAKAO CORP

AI summary

A method of providing a special effect includes, in response to the selection of background music to be applied to a video, applying the background music and a special effect associated with the background music to the video based on a first feature extracted from the background music and a second feature extracted from the video.