Signal Processing Apparatus for Natural Slow-Motion Audio Playback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing techniques for playing back images in slow motion result in unnatural sound experiences due to soundless periods before and after event sounds, and the prolongation of sound pitch, leading to a processed sound that differs from the actual sound.

Innovation Solution

A signal processing apparatus that synthesizes audio signals from a capturing target region and its peripheral area, generating a playback audio signal suitable for slow-motion image playback by using a first obtaining unit for sounds in a shorter period and a second obtaining unit for sounds in a longer period, ensuring continuous and natural sound playback.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If event sound is played back in synchronism with the time during which the image of the event sound is played back during slow-motion playback, then the event sound timing is accurate, but soundless periods occur before and after the event making the sound unnatural

Engineering Contradiction:

Improveevent sound timing accuracyVSAvoidsoundless periods creating unnatural sound

Core Design Contradiction:

Measurement precisionVSObject-generated harmful factors

Solution Approach 1:

The audio signal is segmented into multiple portions corresponding to different time periods: a first portion for sounds before the event, a second portion (event sound) for sounds during the event, and a third portion for sounds after the event. Each portion is processed and played back at appropriate speeds to maintain natural sound continuity while preserving event timing accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The playback speed of different audio portions is dynamically adjusted. The first and third audio portions are played back at normal speed, while the second audio portion (event sound) is played back at slow motion speed to synchronize with the slow-motion image playback, creating a natural and immersive audio experience.

Inventive Principle:

Principle #15Dynamics

2Duration of action of moving object

If the sound is prolonged while holding the pitch during slow-motion playback, then the sound duration matches the image duration, but the pitch change makes the sound processed and unnatural

Engineering Contradiction:

Improvesound duration matching image durationVSAvoidpitch change making sound processed

Core Design Contradiction:

Duration of action of moving objectVSObject-generated harmful factors

Solution Approach 1:

The audio signal is divided into multiple time-based segments (first portion before event, second portion during event, third portion after event). Each segment is processed independently, allowing the event sound to be prolonged to match the slow-motion image duration while maintaining its original pitch and natural characteristics.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different quality characteristics are applied to different audio portions. The event sound portion is prolonged in duration to match the slow-motion playback while maintaining its original pitch and natural sound quality, whereas other portions are processed accordingly to ensure overall naturalness.

Inventive Principle:

Principle #3Local quality

3Duration of action of moving object

If echo is applied to the sound during slow-motion playback, then the sound is repeated multiple times, but the processed sound differs from the actual sound making playback unnatural

Engineering Contradiction:

Improvesound repetition for continuityVSAvoidprocessed sound differing from actual sound

Core Design Contradiction:

Duration of action of moving objectVSObject-generated harmful factors

Solution Approach 1:

The audio signal is segmented into distinct portions corresponding to different time periods. The event sound portion is selectively processed and prolonged to match the slow-motion image duration without applying artificial echo effects, thereby maintaining the natural characteristics of the actual sound while ensuring continuity.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS10734029B2Signal processing apparatus, signal processing method, and non-transitory computer-readable storage medium

Publication Date: 2020.08.04 CANON KK

AI summary

A signal processing apparatus for generating a playback audio signal to be played back together with a slow-motion image based on image capturing in a capturing target region, obtains a first audio signal containing a sound in the capturing target region, and based on sound acquisition in a first period shorter than a capturing period corresponding to the slow-motion image, obtains a second audio signal containing a sound in a peripheral region of the capturing target region, and based on sound acquisition in a second period longer than the capturing period, and synthesizes the first audio signal and the second audio signal, thereby generating the playback audio signal to be played back together with the slow-motion image.