Method and apparatus for sound object following

Abstract

The present disclosure relates to a method and apparatus for processing a multimedia signal. More specifically, the present disclosure relates to a method comprising obtaining a video frame and an audio frame from the multimedia signal; obtaining at least one video object from the video frame and at least one audio object from the audio frame; determining a correlation between the at least one video object and the at least one audio object; and performing a directional rendering on a specific audio object of the at least one audio object, based on a screen location of a specific video object related with the specific audio object from among the at least one video object according to the determined correlation, and an apparatus therefor.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]Pursuant to 35 U.S.C. § 119(e), this application claims the benefit of U.S. Provisional Application No. 62 / 815,361, filed on Mar. 8, 2019, the contents of which are all hereby incorporated by reference herein in its entirety.FIELD[0002]The present disclosure relates to multimedia signal processing, and more particularly, to a method and apparatus for providing sound with a three-dimensional sound effect and a sense of immersion through an apparatus configured to output a video signal and an audio signal.BACKGROUND[0003]With advances in technology, devices equipped with a larger and higher-resolution display and multiple speakers have become widespread. In addition, research on video coding technology for transmitting and receiving more vivid images and audio coding technology for transmitting and receiving immersive audio signals has been actively conducted, and multimedia contents produced or created based on the video coding technology ...

AI Technical Summary

Benefits of technology

[0007]An object of the present disclosure is to provide a method and apparatus for localizing a sound image of an audio object based on the location of a video object related to the audio object to improve the sense of immersion regarding audio.

[0008]Another object of the present disclosure is to provide a method and apparatus for effectively determining a relationship between an audio object and a video object contained in a multimedia signal.

[0026]According to the present disclosure, the sense of immersion may be improved regarding audio by localizing a sound image of an audio object based on the location of a video object related to an audio object.

[0027]In addition, according to the present disclosure, a relationship between an audio object and a video object contained in a multimedia signal may be effectively determined.

Problems solved by technology

Accordingly, creating content based on the conventional 3D audio processing technology may take a lot of time and money.

Further, it may be difficult to apply the conventional 3D audio processing technology to content for broadcasting and real-time streaming.

Thus, 3D audio processing technologies such as Dolby Atmos are limited to some content, such as movies, and not yet applied to most broadcast and streaming content.

For audio content contained in most broadcast and streaming content, a sound image is not properly localized according to the location of the sound source or video object, and therefore the sense of immersion created is limited.

Images