Audio output device and control method thereof

Abstract

An audio output device is disclosed. The audio output device outputs a multi-channel audio signal through a plurality of speakers arranged at different positions. The audio output device includes an input device and a processor, wherein the processor is configured to: obtain scene information related to an audio type included in the multi-channel audio signal and sound image angle information related to an angle formed by a sound image of the audio type included in the multi-channel audio signal based on a virtual user based on the multi-channel audio signal input through the input device; and generate an output signal to be output through a plurality of speakers from the multi-channel audio signal based on the obtained scene information and sound image angle information, wherein the audio type includes at least one of sound effects, shouting, music, and speech, and the number of the plurality of speakers is equal to or greater than the number of channels of the multi-channel audio signal.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims the benefit of priority from Korean Patent Application No. 10-2019-0162659 filed in the Korean Intellectual Property Office on December 9, 2019, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to an audio output device and a control method thereof. More specifically, the present disclosure relates to an audio output device that outputs a multi-channel audio signal through a plurality of speakers and a control method thereof. Background Art

[0004] As electronic technology develops, expectations for high-realistic sound are increasing with the advent of large-screen displays and high-resolution content. Therefore, a technique for providing high-realistic sound through a multi-channel speaker has been attempted, in which an additional channel (e.g., a high channel) is added to an existing stereo channel speaker.

[0005] However, since most audio content encountered in real life has stereo channels or 5.1 channels, when the audio content is played through a multi-channel speaker having a greater number of channels than the audio content, a problem of presenting sound through the multi-channel speaker may occur.

[0006] One or more aspects of the present disclosure provide a method for providing a user with a high-realistic sound by fully considering the production purpose of audio content. Summary of the invention

[0007] According to one aspect of the present disclosure, an audio output device is provided, comprising: a receiver configured to receive a multi-channel audio signal; and a processor configured to: based on the multi-channel audio signal received by the receiver, obtain scene information related to an audio type and sound image angle information related to an angle formed by a sound image of the audio type corresponding to a virtual user from the multi-channel audio signal, and based on the obtained scene information and sound image angle information, generate an output signal to be output through a plurality of speakers from the multi-channel audio signal, wherein the audio type includes at least one of sound effects, shouting, music, and speech, and the number of the plurality of speakers is equal to or greater than the number of channels of the multi-channel audio signal.

[0008] The multi-channel audio signal may include a plurality of audio frames, and the processor may be further configured to obtain scene information from each of one or more audio frames of the plurality of audio frames, and obtain sound image angle information for each frequency band for each of the one or more audio frames.

[0009] The plurality of speakers may include a plurality of first speakers having a first horizontal plane angle corresponding to the virtual user and a plurality of second speakers having a second horizontal plane angle corresponding to the virtual user. The first horizontal plane angle may be greater than the second horizontal plane angle.

[0010] Based on identifying that the audio type included in the scene information is a sound effect, the processor can also be configured to: generate a first output signal to be output through multiple first speakers by performing an inverse Fourier transform on a signal of a frequency band in which the sound and image angle information is equal to or greater than a predetermined angle among the signals of the entire frequency band of the audio frame including the sound effect; and generate a second output signal to be output through multiple second speakers by performing an inverse Fourier transform on a signal of a frequency band in which the sound and image angle information is less than a predetermined angle among the signals of the entire frequency band of the audio frame including the sound effect.

[0011] Based on identifying that the audio type included in the scene information is sound, the processor can also be configured to generate an output signal to be output through multiple first and second speakers by performing an inverse Fourier transform on the signal of the frequency band in the entire frequency band of the audio frame including the sound and the sound image angle information is not 0°.

[0012] Based on recognizing that the audio type included in the scene information is music, the processor may be further configured to generate an output signal to be output through a plurality of second speakers by performing inverse Fourier transform on a signal of an entire frequency band including an audio frame of music.

[0013] The processor may be further configured to obtain at least one audio feature about the multi-channel audio signal, and obtain scene information based on the obtained at least one audio feature.

[0014] The processor may be further configured to obtain scene information using an artificial intelligence model trained to recognize a type of audio included in the multi-channel audio signal.

[0015] The processor may be further configured to obtain the sound image angle information based on at least one of an inter-channel loudness difference (ILD) and an inter-channel time difference (ITD) of the multi-channel audio signal.

[0016] The processor may be further configured to apply a sound effect to the generated output signal based on the scene information.

[0017] A plurality of speakers are provided in the audio output device.

[0018] The audio output device may further include a communicator. The processor may be further configured to: control the communicator to output the output signal through a plurality of speakers existing outside the audio output device.

[0019] Based on recognizing that the audio type included in the scene information includes a plurality of audio types, the processor may be further configured to generate an output signal by reflecting an energy ratio occupied by each of the plurality of audio types in all of the one or more audio frames.

[0020] According to another aspect of the present disclosure, a method for controlling an audio output device is provided, the method comprising: receiving a multi-channel audio signal; acquiring scene information about an audio type from the multi-channel audio signal, and acquiring sound-image angle information about an angle formed by a sound image of the audio type corresponding to a virtual user from the multi-channel audio signal; and generating an output signal to be output through a plurality of speakers from the multi-channel audio signal based on the acquired scene information and sound-image angle information, wherein the audio type comprises at least one of sound effects, shouting, music, and speech, and the number of the plurality of speakers is equal to or greater than the number of channels of the multi-channel audio signal.

[0021] The multi-channel audio signal may include a plurality of audio frames. The scene information may be configured to be obtained from each of one or more audio frames of the plurality of audio frames, and may be configured to obtain the sound image angle information for each frequency band for each of the one or more audio frames.

[0022] The plurality of speakers may include a plurality of first speakers having a first horizontal plane angle corresponding to the virtual user and a plurality of second speakers having a second horizontal plane angle corresponding to the virtual user. The first horizontal plane angle may be greater than the second horizontal plane angle.

[0023] Based on identifying that the audio type included in the scene information is a sound effect, the generation may include: generating a first output signal to be output through multiple first speakers by performing an inverse Fourier transform on a signal of a frequency band whose sound and image angle information is equal to or greater than a predetermined angle among the signals of the entire frequency band of the audio frame including the sound effect, and generating a second output signal to be output through multiple second speakers by performing an inverse Fourier transform on a signal of a frequency band whose sound and image angle information is less than a predetermined angle among the signals of the entire frequency band of the audio frame including the sound effect.

[0024] Based on identifying that the audio type included in the scene information is shouting, the generation may include: performing an inverse Fourier transform on a signal of a frequency band including the entire frequency band of the audio frame of the sound, in which the sound image angle information is not 0°, to generate an output signal to be output through a plurality of first and second speakers.

[0025] Based on recognizing that the audio type included in the scene information is music, the generating may include performing inverse Fourier transform on a signal of an entire frequency band including an audio frame of music to generate an output signal to be output through a plurality of second speakers.

[0026] According to another aspect of the present disclosure, a non-transitory computer-readable medium is provided, which includes a program for executing a method for controlling an audio output device, the method including: receiving a multi-channel audio signal; acquiring scene information about an audio type from the multi-channel audio signal, and acquiring sound-image angle information about an angle formed by a sound image of the audio type corresponding to a virtual user from the multi-channel audio signal; and generating an output signal to be output through a plurality of speakers from the multi-channel audio signal based on the acquired scene information and sound-image angle information, wherein the audio type includes at least one of sound effects, shouting, music, and speech, and the number of the plurality of speakers is equal to or greater than the number of channels of the multi-channel audio signal.

[0027] According to another aspect of the present disclosure, an audio output device is provided, comprising: a memory storing one or more instructions; and a processor configured to execute the one or more instructions to: receive a multi-channel audio signal, obtain scene information about an audio type included in the multi-channel audio signal, obtain sound and image angle information about an angle formed by a sound image of the audio type corresponding to a virtual user from the multi-channel audio signal, and generate an output signal to be output through a plurality of speakers based on the multi-channel audio signal, the obtained scene information, and the sound and image angle information, wherein the number of the plurality of speakers is equal to or greater than the number of channels of the multi-channel audio signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a diagram illustrating an environment in which audio content is provided to an audio output device through a network according to an embodiment of the present disclosure;

[0029] Figure 2 is a block diagram of an audio output device according to an embodiment of the present disclosure;

[0030] Figure 3A is a view showing a sound image angle according to an embodiment of the present disclosure;

[0031] Figure 3B is a view showing a plurality of audio frames included in a multi-channel audio signal according to an embodiment of the present disclosure;

[0032] Figure 3C is a graph showing the sound image angle information of each frequency band according to an embodiment of the present disclosure;

[0033] Figure 4 is a functional block diagram showing a processor according to an embodiment of the present disclosure;

[0034] Figure 5A is an exemplary view of an audio output device according to an embodiment of the present disclosure;

[0035] Figure 5B Is set in Figure 5A a horizontal angled view of a plurality of speakers on an audio output device;

[0036] Figure 6 is a detailed block diagram of an audio output device according to an embodiment of the present disclosure;

[0037] Fig. 7A is a diagram illustrating an audio system including an audio output device according to another embodiment of the present disclosure;

[0038] Figure 7B It shows that the setting Fig. 7A A horizontal angled view of a plurality of speakers on an audio system;

[0039] Figure 8 is a functional block diagram showing a processor according to another embodiment of the present disclosure; and

[0040] Fig. 9 is a flowchart illustrating a method of controlling an audio output device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] An object of the present invention is to provide an audio output device capable of providing a user with high-realistic sound by providing sound in consideration of the purpose of producing audio contents via a multi-channel speaker, and a method of controlling the audio output device.

[0042] In the following description, well-known functions or constructions are not described in detail because they would make the present application unclear in unnecessary detail. In addition, redundant descriptions of the same configurations will be omitted.

[0043] The terms used in the following description are for explaining exemplary embodiments and are not intended to limit the scope. The singular form is intended to include the plural form unless the context clearly indicates otherwise.

[0044] It should be understood that throughout this specification, the term "include" and its variations, such as "comprises", indicate the presence of the features, numbers, steps, operations, components, parts or a combination thereof described in the specification, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or a combination thereof.

[0045] In the description, the terms “first”, “second”, etc. are used to describe various elements regardless of their order and / or importance, and are used to distinguish one element from other elements but do not limit the corresponding elements.

[0046] When an element (e.g., a first element) is operably or communicatively coupled or connected to another element (e.g., a second element), the element may be directly coupled to the other element, or may be coupled to the other element through another element (e.g., a third element). On the other hand, when an element (e.g., a first element) is directly coupled or directly connected to another element (e.g., a second element), there is no element between the element and the other element.

[0047] Terms used in the embodiments of the present disclosure may be construed as meanings generally known to those skilled in the art unless the terms are defined differently.

[0048] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0049] Figure 1 is a diagram showing an environment 10 in which audio content is provided to audio output devices 100 - 1 to 100 - 4 through a communication medium 5 according to an embodiment of the present disclosure.

[0050] like Figure 1 As shown, audio content (or sound source) can be provided to audio output devices 100-1 to 100-4 from a broadcast transmission station 1, a satellite 2, a content providing server 3, etc. via a communication medium 5.

[0051] The audio content may be provided to the audio output devices 100-1 to 100-4 in the form of a stereo channel audio signal or a multi-channel audio signal such as a 5.1 channel audio signal. According to an embodiment, only audio content may be provided or video content may be provided to audio content according to the type of content.

[0052] The broadcast transmission station 1 may include a transmitter or a repeater for transmitting terrestrial broadcast content. The satellite 2 may include a communication satellite for sending data or satellite broadcast content. The content providing server 3 may refer to a server on a communication network, which provides broadcast content of IPTV, broadcast content of cable television, various audio source content and VOD content.

[0053] The communication medium 5 may include an air medium or an established communication network. In this case, the communication network may include a wireless cellular network, the Internet, a wide area network (WAN), a local area network (LAN), a wired telephone network, a cable network, etc.

[0054] The audio output devices 100 - 1 to 100 - 4 include display apparatuses 100 - 1 , 100 - 2 , and 100 - 4 capable of playing or reproducing video and audio together, and an audio apparatus 100 - 3 capable of playing or reproducing only audio content.

[0055] The display devices 100 - 1 , 100 - 2 , 100 - 4 are equipped with displays such as smart TVs, monitors, smart phones, desktop computers, laptop computers, tablet computers, navigation, and digital signage to play videos and output audio through speakers.

[0056] The audio device 100-3 is an electronic device configured to play and output audio only. For example, the audio device 100-3 may include a radio device, an audio device, a recorder, a voice recognition speaker device, a CD player including a speaker, a digital audio player (DAP), an audio device for a vehicle, a home appliance with a speaker, a soundbar, or various devices capable of performing other sound output operations.

[0057] The audio output devices 100-1 to 100-4 can output multi-channel audio signals through multiple speakers arranged at different positions. Specifically, when receiving audio content through the communication medium 5, the audio output devices 100-1 to 100-4 can generate output signals based on the received audio content and output the generated output signals through multiple speakers.

[0058] According to various embodiments of the present disclosure, the number of the plurality of speakers or the number of channels of the plurality of speakers may be greater than or equal to the number of channels of the audio content (or multi-channel audio signal) transmitted to the audio output devices 100 - 1 to 100 - 4 .

[0059] As described above, when audio content is played through a speaker having a larger number of channels than the number of channels of the audio content, a high-realistic sound can be provided to the user by presenting an output in consideration of the purpose for which the audio content was produced.

[0060] Specifically, the purpose of producing content varies greatly depending on the producer, but there may be common rules for each genre. For example, in the case of the sports genre, the sound of the scene is reproduced as much as possible by recording the sound emitted from the fan's viewpoint; in the case of the music genre, the musical instruments and singing are vividly reproduced in a standard 2-channel (stereo) environment; and in the case of the movie genre, the sound matching the video is reproduced in a multi-channel environment. Therefore, when the sound presentation on the multi-channel speaker is performed by using the genre information of the audio content, the audio content can be reproduced by more closely reflecting the purpose of producing the content.

[0061] According to various embodiments of the present disclosure, the audio output devices 100 - 1 to 100 - 4 may generate output signals to be output through speakers based on scene information and sound image angle information related to received multi-channel audio signals to reflect the purpose of producing audio content.

[0062] Scene information is a sub-concept of genre information, and includes information about which of sound effects, shouts, speech, and music an audio type included in a multi-channel audio signal corresponds to. For example, audio content of the "movie" genre may include various audio types, such as speech, sound effects, shouts, music, and the like. In this case, each audio type such as sound effects, shouts, music, and speech may be a scene included in the audio content (or multi-channel audio signal). Sound image angle information may include information about an angle formed by a sound image of an audio type included in a multi-channel audio signal.

[0063] A more detailed description of the audio output devices 100 - 1 to 100 - 4 generating output signals based on scene information and sound image angle information will be described below.

[0064] On the other hand, according to an embodiment, the generated output signals may be output through a plurality of speakers provided in the audio output devices 100 - 1 to 100 - 4 , or may be output through a plurality of individual speakers installed outside the audio output devices 100 - 1 to 100 - 4 .

[0065] As described above, according to various embodiments of the present disclosure, by providing sound according to the purpose of audio content through a multi-channel speaker, a highly realistic sound may be provided to a user.

[0066] On the other hand, in the above description, it has been described that the audio content is provided from the outside of the audio output devices 100 - 1 to 100 - 4 through the communication medium 5 , but it is not limited thereto.

[0067] For example, the audio content may be provided to the audio output devices 100-1 to 100-4 via portable storage media such as a universal serial bus (USB), a secure digital (SD) memory card, or various optical storage media. In addition, the audio content may be stored in the memory of the audio output devices 100-1 to 100-4 themselves (e.g., a hard disk drive (HDD), a solid state drive (SSD), a system memory (Rom, BIOS, etc.)), and when output through a speaker is required, it may be output by the audio output devices 100-1 to 100-4.

[0068] In the following, reference will be made to Figure 2 , FIG. 3A to FIG. 3C , Figure 4 and FIG. 5A to FIG. 5B The specific operation of the audio output device according to various embodiments of the present disclosure is described in detail. Figure 2 , FIG. 3A to FIG. 3C , Figure 4 and FIG. 5A to FIG. 5BIn the description, for ease of description, it will be assumed that the multi-channel audio signal is a 2-channel audio signal and the plurality of speakers are 4-channel speakers. However, the present disclosure is not limited to 2-channel audio signals and 4-channel speakers. Thus, according to another embodiment, the number of multi-channel audio signals may be different, and the number of speakers may be different.

[0069] Figure 2 is a block diagram of an audio output device according to an embodiment of the present disclosure. Figure 2 , the audio output device 100 includes a receiver 110 and a processor 120 .

[0070] The receiver 110 may receive a multi-channel audio signal through an input interface or an input circuit and provide the received multi-channel audio signal to the processor 120. As described above, since the multi-channel audio signal may be provided to the audio output device 100 through the communication medium 5 or through an external portable storage medium, various wired and wireless communication interfaces that receive the multi-channel audio signal may perform the function of the receiver 110.

[0071] In addition, since the multi-channel audio signal may be provided to the processor 120 from the memory included in the audio output device 100 , the memory included in the audio output device 100 may perform the function of the receiver 110 .

[0072] The processor 120 controls the overall operation of the audio output device 100. In particular, when receiving a multi-channel audio signal input through the receiver 110, the processor 120 may acquire sound image information related to an angle formed by a sound image of an audio type based on a virtual user and scene information related to the audio type included in the multi-channel audio signal.

[0073] The scene information includes information on the type of audio included in the multi-channel audio signal as described above, and the audio type may include at least one of sound effects, shouting, and music.

[0074] The sound image angle refers to the angle formed by the sound image of a specific sound and the virtual user when the front direction of the virtual user is 0° based on the virtual user. The sound image is the focus of the sound actually formed. For example, in the case of music content, when it feels that the actual sound comes from the left and right speakers, but the singer's voice comes from the center, the piano's sound comes from a little bit behind the left, and the drum's sound comes from a little bit behind the right, the focus of the singer's voice, the piano's sound, and the drum's sound is felt to become the sound image. In other words, the sound image is the position where a certain sound is focused, and the angle formed by the sound image and the virtual user refers to the angle of the sound image.

[0075] Figure 3A is a view showing the sound image angle according to an embodiment of the present disclosure. Figure 3AAs shown, assuming that there is a virtual user 30, when the sound image of the sound effect included in the multi-channel audio signal is formed at a° on the left side based on the front of the user and when the sound image of the music included in the multi-channel audio signal is formed at b° on the right side relative to the front of the user, the sound image angle of the music is b°. The audio type included in the multi-channel audio signal, that is, the sound image angle of each scene (e.g., sound effects, shouting, music, speech, etc.), can be tuned or set by a producer (e.g., a sound engineer) when producing audio content. When a multi-channel audio signal including a scene with a tuned or set sound image angle is received, the processor 120 can obtain information of each scene included in the multi-channel audio signal from the received multi-channel audio signal as described below.

[0076] When scene information and image angle information of a multi-channel audio signal are obtained, the processor 120 may generate an output signal to be output through a plurality of speakers from the multi-channel audio signal based on the obtained scene information and image angle information, and output the generated output signal through the plurality of speakers.

[0077] Specifically, the processor 120 may divide the received multi-channel audio signal according to the sound image angle information and output it through a plurality of speakers. In this case, the processor 120 may generate an output signal to be output to each of the plurality of speakers from the multi-channel audio signal based on whether the sound image angle falls within a range, and output each of the generated output signals through the plurality of speakers. According to an embodiment, the range may be a predetermined range. The predetermined range may be a floor factor, but the term is not limited thereto.

[0078] Based on the sound image angle of 0°, the predetermined range may exist in pairs of left (L) and right (R). However, it is not limited to this, and the predetermined range may exist only in one of the left (L) and right (R), or may have a first range in the left (L) and a second range in the right (R), wherein the first range and the second range are different. On the other hand, the predetermined range may be applied differently according to scene information.

[0079] For example, the predetermined range may be set to have a start angle x and an end angle y with respect to the left (L) and the right (R), respectively, such as Lx_Scene° to Ly_Scene°, Rx_Scene° to Ry_Scene°, depending on the scene, but not limited thereto.

[0080] Specifically, when the audio type is a sound effect, the developer of the audio output device 100 may set the predetermined range based on radiation patterns of a plurality of speakers, a horizontal plane arrangement, or a listening test.

[0081] In addition, when the audio type is music and the multiple speakers consist of a main speaker (e.g., the second speaker to be described below) and remaining speakers (e.g., the first speaker to be described below), the predetermined range can be set to output most of the output signal from the main speaker.

[0082] In addition, when the audio type is shouting, by setting a predetermined range so that multi-channel audio signals of all sound image angles except the sound image angle of 0° are output through all multiple speakers respectively, an effect of being surrounded by shouting can be provided. On the other hand, according to an embodiment of the present disclosure, the multiple speakers may include multiple first speakers having relatively large horizontal plane angles based on the virtual user and multiple second speakers having relatively small horizontal plane angles based on the virtual user. In addition, the multiple speakers may be embedded in the audio output device 100, or may be installed externally to achieve communication with the audio output device 100.

[0083] Therefore, the processor 120 can generate output signals to be output through multiple first speakers and output signals to be output through multiple second speakers based on scene information and sound image angle information, respectively, and output the generated output signals through multiple first speakers and multiple second speakers to provide the user with sound that is faithful to the purpose of producing audio content, which will be described in more detail below.

[0084] In the following, we will Figure 3B and Figure 3C The scene information and the sound image angle information according to various embodiments of the present disclosure are described in more detail.

[0085] A multi-channel audio signal includes a plurality of audio frames. For example, a two-channel audio signal sampled at a sampling rate of 44100 Hz includes 88,200 audio frames per second combining the left and right channels.

[0086] Figure 3B 2 is a diagram illustrating a plurality of audio frames included in a multi-channel audio signal according to an embodiment of the present disclosure. Figure 3B , the two-channel audio signal includes audio frames 20-1 to 20-4 of the left L channel and audio frames 40-1 to 40-4 of the right R channel.

[0087] exist Figure 3B , with respect to audio frames 20-1 to 20-4 and 40-1 to 40-4, audio frames having the same number after a dash (-) represent audio frames sampled simultaneously. In other words, the audio frame 20-1 of the L channel and the audio frame 40-1 of the R channel, as well as the audio frame 20-2 of the L channel and the audio frame 40-2 of the R channel, respectively, represent audio frames of the L channel and the R channel sampled simultaneously, and the same is true for the remaining audio frames.

[0088] In this case, according to an embodiment of the present disclosure, the processor 120 may acquire scene information of every predetermined number of audio frames among a plurality of audio frames, thereby obtaining information about what type of audio frame each of the corresponding predetermined number of audio frames is.

[0089] For example, in the case of a 2-channel audio signal, the processor 120 may obtain scene information for each audio frame of a set (or a pair) of L and R sampled simultaneously. Figure 3B In the example of , the processor 120 may obtain scene information for audio frames 20-1 and 40-1 and for audio frames 20-2 and 40-2, respectively, and may obtain scene information for each audio frame in a group of L and R. In this case, each group of audio frames may correspond to at least one of a sound effect, a shouting sound, and music.

[0090] However, the embodiment is not limited thereto, and the processor 120 may obtain scene information of every two groups of audio frames. In this case, the processor 120 may obtain scene information of every two groups of audio frames in a manner of obtaining scene information of each of the audio frames 20-1, 40-1, 20-2, and 40-2 and each of the audio frames 20-3, 40-3, 20-4, and 40-4. In this case, each of the two groups of audio frames may correspond to at least one of a sound effect, a shout, and music.

[0091] According to an embodiment, the processor 120 may acquire scene information for every three or more audio frame groups.

[0092] On the other hand, the processor 120 may obtain the sound image angle information of each frequency band for each of a predetermined number of audio frames. Figure 3C is a graph showing the sound image angle information of each frequency band according to an embodiment of the present disclosure. Figure 3C In the above, the horizontal axis represents the frequency band, and the vertical axis represents the sound image angle. In this case, the frequency band of the horizontal axis can be an octave unit, a Mel unit, an equivalent rectangular bandwidth (ERB) unit, a critical band unit, but is not limited thereto.

[0093] For example, when scene information is obtained for each group of audio frames, such as Figure 3C As shown, the processor 120 may obtain a set of audio frames (eg, Figure 3B The audio and video angle information of the audio frames 20-1 and 40-1).

[0094] Similarly, even when scene information is obtained for every two groups of audio frames, such as Figure 3C As shown, the processor 120 may also obtain information about two groups of audio frames (eg Figure 3BThe audio and video angle information of the audio frames 20-1, 40-1, 20-2 and 40-2).

[0095] In various embodiments of the present disclosure as described above, scene information may refer to information obtained according to a unit of a predetermined number of audio frames among a plurality of audio frames included in a multi-channel audio signal, and sound image angle information may refer to information obtained according to a frequency band in each of the predetermined number of audio frames. Figure 4 The configuration and operation of the processor 120 according to various embodiments of the present disclosure are described in more detail. Figure 4 is a functional block diagram of a processor according to an embodiment of the present disclosure. Figure 4 The processor 120 may include a scene analyzer 121 , an audio / video angle analyzer 122 , and an audio mixer 123 .

[0096] The scene analyzer 121 may obtain scene information by analyzing an input signal (i.e., a multi-channel audio signal). Specifically, the scene analyzer 121 may obtain at least one audio feature for a predetermined number of audio frames among a plurality of audio frames included in the multi-channel audio signal, and may obtain scene information for the predetermined number of audio frames based on the obtained audio feature.

[0097] Audio features may include time domain features and frequency domain features, wherein the time domain features may include short term energy (STE), zero crossing rate (ZCR), low short term energy ratio (LSTER), high zero crossing rate (HZCRR), etc., and the frequency domain features may include Mel frequency cepstral coefficients (MFCC), total power spectrum, sub-band power, frequency centroid, bandwidth, pitch frequency, spectral flux (SF), etc.

[0098] For instance, in an example of obtaining scene information from each pair of L and R audio frames, the scene analyzer 121 may extract at least one of the audio features by analyzing a pair of L and R audio frames, and may identify the audio categories included in the corresponding L and R audio frames from sound effects, shouts, and music.

[0099] The specific method of extracting audio features from a multi-channel audio signal and the specific method of identifying which audio type among sound effects, shouts and music is included in a predetermined number of audio frames from the extracted audio features are not limited to the present disclosure, but any suitable method known in the art may be adopted, and a detailed description will be omitted below.

[0100] On the other hand, according to another embodiment of the present disclosure, the scene analyzer 121 may obtain scene information by using an artificial intelligence model that learns to recognize the audio type included in the multi-channel audio. The artificial intelligence model may include at least one of a deep learning model, a convolutional neural network (CNN) model, and a recurrent neural network (RNN) model.

[0101] For instance, in an example of obtaining scene information from each pair of L and R audio frames, the scene analyzer 121 may identify which audio type of sound effects, shouts, and music the corresponding L and R audio frames include by utilizing a trained CNN model and converting a pair of L and R audio frames into a spectrogram of a two-dimensional axis by calculating a matching probability using the audio type.

[0102] On the other hand, the artificial intelligence model may be included in the memory of the audio output device 100 and used by the scene analyzer 121, or may be included in a server existing outside the audio output device 100 and used by the scene analyzer 121 through communication between the server and the audio output device 100.

[0103] The image angle analyzer 122 obtains the image angle information by analyzing the input signal (ie, the multi-channel audio signal). Specifically, the image angle analyzer 122 can obtain the image angle information based on the similarity and coherence between the channels of the frequency components of the multi-channel audio signal.

[0104] In this case, the sound image angle analyzer 122 can convert the audio signal of each channel included in the multi-channel audio signal into the frequency domain through Fourier transform, and calculate the similarity and consistency between the channels of the frequency components of each frequency band in the converted frequency domain to obtain the sound image angle information about each frequency band of the multi-channel audio signal.

[0105] For example, when a stereo channel audio signal is input, the sound image angle analyzer 122 can obtain the sound image angle information for each frequency band based on at least one of the inter-channel loudness difference or interaural level difference (ILD) and the inter-channel time difference or interaural time difference (ITD) of the L channel signal and the R channel signal. When it is assumed that the audio of a certain sound reaches both ears of the user, the ILD refers to the sound level difference between the two ears, and the ITD refers to the time level difference between the two ears.

[0106] On the other hand, as described above, the sound image angle analyzer 122 may obtain the sound image angle information for each frequency band from each audio frame in units of a predetermined number of units from which scene information is obtained.

[0107] The audio mixer 123 may generate an output signal to be output through a plurality of speakers based on the scene information and the sound image angle information obtained from the scene analyzer 121 and the sound image angle analyzer 122. In this case, the audio mixer 123 may generate the output signal by performing an inverse Fourier transform (IFT) on the signal of each frequency band based on the scene information and the sound image angle information.

[0108] In the following, reference will be made to Figure 5A and Figure 5BThe operation of the audio mixer 123 is described in more detail. Figure 5A is an exemplary view of an audio output device according to an embodiment of the present disclosure. Figure 5B Is set in Figure 5A A horizontal angled view of a plurality of speakers on an audio output device.

[0109] like Figure 5A As shown, according to an embodiment of the present disclosure, the audio output device 100-5 may be a flat panel including a plurality of speakers 50-1 to 50-4 disposed at different positions on the flat panel, but is not limited thereto. In this case, the plurality of speakers 50-1 to 50-4 may have a horizontal plane angle and a vertical plane angle according to their positions.

[0110] The horizontal plane angle refers to a horizontal plane at which the plurality of speakers are respectively perceived based on the front of a virtual user when the plurality of speakers are provided at different positions to output sounds through the plurality of speakers.

[0111] Therefore, reference Figure 5B , the audio output device 100 - 5 includes a plurality of first speakers 50 - 3 and 50 - 4 having relatively large horizontal plane angles and a plurality of second speakers 50 - 1 and 50 - 2 having relatively small horizontal plane angles based on the virtual user 30 .

[0112] Information related to the horizontal plane angles of the plurality of speakers can be obtained through speaker characteristics, speaker arrangement information, listening tests, etc. Figure 5A As shown, when multiple speakers 50-1 to 50-4 are embedded in the audio output device 100-5, arrangement information of the multiple speakers 50-1 to 50-4 or information about the horizontal plane angle can be pre-stored in the memory of the audio output device 100 by the developer or manufacturer of the audio output device 100-5.

[0113] On the other hand, Figure 5A In the embodiment of the present invention, when the input audio content is a stereo channel audio signal, the plurality of second speakers 50-1 and 50-2 may be stereo channel speakers corresponding to each channel signal. In addition, the plurality of first speakers 50-3 and 50-4 may be speakers for additional channels based on the stereo channel audio signal. In this case, since the speakers for the additional channels (i.e., the plurality of first speakers 50-3 and 50-4) are arranged at a higher vertical angle than the stereo channel speakers (i.e., the second speakers) 50-1 and 50-2, the speakers for the additional channels may be referred to as high channel speakers, but are not limited thereto.

[0114] In this case, by outputting appropriate sounds through the plurality of first speakers 50 - 3 and 50 - 4 according to the audio type, the purpose of making the audio can be reflected.

[0115] For example, in the case of sound effects, it is important to correctly transmit left / right surround sound. Therefore, the left / right surround sound can be transmitted to the audience through speakers of additional channels with a relatively large horizontal plane angle.

[0116] According to an embodiment of the present disclosure, when the audio type included in the scene information is a sound effect, the audio mixer 123 can perform an inverse Fourier transform on a signal of a frequency band having sound and image angle information equal to or greater than a predetermined angle among the signals of the entire frequency band of the audio frame to generate a first output signal to be output through multiple first speakers 50-3 and 50-4, and can perform an inverse Fourier transform on a signal of a frequency band having sound and image angle information less than a predetermined angle to generate a second output signal to be output through multiple second speakers 50-1 and 50-2.

[0117] In this way, in the case of sound effects, a signal of a frequency band having a sound image angle greater than a predetermined angle is output through multiple first speakers 50-3 and 50-4, and a signal of a frequency band having a sound image angle less than a predetermined angle is output through multiple second speakers 50-1 and 50-2, and the surround sound can be appropriately transmitted to the user, wherein the first speakers 50-3 and 50-4 have a relatively large horizontal plane angle, and the second speakers 50-1 and 50-2 have a relatively small horizontal plane angle.

[0118] On the other hand, shouting in sports has a great influence on the sense of reality felt by viewers. Therefore, in addition to the pitcher and commentator's speech at the sound image angle of 0°, shouting at the sound image angle can enhance the sense of reality by outputting the shouting through the speakers of all channels set in the audio output device 100-5.

[0119] Therefore, according to another embodiment of the present disclosure, when the audio type included in the scene information is shouting, the audio mixer 123 can perform an inverse Fourier transform on the signal of the frequency band whose sound image angle is not 0° among the signals of the entire frequency band of the audio frame including the shouting, so as to generate an output signal to be output through multiple first and second speakers 50-1 to 50-4.

[0120] In this way, in the case of a shouting sound, by outputting signals of all frequency bands having sound image angles other than 0° to all the plurality of speakers 50 - 1 to 50 - 4 , respectively, it is possible to correctly convey a sense of reality to the user.

[0121] On the other hand, it may be important for audio equipment to provide sound according to the purpose of the producer of the music. In other words, if music as a stereo channel audio signal is output to the speakers of the additional channel along with being output to the stereo channel speakers, the sound image intended by the producer and the sound image felt by the audience may be different, so it is necessary to minimize the output of the speakers of the additional channel.

[0122] Therefore, according to another embodiment of the present disclosure, when the audio type included in the scene information is music, the audio mixer 123 may perform inverse Fourier transform on signals of all frequency bands of an audio frame including music to generate output signals to be output through a plurality of second speakers 50-1 and 50-2.

[0123] In the case of music, the production purpose of the music content producer can be fully reflected by outputting sound only through speakers of channels corresponding to the channels of the multi-channel audio signal (ie, stereo channel speakers 50-1 and 50-2 corresponding to the stereo channel audio signal).

[0124] On the other hand, according to an embodiment of the present disclosure, the audio mixer 123 can unconditionally output the center component with a sound image angle of 0° among the frequency band signals through multiple second speakers 50-1 and 50-2 without considering the scene information, and perform the above-mentioned processing on the ambient components of the remaining angles except the sound image angle of 0°.

[0125] In the above description, it has been described that the processor 120 directly analyzes or processes the multi-channel audio signal to obtain scene information and sound image angle information, but the embodiment is not limited thereto. For example, the processor 120 may obtain the scene information and sound image angle information by receiving the scene information and sound image angle information corresponding to the received multi-channel audio signal from an external server, and may generate an output signal based on the scene information and sound image angle information obtained from the external server, wherein the external server generates and manages the scene information and sound image angle information for the audio content.

[0126] On the other hand, the processor 120 may include a central processing unit (CPU), a microcontroller unit (MCU), a microprocessor (MICOM), an electronic control unit (ECU), or an application processor (AP) configured to control the operation of the scene analyzer 121, the sound image angle analyzer 122, and the audio mixer 123, and / or another electronic unit capable of performing various calculation processes and generating control signals (hereinafter referred to as "CPU", etc.). In this case, the CPU, etc. may be provided in the form of being integrated into at least one or a part of the scene analyzer 121, the sound image angle analyzer 122, and the audio mixer 123.

[0127] On the other hand, the scene analyzer 121, the sound and image angle analyzer 122, and the audio mixer 123 may be integrated into one or more functional modules to form the processor 120. For example, the scene analyzer 121 and the sound and image angle analyzer 122 may be integrated to form a single signal processing module, or the scene analyzer 121, the sound and image angle analyzer 122, and the audio mixer 123 may be integrated to form a single signal processing module. In this case, the signal processing module may be a digital signal processor (DSP), but is not limited thereto.

[0128] Figure 6 is a detailed block diagram of an audio output device according to an embodiment of the present disclosure. Figure 6 The audio output device 100 in Figure 2 The description of the overlapping portion of the audio output device 100 in FIG. Figure 6 The audio output device 100 includes a processor 120, a memory 130, a display 140, a user input circuit 150, a communicator 180, and a plurality of speakers 50-1 to 50-n. According to an embodiment, the Figure 6 Some configurations of the audio output device 100 are shown, and other configurations not shown may be added.

[0129] The communicator 180 performs communication with an external device and may include a wireless communicator 181 , a wired communicator 182 , and an input interface 183 .

[0130] The wireless communicator 181 can use wireless communication technology or mobile communication technology to communicate with the external broadcast transmission station 1, satellite 2, content providing server 3 and other terminal devices. Such wireless communication technology may include, for example, Bluetooth, Bluetooth low energy, controller area network (CAN) communication, Wi-Fi, Wi-Fi direct connection, ultra-wideband communication (UWB), zigbee, infrared data association (IrDA) or near field communication (NFC), etc., and mobile communication technology may include 3GPP, Wi-Max, long term evolution (LTE), 5G, etc.

[0131] The wireless communicator 181 may receive audio content from other terminal devices or servers and send the received audio content to the processor 120. For this operation, the wireless communicator 181 may be implemented using an antenna, a communication chip, a substrate, etc. that can transmit electromagnetic waves to the outside or receive electromagnetic waves transmitted from the outside.

[0132] The wired communicator 182 can communicate with the external broadcast transmission station 1, the satellite 2, the content providing server 3 and other terminal devices based on the wired communication network. The wired communication network can be implemented, for example, using physical cables, such as a pair cable, a coaxial cable, an optical fiber cable, an Ethernet cable, etc. The wired communicator 182 can receive audio content from other terminal devices or servers and send the received audio content to the processor 120.

[0133] According to an embodiment, any one of the wireless communicator 181 and the wired communicator 182 may be omitted. Therefore, the audio output device 100 may include only the wireless communicator 181 or the wired communicator 182. In addition, the audio output device 100 may include an integrated communicator that supports wireless access of the wireless communicator 181 and wired access of the wired communicator 182.

[0134] The input interface 183 may be provided to be connectable to other devices provided separately from the audio output apparatus 100, such as an external storage device, and receive audio content from the other devices to transmit the received audio content to the processor 120. For example, the input interface 183 may be a universal serial bus (USB) terminal, and may include at least one of various interface terminals such as a high-definition multimedia interface (HDMI) terminal, a Thunderbolt terminal, and the like.

[0135] exist Figure 6 In the embodiment, the plurality of speakers 50-1 to 50-n are directly connected to the processor 120 of the audio output device 100 (specifically, the audio mixer 123 of the processor 120) and are embedded in the audio output device 100, but are not limited thereto.

[0136] As described above, the output signal generated by the processor 120 may be output through the plurality of external speakers 50-1 to 50-n installed outside the audio output device 100. In this case, the plurality of external speakers 50-1 to 50-n may be connected to the audio output device 100 through the communicator 180 in various wired and wireless methods, and the plurality of output signals generated by the processor 120 may be output through the plurality of external speakers 50-1 to 50-n through the communicator 180.

[0137] In addition, according to an embodiment of the present disclosure, the communicator 180 can communicate with an external server (not shown) and obtain scene information and sound image angle information about a multi-channel audio signal from the external server, wherein the external server generates and manages scene information and sound image angle information about audio content.

[0138] The memory 130 may store audio content temporarily or non-temporarily, and transmit the audio content to the processor 120 according to a call of the processor 120. In addition, the memory 130 may store various types of information required for calculation, processing, or control operations.

[0139] For example, memory 130 may store all or part of various data, applications, filters, algorithms, etc. required for the operation of processor 120 and provide them to processor 120 as needed. Applications may be obtained through wireless communicator 181 or wired communicator 182 via an accessible electronic software distribution network.

[0140] The memory 130 may include, for example, at least one of a main memory device and an auxiliary memory device. The main memory device may be implemented using a semiconductor storage medium (e.g., ROM and / or RAM). The ROM may include, for example, a conventional ROM, EPROM, EEPROM, mask ROM, etc. The RAM may include, for example, a DRAM, SRAM, etc. The auxiliary memory device may be implemented by using at least one storage medium capable of storing data in advance or semi-permanently, such as an optical medium, a magnetic tape, a magneto-optical disk, a floppy disk, etc., wherein the optical medium includes a flash memory device, a secure digital (SD) card, a solid-state drive (SSD), a hard disk drive (HDD), a magnetic drum, a compact disk (CD), a DVD, a laser disk, etc.

[0141] On the other hand, when the plurality of speakers 50-1 to 50-n are embedded in the audio output device 100, information about the horizontal plane angles of the plurality of speakers 50-1 to 50-n may be stored in the memory 130. When the plurality of speakers 50-1 to 50-n are independently provided outside the audio output device 100, information about the horizontal plane angles of the plurality of external speakers 50-1 to 50-n may be obtained through a user's listening test or a user's input, and the information about the horizontal plane angles thus obtained may be stored in the memory 130.

[0142] exist Figure 2 In the audio output device 100 of FIG. 1 , the receiver 110 is defined as a configuration for receiving audio content (ie, a multi-channel audio signal) and providing it to the processor 120. Figure 6 In the embodiment, since the audio content is provided to the processor 120 through the communicator 180 or the memory 130, Figure 6 The communicator 180 and the memory 130 may correspond to the above Figure 2 The receiver 110 in FIG.

[0143] The display 140 displays various images. In particular, when a video signal is received through the communicator 180 or the memory 130, the processor 120 can play the video through the display 140. For this operation, the display 140 may include various types of display panels, such as a liquid crystal display (LCD) panel, an organic light emitting diode (OLED) panel, a plasma display panel (PDP), an inorganic LED panel, a micro LED panel, etc., but is not limited thereto. On the other hand, the display 140 may configure a touch screen together with a touch panel.

[0144] The user input circuit 150 may be a user interface configured to receive various user inputs. For example, the user input circuit 150 may include various buttons, a touch panel, etc., but is not limited thereto.

[0145] The processor 120 controls the overall operation of the audio output device 100. In particular, the processor 120 may execute the above Figures 2 to 5B Describes the operation.

[0146] On the other hand, according to an embodiment of the present disclosure, when a plurality of audio types are included in the scene information, the processor 120 may reflect an energy ratio (or signal strength) occupied by each of the plurality of audio types in all predetermined number of audio frames.

[0147] For example, when the input multi-channel audio signal is a stereo channel audio signal and scene information is acquired for each set of L and R audio frames, sound effects and shouting sounds may be included in the corresponding L and R audio frames. In this case, the processor 120 may identify an energy ratio or an intensity ratio of the signal occupied by each sound effect and shouting sound in the corresponding L and R frames.

[0148] For example, if the corresponding L and R frames include 70% sound effects and 30% shouting, for the signal of each frequency band, the processor 120 can generate a signal with a strength of 70% and a signal with a strength of 30% to generate an output signal, wherein the signal with a strength of 70% performs processing on the aforementioned sound effects (output through the speakers of other channels according to whether the audio and video angle is higher or lower than a predetermined angle), and the signal with a strength of 30% performs processing on the shouting (outputs signals outside 0° via the speakers of all channels).

[0149] More specifically, for example, if the sound image angle of the first frequency band f1 is equal to or greater than a predetermined angle, the processor 120 can generate an output signal for outputting 70% of the signal corresponding to the frequency band f1 through multiple first speakers 50-3 and 50-4 and for outputting 30% of the signal corresponding to the frequency band f1 through all multiple first and second speakers 50-1 to 50-4.

[0150] In addition, if the sound image angle of the second frequency band f2 is less than a predetermined angle, the processor 120 can generate an output signal for outputting 70% of the signal corresponding to the frequency band f2 through multiple first speakers 50-3 and 50-4 and 30% of the signal corresponding to the frequency band f2 through all speakers among the multiple first and second speakers 50-1 to 50-4.

[0151] On the other hand, when the audio content is sent through the memory 130 or the communicator 180, the processor 120 can decode the audio content and convert it into an uncompressed format. Decoding can refer to the process of restoring an audio signal compressed in an audio compression format to an uncompressed audio signal, and the audio compression format is, for example, MPEG Layer 3 (MP3), Advanced Audio Coding (AAC), Audio Codec-3 (AC-3), Digital Theater System (DTS), Free Lossless Audio Codec (FLAC), Windows Media Audio (WMA). If the audio content is not compressed, this decoding process can be omitted. The recovered audio signal may include one or more channels.

[0152] For example, if the audio content is a stereo channel audio signal, the at least one channel may be an L channel and an R channel. In this case, if the number of speakers 50-1 to 50-n is 4, and the information about the horizontal plane angles of the plurality of speakers 50-1 to 50-4 is as follows: Figure 5B As shown, the processor 120 can process the L channel audio signal and the R channel audio signal to perform the above Figures 2 to 5B Such operation is the same regardless of whether the plurality of speakers 50-1 to 50-4 are embedded in the audio output device 100 or disposed outside thereof.

[0153] If the audio content is a 5.1 channel audio signal, at least one channel may be 6 channels (L, R, C, LFE, Ls, Rs) including a subwoofer signal. Even in this case, if the plurality of speakers 50-1 to 50-n include speakers of additional channels equal to or greater than the 5.1 channels, the processor 120 may acquire scene information and sound image angle information about the 5.1 channel audio signal, and generate an output signal to be output through the speakers of the additional channels based on the scene information and the sound image angle information.

[0154] For example, when the multi-channel audio signal is a 5.1-channel audio signal and the speaker includes two high-channel speakers in addition to the 5.1-channel speakers, when the audio type included in the 5.0-channel audio signal is a sound effect, the processor can perform an inverse Fourier transform on the signals of all frequency bands in which the sound image angle is not 0° to generate an output signal to be output through the speakers of all channels (including the 5.1-channel speakers and the high-channel speakers).

[0155] Fig. 7A is a view showing an audio system including an audio output device according to another embodiment of the present disclosure, and Figure 7B It shows that the setting Fig. 7A An angled view of the horizontal plane of multiple speakers on an audio system.

[0156] Specifically, Fig. 7A 5.1 channel speaker environment is shown. C represents the center speaker, SW represents the subwoofer speaker, FL and FR represent the front left and front right speakers, and RL and RR represent the rear left and rear right speakers. In this case, the horizontal plane angle of each speaker is as shown in Figure 7B shown.

[0157] In such an environment, the audio output device 100-6 and multiple speakers C, SW, FL, FR, RL and RR can be connected through various wired or wireless communication methods, and the output signal generated by the audio output device 100-6 can be output through multiple speakers C, SW, FL, FR, RL and RR.

[0158] For example, when a 5.1 channel audio signal is input, the audio output device 100 - 6 may output an audio signal of each channel to a speaker of a corresponding channel.

[0159] However, for example, when a stereo channel audio signal is input, as described above, the audio output device 100-6 can obtain scene information and sound image angle information about the stereo channel audio signal, and generate an output signal to be output through a 5.1 channel speaker based on the obtained scene information and sound image angle information.

[0160] In other words, according to various embodiments of the present disclosure, the stereo channel audio signals input to the audio output devices 100 and 100-6 may be mixed into two or more output signals based on the scene information and the sound image angle information, and the generated output signals may be mixed into two or more output signals based on the scene information and the sound image angle information. Figures 2 to 5B The 4-channel speaker output is built into the audio output device 100, but it can also be output through Fig. 7A A 5.1 channel speaker output is shown mounted externally to the audio output device 100 - 6 .

[0161] exist Fig. 7AIn the embodiment, assuming that scene information is acquired for each L and R audio frame of one of a plurality of audio frames included in a stereo channel audio signal, when the audio type included in the scene information for the L and R audio frames is a sound effect, the processor 120 may perform inverse Fourier transform on a signal of a frequency band having a sound image angle less than a predetermined angle through the speakers of the FL and FR channels, and perform inverse Fourier transform on an output signal to be output through the speakers of the RL and RR channels. In this case, the predetermined angle may be appropriately and experimentally determined by a developer or manufacturer of the audio output device 100-6.

[0162] On the other hand, when the audio type included in the scene information is shouting, the processor 120 can perform inverse Fourier transform on the signal of the frequency band where the sound image angle is not 0° among the signals of the entire frequency band of the audio frame including the shouting, to generate an output signal to be output through the speakers of the FL, FR, RL and RR channels.

[0163] If the audio type included in the scene information is music, the processor 120 may perform inverse Fourier transform on a signal of the entire frequency band including an audio frame of music to generate an output signal to be output through speakers of FL and FR channels.

[0164] on the other hand, Fig. 7A The audio output device 100-6 also includes speakers for channels of specific frequencies or specific sound image angles, such as C and SW. For example, the processor 120 may perform inverse Fourier transform on signals of all frequency bands having a sound image angle of 0° to generate output signals to be output through the C channel speaker, and perform inverse Fourier transform on signals of frequency bands less than a predetermined frequency to generate output signals to be output through the SW channel speaker.

[0165] However, the embodiment is not limited thereto, and a developer or manufacturer of the audio output device 100 - 6 may configure the audio output device 100 - 6 to generate any different forms of output signals based on scene information and sound image angle information of an input stereo channel audio signal.

[0166] On the other hand, according to an embodiment of the present disclosure, the processor 120 may apply a sound effect to the output signal generated as described above based on scene information. Figure 8 is a functional block diagram of a processor according to another embodiment of the present disclosure. Figure 4 Compared with the processor 120, Figure 8 The processor 120 is the same as that of FIG. 1 , except that the processor 120 also includes a sound effect applicator 124 .

[0167] The sound effect applicator 124 can apply the sound effect to the output signal generated based on the scene information as described above. For example, when the audio type included in the scene information is music, the sound effect applicator 124 can apply a virtual stereo effect such as a head-related transfer function (HRTF) to the output signal to widen the sound stage.

[0168] In addition, when the audio type included in the scene information is shouting, the sound effect applicator 124 can apply the effect of being surrounded by shouting by adding out-of-phase components between the multiple second speakers 50-1 and 50-2 and the multiple first speakers 50-3 and 50-4, which is achieved by adding out-of-phase components between the multiple second speakers 50-1 and 50-2 and the multiple first speakers 50-3 and 50-4.

[0169] Fig. 9 is a flowchart illustrating a method of controlling an audio output device according to an embodiment of the present disclosure. Fig. 9 , when a multi-channel audio signal is received (S910), the audio output device 100 may obtain scene information related to the audio type included in the multi-channel audio signal and sound image angle information related to the sound image of the audio included in the multi-channel audio signal based on the angle of the virtual user (S920). In this case, the multi-channel audio signal may include a plurality of audio frames, wherein the scene information may be information obtained for each of a predetermined number of audio frames among the plurality of audio frames, and the sound image angle information may be information obtained for each frequency band for each of the predetermined number of audio frames.

[0170] Therefore, the audio output device 100 can generate an output signal to be output through a plurality of speakers from a multi-channel audio signal based on the obtained scene information and sound image angle information (S930). In this case, based on the virtual user, the plurality of speakers may include a plurality of first speakers having a relatively large horizontal plane angle and a plurality of second speakers having a relatively small horizontal plane angle.

[0171] On the other hand, according to an embodiment of the present disclosure, when the audio type included in the scene information is a sound effect, the audio output device 100 may perform an inverse Fourier transform (IFT) on a signal of a frequency band having sound image angle information equal to or greater than a predetermined angle to generate a first output signal to be output through a plurality of first speakers, and may perform an inverse Fourier transform on a signal of a frequency band having sound image angle information less than a predetermined angle to generate a second output signal to be output through a plurality of second speakers.

[0172] When the audio type included in the scene information is shouting, the audio output device 100 can perform inverse Fourier transform on the signal of the frequency band whose sound image angle information is not 0° among the signals of the entire frequency band of the audio frame including the shouting sound to generate an output signal to be output through the plurality of first and second speakers.

[0173] When the audio type included in the scene information is music, the audio output device 100 may perform inverse Fourier transform on signals of all frequency bands including an audio frame of music to generate output signals to be output through the plurality of second speakers.

[0174] According to various embodiments of the present disclosure as described above, by providing sound faithful to the purpose of producing audio content through a multi-channel speaker, a highly realistic sound can be provided to a user.

[0175] On the other hand, the various embodiments described above may be implemented as an S / W program including instructions stored on a machine-readable (e.g., computer-readable) storage medium. The machine is a device capable of calling the stored instructions from the storage medium and operating according to the called instructions, and may include the audio output devices 100, 100-1 to 100-6 according to the above embodiments.

[0176] When the instruction is executed by the processor, the processor may directly execute the function corresponding to the instruction or use other components under the control of the processor to execute the function corresponding to the instruction. The command may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" only means that the storage medium does not include a signal but is tangible, without distinguishing between the case where data is semi-permanently stored in the storage medium and the case where data is temporarily stored in the storage medium.

[0177] According to an exemplary embodiment, the method according to each of the above exemplary embodiments may be provided as included in a computer program product. The computer program product may be traded between a seller and a consumer as a product. The computer program product may be in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)) or through an application store (e.g., Play Store TM ) Online distribution. In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated at least in a manufacturer's server, an application store's server, or a storage medium such as a memory.

[0178] According to each exemplary embodiment, the corresponding components (e.g., modules or programs) may include a single entity or multiple entities, and some of the above-mentioned corresponding subcomponents may be omitted, or another subcomponent may be further added to each exemplary embodiment. Alternatively or additionally, some components (e.g., modules or programs) may be combined to form a single entity that performs the same or similar functions as those performed by the corresponding elements before being combined. According to each exemplary embodiment, the operations performed by modules, program modules or other components may be performed sequentially, in parallel or jointly, iteratively, or tentatively, or at least some operations may be performed in a different order, omitted, or other operations may be added.

[0179] Although the exemplary embodiments of the present disclosure have been illustrated and described, it should be understood that the present disclosure is not limited to the disclosed embodiments, and various changes can be made without departing from the spirit and scope of the present disclosure. In addition, the technical ideas of the present disclosure are not limited according to the embodiments of the present disclosure, but are used for description, and the scope of the technical ideas of the present disclosure is not limited by these embodiments. Therefore, the protection scope of the present disclosure should be interpreted by the attached claims, and all technical ideas within their equivalent scope should be interpreted as included in the scope of the present disclosure.

Claims

1. Audio output device, including: a receiver configured to receive a multi-channel audio signal; as well as Processor, configured as: Based on the multi-channel audio signal received by the receiver, scene information about an audio type is obtained from the multi-channel audio signal, and sound image angle information about an angle formed by a sound image of the audio type corresponding to a virtual user is obtained from the multi-channel audio signal, and generating an output signal to be output through a plurality of speakers from the multi-channel audio signal based on the obtained scene information and the obtained sound image angle information; The audio type includes at least one of sound effects, shouting, music and speech. The multi-channel audio signal includes a plurality of audio frames. The processor is further configured to: obtain the scene information of each of the multiple groups of audio frames from the multiple audio frames, and obtain the sound image angle information for each frequency band for each of the multiple groups of audio frames, and Each of the multiple groups of audio frames includes an audio frame corresponding to each channel of the multi-channel audio signal.

2. The audio output device according to claim 1, wherein: The plurality of speakers include a plurality of first speakers having a first horizontal plane angle corresponding to the virtual user and a plurality of second speakers having a second horizontal plane angle corresponding to the virtual user, wherein the first horizontal plane angle is greater than the second horizontal plane angle.

3. The audio output device according to claim 2, wherein: The processor is further configured to: based on identifying that the audio type included in the scene information is the sound effect, Generate a first output signal to be output through the plurality of first speakers by performing inverse Fourier transform on a signal of a frequency band in which the sound image angle information is equal to or greater than a predetermined angle among signals of the entire frequency band of the audio frame including the sound effect; as well as A second output signal to be output through the plurality of second speakers is generated by performing inverse Fourier transform on a signal of a frequency band in which the sound image angle information is smaller than the predetermined angle among signals of the entire frequency band of the audio frame including the sound effect.

4. The audio output device according to claim 2, wherein: The processor is also configured to: based on identifying that the audio type included in the scene information is the shouting sound, generate an output signal to be output through the multiple first speakers and the multiple second speakers by performing an inverse Fourier transform on the signal of the frequency band in which the sound image angle information is not 0° within the entire frequency band of the audio frame including the shouting sound.

5. The audio output device according to claim 2, wherein: The processor is further configured to generate an output signal to be output through the plurality of second speakers by performing inverse Fourier transform on a signal of an entire frequency band including an audio frame of the music based on recognizing that the audio type included in the scene information is the music.

6. The audio output device according to claim 1, wherein: The processor is further configured to obtain at least one audio feature about the multi-channel audio signal, and obtain the scene information based on the obtained at least one audio feature.

7. The audio output device according to claim 1, wherein: The processor is further configured to obtain the scene information using an artificial intelligence model trained to recognize the audio type included in the multi-channel audio signal.

8. The audio output device according to claim 1, wherein: The processor is further configured to obtain the sound image angle information based on at least one of an inter-channel loudness difference and an inter-channel time difference of the multi-channel audio signal.

9. The audio output device according to claim 1, wherein: The processor is further configured to apply a sound effect to the generated output signal based on the scene information.

10. The audio output device according to claim 1, wherein: The plurality of speakers are disposed in the audio output device.

11. The audio output device according to claim 1, further comprising: communicator, The processor is further configured to: control the communicator to output the output signal through the plurality of speakers existing outside the audio output device.

12. The audio output device according to claim 1, wherein: The processor is further configured to generate the output signal by reflecting an energy ratio occupied by each of the plurality of audio types in all of the one or more audio frames based on identifying that the audio type included in the scene information includes a plurality of audio types.

13. A method for controlling an audio output device, the method comprising: Receiving multi-channel audio signals; Obtaining scene information about an audio type from the multi-channel audio signal, and obtaining sound image angle information about an angle formed by a sound image of the audio type corresponding to a virtual user from the multi-channel audio signal; as well as generating an output signal to be output through a plurality of speakers from the multi-channel audio signal based on the obtained scene information and the obtained sound image angle information; The audio type includes at least one of sound effects, shouting, music and speech. The multi-channel audio signal includes a plurality of audio frames. Wherein, obtaining scene information about the audio type comprises: obtaining the scene information of each group of multiple groups of audio frames from the multiple audio frames, Wherein, obtaining the sound image angle information comprises: obtaining the sound image angle information for each frequency band for each group of the multiple groups of audio frames, and Each of the multiple groups of audio frames includes an audio frame corresponding to each channel of the multi-channel audio signal.

Images