Spatial audio parameter signaling

Abstract

An apparatus comprising means for: determining, for two or more loudspeaker channel audio signals (102), at least one spatial audio parameter (108, 310) for providing spatial audio reproduction; determining at least one audio signal relationship parameter (112) between the two or more loudspeaker channel audio signals (102), the at least one audio signal relationship parameter (112) being associated with at least one coherence parameter (112), whereby the at least one coherence parameter (112) provides at least one piece of inter-channel coherence information between the two or more loudspeaker channel audio signals (102) for the at least two frequency bands, so as to provide the at least one piece of inter-channel coherence information between the two or more loudspeaker channel audio signals (102) based on the at least one spatial audio parameter (108, 108 '). 310) and at least one audio signal relationship parameter (112) to reproduce the two or more loudspeaker channel audio signals (102); and transmitting the at least one spatial audio parameter (108, 310) and transmitting at least one piece of information associated with the at least one piece of inter-channel coherence information using the at least one determination value (112).

Description

[0001] This application is a divisional application of the Chinese invention patent application with the invention name “Spatial Audio Parameter Signaling” (application number 201980050085.5, application date May 29, 2019). Technical Field

[0002] The present application relates to apparatus and methods for signaling of spatial audio parameters, but not exclusively to signaling of spatial coherence and directional and spherical sector parameters. Background Art

[0003] Parametric spatial audio processing is a field of audio signal processing in which a set of parameters is used to describe the spatial aspects of sound. For example, in parametric spatial audio capture from a microphone array, estimating a set of parameters from the microphone array signal, such as the direction of the sound in the frequency band, and the ratio of the directional to non-directional parts of the captured sound in the frequency band, is a typical and effective choice. It is well known that these parameters describe the perceived spatial characteristics of the captured sound at the position of the microphone array. These parameters can be used accordingly in the synthesis of spatial sound for binaural headphones, speakers, or other formats such as Ambisonics.

[0004] Therefore, the direction and direct-to-total energy ratio in a frequency band is a particularly effective parameterization for spatial audio capture.

[0005] A parameter set including a directional parameter in a frequency band and an energy ratio parameter in a frequency band (indicating the directionality of the sound) can also be used as spatial metadata for an audio codec. For example, these parameters can be estimated from an audio signal captured by a microphone array and other input formats, and, for example, a stereo signal can be generated from the microphone array signal to be transmitted together with the spatial metadata. The stereo signal can be encoded, for example, with an EVS (in a dual-mono configuration) or an AAC encoder. The corresponding decoder can decode the audio signal into a PCM signal and process the sound in the frequency band (using the spatial metadata) to obtain a spatial output, for example, a binaural output.

[0006] The aforementioned solution is particularly applicable to encoding captured spatial sound from microphone arrays (e.g., in mobile phones, VR cameras, stand-alone microphone arrays). It may be desirable for such an encoder to be able to encode metadata parameters to more accurately convey relevant aspects of the input audio signal. Summary of the invention

[0007] According to a first aspect, an apparatus is provided, comprising means for: determining, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determining at least one audio signal relationship parameter between the two or more speaker channel audio signals, the at least one audio signal relationship parameter being associated with at least one coherence parameter, whereby the at least one coherence parameter provides at least one inter-channel coherence information between the two or more speaker channel audio signals for at least two frequency bands, so as to reproduce the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one audio signal relationship parameter; and transmitting the at least one spatial audio parameter and transmitting at least one information associated with the at least one inter-channel coherence information using at least one determined value.

[0008] The component for sending is also used to send at least one audio signal relationship parameter, and the component for sending at least one information associated with at least one inter-channel coherence information using at least one determined value can be used to send at least one of the following: at least one orientation of at least one coherence parameter; at least one width of at least one coherence parameter; and at least one range of at least one coherence parameter.

[0009] The at least one determined value may include at least one of: at least one directional code; at least one width code; and at least one range code.

[0010] The means for determining, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction may be configured to determine, for the two or more speaker channel audio signals, at least one directional parameter and / or at least one energy ratio.

[0011] The component may also be configured to determine a transmission audio signal from two or more speaker channel audio signals, wherein the two or more speaker channel audio signals may be reproduced based on at least one spatial audio parameter, at least one coherence parameter, and / or the transmission audio signal.

[0012] The component for determining at least one coherence parameter between two or more speaker channel audio signals can be used to determine an extended coherence parameter, wherein the extended coherence parameter can be determined based on inter-channel coherence information between two or more speaker channel audio signals that are spatially adjacent to the identified speaker channel audio signal, and the identified speaker channel audio signal is identified based on at least one spatial audio parameter.

[0013] The component for determining the extended coherence parameter may also be used to: determine a stereoscopic parameter associated with an indication that two or more speaker channel audio signals are coherently reproduced using two speaker channel audio signals that are spatially adjacent to the identified speaker channel audio signal, the identified speaker channel audio signal being the speaker channel audio signal that is spatially closest to at least one directional parameter; determine a coherence panning parameter associated with an indication that two or more speaker channel audio signals are coherently reproduced using at least two or more speaker channel audio signals that are spatially adjacent to the identified speaker channel audio signal; and generate the extended coherence parameter based on the stereoscopic parameter and the coherence panning parameter.

[0014] The component for generating extended coherence parameters based on stereo parameters and coherence translation parameters can also be used to: determine a main direction analysis to identify a speaker closest to at least one direction parameter; search from the direction of the identified speaker, and each search area includes an angle from 0 degrees to 180 degrees in a series of angular steps; estimate the average coherence value between the defined main speaker channel and any speaker channel within the search area; based on the average coherence value, determine a substantially constant coherence area; set an extended range to twice the maximum coherence area; and define a coherence translation parameter based on the extended range.

[0015] The means for defining the coherence panning parameter based on the maximum coherence region may be used to: determine the speaker closest to the at least one directional parameter; determine the speaker closest to the at least one directional parameter; determine the normalized coherence c between the speaker and all speakers within the maximum coherence region; a,i ; Ignore speakers with energy below a threshold energy; select minimum coherence from the remaining speakers; determine energy distribution parameters based on the energy distribution among the remaining speakers; multiply the energy distribution parameters by the maximum coherence area to determine the coherence translation parameters.

[0016] The component for determining stereo parameters can also be used to: determine a main direction analysis to identify a speaker closest to at least one direction parameter; search from the direction of the identified speaker, and each search ring is defined by an angle from 0 degrees to 180 degrees in a series of angle steps; estimate the average coherence value and the average energy value of all speakers located near the search ring; determine the maximum coherence ring angle based on the average coherence value and the average energy value; set the extension range to twice the maximum coherence ring angle; and define the stereo parameters based on the extension range.

[0017] A component for defining a stereo parameter based on an extended range can be used to: identify a speaker having maximum energy on a maximum coherence loop; determine a normalized coherence between the identified speaker and other speakers on the maximum coherence loop; determine an average value of the normalized coherences weighted by the corresponding energies; determine an energy ratio of the energy on the maximum coherence loop to the energy within the maximum coherence loop; and multiply the energy ratio by the average value of the normalized coherences to form a stereo parameter.

[0018] According to a second aspect, a method for spatial audio signal processing is provided, comprising: determining, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determining at least one audio signal relationship parameter between the two or more speaker channel audio signals, the at least one audio signal relationship parameter being associated with at least one coherence parameter, whereby the at least one coherence parameter provides at least one inter-channel coherence information between the two or more speaker channel audio signals for at least two frequency bands, so as to reproduce the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one audio signal relationship parameter; and sending the at least one spatial audio parameter and sending at least one information associated with the at least one inter-channel coherence information using at least one determined value.

[0019] Using at least one determined value to transmit at least one information associated with at least one inter-channel coherence information may include transmitting at least one of: at least one orientation of at least one coherence parameter; at least one width of at least one coherence parameter; and at least one range of at least one coherence parameter.

[0020] The at least one determined value may include at least one of: at least one directional code; at least one width code; and at least one range code.

[0021] For the two or more speaker channel audio signals, determining at least one spatial audio parameter for providing spatial audio reproduction may include determining at least one directional parameter and / or at least one energy ratio for the two or more speaker channel audio signals.

[0022] The method may include determining a transmission audio signal from two or more speaker channel audio signals, wherein the two or more speaker channel audio signals may be reproduced based on at least one spatial audio parameter, at least one coherence parameter, and / or the transmission audio signal.

[0023] Determining at least one coherence parameter between two or more speaker channel audio signals may include determining an extended coherence parameter, wherein the extended coherence parameter may be determined based on inter-channel coherence information between two or more speaker channel audio signals that are spatially adjacent to an identified speaker channel audio signal, the identified speaker channel audio signal being identified based on at least one spatial audio parameter.

[0024] Determining the extended coherence parameter may include: determining a stereoscopic parameter associated with indicating that two or more speaker channel audio signals are coherently reproduced using two speaker channel audio signals that are spatially adjacent to an identified speaker channel audio signal, the identified speaker channel audio signal being a speaker channel audio signal that is spatially closest to at least one directional parameter; determining a coherence panning parameter associated with indicating that two or more speaker channel audio signals are coherently reproduced using at least two or more speaker channel audio signals that are spatially adjacent to the identified speaker channel audio signal; and generating the extended coherence parameter based on the stereoscopic parameter and the coherence panning parameter.

[0025] Generating extended coherence parameters based on stereo parameters and coherence translation parameters may include: determining a main directional analysis to identify a speaker closest to at least one directional parameter; searching from the direction of the identified speaker, and each search area includes an angle from 0 degrees to 180 degrees in a series of angular steps; estimating an average coherence value between a defined main speaker channel and any speaker channel within the search area; based on the average coherence value, determining a substantially constant coherence area; setting an extended range to twice the maximum coherence area; and defining a coherence translation parameter based on the extended range.

[0026] Defining the coherence panning parameter based on the maximum coherence region may include: determining a speaker closest to at least one directional parameter; determining a normalized coherence c between the speaker and all speakers within the maximum coherence region; a,i ; Ignore speakers with energy below a threshold energy; select minimum coherence from the remaining speakers; determine energy distribution parameters based on the energy distribution among the remaining speakers; multiply the energy distribution parameters by the maximum coherence area to determine the coherence translation parameters.

[0027] Determining stereo parameters may include: determining a main direction analysis to identify a speaker closest to at least one direction parameter; searching from the direction of the identified speaker, and each search ring is defined by an angle from 0 degrees to 180 degrees in a series of angle steps; estimating an average coherence value and an average energy value of all speakers located near the search ring; determining a maximum coherence ring angle based on the average coherence value and the average energy value; setting an extension range to twice the maximum coherence ring angle; and defining stereo parameters based on the extension range.

[0028] Defining a stereo parameter based on an extended range may include: identifying a speaker having maximum energy on a maximum coherence loop; determining a normalized coherence between the identified speaker and other speakers on the maximum coherence loop; determining an average of the normalized coherences weighted by the corresponding energies; determining an energy ratio of the energy on the maximum coherence loop to the energy within the maximum coherence loop; and multiplying the energy ratio by the average of the normalized coherences to form a stereo parameter.

[0029] According to a third aspect, there is provided an apparatus comprising at least one processor and at least one memory comprising computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the apparatus to at least: determine, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determine at least one audio signal relationship parameter between the two or more speaker channel audio signals, the at least one audio signal relationship parameter being associated with at least one coherence parameter, whereby the at least one coherence parameter provides at least one inter-channel coherence information between the two or more speaker channel audio signals for at least two frequency bands, so as to reproduce the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one audio signal relationship parameter; and send at least one spatial audio parameter and send at least one information associated with the at least one inter-channel coherence information using at least one determined value.

[0030] The device that is caused to use at least one determined value to send at least one information associated with at least one inter-channel coherence information may further cause the device to send at least one of the following: at least one orientation of at least one coherence parameter; at least one width of at least one coherence parameter; and at least one range of at least one coherence parameter.

[0031] The at least one determined value may include at least one of: at least one directional code; at least one width code; and at least one range code.

[0032] The apparatus caused to determine, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction may be caused to determine, for the two or more speaker channel audio signals, at least one directional parameter and / or at least one energy ratio.

[0033] The apparatus may be caused to determine a transmission audio signal from two or more speaker channel audio signals, wherein the two or more speaker channel audio signals may be reproduced based on at least one spatial audio parameter, at least one coherence parameter, and / or the transmission audio signal.

[0034] The device, which is caused to determine at least one coherence parameter between two or more speaker channel audio signals, may be caused to determine an extended coherence parameter, wherein the extended coherence parameter may be determined based on inter-channel coherence information between two or more speaker channel audio signals that are spatially adjacent to an identified speaker channel audio signal, the identified speaker channel audio signal being identified based on the at least one spatial audio parameter.

[0035] The apparatus caused to determine an extended coherence parameter may be caused to: determine a stereoscopic parameter associated with an indication that two or more speaker channel audio signals are coherently reproduced using two speaker channel audio signals that are spatially adjacent to an identified speaker channel audio signal, the identified speaker channel audio signal being a speaker channel audio signal that is spatially closest to at least one directional parameter; determine a coherence panning parameter associated with an indication that two or more speaker channel audio signals are coherently reproduced using at least two or more speaker channel audio signals that are spatially adjacent to the identified speaker channel audio signal; and generate the extended coherence parameter based on the stereoscopic parameter and the coherence panning parameter.

[0036] The device, which is configured to generate extended coherence parameters based on stereo parameters and coherence translation parameters, can be configured to: determine a main directional analysis to identify a speaker closest to at least one directional parameter; search from the direction of the identified speaker, and each search area includes an angle from 0 degrees to 180 degrees in a series of angular steps; estimate the average coherence value between the defined main speaker channel and any speaker channel within the search area; determine a substantially constant coherence area based on the average coherence value; set an extended range to twice the maximum coherence area; and define a coherence translation parameter based on the extended range.

[0037] The apparatus caused to define the coherence panning parameter based on the maximum coherence region may be caused to: determine the loudspeaker closest to at least one directional parameter; determine the normalized coherence c between the loudspeaker and all loudspeakers within the maximum coherence region;a,i ; Ignore speakers with energy below a threshold energy; select minimum coherence from the remaining speakers; determine energy distribution parameters based on the energy distribution among the remaining speakers; multiply the energy distribution parameters by the maximum coherence area to determine the coherence translation parameters.

[0038] The device configured to determine stereo parameters can be configured to: determine a main directional analysis to identify a speaker closest to at least one directional parameter; search from the direction of the identified speaker, and each search ring is defined by an angle from 0 degrees to 180 degrees in a series of angular steps; estimate the average coherence value and the average energy value of all speakers located near the search ring; determine the maximum coherence ring angle based on the average coherence value and the average energy value; set the extension range to twice the maximum coherence ring angle; and define the stereo parameters based on the extension range.

[0039] The device that is enabled to define a stereo parameter based on an extended range can be enabled to: identify a speaker having maximum energy on a maximum coherence loop; determine a normalized coherence between the identified speaker and other speakers on the maximum coherence loop; determine an average value of the normalized coherences weighted by the corresponding energies; determine an energy ratio of the energy on the maximum coherence loop to the energy within the maximum coherence loop; and multiply the energy ratio by the average value of the normalized coherences to form a stereo parameter.

[0040] According to a fourth aspect, there is provided a computer program (or a computer-readable medium comprising program instructions) comprising instructions for causing an apparatus to at least perform the following operations: determine, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determine at least one audio signal relationship parameter between the two or more speaker channel audio signals, the at least one audio signal relationship parameter being associated with at least one coherence parameter, whereby the at least one coherence parameter provides at least one inter-channel coherence information between the two or more speaker channel audio signals for at least two frequency bands, so as to reproduce the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one audio signal relationship parameter; and send at least one spatial audio parameter and use at least one determined value to send at least one information associated with the at least one inter-channel coherence information.

[0041] According to a fifth aspect, a non-transitory computer-readable medium is provided, comprising program instructions for causing an apparatus to perform at least the following operations: determining, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determining at least one audio signal relationship parameter between the two or more speaker channel audio signals, the at least one audio signal relationship parameter being associated with at least one coherence parameter, whereby the at least one coherence parameter provides at least one inter-channel coherence information between the two or more speaker channel audio signals for at least two frequency bands, so as to reproduce the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one audio signal relationship parameter; and sending at least one spatial audio parameter, and using at least one determined value to send at least one information associated with the at least one inter-channel coherence information.

[0042] According to a sixth aspect, a device is provided, comprising: a spatial audio parameter determination circuit, configured to determine at least one spatial audio parameter for providing spatial audio reproduction for two or more speaker channel audio signals; an audio signal relationship parameter determination circuit, configured to determine at least one audio signal relationship parameter between the two or more speaker channel audio signals, the at least one audio signal relationship parameter being associated with at least one coherence parameter, whereby the at least one coherence parameter provides at least one inter-channel coherence information between the two or more speaker channel audio signals for at least two frequency bands, so as to reproduce the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one audio signal relationship parameter; and a transmission circuit, for transmitting the at least one spatial audio parameter, and transmitting at least one information associated with the at least one inter-channel coherence information using at least one determined value.

[0043] According to a seventh aspect, there is provided a computer-readable medium comprising program instructions for causing an apparatus to at least perform the following operations: determine, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determine at least one audio signal relationship parameter between the two or more speaker channel audio signals, the at least one audio signal relationship parameter being associated with at least one coherence parameter, whereby the at least one coherence parameter provides at least one inter-channel coherence information between the two or more speaker channel audio signals for at least two frequency bands, so as to reproduce the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one audio signal relationship parameter; and send at least one spatial audio parameter, and use at least one determined value to send at least one information associated with the at least one inter-channel coherence information.

[0044] An apparatus comprises means for performing the actions of the method as described above.

[0045] A device is configured to perform the actions of the method described above.

[0046] A computer program comprises program instructions for causing a computer to execute the method described above.

[0047] A computer program product stored on a medium can cause an apparatus to execute the method as described herein.

[0048] An electronic device may include the apparatus described herein.

[0049] A chipset may include the device described herein.

[0050] The embodiments of the present application are intended to solve the problems associated with the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] For a better understanding of the present application, reference will now be made by way of example to the accompanying drawings, in which:

[0052] Figure 1 schematically illustrates a system suitable for implementing the apparatus of some embodiments;

[0053] Figure 2 According to some embodiments, Figure 1 A flowchart of the operation of the system shown in;

[0054] Figure 3 Schematically illustrating how Figure 1 The analysis processor shown in ;

[0055] Figures 4a to 4f According to some embodiments, Figure 2 A flowchart of the operation of the analysis processor shown in;

[0056] Figure 5a and Figure 5b illustrates an exemplary virtual speaker node arrangement suitable for application of some embodiments;

[0057] Figure 6a and Figure 6b illustrates exemplary coherence in an array of speaker nodes;

[0058] Figure 7a and Figure 7b showing an exemplary virtual speaker array;

[0059] Figure 8a and Figure 8b shows an exemplary extended coherent directional coded quantization example according to some embodiments;

[0060] Figure 9a and Figure 9b shows an exemplary quantization table showing encoding of extended coherence orientation according to some embodiments;

[0061] Fig.10 shows an exemplary increased ring / region for determination of a coherence parameter;

[0062] Fig.11 Schematically illustrating how Figure 1 The synthesis processor shown in ;

[0063] Fig.12 According to some embodiments, Fig.11 A flowchart of an exemplary operation of a synthesis processor shown in;

[0064] Fig.13 A flowchart illustrating exemplary operations for generating a target covariance matrix according to some embodiments;

[0065] Fig.14 Exemplary devices suitable for implementing the means described herein are schematically shown. DETAILED DESCRIPTION

[0066] Suitable apparatus and possible mechanisms for providing efficient encoding of metadata parameters derived for spatial analysis are described in more detail below.

[0067] As discussed previously, spatial metadata parameters in frequency bands, such as directional and direct-to-total energy ratio (or diffuseness-ratio, absolute energy, or any suitable expression indicating the directionality / non-directionality of sound at a given time-frequency interval) parameters are particularly suitable for expressing the perceptual characteristics of both natural (in other words, captured sound fields) and synthetic (in other words, generated sound fields, such as multi-channel loudspeaker mixes) sound fields.

[0068] An example of a suitable spatial parameter is the coherence parameter.As will be discussed in further detail below the concept is to provide efficient parameter transmission over a wide range of bit rates.

[0069] The concepts as detailed below in examples relate to audio encoding and decoding using sound field related parameterization (directions and ratios in frequency bands), wherein a scheme is provided for improving the reproduction quality of (generated and recorded) loudspeaker surround mixes encoded with the aforementioned parameterization.

[0070] Furthermore, embodiments discuss improved perceived quality of speaker surround mixes by analyzing inter-channel coherence information of speaker signals in frequency bands, wherein the inter-channel coherence information includes directional and width (range) information of inter-channel coherence regions or channels / speaker groups.

[0071] In addition, the examples below show spatial coherence parameters being transmitted along with spatial parameters (i.e., direction and energy ratio), where the orientation and width / range are efficiently provided for encoding using an "orientation code" and, in some embodiments, a "directional code" and a "circular sector code." In some embodiments, these codes may consume 4 bits per direction parameter.

[0072] Examples as discussed below also describe reproduction of sound based on directional parameters and spatial coherence parameters including directional codes and circular sector codes, such that the spatial coherence parameters affect cross-correlation of reproduced audio signals according to the directional codes and circular sector codes.

[0073] The cross-correlation of the output signals may refer to the cross-correlation of the reproduced loudspeaker signal, or the reproduced binaural signal, or the reproduced Ambisonic signal.

[0074] In some of the following examples, the signaling of the "spread coherence" parameter uses the format of regional orientation and range. In this exemplary format, the spread directional code has a 0-180 degree rotation, and in this exemplary format, the circular sector code has a 0-360 degree center angle for the spread range. In some embodiments, a spherical sector code may be used alternatively.

[0075] As such, the concepts discussed in further detail by way of exemplary implementations relate to audio encoding and decoding using spatial audio or sound field related parameterization (e.g., other spatial metadata parameters may include directionality, energy ratio, direct-to-total ratio, directional stability, or other suitable parameters). The concepts also disclose embodiments including methods and apparatuses intended to improve the quality of reproduction of loudspeaker surround mixes encoded using the aforementioned parameterization.

[0076] Conceptual embodiments improve the quality of speaker surround mixing by analyzing inter-channel coherence of speaker signals in frequency bands, transmitting spatial coherence parameters and directional parameters, and reproducing sounds based on the directional parameters and the spatial coherence parameters so that the spatial coherence affects the cross-correlation of the reproduced audio signals.

[0077] In this document, the term "coherence or cross-correlation" need not be strictly interpreted as a specific similarity value between signals, such as a normalized square value, but generally reflects a similarity value between playing audio signals and can be a complex value (with phase), an absolute value, a normalized value, or a square value. A coherence parameter can be more generally expressed as an audio signal relationship parameter that indicates the similarity of audio signals in any way.

[0078] The coherence of the output signal may refer to the coherence of the reproduced loudspeaker signal, the reproduced binaural signal, or the reproduced Ambisonic signal.

[0079] Therefore, the conceptual implementation discussed may provide two relevant parameters, such as:

[0080] spatial coherence across an area in a certain direction, which is related to the directional component of the acoustic energy;

[0081] Surround spatial coherence, which is related to the ambient / non-directional portion of the sound energy.

[0082] Furthermore, as discussed in further detail below, ratio parameters may be modified based on the determined spatial coherence or audio signal relationship parameters for further audio quality improvement.

[0083] A typical scenario is described in the exemplary embodiments detailed below, where the speaker surround mix is ​​a horizontal surround setup. In other embodiments, the spatial coherence or audio signal relationship parameters may also be estimated from a "3D" speaker configuration. In other words, in some embodiments, the spatial coherence or audio signal relationship parameters may be associated with directions located "above" or "below" a defined plane (e.g., speakers that are high or low relative to a defined "horizontal" plane).

[0084] There can be any degree of coherence between any channels in a loudspeaker mix. In theory, in order to accurately describe this perceptually, all the information conveyed by the covariance matrix of the loudspeaker signals in the frequency band should be sent in the spatial metadata. The size of such a covariance matrix is ​​N x N, where N is the number of loudspeaker channels. For a 5-channel system, this means sending 10 complex cross-correlation values ​​for each time-frequency analysis interval, for a 7-channel system 21 complex cross-correlation values, etc. Obviously, for a suitable low bitrate codec, this will produce too much metadata. Therefore, in the following embodiments, examples are described in which the spatial metadata only describes the perceptually essential aspects in order to keep the bitrate low.

[0085] For completeness, and in addition to the embodiments of the present invention, a practical spatial audio encoder that optimizes the transmission of the inter-channel relationships of a speaker mix does not send the entire covariance matrix of the speaker mix, but instead provides a set of upmix parameters to recover a surround sound signal with a covariance matrix substantially similar to the original surround signal on the decoder side. Schemes such as these have been adopted. However, such approaches are dedicated to encoding and decoding only existing speaker mixes. The context of the present invention is spatial audio coding using directional and ratio metadata, which is a parameterization independent of the speaker setup and is particularly suitable for captured spatial audio (hence, the method of the present invention is needed to improve quality in the case of speaker surround input).

[0086] Therefore, these examples focus on addressing the reproduction quality of 5.1 and 7.1 (and other formats) channel speaker mixes using a perceptually determined speaker setup independent parameterized approach as discussed below.

[0087] In practical 5.1 and 7.1 channel loudspeaker mixes, there are three typical cases of spatial coherence that are relevant to the directional-ratio parameterization:

[0088] 1) Sound is reproduced coherently using two speakers to create a "brisk" perception (e.g., using left and right front instead of center);

[0089] 2) the sound is reproduced coherently using three (or more) speakers to create a perception of "closeness" (e.g., using front left, front right, and center instead of just center); and

[0090] 3) Sound is reproduced coherently from all (or nearly all) speakers to create an "in-head" or "above" perception.

[0091] It is shown how these three cases can be estimated and described using only 2 parameters in each time-frequency interval (in addition to the already existing directional and direct-to-total ratio parameters). It is proposed that using this parameter set, a spatial quality for the reproduced output similar to that obtained by reproducing the spatial sound using the information contained in the entire covariance matrix can be obtained.

[0092] It is also shown how to synthesize spatial sounds based on the proposed parameters by using existing synthesis techniques known in the literature.

[0093] about Figure 1, shows an exemplary device and system for implementing an embodiment of the present application. The system 100 is shown to have an "analysis" part 121 and a "synthesis" part 131. The "analysis" part 121 is the part from receiving the multi-channel loudspeaker signal until the encoding of the metadata and the transmitted audio signal, and the "synthesis" part 131 is the part from decoding the encoded metadata and the transmitted audio signal to presenting the synthesized signal (e.g. in the form of a multi-channel loudspeaker).

[0094] The input to the system 100 and the "analysis" part 121 is a multi-channel loudspeaker signal 102. In the following example a 5.1 channel loudspeaker signal input is described, however, in other embodiments any suitable input loudspeaker (or resultant multi-channel) format may be implemented.

[0095] The multi-channel loudspeaker signal is passed to the transmission signal generator 103 and to the analysis processor 105 .

[0096] The transmission signal generator 103 is configured to receive the input signal 102 and generate a suitable transmission audio signal 104. The transmission audio signal may also be referred to as an associated audio signal and is based on a spatial audio signal (implicitly or explicitly containing the directional information of the sound field and input into the system). For example, in some embodiments, the transmission signal generator 103 is configured to down-mix or otherwise select or combine the input audio signal to a determined number of channels and output them as a transmission signal 104. The transmission signal generator 103 may be configured to generate any suitable number of transmission audio signals (or channels), for example, in some embodiments, the transmission signal generator is configured to generate two transmission audio signals. In some embodiments, the transmission signal generator 103 is also configured to encode the audio signal. For example, in some embodiments, the audio signal may be encoded using advanced audio coding (AAC) or enhanced voice service (EVS) compression encoding. In some embodiments, the transmission signal generator 103 may be configured to equalize the audio signal, apply automatic noise control, dynamic processing, or any other suitable processing. In some embodiments, the transmission signal generator 103 may also take the output of the analysis processor 105 as input to facilitate the generation of the transmission signal 104 .

[0097] In some embodiments, the transmission signal generator 103 is optional and the multi-channel loudspeaker signal is passed through unprocessed.

[0098] In some embodiments, the analysis processor 105 is also configured to receive the multi-channel speaker signals and analyze these signals to generate metadata 106 associated with the multi-channel speaker signals and therefore associated with the transmission signal 104. For example, the analysis processor 105 can be a computer (running suitable software stored on a memory and at least one processor), or alternatively can be a specific device, such as utilizing an FPGA or ASIC. As further detailed herein, for each time-frequency analysis interval, the metadata can include a directional parameter 108, an energy ratio parameter 110, a surround coherence parameter 112, and an extended coherence parameter 114. In some embodiments, the directional parameters and the energy ratio parameters can be considered as spatial audio parameters. In other words, the spatial audio parameters include parameters intended to characterize the sound field created by the multi-channel speaker signals (or generally, two or more playback audio signals).

[0099] In some embodiments, the parameters generated may differ from band to band. Thus, for example, in band X, all parameters are generated and sent, while in band Y, a different number of parameters are generated and sent, and furthermore, in band Z, no parameters are generated or sent. A practical example of this may be that for some bands, such as the highest frequency band, some parameters are not needed for perceptual reasons.

[0100] Additionally, the analysis processor 105 or a suitable encoder may be configured to encode the metadata, for example, as described in further detail below.

[0101] The transmission signal 104 and metadata 106 may be transmitted or stored. Figure 1 107. Before the transmission signal 104 and metadata 106 are sent or stored, they may be encoded to reduce the bit rate and multiplexed into a stream. Any suitable scheme may be used to implement encoding and multiplexing, and the encoding of metadata is described in the embodiment.

[0102] At the decoder side, the received or acquired data (stream) can be demultiplexed and the encoded stream can be decoded to obtain the transmission signal and metadata. This reception or acquisition of the transmission signal and metadata is also performed at Figure 1 107 is shown on the right side of the dashed line 107.

[0103] The "Synthesis" portion 131 of the system 100 shows a synthesis processor 109 that is configured to receive the transmission signal 104 and the metadata 106 and recreate the multi-channel loudspeaker signal 110 (or in some embodiments, any suitable output format such as a binaural or Ambisonics signal, depending on the use case) based on the transmission signal 104 and the metadata 106. In some embodiments, the synthesis processor 109 can be a computer (running suitable software stored on a memory and at least one processor), or alternatively can be a specific device, such as using an FPGA or ASIC.

[0104] about Figure 2 , showing Figure 1 An exemplary flow chart of the overview is shown in .

[0105] First, as in Figure 2 As shown in step 201, the system (analysis part) is configured to receive a multi-channel (speaker) audio signal.

[0106] Then, as in Figure 2 As shown in step 203, the system (analysis part) is configured to generate a transmission audio signal.

[0107] In addition, as in Figure 2 As shown in step 205, the system (analysis part) is configured to analyze the loudspeaker signals to generate metadata: directionality; energy ratio; surround coherence; extended coherence.

[0108] Then, as in Figure 2 As shown by step 207, the system is configured to encode the transmission signal and metadata with coherence parameters for storage / transmission.

[0109] After this, as in Figure 2 As shown in step 209, the system may store / send the encoded transmission signal and metadata with coherence parameters.

[0110] The system can acquire / receive the encoded transmission signal and metadata with coherence parameters, such as Figure 2 As shown in step 211.

[0111] Then, as in Figure 2 As shown by step 213, the system is configured to extract the transmission signal and the metadata with the coherence parameters from the encoded transmission signal and the metadata with the coherence parameters.

[0112] As in Figure 2As shown by step 215, the system (synthesis part) is configured to synthesize an output multi-channel audio signal (which, as previously discussed, can be any suitable output format such as a binaural, multi-channel or Ambisonics signal, depending on the use case) based on the extracted transmission signal and the metadata with coherence parameters.

[0113] about Figure 3 , further describing in detail an exemplary analysis processor 105 (eg, Figure 1 In some embodiments, the analysis processor 105 includes a time-frequency domain converter 301.

[0114] In some embodiments, the time-frequency domain transformer 301 is configured to receive the multi-channel loudspeaker signal 102 and apply a suitable time-frequency domain transform, such as a short-time Fourier transform (STFT), to convert the input time domain signal into suitable time-frequency signals 302. These time-frequency signals may be passed to a direction analyzer 303 and a coherence analyzer 305.

[0115] Thus, for example, the time-frequency signal 302 may be represented in the time-frequency domain as:

[0116] s i (b, n)

[0117] Where b is a frequency bin index, n is a frame index, and i is a speaker channel index. In another expression, n can be considered as a time index with a sampling rate lower than the sampling rate of the original time domain signal. The frequency bins can be grouped into multiple subbands, which group one or more bins into frequency band indexes k=0, ..., K-1. Each subband k has a minimum bin b k，low and the highest interval b k，high , and this subband contains k，low to b k，high The width of the subband can be approximated by any suitable distribution, such as the equivalent rectangular bandwidth (ERB) scale or the Bark scale.

[0118] In some embodiments, the analysis processor 105 comprises a direction analyzer 303. The direction analyzer 303 may be configured to receive the time-frequency signals 302 and based on these signals estimate the direction parameter 108. The direction parameter may be determined based on any audio-based determination of "direction".

[0119] For example, in some embodiments, the direction analyzer 203 is configured to estimate the direction using two or more speaker signal inputs.

[0120] Therefore, the direction analyzer 303 may be configured to provide an azimuth angle denoted as θ(k,n) for each frequency band and time frame. If the direction parameter is a 3D parameter, exemplary direction parameters may be the azimuth angle θ(k,n), the elevation angle The direction parameter 108 may also be passed to the coherence analyzer 305 .

[0121] refer to Figure 2 The directional parameters obtained by analyzing the loudspeaker signals in step 205 to generate metadata (and encoded in step 207 for storage or transmission) may be expressed, for example, in terms of azimuth and elevation angles or spherical grid indices.

[0122] In some embodiments, in addition to the direction parameter, the direction analyzer 303 is also configured to determine other suitable parameters associated with the determined direction parameter. For example, in some embodiments, the direction analyzer is caused to determine the energy ratio parameter 110. The energy ratio can be considered as a determination of the energy of an audio signal, wherein the audio signal can be considered to arrive from a direction. For example, the direct total energy ratio r(k, n) can be estimated using a stability metric of the direction estimate, or using any correlation metric, or any other suitable method for obtaining the energy ratio parameter. In other embodiments, the direction analyzer is caused to determine and output a stability metric of the direction estimate, a correlation metric, or other parameters associated with the direction.

[0123] The estimated direction 108 parameters may be output (and will be used in the synthesis processor). The estimated energy ratio parameters 110 may be passed to the coherence analyzer 305. In some embodiments, these parameters may be received in a parameter combiner (not shown), where the estimated direction and energy ratio parameters are combined with the coherence parameters generated by the coherence analyzer 305 as described below.

[0124] In some embodiments, the analysis processor 105 includes a coherence analyzer 305. The coherence analyzer 305 is configured to receive parameters (such as azimuth angle (θ(k, n)) 108 and direct total energy ratio (r(k, n)) 110) from the direction analyzer 303. The coherence analyzer 305 can be further configured to receive the time-frequency signal (s i (b, n)) 302. All of this is in the time-frequency domain; b is the frequency bin index, k is the band index (each band potentially includes several bins b), n is the time index, and i is the loudspeaker channel.

[0125] Although these directions and ratios are expressed here for each time index n, in some embodiments, these parameters can be combined over several time indexes. As already expressed, the same applies to the frequency axis, the direction of several frequency intervals b can be expressed by one direction parameter in a frequency band k comprising several frequency intervals b. The same applies to the spatial parameters discussed herein.

[0126] The coherence analyzer 305 is configured to generate multiple coherence parameters. In the following disclosure, there are two parameters: surround coherence (γ(k, n)) and extended coherence (ζ(k, n)), which are analyzed in the time-frequency domain. In addition, in some embodiments, the coherence analyzer 205 is configured to modify associated parameters (e.g., the estimated energy ratio (r(k, n))).

[0127] In some embodiments, the extended coherence encoder 307 is configured to receive and encode the extended coherence parameters. In some embodiments, the function of the extended coherence encoder 307 is incorporated into the coherence analyzer 305, and the encoded extended coherence parameters 114 are directly output from the coherence analyzer. In some embodiments, the encoding and signaling of the extended coherence parameters are achieved by signaling of the "extended coherence" regional orientation and range parameter pairs. In addition, in some embodiments, the "extended coherence" regional orientation and range parameter pairs are signaled by the following items:

[0128] Extended directional code with 0-180 degree rotation; and

[0129] Circular sector code with a center angle of 0-360 degrees for an extended range.

[0130] In some embodiments, only circular sector codes with center angles of 0-360 degrees for an extended range are used.

[0131] In some embodiments, spherical sector codes may alternatively be used. The exemplary encoding of coherence is intended to produce no or minimal loss at the codec input and to allow efficient transmission given the current bit rate constraints at the audio encoder. For example, in a scenario with communication capabilities, network congestion may severely affect the audio encoding bit rate in a single transmission, resulting in frame-to-frame fluctuations.

[0132] The output of the coherence analyzer 305 (and the extended coherence encoder 307 ), and in particular the extended coherence output, may be passed to an extended coherence encoder configured to encode the output extended coherence and generate suitable encoded extended coherence parameters 114 .

[0133] Therefore, in some embodiments, the coherence analyzer 305 may be configured to calculate the covariance matrix C for a given analysis interval comprising one or more time indices n and frequency bins b. The matrix size is N x N, and the elements are denoted as c ij , where i and j are the speaker channel indices.

[0134] Next, the coherence analyzer 305 may be configured to determine the speaker channel i that is closest to the estimated direction (in this example, the azimuth angle θ) c .

[0135] i c =arg(min(|θ-α i |))

[0136] Among them, α i is the angle of speaker i.

[0137] In some embodiments, for example in the case of a 3D speaker setup, the elevation angle is also taken into account when determining the closest speaker ic. This may be achieved in any suitable manner, for example by considering each orientation separately or calculating all combinations at once (and extracting the orientation from the information).

[0138] Furthermore, in this embodiment, the coherence analyzer 305 is configured to determine the c The left side l and right side i r The closest speaker.

[0139] The normalized coherence between loudspeakers i and j is expressed as:

[0140]

[0141] Using this equation, the coherence analyzer 305 may be configured to calculate i l with i r The normalized coherence c′ between lr . In other words, calculate:

[0142]

[0143] Furthermore, the coherence analyzer 305 may be configured to use the diagonal elements of the covariance matrix to determine the energy of the loudspeaker channel i.

[0144] E i =c ii

[0145] And determine the speaker i l and i r With speaker i l 、ir and i c The energy ratio between them is:

[0146]

[0147] The coherence analyzer 305 may then use these determined variables to generate a "stereoness" parameter.

[0148] μ=c′ lr ξ lr / lrc

[0149] This "stereoscopic" parameter has a value between 0 and 1. A value of 1 means that speaker i l and i r , and this sound dominates the energy of this sector. The reason for this may be, for example, that a speaker mixer uses amplitude panning techniques to create a "brisk" perception of the sound. A value of 0 means that no such technique has been applied, and, for example, the sound may simply be localized to the closest speaker.

[0150] In addition, the coherence analyzer can be configured to detect or at least identify situations where three (or more) speakers are used to coherently reproduce sound to create a "close" perception (e.g., using left front, right front and center instead of just center). This can be because the mixing engineer creates this situation when performing a surround mix on a multi-channel speaker mix.

[0151] In this embodiment, the coherence analyzer uses the same speaker i previously identified. l 、i r and i c The normalized coherence value c′ is determined using the normalized coherence determination discussed previously. cl and c′ cr . In other words, the following values ​​are calculated:

[0152]

[0153] The coherence analyzer 305 may then determine a normalized coherence value c′ describing the coherence between the loudspeakers using the following equation: clr :

[0154] c′ clr =min(c′ cl , c′ cr )

[0155] Additionally, the coherence analyzer may be configured to determine a coherence factor describing how the energy in channel i l 、i r and i cParameters evenly distributed between:

[0156]

[0157] Using these variables, the coherence analyzer can determine the new coherence shift parameter K as:

[0158] κ=c′ clr ξ clr

[0159] The coherence pan parameter κ has a value between 0 and 1. A value of 1 means that l 、i r and i c , and the energy of that sound is evenly distributed between those speakers. The reason for this could be, for example, because the speaker mix was generated using recording mixing techniques designed to create the perception of closer sound sources. A value of 0 means that no such techniques have been applied, for example, the sound may simply be localized to the closest speaker.

[0160] Determine the metric i l and i r in (but not in i c The stereo parameter μ of the coherent sound volume and the measure of all i l 、i r and i c A coherence analyzer which determines the coherence panning parameters K of the coherent sound volume in is configured to use these parameters to determine the coherence parameters to be output as metadata.

[0161] Therefore, the coherence analyzer is configured to combine the stereo parameter μ and the coherence translation parameter K to form an extended coherence ζ parameter with values ​​ranging from 0 to 1. An extended coherence ζ value of 0 indicates a point source, in other words, the source should be detected with as few loudspeakers as possible (e.g., only loudspeaker i c ) to reproduce the sound. As the value of the extended coherence ζ increases, more energy is extended to the speaker i c surrounding speakers; until the value is 0.5, the energy is in speaker i l 、i r and i c When the value of the extended coherence ζ exceeds 0.5, the loudspeaker i c The energy in decreases until the value is 1, at speaker i c There is no energy in the speaker, and all the energy is in the speaker l and i r Place.

[0162] In some embodiments, using the above parameters μ and κ, the coherence analyzer is configured to determine the extended coherence parameter ζ using the following expression:

[0163]

[0164] The above expressions are merely examples, and it should be noted that the coherence analyzer may adopt any other manner to estimate the extended coherence parameter ζ as long as it conforms to the above parameter definition.

[0165] In addition to being configured to detect previous situations, the coherence analyzer may also be configured to detect or at least identify situations in which sound is reproduced coherently from all (or nearly all) speakers to create a perception of being "in the head" or "above".

[0166] In some embodiments, the coherence analyzer may be configured to determine the energy E having the maximum value. i and speaker channel i e to classify.

[0167] The coherence analyzer can then be configured to determine the normalized coherence C′ between this channel and the M other loudest channels ii These normalized coherences c′ between this channel and the M other loudest channels can then be monitored ij Values. In some embodiments, M may be N-1, which would mean monitoring the coherence between the loudest channel and all other speaker channels. However, in some embodiments, M may be a smaller number, for example, N-2. Using these normalized coherence values, the coherence analyzer may be configured to determine the surround coherence parameter γ using the following expression:

[0168]

[0169] in, is the normalized coherence between the loudest channel and the M next loudest channels.

[0170] The surround coherence parameter y has a value from 0 to 1. A value of 1 means that there is coherence between all (or almost all) of the loudspeaker channels. A value of 0 means that there is no coherence between all (or even almost all) of the loudspeaker channels.

[0171] The above expression is merely an example of estimating the surround coherence parameter γ, and any other manner may be used as long as it conforms to the above parameter definition.

[0172] As discussed above, the coherence analyzer can be used to estimate surround coherence and extended coherence parameters. However, in some embodiments, in order to improve audio quality, the coherence analyzer can modify the ratio parameter r when it has been determined that situation 1 (the sound is coherent using two speakers for creating a "brisk" perception and using the left and right front instead of the center speaker) and / or situation 2 (the sound is coherent using three (or more) speakers for creating a "close" perception) occurs within the speaker signal. Therefore, in some embodiments, the extended coherence parameter and the surround coherence parameter can also be used to modify the ratio parameter r.

[0173] As indicated above, the energy ratio r is determined as the energy ratio in the direction (which may be the azimuth angle θ and / or the elevation angle ) to the rest of the energy. If the sound sources are generated as point sources in the surround mix (e.g., the sound is in only one speaker), the directional analysis correctly produces an energy ratio of 1, and the synthesis stage will reproduce the sound as a point source. However, if an audio mixing method for coherent sound in multiple speakers has been applied (such as the aforementioned scenarios 1 and 2), the directional analysis will produce a lower energy ratio (because the sound is no longer a point source). As a result, the synthesis stage will reproduce part of the sound as an environment, which can, for example, lead to the perception of distant sound sources, which is contrary to the goal of the recording mixing engineer when generating the speaker mix.

[0174] Thus, in some embodiments, the coherence analyzer may be configured to modify the energy ratio if it is detected that an audio mixing technique has been used that coherently distributes the sound to multiple speakers.

[0175] Therefore, in some embodiments, the coherence analyzer is configured to determine the speaker i l and i r The ratio of the energy of to the energy of all loudspeakers:

[0176]

[0177] Using this ratio, and c′ determined above lr and γ, the coherence analyzer generates an alternative energy ratio r s :

[0178] r s =c′ lr ξ lr / all -γ

[0179] In some embodiments, the coherence analyzer may be similarly configured to determine the speaker i l 、i r and i c The ratio of the energy of to the energy of all loudspeakers:

[0180]

[0181] Using this ratio, and the c′ calculated above clr and y, the coherence analyzer forms another alternative energy ratio rc:

[0182] r c =c′ clr ξ clr / all -γ

[0183] Using these energy ratios, the coherence analyzer can modify the original energy ratio r to:

[0184] r′=max(r,r s , r c )

[0185] This modified energy ratio r' can be used to replace the original energy ratio r. As a result, for example, in case 1 (the sound is coherently generated using two speakers to create a "brisk" perception and using the left and right front speakers instead of the center speaker), the ratio r' will be close to 1 (and the extended coherence ζ will also be close to 1). As discussed later in the synthesis stage, the sound will be generated from speaker i l and i r coherently reproduced without any decorrelation. Therefore, the perception of the reproduced sound will match the original mix.

[0186] about Figure 4a , 4b , 4c and 4d, show flow charts summarizing the operations described above.

[0187] So, for example, Figure 4a It shows that Figure 3 An exemplary overview of the operation of analysis processor 105 is shown in FIG.

[0188] As in Figure 4a As shown in step 401, the first operation is to receive a time domain multi-channel (speaker) audio signal.

[0189] Next, as in Figure 4a As shown in step 403, a time-to-frequency domain transform (eg, STFT) is applied to generate a suitable time-frequency domain signal for analysis.

[0190] Then, in Figure 4a Applying directional analysis to determine directions and associated parameters (eg, energy ratio parameters) is shown by step 405 .

[0191] Then, in Figure 4aApplying the coherence analysis to determine coherence parameters, such as surround parameters and / or extended coherence parameters, is shown by step 407 .

[0192] In some embodiments, the energy ratio may also be modified in this step based on the determined coherence parameter.

[0193] exist Figure 4a The final operation of encoding the extended coherence parameters and outputting (eg, in a bitstream or other suitable data structure) the determined parameters is shown by step 409 .

[0194] about Figure 4b is an exemplary method for generating extended coherence parameters.

[0195] As in Figure 4b As shown in step 431, the first operation is to calculate the covariance matrix.

[0196] As in Figure 4b As shown in step 433, the next operation is to determine the channel and adjacent channels closest to the estimated direction (ie, i c ,i l ,i r ).

[0197] As in Figure 4b As shown in step 435, the next operation is to normalize the covariance matrix.

[0198] Then, as in Figure 4b As shown by step 437, the method may include: using the diagonal elements of the covariance matrix to determine the energy of the channel.

[0199] Then, as in Figure 4b As shown by step 439, the method may include: determining a normalized coherence value between the left channel and the right channel.

[0200] As in Figure 4b As shown in step 441, the method may include: generating channel i l and i r Energy and channel i l 、i r and i c The ratio between the energies.

[0201] Then, as in Figure 4b As shown in step 443, the stereoscopic parameters can be determined.

[0202] Furthermore, in parallel with steps 439 to 443, the method may include: Figure 4b As shown in step 438, the normalized coherence value between the channels is determined; Figure 4b In step 440, the energy distribution parameters are determined; and Figure 4b As shown in step 442, the coherent translation parameters are determined.

[0203] Finally, as in Figure 4b As shown in step 445, the operation may determine an extended coherence parameter from the stereoscopic parameter and the coherence translation parameter.

[0204] also, Figure 4c An exemplary method for generating surround coherence parameters is shown.

[0205] The first three operations are Figure 4b The first three of the four operations shown are identical, with the first operation being the computation of the covariance matrix, as in Figure 4c As shown in step 451.

[0206] As in Figure 4c As shown in step 453, the next operation is to normalize the covariance matrix.

[0207] Then, as in Figure 4c As shown by step 455, the method may include: using the diagonal elements of the covariance matrix to determine the energy of the channel.

[0208] Then, as in Figure 4c As shown in step 457, the method may include: classifying the energy Ei.

[0209] Then, as in Figure 4c As shown in step 459, the method may include: selecting the channel with the maximum value.

[0210] Then, as in Figure 4c As shown by step 461 in FIG. 4 , the method may include monitoring the normalized coherence between the selected channel and M other maximum energy channels.

[0211] Then, as in Figure 4c As shown in step 463, the surround coherence parameters are determined from the normalized covariance matrix values.

[0212] about Figure 4d , an exemplary method for modifying the energy ratio is shown.

[0213] As in Figure 4d As shown in step 471, the first operation is to determine the speaker i l and i r The ratio of the energy in one loudspeaker to the energy in all loudspeakers.

[0214] Then, in Figure 4dThe coherence analyzer based on this ratio and c′ as determined above is shown by step 473. lr and γ to determine the first substitution ratio rs.

[0215] As in Figure 4d As shown in step 475, the next operation is to determine the speaker i l 、i r and i c The ratio of the energy in one loudspeaker to the energy in all loudspeakers.

[0216] Then, in Figure 4d The coherence analyzer is shown in step 477 based on this ratio and c′ as determined above. clr and γ to determine the second replacement ratio rc.

[0217] Then, as in Figure 4d As shown in step 739, a modified energy ratio may be determined based on the original energy ratio, the first alternative energy ratio, and the second alternative energy ratio and used to replace the current energy ratio.

[0218] The above formulas are described in detail to estimate coherence parameters for surround speaker inputs.A similar process may be performed on audio object inputs by processing the audio objects into audio channels at the determined positions at each temporal parameter estimation interval.

[0219] In addition, coherence parameters, such as extension and surround coherence parameters, can also be estimated for microphone array signals or Ambisonic input signals. For example, the method and apparatus can obtain a first-order Ambisonic (FOA) signal from some microphone arrays using methods known in the literature. The FOA signal includes an omnidirectional signal and three orthogonally aligned figure-of-eight signals, which have positive gain in one direction and negative gain in another direction. In one example of coherence parameter estimation for such an input, the method and apparatus can monitor the relative energy of the omnidirectional and three directional signals of the FOA signal. This is because if the sound is coherently reproduced from the surround direction and the FOA signal is captured, the omnidirectional (0th order FOA) signal includes the sum of these coherent signals. In contrast, the three figure-of-eight (first order FOA) signals have positive and negative gains related to the direction, and therefore, the coherent signals will partially or completely cancel each other at these first order FOA signals. Therefore, the surround coherence parameters can be estimated so that a higher value is provided when the energy of the 0th order FOA signal becomes higher relative to the combined energy of the 1st order FOA signal.

[0220] about Figure 4e, shows another example of determining the extended coherence parameter. In this example, the extended coherence estimation method described above is further generalized by using all input channels instead of only adjacent channels.

[0221] In some embodiments, this may be achieved by implementing a method of searching for a continuous coherent region (and generalizing the case where multiple loudspeakers are used to reproduce a coherent signal).

[0222] In this method, the search pattern can be defined with the parameters angle (φphi, starting from 0°) and step size (Δdelta, with a value of 5°, for example).

[0223] The method may perform an initial principal direction analysis (or receive from direction analyzer 303) to determine one or more directions, such as Figure 4e As shown in step 901.

[0224] The method can then place the input channels on a unit sphere (or create a unit sphere) based on their orientation, as in Figure 4e As shown in step 903.

[0225] The method is then further illustrated as creating a circle on the unit sphere with the principal direction being the center point and (φ) being the angle between the center point vector and the vector pointing to the edge of the circle (or otherwise creating a parameterized circle), as in Figure 4e As shown in step 905.

[0226] The main direction may be provided by a suitable means, such as the method suggested in the above method for directional analysis. The main channel may then be selected as the loudspeaker node or channel that is closest to the estimated main direction. Figure 4e The definition of the main channel is shown by step 907.

[0227] The next operation is to set the initial coherence angle, for example, φCA = 0, as in Figure 4e As shown in step 908.

[0228] Then the coherence area search begins. This search uses the main channel with a search area φ, as in Figure 4e As shown in step 909.

[0229] The next operation is to increase the angle φ by a step size Δ, as in Figure 4e This is shown in step 911. If φ exceeds 180 degrees, it is set to 180 degrees.

[0230] For example, this Fig.10 , where a main direction 1101 and a first angle φ 1103 are shown for a unit sphere 1100, which defines a first search ring 1113 on the surface of the sphere. Fig.10 As shown in , the angle φ can be increased by a step size Δ in further iterations. Fig.10 As shown in , this angle can be increased to a second angle 1105 , a third angle 1107 , and a fourth angle 1119 , thereby producing a second ring 1115 , a third ring 1117 , and a fourth ring 1119 .

[0231] Using this search area defined by direction and angle, it will be checked whether any input channel is within the search loop (within the defined tolerances), e.g. Figure 4e As shown in step 913.

[0232] If there are no input channels, the method returns to step 911 and the search ring is increased by further incrementing the angle φ by a step size Δ.

[0233] For any input channel determined within the search loop, the normalized coherence energy between the detected channel and the main channel is calculated, and their average value is calculated, as in Figure 4e As shown in step 915.

[0234] Then, a check is performed to determine whether the average coherence is above a certain tolerance (e.g., above 0.5). Figure 4e This is shown by step 917.

[0235] If the check determines that the average coherence is above a determined tolerance, the coherence angle φCA is increased to the current angle, in other words, φCA=φ.

[0236] In other words, newly identified channels are added to the area. Figure 4e This is shown by step 919.

[0237] Then, a further check is made to determine whether the search angle φ is 180 degrees, as in Figure 4e As shown in step 921.

[0238] If the search angle is less than 180 degrees, the operation returns to step 911 and the search circle is increased by further incrementing the angle φ by a step size Δ.

[0239] If the coherence energy does not match (or if the angle is 180 degrees), set φCA*2 to the extended range, as in Figure 4e As shown in step 923.

[0240] The subsequent operation after setting φCA*2 to the extended range is to estimate the coherent shift parameters, as in Figure 4e As shown in step 925.

[0241] To estimate the coherent panning parameters, first determine the loudspeaker a that is closest to the direction being analyzed. Next, determine the normalized coherence ca,i between this channel a and all channels i (where i≠a) in the region. Next, ignore channels with energies below a threshold energy (e.g., E τ =0.01E c ) channels and select the minimum coherence from the remaining channels.

[0242] c area =min(c a，i ), i∈area, i≠a, i≠ignored channel

[0243] Next, determine area , which indicates how evenly the energy is distributed among these channels:

[0244]

[0245] ξ area =min(ξi), i∈region, i≠c, i≠ignored channel

[0246] Using these variables, the coherence translation parameters can be formed:

[0247] κ=c area ξ area

[0248] As in Figure 4e As shown in step 925.

[0249] about Figure 4f , another embodiment is shown.

[0250] This other embodiment generalizes the search for coherence edges and is illustrated by a search for coherence rings.

[0251] The method may perform an initial principal direction analysis (or receive from direction analyzer 303) to determine one or more directions, such as Figure 4f As shown in step 1001.

[0252] The method can then place the input channels on a unit sphere (or create a unit sphere) based on their orientation, as in Figure 4f As shown in step 1003.

[0253] The method is then further illustrated as creating a circle on the unit sphere with the principal direction being the center point and (φ) being the angle between the center point vector and the vector pointing to the edge of the circle (or otherwise creating a parameterized circle), as in Figure 4f As shown in step 1005.

[0254] Then the coherence region search is started. This search uses the main channel with angle φ = 0, as in Figure 4f , as shown in step 1007. In this method, the search pattern can be defined by the parameters angle (φ, starting from 0°) and step size (Δ delta, with a value of 5°, for example).

[0255] In addition, the found coherence energy CE value is set to 0, and the coherence angle φCE=0 is defined, as in Figure 4f As shown in step 1009.

[0256] The next operation is to increase the search angle φ using a step size Δ, as in Figure 4f , as shown in step 1011. If φ exceeds 180 degrees, it is set to 180 degrees.

[0257] Using this direction and angle, it will be checked whether there are any input channels near the search ring (within a certain tolerance, such as 10 degrees), such as Figure 4f As shown in step 1013.

[0258] If there are no input channels in the vicinity of the ring, the method returns to step 1011 and the search ring is increased by further incrementing the angle φ by a step size Δ.

[0259] If there are at least two input channels on the search loop (within tolerance), the coherence between all channels on the loop is determined, and the average coherence of the loop is determined.

[0260] The average energy of all channels on the ring is also determined.

[0261] The determined average coherence is then multiplied by the average energy to generate the coherence energy CE of the ring, as in Figure 4f As shown in step 1015.

[0262] Then, a check is made to determine whether the average energy is large enough, as in Figure 4f As shown in step 1017.

[0263] If the average energy is not greater than the minimum value, the next step is 1011, the size of the ring is increased, and the input channels near the ring are searched again.

[0264] If the average energy of the ring is greater than a minimum value (e.g., 0.1), a further check is made to compare the determined coherence energy CE of the ring with the coherence energy of the previous ring. Figure 4f The CE check is shown by step 1019 .

[0265] If the check determines that the coherence energy of a loop is greater than the coherence energy of the previous loop, then the loop is used as the coherence loop. In other words, the found CE is set to the determined CE value of the loop, and φCE=φ is set, as in Figure 4f As shown in step 1021.

[0266] If the coherence energy of the loop is less than the coherence energy of the previous loop, the operation returns to step 1011 and the search loop is increased by further incrementing the angle φ by a step size Δ.

[0267] If the coherence energy is greater, a further check is performed to determine whether the search angle φ is 180 degrees, as in Figure 4f As shown in step 1023.

[0268] If the search angle is less than 180 degrees, the operation returns to step 1011 and the search circle is increased by further incrementing the angle φ by a step size Δ.

[0269] If the search angle is 180 degrees, set the extended range to φCE*2, as in Figure 4f As shown in step 1025.

[0270] The subsequent operation after setting the extended range to φCE*2 is to estimate the stereo parameters, as in Figure 4f As shown in step 1027. The stereo parameter can be determined by the following operations: First, find the ring with the maximum energy E m Then, the normalized coherence cm,i between this channel and the other channels i on the ring is calculated. Next, the average of these coherences weighted by the corresponding energy is calculated:

[0271]

[0272] Then, calculate the energy ratio on the ring to inside the ring:

[0273]

[0274] Using these variables, a stereometric parameter can be formed:

[0275] μ=c ring ξ ring

[0276] After the coherence panning and stereo parameters have been determined, they may be combined as described above to form combined extended coherence parameters.

[0277] Since the above example also generates an extended range parameter, they can be combined in some embodiments. In some embodiments, this combination can be to select the larger extended range of the two results.

[0278] The above algorithm shows an example of a general search pattern using a circle. However, the method is not limited to this, and various shapes and forms can be used instead of a circle. In addition, it is not mandatory to use a 3D search, and we can use only a 2D pattern to search, and include rotations of this 2D pattern.

[0279] These (modified) energy ratio 110, surround coherence 112 and extended coherence 114 parameters can then be output. In addition, as discussed, the extended coherence parameters can be passed to a metadata combiner or processed in any suitable manner, for example, encoded and / or multiplexed with a downmix signal, and stored and / or transmitted (and passed to the synthesis portion of the system). The synthesis method can be a modified least squares optimized signal mixing technique to process the covariance matrix of the signal while attempting to maintain audio quality. The method uses the covariance matrix metric of the input signal as well as the target covariance matrix (as discussed below) and provides a mixing matrix to perform this processing. The method also provides a means for optimally using decorrelated sound when there is not a sufficient amount of independent signal energy at the input.

[0280] Before further discussing the generation and encoding of coherence parameters, an exemplary loudspeaker node arrangement will be discussed. Figure 5a and 5b , showing a first view and a plan view, respectively, of an exemplary immersive audio presentation arrangement. Figure 5a and 5b The array shown in shows 30 speaker nodes that can represent (virtual) speakers. In this example, the array is arranged in three rings, each ring comprising 10 speaker nodes.

[0281] The first ring 513 is a horizontal ring around the listening position 501 at ear level, having a front center speaker 533 (on a reference azimuth that is "directly" in front of the listening position 501), a rear center speaker 543 (on the opposite side of the reference azimuth and "directly" behind the listening position 501), and another speaker 523 as marked.

[0282] The array may also include a first elevated or higher ring 511, which is a horizontal ring around the listening position 501 above ear level, having a front center speaker 531 (at a reference azimuth that is "directly" in front of the listening position 501), a rear center speaker 541 (on the opposite side of the reference azimuth and "directly" behind the listening position 501), and another speaker 521 as labeled.

[0283] The array is also shown as including a lowered or lower ring 515, which is a horizontal ring about the listening position 501 below ear level, having a center speaker 535 (at a reference azimuth that is "directly" in front of the listening position 501), a rear center speaker 545 (on the opposite side of the reference azimuth and "directly" behind the listening position 501), and another speaker 525 as labeled.

[0284] In some embodiments, the array of (virtual) speaker nodes may alternatively completely surround the listening position (i.e., for example, there may be virtual speakers surrounding the user in an equidistant array configuration), thereby providing the user with full freedom of 3DoF rotation without loss of resolution due to the selected viewing / listening direction.

[0285] The spacing between speaker nodes can vary greatly depending on the "looking" direction, and the azimuth distribution may not be equidistant, e.g. Figure 5a and 5b For example, a conventional horizontal speaker configuration such as 5.1 or 7.1 provides higher spatial resolution in front of the user than in other directions. Additionally, in some embodiments, the speaker distribution may be configured to provide a higher ring without providing a lower ring, or to provide more than one higher or lower ring.

[0286] Therefore, although the following examples are described with respect to this exemplary speaker node distribution, the embodiments described hereinafter may be applied to any suitable speaker node distribution.

[0287] about Figure 6a and 6b , shows an example where only the closest neighboring directions (or speaker nodes) are considered for coherence evaluation and the signaling / transmission of coherence parameters creates a large amount of data. Thus, for example, at least four orientations should be considered for a single speaker node 601, shown as vertical orientation 613, horizontal orientation 617, first diagonal orientation 611, and second diagonal orientation 615. Thus, when a single main coherence component is sent, signaling still needs to signal the selected or chosen orientation.

[0288] Once we know the coherent reproduction range, the coherent reproduction directional parameter can be estimated. This parameter is used to support reproduction when circular reproduction is not assumed. The method used to find the directional parameter is to always use the main direction speaker and the closest speaker to estimate the extended coherence parameter (and form the "stereo" and "coherence translation" parameters) for each directional angle in the rotation plane with positive and negative range angles (i.e., ±range / 2). The orientation that obtains the maximum extended coherence parameter is the selected directional angle. If multiple angles use the same "left" and "right" speakers, the average of these angles is used. This further assumes that the directional angles are searched from *90° to 90° with some step size (e.g., 10°).

[0289] In addition, if Figure 7a and 7b As shown in , orientation in large arrays can appear ambiguous, depending on the "center" or orientation, the orientation angle, and the array configuration. Thus, for example, Figure 7a A first orientation is shown, which shows that none of the speaker nodes are ambiguous, as orientation 701 passes through speaker nodes 711, 713, 715, 717, and 719. However, Figure 7b A direction 721 is shown, where it passes through some speaker nodes 731, 737 and 743, but is ambiguous for speaker node pairs 733 and 735, and 739 and 741. This may not be perceptually relevant and may not affect encoding and signaling.

[0290] In the embodiments described below, in addition to the coherence parameter value ("extended coherence"), directional and circular sectors of coherence are defined. In some embodiments, spherical sectors may alternatively or additionally be used. In some embodiments, the definition may also include directional information (and other descriptors, such as flatness).

[0291] It should be noted that in some embodiments where complex shapes of "extended coherence" directions are considered, the output may require very large amounts of metadata, which may result in data rates that are particularly unsuitable for low bitrate codecs without corresponding perceptual advantages. Therefore, in some embodiments, perceptually important aspects are defined and encoded in the spatial metadata.

[0292] Therefore, as discussed previously, the extended coherence encoder can be made to encode the orientation and extent of the extended coherence region:

[0293] - Extended directional code with 0-180 degree rotation; and

[0294] -Circular sector codes with a center angle of 0-360 degrees for an extended range.

[0295] Note that if the circular sector is very small, the influence of the extended coherence parameter on the perceived effect of the reproduction is limited. At small values, the source remains more point-like. On the other hand, at small sector values, small changes in the orientation angle are usually perceptually insignificant.

[0296] about Figure 8a and 8b , shows an exemplary directional encoding having the following form:

[0297]

[0298] Where b is the signaling bit and Qstep is the quantization step size. For a 4-bit description, it is:

[0299]

[0300] Therefore, if Figure 8a Shown are exemplary quantization points for the following quantization: 1-bit quantization 801 (at -pi / 2 or 0), 2-bit quantization 803 (at -2pi / 4, -pi / 4, 0, or +pi / 4), 3-bit quantization 805 (-4pi / 8, -3pi / 8, -2pi / 8, -pi / 8, 0, +pi / 8, 2pi / 8, 3pi / 8), 4-bit quantization 807 (from -8pi / 16 to 7pi / 16, with a step size of pi / 16), and 5-bit quantization 809 (from -15pi / 32 to 14pi / 32, with a step size of pi / 32).

[0301] also, Figure 8b The direction associated with the first bit b0 and the effect when the second bit b1 is 1 are shown, where the first bit b0 defines whether the direction is -pi / 2 (where b0=0) or 0 (where b0=1). For example, when b0b1=01 it is -pi / 4, and when b0b1=11 it is pi / 4.

[0302] also, Figure 9a A table summarizing an exemplary 4-bit embedded code is shown (where a basic offset of -90 degrees is added to correspond to Figure 8a and 8b corresponding).

[0303] In some embodiments, an orientation code may be embedded, in which case the orientation accuracy may be reduced by dropping bits in the encoder. In an embedded code, the baseline description provides a rough orientation (e.g., 90 or 45 degree accuracy), while additional layers of bits define a more accurate orientation.

[0304] Figure 9bAnother table indicating an embedded exemplary code with a 2-bit baseline and two 1-bit embedded fields with exemplary values ​​of 15 and 7.5 degrees, respectively, is shown. Normalization is performed to place all values ​​between -90 and 89.99 degrees, since any directional offset of 180 degrees corresponds to no offset in the directional data.

[0305] The (circular) sector range may be encoded by the implementation of a scalar quantization value. In some embodiments, the quantization may correspond to a virtual speaker array to be used as the intended rendering speaker node array, or in some embodiments it may be an "arbitrary" quantizer.

[0306] In some embodiments, the input channel configuration is signaled to the decoder. In this case, the (circular) sector range (and directional code) can directly exploit this information to maintain the quantization corresponding to the input.

[0307] about Fig.11 , further details of an exemplary synthesis processor 109. The exemplary synthesis processor 109 may be configured to use a modification method such as described in detail in the following document: US20140233762A1 "Optimal mixing matrices and usage of decorrelators in spatial audio processing", Vilkamo, Kuntz, Küch.

[0308] The cited method may be chosen because it is particularly suitable for such situations where inter-channel signal coherence needs to be synthesized or processed.

[0309] The synthesis processor 109 may receive the transmission signal 104 and the metadata 106 .

[0310] The synthesis processor 109 may include a time-frequency domain converter 301 configured to receive the transmission signal 104 and apply a suitable time-frequency domain transform such as a short-time Fourier transform (STFT) to convert the input time domain signal into a suitable time-frequency signal. These time-frequency signals may be passed to a mixing matrix processor 1209 and a covariance matrix estimator 1203.

[0311] The time-frequency signal may then be adaptively processed in the frequency band using a hybrid matrix processor (and possibly also a decorrelation processor) 1209 and the result in the form of a time-frequency output signal 1212 transformed back into the time domain to provide a processed output in the form of a spatialized audio signal 1214. Hybrid matrix processing methods are described in detail, for example, in Vilkamo, and Kuntz, “Optimized covariance domain framework for time-frequency processing of spatial audio,” Audio Engineering Society Journal 61.6 (2013): 403–411.

[0312] In order to apply the mixing matrix processing, a mixing matrix in a frequency band 1210 is required. In some embodiments, the mixing matrix 1210 may be formulated in a mixing matrix determiner 1207. The mixing matrix determiner 1207 is configured to receive an input covariance matrix in a frequency band 1206 and a target covariance matrix in a frequency band 1208.

[0313] The covariance matrix 1206 in the frequency band is determined only in the covariance matrix estimator 1203 and is measured based on the down-mix signal in the frequency band from the time-frequency domain transformer 1201 .

[0314] In some embodiments, the target covariance matrix is ​​formulated in a target covariance matrix determiner 1205 .

[0315] In some embodiments, the target covariance matrix determiner 1205 is configured to determine a target covariance matrix for reproduction to a surround speaker setup. In the following expressions, the time and frequency indices n and k are removed for simplicity (where not required).

[0316] First, the target covariance matrix determiner 1205 may be configured to estimate a total energy E 1204 of the target covariance matrix based on the input covariance matrix from the covariance matrix estimator 1203. In some embodiments, the total energy E may be determined from the sum of the diagonal elements of the input covariance matrix.

[0317] Then, the target covariance matrix determiner 1205 may be configured to determine a target covariance matrix CT in the mutually incoherent part, the directional part CD and the ambient or non-directional part CA.

[0318] Therefore, the target covariance matrix is ​​determined by the target covariance matrix determiner 1205 as CT=CD+CA.

[0319] The ambient part CA represents the spatially surrounding sound energy, which was previously only incoherent, but thanks to the invention it can be incoherent or coherent, or partially coherent.

[0320] Therefore, the target covariance matrix determiner 1205 can be configured to determine the ambient energy as (1-r)E, where r is the direct total energy ratio parameter from the input metadata. Then, the ambient covariance matrix can be determined by the following formula:

[0321]

[0322] Where I is the identity matrix, U is the 1 matrix, and M is the number of output channels. In other words, when γ is 0, the ambient covariance matrix CA is diagonal, and when γ is 1, the ambient covariance matrix makes it certain that all channel pairs are coherent.

[0323] Next, the target covariance matrix determiner 1205 may be configured to determine a direct part covariance matrix CD.

[0324] Therefore, the target covariance matrix determiner 1205 may be configured to determine the direct part energy as rE.

[0325] The target covariance matrix determiner 1205 is then configured to determine a gain vector for the speaker signal based on the metadata. First, the target covariance matrix determiner 1205 is configured to determine a vector of amplitude translation gains for the speaker setting and directional information of the spatial metadata, for example using a vector basic amplitude translation (VBAP). These gains can be represented by a column vector vVBAP, which has at most two non-zero values ​​for the two active speakers in the amplitude translation for the horizontal setting. In some embodiments, the target covariance matrix determiner 1205 can be configured to determine the VBAP covariance matrix as:

[0326]

[0327] The target covariance matrix determiner 1205 may be configured to determine the channel triplet i l 、i r 、i c , where i c is the speaker closest to the estimated direction, while the left and right speakers i l 、i r is determined as follows. First, the extended range is determined, either as a parameter input from the encoder / analysis side, or as a constant, e.g., 60 degrees, if these parameters are not available. Two new directions can be formulated by adjusting the azimuth of the direction parameter to the left and right by half the extended range parameter. The left and right speakers i l 、i r is the speaker closest to these new directions, provided that i l ≠i l ≠i c .

[0328] In some embodiments, when directional angles are provided, the left and right speakers i l and i r The closest loudspeaker is chosen as the plane of rotation other than the horizontal plane in which the plane is rotated as defined by the directional parameters.

[0329] Furthermore, the target covariance matrix determiner 1205 may be configured to determine the translation column vector vLRC to be otherwise zero, but at index i l 、i r 、i c The value is The covariance matrix for this vector is:

[0330]

[0331] When the extended coherence parameter ζ is less than 0.5, i.e., when the sound is between a “direct point source” and a “three-speaker coherent sound”, the target covariance matrix determiner 1205 may be configured to determine the direct part covariance matrix as:

[0332] C D = rE((1-2ζ)C VBAP +2ζC LRC )

[0333] When the extended coherence parameter ζ is between 0.5 and 1, that is, when the sound is between “three-speaker coherent sound” and “two-extended-speaker coherent sound”, the target covariance matrix determiner 1205 may determine the extended distribution vector as:

[0334]

[0335] Then, the target covariance matrix determiner 1205 may be configured to determine the translation vector V DISTR , where the i c The element is v DISTR,3 The first element, i l and the i r The element is v DISTR,3 The target covariance matrix determiner 1205 can then calculate the direct part covariance matrix as:

[0336]

[0337] Then, the target covariance matrix determiner 1205 can obtain the target covariance matrix CT=CD+CA to process the sound. As described above, the ambient part covariance matrix thus explains the ambient energy and spatial coherence contained in the surround coherence parameter γ, while the direct covariance matrix explains the directional energy, the directional parameter, and the extended coherence parameter ζ.

[0338] The target covariance matrix determiner 1205 may be configured to determine a target covariance matrix 1208 for binaural output by being configured to synthesize inter-aural characteristics of the surround sound instead of inter-channel characteristics.

[0339] Therefore, the target covariance matrix determiner 1205 can be configured to determine the ambient covariance matrix CA for binaural sound. The amount of ambient or non-directional energy is (1-r)E, where E is the total energy previously determined. The ambient part covariance matrix can be determined as:

[0340]

[0341] in,

[0342] c(k,n)=γ(k,n)+(1-y(k,n))c bin (k)

[0343] And among them, c bin (k) is the binaural diffuse field coherence for the frequency indexed by the kth frequency. In other words, when γ(k,n) is 1, then the ambient covariance matrix CA is such that a perfect coherence between the left and right ears is determined. When γ(k,n) is 0, then CA is such that a coherence between the left and right ears is determined that is natural for a human listener in a diffuse field (roughly: "zero" at high frequencies, "high" at low frequencies).

[0344] The target covariance matrix determiner 1205 may then be configured to determine the direct part covariance matrix CD. The amount of directional energy is rE. As will be described in detail below, the extended coherence parameter ζ may be synthesized using a method similar to that used in loudspeaker reproduction.

[0345] First, the target covariance matrix determiner 1205 may be configured to determine a 2x1 HRTF vector where θ(k,n) is the estimated azimuth, is the estimated elevation angle. The target covariance matrix determiner 1205 may determine a translation HRTF vector that is equivalent to reproducing the sound coherently in three directions:

[0346]

[0347] Among them, θ Δ The parameter defines how wide the "extended" acoustic energy is relative to the azimuth dimension. If it is provided as a parameter input, it could be, for example, 30 degrees, or half the extended range parameter.

[0348] When the extended coherence parameter ζ is less than 0.5, i.e., when the sound is between “direct point source” and “three speaker coherent sound”, the target covariance matrix determiner 1205 may be configured to determine the direct part HRTF covariance matrix as:

[0349]

[0350] When the extended coherence parameter ζ is between 0.5 and 1, that is, when the sound is between “three-speaker coherent sound” and “two-extended-speaker coherent sound”, the target covariance matrix determiner 1205 can obtain the value of the target covariance matrix by reusing the amplitude distribution vector V DISTR,3 (same as in speaker rendering) to determine the spread distribution. In turn, the combined head-related transfer function (HRTF) vector can be determined as:

[0351]

[0352] The above formula produces a weighted sum of three HRTFs, with weights V DISTR,3 . Then, the direct part HRTF covariance matrix is:

[0353]

[0354] Then, the target covariance matrix determiner 1205 is configured to obtain the target covariance CT=CD+CA to process the sound. As described above, the ambient part covariance matrix thus explains the ambient energy and spatial coherence contained in the surround coherence parameter γ, while the direct covariance matrix explains the directional energy, the directional parameter, and the extended coherence parameter ζ.

[0355] The target covariance matrix determiner 1205 may be configured to determine a target covariance matrix 1208 for the Ambisonic output by being configured to synthesize the inter-channel characteristics of the Ambisonic signal rather than the inter-channel characteristics of the speaker surrounds. Below, a first-order Ambisonic (FOA) output is used as an example, however, it is also straightforward to extend the same principle to higher-order Ambisonic outputs.

[0356] Therefore, the target covariance matrix determiner 1205 can be configured to determine the ambient covariance matrix CA for the Ambisonic sound. The amount of ambient or non-directional energy is (1-r)E, where E is the total energy previously determined. The ambient portion covariance matrix can be determined as:

[0357]

[0358] In other words, when γ(k,n) is 1, the ambient covariance matrix CA is such that only the 0th order component receives the signal. The meaning of such an Ambisonic signal is to reproduce the sound spatially coherently. When γ(k,n) is 0, CA corresponds to the Ambisonic covariance matrix in the diffuse field. The normalization of the above 0th and 1st order elements is according to the known SN3D normalization scheme.

[0359] The target covariance matrix determiner 1205 may then be configured to determine the direct part covariance matrix CD. The amount of directional energy is rE. As described in detail below, the extended coherence parameter ζ may be synthesized using a method similar to that used in loudspeaker reproduction.

[0360] First, the target covariance matrix determiner 1205 can be configured to determine the 4×1 Ambisonic translation vector Where θ(k,n) is the estimated azimuth parameter, is the estimated elevation parameter. Ambisonic translation vector Contains and Directions The corresponding Ambisonic gain. For the FOA output using the known ACN channel ordering scheme, the Ambisonic translation vector is:

[0361]

[0362] The target covariance matrix determiner 1205 may determine a translational Ambisonic vector, which is equivalent to reproducing the sound coherently in three directions:

[0363]

[0364] Where the θΔ parameter defines how wide the "extended" acoustic energy is relative to the azimuthal dimension. If it is provided as a parameter input, it could be, for example, 30 degrees, or half the extended range parameter.

[0365] When the extended coherence parameter ζ is less than 0.5, i.e., when the sound is between “direct point source” and “three speaker coherent sound”, the target covariance matrix determiner 1205 may be configured to determine the direct partial surround sound covariance matrix as:

[0366]

[0367] When the extended coherence parameter ζ is between 0.5 and 1, that is, when the sound is between “three-speaker coherent sound” and “two-extended-speaker coherent sound”, the target covariance matrix determiner 1205 can obtain the value of the target covariance matrix by reusing the amplitude distribution vector V DISTR,3 (same as in speaker rendering) to determine the spread distribution. In turn, the combined Ambisonic translation vector can be determined as:

[0368]

[0369] The above formula produces a weighted sum of three Ambisonic translation vectors, with weights V DISTR,3 . Then, the direct part of the Ambisonic covariance matrix is:

[0370]

[0371] Then, the target covariance matrix determiner 1205 is configured to obtain the target covariance matrix CT=CD+CA to process the sound. As described above, the ambient part covariance matrix thus explains the ambient energy and spatial coherence contained in the surround coherence parameter γ, while the direct covariance matrix explains the directional energy, the directional parameter, and the extended coherence parameter ζ.

[0372] In other words, the same general principles apply to constructing binaural or Ambisonic or loudspeaker target covariance matrices. The main differences are the use of HRTF data or Ambisonic panning data rather than loudspeaker amplitude panning data in the rendering of the direct portion, and the use of binaural coherence (or special Ambisonic ambient covariance matrix processing) rather than inter-channel ("zero") coherence in the rendering of the ambient portion. It should be understood that the processor may be capable of running software to achieve the above, and therefore capable of rendering each of these output types.

[0373] In the above formula, the energies of the direct and ambient parts of the target covariance matrix are weighted based on the total energy estimate E from the estimated input covariance matrix. Optionally, this weighting can be omitted, that is, the direct part energy is determined as r, and the ambient part energy is determined as (1-r). In this case, the estimated input covariance matrix is ​​normalized using the total energy estimate instead, that is, multiplied by 1 / E. The mixing matrix obtained based on this determined target covariance matrix and the normalized input covariance matrix can be completely or substantially the same as the previously provided formula, because it is the relative energies of these matrices rather than their absolute energies that have an impact.

[0374] In the above formula, the extended coherent sound to be reproduced on the same plane on the left and right sides of the direction is determined according to the directional parameter. In another embodiment, the coherent sound is reproduced using the speaker ring and the directional surrounding area parameter. In this embodiment, for example, in the case of speaker reproduction, the determined speaker ring (in the region formed by the center speaker l c In another example, the virtual surface of the determined loudspeaker (within the tolerance defined by the angle α between the central loudspeaker l) is used to reproduce the extended coherent sound corresponding to ζ=1. c The extended coherent sound corresponding to ζ=0.5 is reproduced within an angle α apart. If the angle α is provided as a parameter input, the angle α can be determined to be half of the extended range parameter, or a constant, for example, 30 degrees.

[0375] about Fig.12 , showing an overview of the synthesis operation.

[0376] Therefore, if Fig.12 As shown in step 1601, the method can receive a time domain transmission signal.

[0377] Then, as in Fig.12 As shown in step 1603, these transmission signals can be transformed in the time-frequency domain.

[0378] Then, as in Fig.12 As shown in step 1605, a covariance matrix may be estimated from the input (transmitted audio) signal.

[0379] In addition, as in Fig.12 As shown in step 1602, spatial metadata having a direction parameter, an energy ratio parameter, and a coherence parameter may be received.

[0380] As in Fig.12 As shown in step 1607, a target covariance matrix may be determined from the estimated covariance matrix, directional parameters, energy ratio parameters, and coherence parameters.

[0381] Then, as in Fig.12 As shown in step 1609, an optimal mixing matrix may be determined based on the estimated covariance matrix and the target covariance matrix.

[0382] Then, as in Fig.12 As shown in step 1611, a mixing matrix may be applied to the time-frequency downmixed signal.

[0383] Then, as in Fig.12 As shown in step 1613, the result of applying the mixing matrix to the time-frequency downmixed signal may be inversely transformed into the time-frequency domain to generate a spatialized audio signal.

[0384] about Fig.13 , showing an exemplary method for generating a target covariance matrix according to some embodiments.

[0385] First, as in Fig.13 As shown in step 1621, the total energy E of the target covariance matrix is ​​estimated based on the input covariance matrix.

[0386] Then, as in Fig.13 As shown in step 1623, the method may include: determining the ambient energy as (1-r)E, where r is a direct total energy ratio parameter from the input metadata.

[0387] In addition, as in Fig.13 As shown in step 1625, the method may include: estimating the environmental covariance matrix.

[0388] In addition, as in Fig.13 As shown in step 1624, the method may include: determining the direct partial energy as rE, where r is the direct total energy ratio parameter from the input metadata.

[0389] Then, as in Fig.13 As shown in step 1626, the method may include: determining an amplitude panning gain vector and directional information of the spatial metadata for the speaker setting.

[0390] Next, as in Fig.13 As shown in step 1628, the method may include determining a channel triplet that is the speakers closest to the estimated direction and the closest left and right speakers.

[0391] Then, as in Fig.13 As shown in step 1630, the method may include: estimating the direct covariance matrix.

[0392] Finally, as in Fig.13 As shown in step 1631, the method may include: combining the environment covariance matrix portion and the direct covariance matrix portion to generate a target covariance matrix.

[0393] The above formula discusses the construction of a target covariance matrix. The methods in US20140233762A1 and related journal publications provide more details, more relevantly, the determination and use of a "prototype matrix". The prototype matrix determines the "reference signal" for rendering with respect to which least squares optimized mixing scheme is formulated. If a stereo downmix is ​​provided in the codec as an audio signal, the prototype matrix for speaker rendering may make it possible to determine that the signal for the left speaker is optimal relative to the left channel of the provided stereo track, and similarly for the right side (the central channel may be optimized relative to the sum of the left and right audio channels). For binaural output, the prototype matrix may make it possible to determine that the reference signal for the left ear output signal is the left stereo channel, and similarly for the right ear. The determination of the prototype matrix is ​​simple for a person skilled in the art who has studied the existing literature. With respect to the existing literature, the novelty of the present invention conception is that at the synthesis stage, spatial coherence metadata is also used to construct the target covariance matrix.

[0394] Although not repeated throughout the document, it should be understood that spatial audio processing typically and in this context occurs in frequency bands. Those frequency bands can be, for example, frequency intervals of a time-frequency transform, or frequency bands that combine several frequency intervals. The combination can approximate the characteristics of human hearing, such as Bark frequency resolution. In other words, in some cases, we can measure and process audio in a time-frequency region that combines several frequency intervals b and / or time index n. For simplicity, these aspects are not expressed by all of the above formulas. In the case of combining multiple time-frequency samples, a set of parameters such as one direction is usually estimated for the time-frequency region, and then all time-frequency samples in the region are synthesized based on the parameter set such as the one directional parameters.

[0395] Using a frequency resolution in the parametric analysis that is different from the frequency resolution of the applied filter bank is a typical approach in spatial audio processing systems.

[0396] Therefore, the proposed method can detect or identify loudspeaker signals where the following general multi-channel mixing techniques have been applied:

[0397] 1) Sound is reproduced coherently using two speakers to create a "brisk" perception (e.g., using left and right front instead of center);

[0398] 2) Sound is reproduced coherently using three (or more) speakers to create a perception of "closeness" (e.g., using left front, right front, and center instead of just center);

[0399] 3) The sound is reproduced coherently from all (or nearly all) speakers to create a perception of being "in the head" or "above". In some embodiments, this detection or identification information can be passed from the encoder to the decoder using multiple (time-frequency domain) parameters. Two of these parameters are the extended coherence parameter and the surround coherence parameter. Additionally, the energy ratio parameter can be modified to improve the audio quality where such a situation has been determined as described above.

[0400] In the synthesis stage, the prior art methods (not using the proposed novel parameters) have the following problems, which are respectively:

[0401] 1) The sound is reproduced mostly as ambient: the center speaker emits dry sound, and all speakers emit de-correlated sound. This results in an ambient-like perception, while the perception of the original signal is "airy".

[0402] 2) The sound is partially reproduced as ambient: dry sound is emitted from the center speaker, and de-correlated sound is emitted from all speakers. The sound source is perceived as being far away, while it is close to the original signal.

[0403] 3) Sound is reproduced as ambient: Almost all sounds are reproduced as decorrelated from all speakers. The spatial perception is almost the opposite of the original signal.

[0404] However, in the synthesis phase of implementing the embodiments described herein, the synthesis can reproduce these cases respectively without problems (using the proposed novel parameters):

[0405] 1) The sound is reproduced coherently using two speakers, just like the original signal.

[0406] 2) The sound is reproduced coherently using the three speakers, just like the original signal.

[0407] 3) The sound is reproduced coherently using all speakers, just like the original signal.

[0408] In some embodiments, to accommodate the above analysis embodiments, the synthesis can also use a full set of output channels. In such embodiments, instead of using only three channels, all channels within the extended range are used to reproduce a coherent signal, extending the concept to the case of multiple speakers. Similarly, in some embodiments, the closest speakers around the edge of the extended range are selected as the actual edge. However, as defined in the above synthesis method, a circular area is created to serve as two clear speakers as the edge. In some embodiments, since the speaker nodes or speakers may not be exactly on this circle in all directions, a tolerance zone (e.g., 10 degrees) is defined that also allows for the inclusion of speakers that are slightly outside the extended range, thereby producing a more likely optimal circular edge.

[0409] about Fig.14 , shows an exemplary electronic device that can be used as an analysis or synthesis device. The device can be any suitable electronic device or apparatus. For example, in some embodiments, the device 1400 is a mobile device, a user device, a tablet computer, a computer, an audio playback device, etc.

[0410] In some embodiments, device 1400 includes at least one processor or central processing unit 1407. Processor 1407 may be configured to execute various program codes, such as the methods described herein.

[0411] In some embodiments, the device 1400 includes a memory 1411. In some embodiments, at least one processor 1407 is coupled to the memory 1411. The memory 1411 can be any suitable storage component. In some embodiments, the memory 1411 includes a program code portion for storing program codes that can be implemented on the processor 1407. In addition, in some embodiments, the memory 1411 may also include a storage data portion for storing data (e.g., data that has been processed or will be processed according to the embodiments described herein). Whenever necessary, the processor 1407 can obtain the implementation program code stored in the program code portion and the data stored in the storage data portion via the memory-processor coupling.

[0412] In some embodiments, device 1400 includes a user interface 1405. In some embodiments, user interface 1405 may be coupled to processor 1407. In some embodiments, processor 1407 may control the operation of user interface 1405 and receive input from user interface 1405. In some embodiments, user interface 1405 may enable a user to input commands to device 1400, for example, via a keyboard. In some embodiments, user interface 1405 may enable a user to obtain information from device 1400. For example, user interface 1405 may include a display configured to display information from device 1400 to a user. In some embodiments, user interface 1405 may include a touch screen or touch interface that enables information to be input to device 1400 and also displays information to a user of device 1400. In some embodiments, user interface 1405 may be a user interface for communicating with a position determiner as described herein.

[0413] In some embodiments, the device 1400 includes an input / output port 1409. In some embodiments, the input / output port 1409 includes a transceiver. In such embodiments, the transceiver may be coupled to the processor 1407 and configured to enable communication with other devices or electronic devices, for example, via a wireless communication network. In some embodiments, the transceiver or any suitable transceiver or transmitter and / or receiver device may be configured to communicate with other electronic devices or devices via a wired or wired coupling.

[0414] The transceiver may communicate with other devices via any suitable known communication protocol. For example, in some embodiments, the transceiver or transceiver component may use a suitable Universal Mobile Telecommunications System (UMTS) protocol, a wireless local area network (WLAN) protocol such as, for example, IEEE 802.X, a suitable short-range radio frequency communication protocol such as Bluetooth, or an infrared data communication path (IRDA).

[0415] The transceiver input / output port 1409 may be configured to receive the speaker signal and, in some embodiments, determine parameters as described herein using the processor 1407 executing appropriate code. In addition, the device may generate a suitable downmix signal and parameter output to send to a synthesis device.

[0416] In some embodiments, the apparatus 1400 may be implemented as at least part of a synthesis device. Thus, the input / output port 1409 may be configured to receive a downmix signal and, in some embodiments, receive parameters determined at a capture device or processing device as described herein, and generate a suitable audio signal format output using a processor 1407 executing suitable code. The input / output port 1409 may be coupled to any suitable audio output, such as to a multi-channel speaker system and / or headphones or the like.

[0417] As used in this application, the term "circuitry" may refer to one or more or all of the following:

[0418] (a) hardware circuit implementation only, (such as analog and / or digital circuit implementation only);

[0419] (b) a combination of hardware circuitry and software such as (where applicable):

[0420] (i) a combination of analog and / or digital hardware circuits and software / firmware; and

[0421] (ii) any portion of a hardware processor with software (including a digital signal processor, software and memory, either

[0422] Working together to enable a device such as a mobile phone or server to perform various functions); and

[0423] (c) Hardware circuits and / or processors, such as a microprocessor or portion of a microprocessor, that require software (e.g., firmware) to operate, but may not be present when the software is not required for operation.

[0424] This definition of "circuitry" applies to all uses of this term in this application, including in any claims. As another example, as used in this application, the term "circuitry" also covers an implementation of merely a hardware circuit or processor (or multiple processors) or a portion of a hardware circuit or processor and its accompanying software and / or firmware. The term "circuitry" also covers (for example and if applicable to the specifically claimed element) a baseband integrated circuit or processor integrated circuit for a mobile device, or a similar integrated circuit in a server, cellular network device, or other computing or network device.

[0425] or similar integrated circuits in servers, cellular network devices, or baseband integrated circuits or processor integrated circuits in other computing or networking devices.

[0426] Generally, various embodiments of the present invention may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be shown and described as block diagrams, flow charts, or illustrated or described using some other graphical representation, it should be fully understood that the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0427] Embodiments of the present invention may be implemented by computer software, or by hardware, or by a combination of software and hardware, the computer software being executable by a data processor of a mobile device, such as in a processor entity. In addition, in this regard, it should be noted that any box of the logic flow in the accompanying drawings may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on a physical medium, such as a memory chip, or a memory block implemented within a processor, a magnetic medium such as a hard disk or a floppy disk, and an optical medium such as a DVD and its data variants, a CD.

[0428] The memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The data processor may be of any type suitable for the local technical environment and may include, as non-limiting examples, one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a gate-level circuit, and a processor based on a multi-core processor architecture.

[0429] Embodiments of the present invention may be practiced in various components such as integrated circuit modules. The design of integrated circuits is essentially a highly automated process. Complex and powerful software tools are available to convert a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.

[0430] Programs, such as those offered by Synopsys, Inc. of Mountain View, Calif., and Cadence Design, Inc. of San Jose, Calif., can automatically route conductors and position components on a semiconductor chip using well-established design rules and a library of pre-stored design modules. Once the design of a semiconductor circuit is complete, the design, in a standardized electronic format (e.g., Opus, GDSII, etc.), can be transmitted to a semiconductor manufacturing facility or "fab" for fabrication.

[0431] The foregoing description has provided a complete and informative description of exemplary embodiments of the present invention by way of exemplary and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the relevant art in view of the foregoing description when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications to the teachings of the present invention will still fall within the scope of the present invention as defined in the appended claims.

Claims

1. A device comprising: at least one processor; as well as at least one non-transitory memory including computer program code, The at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least: determining, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determining at least one coherence parameter between the two or more speaker channel audio signals, wherein the at least one coherence parameter is configured to provide at least one inter-channel coherence information between the two or more speaker channel audio signals for a respective frequency band of at least two frequency bands of the two or more speaker channel audio signals; determining at least one value based at least in part on the at least one inter-channel coherence information; and The at least one spatial audio parameter and the at least one determined value are transmitted.

2. The device according to claim 1, wherein: The at least one inter-channel coherence information is configured to enable reproduction of the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one coherence parameter.

3. The device according to claim 1, wherein: The at least one value is configured to indicate at least one information associated with the at least one inter-channel coherence information.

4. The device according to claim 3, wherein: The at least one information associated with the at least one inter-channel coherence information includes at least one of the following: at least one orientation of the at least one coherence parameter; at least one width of the at least one coherence parameter; or At least one range of the at least one coherence parameter.

5. The device according to claim 1, wherein: The at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to determine from the two or more speaker channel audio signals at least one of: at least one directional parameter; at least one energy ratio; or A transmission audio signal, wherein the two or more speaker channel audio signals are configured to be reproduced based on at least one of: the at least one spatial audio parameter, the at least one coherence parameter, or the transmission audio signal.

6. The device according to claim 1, wherein: Determining the at least one coherence parameter comprises the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to: An extended coherence parameter is determined based on inter-channel coherence information between two or more speaker channel audio signals that are spatially adjacent to the identified speaker channel audio signal, the identified speaker channel audio signal being identified based on the at least one spatial audio parameter.

7. The device according to claim 1, wherein: The at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to: determining at least one directional parameter; determining a stereoscopic parameter configured to indicate that the two or more speaker channel audio signals are coherently reproduced using two speaker channel audio signals that are spatially adjacent to the identified speaker channel audio signal, wherein the identified speaker channel audio signal comprises a speaker channel audio signal that is spatially closest to the at least one directional parameter; determining a coherence panning parameter configured to indicate that the two or more speaker channel audio signals are coherently reproduced using at least two speaker channel audio signals that are spatially adjacent to the identified speaker channel audio signal; and An extended coherence parameter is generated based on the stereoscopic parameter and the coherence translation parameter, wherein the at least one coherence parameter includes at least the generated extended coherence parameter.

8. The device according to claim 7, wherein: Generating the extended coherence parameter comprises the at least one memory and the computer program code being configured to, with the at least one processor, further cause the apparatus to: determining a principal directional analysis to identify a speaker channel closest to the at least one directional parameter; Based on the identified loudspeaker channels, performing a search, the search comprising searching an area at a series of angular steps; estimating an average coherence value between the defined main speaker channel and any speaker channel within the zone, wherein the speaker channel is associated with the defined main speaker channel, wherein a corresponding speaker is associated with the any speaker channel; determining a substantially constant coherence region based on the average coherence value; setting the expansion range to twice the maximum coherence area of ​​the substantially constant coherence area; and Based on the extended range, a coherence translation parameter is defined.

9. The device according to claim 8, wherein: Defining the coherence translation parameters comprises the at least one memory and the computer program code being configured to, with the at least one processor, further cause the apparatus to: determining a speaker channel closest to the at least one directional parameter; determining a normalized coherence between the loudspeaker channel and any loudspeaker within the maximum coherence region; Ignore speaker channels with energies below a threshold energy; selecting the minimum coherence from the remaining speaker channels; determining an energy distribution parameter based on the energy distribution among the remaining speaker channels; The energy distribution parameter is multiplied by the maximum coherence area to determine the coherence shift parameter.

10. The device according to claim 7, wherein: Determining the stereoscopic parameter includes the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to: determining a principal directional analysis to identify a speaker channel closest to the at least one directional parameter; searching from the direction of the identified speaker channel, the searching comprising searching an area defined by angles from 0 degrees to 180 degrees in a series of angular steps; estimating an average coherence value and an average energy value for any speaker channel of at least one positioned speaker according to the region; determining a maximum coherence angle of the region based on the average coherence value and the average energy value; setting the extended range to twice the maximum coherence angle; as well as Based on the extended range, the stereoscopic parameter is defined.

11. The device according to claim 10, wherein: Defining the stereoscopic parameters based on the extended range includes the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to: identifying a first loudspeaker channel comprising maximum energy at the maximum coherence angle; determining a normalized coherence between the first loudspeaker channel and other loudspeaker channels at the maximum coherence angle; determining an average value of the normalized coherence weighted by the corresponding energy; determining an energy ratio of energy at the maximum coherence angle to energy within the maximum coherence angle; as well as The energy ratio is multiplied by the average value of the normalized coherence to form the stereoscopic parameter.

12. The device according to claim 1, wherein: The at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to: An indication of an input channel configuration associated with the two or more speaker channel audio signals is transmitted.

13. A method comprising: determining, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determining at least one coherence parameter between the two or more speaker channel audio signals, wherein the at least one coherence parameter is configured to provide at least one inter-channel coherence information between the two or more speaker channel audio signals for a respective frequency band of at least two frequency bands of the two or more speaker channel audio signals; determining at least one value based at least in part on the at least one inter-channel coherence information; and The at least one spatial audio parameter and the at least one determined value are transmitted.

14. The method according to claim 13, wherein: The at least one inter-channel coherence information is configured to enable reproduction of the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one coherence parameter.

15. The method according to claim 13, wherein: The at least one value is configured to indicate at least one information associated with the at least one inter-channel coherence information.

16. The method according to claim 15, wherein: The at least one information associated with the at least one inter-channel coherence information includes at least one of the following: at least one orientation of the at least one coherence parameter; at least one width of the at least one coherence parameter; or At least one range of the at least one coherence parameter.

17. The method according to claim 13, further comprising: determining from the two or more speaker channel audio signals at least one of: at least one directional parameter; at least one energy ratio; or A transmission audio signal, wherein the two or more speaker channel audio signals are configured to be reproduced based on at least one of: the at least one spatial audio parameter, the at least one coherence parameter, or the transmission audio signal.

18. A non-transitory computer readable medium comprising program instructions stored thereon, the program instructions, when executed by at least one processor, causing the at least one processor to: determining, for two or more speaker channel audio signals, at least one spatial audio parameter for providing spatial audio reproduction; determining at least one coherence parameter between the two or more speaker channel audio signals, wherein the at least one coherence parameter is configured to provide at least one inter-channel coherence information between the two or more speaker channel audio signals for a respective frequency band of at least two frequency bands of the two or more speaker channel audio signals; determining at least one value based at least in part on the at least one inter-channel coherence information; and The at least one spatial audio parameter and the at least one determined value are transmitted.

19. The non-transitory computer readable medium of claim 18, wherein: The at least one inter-channel coherence information is configured to enable reproduction of the two or more speaker channel audio signals based on the at least one spatial audio parameter and the at least one coherence parameter.

20. The non-transitory computer readable medium of claim 18, wherein: The at least one value is configured to indicate at least one information associated with the at least one inter-channel coherence information.

21. The non-transitory computer readable medium of claim 20, wherein: The at least one information associated with the at least one inter-channel coherence information includes at least one of the following: at least one orientation of the at least one coherence parameter; at least one width of the at least one coherence parameter; or At least one range of the at least one coherence parameter.

22. The non-transitory computer readable medium of claim 18, wherein: The program instructions stored thereon, when executed with the at least one processor, cause the at least one processor to determine from the two or more speaker channel audio signals at least one of: at least one directional parameter; at least one energy ratio; or A transmission audio signal, wherein the two or more speaker channel audio signals are configured to be reproduced based on at least one of: the at least one spatial audio parameter, the at least one coherence parameter, or the transmission audio signal.

23. The non-transitory computer readable medium of claim 18, wherein: The at least one determined value is configured to influence a cross-correlation of reproduction of the two or more speaker channel audio signals.

Images