Determination of spatial audio parameter encoding and associated decoding

Abstract

A device comprises a module for: receiving values ​​of subbands for an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each subband; determining an allocation of a first number of bits to encode the values ​​of the frame, wherein the first number of bits is fixed; encoding at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; encoding at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a subband-by-subband basis.

Description

Technical Field

[0001] The present application relates to an apparatus and method for encoding parameters related to a sound field, but is not limited to an apparatus and method for encoding parameters related to the time-frequency domain for an audio encoder and decoder. Background Art

[0002] Parametric spatial audio processing is a field of audio signal processing in which a set of parameters is used to describe the spatial aspects of sound. For example, in parametric spatial audio capture from a microphone array, estimating a set of parameters from the microphone array signal (e.g., the direction of the sound in a frequency band and the ratio between the directional and non-directional parts of the captured sound in the frequency band) is a typical and effective choice. It is well known that these parameters describe the perceived spatial properties of the captured sound at the microphone array position very well. These parameters can be used accordingly for the synthesis of spatial sound for binaural headphones, for loudspeakers or other formats such as Ambisonics.

[0003] Therefore, the directional and direct to overall energy ratios in a frequency band are particularly effective parameterizations for spatial audio capture.

[0004] A parameter set consisting of a directional parameter in a frequency band and an energy ratio parameter in a frequency band (indicating the directionality of the sound) can also be used as spatial metadata for an audio codec (which may also include other parameters, such as coherence, extended coherence, number of directions, distance, and other information). For example, these parameters can be estimated from an audio signal captured by a microphone array, and, for example, a stereo signal can be generated from the microphone array signal to be transmitted together with the spatial metadata. The stereo signal can be encoded, for example, with an AAC encoder. The decoder can decode the audio signal into a PCM signal and process the sound in the frequency band (using the spatial metadata) to obtain a spatial output, such as a binaural output.

[0005] The aforementioned solution is particularly suitable for encoding spatial sound captured from a microphone array (e.g., in a mobile phone, a VR camera, a stand-alone microphone array). However, it may be desirable for such an encoder to also have other input types than the signal captured by the microphone array, such as speaker signals, audio object signals, or panoramic sound signals.

[0006] In the scientific literature related to Directional Audio Coding (DirAC) and Harmonic Plane Wave Expansion (Harpex), the first-order Atmos (FOA) input for spatial metadata extraction is extensively analyzed. This is because there are microphone arrays that directly provide FOA signals (or more precisely, its variant, the B-format signal), so the analysis of this input has become a research focus in the said field.

[0007] Another input to the encoder is also a multi-channel speaker input, such as a 5.1 or 7.1 channel surround sound input.

[0008] However, for the directional components of the metadata, these directional components may include the elevation, azimuth (and energy ratio to 1-dispersion) of the resulting direction for each considered time / frequency subband, and the quantization of these directional components is a current research topic. Summary of the invention

[0009] According to a first aspect, a device is provided, comprising a module for: receiving values ​​for subbands of an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each subband; determining an allocation of a first number of bits to encode the values ​​of the frame, wherein the first number of bits is fixed; encoding at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; encoding at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a subband-by-subband basis.

[0010] The module for encoding the at least one energy ratio value of the frame based on the defined allocation of the second number of bits in the first number of bits can also be used to: generate a weighted average of at least one energy ratio value; and encode the weighted average of the at least one energy ratio value based on the second number of bits.

[0011] The means for encoding the weighted average of the at least one energy ratio value based on the second number of bits may also be configured to scalar non-uniformly quantize the at least one weighted average of the at least one energy ratio value.

[0012] The module for encoding the at least one azimuth value and / or at least one elevation value of the frame based on a defined allocation of the third bit number in the first bit number can also be used for the following operations, wherein the third bit number is variably distributed on a sub-band-by-sub-band basis: determining an initial estimate of the distribution of the third bit number on a sub-band-by-sub-band basis, the initial estimate being based on at least one energy ratio value associated with the sub-band; and spatially quantizing the at least one azimuth value and / or at least one elevation value based on the initial estimate of the distribution of the third bit number on a sub-band-by-sub-band basis to generate at least one azimuth index and / or at least one elevation index for each sub-band.

[0013] The module for encoding the at least one azimuth value and / or the at least one elevation value of the frame based on a defined allocation of the third number of bits in the first number of bits may also be used for the following operations, wherein the third number of bits is variably distributed on a sub-band by sub-band basis: encoding is performed on a sub-band by sub-band basis by determining a reduced distribution of the third number of bits on a sub-band by sub-band basis, the reduced estimate being based on the initial estimate and the defined allocation of the second number of bits.

[0014] The module for encoding the at least one azimuth value and / or the at least one elevation value of the frame based on the defined allocation of the third number of bits in the first number of bits can also be used to encode on a sub-band-by-sub-band basis by: determining a bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for the sub-band based on the reduced distribution; estimating the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index; estimating the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index based on the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index being less than the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index for the sub-band; less bit allocation for encoding one elevation index, entropy encoding the at least one azimuth index and / or the at least one elevation index, and otherwise, fixed rate encoding based on the bit allocation; generating signaling bits identifying the encoding of the at least one azimuth index and / or the at least one elevation index; allocating any available bits from the difference between the bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for a subband and the sum of the number of bits for encoding the subband and the signaling bits for another bit allocation for encoding at least one azimuth index and / or the at least one elevation index for another subband, or otherwise reducing the another bit allocation for encoding at least one azimuth index and / or the at least one elevation index for another subband by one bit.

[0015] The module for encoding the at least one azimuth value and / or at least one elevation value of the frame based on the defined allocation of the third number of bits in the first number of bits can also be used for encoding on a sub-band by sub-band basis by the following operations, wherein the third number of bits is variably distributed on a sub-band by sub-band basis: based on the reduced distribution, determining the bit allocation for encoding at least one azimuth index and / or at least one elevation index for the last sub-band; and based on the reduced distribution of the bit allocation, fixed-rate encoding the at least one azimuth index and / or at least one elevation index for the last sub-band.

[0016] The module for entropy encoding the at least one azimuth index and / or the at least one elevation index based on the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index may be a module for Golomb Rice encoding with two GR parameter values.

[0017] The module for encoding on a sub-band by sub-band basis by determining the distribution of the reduction of the third number of bits on a sub-band by sub-band basis can also be used for the following operations, wherein the reduction estimate is based on the initial estimate and the defined allocation of the second number of bits: uniformly reducing the bit allocation for encoding the at least one azimuth index and / or at least one elevation index on a sub-band by sub-band basis.

[0018] The module for encoding the at least one azimuth value and / or the at least one elevation value of the frame based on the defined allocation of the third bit number in the first bit number can also be used for at least one of the following operations, wherein the third bit number is variably distributed on a sub-band by sub-band basis: allocating indices for encoding in increasing order of distance from the frontal direction; allocating indices in increasing order of azimuth values.

[0019] The module may also be configured to store and / or transmit the encoded at least one energy ratio value and at least one azimuth angle value and / or at least one elevation angle value.

[0020] According to a second aspect, a device is provided, comprising a module for: receiving encoded values ​​for subbands of a frame of an audio signal, the values ​​comprising at least one azimuth index, at least one elevation index and at least one energy ratio value for each subband; decoding the encoded values ​​based on a defined bit allocation, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis.

[0021] The module for decoding the encoded value of the frame based on a defined bit allocation can also be used for the following operations, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis: based on at least one energy ratio value for each subband, determining an initial bit allocation distribution for decoding at least one azimuth index and / or at least one elevation index for each subband; determining a reduced bit allocation distribution based on the initial bit allocation distribution and the bit allocation distribution for decoding the at least one energy value of the frame; and decoding at least one azimuth index and / or at least one elevation index of the frame based on the reduced bit allocation distribution.

[0022] The module for decoding the at least one azimuth index and / or at least one elevation index of the frame based on the reduced bit allocation distribution can also be used to: determine the bit allocation for decoding the at least one azimuth index and / or at least one elevation index for a sub-band based on the reduced distribution; entropy decode the at least one azimuth index and / or at least one elevation index based on a signaling bit indicating entropy coding, otherwise perform fixed rate decoding; allocate any available bits from the difference between the bit allocation for encoding the at least one azimuth index and / or at least one elevation index for a sub-band and the sum of the number of bits for decoding the sub-band and the signaling bits for another bit allocation for decoding at least one azimuth index and / or at least one elevation index for another sub-band, or otherwise reduce the another bit allocation for decoding at least one azimuth index and / or at least one elevation index for another sub-band by one bit.

[0023] The module for decoding the at least one azimuth index and / or at least one elevation index of the frame based on the reduced bit allocation distribution can also be used to: determine the bit allocation for decoding at least one azimuth index and / or at least one elevation index for the last sub-band based on the reduced distribution; and perform fixed rate decoding on at least one azimuth index and / or at least one elevation index for the last sub-band based on the reduced bit allocation distribution.

[0024] The means for entropy decoding the at least one azimuth index and / or the at least one elevation index may be a means for Golomb Rice decoding with two GR parameter values.

[0025] According to a third aspect, a method is provided, comprising: receiving values ​​for sub-bands of an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each sub-band; determining an allocation of a first number of bits to encode the values ​​of the frame, wherein the first number of bits is fixed; encoding at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; encoding at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a sub-band by sub-band basis.

[0026] Encoding the at least one energy ratio value of the frame based on the defined allocation of the second number of bits in the first number of bits may also include: generating a weighted average of at least one energy ratio value; encoding the weighted average of the at least one energy ratio value based on the second number of bits.

[0027] The module for encoding the weighted average of the at least one energy ratio value based on the second number of bits may also include scalar non-uniformly quantizing the at least one weighted average of the at least one energy ratio value.

[0028] Encoding the at least one azimuth value and / or at least one elevation value of the frame based on the defined allocation of the third bit number in the first bit number may also include: determining an initial estimate of the distribution of the third bit number on a subband-by-subband basis, the initial estimate being based on at least one energy ratio value associated with the subband; and spatially quantizing the at least one azimuth value and / or at least one elevation value based on the initial estimate of the distribution of the third bit number on a subband-by-subband basis to generate at least one azimuth index and / or at least one elevation index for each subband.

[0029] The module for encoding the at least one azimuth value and / or the at least one elevation value of the frame based on the defined allocation of the third bit number in the first bit number may also include, wherein the third bit number is variably distributed on a sub-band by sub-band basis: encoding is performed on a sub-band by sub-band basis by determining a reduced distribution of the third bit number on a sub-band by sub-band basis, the reduced estimate being based on the initial estimate and the defined allocation of the second bit number.

[0030] Encoding the at least one azimuth value and / or the at least one elevation value of the frame based on the defined allocation of the third number of bits in the first number of bits may also include encoding on a sub-band-by-sub-band basis by: determining a bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for the sub-band based on the reduced distribution; estimating a number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index; and estimating a number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index based on the fact that the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index is less than the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index for the sub-band. bit allocation for encoding angle indices, entropy encoding the at least one azimuth index and / or the at least one elevation index, and otherwise, fixed rate encoding based on the bit allocation; generating signaling bits identifying the encoding of the at least one azimuth index and / or the at least one elevation index; allocating any available bits from the difference between the bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for a sub-band and the sum of the number of bits for encoding the sub-band and the signaling bits for another bit allocation for encoding at least one azimuth index and / or the at least one elevation index for another sub-band, or otherwise reducing the another bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for another sub-band by one bit.

[0031] Encoding the at least one azimuth value and / or at least one elevation value of the frame based on the defined allocation of the third bit number in the first bit number may also include encoding on a sub-band by sub-band basis by the following operations, wherein the third bit number is variably distributed on a sub-band by sub-band basis: based on the reduced distribution, determining the bit allocation for encoding at least one azimuth index and / or at least one elevation index for the last sub-band; and based on the reduced bit allocation distribution, fixed-rate encoding at least one azimuth index and / or at least one elevation index for the last sub-band.

[0032] Entropy encoding the at least one azimuth index and / or the at least one elevation index based on the number of bits required to entropy encode the at least one azimuth index and / or the at least one elevation index may also include Golomb Rice encoding with two GR parameter values.

[0033] Encoding on a sub-band by sub-band basis by determining the distribution of the reduction of the third number of bits on a sub-band by sub-band basis may also include the following operation, wherein the reduction estimate is based on the initial estimate and the defined allocation of the second number of bits: uniformly reducing the bit allocation for encoding the at least one azimuth index and / or at least one elevation index on a sub-band by sub-band basis.

[0034] Encoding the at least one azimuth value and / or the at least one elevation value of the frame based on the defined allocation of the third bit number in the first bit number may also include at least one of the following operations, wherein the third bit number is variably distributed on a sub-band by sub-band basis: allocating indices for encoding in increasing order of distance from the front direction; allocating indices in increasing order of azimuth values.

[0035] The method may further comprise storing and / or transmitting the encoded at least one energy ratio value and at least one azimuth angle value and / or at least one elevation angle value of the frame.

[0036] According to a fourth aspect, a method is provided, comprising: receiving encoded values ​​for subbands of a frame of an audio signal, the values ​​comprising at least one azimuth index, at least one elevation index and at least one energy ratio value for each subband; decoding the encoded values ​​based on a defined bit allocation, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis.

[0037] Decoding the encoded value of the frame based on a defined bit allocation may also include the following operations, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis: determining an initial bit allocation distribution for decoding at least one azimuth index and / or at least one elevation index for each subband based on at least one energy ratio value for each subband; determining a reduced bit allocation distribution based on the initial bit allocation distribution and the bit allocation distribution for decoding the at least one energy value of the frame; and decoding at least one azimuth index and / or at least one elevation index of the frame based on the reduced allocation of the bit distribution.

[0038] Decoding the at least one azimuth index and / or at least one elevation index of the frame based on the reduced bit allocation distribution may also include: determining the bit allocation for decoding the at least one azimuth index and / or at least one elevation index for a sub-band based on the reduced distribution; entropy decoding the at least one azimuth index and / or at least one elevation index based on a signaling bit indicating entropy coding, otherwise performing fixed rate decoding; allocating any available bits from the difference between the bit allocation for encoding the at least one azimuth index and / or at least one elevation index for a sub-band and the sum of the number of bits for decoding the sub-band and the signaling bits for allocating another bit allocation for decoding at least one azimuth index and / or at least one elevation index for another sub-band, or otherwise reducing the another bit allocation for decoding at least one azimuth index and / or at least one elevation index for another sub-band by one bit.

[0039] Decoding the at least one azimuth index and / or the at least one elevation index of the frame based on the reduced bit allocation distribution may also include: determining the bit allocation for decoding the at least one azimuth index and / or the at least one elevation index for the last sub-band based on the reduced distribution; and performing fixed rate decoding on the at least one azimuth index and / or the at least one elevation index for the last sub-band based on the reduced bit allocation distribution.

[0040] Entropy decoding the at least one azimuth index and / or the at least one elevation index may also include Golomb Rice decoding with two GR parameter values.

[0041] According to a fifth aspect, a device is provided, comprising at least one processor and at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the device to at least: receive values ​​for sub-bands of an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each sub-band; determine an allocation of a first number of bits to encode the values ​​of the frame, wherein the first number of bits is fixed; encode at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; encode at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a sub-band by sub-band basis.

[0042] The device caused to encode the at least one energy ratio value of the frame based on the defined allocation of the second number of bits in the first number of bits can also be caused to: generate a weighted average of at least one energy ratio value; and encode the weighted average of the at least one energy ratio value based on the second number of bits.

[0043] The apparatus caused to encode the weighted average of the at least one energy ratio value based on the second number of bits may also be caused to scalar non-uniformly quantize the at least one weighted average of the at least one energy ratio value.

[0044] The device that is caused to encode the at least one azimuth value and / or at least one elevation value of the frame based on the defined allocation of the third bit number in the first bit number can also be caused to perform the following operations, wherein the third bit number is variably distributed on a sub-band basis: determining an initial estimate of the distribution of the third bit number on a sub-band basis, the initial estimate being based on at least one energy ratio value associated with the sub-band; and spatially quantizing the at least one azimuth value and / or at least one elevation value on a sub-band basis based on the initial estimate of the distribution of the third bit number to generate at least one azimuth index and / or at least one elevation index for each sub-band.

[0045] The device caused to encode the at least one azimuth value and / or the at least one elevation value of the frame based on a defined allocation of the third number of bits in the first number of bits may also be caused to perform the following operations, wherein the third number of bits is variably distributed on a sub-band by sub-band basis: encoding on a sub-band by sub-band basis by determining a reduced distribution of the third number of bits on a sub-band by sub-band basis, the reduced estimate being based on the initial estimate and the defined allocation of the second number of bits.

[0046] The device caused to encode the at least one azimuth value and / or the at least one elevation value of the frame based on the defined allocation of the third number of bits in the first number of bits can also be caused to encode on a sub-band by sub-band basis by performing the following operations, wherein the third number of bits is variably distributed on a sub-band by sub-band basis: based on the reduced distribution, determining the bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for the sub-band; estimating the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index; estimating the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index based on the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index being less than the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index for the sub-band. or bit allocation for encoding at least one elevation index, entropy encoding the at least one azimuth index and / or at least one elevation index, and otherwise, fixed rate encoding based on the bit allocation; generating signaling bits identifying the encoding of the at least one azimuth index and / or at least one elevation index; allocating any available bits for another bit allocation for encoding at least one azimuth index and / or at least one elevation index for another sub-band from the difference between the bit allocation for encoding the at least one azimuth index and / or at least one elevation index for a sub-band and the sum of the number of bits for encoding the sub-band and the signaling bits, or otherwise reducing the another bit allocation for encoding at least one azimuth index and / or at least one elevation index for another sub-band by one bit.

[0047] The device that is caused to encode the at least one azimuth value and / or at least one elevation value of the frame based on the defined allocation of the third number of bits in the first number of bits can also be caused to encode on a sub-band by sub-band basis by performing the following operations, wherein the third number of bits is variably distributed on a sub-band by sub-band basis: based on the reduced distribution, determining the bit allocation for encoding at least one azimuth index and / or at least one elevation index for the last sub-band; and based on the reduced bit allocation distribution, fixed-rate encoding the at least one azimuth index and / or at least one elevation index for the last sub-band.

[0048] The device caused to entropy encode the at least one azimuth index and / or the at least one elevation index based on the number of bits required to entropy encode the at least one azimuth index and / or the at least one elevation index may also be caused to perform Golomb Rice encoding with two GR parameter values.

[0049] The device caused to encode on a sub-band by sub-band basis by determining the distribution of the reduction of the third number of bits on a sub-band by sub-band basis may also be caused to perform the following operations, wherein the reduction estimate is based on the initial estimate and the defined allocation of the second number of bits: uniformly reducing the bit allocation for encoding the at least one azimuth index and / or at least one elevation index on a sub-band by sub-band basis.

[0050] The device that is caused to encode the at least one azimuth value and / or the at least one elevation value of the frame based on the defined allocation of the third bit number in the first bit number can also be caused to perform at least one of the following operations, wherein the third bit number is variably distributed on a sub-band by sub-band basis: allocating indices for encoding in increasing order of distance from the frontal direction; allocating indices in increasing order of azimuth values.

[0051] The apparatus may also be caused to perform: storing and / or transmitting the encoded at least one energy ratio value and at least one azimuth angle value and / or at least one elevation angle value.

[0052] According to a sixth aspect, a device is provided, comprising at least one processor and at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the device to at least: receive encoded values ​​for subbands of a frame of an audio signal, the values ​​comprising at least one azimuth index, at least one elevation index and at least one energy ratio value for each subband; and decode the encoded values ​​based on a defined bit allocation, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis.

[0053] The device caused to decode the encoded value of the frame based on a defined bit allocation can also be caused to perform the following operations, wherein the decoding of the at least one azimuth index and / or the at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis: based on at least one energy ratio value for each subband, determining an initial bit allocation distribution for decoding at least one azimuth index and / or at least one elevation index for each subband; determining a reduced bit allocation distribution based on the initial bit allocation distribution and the bit allocation distribution for decoding the at least one energy value of the frame; and decoding at least one azimuth index and / or at least one elevation index of the frame based on the reduced bit allocation distribution.

[0054] The device caused to decode the at least one azimuth index and / or at least one elevation index of the frame based on the reduced bit allocation distribution can also be caused to: determine the bit allocation for decoding the at least one azimuth index and / or at least one elevation index for a sub-band based on the reduced distribution; entropy decode the at least one azimuth index and / or at least one elevation index based on a signaling bit indicating entropy coding, otherwise perform fixed rate decoding; allocate any available bits from the difference between the bit allocation for encoding the at least one azimuth index and / or at least one elevation index for a sub-band and the sum of the number of bits for decoding the sub-band and the signaling bits for another bit allocation for decoding at least one azimuth index and / or at least one elevation index for another sub-band, or otherwise reduce the another bit allocation for decoding at least one azimuth index and / or at least one elevation index for another sub-band by one bit.

[0055] The device caused to decode the at least one azimuth index and / or at least one elevation index of the frame based on the reduced bit allocation distribution can also be caused to: determine the bit allocation for decoding the at least one azimuth index and / or at least one elevation index for the last sub-band based on the reduced distribution; and perform fixed rate decoding on the at least one azimuth index and / or at least one elevation index for the last sub-band based on the reduced bit allocation distribution.

[0056] The means caused to entropy decode the at least one azimuth index and / or the at least one elevation index may also be caused to perform Golomb Rice decoding with two GR parameter values.

[0057] According to the seventh aspect, a device is provided, comprising: a module for receiving values ​​of sub-bands for an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each sub-band; a module for determining an allocation of a first number of bits to encode the values ​​of a frame, wherein the first number of bits is fixed; a module for encoding at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; and a module for encoding at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a sub-band by sub-band basis.

[0058] According to an eighth aspect, a device is provided, comprising a module for receiving encoded values ​​of subbands for a frame of an audio signal, the values ​​comprising at least one azimuth index, at least one elevation index and at least one energy ratio value for each subband; and a module for decoding the encoded values ​​based on a defined bit allocation, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis.

[0059] According to the ninth aspect, a computer program (comprising a computer-readable medium of program instructions) is provided, for causing a device to perform at least the following operations: receiving values ​​of sub-bands for an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each sub-band; determining an allocation of a first number of bits to encode the values ​​of the frame, wherein the first number of bits is fixed; encoding at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; encoding at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a sub-band by sub-band basis.

[0060] According to the tenth aspect, a method is provided comprising a computer program (or a computer-readable medium comprising program instructions), comprising a method for causing an apparatus to at least perform the following operations: receiving encoded values ​​for subbands of a frame of an audio signal, the values ​​comprising at least one azimuth index, at least one elevation index and at least one energy ratio value for each subband; decoding the encoded values ​​based on a defined bit allocation, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis.

[0061] According to the eleventh aspect, a non-transitory computer-readable medium is provided, comprising program instructions for causing a device to perform at least the following operations: receiving values ​​for sub-bands of an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each sub-band; determining an allocation of a first number of bits to encode the values ​​of the frame, wherein the first number of bits is fixed; encoding at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; encoding at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a sub-band by sub-band basis.

[0062] According to the twelfth aspect, a non-transitory computer-readable medium is provided, comprising program instructions for causing a device to perform at least the following operations: receiving encoded values ​​for subbands of a frame of an audio signal, the values ​​comprising at least one azimuth index, at least one elevation index and at least one energy ratio value for each subband; decoding the encoded values ​​based on a defined bit allocation, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis.

[0063] According to the thirteenth aspect, a device is provided, comprising: a receiving circuit configured to receive values ​​of sub-bands for an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each sub-band; an allocation circuit configured to determine an allocation of a first number of bits to encode the value of the frame, wherein the first number of bits is fixed; an encoding circuit configured to encode at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; and an encoding circuit configured to encode at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a sub-band by sub-band basis.

[0064] According to the fourteenth aspect, a device is provided, comprising: a receiving circuit configured to: receive encoded values ​​for sub-bands of a frame of an audio signal, the values ​​comprising at least one azimuth index, at least one elevation index and at least one energy ratio value for each sub-band; a decoding circuit configured to decode the encoded values ​​based on a defined bit allocation, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a sub-band-by-sub-band basis.

[0065] According to the fifteenth aspect, a computer-readable medium is provided, comprising program instructions for causing a device to perform at least the following operations: receiving values ​​of sub-bands for an audio signal frame, the values ​​comprising at least one azimuth value, at least one elevation value and at least one energy ratio value for each sub-band; determining an allocation of a first number of bits to encode the values ​​of the frame, wherein the first number of bits is fixed; encoding at least one energy ratio value of the frame based on a defined allocation of a second number of bits in the first number of bits; encoding at least one azimuth value of the frame and / or at least one elevation value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a sub-band by sub-band basis.

[0066] According to the sixteenth aspect, a computer-readable medium is provided, comprising program instructions for causing a device to perform at least the following operations: receiving encoded values ​​for subbands of a frame of an audio signal, the values ​​comprising at least one azimuth index, at least one elevation index and at least one energy ratio value for each subband; decoding the encoded values ​​based on a defined bit allocation, wherein the decoding of the at least one azimuth index and / or at least one elevation index of the frame uses a bit allocation that is variably distributed on a subband-by-subband basis.

[0067] An apparatus comprises modules for performing the actions of the method as described above.

[0068] A device is configured to perform the actions of the method described above.

[0069] A computer program includes program instructions for causing a computer to execute the method as described above.

[0070] A computer program product stored on a medium can cause an apparatus to execute the method described herein.

[0071] An electronic device may include the device described herein.

[0072] A chipset may include the device described herein.

[0073] The embodiments of the present application are intended to solve the problems associated with the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] For a better understanding of the present application, reference will now be made by way of example to the accompanying drawings, in which:

[0075] Figure 1 A system of apparatus suitable for implementing some embodiments is schematically shown;

[0076] Figure 2 schematically illustrates a metadata encoder according to some embodiments;

[0077] Figure 3 According to some embodiments, Figure 2 A flowchart of the operation of the metadata encoder is shown;

[0078] Figure 4 schematically illustrates a metadata decoder according to some embodiments;

[0079] Figure 5 According to some embodiments, Figure 4 A flowchart of the operation of the metadata decoder is shown; and

[0080] Figure 6An exemplary device suitable for implementing the illustrated means is schematically illustrated. DETAILED DESCRIPTION

[0081] Suitable devices and possible mechanisms for providing metadata parameters derived from effective spatial analysis are described in further detail below. In the following discussion, multi-channel systems are discussed with respect to multi-channel microphone implementations. However, as described above, the input format can be any suitable input format, such as multi-channel speakers, panoramic sound (FOA / HOA), etc. It should be understood that in some embodiments, the channel position is based on the position of the microphone, or a virtual position or direction. In addition, the output of the example system is a multi-channel speaker equipment. However, it should be understood that the output can be rendered to the user by a module other than the speaker. In addition, the multi-channel speaker signal can be summarized as two or more playback audio signals.

[0082] For each considered time / subband, the metadata includes at least the elevation, azimuth and energy ratio of the resulting direction. The directional parameter components, azimuth and elevation are extracted from the audio data and then quantized to a given quantization resolution. For efficient transmission, the generated index must be further compressed. For high bit rates, high-quality lossless encoding of the metadata is required.

[0083] The concept discussed below is to combine a fixed bit rate coding approach with a variable bit rate coding which distributes the coding bits of the data to be compressed between different segments so that the total bit rate per frame is fixed. Within a time-frequency block, bits can be transferred between sub-bands.

[0084] about Figure 1 , shows an example device and system for implementing an embodiment of the present application. The system 100 is shown as having an "analysis" part 121 and a "synthesis" part 131. The "analysis" part 121 is the part from receiving the multi-channel loudspeaker signal until the encoding of the metadata and the downmix signal, and the "synthesis" part 131 is the part from decoding the encoded metadata and the downmix signal to the presentation of the regenerated signal (e.g., in a multi-channel loudspeaker form).

[0085] The input to the system 100 and the "analysis" section 121 is a multi-channel signal 102. In the following example, a microphone channel signal input is described, but any suitable input (or synthesized multi-channel) format may be implemented in other embodiments. For example, in some embodiments, the spatial analyzer and spatial analysis may be implemented outside the encoder. For example, in some embodiments, spatial metadata associated with the audio signal may be provided to the encoder as a separate bitstream. In some embodiments, the spatial metadata may be provided as a set of spatial (directional) index values.

[0086] The multi-channel signal is passed to a downmixer 103 and an analysis processor 105 .

[0087] In some embodiments, downmixer 103 is configured to receive a multi-channel signal and downmix the signal to a determined number of channels and output a downmix signal 104. For example, downmixer 103 can be configured to generate a 2-channel audio downmix of the multi-channel signal. The determined number of channels can be any suitable number of channels. In some embodiments, downmixer 103 is optional, and the multi-channel signal is passed to encoder 107 unprocessed in the same manner as the downmix signal in this example.

[0088] In some embodiments, the analysis processor 105 is further configured to receive the multi-channel signal and analyze the signal to generate metadata 106 associated with the multi-channel signal and thus associated with the downmix signal 104. The analysis processor 105 can be configured to generate metadata, which can include a direction parameter 108 and an energy ratio parameter 110 (as well as a coherence parameter and a diffuseness parameter in some embodiments) for each time-frequency analysis interval. In some embodiments, the direction and energy ratio can be considered as spatial audio parameters. In other words, the spatial audio parameters include parameters intended to characterize the sound field created by the multi-channel signal (or generally two or more playback audio signals).

[0089] In some embodiments, the parameters generated may differ between frequency bands. Thus, for example, in frequency band X, all parameters are generated and transmitted, while in frequency band Y, only one parameter is generated and transmitted, and furthermore, in frequency band Z, no parameters are generated or transmitted. A practical example of this may be that for some frequency bands, such as the highest frequency band, some parameters are not required for perceptual reasons. The downmix signal 104 and the metadata 106 may be passed to an encoder 107.

[0090] The encoder 107 may include an audio encoder core 109 configured to receive the downmix (or other) signals 104 and generate appropriate encodings of these audio signals. In some embodiments, the encoder 107 may be a computer (running appropriate software stored on a memory and at least one processor), or a specific device using, for example, an FPGA or ASIC. Any suitable scheme may be used to implement the encoding. The encoder 107 may also include a metadata encoder / quantizer 111 configured to receive metadata and output an encoded or compressed form of the information. In some embodiments, Figure 1 The encoder 107 may further interleave, multiplex into a single data stream or embed metadata into the encoded downmix signal before transmission or storage as shown in dashed lines. Any suitable scheme may be used to achieve multiplexing.

[0091] On the decoder side, the received or retrieved data (stream) may be received by a decoder / demultiplexer 133. The decoder / demultiplexer 133 may demultiplex the encoded stream and pass the audio encoded stream to a downmix extractor 135, which is configured to decode the audio signal to obtain a downmix signal. Similarly, the decoder / demultiplexer 133 may include a metadata extractor 137, which is configured to receive the encoded metadata and generate metadata. In some embodiments, the decoder / demultiplexer 133 may be a computer (running appropriate software stored in a memory and at least one processor), or a specific device such as an FPGA or ASIC may be used.

[0092] The decoded metadata and downmixed audio signal may be passed to a synthesis processor 139 .

[0093] The “Synthesis” portion 131 of the system 100 also shows a synthesis processor 139 which is configured to receive the downmix and metadata and to recreate synthesized spatial audio in the form of multi-channel signals 110 in any suitable format based on the downmix signal and the metadata (these may be multi-channel speaker formats, or other formats in some embodiments, such as binaural or panoramic signals, depending on the use case).

[0094] Therefore, in summary, firstly the system (analysis component) is configured to receive a multi-channel audio signal.

[0095] The system (analysis part) is then configured to generate a downmix or otherwise generate a suitable transmission audio signal (eg by selecting some audio signal channels).

[0096] The system is then configured to downmix (or more generally transmit) the signal for encoding for storage / transmission.

[0097] Thereafter, the system may store / transmit the encoded downmix and metadata.

[0098] The system may retrieve / receive the encoded downmix and metadata.

[0099] The system is then configured to extract the downmix and metadata from the encoded downmix and metadata parameters, eg demultiplex and decode the encoded downmix and metadata parameters.

[0100] The system (synthesis section) is configured to synthesize the output multi-channel audio signal based on the downmix of the extracted multi-channel audio signal and the metadata.

[0101] about Figure 2 , further details an example analysis processor 105 and metadata encoder / quantizer 111 (eg, Figure 1 shown).

[0102] In some embodiments, the analysis processor 105 comprises a time-frequency domain transformer 201 .

[0103] In some embodiments, the time-frequency domain converter 201 is configured to receive the multi-channel signal 102 and apply an appropriate time-frequency domain transform, such as a short-time Fourier transform (STFT), to convert the input time domain signal into a suitable time-frequency signal. These time-frequency signals can be passed to the spatial analyzer 203 and the signal analyzer 205.

[0104] Thus, for example, the time-frequency signal 202 may be represented in the time-frequency domain as

[0105] s i (b, n),

[0106] Where b is the frequency bin index, n is the time-frequency block (frame) index, and i is the channel index. In another expression, n can be considered as a time index with a lower sampling rate than the original time domain signal. These frequency bins can be grouped into subbands, which group one or more frequency bins into subbands with band indices k=0, ..., K-1. Each subband k has the lowest bin b k，low and the highest warehouse b k，high , and the subband contains k，low to b k，high The width of the sub-bands can be approximated by any suitable distribution, such as the Equivalent Rectangular Bandwidth (ERB) scale or the Bark scale.

[0107] In some embodiments, the analysis processor 105 comprises a spatial analyzer 203. The spatial analyzer 203 may be configured to receive the time-frequency signals 202 and based on these signals estimate the directional parameters 108. The directional parameters may be determined based on any audio-based determination of "direction".

[0108] For example, in some embodiments, spatial analyzer 203 is configured to estimate direction using two or more signal inputs. This represents the simplest configuration for estimating "direction", and more complex processing can be performed using more signals.

[0109] Therefore, the spatial analyzer 203 may be configured to provide at least one azimuth and elevation angle for each frequency band and time-frequency block within a frame of the audio signal, denoted as azimuth angle and the elevation angle θ(k,n). The direction parameter 108 may also be passed to the direction index generator 205.

[0110] The spatial analyzer 203 may also be configured to determine an energy ratio parameter 110. The energy ratio may be considered as a determination of the energy of the audio signal arriving from a direction. For example, a stability metric of the directional estimate may be used, or any correlation metric may be used, or any other suitable method may be used to obtain a ratio parameter to estimate the direct to overall energy ratio r(k, n). The energy ratio may be passed to an energy ratio analyzer 221 and an energy ratio encoder 223.

[0111] Thus, in summary, the analysis processor is configured to receive a time domain multi-channel or other format, such as a microphone or panoramic sound audio signal.

[0112] Thereafter, the analysis processor may apply a time-domain to frequency-domain transform (eg, STFT) to generate a suitable time-frequency domain signal for analysis, and then apply directional analysis to determine direction and energy ratio parameters.

[0113] The analysis processor may then be configured to output the determined parameters.

[0114] Although directions and ratios are expressed here for each time index n, in some embodiments the parameters may be combined over multiple time indices. As already expressed, the same applies to the frequency axis, the directions of multiple frequency bins b may be expressed by one directional parameter in a frequency band k consisting of multiple frequency bins b. The same applies to all spatial parameters discussed herein.

[0115] Also like Figure 2 As shown, an exemplary metadata encoder / quantizer 111 is shown according to some embodiments.

[0116] The metadata encoder / quantizer 111 may include an energy ratio analyzer (or quantization resolution determiner) 221. The energy ratio analyzer 221 may be configured to receive the energy ratio and generate from the analysis the quantization resolution of the directional parameter (in other words, the quantization resolution for the elevation and azimuth values) for all time-frequency blocks in the frame. The bit allocation may be defined, for example, by bits_dir0[0:N-1][0:M-1].

[0117] The metadata encoder / quantizer 111 may include a direction index generator 205. The direction index generator 205 is configured to receive a direction parameter (such as an azimuth angle and elevation angles 108 and quantization bit allocation) and thereby generate a quantized output. In some embodiments, the quantization is based on an arrangement of spheres forming a spherical grid arranged in a ring on a "surface" sphere, the "surface" sphere being defined by a lookup table defined by a determined quantization resolution. In other words, the idea of ​​the spherical grid usage is to cover a sphere with smaller spheres and to consider the centers of the smaller spheres as points defining a grid of nearly equidistant directions. The smaller spheres thus define a cone or solid angle around the center point, which cone or solid angle can be indexed according to any suitable indexing algorithm. Although spherical quantization is described here, any suitable quantization, linear or non-linear, may be used.

[0118] For example, in some embodiments, the bits of the direction parameters (azimuth and elevation) are allocated according to the bits_direction[] table; if the energy ratio has index i, the number of bits for the direction is bits_direction[i].

[0119] const short bits_direction[]={

[0120] 3, 5, 6, 8, 9, 10, 11, 11};

[0121] The following variants give the structure of the directional quantizer for different bit resolutions:

[0122] const short no_theta[] = / * from 1 to 11 bits * /

[0123] { / *1, -1bit

[0124] 1,* / / *2bits* /

[0125] 1, / *3 bits* /

[0126] 2, / *4 bits* /

[0127] 4, / *5 bits* /

[0128] 5, / *6 bits* /

[0129] 6, / *7 bits* /

[0130] 7, / *8 bits* /

[0131] 10, / *9 bits* /

[0132] 14, / *10 bits* /

[0133] 19 / *11bits* /

[0134] };

[0135] const short no_phi[][MAX_NO_THETA] = / * from 1 to 11 bits * /

[0136] {

[0137] {2},

[0138] {4},

[0139] {8},

[0140] {12, 4}, / * There is no point at the pole * /

[0141] {12, 7, 2, 1},

[0142] {14, 13, 9, 2, 1},

[0143] {22, 21, 17, 11, 3, 1},

[0144] {33, 32, 29, 23, 17, 9, 1},

[0145] {48, 47, 45, 41, 35, 28, 20, 12, 2, 1},

[0146] {60, 60, 58, 56, 54, 50, 46, 41, 36, 30, 23, 17, 10, 1},

[0147] {89, 89, 88, 86, 84, 81, 77, 73, 68, 63, 57, 51, 44, 38, 30, 23, 15, 8, 1}

[0148] };

[0149] "no_theta" corresponds to the number of elevation values ​​in the "northern hemisphere" of the directional sphere (including the equator). "no_phi" corresponds to the number of azimuth values ​​per quantizer at each elevation.

[0150] For example, for 5 bits, there are 4 elevation values ​​corresponding to [0, 30, 60, 90], and 4-1=3 negative elevation values ​​[-30, -60, -90]. For the first elevation value 0, there are 12 equidistant azimuth values, for elevation values ​​30 and -30, there are 7 equidistant azimuth values, and so on.

[0151] Except for the structure corresponding to 4 bits, the difference between consecutive elevation values ​​for all quantization structures is 90 degrees divided by the number of elevation values ​​"no_theta". The structure corresponding to 4 bits has points only for 0 and +45 degree elevation values. The structure has no points below the equator. This is an example, and any other suitable distribution can be implemented. For example, in some embodiments, a spherical grid for 4 bits can be implemented, which also has points below the equator. Similarly, a 3-bit distribution can be extended on a sphere or limited to the equator.

[0152] The quantized indices for the subbands within a set of time blocks may then be passed to the directional index encoder 225. The directional index encoder 225 may then be configured to encode the index values ​​on a subband basis.

[0153] The direction index encoder 225 may therefore be configured to reduce the number of allocated bits bits_dir1[0:N-1][0:M-1] such that the sum of the allocated bits is equal to the number of available bits remaining after the coding energy ratio.

[0154] In some embodiments, the reduction of the number of initially allocated bits, in other words, from bits_dir0[0:N-1][0:M-1] to bits_dir1[0:N-1][0:M-1], may be achieved by:

[0155] First, the number of bits on the time / frequency block is uniformly reduced by a number of bits given by the integer division between the bits to be reduced and the number of time / frequency blocks;

[0156] Secondly, the bits that still need to be subtracted start from subband 0, time-frequency block 0, and one is subtracted for each time-frequency block.

[0157] For example, this can be achieved with the following C code:

[0158]

[0159]

[0160]

[0161] In some embodiments, a minimum number of bits greater than 0 may be imposed for each block.

[0162] The direction index encoder 225 may then be configured to implement the reduced number of bits allowed on a sub-band basis.

[0163] For example, the directional index encoder 225 may be configured to determine from the first subband to the second last subband based on the calculated allowed number of bits for the current subband. In other words, from i=1 to N-1, bits_allowed=sum(bits_dir1[i][0:M-1]).

[0164] The directional index encoder can then be configured to attempt to encode the directional parameter index using a suitable entropy coding and determine how many bits are required for the current subband (bits_ec). If this is less than a suitable fixed rate coding mechanism using the determined reduced number of allocated bits (bits_fixed=bits_allowed), entropy coding is selected. Otherwise, a fixed rate coding method is selected.

[0165] In addition, one bit is used to indicate the selected method.

[0166] In other words, the number of bits used to encode the subband direction index is:

[0167] nb=min(bits_fixed, bits_ec)+1;

[0168] The directional index encoder may then be configured to determine whether there are bits remaining from the sub-band "pool" of available bits.

[0169] For example, the direction index encoder 225 may be configured to determine the difference

[0170] diff = (allowed_bits - nb)

[0171] In case diff > 0, in other words there are unused bits from the allocation, these bits may be reallocated to subsequent sub-bands, for example by updating the distribution defined by the array bits_dir1[i+1:N-1][0:M-1].

[0172] In case diff=0 or < 0, then one bit is subtracted from the allocation of the subsequent subband. For example, by updating the distribution defined by the array bits_dir1[i+1][0]

[0173] After all subbands except the last have been encoded, the last subband index value is encoded using fixed rate coding using the bit allocation defined by dir1[N-1][0:M-1] bits.

[0174] These may then be passed to the combiner 207 .

[0175] In some embodiments, the encoder includes an energy ratio encoder 223. The energy ratio encoder 223 may be configured to receive the determined energy ratios (eg, direct to overall energy ratio, and further diffuse to overall energy ratio and residual to overall energy ratio) and encode / quantize them.

[0176] For example, in some embodiments, the energy ratio encoder 223 is configured to apply scalar non-uniform quantization using 3 bits per sub-band.

[0177] Furthermore, in some embodiments, the energy ratio encoder 223 is configured to generate a weighted average per frequency subband. In some embodiments, the average is calculated by considering the total energy of each time-frequency block and applying a weighting based on the frequency subband having more energy.

[0178] The energy ratio encoder 223 may then pass it to a combiner configured to combine the metadata and output combined encoded metadata.

[0179] about Figure 3 , showing that Figure 2 The operation of the metadata encoder / quantizer 111 is shown.

[0180] like Figure 3 As shown in step 301, the initial operation is an operation of obtaining metadata (azimuth angle value, elevation angle value, energy ratio).

[0181] The metadata for each sub-band (i=1:N) has been obtained and is ready for initial distribution or allocation, and as Figure 3 As shown in step 303, the corresponding energy ratio value is encoded using 3 bits, and then the quantization resolution for azimuth and elevation is set for all time-frequency blocks of the current sub-band. The quantization resolution is set by allowing a predefined number of bits to be given by the value of the energy ratio, bits_dir0[0:N-1][0:M-1].

[0182] like Figure 3 As shown in step 305, after the initial allocation is generated, the number of allocated bits is reduced, namely bits_dir1[0:N-1][0:M-1] (the sum of the number of allocated bits = the number of available bits remaining after the coding energy ratio).

[0183] The reduced bit allocation is then achieved by implementing the following operations for the subbands up to the penultimate subband (in other words, for each subband i=1:N-1) (or, if zero bits are allocated to the last subband, the "bit pass" process can be implemented only in the subband before the penultimate subband (1:N-2)): Calculate the allowed bits for the current subband: bits_allowed=sum(bits_dir1[i][0:M-1]). The directional parameter index is encoded using the reduced number of allocated bits (using fixed rate coding or entropy coding, whichever uses fewer bits) and indicates the coding selection. If there are available bits relative to the allowed bits: reallocate the difference to the subsequent subbands (by updating bits_dir1[i+1:N-1][0_M-1]), otherwise subtract one bit from bits_dir1[i+1][0]. This is done in Figure 3 This is shown by step 307.

[0184] like Figure 3 As shown in step 309 , for the last sub-band, bits_dir1[N−1][0:M−1] bits are used to encode the directional parameter index for the last sub-band in a fixed rate method.

[0185] about Figure 4 , an example metadata extractor 137 is shown as part of the decoder 133.

[0186] In some embodiments, the encoded data stream is passed to a demultiplexer 401. The demultiplexer 401 is shown extracting the encoded energy ratio and the encoded directional index, and in some embodiments may also extract other metadata and a transmission audio signal (not shown).

[0187] The energy ratio may be output and may also be passed to an energy ratio analyzer (quantization resolution determiner) where an analysis similar to that performed within the metadata encoder energy ratio analyzer (quantization resolution determiner) generates an initial bit allocation for the direction information. This is passed to the direction index decoder 405.

[0188] The direction index decoder 405 may also receive the encoded direction index from the demultiplexer.

[0189] The direction index decoder 405 may be configured to determine a reduced bit allocation for the direction value in a manner similar to that performed within the encoder.

[0190] Then, the direction index decoder 405 may also be configured to read one bit to determine whether all elevation angle data are 0 (in other words, the direction value is 2D).

[0191] In case the direction value is 3D, the count value of nb_last for the last sub-band allocation is determined.

[0192] If the value nb_last is 0, the last subband to be decoded is N-1, otherwise the last subband to be decoded is N.

[0193] Then, on a subband-by-subband basis from the first subband to the last subband (N or N-1 according to the previous determination), the directional index decoder 405 is configured to determine whether the encoding of the current subband uses a fixed rate code or a variable rate code.

[0194] In case a fixed rate code is used at the encoder, then the spherical index (or other index distribution) is read and decoded to obtain the elevation and azimuth values ​​and the bit allocation for the next subband is reduced by one.

[0195] In the case where a variable rate code is used at the encoder, the entropy coded index is read and decoded to generate the elevation and azimuth values. The number of bits used in the entropy coded information is then counted and the difference between the allowed bits for the current subband and the bits used in the entropy coding is determined. Thereafter, the difference bits are allocated to subsequent subbands.

[0196] Then, the last subband is decoded based on the fixed rate code.

[0197] In case the directional value is 2D, then for each subband, the index is decoded based on the fixed rate coded azimuth index.

[0198] about Figure 5 , a flowchart for decoding of an example encoded bit stream is shown.

[0199] Thus, for example, the first operation would be to obtain metadata (azimuth value, elevation value, energy ratio), such as Figure 5 As shown in step 501.

[0200] Then, the method can estimate the initial bit allocation for the directional information based on the energy ratio, such as Figure 5 As shown in step 503.

[0201] Then as Figure 5 As shown in step 505, the available bit allocation can be reduced, bits_dir1[0:N-1][0:M-1] (the sum of the allocated bits = the number of remaining available bits that can be used to decode the direction information).

[0202] One bit is then read to determine whether all elevation angle data is 0 (2D data).

[0203] If the directional data is 3D, then Figure 5 As shown in step 509,

[0204] Count the number of available bits for the last subband (nb_last), where (nb_last=0), then last_j=N-1, otherwise Last_j=N;

[0205] For each subband from j=1:last_j-1

[0206] Read 1 bit to determine whether the encoding is fixed rate or variable rate

[0207] If it is fixed rate encoding:

[0208] Read and decode the spherical index for directivity information, obtain the elevation and azimuth values, and subtract 1 bit from the bit for the next sub-band;

[0209] otherwise, reading and decoding entropy coding indices for elevation and azimuth, counting the number of bits used in the entropy coding information, calculating the difference between the allowed bits for the current subband and the bits used in the entropy coding, and allocating the difference bits for the next subband;

[0210] End for(loop);

[0211] For each subband from j=last_j:N: read and decode the fixed rate coded sphere index for directionality data.

[0212] If the directional data is 2D, then for each subband from j=1:N: decode the fixed rate coded azimuth index, such as Figure 5 As shown in step 511.

[0213] In some embodiments, the entropy coding / decoding of the azimuth and elevation indices can be implemented using the Golomb Rice coding method with two possible values ​​of the Golomb Rice parameter. In some embodiments, any suitable entropy coding technique (e.g., Huffman, arithmetic coding ...) can also be used to implement entropy coding.

[0214] In some embodiments, when encoding / decoding the elevation index, there may be a few exceptions for cases where the number of bits used for quantization is less than or equal to 3. For these cases, there is only one elevation value, so the index of the elevation angle does not need to be encoded / decoded, and only the azimuth index is needed.

[0215] If all time-frequency blocks for a subband use less than 4 bits, no bits are sent for elevation coding, otherwise one bit is sent to specify the Golomb Rice parameter, and the remaining bits correspond to the Golomb Rice code for time-frequency blocks using more than 3 bits. The Golomb Rice parameter is either 1 or 0. The choice of the GR parameter value is based on the estimated bit consumption in each case, and the one with the smaller number of bits is selected.

[0216] For example, this can be achieved using the following C code

[0217]

[0218]

[0219]

[0220] The encoding / decoding of the azimuth angle can be achieved using Golomb Rice coding with two GR parameter values. The two values ​​are 1 and 2. The selection of the GR parameter value is done by estimating the number of bits in the two cases and selecting the one with the smaller number of bits. A special case is considered when the number of allocated bits for at least one time-frequency block of a subband is less than or equal to 1. If this is the case (the "use_context" case in the C function below), the corresponding block information is encoded with 1 or 0 bits based on the number of allocated bits, while the remaining time-frequency blocks are encoded using GR coding with parameter equal to 1.

[0221]

[0222]

[0223]

[0224]

[0225] In some embodiments, the indexing of the azimuth values ​​is implemented so that instead of assigning indices in increasing order of azimuth values, the indices are assigned in increasing order of distance from the frontal direction. In other words, if the quantized azimuth values ​​are -180, -135, -90, -45, 0, 45, 90, 135, respectively, they do not get indices: 0, 1, 2, 3, 4, 5, 6, 7, but rather 7, 5, 3, 1, 0, 2, 4, 6. This can ensure in some embodiments that the average value of the azimuth index is lower and entropy coding is more efficient.

[0226] The holistic coding approach allows to shift bits from one sub-band to another, thus allowing for a better adaptation to the local data statistics.

[0227] about Figure 6 , shows an example electronic device that can be used as an analysis or synthesis device. The device can be any suitable electronic device or apparatus. For example, in some embodiments, the device 1400 is a mobile device, a user device, a tablet computer, a computer, an audio playback device, etc.

[0228] In some embodiments, device 1400 includes at least one processor or central processing unit 1407. Processor 1407 may be configured to execute various program codes, such as the methods described herein.

[0229] In some embodiments, the device 1400 includes a memory 1411. In some embodiments, at least one processor 1407 is coupled to the memory 1411. The memory 1411 can be any suitable storage device. In some embodiments, the memory 1411 includes a program code portion for storing program code that can be implemented on the processor 1407. In addition, in some embodiments, the memory 1411 can further include a storage data portion for storing data (e.g., data that has been processed or will be processed according to the embodiments described herein). Whenever necessary, the implemented program code stored in the program code portion and the data stored in the storage data portion can be retrieved by the processor 1407 through the memory-processor coupling.

[0230] In some embodiments, device 1400 includes a user interface 1405. In some embodiments, user interface 1405 can be coupled to processor 1407. In some embodiments, processor 1407 can control the operation of user interface 1405 and receive input from user interface 1405. In some embodiments, user interface 1405 can enable a user to enter commands to device 1400, such as through a keyboard. In some embodiments, user interface 1405 can enable a user to obtain information from device 1400. For example, user interface 1405 can include a display configured to display information from device 1400 to a user. In some embodiments, user interface 1405 can include a touch screen or touch interface that both enables information to be entered into device 1400 and displays information to a user of device 1400. In some embodiments, user interface 1405 can be a user interface for communicating with a position determiner as described herein.

[0231] In some embodiments, the device 1400 includes an input / output port 1409. In some embodiments, the input / output port 1409 includes a transceiver. In such embodiments, the transceiver can be coupled to the processor 1407 and configured to enable communication with other devices or electronic devices, for example, via a wireless communication network. In some embodiments, the transceiver or any suitable transceiver or transmitter and / or receiver device can be configured to communicate with other electronic devices or devices via a wired or wired coupling.

[0232] The transceiver can communicate with another device via any suitable known communication protocol. For example, in some embodiments, the transceiver can use a suitable Universal Mobile Telecommunications System (UMTS) protocol, a wireless local area network (WLAN) protocol, such as IEEE 802.X, a suitable short-range radio frequency communication protocol, such as Bluetooth, or an infrared data communication path (IRDA).

[0233] As described herein, transceiver input / output port 1409 may be configured to receive signals and, in some embodiments, determine parameters by executing appropriate code using processor 1407. In addition, the device may generate appropriate downmix signals and parameter outputs for transmission to a synthesis device.

[0234] In some embodiments, the apparatus 1400 can be used as at least part of a synthesis device. Thus, the input / output port 1409 can be configured to receive the downmix signal and the parameters determined at the capture device or processing device as described herein in some embodiments, and generate a suitable audio signal format output by executing suitable code using the processor 1407. The input / output port 1409 can be coupled to any suitable audio output, such as a multi-channel speaker system and / or headphones, etc.

[0235] Generally, various embodiments of the present invention may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software, which may be performed by a controller, microprocessor, or other computing device, although the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described as block diagrams, flow charts, or using some other graphical representation, it is understood that the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0236] Embodiments of the present invention can be implemented by computer software executable by a data processor of a mobile device, for example in a processor entity, or by hardware, or by a combination of software and hardware. In addition, it should be noted that any block of the logic flow in the figure can represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software can be stored in physical media such as memory chips or storage blocks implemented in processors, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and data variants thereof, CDs, etc.

[0237] The memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based storage devices, magnetic storage devices and systems, optical storage devices and systems, fixed memory, and removable memory. As non-limiting examples, the data processor may be of any type suitable for the local technical environment and may include general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), gate-level circuits, and one or more processors based on multi-core processor architectures.

[0238] Embodiments of the present invention may be practiced in various components such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available to convert logic-level designs into semiconductor circuit designs that are easily etched and formed on semiconductor substrates.

[0239] Programs, such as those offered by Synopsys, Inc. of Mountain View, Calif., and Cadence Design, of San Jose, Calif., can automatically route and position components on a semiconductor semiconductor chip using well-established design rules and pre-stored design libraries. Once the design of a semiconductor circuit is complete, the final design in a standardized electronic format (e.g., Opus, GDSII, etc.) can be transmitted to a semiconductor fabrication facility or "fab" for fabrication.

[0240] The foregoing description has provided a complete and beneficial description of exemplary embodiments of the present invention by way of exemplary and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the relevant art in view of the foregoing description when read in conjunction with the accompanying drawings and the appended claims, but all such and similar modifications of the teachings of the present invention will still fall within the scope of the present invention as defined by the appended claims.

Claims

1. An apparatus for audio processing, comprising a module for the following operations: receiving values ​​for frequency subbands of a frame of an audio signal, the values ​​comprising at least one azimuth angle value, at least one elevation angle value, and at least one energy ratio value for each frequency subband; determining an allocation of a first number of bits to encode the value for the subband of the frame, wherein the first number of bits is fixed; encoding at least one energy ratio value for each subband of the frame based on a defined allocation of a second number of bits in the first number of bits; as well as By the apparatus further comprising means for determining and spatially quantizing operations, encoding at least one azimuth angle value for each sub-band of the frame and / or at least one elevation angle value for each sub-band of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a sub-band-by-sub-band basis, the determining and spatially quantizing operations are: determining an initial estimate of a distribution of the third number of bits on a frequency subband-by-frequency subband basis, the initial estimate being based on at least one energy ratio value associated with the frequency subband; as well as The at least one azimuth angle value and / or the at least one elevation angle value are spatially quantized based on an initial estimate of the distribution of the third number of bits on a subband-by-subband basis to generate at least one azimuth angle index and / or at least one elevation angle index for each subband.

2. The device according to claim 1, wherein: The means for encoding the at least one energy ratio value for each subband of the frame based on a defined allocation of the second number of bits in the first number of bits also includes means for scalar non-uniformly quantizing the at least one energy ratio value for each subband of the frame.

3. The device according to claim 1, wherein: The module for encoding the at least one azimuth value and / or the at least one elevation value for each subband of the frame based on a defined allocation of the third number of bits in the first number of bits also includes a module for the following operations, wherein the third number of bits is variably distributed on a subband-by-subband basis: encoding on a subband-by-subband basis by determining a reduced distribution of the third number of bits on a subband-by-subband basis, the reduced distribution being based on the initial estimate and the defined allocation of the second number of bits.

4. The device according to claim 3, wherein: The means for encoding the at least one azimuth angle value and / or the at least one elevation angle value for each subband of the frame based on a defined allocation of the third number of bits in the first number of bits further comprises means for encoding on a subband-by-subband basis, wherein the third number of bits is variably distributed on a subband-by-subband basis by: determining, based on the reduced distribution, a bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for a frequency subband; estimating a number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index for each frequency subband; entropy encoding the at least one azimuth index and / or the at least one elevation index based on a number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index being less than a bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for the subband, and otherwise, performing fixed rate encoding based on the bit allocation; generating coded signaling bits identifying the at least one azimuth index and / or the at least one elevation index for each frequency sub-band; as well as From the difference between the bit allocation for encoding the at least one azimuth index and / or the at least one elevation index for the subband and the number of bits for entropy encoding the subband and the sum of the signaling bits, any available bits are allocated for another bit allocation for encoding at least one azimuth index and / or the at least one elevation index for another subband, or otherwise the another bit allocation for encoding at least one azimuth index and / or the at least one elevation index for the other subband is reduced by one bit.

5. The device according to claim 3, wherein: The means for encoding the at least one azimuth angle value and / or the at least one elevation angle value for each subband of the frame based on a defined allocation of the third number of bits in the first number of bits further comprises means for encoding on a subband-by-subband basis, wherein the third number of bits is variably distributed on a subband-by-subband basis by: determining, based on the reduced distribution, a bit allocation for encoding at least one azimuth index and / or the at least one elevation index for a last sub-band; and Based on the reduced distribution of the bit allocation, at least one azimuth index and / or at least one elevation index for the last frequency subband is fixed rate encoded.

6. The device according to claim 5, wherein: The module for entropy encoding the at least one azimuth index and / or the at least one elevation index based on the number of bits required for entropy encoding the at least one azimuth index and / or the at least one elevation index is a module for Golomb Rice encoding with two GR parameter values.

7. The device according to any one of claims 3 to 6, wherein: The module for encoding on a sub-band by sub-band basis by determining a distribution of the reduction of the third number of bits on a sub-band by sub-band basis also includes a module for the following operations, wherein the reduced estimate is based on the initial estimate and the defined allocation of the second number of bits: uniformly reducing the bit allocation for encoding the at least one azimuth index and / or the at least one elevation index on the sub-band by sub-band basis.

8. The device according to claim 1, wherein: The means for encoding the at least one azimuth angle value and / or the at least one elevation angle value for each subband of the frame based on a defined allocation of the third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a subband-by-subband basis, further comprises means for at least one of the following operations: assigning indices for encoding in increasing order of distance from the front direction; and The indices are assigned in increasing order of the azimuth values.

9. The device according to claim 1, wherein: The apparatus further comprises means for storing and / or transmitting the encoded at least one energy ratio value and at least one azimuth angle value and / or at least one elevation angle value for each sub-band of the frame.

10. A method for audio processing, comprising: receiving values ​​for frequency subbands of a frame of an audio signal, the values ​​comprising at least one azimuth angle value, at least one elevation angle value, and at least one energy ratio value for each frequency subband; determining an allocation of a first number of bits to encode the value for the subband of the frame, wherein the first number of bits is fixed; encoding at least one energy ratio value for each subband of the frame based on a defined allocation of a second number of bits in the first number of bits; as well as The method further comprises determining and spatially quantizing operations to encode at least one azimuth angle value for each subband of the frame and / or at least one elevation angle value of the frame based on a defined allocation of a third number of bits in the first number of bits, wherein the third number of bits is variably distributed on a subband-by-subband basis, the determining and spatially quantizing operations being: determining an initial estimate of a distribution of the third number of bits on a frequency subband-by-frequency subband basis, the initial estimate being based on at least one energy ratio value associated with the frequency subband; as well as The at least one azimuth angle value and / or the at least one elevation angle value are spatially quantized based on an initial estimate of the distribution of the third number of bits on a subband-by-subband basis to generate at least one azimuth angle index and / or at least one elevation angle index for each subband.

Images