Parametric Spatial Audio Encoding

Abstract

The apparatus comprises means for obtaining values ​​of parameters representative of the audio signal, the values ​​including at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowable number of bits for encoding the at least one direction value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and encoding the at least one direction value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, one of the entropy encoding of at least one resolution further comprising: performing entropy encoding of a first resolution of the at least one direction value, determining a number of bits used for encoding the at least one direction value based on the first entropy encoding; and encoding the at least one reduced direction value based on the first entropy encoding of the first resolution of the entropy encoding of the at least one reduced direction value. performing entropy encoding of the at least one reduced directional value based on the first resolution to determine a number of bits used for encoding the at least one reduced directional value based on the entropy encoding of the first resolution; selecting the entropy encoding of the first resolution of the at least one reduced directional value when the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of allowed bits for encoding the at least one directional value; and selecting the entropy encoding of the first resolution of the at least one reduced directional value when the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of allowed bits for encoding the at least one directional value and the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the fraction of the number of allowed bits for encoding the at least one directional value.

Description

[Technical field]

[0001] This application relates to an apparatus and method for spatial audio representation and encoding, but not exclusively to audio representation in an audio encoder. [Background technology]

[0002] Immersive audio codecs have been implemented that support a number of operating points ranging from low bitrate operation to transparency. An example of such a codec is the Immersive Voice and Audio Services (IVAS) codec, which is designed to be suitable for use over communication networks such as 3GPP 4G / 5G networks, including use in immersive services such as immersive voice and audio for virtual reality (VR). The audio codec is expected to handle the encoding, decoding, and rendering of speech, music, and general-purpose audio. Furthermore, the audio codec is expected to support channel-based and scene-based audio inputs, including spatial information about the sound field and sound sources. The codec is also expected to operate with low latency to enable conversational services as well as to support high error resilience under various transmission conditions.

[0003] Metadata-assisted spatial audio (MASA) is one input format proposed for IVAS. Metadata-assisted spatial audio uses audio signals along with corresponding spatial metadata. The spatial metadata includes parameters that define spatial aspects of the audio signal, for example these parameters can include direction and the ratio of direct energy to total energy in frequency bands. The MASA stream can be obtained, for example, by capturing spatial audio using microphones of a suitable capture device. For example, a mobile device including multiple microphones can be configured to capture microphone signals, and a set of spatial metadata can be estimated based on the captured microphone signals. The MASA stream can also be obtained from other sources, such as a specific spatial audio microphone (such as Ambisonics), a studio mix (e.g., a 5.1 audio channel mix), or other content with appropriate format conversion. Summary of the Invention

[0004] According to a first aspect, an apparatus is provided comprising means for obtaining values ​​of parameters representative of an audio signal, the values ​​including at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowable number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and encoding the at least one directional value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, one of the entropy encoding of at least one resolution further comprising: performing entropy encoding of a first resolution of the at least one directional value, determining a number of bits used for encoding the at least one directional value based on the first entropy encoding; and encoding the at least one reduced directional value based on the first entropy encoding of the at least one reduced directional value. and determining a number of bits used for encoding the at least one reduced directional value based on the first entropy encoding; selecting the entropy encoding of the first resolution of the at least one directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of allowed bits for encoding the at least one directional value; and selecting the entropy encoding of the first resolution of the at least one reduced directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of allowed bits for encoding the at least one directional value and the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the fraction of the number of allowed bits for encoding the at least one directional value.

[0005] The means for performing entropy encoding of a first resolution of at least one directional value and determining a number of bits used for encoding the at least one directional value based on the first entropy encoding may be for entropy encoding of at least one resolution of at least one value determined from a difference of the at least one directional value compared between an average directional value of the frame and determining a number of bits used for encoding the at least one value, and the means for performing entropy encoding of a first resolution of at least one reduced directional value and determining a number of bits used for encoding the at least one reduced value based on the first entropy encoding is for entropy encoding of at least one resolution of at least one reduced value from a reduced difference based on the difference of the at least one directional value compared between an average directional value of the frame and determining a number of bits used for encoding the at least one reduced value.

[0006] The means for encoding at least one directional value for at least one subframe of each subband of the frame based on the entropy encoding of the at least one resolution may further be for performing entropy encoding of a second resolution of the at least one value based on the at least one directional value and determining a number of bits to be used for encoding the at least one value based on the entropy encoding of the second resolution, where the entropy encoding of the second resolution may be a lower resolution encoding than the entropy encoding of the first resolution and exploits similarity between time-frequency tiles in the subbands in the frame when the frame includes more than one time-frequency tile within the subband; and selecting the entropy encoding of the second resolution of the at least one value when the number of bits used for encoding the at least one value based on the entropy encoding of the second resolution is less than or equal to a portion of the number of bits allowed for encoding the at least one directional value.

[0007] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.

[0008] The means may further be for selecting entropy encoding of the second resolution of the at least one value based on the at least one directional value when a number of bits used to encode the at least one value based on the entropy encoding of the second resolution is greater than a portion of the allowed number of bits for encoding the at least one directional value but less than the determined relaxed number of bits.

[0009] The relaxed number of bits may be a number of bits relative to a portion of the number of allowed bits for encoding at least one direction value.

[0010] The means may be for encoding at least one directional value for at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, one of the entropy encoding of the at least one resolution, further for performing entropy encoding of a third resolution of the at least one value based on the at least one directional value and determining a number of bits to be used for encoding the at least one value based on the entropy encoding of the third resolution, wherein a quantization resolution of the third resolution is lower than the entropy encoding of the first and second resolutions, and for selecting the entropy encoding of the third resolution of the at least one value based on the at least one directional value when the number of bits used for encoding the at least one value based on the entropy encoding of the first or second resolution is greater than a portion of the number of bits allowed for encoding the at least one value.

[0011] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.

[0012] The means may further be for encoding at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal.

[0013] The means for encoding at least one energy ratio value for at least one subframe of each subband of the frame of the audio signal may be for generating a weighted average of the at least one energy ratio value and for encoding the weighted average of the at least one energy ratio value.

[0014] The means for encoding the weighted average of the at least one energy ratio value may further be for performing a scalar non-uniform quantization of the at least one weighted average of the at least one energy ratio value.

[0015] At least one of the entropy encodings may be Golomb Rice encoding.

[0016] The means may further be for storing and / or transmitting the encoded at least one direction value.

[0017] According to a second aspect, there is provided a method comprising: obtaining values ​​of parameters representative of an audio signal, the values ​​comprising at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowable number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and encoding the at least one directional value for the at least one subframe of the frame based on entropy encoding of at least one resolution, one of the entropy encoding of at least one resolution further comprising: performing entropy encoding of a first resolution of the at least one directional value, determining a number of bits used for encoding the at least one directional value based on the first entropy encoding; and encoding the at least one directional value for the at least one subframe of the frame based on entropy encoding of at least one reduced resolution of the at least one reduced directional value. and determining a number of bits used for encoding the at least one reduced directional value based on the first entropy encoding; selecting the entropy encoding of the first resolution of the at least one directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of allowed bits for encoding the at least one directional value; and selecting the entropy encoding of the first resolution of the at least one reduced directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of allowed bits for encoding the at least one directional value and the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the fraction of the number of allowed bits for encoding the at least one directional value.

[0018] Performing entropy encoding of a first resolution of the at least one directional value and determining a number of bits used to encode the at least one directional value based on the first entropy encoding may include determining entropy encoding of at least one resolution of at least one value determined from a difference of the at least one directional value compared to an average directional value of the frame and a number of bits used to encode the at least one value, and performing entropy encoding of a first resolution of the at least one reduced directional value and determining a number of bits used to encode the at least one reduced value based on the first entropy encoding may include determining entropy encoding of at least one resolution of the at least one reduced value from a reduced difference based on a difference of the at least one directional value compared to an average directional value of the frame and a number of bits used to encode the at least one reduced value.

[0019] Encoding at least one directional value for at least one subframe of each subband of the frame based on entropy encoding of at least one resolution may include performing entropy encoding of a second resolution of the at least one value based on the at least one directional value and determining a number of bits to be used for encoding the at least one value based on the entropy encoding of the second resolution, where the entropy encoding of the second resolution is a lower resolution encoding than the entropy encoding of the first resolution and exploits similarities between time-frequency tiles within the subbands in the frame when the frame includes two or more time-frequency tiles within the subband; and selecting entropy encoding of the second resolution of the at least one value when the number of bits used for encoding the at least one value based on the entropy encoding of the second resolution is less than or equal to a portion of the number of bits allowed for encoding the at least one directional value.

[0020] The at least one direction value may be at least one value based on the at least one direction value and may be at least one difference value from the at least one direction value compared to an average direction value for the frame.

[0021] The method may further include selecting entropy encoding of the second resolution of the at least one value based on the at least one directional value when a number of bits used to encode the at least one value based on the entropy encoding of the second resolution is greater than a portion of the allowed number of bits for encoding the at least one directional value but less than the determined relaxed number of bits.

[0022] The relaxed number of bits may be a number of bits relative to a portion of the number of allowed bits for encoding at least one direction value.

[0023] The method may include encoding at least one directional value for at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, one of the entropy encoding of the at least one resolution further comprising performing entropy encoding of a third resolution of the at least one value based on the at least one directional value and determining a number of bits to be used for encoding the at least one value based on the entropy encoding of the third resolution, wherein the quantization resolution of the third resolution is lower than the entropy encoding of the first and second resolutions, and selecting the entropy encoding of the third resolution of the at least one value based on the at least one directional value if the number of bits used for encoding the at least one value based on the entropy encoding of the first or second resolution is greater than a portion of the number of bits allowed for encoding the at least one directional value.

[0024] The at least one value may be at least one difference value from the at least one orientation value compared to an average orientation value for the frame.

[0025] The method may further include encoding at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal.

[0026] Encoding at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal may include generating a weighted average of the at least one energy ratio value and encoding the weighted average of the at least one energy ratio value.

[0027] Encoding the weighted average of the at least one energy ratio value may further include performing a scalar non-uniform quantization of the at least one weighted average of the at least one energy ratio value.

[0028] At least one of the entropy encodings may be Golomb Rice encoding.

[0029] The method may further include storing and / or transmitting the encoded at least one direction value.

[0030] According to a third aspect, there is provided an apparatus comprising at least one processor and at least one memory containing computer program code, the at least one memory and the computer program code configured, together with the at least one processor, to cause the apparatus to at least perform the following: obtain values ​​of parameters representative of an audio signal, the values ​​including at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtain an allowed number of bits for encoding the at least one direction value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and encode the at least one direction value for the at least one subframe of each subband of the frame based on an entropy encoding of at least one resolution, one of the entropy encodings further includes performing entropy encoding of a first resolution of the at least one directional value and determining a number of bits used for encoding the at least one directional value based on the first entropy encoding; performing entropy encoding of a first resolution of the at least one reduced directional value and determining a number of bits used for encoding the at least one reduced directional value based on the first entropy encoding; selecting the entropy encoding of the first resolution of the at least one directional value when the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of bits allowed for encoding the at least one directional value; and selecting the entropy encoding of the first resolution of the at least one directional value when the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of bits allowed for encoding the at least one directional value.and selecting an entropy encoding of the first resolution of the at least one reduced directional value when a number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a portion of a number of bits allowed for encoding the at least one directional value.

[0031] An apparatus caused to perform entropy encoding of a first resolution of at least one directional value and determine a number of bits used for encoding the at least one directional value based on the first entropy encoding may be caused to perform entropy encoding of at least one resolution of at least one value determined from a difference of the at least one directional value compared between an average directional value of the frame and determine a number of bits used for encoding the at least one value, and an apparatus caused to perform entropy encoding of a first resolution of at least one reduced directional value and determine a number of bits used for encoding the at least one reduced directional value based on the first entropy encoding is caused to perform entropy encoding of at least one resolution of at least one reduced value from a reduced difference based on the difference of the at least one directional value compared between an average directional value of the frame and determine a number of bits used for encoding the at least one reduced value.

[0032] An apparatus caused to encode at least one directional value for at least one subframe of each subband of a frame based on entropy encoding of at least one resolution may be further caused to perform entropy encoding of a second resolution of the at least one value based on the at least one directional value, and determine a number of bits to be used for encoding the at least one value based on the entropy encoding of the second resolution, where the entropy encoding of the second resolution is an encoding of a lower resolution than the entropy encoding of the first resolution and exploits similarities between time-frequency tiles in the subbands in the frame when the frame includes two or more time-frequency tiles in the subband, and to select entropy encoding of the second resolution of the at least one value when the number of bits used for encoding the at least one value based on the entropy encoding of the second resolution is less than or equal to a fraction of a number of bits allowed for encoding the at least one directional value.

[0033] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.

[0034] The apparatus may further be caused to select entropy encoding of the second resolution of the at least one value based on the at least one directional value when a number of bits used to encode the at least one value based on the entropy encoding of the second resolution is greater than a portion of the allowed number of bits for encoding the at least one directional value but less than the determined relaxed number of bits.

[0035] The relaxed number of bits may be a number of bits relative to a portion of the number of allowed bits for encoding at least one direction value.

[0036] The apparatus may be caused to encode at least one directional value for at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, one of the entropy encodings of the at least one resolution, perform entropy encoding of a third resolution of the at least one value based on the at least one directional value, and determine a number of bits to be used for encoding the at least one value based on the entropy encoding of the third resolution, wherein a quantization resolution of the third resolution is lower than the entropy encoding of the first and second resolutions, and select the entropy encoding of the third resolution of the at least one value based on the at least one directional value if the number of bits used for encoding the at least one value based on the entropy encoding of the first or second resolution is greater than a portion of the number of bits allowed for encoding the at least one value.

[0037] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.

[0038] The apparatus may be further caused to encode at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal.

[0039] An apparatus caused to encode at least one energy ratio value for at least one subframe of each subband of a frame of an audio signal may be caused to generate a weighted average of the at least one energy ratio value and to encode the weighted average of the at least one energy ratio value.

[0040] The apparatus caused to encode the weighted average of the at least one energy ratio value may be further caused to perform a scalar non-uniform quantization of the at least one weighted average of the at least one energy ratio value.

[0041] At least one of the entropy encodings may be Golomb Rice encoding.

[0042] The device may further be caused to store and / or transmit the encoded at least one direction value.

[0043] According to a fourth aspect, there is provided an apparatus comprising: an acquisition circuit configured to acquire values ​​of parameters representative of an audio signal, the values ​​including at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; an acquisition circuit configured to acquire an allowable number of bits for encoding the at least one direction value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and an encoding circuit configured to encode the at least one direction value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, wherein one of the entropy encoding of at least one resolution comprises performing entropy encoding of a first resolution of the at least one direction value, determining a number of bits used for encoding the at least one direction value based on the first entropy encoding, and encoding the at least one direction value based on the at least one entropy encoding. performing entropy encoding of a first resolution of at least one reduced directional value and determining a number of bits used for encoding the at least one reduced directional value based on the first entropy encoding; selecting the entropy encoding of the first resolution of the at least one directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of bits allowed for encoding the at least one directional value; and selecting the entropy encoding of the first resolution of the at least one reduced directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of bits allowed for encoding the at least one directional value and the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the fraction of the number of bits allowed for encoding the at least one directional value.

[0044] According to a fifth aspect, there is provided a computer program (or a computer readable medium comprising program instructions) comprising instructions to cause an apparatus to at least: obtain values ​​of parameters representative of an audio signal, the values ​​comprising at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtain an allowable number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and encode the at least one directional value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, wherein one of the entropy encoding of at least one resolution further comprises performing an entropy encoding of a first resolution of the at least one directional value and encoding at least one directional value based on the first entropy encoding. determining a number of bits used to encode one directional value; performing entropy encoding of a first resolution of the at least one reduced directional value and determining a number of bits used to encode the at least one reduced directional value based on the first entropy encoding; selecting an entropy encoding of the first resolution of the at least one directional value if the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of bits allowed for encoding the at least one directional value; selecting an entropy encoding of the first resolution of the at least one directional value if the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of bits allowed for encoding the at least one directional value and if the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of bits allowed for encoding the at least one directional value;and selecting a first resolution entropy encoding of the at least one reduced directional value.

[0045] According to a sixth aspect, there is provided a non-transitory computer readable medium including program instructions to cause an apparatus to at least: obtain values ​​of parameters representative of an audio signal, the values ​​including at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtain an allowed number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and encode the at least one directional value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, one of the entropy encoding of at least one resolution further comprising: performing entropy encoding of a first resolution of the at least one directional value and determining a number of bits used for encoding the at least one directional value based on the first entropy encoding; performing entropy encoding of a first resolution of one reduced directional value and determining a number of bits used for encoding the at least one reduced directional value based on the first entropy encoding; selecting the entropy encoding of the first resolution of the at least one directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of bits allowed for encoding the at least one directional value; and selecting the entropy encoding of the first resolution of the at least one reduced directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of bits allowed for encoding the at least one directional value and the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the fraction of the number of bits allowed for encoding the at least one directional value.

[0046] According to a seventh aspect, there is provided an apparatus comprising: means for obtaining values ​​of parameters representative of an audio signal, the values ​​including at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; means for obtaining an allowable number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of a frame of the audio signal; and means for encoding the at least one directional value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, one of the entropy encoding of at least one resolution further comprising: performing entropy encoding of a first resolution of the at least one directional value, determining a number of bits used for encoding the at least one directional value based on the first entropy encoding; and encoding the at least one reduced directional value based on the first entropy encoding of the at least one reduced directional value. and determining a number of bits used for encoding the at least one reduced directional value based on the first entropy encoding; selecting the entropy encoding of the first resolution of the at least one directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of allowed bits for encoding the at least one directional value; and selecting the entropy encoding of the first resolution of the at least one reduced directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of allowed bits for encoding the at least one directional value and the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the fraction of the number of allowed bits for encoding the at least one directional value.

[0047] According to an eighth aspect, there is provided a computer-readable medium comprising program instructions to cause an apparatus to at least: obtain values ​​of parameters representative of an audio signal, the values ​​including at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtain an allowed number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and encode the at least one directional value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, one of the entropy encoding of at least one resolution further comprising: performing entropy encoding of a first resolution of the at least one directional value and determining a number of bits used for encoding the at least one directional value based on the first entropy encoding; performing entropy encoding of a first resolution of one reduced directional value and determining a number of bits used for encoding the at least one reduced directional value based on the first entropy encoding; selecting the entropy encoding of the first resolution of the at least one directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a fraction of the number of allowed bits for encoding the at least one directional value; and selecting the entropy encoding of the first resolution of the at least one reduced directional value if the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is greater than the fraction of the number of allowed bits for encoding the at least one directional value and the number of bits used for encoding the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the fraction of the number of allowed bits for encoding the at least one directional value.

[0048] The apparatus comprises means for carrying out the operations of the methods as described above.

[0049] The apparatus is configured to perform the method operations as described above.

[0050] The computer program comprises program instructions for causing a computer to carry out the method as described above.

[0051] A computer program product stored on the medium may cause an apparatus to perform the methods as described herein.

[0052] The electronic device may comprise an apparatus as described herein.

[0053] The chipset may comprise an apparatus as described herein.

[0054] SUMMARY OF THE PRESENT APPLICATION The embodiments of the present application aim to address problems associated with the state of the art.

[0055] For a better understanding of the present application, reference will now be made, by way of example, to the accompanying drawings in which: [Brief description of the drawings]

[0056] [Figure 1] FIG. 1 illustrates a schematic diagram of a system of apparatus suitable for implementing some embodiments. [Diagram 2] 2 illustrates a schematic diagram of a decoder shown in the system of the device shown in FIG. 1 according to some embodiments; [Diagram 3] FIG. 3 illustrates a flow diagram of the operation of the exemplary decoder shown in FIG. 2 according to some embodiments. [Figure 4] FIG. 3 is a schematic diagram of an exemplary synthesis processor shown in FIG. 2, according to some embodiments. [Diagram 5]FIG. 5 illustrates a flow diagram of the operation of the exemplary synthesis processor shown in FIG. 4 according to some embodiments. [Figure 6] FIG. 2 shows an exemplary device suitable for implementing the apparatus shown in the previous figure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0057] In the following, suitable apparatus and possible mechanisms for encoding a parametric spatial audio stream containing a transport audio signal and spatial metadata are described in further detail.

[0058] As mentioned above, Metadata Assisted Spatial Audio (MASA) is an example of a parametric spatial audio format and representation that is suitable as an input format for IVAS.

[0059] Metadata-assisted spatial audio can be seen as an audio representation composed of "N channels + spatial metadata". Metadata-assisted spatial audio is a scene-based audio format that is particularly suitable for spatial audio capture on practical devices such as smartphones. The idea is to describe a sound scene in terms of the direction and e.g. energy ratio of sound sources that vary with time and frequency. Sound energy that is not defined (described) by direction is described as diffuse (coming from all directions).

[0060] As mentioned above, the spatial metadata associated with the audio signal may include multiple parameters per time-frequency tile (e.g., direct-to-global ratio, spread coherence, distance, etc., for multiple directions and associated with each direction (or direction value)). The spatial metadata may also include or be associated with other parameters (e.g., surround coherence, ratio of diffuse energy to total energy, ratio of residual energy to total energy, etc.) that are considered non-directional but that may be used to define characteristics of the audio scene when combined with the directional parameters. For example, a reasonable design choice that can produce a good quality output is one in which it is determined that the spatial metadata includes one or more directions per time-frequency subframe (and the direct-to-global ratio, spread coherence, distance value, etc., associated with each direction).

[0061] As mentioned above, parametric spatial metadata representation can use multiple simultaneous spatial directions. In MASA, the proposed maximum number of simultaneous directions is two. For each simultaneous direction, there may be associated parameters such as direction index, direct-to-total ratio, spread coherence, and distance. In some embodiments, other parameters are defined such as diffuse-to-total energy ratio, surround coherence, and residual-to-total energy ratio.

[0062] At very low bit rates (e.g., around 13.2 to 16.4 kbps), the number of bits available for encoding metadata becomes very low: for example, in order to have a sufficient bit rate for the audio signal codec, only about 3 kbps may be used for encoding metadata.

[0063] In order to have sufficient frequency and time resolution (e.g., having five frequency bands and a time resolution of 20 ms), often only a few bits per value (e.g., directional parameters) can be used. In practice, this means that the quantization steps are relatively large. Thus, for example, for a given time-frequency tile, the quantization points are at 0 degrees, ±45 degrees, ±90 degrees, ±135 degrees, and 180 degrees of azimuth.

[0064] To try to improve the resulting encoder output, dynamic resolution (or dynamic quantization resolution) may be implemented. For example, an entropy coder is implemented where the angular resolution is given by the energy ratio of each subband as described in GB1811071.8. If the resulting number of bits is larger than the maximum allowed number of bits, the quantization resolution is reduced and an entropy coder as described in EP3861548 is used. However, the reduction in quantization resolution may be too large for some frames, since the directional resolution of human hearing is about 1-2 degrees in the azimuth direction, and any azimuth jump, for example, from 0 to 45 degrees, can be easily perceived, clearly degrading the audio quality and making the playback unnatural.

[0065] The concept as discussed in the embodiments herein attempts to counter the loss of angular resolution. In some embodiments, the limit on the maximum number of bits allowed is relaxed. Furthermore, in some embodiments, a check is made whether a slightly less accurate quantization of the angle can be achieved in the entropy coder by implementing a pseudo-embedded bitstream. In some embodiments, the quantization is further modified in situations where the input spatial metadata has only one subframe subband.

[0066] Embodiments are described with respect to an exemplary capture (or encoder / analyzer) and playback (or decoder / synthesizer) device or system 100 as shown in Figure 1. In the examples that follow, the audio signal input is from a microphone array, but it will be appreciated that the audio input can be in any suitable audio input format, and the following description provides details in situations where differences in processing occur when a different input format is employed.

[0067] The system 100 is shown with a capture section and a playback (decoder / synthesizer) section.

[0068] In some embodiments, the capture unit comprises a microphone array audio signal input 102. The input audio signal can be from any suitable source, for example, two or more microphones attached to a mobile phone, other microphone arrays (e.g., B-format microphones or Eigenmics). In some embodiments, as previously described, the input can be any suitable audio signal input, such as an Ambisonic signal, for example, first-order Ambisonics (FOA), higher-order Ambisonics (HOA), or loudspeaker surround mix and / or objects.

[0069] A microphone array audio signal input 102 may be provided to a microphone array front end 103. The microphone array front end is in some embodiments configured to perform the functions of an analysis processor configured to generate or determine appropriate (spatial) metadata associated with the audio signal, and to perform the functions of a suitable transport signal generator to generate a transport audio signal.

[0070] The functionality of the analysis processor is therefore configured to perform spatial analysis on the input audio signal and generate appropriate spatial metadata 106 in the frequency band. For all the aforementioned input types, there are known methods for generating appropriate spatial metadata, e.g., directional and direct to total energy ratios in the frequency band (or similar parameters, such as diffuse or ambient to total ratios). These methods are not detailed herein, but some examples may include performing an appropriate time-frequency transformation on the input signal and then estimating, in the frequency band, a delay value between pairs of microphones that maximizes the correlation between the microphones when the input is a microphone array on a mobile phone, formulating a directional value corresponding to that delay (as described in GB Patent Application No. 1619573.7 and PCT Patent Application No. PCT / FI2017 / 050778) and formulating a ratio parameter based on the correlation value. The ratio parameter of direct energy to total energy of a microphone array signal captured by multi-channels can be estimated based on a normalized cross-correlation parameter cor'(k,n) between a pair of microphones in band k, where the value of the cross-correlation parameter is between -1 and 1. The ratio parameter r(k,n) of direct energy to total energy can be expressed as

[0071]

number

[0072] The metadata can be of various forms and in some embodiments can include spatial and other metadata. A typical parameterization of the spatial metadata is the azimuth angle values ​​φ(k,n) and elevation angle values ​​θ(k,n) and a unidirectional parameter in each frequency band characterized as the ratio of the associated direct energy to the total energy in each frequency band r(k,n), where k is the frequency band index and n is the time frame index.

[0073] In some embodiments, the generated parameters may be different for each frequency band. Thus, for example, in band X, all of the parameters are generated and transmitted, while in band Y, only one of the parameters is generated and transmitted, and in band Z, no parameters are generated or transmitted. A practical example of this may be that for some frequency bands, such as the highest band, some of the parameters are not needed for perceptual reasons.

[0074] In some embodiments, if the audio input is a FOA signal or a B-format microphone, the functionality of the analysis processor may be configured to perform the following: determining parameters such as intensity vectors, based on which directional parameters are obtained, and comparing the length of the intensity vector with an overall sound field energy estimate to determine a ratio parameter, a method known in the literature as Directional Audio Coding (DirAC).

[0075] In some embodiments, if the input is an HOA signal, the analysis processor function may receive a FOA subset of the signal and use the above method, or split the HOA signal into multiple sectors, each of which utilizes the above method. Sector-based methods are known in the literature as higher order DirAC (HO-DirAC), where there are multiple simultaneous directional parameters, two or more per frequency band.

[0076] In some embodiments, if the input format is a loudspeaker surround mix and / or objects, the analysis processor functionality may be configured to convert the signal into a FOA signal (by use of spherical harmonic encoding gains) and analyze the directional and proportional parameters as described above.

[0077] Thus, the output of the analysis processor function is (spatial) metadata 106 determined in frequency bands. The (spatial) metadata 106 may include orientation and energy ratios in frequency bands, but may also include any of the metadata types listed previously. The (spatial) metadata 106 can vary over time and over frequency.

[0078] In some embodiments, the analysis function is performed outside of the system 100. For example, in some embodiments, spatial metadata associated with the input audio signal may be provided to the encoder 107 as a separate bitstream. In some embodiments, the spatial metadata may be provided as a set of spatial (directional) index values.

[0079] The microphone array front end 103 is further configured to perform a transport signal generator function, as described above, to generate a suitable transport audio signal 104. The transport signal generator function 113 is configured to receive an input audio signal, which may be, for example, the microphone array audio signal 103, and to generate a transport audio signal 104. The transport audio signal may be a multi-channel, stereo, binaural, or mono audio signal. The generation of the transport audio signal 104 may be performed using any suitable method as summarized below.

[0080] If the input is a microphone array audio signal, the function of the transport signal generator may be to select a left and right microphone pair and apply appropriate processing to the signal pair, such as automatic gain control, microphone noise cancellation, wind noise cancellation, and equalization.

[0081] If the input is a FOA / HOA signal or a B-format microphone, the transport signal 104 may be a directional beam signal oriented in left and right directions, such as two opposing cardioid signals.

[0082] If the input is a loudspeaker surround mix and / or objects, the transport signal 104 may be a downmix signal that combines the left channels into a left downmix channel and the same for the right side, adding a center channel to both transport channels with an appropriate gain.

[0083] In some embodiments, the transport signal 104 is an input audio signal, e.g., a microphone array audio signal. For example, in some cases, analysis and synthesis may occur in a single processing step in the same device, without intermediate encoding. The number of transport channels may also be any suitable number (rather than one or two channels as described in the examples).

[0084] In some embodiments, the capture unit may comprise an encoder 107. The encoder 107 may be configured to receive the transport audio signal 104 and the spatial metadata 106. The encoder 107 may be further configured to generate a bitstream 108 that includes the metadata information and an encoded or compressed form of the transport audio signal.

[0085] The encoder 107 may be implemented, for example, as an IVAS encoder or any other suitable encoder, and in such an embodiment, is configured to encode the audio signal and the metadata to form an IVAS bitstream.

[0086] This bitstream 108 may then be transmitted / stored as indicated by the dashed line.

[0087] The system 100 may further comprise a decoder 109 portion configured to receive, extract or otherwise obtain the bitstream 108 and generate from this bitstream a suitable spatial audio signal 110 for presentation to a listener / listener's playback device.

[0088] Thus, the decoder 109 is configured to receive the bitstream 108, demultiplex the encoded stream and then decode the audio signal to obtain the transport signal and the metadata.

[0089] The decoder 109 may further be configured to generate, from the transport audio signal and the spatial metadata, a spatial audio signal output 110, for example a binaural audio signal that can be played via headphones.

[0090] With reference to FIG. 2, a schematic example of the encoder 107 is shown in further detail.

[0091] The encoder 107 is shown in Fig. 2 with the transport signal 104 being input to a transport signal encoder 201. The transport signal encoder 201 can be any suitable audio signal encoder. For example, an implementation of an Enhanced Voice Services (EVS) or Immersive Voice and Audio Services (IVAS) stereo core encoder can be applied to the transport (audio) signal to generate a suitable encoded transport audio signal 204 that can be passed to a bitstream generator 207 or output as a separate bitstream to spatial metadata parameters.

[0092] The encoder 107, in some embodiments, is configured to receive the spatial metadata 106 or spatial parameters and pass them to a parameter quantizer 203. For example, the determined orientation parameters (azimuth and elevation or other coordinate system) may be quantized by the parameter quantizer 203 and an index that identifies the quantized value that is passed to a quantized parameter entropy encoder 205.

[0093] The encoder 107, in some embodiments, further comprises a quantization parameter entropy encoder 205 configured to obtain or receive quantized parameters and encode them to generate encoded spatial metadata 202, which may be passed to a bitstream generator 207.

[0094] The encoder 107 further comprises, in some embodiments, a bitstream generator 207 configured to obtain or receive the encoded transport audio signal 204 and the encoded spatial metadata 202 including the spread and surround coherence parameters, and to generate the bitstream 108 or a separate bitstream.

[0095] In the following example, the encoder 107 is configured to encode spatial audio parameters (MASA), in other words spatial metadata 106. For example, the direction values ​​(azimuth and elevation values ​​φ(k,n) and θ(k,n)) may be first quantized according to a spherical quantization scheme. Such a scheme is described in patent publication EP3707706. As mentioned above, each type of spatial audio parameter is first quantized to obtain a quantization index.

[0096] The resulting quantization indexes of the spatial audio parameters (e.g., MASA parameters) may then be entropy encoded with different coding rates depending on a factor that specifies the number of bits allocated to the task. Furthermore, the codec may use multiple different coding rates to apply to the encoding of the indices of the spatial audio parameters.

[0097] So in the example that may be returned below, the audio metadata includes azimuth, elevation, and energy ratio data for each subband. The audio metadata may also include spread and surround coherence, but these are encoded first and the remaining available bits are calculated by subtracting the coherence bits from the total bits.

[0098] In the MASA format, the orientation data is represented by 16 bits, such that the azimuth angle is approximately represented by 9 bits and the elevation angle by 7 bits. The energy ratio is represented by 8 bits. There are between N=5 and M=4 time blocks per frame, so (16+8)×M×N bits are needed to store the uncompressed metadata per frame. At higher frequency resolutions, there may be 20 or 24 frequency subbands.

[0099] In the following example, the encoder and decoder operate on N=5 time blocks, but in other examples the number of time blocks, bits and subbands used, and parameters may differ.

[0100] With reference to FIG. 3, an exemplary quantization parameter entropy encoder 205 is shown. The quantization parameter entropy encoder 205 comprises, in some embodiments, an energy ratio value encoder 301. The energy ratio value encoder 301 is configured to receive the quantized energy ratio values ​​300 and generate encoded energy ratio values ​​302. In some examples, 3 bits are used to encode each energy ratio value. In addition, instead of transmitting all energy ratio values ​​of all TF blocks, only one weighted average value is transmitted per subband. The average is calculated by considering the total energy of each time block, thus favoring the values ​​of the subbands with more energy.

[0101] In some embodiments, an exemplary quantized entropy encoder comprises two or more entropy encoders configured to receive the quantized directional values ​​302. In this example, a first directional (average / differential) entropy encoder 303, a second lower resolution entropy encoder 305, and a third lowest resolution average / differential entropy encoder 307 are shown. The first directional (average / differential) entropy encoder 303 (EC1), the second lower resolution entropy encoder 305 (EC2), and the third lowest resolution entropy encoder 307 (EC3) are configured to receive the quantized directional values ​​and generate encoded values ​​that are passed to an encoding selector 309.

[0102] The exemplary quantized entropy encoder, in some embodiments, comprises an encoding selector 309 that receives the outputs of the first, second, and third entropy encoders and selects one of these outputs to output as the encoded directional value 310. The selection, in some embodiments, may be based on the number of bits generated by each of the encoders and the number of allowed bits as described in the following encoding of the index values ​​of the directional parameters of all TF tiles in the frame.

[0103] 3, the first, second and third entropy encoders are concatenated or connected in series such that the first entropy encoder 303 is operated first and if the first entropy encoder 303 cannot acceptably encode the parameters, then the second entropy encoder 305 is operated. Similarly, if the second entropy encoder 305 cannot acceptably encode the parameters, then the third entropy encoder 307 is operated. The encoding selector is then configured to select the output of the encoder that was received last or selected based on an ordering of the third encoder / second encoder / first encoder etc.

[0104] However, in some embodiments, all three encoders are operated in parallel or substantially in parallel, and then one of the three encoder outputs is selected by the encoding selector based on which output is acceptable. In such embodiments, the first encoder output is checked and if acceptable it is output, otherwise the second encoder output is checked and if acceptable it is output, otherwise the third encoder output is output.

[0105] The operation of the encoder selector or encoder may be implemented or perform the encoding selection, for example, based on the following pseudocode: Input: The index of the quantized direction parameters (azimuth and elevation) and the number of bits allowed B allowed 1. Use EC1 to encode parameters 2. If bits_EC1 < B_allowed a. Encode using EC1 3.Else a. Use EC2 bandwidth encoding (which may reduce quantization resolution) b. If bits_EC2 < B_allowed i.Encode using EC2 c.Else i. Reduce the quantization resolution ii. Use EC3 d. End if 4.End if

[0106] In the above, the first directional (average / differential) entropy encoder 303 (EC1) corresponds to a first entropy encoding scheme in which the azimuth and elevation indices may be encoded separately. In some embodiments, this scheme uses an optimized fixed-value average index that is subtracted from each index, resulting in a differential index for each directional index. Each resulting differential index may then be converted to a positive value and then entropy encoded using the Golomb Rice scheme. The optimized average index may be entropy encoded for transmission to the decoder. Furthermore, in some embodiments, the directional entropy encoder is based on a time-averaged directional value and a difference with a time-averaged directional value. For example, the differential index value is the difference between the azimuth or elevation value of the current direction and the average or azimuth or elevation value of the previous frame or subframe, or in some embodiments, a difference based on a reference azimuth or elevation value. In some embodiments, the value of the average (the number to be subtracted) is selected such that the resulting number of bits for encoding is minimized. Thus, in some embodiments, the device tests the values ​​given by the average, and also tests variations around the average (e.g., values ​​of the average ±1 or even more), and selects as the "average" the value that produces the smallest number of encoded bits that are transmitted to the decoder.

[0107] The second entropy encoder 305 (EC2) corresponds to a second entropy encoding scheme that encodes the difference index with less or lower resolution than EC1. Details of a suitable second entropy encoding scheme are described in patent publication WO2021 / 048468.

[0108] The third entropy encoder 307 (EC3) corresponds to a third entropy encoding scheme that encodes the difference index with less resolution than EC2. In this respect, EC3 may constitute the lowest resolution quantization scheme in the above general framework. Details of a suitable scheme for use are given in patent publication EP3861548.

[0109] From the general framework above, the choice of encoding rate (and therefore encoding scheme) depends on the parameter B, which indicates the number of bits allowed for encoding the frame orientation index. allowed It can be understood that the time may be determined in part by allowed may be a parameter determined by the encoding system according to the overall operating point / bitrate of the encoder for a particular time frame.

[0110] As shown above, parameter B allowed Basically, the bits required for the entropy encoding scheme are determined by the parameter B allowed This checking process is performed in descending order of the number of bits required for the entropy encoding schemes. allowed The highest entropy encoding scheme (in terms of number of encoding bits) that satisfies the constraints is selected.

[0111] For example, the number of bits required for the first entropy encoding scheme EC1 (bits_EC1) is B allowed If the number of bits required for EC1 is less than the constraint B, then the first entropy encoding scheme is used. allowed If it is determined that the number of bits required for the second entropy encoding scheme EC2 (bits_EC2) is greater than B allowedFurthermore, in some embodiments, the second entropy encoding scheme EC2 is only tested for the non-2D case (i.e., when the elevation angle is non-zero on all tiles in the frame). This second check ensures that the number of bits required for EC2 is B allowed If it indicates that the directional index of the frame is smaller than B, then a second entropy encoding scheme EC2 is used to entropy encode the directional index of the frame. However, this second check indicates that the number of bits required for EC2 is smaller than B. allowed If it indicates that θ is greater than (or equal to) θ, then the third entropy encoding scheme EC3 is selected to encode the directional index.

[0112] The general framework above can be extended to any number of encoding rates, with each entropy encoding scheme having a number of bits required (bits_ECn) and a number of bits allowed B allowed is selected according to.

[0113] With respect to Figure 4, a method or operation performed by a first entropy encoder (EC1) according to some embodiments is illustrated. In this example, the first entropy encoder is configured to perform entropy encoding (EC1) of the direction encoded in a pseudo-embedded manner.

[0114] Thus, for example, as indicated by step 401 in FIG. 4, the average direction over all time-frequency tiles for which the energy ratio is higher than a threshold is calculated.

[0115] Next, as indicated by step 403 in FIG. 4, the elevation and azimuth angles of the remaining TF tiles are jointly encoded using one spherical index per tile.

[0116] The mean direction is then encoded by transmitting the elevation and azimuth angles separately. As shown by step 405 in Figure 4, this encoding uses the number of bits given by the maximum alphabet of the elevation and azimuth angles, respectively, from the TF tile considered.

[0117] The elevation and azimuth differences from the average are encoded separately: in these embodiments, one stream for the azimuth difference values, and one stream for the elevation difference values, as shown by step 407 in FIG.

[0118] Next, as indicated by step 409 in FIG. 4, for each angle value, the difference to the average is calculated with respect to the average projected at the resolution of the corresponding tile.

[0119] Then, within the index domain, the difference to the mean is transformed into the positive domain using the following function, as indicated by step 411 in FIG. for (i = 0; i < len; i++) { if ( dif_idx[i] < 0 ) { dif_idx[i] = -2 * dif_idx[i]; } else if (dif_idx[i] > 0) { dif_idx[i] = dif_idx[i] * 2 - 1; } else { dif_idx[i] = 0; } }

[0120] Then, the positive difference indexes are encoded using the (optimal) determined order GR codes. The determined (optimal) GR order is such that for each data set, one GR order value for the azimuth difference index and one GR order value for the elevation difference index are calculated.

[0121] Larger values ​​result in longer GR codes and smaller values ​​result in shorter GR codes, so for a differential index, if the encoder uses the smaller value of the two units, the difference to the average will be of the same sign but smaller, and the number of bits required for encoding will be smaller.

[0122] Thus, in some embodiments, a reduced difference index encoding is generated and then a number of bits that may be gained by reducing a portion of the difference index is determined, as illustrated by step 415 in FIG.

[0123] Then, as shown by step 417 in Fig. 4, an encoded index or a reduced encoded differential index is selected based on the number of bits that have been obtained. For example, if the first entropy encoder (EC1) causes an encoding that results in a number of bits that is greater than the number of allowed bits by a value NB, and the maximum number of bits that can be obtained is greater than NB, then the reduced differential index value is selected. If neither the encoding by the first entropy encoder nor the reduced encoding can obtain the required number of bits, then the method of the second entropy encoder EC2 or the third entropy encoder EC3 is used.

[0124] The conditions for checking if the difference index can be decreased are that the difference must be greater than 0 and the angular resolution must be greater than a certain threshold. The angular resolution may be given by the length of the alphabet of the angle values ​​in some embodiments. In one example, 20 degrees may be used as the minimum threshold for the elevation alphabet and 40 degrees may be used as the minimum threshold for the azimuth alphabet. In some embodiments, the portion corresponding to the elevation angle may only be applied if the azimuth alphabet is adjusted based on the changed elevation angle value. In some implementations, the elevation angle change is not used. However, if these values ​​were used, when checking the azimuth angle, the azimuth alphabet should be updated and the original value should be requantized.

[0125] An exemplary implementation in C for determining the number of bits that can be gained may be as follows: if ( nbands > 1 && direction_bits_ec - max_bits > 0 && direction_bits_ec - max_bits < nblocks * nbands ) { / * Check how many bits can be obtained * / for ( idx = 0; idx < dist_count; idx++ ) { if ( q_direction->not_in_2D > 0 ) { if ( dist_elevation_alphabets[idx] > 20 && dist_elevation_indexes_best[idx] > 1 ) { bits_gained += ivas_qmetadata_encode_extended_gr_length( dist_elevation_indexes_best[idx], dist_elevation_alphabets[idx], gr_param_elevation_best ) - ivas_qmetadata_encode_extended_gr_length( dist_elevation_indexes_best[idx] - 2, dist_elevation_alphabets[idx], gr_param_elevation_best ); } else if ( dist_elevation_alphabets[idx] > 20 && dist_elevation_indexes_best[idx] == 1 ) { bits_gained += ivas_qmetadata_encode_extended_gr_length( dist_elevation_indexes_best[idx], dist_elevation_alphabets[idx], gr_param_elevation_best ) - ivas_qmetadata_encode_extended_gr_length( dist_elevation_indexes_best[idx] - 1, dist_elevation_alphabets[idx], gr_param_elevation_best ); } } } for ( idx = 0; idx < dist_count; idx++ ) { if ( dist_azimuth_alphabets[idx] > 40 && dist_azimuth_indexes_best[idx] > 1 ) { bits_gained += ivas_qmetadata_encode_extended_gr_length( dist_azimuth_indexes_best[idx], dist_azimuth_alphabets[idx], gr_param_azimuth_best ) - ivas_qmetadata_encode_extended_gr_length( dist_azimuth_indexes_best[idx] - 2, dist_azimuth_alphabets[idx], gr_param_azimuth_best ); } else if ( dist_azimuth_alphabets[idx] > 40 && dist_azimuth_indexes_best[idx] == 1 ) { bits_gained += ivas_qmetadata_encode_extended_gr_length( dist_azimuth_indexes_best[idx], dist_azimuth_alphabets[idx], gr_param_azimuth_best ) - ivas_qmetadata_encode_extended_gr_length( dist_azimuth_indexes_best[idx] - 1, dist_azimuth_alphabets[idx], gr_param_azimuth_best ); } } printf( “frame = %ld bits_gained = %d %d \n”, frame, bits_gained, direction_bits_ec - max_bits ); if ( bits_gained >= direction_bits_ec - max_bits ) { make_gain = 1; }

[0126] In some embodiments, the maximum bit limit when reducing the quantization resolution may be relaxed. For example, the number of bits that need to be reduced before the third entropy encoder (EC3) method is implemented is limited to a maximum of the number of TF tiles for which encoding is performed. As a result of implementing such a relaxation of the bit limit, the bit consumption may exceed the maximum number of bits allowed for metadata for some frames. However, the encoding is configured to handle situations where the encoder normally operates below the required bit limit, and therefore the number of bits used on average can be relaxed without exceeding the total number of bits over a reasonable period of time.

[0127] In some embodiments, if the encoder determines that there is only one sub-frame per sub-band in the input spatial audio data, the second entropy encoder is disabled or stopped and the method of the second entropy encoder (EC2) is not considered or signaled. The disabling / stopping of the second entropy encoder occurs because the method used in the second entropy encoder is one in which similarities between TF tiles within a sub-band are examined and exploited, but in this case there is only one tile per sub-band and therefore no similarities exist.

[0128] These embodiments may be implemented for lower bit rates.

[0129] In some embodiments, the decoder 109 comprises a demultiplexer (not shown) configured to accept and demultiplex the bitstream to obtain the encoded transport audio signal and encoded spatial audio parameter metadata (MASA metadata) including the encoded energy ratio values ​​302 and the encoded directional values ​​310.

[0130] In some embodiments, the decoder 109 further comprises a transport audio signal decoder (not shown) configured to decode the encoded transport audio signal, thereby generating a transport audio signal stream that is passed to the spatial synthesizer. The decoding process performed by the transport audio signal decoder may be an audio signal decoding scheme suitable for the encoded transport audio signal, such as an EVS decoder if EVS encoding is used.

[0131] FIG. 5 shows in further detail a metadata decoder 509 configured to accept encoded spatial metadata (encoded energy ratio values ​​302 and encoded directional values ​​310) and decode this metadata to generate decoded spatial metadata (energy ratio values ​​502 and directional values ​​or indices 504).

[0132] In some embodiments, the metadata decoder 509 comprises an energy ratio value decoder 501 configured to receive the encoded energy ratio value 302, based on which the energy ratio value is determined.

[0133] The metadata decoder 509 further comprises an entropy decoder 503 configured to obtain the encoded direction values ​​310 and to output the direction values ​​504 .

[0134] Above, the difference is determined with respect to other directional values ​​within the subframe or frame. However, in some embodiments, the difference may be determined with respect to a past subframe. In other words, the average may be determined within the current subframe, within the current frame, or over multiple time frames.

[0135] An exemplary electronic device that may be used as any of the apparatus portions of the system as described above with respect to FIG. 6. The device may be any suitable electronic device or apparatus. For example, in some embodiments, device 1400 is a mobile device, user equipment, tablet computer, computer, audio playback device, etc. The device may be configured to implement, for example, an encoder / analyzer portion and / or a decoder portion as shown in FIG. 1, or any of the functional blocks as described above.

[0136] In some embodiments, device 1400 includes at least one processor or central processing unit 1407. Processor 1407 may be configured to execute various program code, such as the methods described herein.

[0137] In some embodiments, the device 1400 comprises at least one memory 1411. In some embodiments, the at least one processor 1407 is coupled to the memory 1411. The memory 1411 can be any suitable storage means. In some embodiments, the memory 1411 includes a program code section for storing program code executable on the processor 1407. Furthermore, in some embodiments, the memory 1411 can further include a storage data section for storing data (e.g., data that has been processed or will be processed according to embodiments as described herein). The executed program code stored in the program code section and the data stored in the storage data section can be retrieved by the processor 1407 via the memory-processor coupling whenever needed.

[0138] In some embodiments, the device 1400 comprises a user interface 1405. The user interface 1405 may be coupled to a processor 1407 in some embodiments. In some embodiments, the processor 1407 may control the operation of the user interface 1405 and receive input from the user interface 1405. In some embodiments, the user interface 1405 may allow a user to input commands into the device 1400, for example, via a keypad. In some embodiments, the user interface 1405 may allow a user to obtain information from the device 1400. For example, the user interface 1405 may comprise a display configured to display information from the device 1400 to a user. The user interface 1405 may comprise a touch screen or touch interface in some embodiments capable of both allowing information to be input into the device 1400 and further displaying information to a user of the device 1400. In some embodiments, the user interface 1405 may be a user interface for communication.

[0139] In some embodiments, device 1400 comprises an input / output port 1409, which in some embodiments comprises a transceiver. The transceiver, in such embodiments, may be coupled to processor 1407 and configured to enable communication with other apparatus or electronic devices, for example via a wireless communication network. The transceiver or any suitable transceiver or transmitter and / or receiver means, in some embodiments, may be configured to communicate with other electronic devices or apparatuses via a wire or wired coupling.

[0140] The transceiver may communicate with the further device by any suitable known communication protocol. For example, in some embodiments, the transceiver may be connected to a wireless network using a long term evolution advanced (LTE-A) or new radio (NR) (which may also be referred to as 5G), universal mobile telecommunications system (UMTS) radio access network (UTRAN or E-UTRAN), long term evolution (LTE, also known as E-UTRA), 2G network (a traditional network technology), wireless local area network (WLAN or Wi-Fi), worldwide interoperability for microwave access (WiMAX), Bluetooth, personal communications services (PCS), ZigBee, wideband code division multiple access (WCDMA), systems using ultra-wideband (UWB) technology, sensor networks, mobile ad-hoc networks (MANET), or other wireless technologies. Any suitable radio access architecture may be used, such as based on cellular Internet of Things (IoT) RAN and Internet Protocol multimedia subsystems (IMS), any other suitable alternative, and / or any combination thereof.

[0141] The transceiver input / output port 1409 may be configured to receive a signal.

[0142] In some embodiments, device 1400 may be employed as at least a portion of a synthesis device. Input / output port 1409 may be coupled to headphones (which may be head-tracked or non-tracked headphones) or the like and loudspeakers.

[0143] In general, various embodiments of the present invention may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described as block diagrams, flow charts, or using any other graphical representation, it is appreciated that these blocks, apparatus, systems, techniques, or methods described herein may be implemented in, by way of non-limiting examples, hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or any combination thereof.

[0144] The embodiments of the present invention may be implemented by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Moreover, in this regard, it should be noted that any block of logic flow as in the figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on a physical medium such as a memory chip, or a memory block implemented within a processor, a magnetic medium such as a hard disk or floppy disk, and an optical medium, such as, for example, a DVD and its data variations, a CD, etc.

[0145] The memory may be of any type suitable for the local technology environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memories, and removable memories. The data processor may be of any type suitable for the local technology environment and may include, by way of non-limiting examples, one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), gate level circuits, and processors based on multi-core processor architectures.

[0146] Embodiments of the present invention may be practiced in a variety of components, such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available to convert logic level designs into semiconductor circuit designs ready to be etched and formed into semiconductor substrates.

[0147] Programs such as those offered by Synopsys, Inc. of Mountain View, Calif., and Cadence Design Systems, Inc. of San Jose, Calif., use established rules of design and libraries of prestored design modules to automatically route conductors and place components on a semiconductor chip. After the design of a semiconductor circuit is complete, the resulting design in a standardized electronic format (e.g., Opus, GDSII, etc.) may be sent to a semiconductor manufacturing facility or "fab" for fabrication.

[0148] The foregoing description has provided a complete and informative description of example embodiments of the present invention as illustrative and non-limiting examples. However, various modifications and adaptations may become apparent to those skilled in the art in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of this invention will still fall within the scope of the present invention as defined in the appended claims.

Claims

1. obtaining values ​​of parameters representative of an audio signal, said values ​​including at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowable number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; encoding the at least one orientation value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution; wherein one of the entropy encoding of the at least one resolution further comprises: performing an entropy encoding of a first resolution of the at least one directional value and determining the number of bits used to encode the at least one directional value based on the first entropy encoding; performing an entropy encoding of a first resolution of at least one reduced directional value and determining the number of bits used to encode the at least one reduced directional value based on the first entropy encoding; selecting an entropy encoding of the first resolution of the at least one directional value if the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a portion of the number of bits allowed for encoding the at least one directional value; selecting an entropy encoding of the first resolution of the at least one reduced directional value if the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is greater than the portion of the number of allowed bits for encoding the at least one directional value and the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the portion of the number of allowed bits for encoding the at least one directional value; The device is for

2. the means for performing entropy encoding of a first resolution of the at least one directional value and determining the number of bits used to encode the at least one directional value based on the first entropy encoding is for determining entropy encoding of at least one resolution of at least one value determined from a difference of the at least one directional value compared with an average directional value of the frame and the number of bits used to encode the at least one value; 2. The apparatus of claim 1, wherein the means for performing entropy encoding of a first resolution of at least one reduced directional value and determining the number of bits used to encode the at least one reduced directional value based on the first entropy encoding is for entropy encoding of at least one resolution of at least one reduced value from a reduced difference based on the difference of the at least one directional value compared between the average directional value of the frame and determining the number of bits used to encode the at least one reduced value.

3. The means for encoding the at least one orientation value for at least one subframe of each subband of a frame based on entropy encoding of at least one resolution further comprises: performing entropy encoding of a second resolution of at least one value based on the at least one directional value and determining the number of bits used to encode the at least one value based on the entropy encoding of the second resolution, the entropy encoding of the second resolution being a lower resolution encoding than the entropy encoding of the first resolution, and exploiting similarities between time-frequency tiles within subbands in the frame when the frame includes more than one time-frequency tile within a subband; selecting an entropy encoding of the second resolution of the at least one value if the number of bits used to encode the at least one value based on the entropy encoding of the second resolution is less than or equal to the portion of the number of allowed bits for encoding the at least one directional value; 3. The device according to claim 1 or 2, for

4. The apparatus of claim 3 , wherein the at least one value based on the at least one direction value is at least one difference value from the at least one direction value compared to an average direction value for the frame.

5. 5. The apparatus of claim 3, wherein the means is further for selecting the entropy encoding of the second resolution of the at least one value based on the at least one directional value when the number of bits used to encode the at least one value based on the entropy encoding of the second resolution is greater than the portion of the allowed number of bits for encoding the at least one directional value but less than a determined relaxed number of bits.

6. The apparatus of claim 5 , wherein the relaxed number of bits is a number of bits relative to the portion of the allowed number of bits for encoding the at least one directional value.

7. The means for encoding the at least one orientation value for the at least one subframe of each subband of the frame based on an entropy encoding of the at least one resolution, one of the entropy encodings of the at least one resolution further comprising: performing entropy encoding of a third resolution of at least one value based on the at least one directional value and determining the number of bits used to encode the at least one value based on the entropy encoding of the third resolution, wherein a quantization resolution of the third resolution is lower than the entropy encoding of the first and second resolutions; selecting an entropy encoding of the third resolution of the at least one value based on the at least one directional value if the number of bits used to encode the at least one value based on the entropy encoding of the first or second resolution is greater than the portion of the number of allowed bits for encoding the at least one value; 7. The device according to claim 5 or 6, for

8. The apparatus of claim 7 , wherein the at least one value based on the at least one direction value is at least one difference value from the at least one direction value compared to an average direction value for the frame.

9. The apparatus of any one of claims 1 to 8, wherein said means is further for encoding said at least one energy ratio value for said at least one subframe of each subband of said frame of said audio signal.

10. said means for encoding said at least one energy ratio value for said at least one subframe of each subband of a frame of said audio signal further comprising: generating a weighted average of the at least one energy ratio value; encoding the weighted average of the at least one energy ratio value; The device according to claim 6 for:

11. 11. The apparatus of claim 10, wherein the means for encoding the weighted average of the at least one energy ratio value is further for performing a scalar non-uniform quantization of the at least one weighted average of the at least one energy ratio value.

12. The apparatus according to any one of claims 1 to 11, wherein said at least one entropy encoding is Golomb Rice encoding.

13. Apparatus according to any of the preceding claims, wherein said means are further for storing and / or transmitting said encoded at least one directional value.

14. obtaining values ​​of parameters representative of an audio signal, said values ​​including at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowable number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; and encoding the at least one orientation value for the at least one subframe of each subband of the frame based on entropy encoding of at least one resolution, wherein one of the entropy encoding of the at least one resolution further comprises: performing an entropy encoding of a first resolution of the at least one directional value and determining the number of bits used to encode the at least one directional value based on the first entropy encoding; performing an entropy encoding of a first resolution of at least one reduced directional value and determining the number of bits used to encode the at least one reduced directional value based on the first entropy encoding; selecting an entropy encoding of the first resolution of the at least one directional value if the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is less than or equal to a portion of the number of bits allowed for encoding the at least one directional value; selecting an entropy encoding of the first resolution of the at least one reduced directional value if the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is greater than the portion of the number of allowed bits for encoding the at least one directional value and the number of bits used to encode the at least one directional value based on the entropy encoding of the first resolution is less than or equal to the portion of the number of allowed bits for encoding the at least one directional value; A method for

15. 15. The method of claim 14, wherein performing entropy encoding of a first resolution of the at least one directional value and determining the number of bits used to encode the at least one directional value based on the first entropy encoding comprises determining entropy encoding of at least one resolution of at least one value determined from a difference of the at least one directional value compared to an average directional value of the frame and the number of bits used to encode the at least one value, and performing entropy encoding of a first resolution of at least one reduced directional value and determining the number of bits used to encode the at least one reduced value based on the difference of the at least one directional value compared to the average directional value of the frame comprises determining entropy encoding of at least one resolution of at least one reduced value from a reduced difference based on the difference of the at least one directional value compared to the average directional value of the frame and the number of bits used to encode the at least one reduced value.

16. encoding the at least one orientation value for at least one subframe of each subband of a frame based on entropy encoding of at least one resolution; performing entropy encoding of a second resolution of at least one value based on the at least one directional value and determining the number of bits used to encode the at least one value based on the entropy encoding of the second resolution, the entropy encoding of the second resolution being a lower resolution encoding than the entropy encoding of the first resolution, and exploiting similarities between time-frequency tiles within subbands in the frame when the frame includes more than one time-frequency tile within a subband; selecting an entropy encoding of the second resolution of the at least one value if the number of bits used to encode the at least one value based on the entropy encoding of the second resolution is less than or equal to the portion of the number of allowed bits for encoding the at least one directional value; 16. The method of claim 14 or 15, comprising:

17. The method of claim 16 , wherein the at least one value based on the at least one directional value is at least one difference value from the at least one directional value compared to an average directional value for the frame.

18. 18. The method of claim 16 or 17, further comprising selecting the entropy encoding of the second resolution of the at least one value based on the at least one directional value when the number of bits used to encode the at least one value based on the entropy encoding of the second resolution is greater than the portion of the allowed number of bits for encoding the at least one directional value but less than a determined relaxed number of bits.

19. 20. The method of claim 18, wherein the relaxed number of bits is a number of bits relative to the portion of the allowed number of bits for encoding the at least one directional value.

20. encoding the at least one orientation value for the at least one subframe of each subband of the frame based on an entropy encoding of the at least one resolution, one of the entropy encodings of the at least one resolution further comprising: performing entropy encoding of a third resolution of at least one value based on the at least one directional value and determining the number of bits used to encode the at least one value based on the entropy encoding of the third resolution, wherein the quantization resolution of the third resolution is lower than the entropy encoding of the first and second resolutions; selecting an entropy encoding of the third resolution of the at least one value based on the at least one directional value if the number of bits used to encode the at least one value based on the entropy encoding of the first or second resolution is greater than the portion of the number of allowed bits for encoding the at least one value; 20. The method of claim 18 or 19, comprising:

Images