Parameterized spatial audio coding
By using dynamic resolution entropy coding and Golomb Rice coding techniques, the problem of angular resolution loss in spatial audio signal coding at low bit rates is solved, achieving efficient coding and high-quality audio reproduction.
Patent Information
- Application Number
- CN202280093921.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-22
- Publication Date
- 2024-11-12
AI Technical Summary
Existing technologies struggle to maintain audio quality effectively when encoding spatial audio signals at low bit rates, especially due to losses in angular resolution that result in unnatural audio reproduction.
Dynamic resolution entropy coding technology is adopted. Through multi-layer entropy coding scheme (first, second, and third resolution entropy coding) and Golomb Rice coding, combined with pseudo-embedded bit stream, the coding rate is dynamically adjusted to optimize bit usage and reduce angular resolution loss.
It improves the coding efficiency and audio quality of spatial audio signals at low bit rates, maintains the natural reproduction of audio signals, and is suitable for various transmission conditions and error robustness.
Smart Images

Figure CN118946930A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to an apparatus and method for spatial audio representation and encoding, but is not limited to an apparatus and method for audio representation for an audio encoder. Background Art
[0002] Immersive audio codecs are being implemented to support a large number of operating points ranging from low bitrate operation to transparency. An example of such a codec is the Immersive Voice and Audio Service (IVAS) codec, which is designed to be suitable for use on communication networks such as 3GPP 4G / 5G networks, including use in immersive services such as immersive voice and audio for virtual reality (VR). The audio codec is expected to handle the encoding, decoding, and rendering of speech, music, and general audio. In addition, it is also expected to support channel-based audio and scene-based audio input, including spatial information about sound fields and sound sources. The codec is also expected to operate with low latency to enable conversational services, and to support high error robustness under various transmission conditions.
[0003] Metadata Assisted Spatial Audio (MASA) is an input format proposed for IVAS. It uses an audio signal and corresponding spatial metadata. The spatial metadata includes parameters that define the spatial aspects of the audio signal, and it may include, for example, direction and direct-to-total energy ratio in a frequency band. A MASA stream may be obtained, for example, by capturing spatial audio using a microphone of a suitable capture device. For example, a mobile device including multiple microphones may be configured to capture microphone signals, wherein a set of spatial metadata may be estimated based on the captured microphone signals. A MASA stream may also be obtained from other sources (such as specific spatial audio microphones (such as Ambisonics), studio mixes (e.g., 5.1 audio channel mixes)) or other content through suitable format conversion. Summary of the invention
[0004] According to a first aspect, a device is provided, comprising a component for performing the following operations: obtaining a value of a parameter representing an audio signal, the value comprising at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowed number of bits for encoding at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; encoding at least one direction value for at least one subframe of each subband of the frame based on at least one resolution entropy coding, wherein one resolution entropy coding of the at least one resolution entropy coding is further used to: perform a first resolution entropy coding on at least one direction value and determine the number of bits used to encode at least one direction value based on the first entropy coding; encoding at least one reduced direction value; The invention relates to a method for encoding a direction value by first-resolution entropy coding and determining a number of bits used for encoding at least one reduced direction value based on the first entropy coding; when the number of bits used for encoding at least one direction value based on the first-resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value, selecting the first-resolution entropy coding of at least one direction value; and when the number of bits used for encoding at least one direction value based on the first-resolution entropy coding is greater than a portion of the allowed number of bits for encoding at least one direction value and the number of bits used for encoding at least one direction value based on the first-resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value, selecting the first-resolution entropy coding of at least one reduced direction value.
[0005] The component for performing a first resolution entropy encoding on at least one direction value and determining the number of bits used to encode the at least one direction value based on the first entropy encoding can be used to: perform at least one resolution entropy encoding on at least one value determined from a difference between the at least one direction value and an average direction value for a frame, and determine the number of bits used to encode the at least one value, and the component for performing a first resolution entropy encoding on at least one reduced direction value and determining the number of bits used to encode the at least one reduced direction value based on the first entropy encoding can be used to: perform at least one resolution entropy encoding on at least one reduced value from a reduced difference based on a difference between the at least one direction value and the average direction value for a frame, and determine the number of bits used to encode the at least one reduced value.
[0006] The component for encoding at least one directional value for at least one subframe of each subband of a frame based on at least one resolution entropy coding can be further used to: perform second resolution entropy coding on at least one value based on at least one directional value, and determine the number of bits used to encode the at least one value based on the second resolution entropy coding, wherein, when the frame includes more than one time-frequency tile within the subband, the second resolution entropy coding can be a lower resolution coding than the first resolution entropy coding and utilizes the similarity between the time-frequency tiles within the subband within the frame; and when the number of bits used to encode the at least one value based on the second resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one directional value, select the second resolution entropy coding for the at least one value.
[0007] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.
[0008] The above-mentioned component can be further used to: when the number of bits used for encoding the at least one value based on the second resolution entropy coding is greater than the portion of the allowed number of bits for encoding the at least one direction value but less than the determined relaxed number of bits, select the second resolution entropy coding of the at least one value based on the at least one direction value. The relaxed number of bits can be a number of bits relative to the portion of the allowed number of bits for encoding the at least one direction value.
[0009] The above-mentioned component is used to encode at least one directional value of at least one subframe for each subband of a frame based on at least one resolution entropy coding, wherein one resolution entropy coding in the at least one resolution entropy coding can be further used to: perform third resolution entropy coding on at least one value based on at least one directional value, and determine the number of bits used to encode the at least one value based on the third resolution entropy coding, wherein the quantization resolution of the third resolution is lower than the first resolution entropy coding and the second resolution entropy coding; and when the number of bits used to encode the at least one value based on the first resolution entropy coding or the second resolution entropy coding is greater than a portion of the allowed number of bits for encoding the at least one value, select the third resolution entropy coding of the at least one value based on at least one directional value.
[0010] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.
[0011] The above-mentioned means may be further configured to: encode at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal.
[0012] The means for encoding at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal may be configured to: generate a weighted average of the at least one energy ratio value; and encode the weighted average of the at least one energy ratio value.
[0013] The means for encoding the weighted average of at least one energy ratio value may be further configured to: perform scalar non-uniform quantization on at least one weighted average of at least one energy ratio value.
[0014] At least one of the entropy codes may be a Golomb Rice code.
[0015] The above-mentioned component is further used for: storing and / or sending the encoded at least one direction value.
[0016] According to a second aspect, a method is provided, comprising: obtaining a value of a parameter representing an audio signal, the value comprising at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowed number of bits for encoding at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; encoding at least one direction value for at least one subframe of each subband of the frame based on at least one resolution entropy coding, wherein one resolution entropy coding of the at least one resolution entropy coding is further used to: perform a first resolution entropy coding on at least one direction value and determine the number of bits used to encode at least one direction value based on the first entropy coding; performing a first resolution entropy coding on at least one reduced direction value; The invention relates to a method for encoding a direction value according to the embodiment of the present invention and to a method for encoding a direction value according to the embodiment of the present invention. The method comprises: performing entropy coding of the at least one direction value based on the first resolution entropy coding and determining a number of bits used for encoding at least one reduced direction value based on the first entropy coding; selecting the first resolution entropy coding of the at least one direction value when the number of bits used for encoding the at least one direction value based on the first resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one direction value; and selecting the first resolution entropy coding of the at least one reduced direction value when the number of bits used for encoding the at least one direction value based on the first resolution entropy coding is greater than a portion of the allowed number of bits for encoding the at least one direction value and the number of bits used for encoding the at least one direction value based on the first resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one direction value.
[0017] Performing a first resolution entropy encoding on at least one direction value and determining the number of bits used to encode the at least one direction value based on the first entropy encoding may include: performing at least one resolution entropy encoding on at least one value determined from a difference between the at least one direction value and an average direction value for the frame, and determining the number of bits used to encode the at least one value, and the component for performing a first resolution entropy encoding on at least one reduced direction value and determining the number of bits used to encode the at least one reduced direction value based on the first entropy encoding may be used to: perform at least one resolution entropy encoding on at least one reduced value from a reduced difference based on a difference between the at least one direction value and the average direction value for the frame, and determine the number of bits used to encode the at least one reduced value.
[0018] Encoding at least one directional value for at least one subframe of each subband of a frame based on at least one resolution entropy coding may include: performing second resolution entropy coding on at least one value based on at least one directional value, and determining a number of bits used to encode the at least one value based on the second resolution entropy coding, wherein, when the frame includes more than one time-frequency tile within a subband, the second resolution entropy coding may be a lower resolution coding than the first resolution entropy coding, and utilizes similarities between time-frequency tiles within a subband within the frame; and selecting the second resolution entropy coding for the at least one value when the number of bits used to encode the at least one value based on the second resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one directional value.
[0019] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.
[0020] The method may further include: when the number of bits used to encode at least one value based on the second resolution entropy coding is greater than a portion of the allowed number of bits for encoding at least one directional value but less than the determined relaxed number of bits, selecting the second resolution entropy coding of at least one value based on at least one directional value.
[0021] The relaxed number of bits may be a number of bits relative to a fraction of an allowed number of bits for encoding at least one direction value.
[0022] Based on at least one resolution entropy coding, at least one directional value of at least one subframe for each subband of the frame is encoded, wherein one resolution entropy coding in the at least one resolution entropy coding may further include: performing third resolution entropy coding on at least one value based on the at least one directional value, and determining a number of bits used to encode the at least one value based on the third resolution entropy coding, wherein a quantization resolution of the third resolution is lower than that of the first resolution entropy coding and the second resolution entropy coding; and selecting the third resolution entropy coding of the at least one value based on the at least one directional value when the number of bits used to encode the at least one value based on the first resolution entropy coding or the second resolution entropy coding is greater than a portion of an allowed number of bits for encoding the at least one value.
[0023] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.
[0024] The method may further comprise encoding at least one energy ratio value for at least one subframe of each subband of the frame of the audio signal.
[0025] Encoding at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal may include: generating a weighted average of the at least one energy ratio value; and encoding the weighted average of the at least one energy ratio value.
[0026] Encoding the weighted average of at least one energy ratio value may further include scalar non-uniform quantizing the at least one weighted average of the at least one energy ratio value.
[0027] At least one of the entropy codes may be a Golomb Rice code.
[0028] The method may further comprise storing and / or transmitting the encoded at least one direction value.
[0029] According to a third aspect, a device is provided, comprising: at least one processor and at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, enable the device to at least: obtain a value representing a parameter of an audio signal, the value comprising at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtain an allowed number of bits for encoding at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; encode at least one direction value for at least one subframe of each subband of the frame based on at least one resolution entropy coding, wherein one resolution entropy coding of the at least one resolution entropy coding is further used to: perform a first resolution entropy coding on the at least one direction value, and determine a resolution entropy coding for the at least one direction value based on the first entropy coding. The invention relates to a method for encoding a direction value of a first resolution entropy coding method, wherein the number of bits used for encoding the direction value is determined; performing first-resolution entropy coding on at least one reduced direction value and determining the number of bits used for encoding the at least one reduced direction value based on the first entropy coding; when the number of bits used for encoding the at least one direction value based on the first-resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one direction value, selecting the first-resolution entropy coding of the at least one direction value; and when the number of bits used for encoding the at least one direction value based on the first-resolution entropy coding is greater than a portion of the allowed number of bits for encoding the at least one direction value and the number of bits used for encoding the at least one direction value based on the first-resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one direction value, selecting the first-resolution entropy coding of the at least one reduced direction value.
[0030] The device, which is caused to perform first resolution entropy encoding on at least one direction value and determine the number of bits used to encode the at least one direction value based on the first entropy encoding, can be caused to: perform at least one resolution entropy encoding on at least one value determined from the difference between the at least one direction value and the average direction value for the frame, and determine the number of bits used to encode the at least one value, and the component for performing first resolution entropy encoding on at least one reduced direction value and determining the number of bits used to encode the at least one reduced direction value based on the first entropy encoding can be used to: perform at least one resolution entropy encoding on at least one reduced value from the reduced difference based on the difference between the at least one direction value and the average direction value for the frame, and determine the number of bits used to encode the at least one reduced value.
[0031] The device, which is caused to encode at least one directional value for at least one subframe of each subband of a frame based on at least one resolution entropy coding, may be caused to: perform second resolution entropy coding on at least one value based on at least one directional value, and determine the number of bits used to encode the at least one value based on the second resolution entropy coding, wherein, when the frame includes more than one time-frequency tile within the subband, the second resolution entropy coding may be a lower resolution coding than the first resolution entropy coding and utilize the similarity between the time-frequency tiles within the subband within the frame; and select the second resolution entropy coding of the at least one value when the number of bits used to encode the at least one value based on the second resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one directional value.
[0032] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.
[0033] The device may further be configured to: select second resolution entropy encoding of at least one value based on at least one directional value when the number of bits used to encode at least one value based on the second resolution entropy encoding is greater than a portion of the allowed number of bits for encoding at least one directional value but less than the determined relaxed number of bits.
[0034] The relaxed number of bits may be a number of bits relative to a fraction of an allowed number of bits for encoding at least one direction value.
[0035] The device can be enabled to: encode at least one directional value for at least one subframe of each subband of a frame based on at least one resolution entropy coding, wherein one resolution entropy coding among the at least one resolution entropy coding can be enabled to: perform third resolution entropy coding on at least one value based on at least one directional value, and determine the number of bits used to encode the at least one value based on the third resolution entropy coding, wherein a quantization resolution of the third resolution is lower than that of the first resolution entropy coding and the second resolution entropy coding; and select the third resolution entropy coding of at least one value based on at least one directional value when the number of bits used to encode the at least one value based on the first resolution entropy coding or the second resolution entropy coding is greater than a portion of the allowed number of bits for encoding the at least one value.
[0036] The at least one value based on the at least one direction value may be at least one difference value from the at least one direction value compared to an average direction value for the frame.
[0037] The apparatus may be further caused to: encode at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal.
[0038] The apparatus caused to encode at least one energy ratio value for at least one subframe of each subband of a frame of an audio signal may be caused to: generate a weighted average of the at least one energy ratio value; and encode the weighted average of the at least one energy ratio value.
[0039] The apparatus caused to encode the weighted average of at least one energy ratio value may be further caused to: perform scalar non-uniform quantization on the at least one weighted average of the at least one energy ratio value.
[0040] At least one of the entropy codes may be a Golomb Rice code.
[0041] The apparatus may further be caused to: store and / or transmit the encoded at least one direction value.
[0042] According to a fourth aspect, a device is provided, comprising: an obtaining circuit configured to obtain a value of a parameter representing an audio signal, the value comprising at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; an obtaining circuit configured to obtain an allowed number of bits for encoding at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; an encoding circuit configured to encode at least one direction value for at least one subframe of each subband of the frame based on at least one resolution entropy encoding, wherein one resolution entropy encoding in the at least one resolution entropy encoding is: performing a first resolution entropy encoding on at least one direction value and determining the number of bits used for encoding at least one direction value based on the first entropy encoding; encoding at least one direction value for at least one subframe of each subband of the frame; The invention relates to a method for encoding a first-resolution entropy coding of at least one reduced direction value, and determining a number of bits used for encoding at least one reduced direction value based on the first entropy coding; when the number of bits used for encoding at least one direction value based on the first-resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value, selecting the first-resolution entropy coding of at least one direction value; and when the number of bits used for encoding at least one direction value based on the first-resolution entropy coding is greater than a portion of the allowed number of bits for encoding at least one direction value and the number of bits used for encoding at least one direction value based on the first-resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value, selecting the first-resolution entropy coding of at least one reduced direction value.
[0043] According to a fifth aspect, there is provided a computer program comprising instructions [or a computer-readable medium comprising program instructions], the instructions / program instructions being used to cause an apparatus to at least perform the following operations: obtain a value representing a parameter of an audio signal, the value comprising at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtain an allowed number of bits for encoding at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; encode at least one direction value for at least one subframe of each subband of the frame based on at least one resolution entropy coding, wherein one resolution entropy coding of the at least one resolution entropy coding is further used to: perform a first resolution entropy coding on at least one direction value, and determine a bit number used to encode at least one direction value based on the first entropy coding. The invention relates to a method for encoding a direction value of a first resolution and a method for encoding a direction value of a first resolution. The method comprises the steps of: performing first-resolution entropy coding on at least one reduced direction value and determining the number of bits used to encode at least one reduced direction value based on the first entropy coding; selecting first-resolution entropy coding of at least one direction value when the number of bits used to encode at least one direction value based on the first resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value; and selecting first-resolution entropy coding of at least one reduced direction value when the number of bits used to encode at least one direction value based on the first resolution entropy coding is greater than a portion of the allowed number of bits for encoding at least one direction value and the number of bits used to encode at least one direction value based on the first resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value.
[0044] According to a sixth aspect, there is provided a non-transitory computer-readable medium comprising program instructions, the program instructions being used to cause an apparatus to at least perform the following operations: obtain a value representing a parameter of an audio signal, the value comprising at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtain an allowed number of bits for encoding at least one direction value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; encode at least one direction value for at least one subframe of each subband of the frame based on at least one resolution entropy coding, wherein one resolution entropy coding of the at least one resolution entropy coding is further used to: perform a first resolution entropy coding on at least one direction value, and determine a number of bits used to encode at least one direction value based on the first entropy coding; Perform first resolution entropy coding on at least one reduced direction value, and determine the number of bits used to encode the at least one reduced direction value based on the first entropy coding; when the number of bits used to encode the at least one direction value based on the first resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one direction value, select the first resolution entropy coding of the at least one direction value; and when the number of bits used to encode the at least one direction value based on the first resolution entropy coding is greater than a portion of the allowed number of bits for encoding the at least one direction value and the number of bits used to encode the at least one direction value based on the first resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding the at least one direction value, select the first resolution entropy coding of the at least one reduced direction value.
[0045] According to a seventh aspect, a device is provided, comprising: a component for obtaining a value of a parameter representing an audio signal, wherein the value comprises at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; a component for obtaining an allowed number of bits for encoding at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; a component for encoding at least one directional value for at least one subframe of each subband of the frame based on at least one resolution entropy coding, wherein one resolution entropy coding of the at least one resolution entropy coding is further used to: perform a first resolution entropy coding on at least one directional value and determine the number of bits used to encode at least one directional value based on the first entropy coding; The reduced direction value is entropy encoded at a first resolution, and a number of bits used to encode at least one reduced direction value based on the first entropy encoding is determined; when the number of bits used to encode at least one direction value based on the first resolution entropy encoding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value, the first resolution entropy encoding of the at least one direction value is selected; and when the number of bits used to encode at least one direction value based on the first resolution entropy encoding is greater than a portion of the allowed number of bits for encoding at least one direction value and the number of bits used to encode at least one direction value based on the first resolution entropy encoding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value, the first resolution entropy encoding of the at least one reduced direction value is selected.
[0046] According to an eighth aspect, a computer-readable medium comprising program instructions is provided, wherein the program instructions are used to cause an apparatus to perform at least the following operations: obtain a value of a parameter representing an audio signal, the value comprising at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtain an allowed number of bits for encoding at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; encode at least one directional value for at least one subframe of each subband of the frame based on at least one resolution entropy coding, wherein one resolution entropy coding of the at least one resolution entropy coding is further used to: perform a first resolution entropy coding on at least one directional value and determine the number of bits used to encode at least one directional value based on the first entropy coding; The invention relates to a method for encoding at least one reduced direction value for first resolution entropy coding and determining a number of bits used for encoding at least one reduced direction value based on the first entropy coding; when the number of bits used for encoding at least one direction value based on the first resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value, selecting the first resolution entropy coding of at least one direction value; and when the number of bits used for encoding at least one direction value based on the first resolution entropy coding is greater than a portion of the allowed number of bits for encoding at least one direction value and the number of bits used for encoding at least one direction value based on the first resolution entropy coding is less than or equal to a portion of the allowed number of bits for encoding at least one direction value, selecting the first resolution entropy coding of at least one reduced direction value.
[0047] An apparatus comprises means for performing the actions of the method as described above.
[0048] A device is configured to perform the actions of the method described above.
[0049] A computer program includes program instructions for causing a computer to execute the method described above.
[0050] A computer program product stored on a medium may cause an apparatus to perform the method described herein.
[0051] An electronic device may include an apparatus as described herein.
[0052] A chipset may include the apparatus as described herein.
[0053] The embodiments of the present application are intended to solve the problems associated with the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] For a better understanding of the present application, reference will now be made by way of example to the accompanying drawings, in which:
[0055] Figure 1 Schematically illustrates a device system suitable for implementing some embodiments;
[0056] Figure 2 Schematically illustrating how Figure 1 The decoder shown in the device system shown in ;
[0057] Figure 3 According to some embodiments, Figure 2 A flowchart of the operation of the example decoder shown in;
[0058] Figure 4 Schematically illustrating how Figure 2 The example synthesis processor shown in ;
[0059] Figure 5 According to some embodiments, Figure 4 A flowchart of the operation of the example synthesis processor shown in ; and
[0060] Figure 6 An example apparatus suitable for implementing the means shown in the previous figures is shown. DETAILED DESCRIPTION
[0061] Suitable apparatus and possible mechanisms for encoding a parametric spatial audio stream comprising a transport audio signal and spatial metadata are described in more detail below.
[0062] As mentioned above, Metadata Assisted Spatial Audio (MASA) is an example of a parametric spatial audio format and representation suitable as an input format for IVAS.
[0063] It can be viewed as an audio representation consisting of "N channels + spatial metadata". It is a scene-based audio format, particularly suitable for spatial audio capture on practical devices such as smartphones. The idea is to describe the sound scene in terms of time-frequency varying sound source directions and, for example, energy ratios. Sound energy that is not defined (described) by direction is described as diffuse (coming from all directions).
[0064] As discussed above, spatial metadata associated with an audio signal may include multiple parameters per time-frequency block (such as multiple directions and a direct-to-total ratio, spread coherence, distance, etc. associated with each direction (or directional value). The spatial metadata may also include other parameters, or may be associated with other parameters that are considered non-directional (such as surround coherence, diffuse-to-total energy ratio, remainder-to-total energy ratio), but when combined with directional parameters, can be used to define characteristics of an audio scene. For example, a reasonable design choice that can produce good quality output is to determine that the spatial metadata includes one or more directions (and a direct-to-total ratio, spread coherence, distance value, etc. associated with each direction) for each time-frequency subframe.
[0065] As described above, the parameterized spatial metadata representation can use multiple concurrent spatial directions. Using MASA, the maximum number of concurrent directions proposed is two. For each concurrent direction, there can be associated parameters such as: direction index; direct to total ratio; extended coherence; and distance. In some embodiments, other parameters are defined, such as diffuse to total energy ratio; surround coherence; and residual to total energy ratio.
[0066] At very low bit rates (e.g., about 13.2-16.4 kbps), the number of bits available for encoding metadata is very small. For example, only about 3 kbps may be used for encoding metadata to obtain a sufficient bit rate for the audio signal codec.
[0067] In order to have sufficient frequency and time resolution (e.g., with 5 frequency bands and with a time resolution of 20 ms), in many cases only a few bits can be used for each value (e.g., direction parameter). In practice, this means that the quantization step size is relatively large. So, for example, for a certain time-frequency tile, the quantization points are at 0, ±45, ±90, ±135 and 180 degrees azimuth.
[0068] Dynamic resolution (or dynamic quantization resolution) may be implemented to try to improve the resulting encoder output. For example, as described in GB1811071.8, an entropy coder is implemented in which the angular resolution is provided by the energy ratio of each subband. If the resulting number of bits is higher than the maximum allowed number of bits, the quantization resolution is reduced and an entropy coder such as that described in EP3861548 is used. However, for some frames, the quantization resolution reduction may be too high, because the directional resolution of human hearing is about 1–2 degrees in the azimuth direction, and any azimuth jump, for example from 0 degrees to 45 degrees, is easily perceived and significantly reduces the audio quality, making the reproduction unnatural.
[0069] Concepts as discussed in the embodiments herein attempt to counteract the loss of angular resolution. In some embodiments, the maximum number of bits allowed is relaxed. In addition, in some embodiments, a check is performed to see if a slightly less accurate quantization of angles can be achieved within the entropy encoder by implementing a pseudo-embedded bitstream. In some embodiments, the quantization is further modified in the case where the input spatial metadata has only 1 subframe subband.
[0070] Reference will be made to Figure 1 The embodiments are described with reference to the example capture (or encoder / analyzer) and playback (or decoder / synthesizer) device or system 100 shown in FIG. In the following example, the audio signal input is an audio signal input from a microphone array, but it will be understood that the audio input can be in any suitable audio input format, and the following description will be detailed, where differences in processing will occur when different input formats are used.
[0071] System 100 is shown as having a capture portion and a playback (decoder / synthesizer) portion.
[0072] In some embodiments, the capture portion includes a microphone array audio signal input 102. The input audio signal can come from any suitable source, for example: two or more microphones mounted on a mobile phone, other microphone arrays, for example, B-format microphones or Eigenmike. In some embodiments, as described above, the input can be any suitable audio signal input, such as an Ambisonic signal, for example, first-order Ambisonics (FOA), higher-order Ambisonics (HOA), or speaker surround mixes and / or objects.
[0073] The microphone array audio signal input 102 may be provided to a microphone array front end 103. In some embodiments, the microphone array front end is configured to implement an analysis processor function configured to generate or determine appropriate (spatial) metadata associated with the audio signal, and to implement an appropriate transmission signal generator function to generate a transmission audio signal.
[0074] Therefore, the analysis processor function is configured to perform spatial analysis on the input audio signal, thereby generating suitable spatial metadata 106 in the frequency band. For all the above input types, there are known methods to generate suitable spatial metadata in the frequency band, for example, direction and direct to total energy ratio (or similar parameters such as diffuseness, i.e., ambient-to-total ratio). These methods are not described in detail herein, however, some examples may include performing a suitable time-frequency transform on the input signal, thereby estimating the delay value between the microphone pair that maximizes the inter-microphone correlation in the frequency band when the input is a mobile phone microphone array, formulating the corresponding direction value for the delay (as described in GB patent application number 1619573.7 and PCT patent application number PCT / FI2017 / 050778), and formulating the ratio parameter based on the correlation value. The direct to total energy ratio parameter for a multi-channel captured microphone array signal can be estimated based on the normalized cross-correlation parameter cor′(k,n) between the microphone pairs in the frequency band k, and the value of the cross-correlation parameter is between -1 and 1. The total energy ratio parameter r(k,n) can be directly calculated by converting the normalized cross-correlation parameter to the diffuse field normalized cross-correlation parameter cor′ D (k,n) is compared and determined as The direct to total energy ratio is further described in PCT publication WO2017 / 005978, which is incorporated herein by reference.
[0075] The metadata may take various forms, and in some embodiments include spatial metadata and other metadata. A typical parameterization for spatial metadata is a directional parameter in each frequency band, characterized by azimuth values φ(k,n) and elevation values θ(k,n), and an associated direct-to-total energy ratio r(k,n) in each frequency band, where k is the frequency band index and n is the time frame index.
[0076] In some embodiments, the parameters generated vary from band to band. Thus, for example, in band X, all parameters are generated and sent, while in band Y, only one parameter is generated and sent, and furthermore, in band Z, no parameters are generated or sent. A practical example may be that for some bands, such as the highest frequency band, some of the parameters are not needed for perceptual reasons.
[0077] In some embodiments, when the audio input is a FOA signal or a B-format microphone, the analysis processor function can be configured to determine parameters such as an intensity vector, obtain a direction parameter based on the intensity vector parameter, and compare the intensity vector length with the total sound field energy estimate to determine a ratio parameter. This method is referred to in the literature as directional audio coding (DirAC).
[0078] In some embodiments, when the input is a HOA signal, the analysis processor function can take a FOA subset of the signal and use the above method, or divide the HOA signal into multiple parts / sectors and use the above method in each part / sector. This part / sector based method is called high-order DirAC (HO-DirAC) in the literature. In this case, there is more than one simultaneous direction parameter per frequency band.
[0079] In some embodiments, when the input format is a speaker surround mix and / or object, the analysis processor function can be configured to convert the signal into a FOA signal (by using spherical harmonics coding gain) and analyze the above-mentioned direction and ratio parameters.
[0080] The output of the analysis processor function is therefore (spatial) metadata 106 determined in the frequency bands. The (spatial) metadata 106 may relate to directions and energy ratios in the frequency bands, but may also be of any of the metadata types listed above. The (spatial) metadata 106 may vary in time and frequency.
[0081] In some embodiments, the analysis functionality is implemented external to the system 100. For example, in some embodiments, spatial metadata associated with the input audio signal may be provided as a separate bitstream to the encoder 107. In some embodiments, the spatial metadata may be provided as a set of spatial (directional) index values.
[0082] As described above, the microphone array front end 103 is further configured to implement a transmission signal generator function to generate a suitable transmission audio signal 104. The transmission signal generator function 113 is configured to receive an input audio signal, which may be, for example, a microphone array audio signal 103, and generate a transmission audio signal 104. The transmission audio signal may be a multi-channel, stereo, binaural or mono audio signal. The generation of the transmission audio signal 104 may be implemented using any suitable method, as outlined below.
[0083] When the input is a microphone array audio signal, the transmit signal generator function can select the left and right microphone pairs and apply appropriate processing to the signal pairs, such as automatic gain control, microphone noise removal, wind noise removal, and equalization.
[0084] When the input is a FOA / HOA signal or a B-format microphone, the transmission signal 104 may be a directional beam signal oriented toward left and right directions, such as two opposing cardioid signals.
[0085] When the input is a loudspeaker surround mix and / or an object, the transmit signal 104 may be a downmix signal combining the left channels into a left downmix channel, and likewise for the right, with the center channel added to both transmit channels with an appropriate gain.
[0086] In some embodiments, the transmission signal 104 is an input audio signal, such as a microphone array audio signal. For example, in some cases, analysis and synthesis occur in a single processing step at the same device without intermediate encoding. The number of transmission channels can also be any suitable number (rather than one or two channels discussed in the example).
[0087] In some embodiments, the capture portion may include an encoder 107. The encoder 107 may be configured to receive the transmission audio signal 104 and the spatial metadata 106. The encoder 107 may also be configured to generate a bitstream 108 including the metadata information and the transmission audio signal in an encoded or compressed form.
[0088] For example, the encoder 107 may be implemented as an IVAS encoder or any other suitable encoder. In such an embodiment, the encoder 107 is configured to encode the audio signal and metadata and form an IVAS bitstream.
[0089] In turn, the bitstream 108 may be transmitted / stored as shown by the dashed line.
[0090] The system 100 may also include a decoder 109 portion. The decoder 109 is configured to receive, retrieve or otherwise obtain the bitstream 108 and generate a suitable spatial audio signal 110 from the bitstream for presentation to a listener / listener playback device.
[0091] Thus, the decoder 109 is configured to receive the bitstream 108 and demultiplex the encoded stream and thereby decode the audio signal to obtain the transport signal and the metadata.
[0092] The decoder 109 may also be configured to generate a spatial audio signal output 110 from the transport audio signal and the spatial metadata, such as a binaural audio signal that may be reproduced by headphones.
[0093] refer to Figure 2 , a schematic example of encoder 107 is shown in further detail.
[0094] exist Figure 21, wherein the transmission signal 104 is input to the transmission signal encoder 201. The transmission signal encoder 201 can be any suitable audio signal encoder. For example, an enhanced voice service (EVS) or immersive voice and audio service (IVAS) stereo core encoder implementation can be applied to the transmission (audio) signal to generate a suitable encoded transmission audio signal 204, which can be passed to the bitstream generator 207 or output as a separate bitstream to the spatial metadata parameters.
[0095] In some embodiments, the encoder 107 is configured to receive the spatial metadata 106 or spatial parameters and pass these to the parameter quantizer 203. For example, the determined direction parameters (azimuth and elevation or other coordinate systems) can be quantized by the parameter quantizer 203, and the index identifying the quantized value is passed to the quantization parameter entropy encoder 205.
[0096] In some embodiments, the encoder 107 further includes a quantization parameter entropy encoder 205 configured to obtain or receive quantization parameters and encode them to generate encoding spatial metadata 202 that can be passed to the bitstream generator 207 .
[0097] In some embodiments, the encoder 107 further comprises a bitstream generator 207 configured to obtain or receive the encoded transmission audio signal 204 and the encoded spatial metadata 202 (including extension and surround coherence parameters) and generate the bitstream 108 or a separate bitstream.
[0098] In the following example, the encoder 107 is configured to encode spatial audio parameters (MASA), in other words, to encode spatial metadata 106. For example, the direction values (azimuth and elevation values φ(k,n) and θ(k,n)) may first be quantized according to a spherical quantization scheme. Such a scheme may be found in patent publication EP3707706. As described above, each type of spatial audio parameter is first quantized to obtain a quantization index.
[0099] Furthermore, the resulting quantized index for the spatial audio parameter (e.g., MASA parameter) can be entropy encoded at different coding rates in response to a factor that specifies the number of bits allocated for the task. In addition, the codec can use multiple different coding rates and apply them to the encoding of the index of the spatial audio parameter.
[0100] Thus, an example of what may be returned below is that the audio metadata includes azimuth, elevation, and energy ratio data for each subband. The audio metadata may also include extended and surround coherence, but this is encoded first and then the remaining number of available bits is calculated by subtracting the coherence bits from the total number of bits.
[0101] In the MASA format, the directional data is represented in 16 bits, so that the azimuth is represented in approximately 9 bits and the elevation is represented in 7 bits. The energy ratio is represented in 8 bits. For each frame, there are between N=5 and M=4 time blocks, so (16+8)xMxN bits are required to store the uncompressed metadata for each frame. At higher frequency resolutions, there may be 20 or 24 frequency subbands.
[0102] In the following examples, the encoder and decoder operate with N=5 time blocks, but in other examples the number of time blocks, the bits used, and the subbands and parameters may be different.
[0103] about Figure 3 , an example quantization parameter entropy encoder 205 is shown. In some embodiments, the quantization parameter entropy encoder 205 includes an energy ratio value encoder 301. The energy ratio value encoder 301 is configured to receive the quantized energy ratio values 300 and generate encoded energy ratio values 302. In some examples, 3 bits are used to encode each energy ratio value. In addition, instead of sending all energy ratio values for all TF tiles, only one weighted average per subband is sent. The average is calculated by considering the total energy of each time block, thus favoring values for subbands with more energy.
[0104] In some embodiments, the example quantization entropy encoder includes more than one entropy encoder configured to receive the quantization direction value 302. In this example, a first direction (average / difference) entropy encoder 303, a second lower resolution entropy encoder 305, and a third lowest resolution average / difference entropy encoder 307 are shown. The first direction (average / difference) entropy encoder 303 (EC1), the second lower resolution entropy encoder 305 (EC2), and the third lowest resolution entropy encoder 307 (EC3) are configured to receive the quantization direction value and generate an encoding value that is passed to the encoding selector 309.
[0105] In some embodiments, the example quantized entropy encoder includes an encoding selector 309 that receives the output of the first, second, and third entropy encoders and selects one of these to output as an encoded direction value 310. In some embodiments, the selection can be based on the number of bits generated by each encoder and the number of bits allowed, such as described below in the encoding of index values for direction parameters for all TF tiles in a frame.
[0106] exist Figure 3In the arrangement shown in , the first, second and third entropy encoders are linked or connected in series so that, as described in further detail, the first entropy encoder 303 is operated first, and when the first entropy encoder 303 cannot encode the parameters in an acceptable manner, the second entropy encoder 305 is operated or activated. Similarly, when the second entropy encoder 305 cannot encode the parameters in an acceptable manner, the third entropy encoder 307 is operated or activated. In turn, the encoding selector is configured to select the output of the last received encoder or the output of an encoder selected based on a sequence such as third encoder / second encoder / first encoder.
[0107] However, in some embodiments, all three encoders operate in parallel or substantially in parallel, and then, one of the three encoder outputs is selected by the encoder selector based on which output is acceptable. In such embodiments, the first encoder output is checked and, if acceptable, outputted, otherwise the second encoder output is checked and, if acceptable, outputted, otherwise the third encoder output is outputted.
[0108] For example, an encoder selector or encoder operation may implement or perform encoding selection based on the following pseudo code.
[0109] Input: Index of quantized direction parameters (azimuth and elevation) and the number of bits allowed B allowed
[0110]
[0111]
[0112] In the above, the first direction (average / difference) entropy encoder 303 (EC1) corresponds to a first entropy coding scheme, in which the azimuth and elevation indexes can be encoded separately. In some embodiments, the scheme uses an optimized fixed value average index, which is subtracted from each index to obtain a difference index for each direction index. Furthermore, each obtained difference index can be converted to a positive value and then entropy encoded using the Golomb-Rice scheme. The optimized average index can also be entropy encoded to be sent to the decoder. In addition, in some embodiments, the directional entropy encoder is also based on the time-averaged direction value and the difference from the time-averaged direction value. For example, the difference index value is the difference between the current direction azimuth or elevation value and the azimuth or elevation value averaged by the previous frame or subframe, or, in some embodiments, is based on the difference of the reference azimuth or elevation value. In some embodiments, the average value (the amount of subtraction) is selected so that the number of bits obtained for encoding is minimized. Thus, in some examples, the device tests the value given by the average and also tests variations around that average (e.g., the average and values of ±1 or more), and selects the value that produces the smallest number of coded bits to send to the decoder as the “average”.
[0113] The second entropy encoder 305 (EC2) corresponds to a second entropy coding scheme that encodes the difference index at a less or lower resolution than EC1. Details of a suitable second entropy coding scheme can be found in patent publication WO2021 / 048468.
[0114] The third entropy encoder 307 (EC3) corresponds to a third entropy coding scheme that encodes the difference index at a resolution less than EC2. In this regard, EC3 may constitute the lowest resolution quantization scheme in the general framework described above. Details of a scheme suitable for use may be found in patent publication EP3861548.
[0115] From the general framework above, it can be seen that the choice of coding rate (and coding scheme) can be determined in part by the parameter B allowed The parameter indicates the number of bits allowed to be used to encode the direction index for a frame. allowed It may be a parameter determined by the encoding system according to the overall operating point / bitrate of the encoder for a particular time frame.
[0116] From the above, we can see that parameter B allowed It can be used to basically check if the number of bits required by the entropy coding scheme is less than the parameter B allowed The entropy coding scheme is determined. The checking process is performed in descending order of the number of bits required by the entropy coding scheme. The result of the checking process is to select allowed A constrained (number of coded bits) highest order entropy coding scheme.
[0117] For example, if the number of bits (bits_EC1) required by the first entropy coding scheme EC1 is less than B allowed , then the first entropy coding scheme is used. However, if it is determined that the bits required for EC1 are greater than constraint B allowed , then relative to B allowed The number of bits required for the second entropy coding scheme EC2 (bits_EC2) is checked. Furthermore, in some embodiments, the second entropy coding scheme EC2 is tested only for the non-2D case (i.e., when the elevation angles are non-zero on all tiles in the frame). If this second check indicates that the bits required for EC2 are less than B allowed , then the second entropy coding scheme EC2 is used to entropy encode the direction index for the frame. However, if the second check indicates that the bits required for EC2 are greater than (or equal to) B allowed , then the third entropy coding scheme EC3 is selected to encode the direction index. The above general framework can be extended for any number of coding rates, where each entropy coding scheme is based on the required number of bits (bits_ECn) and the allowed number of bits B allowed To choose.
[0118] about Figure 4 , illustrates a method or operation implemented by a first entropy encoder (EC1) according to some embodiments. In this example, the first entropy encoder is configured to perform entropy encoding (EC1) on the direction being encoded in a pseudo-embedded manner.
[0119] So, for example, Figure 4 As shown in step 401, the average direction of all time-frequency image blocks whose energy ratio is higher than a threshold is calculated.
[0120] Furthermore, if Figure 4 As shown in step 403, the remaining TF tiles are jointly encoded in elevation and azimuth, with one spherical index per tile.
[0121] The average direction is then encoded by sending the elevation and azimuth angles separately. Figure 4 As shown by step 405 in , this uses the number of bits given by the maximum alphabet of elevation and azimuth angles respectively from the considered TF tile.
[0122] The elevation and azimuth differences for the average are encoded separately. In these embodiments, Figure 4 As shown in step 407, one stream is used for azimuth difference values and one stream is used for elevation difference values.
[0123] Furthermore, if Figure 4As shown in step 409, for each angle value, a difference is calculated with respect to the average projected at the resolution of the corresponding tile.
[0124] Then, if Figure 4 As shown in step 411, the difference with respect to the mean in the index domain is transformed into the positive domain using the following function:
[0125]
[0126] Then, the positive difference index is encoded using the GR code of the determined (optimal) order. The determined (optimal) GR order is calculated on each data set, with one GR order value for the azimuth difference index and one GR order value for the elevation difference index.
[0127] Since the GR code is longer for larger values and shorter for smaller values, this means that if the encoder uses a smaller value of two units for the difference index, the difference on the average will have the same sign but be smaller, and the number of bits required for encoding will also be smaller.
[0128] Therefore, in some embodiments, a reduced / decreased difference index encoding is generated and furthermore it is determined how many bits can be gained by reducing / decreasing some difference indices. Figure 4 This is shown by step 415.
[0129] Furthermore, if Figure 4 In step 417, based on the number of bits obtained, a coding index or a reduced / reduced difference coding index is selected. For example, if the number of bits obtained by the encoding produced by the first entropy encoder (EC1) is higher than the allowed number of bits by a value NB, and the maximum number of bits that can be obtained is higher than NB, then the reduced difference index value is selected. If neither the encoding performed by the first entropy encoder nor the reduced / reduced encoding can obtain the required number of bits, the second entropy encoder EC2 or the third entropy encoder EC3 method is used.
[0130] The condition for checking whether the difference index can be reduced / reduced is that the difference must be greater than 0 and the angular resolution is above a given threshold. In some embodiments, the angular resolution can be given by the length of the alphabet of angle values. In an example, 20 degrees can be used as the minimum threshold for the elevation alphabet, and 40 degrees can be used as the minimum threshold for the azimuth alphabet. In some embodiments, the portion corresponding to the elevation angle can be applied only when the azimuth angle alphabet is adjusted based on the modified elevation angle value. In some implementations, the modification of the elevation angle is not used. However, if these values are used, the azimuth angle alphabet should be updated when checking the azimuth angle, and the original value should be requantized.
[0131] An example implementation in C for determining the number of bits available may be as follows:
[0132]
[0133]
[0134] In some embodiments, the maximum number of bits when reducing the quantization resolution can be relaxed. For example, before implementing the third entropy encoder (EC3) method, the number of bits that need to be reduced is limited to the number of TF tiles for which encoding is completed. The result of implementing this relaxed bit limit is that for some frames, the bit consumption may exceed the maximum bits allowed for metadata. However, the encoding is configured to handle this situation because the encoder typically operates below the required bit limit, and therefore on average, the number of bits used can be relaxed without exceeding the total number of bits over a reasonable period of time.
[0135] In some embodiments, when the encoder determines that there is only one subframe per subband in the input spatial audio data, the second entropy encoder is disabled or deactivated, and the second entropy coding (EC2) method is not considered and signaled. The second entropy encoder is disabled / deactivated because the method used in the second entropy encoder is a method of checking and utilizing the similarity between TF tiles within a subband, but for this case, there is only one tile per subband, and therefore there will be no similarity.
[0136] These embodiments may be implemented for lower bit rates.
[0137] In some embodiments, the decoder 109 includes a demultiplexer (not shown) configured to receive and demultiplex the bitstream to obtain the encoded transport audio signal and encoded spatial audio parameter metadata (MASA metadata) including the encoded energy ratio value 302 and the encoded direction value 310.
[0138] In some embodiments, the decoder 109 further includes a transmission audio signal decoder (not shown) configured to decode the encoded transmission audio signal to generate a transmission audio signal stream passed to the spatial synthesizer. The decoding process performed by the transmission audio signal decoder may be a suitable audio signal decoding scheme for the encoded transmission audio signal, such as an EVS decoder when EVS encoding is used.
[0139] Figure 5 Shown in further detail is a metadata decoder 509 configured to accept encoded spatial metadata (encoded energy ratio value 302 and encoded direction value 310) and decode the metadata to produce decoded spatial metadata (energy ratio value 502 and direction value or index 504).
[0140] In some embodiments, the metadata decoder 509 includes an energy ratio value decoder 501 configured to receive the encoded energy ratio value 302 and determine an energy ratio value based thereon.
[0141] Furthermore, the metadata decoder 509 further comprises an entropy decoder 503 , which is configured to obtain the encoded direction value 310 and output a direction value 504 .
[0142] In the above, the difference is determined relative to other direction values within a subframe or frame. However, in some embodiments, the difference can be determined relative to past subframes. In other words, the average can be determined within the current subframe, within the current frame, or over several time frames.
[0143] about Figure 6 , shows an example electronic device that can be used as part of any device of the above system. The device can be any suitable electronic device or device. For example, in some embodiments, the device 1400 is a mobile device, a user device, a tablet computer, a computer, an audio playback device, etc. The device can be configured to implement, for example, Figure 1 The encoder / analyzer part and / or decoder part shown in, or any functional blocks as described above.
[0144] In some embodiments, device 1400 includes at least one processor or central processing unit 1407. Processor 1407 may be configured to execute various program codes, such as the methods described herein.
[0145] In some embodiments, the device 1400 includes at least one memory 1411. In some embodiments, at least one processor 1407 is coupled to the memory 1411. The memory 1411 can be any suitable storage component. In some embodiments, the memory 1411 includes a program code portion for storing program codes that can be implemented on the processor 1407. In addition, in some embodiments, the memory 1411 can also include a storage data portion for storing data (e.g., data that has been processed or will be processed according to the embodiments described herein). The implemented program code stored in the program code portion and the data stored in the storage data portion can be retrieved by the processor 1407 via the memory-processor coupling when needed.
[0146] In some embodiments, the device 1400 includes a user interface 1405. In some embodiments, the user interface 1405 may be coupled to a processor 1407. In some embodiments, the processor 1407 may control the operation of the user interface 1405 and receive input from the user interface 1405. In some embodiments, the user interface 1405 may enable a user to enter commands to the device 1400, for example, via a keyboard. In some embodiments, the user interface 1405 may enable a user to obtain information from the device 1400. For example, the user interface 1405 may include a display configured to display information from the device 1400 to the user. In some embodiments, the user interface 1405 may include a touch screen or touch interface that enables information to be entered into the device 1400 and also displays information to a user of the device 1400. In some embodiments, the user interface 1405 may be a user interface for communication.
[0147] In some embodiments, the device 1400 includes an input / output port 1409. In some embodiments, the input / output port 1409 includes a transceiver. In such embodiments, the transceiver can be coupled to the processor 1407 and configured to enable communication with other devices or electronic devices, such as via a wireless communication network. In some embodiments. The transceiver or any suitable transceiver or transmitter and / or receiver components can be configured to communicate with other electronic devices or devices via wired or wired coupling.
[0148] The transceiver may communicate with other devices via any suitable known communication protocol. For example, in some embodiments, the transceiver may use a suitable radio access architecture based on: Advanced Long Term Evolution (LTE Advanced, LTE-A) or New Radio (NR) (or may be referred to as 5G), Universal Mobile Telecommunications System (UMTS) Radio Access Network (UTRAN or E-UTRAN), Long Term Evolution (LTE, the same as E-UTRA), 2G network (legacy network technology), Wireless Local Area Network (WLAN or Wi-Fi), Worldwide Interoperability for Microwave Access (WiMAX), Personal Communications Service (PCS), Wideband Code Division Multiple Access (WCDMA), systems using Ultra-Wideband (UWB) technology, sensor networks, mobile ad hoc networks (MANET), cellular Internet of Things (IoT) RAN and Internet Protocol Multimedia Subsystem (IMS), any other suitable options and / or any combination thereof.
[0149] The transceiver input / output port 1409 may be configured to receive signals.
[0150] In some embodiments, device 1400 may be used as at least part of a composite device.Input / output port 1409 may be coupled to headphones (which may be head-tracked or non-tracked headphones) or the like and speakers.
[0151] In general, various embodiments of the present invention may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described as block diagrams, flow charts, or using some other graphical representation, it is well known that the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.
[0152] Embodiments of the present invention may be implemented by computer software executable by a data processor of a mobile device (such as in a processor entity), or by hardware, or by a combination of software and hardware. In addition, in this regard, it should be noted that any block of the logic flow as in the accompanying drawings may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on a physical medium such as a memory chip or a memory block implemented within a processor, on a magnetic medium such as a hard disk or a floppy disk, and on an optical medium such as a DVD and its data variants, a CD.
[0153] The memory may be of any type suitable for the local technical environment and may be implemented using any appropriate data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The data processor may be of any type suitable for the local technical environment and may include, as non-limiting examples, one or more of a general purpose computer, a special purpose computer, a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a gate level circuit based on a multi-core processor architecture, and a processor.
[0154] Embodiments of the present invention may be practiced in various components such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available to convert a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.
[0155] Programs, such as those offered by Synopsys, Inc. of Mountain View, Calif., and Cadence Design, of San Jose, Calif., use well-established design rules and a library of pre-stored design modules to automatically route conductors and position components on a semiconductor chip. Once the design of a semiconductor circuit is complete, the resulting design in a standardized electronic format (e.g., Opus, GDSII, etc.) can be transmitted to a semiconductor manufacturing facility or "fab" for fabrication.
[0156] The foregoing description has provided a complete and useful description of exemplary embodiments of the present invention by way of exemplary and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the relevant art in view of the above description when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of the present invention will still fall within the scope of the present invention as defined by the appended claims.
Claims
1. An apparatus comprising components for performing the following operations: obtaining values of parameters representing an audio signal, the values comprising at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowed number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; encoding the at least one directional value of the at least one subframe for each subband of the frame based on at least one resolution entropy coding, wherein one of the at least one resolution entropy coding is further used to: performing a first resolution entropy encoding on the at least one direction value, and determining a number of bits used to encode the at least one direction value based on the first entropy encoding; performing a first resolution entropy encoding on at least one reduced direction value, and determining a number of bits to be used to encode the at least one reduced direction value based on the first entropy encoding; selecting the first resolution entropy encoding of the at least one directional value when the number of bits used to encode the at least one directional value based on the first resolution entropy encoding is less than or equal to a portion of the allowed number of bits for encoding the at least one directional value; and The first resolution entropy encoding of the at least one reduced directional value is selected when the number of bits used to encode the at least one directional value based on the first resolution entropy encoding is greater than the portion of the allowed number of bits for encoding the at least one directional value and the number of bits used to encode the at least one directional value based on the first resolution entropy encoding is less than or equal to the portion of the allowed number of bits for encoding the at least one directional value.
2. The device according to claim 1, wherein: The means for entropy encoding the at least one directional value at a first resolution and determining a number of bits to be used to encode the at least one directional value based on the first entropy encoding is configured to: entropy encode at least one value determined from a difference of the at least one directional value compared to an average directional value for the frame at least at a first resolution and determine a number of bits to be used to encode the at least one value, and The means for performing a first resolution entropy encoding of the at least one reduced direction value and determining a number of bits used to encode the at least one reduced direction value based on the first entropy encoding is used to: perform at least one resolution entropy encoding of at least one reduced value from a reduced difference based on the difference of the at least one direction value compared to the average direction value for the frame, and determine the number of bits used to encode the at least one reduced value.
3. The device according to any one of claims 1 or 2, wherein: The means for encoding the at least one directional value for at least one subframe of each subband of the frame based on at least one resolution entropy coding is further configured to: performing a second resolution entropy encoding on at least one value based on the at least one directional value, and determining a number of bits used to encode the at least one value based on the second resolution entropy encoding, wherein the second resolution entropy encoding is a lower resolution encoding than the first resolution entropy encoding when the frame includes more than one time-frequency tile within a subband and utilizes similarities between time-frequency tiles within a subband within the frame; and The second resolution entropy encoding of the at least one value is selected when the number of bits used to encode the at least one value based on the second resolution entropy encoding is less than or equal to the portion of the allowed number of bits used to encode the at least one directional value.
4. The device according to claim 3, wherein: The at least one value based on the at least one direction value is at least one difference value from the at least one direction value compared to an average direction value for the frame.
5. The device according to any one of claims 3 or 4, wherein: The component is further used to: when the number of bits used to encode the at least one value based on the second resolution entropy coding is greater than the portion of the allowed number of bits used to encode the at least one directional value but less than the determined relaxed number of bits, select the second resolution entropy coding of the at least one value based on the at least one directional value.
6. The device according to claim 5, wherein: The relaxed number of bits is a number of bits relative to the portion of the allowed number of bits for encoding the at least one direction value.
7. The device according to claim 5 or 6, wherein: The means is configured to encode the at least one directional value for the at least one subframe of each subband of the frame based on at least one resolution entropy coding, wherein one of the at least one resolution entropy coding is further configured to: performing a third resolution entropy encoding on at least one value based on the at least one directional value, and determining a number of bits used to encode the at least one value based on the third resolution entropy encoding, wherein the third resolution has a lower quantization resolution than the first resolution entropy encoding and the second resolution entropy encoding; and When the number of bits used to encode the at least one value based on the first resolution entropy coding or the second resolution entropy coding is greater than the portion of the allowed number of bits for encoding the at least one value, the third resolution entropy coding of the at least one value based on the at least one directional value is selected.
8. The device according to claim 7, wherein: The at least one value based on the at least one direction value is at least one difference value from the at least one direction value compared to an average direction value for the frame.
9. The device according to any one of claims 1 to 8, wherein: The means are further configured to encode the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal.
10. The device according to claim 6, wherein: The means for encoding the at least one energy ratio value for the at least one subframe of each subband of a frame of the audio signal is configured to: generating a weighted average of the at least one energy ratio value; and The weighted average of the at least one energy ratio value is encoded.
11. The device according to claim 10, wherein: The means for encoding the weighted average of the at least one energy ratio value is further configured to: perform scalar non-uniform quantization on the at least one weighted average of the at least one energy ratio value.
12. The device according to any one of claims 1 to 11, wherein: The at least one entropy coding is a GolombRice coding.
13. The device according to any one of claims 1 to 12, wherein: The component is further configured to store and / or transmit the encoded at least one direction value.
14. A method comprising: obtaining values of parameters representing an audio signal, the values comprising at least one directional value and at least one energy ratio value for at least one subframe of each subband of a frame of the audio signal; obtaining an allowed number of bits for encoding the at least one directional value and the at least one energy ratio value for the at least one subframe of each subband of the frame of the audio signal; encoding the at least one directional value of the at least one subframe for each subband of the frame based on at least one resolution entropy coding, wherein one of the at least one resolution entropy coding is further used to: performing a first resolution entropy encoding on the at least one direction value, and determining a number of bits used to encode the at least one direction value based on the first entropy encoding; performing a first resolution entropy encoding on at least one reduced direction value, and determining a number of bits to be used to encode the at least one reduced direction value based on the first entropy encoding; selecting the first resolution entropy encoding of the at least one directional value when the number of bits used to encode the at least one directional value based on the first resolution entropy encoding is less than or equal to a portion of the allowed number of bits for encoding the at least one directional value; and The first resolution entropy encoding of the at least one reduced directional value is selected when the number of bits used to encode the at least one directional value based on the first resolution entropy encoding is greater than the portion of the allowed number of bits for encoding the at least one directional value and the number of bits used to encode the at least one directional value based on the first resolution entropy encoding is less than or equal to the portion of the allowed number of bits for encoding the at least one directional value.
15. The method according to claim 14, wherein: Entropy encoding the at least one directional value at a first resolution and determining a number of bits to use to encode the at least one directional value based on the first entropy encoding comprises entropy encoding at least one value determined from a difference of the at least one directional value compared to an average directional value for the frame at least at a first resolution and determining a number of bits to use to encode the at least one value, and Performing a first resolution entropy encoding on the at least one reduced directional value and determining a number of bits used to encode the at least one reduced directional value based on the first entropy encoding includes performing at least one resolution entropy encoding on at least one reduced value from a reduced difference based on the difference of the at least one directional value compared to the average directional value for the frame, and determining a number of bits used to encode the at least one reduced value.
16. The method according to any one of claims 14 or 15, wherein: Encoding the at least one directional value for at least one subframe of each subband of the frame based on at least one resolution entropy encoding further comprises: performing a second resolution entropy encoding on at least one value based on the at least one directional value, and determining a number of bits used to encode the at least one value based on the second resolution entropy encoding, wherein the second resolution entropy encoding is a lower resolution encoding than the first resolution entropy encoding when the frame includes more than one time-frequency tile within a subband and utilizes similarities between time-frequency tiles within a subband within the frame; and The second resolution entropy encoding of the at least one value is selected when the number of bits used to encode the at least one value based on the second resolution entropy encoding is less than or equal to the portion of the allowed number of bits used to encode the at least one directional value.
17. The method according to claim 16, wherein: The at least one value based on the at least one direction value is at least one difference value from the at least one direction value compared to an average direction value for the frame.
18. The method according to any one of claims 16 or 17, wherein: The method further comprises: When the number of bits used to encode the at least one value based on the second resolution entropy coding is greater than the portion of the allowed number of bits for encoding the at least one directional value but less than the determined relaxed number of bits, the second resolution entropy coding of the at least one value based on the at least one directional value is selected.
19. The method according to claim 18, wherein: The relaxed number of bits is a number of bits relative to the portion of the allowed number of bits for encoding the at least one direction value.
20. The method according to any one of claims 18 or 19, wherein: Encoding the at least one directional value of the at least one subframe for each subband of the frame based on at least one resolution entropy coding, wherein one resolution entropy coding of the at least one resolution entropy coding further comprises: performing a third resolution entropy encoding on at least one value based on the at least one directional value, and determining a number of bits used to encode the at least one value based on the third resolution entropy encoding, wherein the third resolution has a lower quantization resolution than the first resolution entropy encoding and the second resolution entropy encoding; and When the number of bits used to encode the at least one value based on the first resolution entropy coding or the second resolution entropy coding is greater than the portion of the allowed number of bits for encoding the at least one value, the third resolution entropy coding of the at least one value based on the at least one directional value is selected.
