Encoding of spatial audio parameters and associated decoding - Patents.com

Abstract

1. An apparatus comprising: means configured to obtain directional parameter values ​​(108) associated with at least two time-frequency portions (202) of at least one audio signal (102); and to encode the obtained directional parameter values ​​based on a codebook (206), the codebook including two or more quantization levels, wherein a first quantization level includes a first set of quantization values, and a second or subsequent quantization level includes a second or further set of quantization values ​​and the quantization values ​​of the previous quantization level.

Description

[Technical field]

[0001] The present application relates to an apparatus and method for sound field related parameter encoding, but not exclusively for time-frequency domain direction related parameter encoding for audio coders and decoders. [Background technology]

[0002] Parametric spatial audio processing is a technical field of audio signal processing in which the spatial aspects of the sound are described using a set of parameters. For example, in parametric spatial audio capture from a microphone array, it is a typical and effective choice to estimate a set of directional metadata parameters from the microphone array signal, such as the direction of the sound in a frequency band and the ratio between the directional and omnidirectional parts of the captured sound in the frequency band. These parameters are known to well describe the perceptual spatial characteristics of the captured sound at the position of the microphone array. Therefore, these parameters can be utilized in the synthesis of spatial audio for headphones in a binaural manner, for loudspeakers, or into other formats such as Ambisonics.

[0003] Therefore, directional metadata such as direction within a frequency band and the direct-to-total energy ratio are parameterizations that are particularly useful for spatial audio capture.

[0004] A directional metadata parameter set consisting of one or more directional values ​​per frequency band and an energy ratio parameter associated with each directional value can also be utilized as spatial metadata (which may also include other parameters such as spread coherence, number of directions, distance, etc.) for an audio codec. The directional metadata parameter set may also include other parameters or may be associated with other parameters that are considered to be non-directional (surround coherence, diffuse-to-total energy ratio, remainder-to-total energy ratio, etc.). For example, these parameters can be estimated from an audio signal captured by a microphone array, from which, for example, a stereo signal to be conveyed using the spatial metadata can be generated.

[0005] Since some codecs are expected to operate at different bit rates ranging from very low to relatively high bit rates, different strategies are needed for the compression of spatial metadata to optimize the codec performance per operating point. The raw bit rate of the coded parameters (metadata) is relatively high, and therefore it is expected that only the most important parts of the metadata can be conveyed from the encoder to the decoder, especially at lower bit rates.

[0006] The decoder can decode the audio signal into a PCM signal, process the sound in frequency bands (using spatial metadata) and obtain a spatial output, e.g. a binaural output.

[0007] The above solutions are particularly suitable for encoding spatial audio captured from a microphone array (e.g. in a mobile phone, a video camera, a VR camera, a standalone microphone array). However, it may be desirable for such an encoder to also have other forms of input than the signals captured by the microphone array, e.g. loudspeaker signals, audio object signals, or Ambisonics signals. Summary of the Invention

[0008] According to a first aspect, an apparatus is provided comprising means configured for obtaining directional parameter values ​​associated with at least two time-frequency portions of at least one audio signal and for encoding the obtained directional parameter values ​​based on a codebook, the codebook including two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second or subsequent quantization level comprises a second or further set of quantization values ​​and a quantization value of a preceding quantization level.

[0009] The means configured to encode the obtained directional parameter values ​​based on the codebook may further be configured to determine for each obtained directional parameter value a closest quantization value from the set of quantization values ​​of the determined quantization level and the quantization value of the previous quantization level, and to generate a code word for each obtained directional parameter value based on the associated closest quantization value.

[0010] The means configured to encode the obtained directional parameter values ​​based on the codebook may be further configured to entropy encode the generated codewords for the subbands within the frame comprising the subbands and the time block.

[0011] The means configured to encode the obtained directional parameter value based on the codebook may further be configured to iteratively compare the number of bits needed to entropy encode a codeword for the directional parameter value for a sub-band in a frame comprising the sub-band and the time block based on the selected quantization level with the number of allocated bits until the number of bits needed to entropy encode the codeword for the directional parameter value is less than or equal to the number of allocated bits, and in response to the number of bits needed to entropy encode the codeword for the directional parameter value being greater than the number of allocated bits, select a previous quantization level and re-encode the obtained directional parameter value based on the previous quantization level.

[0012] The means configured to encode the directional parameter value based on the codebook may be further configured to: order the determined closest quantization values ​​for the directional parameter value for the sub-bands in the frame comprising the sub-bands and the time block based on the determined angular quantization distortion; select one of the closest quantization values ​​iteratively and in the determined order until a number of bits required to entropy encode a codeword for the directional parameter value is less than or equal to the allocated number of bits; determine whether the selected closest quantization value is a factor of a quantization value of a previous quantization level; and re-encode the obtained directional parameter value with the quantization value of the previous quantization level.

[0013] The means configured to encode the obtained directional parameter value based on the codebook may be further configured to encode the azimuth directional parameter value based on the codebook, and to encode the elevation directional parameter value based on at least one average elevation directional parameter value for the subbands in the frame including the subbands and the time block.

[0014] The means may be further configured to determine an allocated number of bits for encoding each subband in a frame including the subband and the time block based on a value of the energy ratio value associated with the obtained directional parameter value.

[0015] According to a second aspect, an apparatus is provided comprising means configured to obtain at least one encoded bitstream comprising at least one codebook encoded directional parameter value, the codebook comprising two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second or next quantization level comprises a second or further set of quantization values ​​and a quantization value of the previous quantization level; and decoding the at least one codebook encoded directional parameter value.

[0016] According to a third aspect, there is provided a method comprising obtaining directional parameter values ​​associated with at least two time-frequency portions of at least one audio signal, and encoding the obtained directional parameter values ​​based on a codebook, the codebook comprising two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second or next quantization level comprises a second or further set of quantization values ​​and the quantization values ​​of the previous quantization level.

[0017] Encoding the obtained directional parameter values ​​based on the codebook may include: determining, for each of the obtained directional parameter values, a closest quantization value from a set of quantization values ​​of the determined quantization level and the quantization value of the previous quantization level; and generating a codeword for each of the obtained directional parameter values ​​based on the associated closest quantization value.

[0018] Coding the resulting directional parameter values ​​based on the codebook may further include entropy encoding the generated codewords for the subbands within the frame that includes the subbands and the time block.

[0019] Coding the obtained directional parameter value based on the codebook may further include: iteratively comparing the number of bits needed to entropy code a codeword for a directional parameter value for a sub-band in a frame including the sub-band and the time block based on the selected quantization level with the allocated number of bits until the number of bits needed to entropy code a codeword for the directional parameter value is less than or equal to the allocated number of bits, and in response to the number of bits needed to entropy code a codeword for the directional parameter value being greater than the allocated number of bits, selecting a previous quantization level and re-encoding the obtained directional parameter value based on the previous quantization level.

[0020] Encoding the directional parameter value based on the codebook may further include ordering the determined closest quantized values ​​for the directional parameter value for the subbands in the frame including the subbands and the time block based on the determined angular quantization distortion, selecting one of the closest quantized values ​​iteratively and in the determined order until a number of bits required to entropy encode a codeword for the directional parameter value is less than or equal to the allocated number of bits, determining whether the selected closest quantized value is a factor of a quantized value of a previous quantization level, and re-encoding the resulting directional parameter value with the quantized value of the previous quantization level.

[0021] Encoding the obtained direction parameter value based on the codebook may further include encoding the azimuth direction parameter value based on the codebook, and encoding the elevation direction parameter value based on at least one average elevation direction parameter value for the subbands in the frame including the subbands and the time block.

[0022] The method may further include determining an allocated number of bits for encoding each subband within a frame including the subband and the time block based on the value of the energy ratio value associated with the obtained directional parameter value.

[0023] According to a fourth aspect, there is provided a method comprising: obtaining at least one coded bitstream comprising at least one codebook coded directional parameter value, the codebook comprising two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second or subsequent quantization level comprises a second or further set of quantization values ​​and a quantization value of the previous quantization level; and decoding the at least one codebook coded directional parameter value.

[0024] According to a fifth aspect, there is provided an apparatus comprising at least one processor and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to cause, using the at least one processor, the apparatus to at least obtain directional parameter values ​​associated with at least two time-frequency portions of at least one audio signal, and to encode the obtained directional parameter values ​​based on a codebook, the codebook including two or more quantization levels arranged such that a first quantization level includes a first set of quantization values ​​and a second or next quantization level includes a second or further set of quantization values ​​and the quantization values ​​of the previous quantization level.

[0025] The apparatus for encoding the obtained directional parameter values ​​based on the codebook may further be made to determine for each obtained directional parameter value a closest quantization value from the set of quantization values ​​of the determined quantization level and the quantization value of the previous quantization level, and to generate a code word for each obtained directional parameter value based on the associated closest quantization value.

[0026] The apparatus that is caused to encode the obtained directional parameter values ​​based on the codebook may further be caused to entropy encode the generated codewords for the subbands within a frame that includes the subbands and the time block.

[0027] The apparatus that is caused to code the obtained directional parameter value based on the codebook may further be caused to iteratively compare the number of bits needed to entropy code a codeword for a directional parameter value for a subband in a frame comprising the subband and the time block based on the selected quantization level with the number of allocated bits until the number of bits needed to entropy code a codeword for the directional parameter value is less than or equal to the number of allocated bits, and in response to the number of bits needed to entropy code a codeword for the directional parameter value being greater than the number of allocated bits, select a previous quantization level and re-code the obtained directional parameter value based on the previous quantization level.

[0028] The apparatus for encoding the directional parameter value based on a codebook may further be made to order the determined closest quantization values ​​for the directional parameter value for the subbands in a frame including the subbands and the time block based on the determined angular quantization distortion, select one of the closest quantization values ​​iteratively and in the determined order until a number of bits required to entropy encode a codeword for the directional parameter value is less than or equal to the number of allocated bits, determine whether the selected closest quantization value is a factor of a quantization value of a previous quantization level, and re-encode the obtained directional parameter value using the quantization value of the previous quantization level.

[0029] The apparatus that is caused to encode the obtained direction parameter value based on the codebook may further be caused to encode the azimuth direction parameter value based on the codebook, and to encode the elevation direction parameter value based on at least one average elevation direction parameter value for subbands in a frame that includes the subband and the time block.

[0030] The apparatus may be further configured to determine an allocated number of bits for encoding each subband within a frame comprising the subband and the time block based on the value of the energy ratio value associated with the obtained directional parameter value.

[0031] According to a sixth aspect, there is provided an apparatus comprising at least one processor and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to cause, using the at least one processor, the apparatus to obtain at least one encoded bitstream including at least one codebook encoded directional parameter value, wherein the codebook includes two or more quantization levels arranged such that a first quantization level includes a first set of quantization values ​​and a second or next quantization level includes a second or further set of quantization values ​​and a quantization value of the previous quantization level, and to decode the at least one codebook encoded directional parameter value.

[0032] According to a seventh aspect, there is provided an apparatus comprising: means for obtaining directional parameter values ​​associated with at least two time-frequency portions of at least one audio signal; and means for encoding the obtained directional parameter values ​​based on a codebook, the codebook including two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second or next quantization level comprises a second or further set of quantization values ​​and the quantization values ​​of the previous quantization level.

[0033] According to an eighth aspect, there is provided an apparatus comprising: means for obtaining at least one encoded bitstream comprising at least one codebook encoded directional parameter value, the codebook comprising two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second or next quantization level comprises a second or further set of quantization values ​​and the quantization value of the previous quantization level; and means for decoding the at least one codebook encoded directional parameter value.

[0034] According to a ninth aspect, there is provided a computer program [or a computer readable medium comprising program instructions] comprising instructions to cause an apparatus to at least: obtain directional parameter values ​​associated with at least two time-frequency portions of at least one audio signal; and encode the obtained directional parameter values ​​based on a codebook comprising two or more quantization levels, wherein a first quantization level comprises a first set of quantization values ​​and a second or next quantization level comprises a second or further set of quantization values ​​and the quantization values ​​of the previous quantization level.

[0035] According to a tenth aspect, there is provided a computer program [or a computer readable medium comprising program instructions] comprising instructions to cause an apparatus to at least: obtain at least one encoded bitstream comprising at least one codebook encoded directional parameter value, the codebook comprising two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second or subsequent quantization level comprises a second or further set of quantization values ​​and a quantization value of the previous quantization level; and decode the at least one codebook encoded directional parameter value.

[0036] According to an eleventh aspect, there is provided a non-transitory computer-readable medium comprising program instructions to cause an apparatus to at least: obtain directional parameter values ​​associated with at least two time-frequency portions of at least one audio signal; and encode the obtained directional parameter values ​​based on a codebook, the codebook comprising two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second, or next, quantization level comprises a second, or further set of quantization values ​​and the quantization values ​​of the previous quantization level.

[0037] According to a twelfth aspect, there is provided a non-transitory computer-readable medium comprising program instructions to cause an apparatus to at least: obtain at least one encoded bitstream comprising at least one codebook encoded directional parameter value, wherein the codebook comprises two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second, or next, quantization level comprises a second, or further set of quantization values ​​and a quantization value of the previous quantization level; and decode the at least one codebook encoded directional parameter value.

[0038] According to a thirteenth aspect, there is provided an apparatus comprising: obtaining circuitry configured to obtain directional parameter values ​​associated with at least two time-frequency portions of at least one audio signal; and encoding circuitry configured to encode the obtained directional parameter values ​​based on a codebook, the codebook including two or more quantization levels arranged such that a first quantization level includes a first set of quantization values ​​and a second or next quantization level includes a second or further set of quantization values ​​and the quantization values ​​of the previous quantization level.

[0039] According to a fourteenth aspect, there is provided an apparatus comprising: acquisition circuitry configured to obtain at least one encoded bitstream comprising at least one codebook encoded directional parameter value, the codebook comprising two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second, or next, quantization level comprises a second, or further set of quantization values ​​and a quantization value of the previous quantization level; and decoding circuitry configured to decode the at least one codebook encoded directional parameter value.

[0040] According to a fifteenth aspect, there is provided a computer-readable medium comprising program instructions to cause an apparatus to at least: obtain directional parameter values ​​associated with at least two time-frequency portions of at least one audio signal; and encode the obtained directional parameter values ​​based on a codebook, the codebook comprising two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second or next quantization level comprises a second or further set of quantization values ​​and the quantization values ​​of the previous quantization level.

[0041] According to a sixteenth aspect, there is provided a computer-readable medium comprising program instructions to cause an apparatus to at least: obtain at least one encoded bitstream comprising at least one codebook encoded directional parameter value, wherein the codebook comprises two or more quantization levels arranged such that a first quantization level comprises a first set of quantization values ​​and a second, or next, quantization level comprises a second, or further set of quantization values ​​and a quantization value of the previous quantization level; and decode the at least one codebook encoded directional parameter value.

[0042] An apparatus comprising means for performing the actions of the method as set forth above.

[0043] An apparatus configured to perform the actions of a method as described above.

[0044] A computer program comprising program instructions for causing a computer to carry out the method as described above.

[0045] A computer program product stored on the medium may cause an apparatus to perform the methods as described herein.

[0046] The electronic device may comprise an apparatus as described herein.

[0047] A chipset may comprise an apparatus as described herein.

[0048] SUMMARY OF THE PRESENT APPLICATION The embodiments of the present application aim to address problems associated with the state of the art.

[0049] For a better understanding of the present application, reference is now made, by way of example only, to the accompanying drawings in which: [Brief description of the drawings]

[0050] [Figure 1] FIG. 1 illustrates a schematic diagram of a system of apparatus suitable for implementing some embodiments. [Diagram 2] FIG. 2 illustrates a schematic diagram of an encoder according to some embodiments; [Diagram 3] 3 shows a flow diagram of the operation of an encoder as shown in FIG. 2 according to some embodiments. [Figure 4] FIG. 3 is a schematic diagram of a directional encoder as shown in FIG. 2 according to some embodiments. [Diagram 5] 5 shows a flow diagram of the operation of a directional encoder as shown in FIG. 4 according to some embodiments. [Figure 6] FIG. 5 illustrates a schematic diagram of a flexible border codebook encoder portion of the directional encoder as shown in FIGS. 2 and 4 according to some embodiments. [Figure 7] 7 shows a flow diagram of the operation of the encoder of the flexible boundary codebook encoder portion as shown in FIG. 6 according to some embodiments. [Figure 8] FIG. 2 illustrates an example codebook angle assignment according to some embodiments. [Figure 9] 7 shows a further flow diagram of the operation of flexible boundary codebook selection for the encoder as shown in FIG. 6 according to some embodiments. [Figure 10]FIG. 2 shows a schematic diagram of an exemplary device suitable for implementing the depicted apparatus; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0051] The following describes in more detail suitable apparatus and possible mechanisms for providing combining and encoding spatial analysis derived metadata parameters. In the following description, the multi-channel system is described with respect to a multi-channel microphone implementation. However, as mentioned above, the input format can be any suitable input format, such as multi-channel loudspeaker, Ambisonics (FOA / HOA), etc. It is understood that in some embodiments, the channel locations are based on microphone locations or are virtual locations or directions.

[0052] Additionally, in the following examples, the output of the exemplary system is a multi-channel loudspeaker configuration. In other embodiments, the output may be rendered to a user via means other than loudspeakers. The multi-channel loudspeaker signal may also be generalized to two or more reproduced audio signals.

[0053] As described above, the directional metadata associated with the audio signal may include multiple parameters (such as multiple directions and a directivity-to-aggregate ratio, distance, etc. associated with each direction) per time-frequency tile. The directional metadata may also include other parameters, or may be considered non-directional but associated with other parameters (ambient coherence, diffuse-to-aggregate energy ratio, residual-to-aggregate energy ratio, etc.) that may be used to characterize the audio scene when combined with the directional parameters. For example, a reasonable design choice that can produce a good quality output is determined to be that the directional metadata includes two directions (and a directivity-to-aggregate ratio, distance value, etc. associated with each direction) per time-frequency subframe. However, as also explained above, bandwidth and / or storage limitations may require the codec not to transmit directional metadata parameter values ​​for every frequency band and time subframe.

[0054] Proposals include those disclosed in UK Patent Application No. 1811071.8 which considers lossy compression of metadata, and for International Application PCT / FI2019 / 050675 a vector quantization approach was discussed when the number of bits available for a given subband is very low. Even for codebooks with only a maximum of 9 bits, the vector quantizer approach increases the table ROM of the codec, with approximately 4kB of memory used for 4-dimensional codebooks of 2, 3, 4, ..., and 9 bits.

[0055] The concept described in the following embodiments relates to encoding spatial audio streams with transport audio signals and (spatial) direction metadata, where an apparatus and method are described that implements an embedded codebook with flexible boundaries rather than using a vector quantizer. To reduce memory consumption, in some embodiments, the codebook is one-dimensional and the codewords are indexed such that those forming the 1-bit codebook appear first, followed by those belonging to the 2-bit codebook but not the 1-bit codebook, then those belonging to the 3-bit codebook but not the 1- and 2-bit ones, and so on. In such embodiments, one subband is coded at a time. Based on the number of bits available for the current subband and the distance to the unquantized audio direction data, the codewords for each time-frequency tile are selected and their indices are coded using a suitable entropy coding method, for example Golomb-Rice coding.

[0056] With reference to Figure 1, an exemplary apparatus and system for implementing embodiments of the present application is shown. System 100 is shown having an "analysis" portion 121 and a "synthesis" portion 131. The "analysis" portion 121 is from receiving a multi-channel signal to encoding directional metadata and a transport signal, and the "synthesis" portion 131 is from decoding the encoded directional metadata and transport signal to presenting a reproduced signal (e.g., in the form of a multi-channel loudspeaker).

[0057] In the following description, the "analysis" portion 121 is described as a series of portions, but in some embodiments the portions may be implemented as functions within the same functional unit or portion. In other words, in some embodiments the "analysis" portion 121 is an encoder that includes at least one of a transport signal generator or an analysis processor as described below.

[0058] The input to the system 100 and the "analysis" portion 121 is a multi-channel signal 102. The "analysis" portion 121 may include a transport signal generator 103, an analysis processor 105, and an encoder 107. In the following example, a microphone channel signal input is described, but in other embodiments, any suitable input (or composite multi-channel) format may be implemented. In such an embodiment, directional metadata associated with the audio signal may be provided to the encoder as a separate bitstream. The multi-channel signal is passed to the transport signal generator 103 and the analysis processor 105.

[0059] In some embodiments, the transport signal generator 103 is configured to receive a multi-channel signal and generate a suitable audio signal format for encoding. The transport signal generator 103 can generate, for example, a stereo or mono audio signal. The transport audio signal generated by the transport signal generator can be in any known format. For example, when the input is an input where the incoming audio signal is a mobile phone microphone array audio signal, the transport signal generator 103 can be configured to select a left and right microphone pair and apply any suitable processing to the audio signal pair, such as automatic gain control, microphone noise cancellation, wind noise cancellation, and equalization. In some embodiments, when the input is a first order Ambisonic / higher order Ambisonic (FOA / HOA) signal, the transport signal generator can be configured to build directional beam signals pointing in the left and right directions, such as two opposing cardioid signals. Additionally, in some embodiments, when the input is a loudspeaker surround mix and / or objects, then the transport signal generator 103 may be configured to generate a downmix signal combining the left channel into a left downmix channel and the right channel into a right downmix channel, and adding the centre channel with a suitable gain to both transport channels.

[0060] In some embodiments, the transport signal generator is bypassed (or in other words, optional). For example, in some situations where analysis and synthesis are performed in the same device in a single processing step without intermediate processing, there is no generation of a transport signal and the input audio signal is passed on unprocessed. The number of transport channels generated can be any suitable number, e.g., not just one or two channels.

[0061] The output of the transport signal generator 103 may be passed to an encoder 107 .

[0062] In some embodiments, the analysis processor 105 is also configured to receive the multi-channel signal, analyze the signal, and create directional metadata 106 associated with the multi-channel signal, and therefore associated with the transport signal 104.

[0063] The analysis processor 105 may be configured to generate, for each time-frequency analysis interval, directional metadata parameters, which may include at least one directional parameter 108 and at least one energy ratio parameter 110 (as well as other parameters, in some embodiments, a non-exhaustive list including number of directions, ambient coherence, diffuse-to-total energy ratio, residual-to-total energy ratio, diffuse coherence parameters, and distance parameters). The directional parameters may be expressed in any suitable manner, for example, as spherical coordinates expressed as azimuth angle φ(k,n) and elevation angle θ(k,n).

[0064] In some embodiments, the number of directional metadata parameters may vary from one time-frequency tile to another. Thus, for example, in band X, all of the directional metadata parameters are obtained (generated) and transmitted, whereas in band Y, only one of the directional metadata parameters is obtained and transmitted, and in band Z, no parameters are obtained or transmitted. An example of this may be that for some time-frequency tiles corresponding to the highest frequency bands, some of the directional metadata parameters are not needed for perceptual reasons. The directional metadata 106 may be passed to the encoder 107.

[0065] In some embodiments, the analysis processor 105 is configured to apply a time-frequency transformation for the input signal. Then, for example, in a time-frequency tile, when the input is a cell phone microphone array, the analysis processor could be configured to estimate delay values ​​between pairs of microphones that maximize the correlation between the microphones. Then, based on these delay values, the analysis processor could be configured to construct corresponding direction values ​​for the directional metadata. Furthermore, the analysis processor could be configured to construct a directivity-to-aggregate ratio parameter based on the correlation values.

[0066] For example, in some embodiments where the input is a FOA signal, the analysis processor 105 may be configured to determine a magnitude vector. The analysis processor may then be configured to determine a directional parameter value for the directional metadata based on the magnitude vector. A diffuse-to-aggregate ratio may then be determined, from which a directivity-to-aggregate ratio parameter value for the directional metadata may be determined. This analysis method is known in the literature as Directional Audio Coding (DirAC).

[0067] For example, in some examples where the input is an HOA signal, the analysis processor 105 may be configured to split the HOA signal into multiple sectors, with the above-mentioned method being utilized in each of the sectors. This sector-based method is known in the literature as higher order DirAC (HO-DirAC). In these examples, there is more than one simultaneous directional parameter value for each time-frequency tile corresponding to multiple sectors.

[0068] Additionally, in some embodiments where the input is a loudspeaker surround mix and / or audio object-based signals, the analysis processor may be configured to convert the signals into a FOA / HOA signal format and obtain directional and directivity-to-aggregate ratio parameter values ​​as described above.

[0069] The encoder 107 may include an audio encoder core 109 configured to receive the transport audio signals 104 and generate a suitable encoding of these audio signals. In some embodiments, the encoder 107 may be a computer (executing suitable software stored in memory and on at least one processor) or alternatively a specific device utilizing, for example, an FPGA or an ASIC. The audio encoding may be performed using any suitable scheme.

[0070] The encoder 107 may further include a directional metadata encoder / quantizer 111 configured to receive the directional metadata and output an encoded or compressed form of the information. In some embodiments, the encoder 107 may further interleave, multiplex into a single data stream, or embed the directional metadata within the encoded downmix signal before transmission or storage, indicated by the dashed lines in Fig. 1. The multiplexing may be performed using any suitable scheme.

[0071] In some embodiments, the transport signal generator 103 and / or the analysis processor 105 may be located on a separate device (or otherwise separate) from the encoder 107. For example, in such an embodiment, directional metadata (and related non-directional metadata) parameters associated with the audio signal may be provided to the encoder as a separate bitstream.

[0072] In some embodiments, the transport signal generator 103 and / or the analysis processor 105 are part of the encoder 107, ie, located inside the encoder, and may be on the same device.

[0073] In the following description, the "composite" portion 131 is described as a series of portions, however, in some embodiments, the portions may be implemented as functions within the same functional device or portion.

[0074] On the decoder side, the received or acquired data (stream) may be received by a decoder / demultiplexer 133. The decoder / demultiplexer 133 may demultiplex the encoded stream and pass the audio encoded stream to a transport signal decoder 135 configured to decode the audio signal and obtain a transport audio signal. Similarly, the decoder / demultiplexer 133 may include a metadata extractor 137 configured to receive the encoded directional metadata (e.g., directional indices representing directional parameter values) and generate directional metadata.

[0075] The decoder / demultiplexer 133 may, in some embodiments, be a computer (executing suitable software stored in memory and on at least one processor) or, alternatively, a specific device utilizing, for example, an FPGA or an ASIC.

[0076] The decoded metadata and the transport audio signal may be passed to a synthesis processor 139 .

[0077] The “synthesis” part 131 of the system 100 further indicates a synthesis processor 139 configured to receive the transport audio signal and the directional metadata and, based on the transport signal and the directional metadata, reproduce synthetic spatial audio in any suitable format in the form of a multi-channel signal 110 (which may be in multi-channel loudspeaker format or, in some embodiments, any suitable output format such as a binaural or Ambisonics signal, depending on the use case).

[0078] The synthesis processor 139 therefore creates an output audio signal, e.g. a multi-channel loudspeaker signal or a binaural signal, based on any suitable known method. This will not be described in further detail here. However, as a simplified example, the rendering can be performed for the loudspeaker output according to any of the following methods: For example, the transport audio signal can be split into a directional and an ambient stream based on the directional-to-aggregate and diffuse-to-aggregate energy ratios. The direct stream can then be rendered based on a directional parameter using amplitude panning. The ambient stream can further be rendered using decorrelation. The directional and ambient streams can then be combined.

[0079] The output signal may be reproduced using a multi-channel loudspeaker arrangement or headphones, which may be head-tracked.

[0080] It should be noted that the processing blocks of Figure 1 can be located in the same or different processing entities. For example, in some embodiments, a microphone signal from a mobile device is processed using a spatial audio capture system (including an analysis processor and a transport signal generator), and the resulting spatial metadata (e.g., in the form of a MASA stream) and a transport audio signal are forwarded to an encoder (e.g., an IVAS encoder), including an encoder. In other embodiments, an input signal (e.g., a 5.1 channel audio signal) is forwarded directly to an encoder (e.g., an IVAS encoder), including an analysis processor, a transport signal generator, and an encoder.

[0081] In some embodiments, there may be two (or more) input audio signals, where a first audio signal is processed by the device shown in Figure 1 (leading to data as input for the encoder) and a second audio signal is forwarded directly to an encoder (e.g., an IVAS encoder), which includes an analysis processor, a transport signal generator, and an encoder. The audio input signals may then be coded independently in the encoder, or they may be combined in the parametric domain, for example, according to what may be called MASA mixing, for example.

[0082] In some embodiments, there may be a compositing section that includes separate decoder and compositing processor entities or devices, or the compositing section may include a single entity that includes both a decoder and a compositing processor. In some embodiments, the decoder block may process more than one incoming data stream in parallel. In this application, the term compositing processor may be interpreted as an internal or external renderer.

[0083] So, in summary, first the system (analysis part) is configured to receive a multi-channel audio signal. Then the system (analysis part) is configured to generate a suitable transport audio signal (e.g. by selecting some of the audio signal channels). Then the system is configured to encode the transport audio signal for storage / transmission. After this, the system may store / transmit the encoded transport audio signal and metadata. The system may obtain / receive the encoded transport audio signal and metadata. Then the system is configured to extract the transport audio signal and metadata from the encoded transport audio signal and metadata parameters, e.g. demultiplex and decode the encoded transport audio signal and metadata parameters.

[0084] The system (synthesis portion) is configured to synthesize an output multi-channel audio signal based on the extracted transport audio signal and the metadata.

[0085] With respect to FIG. 2, an exemplary analysis processor 105 and metadata encoder / quantizer 111 (as shown in FIG. 1) according to some embodiments will be described in further detail.

[0086] The analysis processor 105 in some embodiments includes a time-frequency domain transformer 201 .

[0087] In some embodiments, the time-frequency domain transformer 201 is configured to receive the multi-channel signal 102 and apply a suitable time-to-frequency domain transform, such as a Short Time Fourier Transform (STFT), to transform the input time-domain signal into suitable time-frequency signals that may be passed to the spatial analyzer 203 and the signal analyzer 205.

[0088] Thus, for example, the time-frequency signal 202 may be represented in a time-frequency domain representation by: s i (b,n), where b is the frequency bin index, n is the time frequency block (frame) index, and i is the channel index. In another formula, n can be considered as a time index with a lower sampling rate than that of the original time domain signal. These frequency bins can be grouped into subbands, grouping one or more of the bins into subbands of band index k=0,...,K-1. Each subband k is the lowest bin b k,low and the highest bin b k,high and the sub-bands are b k,low ~b k,high The widths of the subbands can approximate any suitable distribution, for example the Equivalent Rectangular Bandwidth (ERB) scale or the Bark scale.

[0089] In some embodiments, the analysis processor 105 includes a spatial analyzer 203. The spatial analyzer 203 may be configured to receive the time-frequency signals 202 and estimate the direction parameters 108 based on these signals. The direction parameters may be determined based on any audio-based "direction" determination.

[0090] For example, in some embodiments, spatial analyzer 203 is configured to estimate direction using two or more signal inputs. This represents the simplest configuration for estimating "direction", and more complex processing may be performed using even more signals.

[0091] Therefore, the spatial analyzer 203 may be configured to provide at least one azimuth angle and elevation angle for each frequency band and temporal time-frequency block within a frame of the audio signal, denoted as azimuth angle φ(k,n) and elevation angle θ(k,n). The direction parameters 108 may also be passed to a direction index generator 205.

[0092] The spatial analyzer 203 may also be configured to determine an energy ratio parameter 110. The energy ratio may be considered to be a determination of the energy of an audio signal that can be considered to come from a certain direction. The directivity to aggregate energy ratio r(k,n) may be estimated, for example, using a stability measure of the directivity estimate, or using any correlation measure, or any other suitable method for obtaining the ratio parameter. The energy ratio may be passed to an energy ratio encoder 207.

[0093] The spatial analyzer 203 may be further configured to determine a number of coherence parameters 112, which may include ambient coherence (γ(k,n)) and diffuse coherence (ζ(k,n)), both of which are analyzed in the time-frequency domain.

[0094] In summary therefore, the analysis processor is arranged to receive time domain multi-channel or other formats such as microphone or Ambisonic audio signals.

[0095] Following this, the analysis processor may apply a time domain to frequency domain transform (eg, STFT) to generate a time-frequency domain signal suitable for analysis, and then apply directional analysis to determine direction and energy ratio parameters.

[0096] The analysis processor may then be configured to output the determined parameters.

[0097] The direction, energy ratio and coherence parameters are expressed here per time index n, although in some embodiments the parameters may be aggregated over several time indexes. The same applies for the frequency axis, where as shown, a direction for several frequency bins b could be represented by one direction parameter in a band k of several frequency bins b. The same applies for all of the spatial parameters described herein.

[0098] In some embodiments, the direction data may be represented using 16 bits, such that each azimuth parameter is approximately represented using 9 bits and the elevation angle using 7 bits. In such an embodiment, the energy ratio parameter may be represented using 8 bits. For each frame, there may be N subbands (where N may be 1-24 and may be fixed at 5), M time frequency (TF) blocks (where the value of M may be M=4). Therefore, in this example, there are (16+8)×M×N bits required to store the uncompressed direction and energy ratio metadata for each frame.

[0099] As also shown in FIG. 2, an exemplary metadata encoder / quantizer 111 according to some embodiments is shown.

[0100] The metadata encoder / quantizer 111 may include a direction encoder 205. The direction encoder 205 is configured to receive the direction parameters 108 (such as azimuth angle φ(k,n) and elevation angle θ(k,n) (as well as, in some embodiments, the expected bit allocation) and generate therefrom a suitable encoded output. In some embodiments, the encoding is based on an arrangement of spheres forming a spherical lattice arranged in an annulus on a "surface" sphere defined by a look-up table defined by a determined quantization resolution. In other words, the spherical lattice uses the idea of ​​covering a sphere with smaller spheres and considering the center of the smaller sphere as a point that defines a lattice of approximately equidistant directions. The smaller spheres thus define a cone or solid angle around the center point that can be indexed according to any suitable indexing algorithm. Although spherical quantization is described here, any suitable quantization, linear or non-linear, may be used.

[0101] The quantized values ​​may then be further combined by determining whether the elevation values ​​of the corresponding directional parameters are similar enough to employ an embedded flexible boundary codebook.

[0102] The encoded direction parameters 206 may then be passed to a combiner 211 .

[0103] The metadata encoder / quantizer 111 may include an energy ratio encoder 207. The energy ratio encoder 207 is configured to receive the energy ratios and determine a suitable encoding for compressing the energy ratios for the sub-bands and time-frequency blocks. For example, in some embodiments, the energy ratio encoder 207 is configured to use 3 bits to encode each energy ratio parameter value.

[0104] Furthermore, in some embodiments, instead of transmitting or storing all energy ratio values ​​for all TF blocks, only one weighted average value per subband is transmitted or stored. The average may be determined by considering the total energy of each time block, thus favoring the values ​​of subbands with more energy.

[0105] In such an embodiment, the quantized energy ratio value 208 is the same for all TF blocks of a given subband.

[0106] In some embodiments, the energy ratio encoder 207 is further configured to pass the quantized (encoded) energy ratio value 208 to a combiner 211 .

[0107] The metadata encoder / quantizer 111 may include a combiner 211. The combiner is configured to receive the encoded (or quantized / compressed) directional parameters and the energy ratio parameters, combine them, and generate a suitable output (e.g., a metadata bitstream that can be combined with the transport signal or transmitted or stored separately from the transport signal).

[0108] With respect to FIG. 3, an exemplary operation of a metadata encoder / quantizer as shown in FIG. 2 is shown, according to some embodiments.

[0109] The first action is to obtain metadata (such as azimuth values, elevation values, energy ratios, etc.), as indicated by step 301 in FIG.

[0110] Next, as indicated by step 303 in FIG. 3, the direction values ​​(elevation, azimuth) may be compressed or encoded (eg, by applying spherical quantization, or any suitable compression).

[0111] In FIG. 3, as indicated by step 305, the energy ratio values ​​are compressed or encoded (eg, by generating a weighted average for each subband and then quantizing these as a 3-bit value).

[0112] Next, as indicated by step 307 in FIG. 3, the encoded direction values, energy ratios and coherence values ​​are combined to generate encoded metadata.

[0113] The direction encoder 205 is shown in further detail with respect to FIG.

[0114] The directional encoder may, in some embodiments, include a quantization decider 401. The quantization decider is configured to receive the coded / quantized energy ratio 208 per subband and determine from this value the quantization resolution for the azimuth and elevation angles for all time blocks of the current subband. The quantization resolution is set by allowing a predefined number of bits given by the value of the energy ratio, bits_dir0[0:N-1][0:M-1], which can be output to the bit allocation manager 403.

[0115] The directional encoder 205 may further include a bit allocation manager 403 configured to receive the quantization resolution determined for the azimuth and elevation angles for all time blocks of the current sub-band based on the energy ratio values ​​bits_dir0[0:N-1][0:M-1] and the allocated bits of the frame, and to change the quantization resolution to reduce the number of allocated bits to bits_dir1[0:N-1][0:M-1] such that the sum of the allocated bits is equal to the number of available bits left after encoding the energy ratios. The reduced bit allocation bits_dir1[0:N-1][0:M-1] may then be passed to the sub-band-based directional encoder 403.

[0116] The directional encoder 205 may further include a sub-band based directional encoder 405 configured to receive the (reduced) bit allocation from the bit allocation manager 403. The sub-band based directional encoder 405 is also configured to receive the directional parameters 108 and to encode them based on the bit allocation per sub-band.

[0117] With reference to FIG. 5, a flow diagram illustrating the operation of the direction encoder 205 as shown in FIG. 4 is shown.

[0118] The first operation, as indicated by step 501 in FIG. 5, is to obtain directional metadata (such as azimuth values, elevation values, etc.), coded energy ratio values, and bit allocations.

[0119] Next, in FIG. 5, as indicated by step 503, the quantization resolution is first determined based on the energy ratio value.

[0120] Next, as indicated by step 505 in FIG. 5, the quantization resolution may be changed based on the allocated bits for the frame.

[0121] Next, as indicated by step 507 in FIG. 5, the directional parameters for each sub-band may be compressed / encoded based on the modified quantized resolution.

[0122] With respect to FIG. 6, the sub-band-based directional encoder 405 as shown in FIG. 4 is shown in more detail.

[0123] The subband-based directional encoder 405, in some embodiments, includes a subband bit allocator 601. The subband bit allocator 601 is configured to receive the (reduced) bit allocation bits_dir1[0:N-1][0:M-1] and determine the number of bits allowed for a subband, e.g., bits_allowed=sum(bits_dir1[i][0:M-1]).

[0124] The sub-band-based directional encoder 405 includes, in some embodiments, a bit limit determiner 603. The bit limit determiner 603 is configured to find the maximum number of bits allocated per TF block of the current sub-band, max_b=max(bits_dir1[i][0:M-1]), and further determine whether the maximum number of bits allocated per TF block of the current sub-band is less than or equal to the determined limit number of bits. In other cases, whether max_b<=LIMIT_BITS_PER_TF. The value of the determined limit number of bits, LIMIT_BITS_PER_TF, is the limit number of bits per time-frequency (TF) tile for which a decision is made as to whether an embedded quantizer should be used instead of joint coding. For example, for an example array of TF tiles (blocks) with M=4 in a subband, the number of bits in a tile can be (3 2 1 3), or (3 3 3 3), or (2 1 1 2), respectively, when LIMIT_BITS_PER_TF=3, then the method may start checking whether an embedded quantizer can be used.

[0125] The subband-based directional encoder 405, in some embodiments, includes a distance determiner 605. The distance determiner 605 may be controlled by the bit limit determiner 603 such that when the maximum number of bits allocated per TF block of the current subband is less than or equal to the allowed bits, then the distances d1, d2 are determined, where the angular distance is calculated as follows:

[0126]

number

[0127] In some embodiments, the estimation is performed based on the unquantized angles and the actual values ​​in each of the codebooks, without calculating the quantized values.

[0128] In some embodiments, if the variance of the elevation angle is greater than the determined value, then more than one elevation angle value for the sub-band is encoded, so that the variance of the elevation angle is taken into account, as further detailed in International Application PCT / FI2019 / 050675.

[0129] The distance determiner 605 is configured to determine whether the distance d2 is less than the distance d1.

[0130] The sub-band-based orientation encoder 405, in some embodiments, includes a joint elevation / azimuth encoder 607. The joint elevation / azimuth encoder 607 can be controlled to jointly encode the elevation and azimuth values ​​of each TF block within the number of bits allocated for the current sub-band when the maximum number of bits allocated per TF block of the current sub-band is greater than the allowed bits, as determined by the bit limit determiner 603.

[0131] In addition, the joint elevation / azimuth encoder 607 may be controlled by the distance determiner 605 to jointly encode the elevation and azimuth values ​​of each TF block within the number of bits allocated for the current subband when the distance d2 is greater than the distance d1, in other words, when the estimated quantization distortion when using joint encoding is less than the estimated quantization distortion when using the flexible embedded codebook.

[0132] The subband-based direction encoder 405, in some embodiments, includes an average elevation encoder / flexible border embedded codebook encoder 609. The average elevation encoder / flexible border embedded codebook encoder 609 can be operable to receive the distance and direction parameters and determine, based on the distance determiner 605, that the distance d2 is greater than the distance d1 (and when the maximum number of bits allocated per TF block of the current subband is less than or equal to the allowed bits).

[0133] The mean elevation angle encoder / flexible boundary embedded codebook encoder 609 is configured to encode the mean elevation angle value using 1 or 2 bits (1 bit for value 0 degrees and 2 bits for + / - 36 degrees) in some embodiments. Additionally, the mean elevation angle encoder / flexible boundary embedded codebook encoder 609 is configured to employ a flexible boundary embedded codebook for the azimuth angle value of each of the considered TF tiles.

[0134] The operation of the subband-based directional encoder 405 is further explained with respect to the flow diagram shown in FIG.

[0135] The first operation may be to allocate sub-band bits based on the changed quantization resolution, as indicated by step 701 in FIG.

[0136] Next, in FIG. 7, as indicated by step 703, determine the maximum bits for each quantization resolution based on the energy ratio value, and perform a check to see if the maximum number of bits is less than the limit of the bits for each time-frequency block.

[0137] If the check determines that the maximum number of bits is greater than the limit, then next, in FIG. 7, as indicated by step 710, jointly encode the elevation and azimuth values within the number of bits assigned for the sub-band for each time-frequency block.

[0138] If the check determines that the maximum number of bits is less than or equal to the limit, then next, determine the distances d1 and d2 for the sub-frame of the current sub-band.

[0139] Next, in FIG. 7, there is a check as to whether d2 < d1, as indicated by step 707.

[0140] If the distance d2 >= d1, then next, in FIG. 7, as indicated by step 710, jointly encode the elevation and azimuth values within the number of bits assigned for the sub-band for each time-frequency block.

[0141] If the distance d2 < d1, then next, in FIG. 7, as indicated by step 709, encode the average elevation value using 1 or 2 bits (1 bit for a value of 0 degrees and 2 bits for + / - 36 degrees), and encode the azimuth value using a flexible boundary embedding codebook for each of the TF tiles considered.

[0142] Next, in FIG. 7, as indicated by step 711, the sub-encoded value can be output.

[0143] An exemplary pseudo-code form of the energy ratio / direction encoding operation can be as follows: 1. For each sub-band i = 1:N a. Encode the corresponding energy ratio value using 3 bits b. Set the quantization resolution for the azimuth and elevation for all time blocks of the current sub - band. The quantization resolution is set by allowing a given number of bits given by the value of the energy ratio, bits_dir0[0:N - 1][0:M - 1] 2. End for 3. Reduce the number of allocated bits, bits_dir1[0:N - 1][0:M - 1], such that the total number of allocated bits equals the number of available bits remaining after encoding the energy ratio 4. For each sub - band i = 1:N a. Calculate the bits allowed for the current sub - band: bits_allowed = sum(bits_dir1[i][0:M - 1]) b. Find the maximum number of bits max_b = max(bits_dir1[i][0:M - 1]) allocated to each TF block of the current sub - band c. If (max_b <= LIMIT_BITS_PER_TF), then i. Calculate two distances d1 and d2 for the sub - frame data of the current sub - band ii. If d2 < d1, then 1. Encode the average elevation value using 1 or 2 bits (1 bit for value 0 degrees and 2 bits for + / - 36 degrees) 2. Use a flexible boundary embedding codebook for each azimuth value of the TF tiles considered iii. Otherwise 1. Jointly encode the elevation and azimuth values of each TF block within the number of bits allocated for the current sub - band iv. End if d. Otherwise i. Jointly encode the elevation and azimuth values of each TF block within the number of bits allocated for the current sub - band e. End if 5. End for

[0144] An example of an embedded codebook for azimuth angles is shown with respect to Figure 8. This example shows a 1-bit codebook (indices 0 and 1) accounting for the forward 801 and backward 803 directions. A 2-bit codebook further incorporates left and right directions (indices 2 and 3). A 3-bit codebook further incorporates intermediate left and right (indices 4, 5, 6, 7) for the forward and backward directions.

[0145] The selection of which codebook to use to encode the azimuth angle value based on the embedded codebook may be illustrated by the example method shown in FIG.

[0146] The first operation is to calculate the azimuth angle φ using a B-bit codebook, as shown by step 901 in FIG. i of

number

[0147] The next operation is to determine the number of bits (nbits) required to encode the exponent using an entropy coder, for example a Golomb-Rice coder of order 0. The number of bits (nbits) is then checked against the number of bits allowed (allowed_bits), indicated by step 903 in FIG.

[0148] If nbits<=allowed_bits, then entropy coding is performed using the exponent, as indicated by step 904 in FIG.

[0149] If nbits>allowed_bits, then calculate the angle quantization distortion for each of the quantization values ​​taking into account the quantized elevation angle values, as shown by step 905 in FIG.

[0150] Next, the time-frequency tiles are sorted in ascending order of their angle quantization distortion, as indicated by step 907 in FIG.

[0151] Next, as indicated by step 909 in FIG. 9, we begin a loop through the time-frequency tiles in ascending order of their angle quantization distortion.

[0152] The loop determines in FIG. 9 by step 911 that if the quantized azimuth angle belongs only to the B-bit codebook and not to the B-1-bit codebook, then requantize it in the B-1-bit codebook and then recalculate the number of bits required to code the (requantized) exponent using an entropy coder (such as a Golomb-Rice coder).

[0153] Next, in FIG. 9, as indicated by step 913, a check is made to determine whether the number of bits (nbits) is less than or equal to the allowed bits.

[0154] If the check determines that the number of bits (nbits) is less than or equal to the allowed bits, then entropy coding is performed using the exponent, as indicated by step 904 in FIG.

[0155] If the check determines that the number of bits (nbits) is more than the allowed bits, then the loop is checked, as indicated by step 915 in FIG.

[0156] If there are more TF tiles to be tested, then the loop returns to the next increasing angle quantization distortion value.

[0157] If there are no more TF tiles to be processed, then a further check is made to determine whether the number of bits (nbits) is less than or equal to the allowed bits, as indicated by step 917 in FIG. 9.

[0158] If the check determines that the number of bits (nbits) is less than or equal to the allowed bits, then entropy coding is performed using the exponent, as indicated by step 904 in FIG.

[0159] If the check determines that the number of bits (nbits) is more than the allowed bits, then in FIG. 9, reduce the codebook level, set B=B−1, and estimate the angle quantization distortion for each quantization value for the new quantization value, as shown by step 919.

[0160] This can be summarized by the following pseudocode: 1. Azimuth angle φ in B-bit codebook i of

number

[0161] An exemplary c code implementation may be as follows: short truncGR0(float * data, float * data_hat, unsigned short * data_idx, short len, short bits_allowed, float * st, float * ct) { short i; short bits; short i_az; short remap3b[] = { 1,6,2,4,0,5,3,7}, remap2b[] = {1,2,0,3}; float diff[MASA_SUBFRAMES]; int indx[MASA_SUBFRAMES]; i_az = 0; bits = 0; set_f(data_hat, 0.0f, len); set_f(diff, 10000.0f, len); for (i = 0; i < len; i++) { if (data_idx[i] < MASA_NO_INDEX) / * MASA_NO_INDEX signals the case where the exponent does not need to be encoded, e.g. because it has a default value * / { data_idx[i] = quantize_phi(data[i]+180, 0, &data_hat[i], 8); data_hat[i] -= 180; data_idx[i] = remap3b[data_idx[i]]; bits += 1 + data_idx[i]; i_az++; diff[i] = -st[i] - ct[i]*cos(data[i] - data_hat[i]); / *(data[i] - data_hat[i])*(data[i] - data_hat[i]);* / } } if (bits_allowed < i_az) { bits = bits_allowed; for (i = 0; i < len; i++) { if (data_idx[i] < MASA_NO_INDEX) { data_idx[i] = 0; data_hat[i] = 0.0f; } } return bits; } i = 0; if (bits > bits_allowed) { sort_desc_ind(diff, len, indx); for (i = len-1; i >= 0; i--) { if (data_idx[indx[i]] < MASA_NO_INDEX) { if (data_idx[indx[i]] > 3) { bits -= data_idx[indx[i]]; data_idx[indx[i]] = quantize_phi(data[indx[i]] + 180, 0, &data_hat[indx[i]], 4); data_hat[indx[i]] -= 180; data_idx[indx[i]] = remap2b[data_idx[indx[i]]]; bits += data_idx[indx[i]]; diff[indx[i]] = -st[i] - ct[i] * cos(data[i] - data_hat[i]); } if (bits <= bits_allowed) { break; } } } } if (bits > bits_allowed) { sort_desc_ind(diff, len, indx); for (i = len - 1; i >= 0; i--) { if (data_idx[indx[i]] < MASA_NO_INDEX) { if (data_idx[indx[i]] > 1) { bits -= data_idx[indx[i]]; if (fabs(data[indx[i]]) <= 90) { data_idx[indx[i]] = 0; data_hat[indx[i]] = 0; } else { data_idx[indx[i]] = 1; data_hat[indx[i]] = -180; } bits += data_idx[indx[i]]; diff[indx[i]] = -st[i] - ct[i] * cos(data[i] - data_hat[i]); } if (bits <= bits_allowed) { break; } } } } if (bits > bits_allowed) { sort_desc_ind(diff, len, indx); for (i = len - 1; i >= 0; i--) { if (data_idx[indx[i]] < MASA_NO_INDEX) { if (data_idx[indx[i]] > 0) { bits -= data_idx[indx[i]]; data_idx[indx[i]] = 0; data_hat[indx[i]] = 0; } if (bits <= bits_allowed) { break; } } } } return bits; } short common_direction(IVAS_QDIRECTION * qdirection, short band_idx, short len, short bits_allowed, BSTR_ENC_HANDLE hMasaMetaData, float* elevation_orig, float * azimuth_orig ) { short nbits; float av_el; short no_th, i, id_th, k; float theta_cb[3]; short data, bits_crt; float dist, best_dist; float ct[MASA_SUBFRAMES], st[MASA_SUBFRAMES]; nbits = 0; if (bits_allowed == 0) { for (i = 0; i < len; i++) { qdirection->elevation[band_idx][i] = 0; qdirection->azimuth[band_idx][i] = 0; } return 0; } / * If less than 5 bits are available then the elevation is 0 and all bits are used for the azimuth * / if (bits_allowed <= 5) { set_f(qdirection->elevation[band_idx], 0.0f, len); set_f(st, 0.0f, len); for (i = 0; i < len; i++) { ct[i] = cos(elevation_orig[i]);} nbits = truncGR0(azimuth_orig, qdirection->azimuth[band_idx], qdirection->azimuth_index[band_idx], len, bits_allowed, st, ct); return nbits; } av_el = mean(qdirection->elevation[band_idx], len); no_th = no_theta_masa[0] +1 ; / * only 3 values ​​for theta * / theta_cb[0] = 0; theta_cb[1] = delta_theta_masa[2]; theta_cb[2] = -theta_cb[1]; best_dist = 500000.0f; id_th = 0; for (i = 0; i < no_th; i++) { dist = 0.0f; for (k = 0; k < len; k++) { dist += (elevation_orig[k] - theta_cb[i])* (elevation_orig[k] - theta_cb[i]); } if (dist < best_dist) { id_th = i; best_dist = dist; } } set_f(qdirection->elevation[band_idx], theta_cb[id_th], len); for (i = 0; i < len; i++) { qdirection->elevation_index[band_idx][i] = id_th; } if (id_th == 0) { push_next_indice(hMasaMetaData, 0, 1); / * Average theta index * / set_f(st, 0.0f, len); for (i = 0; i < len; i++) { ct[i] = cos(elevation_orig[i]); } nbits = truncGR0(azimuth_orig, qdirection->azimuth[band_idx], qdirection->azimuth_index[band_idx], len, bits_allowed - 1, st, ct) + 1; } else { set_f(st, sin(theta_cb[id_th]), len); set_f(ct, cos(theta_cb[id_th]), len); for (i = 0; i < len; i++) { st[i] *= sin(elevation_orig[i]); ct[i] *= cos(elevation_orig[i]); } push_next_indice(hMasaMetaData, id_th + 1, 2); / * Average theta index * / nbits = truncGR0(azimuth_orig,qdirection->azimuth[band_idx], qdirection->azimuth_index[band_idx], len, bits_allowed-2, st, ct) +2; } data = 0; for (i = 0; i < len; i++) { if (qdirection->azimuth_index[band_idx][i] < MASA_NO_INDEX) { data = GR_data(data, 0, &bits_crt); push_next_indice(hMasaMetaData, data, bits_crt); } } return nbits; }

[0162] With respect to the decoder, the metadata decoder may be configured to determine if the mean elevation angle is signaled, where the mean elevation angle is read using one or two bits, "0" for value 0, "10" for value +36, and "11" for value -36. Other values ​​may be used instead of 36 degrees. Furthermore, in some embodiments, more than one bit is used to encode the mean elevation angle, and therefore may have between five values ​​to choose from -0, + / -+theta_1, + / -\theta_2, where the exponents for the azimuth angles are then read using a 0th order Golomb-Rice decoder, and no further signaling of which codebook they belong to is required.

[0163] Therefore, the embodiments as described in this specification allow both a significant reduction in the table ROM for encoding the orientation and a reduction of up to 30% in the resulting angle quantization distortion, made possible by the fact that the optimization is performed in the angle distortion space, with each component being checked individually.

[0164] 10, an exemplary electronic device that may be used as an analysis or synthesis device is shown. The device may be any suitable electronic device or device. For example, in some embodiments, device 1400 is a mobile device, user equipment, tablet computer, computer, audio playback device, etc.

[0165] In some embodiments, device 1400 includes at least one processor or central processing unit 1407. Processor 1407 can be configured to execute various program code, such as methods, such as those described herein.

[0166] In some embodiments, the device 1400 comprises a memory 1411. In some embodiments, at least one processor 1407 is coupled to the memory 1411. The memory 1411 can be any suitable storage means. In some embodiments, the memory 1411 includes a program code section for storing program code executable on the processor 1407. Furthermore, in some embodiments, the memory 1411 can further include a storage data section for storing data, e.g., data that has been processed or is to be processed according to an embodiment as described herein. The implementation program code stored in the program code section and the data stored in the storage data section can be retrieved by the processor 1407 via the memory-processor coupling whenever needed.

[0167] In some embodiments, the device 1400 comprises a user interface 1405. The user interface 1405 may be coupled to a processor 1407 in some embodiments. In some embodiments, the processor 1407 may control the operation of the user interface 1405 and receive input from the user interface 1405. In some embodiments, the user interface 1405 may allow a user to input commands into the device 1400, for example, via a keypad. In some embodiments, the user interface 1405 may allow a user to obtain information from the device 1400. For example, the user interface 1405 may include a display configured to display information from the device 1400 to a user. The user interface 1405 may include a touch screen or touch interface in some embodiments that has the capability of both allowing information to be input into the device 1400 and further displaying information to a user of the device 1400. In some embodiments, the user interface 1405 may be a user interface for communicating with a position determiner as described herein.

[0168] In some embodiments, device 1400 includes an input / output port 1409. Input / output port 1409, in some embodiments, includes a transceiver. The transceiver in such an embodiment is coupled to processor 1407 and may be configured to enable communication with other apparatus or electronic devices, for example, via a wireless communication network. The transceiver, or any suitable transceiver or transmitter and / or receiver means, in some embodiments, may be configured to communicate with other electronic devices or apparatuses via wiring or a wired coupling.

[0169] The transceiver may communicate with the further device by any suitable known communication protocol, for example, in some embodiments, the transceiver may use a suitable universal mobile telecommunications system (UMTS) protocol, a wireless local area network (WLAN) protocol such as, for example, IEEE 802.X, a suitable short-range radio frequency communication protocol such as Bluetooth, or an infrared data communication pathway (IRDA).

[0170] The transceiver input / output port 1409 may be configured to receive signals and, in some embodiments, determine parameters as described herein by using a processor 1407 executing suitable code.

[0171] Generally, various embodiments of the present invention may be implemented in the form of hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in the form of hardware, while other aspects may be implemented in the form of firmware or software that may be executed by a controller, microprocessor, or other computing device. However, the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described as block diagrams, flow charts, or using any other graphical representation, it is understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in the form of hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controller, or other computing device, or any combination thereof, as non-limiting examples.

[0172] The embodiments of the present invention may be implemented by computer software executable by a data processor of a mobile device, for example in a processor entity, or by hardware, or by a combination of software and hardware. Furthermore, in this regard, it should be noted that any block of logic flow as in the figures may represent program steps, or interconnected logic circuits, blocks, and functions, or a combination of program steps and logic circuits, blocks, and functions. The software may be stored on a physical medium such as a memory chip, or a memory block implemented in a processor, a magnetic medium such as a hard disk or floppy disk, and an optical medium such as, for example, a DVD and its data variants, a CD, etc.

[0173] The memory may be of any type suitable for the local technology environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed and removable memories, etc. The data processor may be of any type suitable for the local technology environment and may include, by way of non-limiting examples, one or more of a general purpose computer, a special purpose computer, a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a gate level circuit, and a processor based on a multi-core processor architecture.

[0174] Embodiments of the present invention may be implemented in a variety of components, such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available for converting logic level designs into semiconductor circuit designs ready to be etched and formed on semiconductor substrates.

[0175] Programs such as those offered by Synopsys, Inc. of Mountain View, California, and Cadence Design of San Jose, California, use well-established design rules and libraries of pre-stored design modules to automatically route conductors and determine the locations of components on a semiconductor chip. Once a design for a semiconductor circuit is completed, the resulting design in a standardized electronic format (e.g., Opus, GDSII, or the like) can be transmitted to a semiconductor manufacturing facility or "fab" for fabrication.

[0176] The above description has provided a complete and informative description of exemplary embodiments of the present invention by way of illustrative and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the art in light of the above description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of this invention will still fall within the scope of the present invention as defined in the appended claims.

Claims

1. An apparatus comprising: obtaining directional parameter values ​​associated with each of at least two time-frequency tiles of at least one audio signal; encoding each directional parameter value based on a codebook, the codebook including two or more quantization resolutions arranged such that a first quantization resolution includes a set of quantization values ​​and at least a second or subsequent quantization resolution includes a further set of quantization values, the further set of quantization values ​​incorporating the set of quantization values ​​of the first quantization resolution; a means for detecting a temperature difference between the first and second electrodes, said means configured to encode each directional parameter value based on said codebook; determining for each directional parameter value a closest quantization value from the further set of quantization values ​​of the second quantization resolution; generating a codeword index for the closest quantized value for each of said directional parameter values; determining a number of bits required to entropy code the codeword exponents associated with the at least two time-frequency tiles; comparing a number of bits required to entropy code the codeword exponents associated with the at least two time-frequency tiles to a number of allocated bits; configured to: When the number of bits required to entropy code the codeword exponent is greater than the number of allocated bits, the apparatus ordering the at least two time-frequency tiles based on a determined angle quantization distortion of the determined closest quantization value for the directional parameter value associated with the at least two time-frequency tiles; entropy coding the codeword exponents associated with the at least two time-frequency tiles, one ordered time-frequency tile at a time, in the order of the at least two ordered time-frequency tiles, until a number of bits required to entropy code the codeword exponents associated with the at least two time-frequency tiles is less than or equal to a number of allocated bits. selecting the closest quantized value of the directional parameter value associated with the ordered time-frequency tile; determining whether the selected closest quantization value is an element of the further set of quantization values ​​but is not an element of the set of quantization values ​​of the first quantization resolution; when the determination is positive, regenerating codeword exponents for the directional parameter values ​​associated with the ordered time-frequency tiles using the set of quantization values ​​of the first quantization resolution and determining a number of bits required to entropy code the codeword exponents associated with the at least two time-frequency tiles; An apparatus comprising:

2. The means configured to encode the directional parameter value based on the codebook, encoding an azimuth direction parameter value based on the codebook; encoding an elevation parameter value based on at least one average elevation parameter value for the at least two time-frequency tiles; The apparatus of claim 1 , further configured to:

3. 3. The apparatus of claim 1, wherein the means is further configured to determine the number of allocated bits for encoding the directional parameter value associated with each of the at least two time-frequency tiles based on a value of an energy ratio value associated with the directional parameter value.

4. 1. A method comprising: obtaining directional parameter values ​​associated with each of at least two time-frequency tiles of at least one audio signal; encoding each directional parameter value based on a codebook, the codebook including two or more quantization resolutions arranged such that a first quantization resolution includes a set of quantization values ​​and at least a second or subsequent quantization resolution includes a further set of quantization values, the further set of quantization values ​​incorporating the set of quantization values ​​of the first quantization resolution; Including, encoding each directional parameter value based on the codebook; determining for each directional parameter value a closest quantization value from the further set of quantization values ​​of the second quantization resolution; generating a codeword index for the closest quantized value for each of said directional parameter values; Including, determining a number of bits required to entropy code the codeword exponents associated with the at least two time-frequency tiles; comparing a number of bits required to entropy code the codeword exponents associated with the at least two time-frequency tiles to a number of allocated bits; configured to: When the number of bits required to entropy code the codeword exponent is greater than the number of allocated bits, the method comprises: ordering the at least two time-frequency tiles based on a determined angle quantization distortion of the determined closest quantization value for the directional parameter value associated with the at least two time-frequency tiles; entropy coding the codeword exponents associated with the at least two time-frequency tiles, one ordered time-frequency tile at a time, in the order of the at least two ordered time-frequency tiles, until a number of bits required to entropy code the codeword exponents associated with the at least two time-frequency tiles is less than or equal to a number of allocated bits. selecting the closest quantized value of the directional parameter value associated with the ordered time-frequency tile; determining whether the selected closest quantization value is an element of the further set of quantization values ​​but is not an element of the set of quantization values ​​of the first quantization resolution; when the determination is positive, regenerating codeword exponents for the directional parameter values ​​associated with the ordered time-frequency tiles using the set of quantization values ​​of the first quantization resolution and determining a number of bits required to entropy code the codeword exponents associated with the at least two time-frequency tiles; and performing the steps of:

5. encoding the directional parameter value based on the codebook, encoding an azimuth direction parameter value based on the codebook; encoding an elevation parameter value based on at least one average elevation parameter value for the at least two time-frequency tiles; The method of claim 4 further comprising:

6. 6. The method of claim 4 or 5, further comprising determining the number of allocated bits for encoding the directional parameter values ​​associated with each of the at least two time-frequency tiles based on a value of an energy ratio value associated with the directional parameter values.

Images