Combined Input Format Spatial Audio Encoding

Abstract

1. An apparatus for encoding audio object parameters, the apparatus comprising: means for identifying one of at least two audio objects in an audio environment to be encoded separately; obtaining ratio parameters of the at least two audio objects for a time-frequency element of a frame comprising at least one time element and at least one frequency element; grouping ratio parameters associated with the at least two audio objects for the time-frequency element; quantizing the ratio parameters within the group, the quantization of the ratio parameters being configured to generate integer representations of ratio parameter values ​​that sum to a defined integer value for a particular group; encoding the integer representations of the ratio parameter values ​​in a first group as enumeration indexes; and encoding subsequent groups as difference values ​​relative to the first group.

Description

[Technical Field]

[0001] This application relates to an apparatus and method for combined input format spatial audio encoding, but is not limited to combined metadata-assisted spatial audio and object input format parameters. [Background technology]

[0002] Immersive audio codecs are being implemented to support multiple operating points, ranging from low bitrate operation to transparency. An example of such a codec is the Immersive Voice and Audio Services (IVAS) codec, which is designed for use over communication networks such as 3GPP 4G / 5G networks, including for immersive services such as immersive voice and audio for virtual reality (VR). This audio codec is expected to handle the encoding, decoding, and rendering of speech, music, and general-purpose audio. It is also expected to support channel-based and scene-based audio inputs, including spatial information about the sound field and sound sources. The codec is also expected to operate with low latency to enable conversational services, as well as support high error robustness under various transmission conditions.

[0003] A stereo signal may be encoded with, for example, an AAC encoder, and a mono signal may be encoded with an EVS encoder. The decoder may decode the audio signal into a PCM signal and process the sounds in the frequency band (using spatial metadata) to obtain a spatial output, for example a binaural output.

[0004] The aforementioned immersive audio codecs are particularly suited to encoding spatial sound captured from microphone arrays (e.g., in mobile phones, VR cameras, standalone microphone arrays), although such encoders can have other input types, e.g., loudspeaker signals, audio object signals, ambisonic signals. Summary of the Invention

[0005] According to a first aspect, there is provided an apparatus for encoding audio object parameters, the apparatus comprising: identifying one of at least two audio objects in an audio environment to be encoded separately; obtaining ratio parameters of the at least two audio objects for a time-frequency element of a frame comprising at least one time element and at least one frequency element, the ratio parameters being configured to identify a distribution of the particular audio object within an object portion of the overall audio environment and for the particular time-frequency element; grouping, for the time-frequency element, ratio parameters associated with the at least two audio objects; and quantizing the ratio parameters within the group, the quantizing the ratio parameters so as to generate integer representations of ratio parameter values ​​that sum up for the particular group to a defined integer value. and encoding the subsequent groups as differential values ​​relative to the first group or first set of groups, wherein the means for encoding the subsequent groups further comprises: selecting a subset of the integer representations of the ratio parameter values ​​for the subsequent groups; and encoding the selected subset of the integer representations of the ratio parameter values ​​for the subsequent groups as differential values ​​relative to the first group or first set of groups. The means for grouping orders the ratio parameters within the groups such that the selected subset of the integer representations of the ratio parameter values ​​includes integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded.

[0006] The means for selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group may be means for selecting all but one of the integer representations of the particular group, since the integer representations of the ratio parameter values ​​of the group are values ​​defined for all groups.

[0007] The means for encoding a selected subset of integer representations of ratio parameter values ​​of a subsequent group as difference values ​​relative to the first group or first set of groups may comprise: determining that integer representations of ratio parameter values ​​corresponding to one of the at least two audio objects in the audio environment to be encoded separately are zero for all frequency elements of the first time element; and encoding all but two of the integer representations for a particular group, wherein the encoding does not include the integer representation of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be encoded separately if the integer representation of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be encoded separately is zero for all frequency elements of the first time element.

[0008] The means for selecting a subset of integer representations of ratio parameter values ​​for a subsequent group may be means for determining that integer representations of ratio parameter values ​​corresponding to one of the at least two audio objects in the audio environment to be separately encoded are zero for all frequency elements of the first time element, and selecting, for the particular group, all but two of the integer representations, wherein the selecting does not include the integer representations of ratio parameter values ​​corresponding to one of the at least two audio objects in the audio environment to be separately encoded.

[0009] The means for encoding integer representations of ratio parameter values ​​in the first group or first set of groups as enumeration indexes may be means for generating integer values ​​based on indexing from the groups, the generated integer values ​​representing ratio parameters of the audio objects.

[0010] The means for quantizing ratio parameters within a group, where the quantization of the ratio parameters is configured to generate integer representations of ratio parameter values ​​that sum to a defined integer value for a particular group, can be means for quantizing: generating a single value by adding elements from the group of ratio parameters; and generating an integer representation from the single value by running an iteration loop from the zeroth iteration up to a single iteration number and sequentially associating index values ​​with iteration numbers of the iteration loop having valid selections of ratio parameters, where the integer value is the highest index value.

[0011] The means for quantizing the ratio parameters in the group may be means for quantizing the ratio values ​​in a particular selection using minimum nearest neighbor scalar quantization to obtain quantization index values, calculating a reconstructed value of the ratio parameter for the particular selection, calculating an error value based on a difference between the reconstructed ratio value and the particular selection of ratio parameter value, determining a sum of the quantization index values, and selecting at least one quantization index value to increment so that the sum of the quantization index values ​​equals the sum of the expected index values.

[0012] The means for selecting at least one quantization index value to increment so that the sum of the quantization index values ​​equals the sum of the expected index values ​​may be means for doing one of: selecting the at least one quantization index value to increment based on identifying a maximum decrease in the error value when the index value is incremented; or selecting the at least one quantization index value to increment based on identifying a minimum increase in the error value when the index value is incremented.

[0013] The means for encoding the selected subset of integer representations of ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups may be means for determining, for the selected subset of integer representations of ratio parameter values ​​of the subsequent group for a particular time element of the frame, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required for entropy coding differences between quantized frequency elements; determining, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required for entropy coding differences between quantized time elements; selecting, for the particular time element, the first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding differences within the particular time element of the frame; and selecting one of entropy coding of differences between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding differences within the particular time element of the frame.

[0014] The means for encoding a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups may be means for encoding a selected one of entropy coding of differences between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or second entropy coding parameter.

[0015] The means for encoding a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​for the first group or first set of groups may be means for performing, for a selected set of ratio parameters for a particular frequency element of the frame, determining, for a first entropy coding parameter and a second entropy coding parameter, a number of bits required for entropy coding of quantized differences between frequency elements; determining, for the first entropy coding parameter and a second entropy coding parameter, a number of bits required for entropy coding of quantized differences between time elements; selecting, for the particular frequency element, the first entropy coding parameter or the second entropy coding parameter based on a smaller number of bits required for coding differences within the particular time element of the frame; and selecting, for entropy coding based on a selected first entropy coding parameter or a second entropy coding parameter based on a smaller number of bits required for coding differences within the particular time element of the frame for the particular frequency element.

[0016] The means for encoding a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups may be means for encoding a selected one of entropy coding of differences between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or second entropy coding parameter.

[0017] The means for encoding a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups may be means for generating an indicator indicative of a selected first or second entropy coding parameter, and for entropy coding based on the selected first or second entropy coding parameter, generating an indicator indicative of a selected one of entropy coding of differences between frequency elements or time elements.

[0018] The entropy coding may be Golomb-Rice entropy coding, the first entropy coding parameter may be Golomb-Rice entropy coding order 0, and the second entropy coding parameter may be Golomb-Rice entropy coding order 1.

[0019] The means for encoding a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as differential values ​​relative to the first group or set of groups may be means for differentially encoding a selection of the ratio parameters based on a leading indexed time element group of the ratio parameter in the absence of a leading indexed frequency element group of the ratio parameter.

[0020] The ratio parameter configured to identify the distribution of a particular object within the object portion of the overall audio environment may be an ISM ratio.

[0021] The means for selecting a subset of integer representations of ratio parameter values ​​of the subsequent group may further comprise means for determining that integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded are substantially zero for all frequency elements of the first group or first set of groups, and encoding only the remaining selected subset of integer representations of ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups.

[0022] The first group may be a first group of time and frequency elements of the group, and the first group set is a group of frequency element groups associated with the first time element.

[0023] The means may further be means for separately encoding the identified one of the at least two audio objects.

[0024] According to a second aspect, an apparatus for decoding audio object parameters is provided, the apparatus comprising: means for obtaining a bitstream including encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment including more than two audio objects, the ratio parameters being configured to identify a distribution of the particular object within an object portion of the overall audio environment and with respect to the particular time-frequency element; identifying that one of the at least two audio objects in the audio environment is separately encoded; decoding an enumeration index in a first group or a first set of groups to generate a decoded integer representation of a ratio parameter value for the first group or the first set of groups; and decoding subsequent groups as difference values ​​with respect to the first group or the first set of groups, wherein the means for decoding the subsequent groups further determines, for the subsequent groups, a further ratio parameter value based on a difference between a defined total ratio value and a sum of the ratio values ​​in the group.

[0025] The means for decoding the subsequent group as a difference value relative to the first group or first set of groups may further comprise means for determining that a ratio parameter value corresponding to one of the at least two audio objects in the separately encoded audio environment is zero for all frequency elements of the first time element, and for the subsequent group, determining an additional ratio parameter value having a value of zero that represents one of the at least two audio objects in the separately encoded audio environment.

[0026] A group may be a vector of ratio parameters.

[0027] The means for decoding the enumeration index within the first group or first set of groups to generate a decoded integer representation of the ratio parameter value for the first group or first set of groups may be means for obtaining an integer value representing the encoded ratio parameter, converting the integer value to a selection of ratio parameters based on the indexing of the vector, and regenerating at least one further ratio parameter from the selection of ratio parameters.

[0028] The means for decoding the subsequent group as a differential value relative to the first group or first set of groups may be means for obtaining a differential indicator that identifies frequency differential or time differential encoding, obtaining an entropy encoding indicator that identifies entropy encoding parameters, and decoding a remaining selection of ratio parameters for the frame based on the differential indicator and the entropy encoding indicator.

[0029] The ratio parameter configured to identify the distribution of a particular object within the object portion of the overall audio environment may be an ISM ratio.

[0030] According to a third aspect, there is provided a method for encoding audio object parameters, the method comprising: identifying one of at least two audio objects in an audio environment to be encoded separately; obtaining ratio parameters of the at least two audio objects for a time-frequency element of a frame comprising at least one time element and at least one frequency element, the ratio parameters being configured to identify a distribution of the particular audio object within an object portion of the overall audio environment and for the particular time-frequency element; grouping ratio parameters associated with the at least two audio objects for the time-frequency element; and quantizing the ratio parameters within the group, wherein the quantization of the ratio parameters generates integer representations of ratio parameter values ​​that sum to a defined integer value for the particular group. and encoding the subsequent groups as differential values ​​relative to the first group or first set of groups, wherein the means for encoding the subsequent groups further comprises: selecting a subset of the integer representations of the ratio parameter values ​​for the subsequent groups; and encoding the selected subset of the integer representations of the ratio parameter values ​​for the subsequent groups as differential values ​​relative to the first group or first set of groups. The means for grouping is configured to order the ratio parameters within the groups such that the selected subset of the integer representations of the ratio parameter values ​​includes integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded.

[0031] Selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group may include selecting all but one of the integer representations of the particular group, since the integer representations of the ratio parameter values ​​of the group are values ​​defined for all groups.

[0032] Encoding a selected subset of the integer representations of ratio parameter values ​​of a subsequent group as difference values ​​relative to the first group or first set of groups may include: determining that the integer representations of ratio parameter values ​​corresponding to one of the at least two audio objects in the audio environment to be encoded separately are zero for all frequency elements of the first time element; and encoding all but two of the integer representations for the particular group, wherein the encoding does not include the integer representation of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be encoded separately if the integer representations of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be encoded separately are zero for all frequency elements of the first time element.

[0033] Selecting a subset of integer representations of ratio parameter values ​​for a subsequent group may include determining that integer representations of ratio parameter values ​​corresponding to one of the at least two audio objects in the audio environment to be separately encoded are zero for all frequency elements of the first time element, and selecting, for the particular group, all but two of the integer representations, wherein the selecting does not include integer representations of ratio parameter values ​​corresponding to one of the at least two audio objects in the audio environment to be separately encoded.

[0034] Encoding integer representations of ratio parameter values ​​in the first group or first set of groups as enumeration indexes may include generating integer values ​​based on indexing from the groups, the generated integer values ​​representing ratio parameters of the audio objects.

[0035] Quantizing the ratio parameters in the group may include generating a single value by adding elements from the group of ratio parameters, and generating an integer representation from the single value by executing an iteration loop from the zeroth iteration up to a single iteration number and sequentially associating index values ​​with iteration numbers of the iteration loop having valid selections of the ratio parameters, where the integer value is the highest index value.

[0036] Quantizing the ratio parameters in the group may include quantizing the ratio values ​​in a particular selection using minimum nearest neighbor scalar quantization to obtain quantization index values, calculating a reconstructed value of the ratio parameter for the particular selection, calculating an error value based on a difference between the reconstructed ratio value and the particular selection of ratio parameter value, determining a sum of the quantization index values, and selecting at least one quantization index value to increment so that the sum of the quantization index values ​​equals the sum of the expected index values.

[0037] Selecting at least one quantization index value to increment so that the sum of the quantization index values ​​equals the sum of the expected index values ​​may include one of selecting the at least one quantization index value to increment based on identifying a maximum decrease in the error value when the index value is incremented, or selecting the at least one quantization index value to increment based on identifying a minimum increase in the error value when the index value is incremented.

[0038] Encoding the selected subset of integer representations of ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups may include, for the selected subset of integer representations of ratio parameter values ​​of the subsequent group for a particular time element of the frame, performing: determining, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required for entropy coding the differences between the quantized frequency elements; determining, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required for entropy coding the differences between the quantized time elements; selecting, for the particular time element, the first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding the differences within the particular time element of the frame; and selecting one of the entropy coding of the differences between the frequency elements or the time elements for entropy coding based on the selected first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding the differences within the particular time element of the frame.

[0039] Encoding the selected subset of integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups may include encoding a selected one of entropy coding of differences between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or second entropy coding parameter.

[0040] Encoding a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​for the first group or first set of groups may include, for a selected set of ratio parameters for a particular frequency element of the frame, determining, for a first entropy coding parameter and a second entropy coding parameter, a number of bits required for entropy coding the quantized differences between the frequency elements; determining, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required for entropy coding the quantized differences between the time elements; selecting, for the particular frequency element, the first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding the differences within the particular time element of the frame; and selecting, for the particular frequency element, one of the entropy coding of the differences between the frequency elements or the time elements for entropy coding based on the selected first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding the differences within the particular time element of the frame.

[0041] Encoding the selected subset of integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups may include encoding a selected one of entropy coding of differences between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or second entropy coding parameter.

[0042] Encoding a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups may include generating an indicator indicating a selected first entropy coding parameter or a second entropy coding parameter, and generating an indicator indicating a selected one of entropy coding of the difference between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the second entropy coding parameter.

[0043] The entropy coding may be Golomb-Rice entropy coding, the first entropy coding parameter may be Golomb-Rice entropy coding order 0, and the second entropy coding parameter may be Golomb-Rice entropy coding order 1.

[0044] Encoding a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as differential values ​​relative to the first group or first set of groups may include differentially encoding a selection of the ratio parameters based on a leading-indexed time element group of the ratio parameter in the absence of a leading-indexed frequency element group of the ratio parameter.

[0045] The ratio parameter configured to identify the distribution of a particular object within the object portion of the overall audio environment may be an ISM ratio.

[0046] Selecting a subset of integer representations of ratio parameter values ​​of the subsequent group may include determining that integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded are substantially zero for all frequency elements of the first group or first set of groups, and encoding only the remaining selected subset of integer representations of ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups.

[0047] The first group may be a first group of time and frequency elements of the group, and the first group set is a group of frequency element groups associated with the first time element.

[0048] The method may further include separately encoding the identified one of the at least two audio objects.

[0049] According to a fourth aspect, there is provided a method for an apparatus for decoding audio object parameters, the method comprising: obtaining a bitstream including encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment including more than two audio objects, the ratio parameters being configured to identify a distribution of the particular object within an object portion of the overall audio environment and with respect to the particular time-frequency element; identifying that one of the at least two audio objects in the audio environment is separately encoded; decoding an enumeration index in a first group or a first set of groups to generate a decoded integer representation of a ratio parameter value for the first group or the first set of groups; and decoding a subsequent group as a difference value with respect to the first group or the first set of groups, wherein the means for decoding the subsequent group further determines, for the subsequent group, a further ratio parameter value based on a difference between a defined total ratio value and a sum of the ratio values ​​in the group.

[0050] Decoding the subsequent group as a difference value relative to the first group or first set of groups may further include determining that a ratio parameter value corresponding to one of the at least two audio objects in the separately encoded audio environment is zero for all frequency elements of the first time element, and determining, for the subsequent group, an additional ratio parameter value having a value of 0 that represents one of the at least two audio objects in the separately encoded audio environment.

[0051] A group may be a vector of ratio parameters.

[0052] Decoding the enumeration index within the first group or first set of groups to generate a decoded integer representation of the ratio parameter value for the first group or first set of groups may include obtaining an integer value representing the encoded ratio parameter, converting the integer value to a selection of ratio parameters based on the indexing of the vector, and regenerating at least one further ratio parameter from the selection of ratio parameters.

[0053] Decoding the subsequent group as a differential value relative to the first group or first set of groups may include obtaining a differential indicator that identifies frequency differential or time differential encoding, obtaining an entropy encoding indicator that identifies an entropy encoding parameter, and decoding a remaining selection of ratio parameters for the frame based on the differential indicator and the entropy encoding indicator.

[0054] The ratio parameter configured to identify the distribution of a particular object within the object portion of the overall audio environment may be an ISM ratio.

[0055] According to a fifth aspect, there is provided an apparatus for encoding audio object parameters, the apparatus comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause a system to at least: identify one of at least two audio objects in an audio environment to be encoded separately; obtain, for a time-frequency element of a frame comprising at least one time element and at least one frequency element, ratio parameters of the at least two audio objects, the ratio parameters being configured to identify a distribution of the particular audio object within an object portion of the overall audio environment and for the particular time-frequency element; grouping, for the time-frequency element, ratio parameters associated with the at least two audio objects; and quantizing the ratio parameters within the group, the quantization of the ratio parameters being configured to identify a distribution of the particular group. the means for encoding the subsequent group further comprises: selecting a subset of the integer representations of the ratio parameter values ​​for the subsequent group; and encoding the selected subset of the integer representations of the ratio parameter values ​​for the subsequent group as difference values ​​relative to the first group or first set of groups. The means for causing the encoding and grouping to occur orders the ratio parameters within the group such that the selected subset of the integer representations of the ratio parameter values ​​includes integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded.

[0056] An apparatus that performs selecting a subset of the integer representations of the ratio parameter values ​​of a subsequent group may perform selecting all but one of the integer representations of a particular group, since the integer representations of the ratio parameter values ​​of the group are values ​​defined for all groups.

[0057] An apparatus that performs encoding a selected subset of integer representations of ratio parameter values ​​of a subsequent group as difference values ​​relative to a first group or first set of groups may perform the following: determining that the integer representations of ratio parameter values ​​corresponding to one of at least two audio objects in an audio environment to be encoded separately are zero for all frequency elements of a first time element; and encoding, for a particular group, all but two of the integer representations, wherein the encoding does not include the integer representation of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be encoded separately if the integer representations of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be encoded separately are zero for all frequency elements of the first time element.

[0058] An apparatus that performs selecting a subset of integer representations of ratio parameter values ​​for a subsequent group may perform the following: determining that integer representations of ratio parameter values ​​corresponding to one of at least two audio objects in the audio environment to be separately encoded are zero for all frequency elements of a first time element; and selecting, for the particular group, all but two of the integer representations, wherein the selecting does not include the integer representations of ratio parameter values ​​corresponding to one of the at least two audio objects in the audio environment to be separately encoded.

[0059] An apparatus that performs encoding integer representations of ratio parameter values ​​in a first group or a first set of groups as enumeration indexes may perform generating means for generating integer values ​​based on indexing from the groups, the generated integer values ​​representing ratio parameters of the audio objects.

[0060] An apparatus that performs quantizing of ratio parameters in a group, where the quantization of the ratio parameters is configured to generate integer representations of ratio parameter values ​​that sum to a defined integer value for a particular group, may perform the following steps: generating a single value by adding elements from the group of ratio parameters; and generating an integer representation from the single value by executing an iteration loop from the zeroth iteration up to a single iteration number, and sequentially associating index values ​​with iteration numbers of the iteration loop having valid selections of ratio parameters, where the integer value is the highest index value.

[0061] An apparatus that performs quantizing ratio parameters in a group may perform the following: quantizing the ratio values ​​in a particular selection using minimum nearest neighbor scalar quantization to obtain quantization index values; calculating a reconstructed value of the ratio parameter for the particular selection; calculating an error value based on a difference between the reconstructed ratio value and the particular selection of ratio parameter value; determining a sum of the quantization index values; and selecting at least one quantization index value to increment so that the sum of the quantization index values ​​equals the sum of the expected index values.

[0062] An apparatus that performs selecting at least one quantization index value to increment so that a sum of quantization index values ​​equals a sum of expected index values ​​may perform selecting the at least one quantization index value to increment based on identifying a maximum decrease in an error value when the index value is incremented, or selecting the at least one quantization index value to increment based on identifying a minimum increase in the error value when the index value is incremented.

[0063] An apparatus that performs encoding a selected subset of integer representations of ratio parameter values ​​of a subsequent group as difference values ​​for a first group or a first set of groups may perform, for a selected subset of integer representations of ratio parameter values ​​of a subsequent group for a particular time element of a frame, determining, for a first entropy coding parameter and a second entropy coding parameter, a number of bits required for entropy coding differences between quantized frequency elements; determining, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required for entropy coding differences between quantized time elements; selecting, for the particular time element, the first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding differences within the particular time element of the frame; and selecting one of entropy coding of differences between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding differences within the particular time element of the frame.

[0064] An apparatus that performs encoding a selected subset of integer representations of ratio parameter values ​​of a subsequent group as difference values ​​relative to a first group or first set of groups may perform encoding a selected one of entropy coding of differences between frequency elements or time elements for entropy coding based on a selected first entropy coding parameter or second entropy coding parameter.

[0065] An apparatus that performs encoding a selected subset of integer representations of ratio parameter values ​​of a subsequent group as difference values ​​for a first group or a first set of groups may perform, for a selected set of ratio parameters for a particular frequency element of a frame, determining, for a first entropy coding parameter and a second entropy coding parameter, a number of bits required for entropy coding of quantized differences between frequency elements; determining, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required for entropy coding of quantized differences between time elements; selecting, for the particular frequency element, the first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding differences within the particular time element of the frame; and selecting, for entropy coding based on the selected first entropy coding parameter or the second entropy coding parameter based on the smaller number of bits required for coding differences within the particular time element of the frame for the particular frequency element.

[0066] An apparatus that performs encoding a selected subset of integer representations of ratio parameter values ​​of a subsequent group as difference values ​​relative to a first group or first set of groups may perform encoding a selected one of entropy coding of differences between frequency elements or time elements for entropy coding based on a selected first entropy coding parameter or second entropy coding parameter.

[0067] An apparatus that performs encoding a selected subset of integer representations of ratio parameter values ​​of a subsequent group as difference values ​​relative to a first group or first set of groups may perform the following: generating an indicator indicating a selected first entropy coding parameter or a second entropy coding parameter; and generating an indicator indicating a selected one of entropy coding of differences between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the second entropy coding parameter.

[0068] The entropy coding may be Golomb-Rice entropy coding, the first entropy coding parameter may be Golomb-Rice entropy coding order 0, and the second entropy coding parameter may be Golomb-Rice entropy coding order 1.

[0069] An apparatus that performs encoding a selected subset of integer representations of ratio parameter values ​​of a subsequent group as differential values ​​relative to a first group or set of groups may perform differential encoding of a selection of ratio parameters based on a leading-indexed time element group of the ratio parameter in the absence of a leading-indexed frequency element group of the ratio parameter.

[0070] The ratio parameter configured to identify the distribution of a particular object within the object portion of the overall audio environment may be an ISM ratio.

[0071] The apparatus that performs selecting a subset of integer representations of ratio parameter values ​​of a subsequent group may further perform: determining that integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded are substantially zero for all frequency elements of the first group or first set of groups; and encoding only the remaining selected subset of integer representations of ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups.

[0072] The first group may be a first group of time and frequency elements of the group, and the first group set is a group of frequency element groups associated with the first time element.

[0073] The apparatus may further perform separately encoding the identified one of the at least two audio objects.

[0074] According to a sixth aspect, there is provided an apparatus for decoding audio object parameters, the apparatus comprising: at least one processor; and at least one memory storing instructions that when executed by the at least one processor cause a system to obtain a bitstream comprising encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment comprising more than two audio objects, the ratio parameters being particular to object portions within the overall audio environment and for particular time-frequency elements. and identifying that one of at least two audio objects in the audio environment is separately encoded; decoding an enumeration index within the first group or first set of groups to generate a decoded integer representation of a ratio parameter value for the first group or first set of groups; and decoding a subsequent group as a difference value relative to the first group or first set of groups, wherein the means for decoding the subsequent group further determines, for the subsequent group, a further ratio parameter value based on a difference between the defined total ratio value and a sum of the ratio values ​​within the group.

[0075] The apparatus that performs decoding the subsequent group as a difference value relative to the first group or first set of groups may further perform: determining that a ratio parameter value corresponding to one of the at least two audio objects in the separately encoded audio environment is zero for all frequency elements of the first time element; and determining, for the subsequent group, an additional ratio parameter value having a value of 0 that represents one of the at least two audio objects in the separately encoded audio environment.

[0076] A group may be a vector of ratio parameters.

[0077] An apparatus that performs decoding the enumeration index within the first group or first set of groups to generate a decoded integer representation of a ratio parameter value for the first group or first set of groups may perform the following: obtain an integer value representing the encoded ratio parameter; convert the integer value to a selection of ratio parameters based on the indexing of the vector; and regenerate at least one further ratio parameter from the selection of ratio parameters.

[0078] An apparatus that performs decoding the subsequent group as a differential value relative to the first group or first set of groups may further perform obtaining a differential indicator that identifies frequency differential or time differential encoding, obtaining an entropy encoding indicator that identifies an entropy encoding parameter, and decoding a remaining selection of ratio parameters for the frame based on the differential indicator and the entropy encoding indicator.

[0079] The ratio parameter configured to identify the distribution of a particular object within the object portion of the overall audio environment may be an ISM ratio.

[0080] According to a seventh aspect, there is provided an apparatus for encoding audio object parameters, the apparatus comprising: an identification circuit configured to identify one of at least two audio objects in an audio environment to be encoded separately; an acquisition circuit configured to acquire ratio parameters of the at least two audio objects for a time-frequency element of a frame comprising at least one time element and at least one frequency element, the ratio parameters being configured to identify a distribution of the particular audio object within an object portion of the overall audio environment and for the particular time-frequency element; a grouping circuit configured to group ratio parameters associated with the at least two audio objects for the time-frequency element; and quantizing the ratio parameters within the group, the quantization of the ratio parameters being configured to generate integer representations of ratio parameter values ​​that sum up to a defined integer value for the particular group. and an encoding circuit configured to perform the quantization; and an encoding circuit configured to encode integer representations of the ratio parameter values ​​in the first group or first set of groups as enumeration indices; and an encoding circuit configured to encode subsequent groups as difference values ​​relative to the first group or first set of groups, wherein the means for encoding the subsequent groups further comprises a selection circuit configured to select a subset of the integer representations of the ratio parameter values ​​of the subsequent group; and an encoding circuit configured to encode the selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or first set of groups, wherein the grouping orders the ratio parameters in the group such that the selected subset of the integer representations of the ratio parameter values ​​includes integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded.

[0081] According to an eighth aspect, there is provided an apparatus for decoding audio object parameters, the apparatus comprising: an acquisition circuit configured to acquire a bitstream comprising encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment comprising more than two audio objects, the ratio parameters being configured to identify a distribution of particular objects within object portions of the overall audio environment and with respect to particular time-frequency elements; and an acquisition circuit configured to acquire a bitstream comprising encoded integer representations of ratio parameters for at least two audio objects in the audio environment. The audio encoding system includes an identification circuit configured to identify that one of the audio objects is separately encoded; a decoding circuit configured to decode an enumeration index within the first group or first set of groups to generate a decoded integer representation of a ratio parameter value for the first group or first set of groups; and a decoding circuit configured to decode a subsequent group as a difference value relative to the first group or first set of groups, wherein decoding the subsequent group further includes determining, for the subsequent group, a further ratio parameter value based on a difference between the defined total ratio value and a sum of the ratio values ​​within the group.

[0082] According to a ninth aspect, there is provided a computer program comprising instructions (or a computer readable medium comprising program instructions) that cause an apparatus for encoding audio object parameters to at least: identify one of at least two audio objects in an audio environment to be encoded separately; obtain, for a time-frequency element of a frame comprising at least one time element and at least one frequency element, ratio parameters of the at least two audio objects, the ratio parameters being configured to identify a distribution of the particular audio object within an object portion of the overall audio environment and for the particular time-frequency element; group, for the time-frequency element, ratio parameters associated with the at least two audio objects; and quantize the ratio parameters within the group, the quantization of the ratio parameters being summed for the particular group. the encoding step is configured to generate integer representations of the ratio parameter values ​​that result in an integer value defined as a quantization step; encoding the integer representations of the ratio parameter values ​​in the first group or the first set of groups as enumeration indices; and encoding subsequent groups as difference values ​​relative to the first group or the first set of groups, wherein encoding the subsequent groups further comprises selecting a subset of the integer representations of the ratio parameter values ​​for the subsequent groups; and encoding the selected subset of the integer representations of the ratio parameter values ​​for the subsequent groups as difference values ​​relative to the first group or the first set of groups; and the grouping step comprises ordering the ratio parameters in the groups such that the selected subset of the integer representations of the ratio parameter values ​​includes integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded.

[0083] According to a tenth aspect, there is provided a computer program (or a computer readable medium comprising program instructions) comprising instructions to cause an apparatus for decoding audio object parameters to at least obtain a bitstream comprising encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment comprising more than two audio objects, the ratio parameters being proportional to the proportions of particular objects within an object portion of the overall audio environment and for particular time-frequency elements. The audio processing system performs the steps of: acquiring, configured to identify a fabric; identifying that one of at least two audio objects in the audio environment is separately encoded; decoding an enumeration index in a first group or a first set of groups to generate a decoded integer representation of a ratio parameter value for the first group or the first set of groups; and decoding a subsequent group as a difference value relative to the first group or the first set of groups, wherein decoding the subsequent group further includes determining, for the subsequent group, a further ratio parameter value based on a difference between the defined total ratio value and a sum of the ratio values ​​in the group.

[0084] According to an eleventh aspect, there is provided a non-transitory computer-readable medium comprising program instructions that cause an apparatus for encoding audio object parameters to at least: identify one of at least two audio objects in an audio environment to be encoded separately; obtain, for a time-frequency element of a frame comprising at least one time element and at least one frequency element, ratio parameters of the at least two audio objects, the ratio parameters being configured to identify a distribution of the particular audio object within an object portion of the overall audio environment and for the particular time-frequency element; group, for the time-frequency element, ratio parameters associated with the at least two audio objects; and quantize the ratio parameters within the group, the quantization of the ratio parameters being defined as summing for the particular group. the encoding step is configured to generate integer representations of the ratio parameter values ​​that result in a selected integer value; encoding the integer representations of the ratio parameter values ​​in the first group or first set of groups as enumeration indices; and encoding subsequent groups as difference values ​​relative to the first group or first set of groups, wherein encoding the subsequent groups further comprises selecting a subset of the integer representations of the ratio parameter values ​​for the subsequent groups; and encoding the selected subset of the integer representations of the ratio parameter values ​​for the subsequent groups as difference values ​​relative to the first group or first set of groups; and the grouping step comprises ordering the ratio parameters in the groups such that the selected subset of the integer representations of the ratio parameter values ​​includes integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded.

[0085] According to a twelfth aspect, there is provided a non-transitory computer readable medium comprising program instructions that cause an apparatus for decoding audio object parameters to at least obtain a bitstream comprising encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment comprising more than two audio objects, and the ratio parameters identifying a distribution of particular objects within object portions of the overall audio environment and for particular time-frequency elements. and decoding a subsequent group as a difference value relative to the first group or first set of groups, wherein decoding the subsequent group further includes determining, for the subsequent group, a further ratio parameter value based on a difference between the defined total ratio value and a sum of the ratio values ​​in the group.

[0086] According to a thirteenth aspect, there is provided an apparatus for encoding audio object parameters, comprising: means for identifying one of at least two audio objects in an audio environment to be encoded separately; means for obtaining, for a time-frequency element of a frame comprising at least one time element and at least one frequency element, ratio parameters of the at least two audio objects, the ratio parameters being configured to identify a distribution of the particular audio object within an object portion of the overall audio environment and for the particular time-frequency element; means for grouping, for the time-frequency element, ratio parameters associated with the at least two audio objects; and means for quantizing the ratio parameters within the group, wherein the quantization of the ratio parameters generates integer representations of ratio parameter values ​​that sum up for the particular group to a defined integer value. and means for encoding a subsequent group as a differential value relative to the first group or first set of groups, wherein the means for encoding the subsequent group further comprises: means for selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group; and means for encoding the selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as a differential value relative to the first group or first set of groups, and the means for grouping includes means for ordering the ratio parameters within the group such that the selected subset of the integer representations of the ratio parameter values ​​comprises integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded.

[0087] According to a fourteenth aspect, there is provided an apparatus for decoding audio object parameters, the apparatus comprising: obtaining a bitstream including encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment including more than two audio objects, and the ratio parameters being configured to identify a distribution of the particular object within an object portion of the overall audio environment and with respect to the particular time-frequency element; means for identifying that one of the at least two audio objects in the audio environment is separately encoded; means for decoding an enumeration index in a first group or a first set of groups to generate a decoded integer representation of a ratio parameter value for the first group or the first set of groups; and means for decoding a subsequent group as a difference value with respect to the first group or the first set of groups, the means for decoding the subsequent group further comprising means for determining, for the subsequent group, a further ratio parameter value based on a difference between a defined total ratio value and a sum of the ratio values ​​in the group.

[0088] According to a fifteenth aspect, there is provided a computer-readable medium comprising program instructions that cause an apparatus for encoding audio object parameters to at least: identify one of at least two audio objects in an audio environment to be encoded separately; obtain, for a time-frequency element of a frame comprising at least one time element and at least one frequency element, ratio parameters of the at least two audio objects, the ratio parameters being configured to identify a distribution of the particular audio object within an object portion of the overall audio environment and for the particular time-frequency element; group, for the time-frequency element, ratio parameters associated with the at least two audio objects; and quantize the ratio parameters within the group, the quantization of the ratio parameters being defined as summing for the particular group. the encoding includes: quantizing, configured to generate integer representations of the ratio parameter values ​​that result in integer values; encoding the integer representations of the ratio parameter values ​​in the first group or the first set of groups as enumeration indices; and encoding subsequent groups as difference values ​​relative to the first group or the first set of groups, wherein encoding the subsequent groups further includes selecting a subset of the integer representations of the ratio parameter values ​​for the subsequent groups; and encoding the selected subset of the integer representations of the ratio parameter values ​​for the subsequent groups as difference values ​​relative to the first group or the first set of groups; and the grouping includes ordering the ratio parameters in the groups such that the selected subset of the integer representations of the ratio parameter values ​​includes integer representations of ratio parameter values ​​associated with a selected one of the at least two audio objects to be separately encoded.

[0089] According to a sixteenth aspect, there is provided a computer readable medium comprising program instructions that cause an apparatus for decoding audio object parameters to at least obtain a bitstream comprising encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment comprising more than two audio objects, the ratio parameters being adapted to identify a distribution of particular objects within object portions of the overall audio environment and for particular time-frequency elements. identifying that one of at least two audio objects in the audio environment is separately encoded; decoding an enumeration index in a first group or a first set of groups to generate a decoded integer representation of a ratio parameter value for the first group or the first set of groups; and decoding a subsequent group as a difference value relative to the first group or the first set of groups, wherein decoding the subsequent group further includes determining, for the subsequent group, a further ratio parameter value based on a difference between a defined total ratio value and a sum of the ratio values ​​in the group.

[0090] An apparatus comprising means for performing the actions of the method as described above.

[0091] An apparatus configured to perform the actions of the method as described above.

[0092] A computer program comprising program instructions for causing a computer to carry out the method as described above.

[0093] A computer program product stored on the medium may cause an apparatus to perform the methods described herein.

[0094] The electronic device may include the apparatus described herein.

[0095] The chipset may include the devices described herein.

[0096] SUMMARY OF THE INVENTION Embodiments of the present application aim to address problems associated with the current state of the art.

[0097] For a better understanding of the present application, reference will now be made, by way of example, to the accompanying drawings, in which: [Brief explanation of the drawings]

[0098] [Figure 1] 1 illustrates a schematic diagram of a system of apparatus suitable for implementing some embodiments. [Figure 2] 2 illustrates a schematic diagram of an exemplary encoding mode selector shown in the system of the device illustrated in FIG. 1, according to some embodiments. [Figure 3] 3 illustrates a flow diagram of the operation of the exemplary encoding mode selector shown in FIG. 2, according to some embodiments. [Figure 4] 4 illustrates a flow diagram of the operation of the exemplary first, lowest, or MASA-only bitrate encoding mode shown in FIG. 3, according to some embodiments. [Figure 5] 4 illustrates a flow diagram of the operation of the exemplary second, lower, or object information encoding mode shown in FIG. 3, according to some embodiments. [Figure 6] 4 illustrates a flow diagram of the exemplary third, higher, or single object encoding mode of operation shown in FIG. 3, according to some embodiments. [Figure 7] 4 illustrates a flow diagram of the operation of the exemplary fourth, top, or independent object and multiple input encoding mode shown in FIG. 3 , according to some embodiments. [Figure 8] 2 illustrates a schematic diagram of the exemplary audio object metadata encoder shown in FIG. 1, according to some embodiments. [Figure 9] 9 illustrates a flow diagram of the operation of the exemplary audio object analyzer and audio object metadata encoder encoding mode selector shown in FIG. 8, according to some embodiments. [Figure 10] 1 illustrates an exemplary device suitable for implementing the apparatus shown in the above figures. DETAILED DESCRIPTION OF THE INVENTION

[0099] In the following, suitable devices and possible mechanisms for encoding a parametric spatial audio signal including a transport audio signal and spatial metadata are described in more detail. As mentioned above, immersive audio codecs (such as 3GPP IVAS) are planned that support a number of operating points, ranging from low-bitrate operation to transparency. They are expected to support channel-based, object-based, and scene-based audio inputs that include spatial information about the sound field and sound sources. In the following, an exemplary codec is configured to receive multiple input formats. In particular, the codec is configured to acquire or receive multi-audio signals (e.g., received from a microphone array, or as multi-channel audio format input, or Ambisonics format input) and audio object signals (sometimes referred to as Independent Stream with Metadata (ISM) format). Furthermore, in some situations, the codec is configured to process multiple input formats at once. This combined (input) format mode can, for example, enable simultaneous encoding of two different audio input formats. An example of two different audio input formats currently considered is a MASA format combined with an audio object format.

[0100] Metadata-Assisted Spatial Audio (MASA) is an example of a parametric spatial audio format and representation that is suitable as an input format for IVAS.

[0101] MASA can be seen as an audio representation consisting of "N channels + spatial metadata". MASA is a scene-based audio format particularly suited for spatial audio capture on practical devices such as smartphones. The idea is to describe a sound scene in terms of the direction and, for example, energy ratio of time- and frequency-varying sound sources. Sound energy not specified (described) by direction is described as diffuse (coming from all directions).

[0102] As mentioned above, spatial metadata associated with an audio signal may include multiple parameters per time-frequency tile (multiple directions and a direct-to-total energy ratio, spread coherence, distance, etc. associated with each direction (or direction value)). The spatial metadata may also include other parameters, or may be associated with other parameters that are considered omnidirectional (such as surround coherence, diffuse-to-total energy ratio, residual-to-total energy ratio, etc.), but which, when combined with the directional parameters, can be used to define characteristics of the audio scene. For example, a reasonable design choice that can produce quality output is one in which the spatial metadata includes determining one or more directions per time-frequency subframe (and a direct-to-total ratio, spread coherence, distance value, etc. associated with each direction).

[0103] Examples of MASA spatial metadata are shown in the table below. These values ​​can be used for each time-frequency tile. In some implementations, the frame is subdivided into 24 frequency bands and four time subframes. Other implementations may use other divisions of frequency and time. Furthermore, in some implementations, the frame size (e.g., as implemented in IVAS) is 20 ms (and thus the time subframes are 5 ms). However, similarly, other embodiments may employ other frame lengths. In some embodiments, the MASA analyzer is configured to determine one or two directions for each time-frequency tile (i.e., there are one or two direction indices, direct-to-total energy ratios, and spread coherence parameters for each time-frequency tile). However, in some embodiments, the analyzer is configured to generate more than two directions for a time-frequency tile. [Table 1-1] [Table 1-2]

[0104] The MASA stream can be rendered to a variety of outputs, such as a multi-channel loudspeaker signal (eg, 5.1) or a binaural signal.

[0105] Another input format supported by IVAS is the ISM (Independent Stream with Metadata) format, as mentioned above. The ISM format is intended to represent individual sound sources (audio objects) within a sound scene, along with associated metadata that describes how the rendering of the audio signal should be implemented. This contrasts with the MASA format, where the audio signal and associated metadata perceptually describe the entire sound scene. Examples of metadata in the ISM format might include the following parameters: Position (e.g. azimuth, altitude or azimuth, altitude and radius), direction, Range / extent, Distance attenuation, and Directivity pattern.

[0106] In some embodiments, some example metadata parameters, such as position, orientation, and range / spread, vary with time but do not vary with frequency, while some parameters, such as distance attenuation and directional pattern, vary with frequency but typically do not vary with time, although parameters may have time and frequency variability (or invariance) beyond the examples given above and herein.

[0107] In some embodiments, ISM-formatted input can be obtained by capturing individual sources in a scene, for example, using close-up or lavalier microphones (located at or near the individual sources). Examples of such individual sources include individual speakers, singers, or individual instruments in a teleconference. These ISM-formatted signals can then be associated with metadata automatically, for example, by a position tracker, or manually by a mixing expert. Alternatively, mixing can create the appropriate associated metadata and generate the ISM-formatted signals. An example of a fully generated audio source is in a game audio engine.

[0108] As mentioned above, research is underway to enable the IVAS codec to support joint coding of multiple audio formats.

[0109] The combined input format mode allows simultaneous encoding of two different audio input formats. The first example we consider is the combination of a MASA format with an audio object (i.e., an independent mono stream (ISM) with metadata / independent mono stream without metadata), which is the topic of the next example. To transmit this combined format at moderate or intermediate bitrates, there is a parametric model involving ISM energy ratios, which define the proportion of the audio scene created by each object within the audio scene created by all objects. For each time-frequency tile, there are groups of O such values, where O is the number of objects in the scene. Since such values ​​can be very large (e.g., 20xO), it is important to encode them efficiently.

[0110] Therefore, in the example described here we consider the case where one object of greatest importance is coded separately and the remaining objects are encoded in parametric mode.

[0111] Current solutions propose an informed transformation of the ISM ratio index vector before coding for isolated single object coding modes. The transformation is informed by the index of the isolated object. In some examples, embodiments have been shown to reduce the required bitrate by up to 15%.

[0112] In this regard, Figure 1 illustrates an exemplary apparatus 100 and system for implementing embodiments of the present application. The system is shown to include an analyzer / encoder 101. The analyzer / encoder 101 is the part that receives the input signal and transmits or stores the encoded bitstream (including the transport audio signal and metadata).

[0113] The input to the system analyzer / encoder 101 is a multi-channel audio signal 102. In the following example, a microphone channel signal input is described, but in other embodiments, any suitable input (or composite multi-channel) format may be implemented. For example, in some embodiments, the spatial analyzer and spatial analysis may be implemented external to the encoder. For example, in some embodiments, spatial (MASA) metadata associated with the audio signal may be provided to the encoder as a separate bitstream. In some embodiments, the spatial (MASA) metadata may be provided as a set of spatial (directional) index values.

[0114] 1 also shows a number of audio objects 104 as further inputs to the analysis portion. As mentioned above, these multiple audio objects (or audio object streams) 104 may represent various sound sources in physical space. Each audio object may be characterized by an audio (object) signal and accompanying metadata including directional data (in the form of azimuth and altitude values) that indicate the location or direction of the audio object in physical space for each audio frame.

[0115] The multi-channel signal 102 is passed to an analyzer and encoder 101 , specifically to a transport signal generator 105 and a metadata generator 103 .

[0116] In some embodiments, the metadata generator 103 is also configured to receive the multi-channel signal and analyze the signal in order to generate metadata 104 associated with the multi-channel signal and, therefore, associated with the transport signal 106. The analysis processor 103 may be configured to generate, for each time-frequency analysis interval, metadata that may include a direction parameter, an energy ratio parameter, and a coherence parameter (and, in some embodiments, a diffuseness parameter). The direction, energy ratio, and coherence parameters may, in some embodiments, be considered to be MASA spatial audio parameters (or MASA metadata). In other words, spatial audio parameters include parameters that aim to characterize the sound field created / captured by the multi-channel signal (or two or more audio signals in general).

[0117] In some embodiments, the generated parameters may differ for each frequency band. Thus, for example, in band X, all of the parameters are generated and transmitted, while in band Y, only one of the parameters is generated and transmitted, and further, in band Z, no parameters are generated or transmitted. A practical example of this may be that in some frequency bands, such as the highest band, some of the parameters are not needed for perceptual reasons. The transport signal 106 and metadata 104 may be passed to a joint encoder core 109.

[0118] In some embodiments, transport signal generator 105 is configured to receive the multi-channel signal, generate an appropriate transport signal including the determined number of channels, and output transport signal 106 (MASA transport audio signal). For example, transport signal generator 105 may be configured to generate a two-audio-channel downmix of the multi-channel signal. The determined number of channels may be any appropriate number of channels. In some embodiments, transport signal generator is configured to select or combine input audio signals for the determined number of channels in other ways, such as by beamforming techniques, and output them as a transport signal.

[0119] In some embodiments, the transport signal generator 105 is optional and the multi-channel signal is passed to the combined encoder core 109 unprocessed, in this example in the same way as the transport signal.

[0120] The audio objects 104 may be passed to an audio object analyzer 107 for processing. In some embodiments, the audio object analyzer 107 analyzes the audio object input stream 104 to generate an appropriate audio object transport signal and audio object metadata. For example, the audio object analyzer may be configured to generate an audio object transport signal by combining and downmixing the audio signals of the audio objects to stereo channels using amplitude panning based on the associated audio object direction. Furthermore, the audio object analyzer may also be configured to generate audio object metadata associated with the audio object input stream 104. The audio object metadata may include direction values ​​applicable to all subbands. Thus, if four objects are present, there are four directions. In the example described herein, the direction values ​​also apply across all of the subframes of a frame, but in some embodiments, the temporal resolution of the direction values ​​can be different, and the direction values ​​apply to one or more subframes of a frame. Furthermore, an energy ratio (or ISM ratio) may be determined for each object. The energy ratio (ISM ratio) defines the contribution of an object within the object portion of the overall audio environment. In the example below, the energy ratio (or ISM ratio) is for each time-frequency tile for each object.

[0121] The acquisition of ISM ratios is presented, for example, in WO2022 / 200666. For example, they can be acquired as follows:

[0122] First, the object audio signal s obj (t,i) is the time-frequency domain S obj(b,n,i), where t is the time sample index, b is the frequency bin index, n is the time frame (or subframe or slot) index, and i is the object index. The time-frequency domain signal can be obtained, for example, via a short-time Fourier transform (STFT) or a complex-modulated quadrature filter bank (QMF) (or their low-delay variants).

[0123] The energy of the object is then calculated in the frequency band as follows:

number

number

[0124] In some embodiments, the temporal resolution of the ISM ratio is determined by the time-frequency domain audio signal S obj The temporal resolution of (b,n,i) may differ (i.e., the temporal resolution of the spatial metadata may differ from the temporal resolution of the time-frequency transform). In such cases, the calculation (of the energy and / or ISM ratios) may involve summation over multiple time frames of the time-frequency domain audio signals and / or energy values.

[0125] ISM ratios are numbers between 0 and 1, and correspond to the proportion of the audio scene created by all objects that an object is active in. For each object, there is one ISM ratio per frequency subband and time subframe. By definition, the ratios across all objects sum to 1 for each subband and subframe.

[0126] In some embodiments, the audio object analyzer 107 may be located elsewhere and the audio objects 104 input to the analyzer and encoder 101 are audio object transport signals and audio object metadata.

[0127] The analyzer and encoder 101 may include a combined encoder core 109 configured to receive a transport audio (e.g., downmix) signal 106 and an audio object transport signal 128 and to generate appropriate encodings of these audio signals.

[0128] The analyzer and encoder 101 may also include an audio object metadata encoder 111 , which is also configured to receive the audio object metadata 108 and output an encoded or compressed form of the input information as encoded audio object metadata 112 .

[0129] In some embodiments, the combined encoder core 109 may be configured to implement a stream separation metadata determiner and encoder, which may be configured to determine the relative contributions of the multi-channel signal 102 (sometimes known as a MASA audio signal) and the audio objects 104 to an overall audio scene. While the following examples describe combining a multi-channel audio signal and audio objects, in some embodiments, the multi-channel audio signal may be generalized as a spatial audio signal. This measure of proportionality generated by the stream separation metadata determiner and encoder may be used to determine the proportion of quantization and encoding “effort” spent on the input multi-channel signal 102 and audio objects 104. In other words, the stream separation metadata determiner and encoder may generate a metric that quantifies the proportion of encoding effort spent on the multi-channel audio signal 102 compared to the encoding effort spent on the audio objects 104. This metric may be used to drive the encoding of the audio object metadata 108 and the metadata 104. Furthermore, the metrics determined by the separation metadata determiner and encoder may also be used as influencing factors in the process of encoding the transport audio signal 106 and the audio object transport audio signal 128 performed by the combined encoder core 109. The output metrics from the stream separation metadata determiner and encoder may further be represented as encoded stream separation metadata and may be combined into the encoded metadata stream from the combined encoder core 109.

[0130] In some embodiments, the analyzer and encoder 101 includes a bitstream generator 113 configured to obtain the encoded metadata 116, the encoded transport audio signal 138, and the encoded audio object metadata 112 and generate a bitstream 118 for potential transmission or storage.

[0131] In some embodiments, the analyzer and encoder 101 includes an encoder controller 115. The encoder controller 115, in some embodiments, can control the encoding implemented by the audio object metadata encoder 111 and the combined encoder core 109. In some embodiments, the encoder controller 115 is configured to determine a bit rate for the bit stream 118 and control the encoding based on the bit rate. In some embodiments, the encoder controller 115 is further configured to control at least one of the audio object analyzer 107, the transport signal generator 105, and the metadata generator in generating parameters.

[0132] Analyzer and Encoder 101, in some embodiments, can be a computer or mobile device (executing appropriate software stored in memory and at least one processor), or alternatively, a specialized device utilizing, for example, an FPGA or ASIC. Encoding can be implemented using any suitable scheme. In some embodiments, Encoder 107 can further interleave, multiplex into a single data stream, or embed the encoded MASA metadata, audio object metadata, and stream separation metadata within the encoded (downmixed) transport audio signal prior to transmission or storage, as indicated by the dashed lines in FIG. 1. Multiplexing can be implemented using any suitable scheme.

[0133] 1, an associated decoder and renderer 109 is shown, configured to take the bitstream 118 containing the encoded metadata 116, the encoded transport audio signal 138, and the encoded audio object metadata 112, and to generate therefrom an appropriate spatial audio output signal. The decoding and processing of such audio signals is known in principle and will not be described in detail hereinafter, other than the decoding of the encoded ISM ratio metadata.

[0134] With reference to FIG. 2, the encoder controller 115 according to some embodiments is shown in further detail.

[0135] In this example, encoder controller 115 includes a bitrate determiner / monitor 201 configured to determine and / or monitor the available bitrate for the encoded audio and metadata bandwidth, which may be determined based on transmission path bandwidth estimates (and, for example, based on estimated signal strength), based on bandwidth storage decisions to keep files below a required size for a determined duration, or by any suitable method.

[0136] Additionally, in some embodiments, the encoder controller 115 comprises an object monitor 202 configured to determine the number of objects and pass this information to the encoding mode selector 203 .

[0137] Furthermore, the bitrate determiner / monitor 201 and the object monitor 202 may be configured to control the encoding mode selector 203 .

[0138] The encoder controller 115 may include an encoding mode selector 203, which may be configured to select an encoding mode, for example based on a determined bandwidth or bit rate, and then control the encoders, for example the combined encoder core 109 and the audio object metadata encoder 111.

[0139] With reference to Figure 3, there is shown a flow diagram of an exemplary operation of the encoder controller shown in Figure 2. In this example, there is an initial operation of receiving, obtaining, or otherwise determining the encoded parameters and the bit rate or bandwidth of the audio data, as well as the number of objects, as shown in Figure 3 at step 301.

[0140] Once the available bandwidth or bitrate and number of objects are obtained, one of several encoding modes can be determined based on the bandwidth / bitrate and number of objects, as shown in step 303 of Figure 3.

[0141] In some embodiments, the mode selection can be based on the following table: [Table 2]

[0142] Mode A is a mode in which the encoder can be controlled to encode only the transport channel and MASA metadata, as shown in step 304 of FIG.

[0143] Mode B is a mode in which the encoder can be controlled to encode the transport channel, MASA metadata, ISM metadata (all objects), MASA to total ratio, and ISM ratio, as shown in step 306 of FIG.

[0144] Mode C is a mode in which the encoder can be controlled to encode transport channels, MASA metadata, ISM metadata (all objects), MASA to total ratio, ISM ratio, and one object audio data together with one object identifier, as shown in step 308 of Figure 3.

[0145] Mode D is a mode in which the encoder can be controlled to encode the transport channel, MASA metadata, and ISM metadata (all objects), as shown in step 310 of FIG.

[0146] It will be understood that the bitrate / number of objects based mode selections shown herein are examples and may be other specific values.

[0147] For example, Figure 4 shows in more detail the Mode A encoding method, the first (or lowest or combined) encoding mode shown in step 304 of Figure 3. Thus, for the lowest overall bitrate, all encoding is performed using the MASA representation.

[0148] Thus, for example, as indicated by step 401 in FIG. 4, there are operations to receive / acquire object-based streams (independent streams with metadata) and multi-channel-based (MASA stream) transport audio signals and metadata.

[0149] Next, there is the operation of generating an object-based MASA stream from the object stream (independent stream with metadata), as indicated by step 403 in Figure 4. In some embodiments, this object-based MASA stream can be created from the object stream using, for example, the methods presented in WO2019086757A1.

[0150] The object-based MASA stream and the multi-channel-based MASA stream are then combined, as shown by step 405 in Figure 4. In some embodiments, the original MASA stream and the MASA stream created from the objects can be combined using the methods set out in GB2574238. The decoder obtains the objects and the MASA audio content in MASA format.

[0151] The combined stream is then output, as indicated by step 407 in Figure 4. In such an embodiment, the object audio content is present in the decoded audio scene (along with the MASA audio content), but the object cannot be edited or separated from the scene at the decoder.

[0152] Figure 5 illustrates the Mode B encoding method, the second (or lower or object metadata) encoding mode, as indicated by step 306 in Figure 3. Since the number of available bits is higher compared to the Mode A encoding method (for the same number of objects), there is a possibility to parameterize the audio scene by transmitting one common audio data downmix, MASA metadata, ISM metadata, and for each time-frequency tile an additional set of parameters indicating the amount of signal corresponding to MASA components outside the total audio scene (in other words, this can be represented or indicated by a MASA to total energy ratio) and a ratio indicating how the audio scene components corresponding to the objects are distributed among the ISMs (in other words, this can be represented or indicated by an ISM ratio).

[0153] Thus, there are method steps for receiving / obtaining object-based streams (independent streams with metadata) and multi-channel-based (MASA stream) transport audio signals and metadata, for example as indicated by step 501 in FIG.

[0154] Next, a combined MASA and object-based downmix (channel-to-element) audio signal is generated, as shown by step 503 in Figure 5. In other words, the audio content of the MASA and objects is downmixed to two channels (channel-to-element CPE).

[0155] The MASA to total ratio and ISM ratio may be determined as shown by step 505 in FIG.

[0156] The MASA-to-sum ratio and the ISM ratio may then be encoded based on any suitable encoding method. For example, the ISM ratio may be encoded using a lattice encoding method, or the MASA-to-sum ratio may be encoded by entropy coding after a DCT transform (e.g., as described in WO2022 / 200666). The encoding of the MASA-to-sum ratio and the ISM ratio is illustrated by step 507 in FIG. 5.

[0157] The further MASA metadata may then be encoded based on any suitable MASA metadata encoding method, as indicated by step 509 of FIG.

[0158] The combined audio signal may then be encoded based on any suitable audio signal encoding method as indicated by step 511 in FIG.

[0159] The encoder may then output the encoded MASA metadata, the MASA-to-total ratio, the ISM ratio, and the combined transport audio signal, as indicated by step 513 in FIG.

[0160] FIG. 6 illustrates the Mode C encoding method, a third (or higher or single object) encoding mode, indicated by step 308 in FIG. 3. Thus, at a bitrate higher than that of the Mode A or B encoding method (for the same number of objects), the audio content of (only) one object is separated and transmitted separately. Furthermore, a downmix shaped from the MASA transport channel and the remainder of the object is transmitted in MASA format, along with additional parameters for the MASA-to-total energy ratio and the ISM ratio. Furthermore, ISM metadata is transmitted, along with an identifier describing which object has been separated. For each frame, it is determined which object needs to be separated. This determination may be based, for example, on the relative level of the object to the other objects (e.g., separating the loudest object). This is described in detail in WO 2022 / 214730.

[0161] Thus, there are method steps for receiving / obtaining object-based streams (independent streams with metadata) and multi-channel-based (MASA stream) transport audio signals and metadata, for example as indicated by step 601 in FIG.

[0162] Next, as shown by step 603 in Figure 6, one audio object is selected, and an object identifier is generated based on the selected audio object. Furthermore, an audio signal associated with the selected audio object is encoded. Any suitable audio signal encoder may be used to encode the audio signal of the selected object. For example, an audio signal encoder the same as or similar to that used to encode the MASA audio signal(s) may be employed.

[0163] Next, a combined MASA and remaining (or unselected) object-based transport audio signal (or downmix) is generated, as indicated by step 605 in FIG. 6. The object transport signal can be created in the same way as presented in the previous mode, Mode B, with the difference that the selected or separated objects are not included in the mix. For example, a multichannel or MASA audio signal and a (unselected) object transport signal can be added together to generate the combined transport audio signal. Note that objects are not separated from the mix or (re)inserted into the mix abruptly, but rather inserted through an appropriate fade-in / fade-out mode by setting the corresponding ISM ratio in the mix to a non-zero value. Therefore, even if one object is separated from the mix, the ISM ratios of all objects must be encoded.

[0164] There is one exception: for a given subband, the isolated object ratio is zero in the first subframe, i.e., all subsequent subframes within the same frame have the same value of isolated object ISM ratio and do not need to be encoded and transmitted.

[0165] The MASA to total ratio and ISM ratio may be determined as shown by step 607 in FIG.

[0166] The object identifier, MASA metadata, MASA-to-total ratio, and ISM ratio may then be encoded based on any suitable lattice or entropy encoding method, as shown by step 609 in Figure 6. The encoding of the MASA-to-total energy ratio may be implemented in the manner described in WO2022 / 200666. The encoding of the ISM ratio is described in further detail below.

[0167] The combined audio signal may then be encoded based on any suitable MASA audio signal encoding method, as indicated by step 611 of Figure 6. The encoding of the combined transport audio signal may employ any suitable transport audio signal encoding, for example, an audio signal(s) encoder of an IVAS encoder.

[0168] In other words, the separated objects are determined, separated and encoded as described in WO2022 / 214730, and for the remaining objects and MASA streams the processing works as described in WO2022 / 200666.

[0169] The encoder can then output the encoded object identifiers, MASA metadata, MASA-to-total ratio, ISM ratio, object metadata (for all objects), selected single object audio signals, and combined transport audio signals, as indicated by step 613 in FIG. 6.

[0170] Figure 7 illustrates the Mode D encoding method, the fourth (or highest, or all-object) encoding mode, as shown in step 310 of Figure 3. Thus, at higher bit rates (for a given number of objects, and compared to the bit rates of the Mode A, B, or C encoding methods), the two input audio formats, MASA and ISM, are encoded independently and transmitted in the same bitstream (in other words, using a single instance of the IVAS codec).

[0171] Thus, there are method steps for receiving / obtaining object-based streams (independent streams with metadata) and multi-channel-based (MASA stream) transport audio signals and metadata, for example as illustrated by step 701 in FIG.

[0172] Next, as indicated by step 703 of FIG. 7, the multi-channel-based (MASA stream) transport audio signal and metadata are encoded based on any suitable MASA encoding method.

[0173] The objects (independent streams with metadata) and associated metadata can be further encoded as indicated by step 705 in Figure 7. Any suitable mono encoder can be employed to implement the encoding, such as an EVS-based mono encoder block.

[0174] The encoder can then output the independently encoded objects (independent streams with metadata) and associated metadata, as well as the independently encoded multi-channel based (MASA stream) transport audio signal and metadata, as indicated by step 707 in FIG. 7.

[0175] The generation and encoding of ISM ratio values ​​as determined and encoded within encoding modes B and C will be described in further detail below.

[0176] 8, an audio object metadata encoder 111 according to some embodiments is shown in further detail. In some embodiments, the MASA-to-sum ratio and direction (i.e., azimuth and elevation angles for each object) are forwarded and encoded by the audio object metadata encoder 111, although the specific encoding of the direction and MASA-to-sum ratio will not be described in further detail herein. For example, WO2022 / 200666 describes a suitable MASA-to-sum ratio encoding method, and PCT / EP2017 / 078948 and US11475904 describe a suitable direction value encoding method.

[0177] As mentioned above, the ISM ratio is passed to the audio object metadata encoder 111 .

[0178] In some embodiments, the audio object metadata encoder 111 includes an ISM ratio vector generator 803 configured to receive ISM ratio values ​​and generate a vector representation of the ISM ratios for the subbands and subframes. In other words, the vector describes the ISM values ​​of all objects in a given time-frequency tile. The vector of ISM ratio values ​​804 may then be passed to a vector (ISM ratio) quantizer 805. A vector is sometimes known as an array or group of ISM ratio values.

[0179] In some embodiments, the audio object metadata encoder 111 includes a vector (ISM ratio) quantizer 805 configured to receive the vector 804 of ISM ratios and quantize them.

[0180] In some embodiments, for each subband and temporal subframe, the ratios can be scalar quantized with nb = 3 bits. Thus, quantization of each ratio returns a positive integer value between 000 and 111 in binary (or between 0 and 7 in decimal or base 10). In other embodiments, quantization can be performed using any suitable number of bits. Thus, the following example shows a uniform scalar quantizer based on 3 bits per value. It can also be a non-uniform scalar quantizer. The distribution of the indexes does not affect the indexing; however, this can be taken into account by observing that, in principle, some vector indices are more likely than others. In some embodiments, a quantizer based on more than 3 bits can be employed.

[0181] By definition, for each subband and subframe, the sum over all objects is 1. For each subband and temporal subframe, the values ​​are scalar quantized with nb=3 bits. Since the ISM ratios sum to at most 1, there is a corresponding relationship between the quantization indexes, and they will sum to at most 2^nb-1 (=7). This allows for a reduction in the number of transmitted indexes; they can be transmitted with one less object per subband. However, due to the nonlinearity of the quantization operation, reconstruction at the decoder may not be optimal with respect to the condition of summing to a constant within the index domain. Thus, in some embodiments, the quantization operation may further include a quantization optimization operation, characterized by constrained vector quantization.

[0182] Thus, in some embodiments, the quantization of the per-subband and per-subframe indices may be implemented based on the following operations: 1. For o=0:O-1 a. Quantize the ISM ratio rISM(o) to the smallest nearest neighbor and obtain the index idx(o) (i.e., from two adjacent possible quantized values, select the one with the lower value). 2. End 3. Formula

number

[0183] Note that in scalar quantization the quantization operation is forced to always take the lowest neighbors, so the modification of the indices is performed only by increasing their values.

[0184] In some embodiments, when encoding one isolated object mode, if the isolated mode's ISM ratio is zero, its corresponding quantization value remains zero and cannot be changed as described above. Therefore, any changes made to comply with the sum constraint are only made to the ISM ratio index of other objects.

[0185] This quantization process ensures that the sum of the indices across all objects is equal to K.

[0186] The vector of indices of the quantized ISM ratio values ​​806 may be passed to a quantization vector encoder 807 .

[0187] The audio object metadata encoder 111, in some embodiments, may include a quantization vector encoder 807. The quantization vector encoder 807 may be configured to obtain a vector of indices of the quantized ISM ratio values ​​806 and generate therefrom appropriate encoded quantized ISM ratio values ​​808, which may be passed to the bitstream generator 113, for example, to be included in the bitstream 118.

[0188] In some embodiments, two differential coding modes are tested: index difference with respect to previous subframe data and index difference with respect to previous subband data.

[0189] In some embodiments, differential encoding with respect to the previous subframe is only applied to subframes starting from the second subframe.

[0190] There are BxN O-dimensional integer vectors of ISM ratio indices. The encoding of the ISM ratio indices is presented as follows: 1. The first subframe ISM rate quantization data is encoded with an enumeration index a. For each subband in the first subframe i. As the enumeration index, encode a vector of O integer values ​​summing up to the value 2^nb-1 (=7). b. End for 2. For each subframe 1 to N-1 a. For each subband from 0 to B-1 i. Calculate the difference index for each object relative to the previous subframe ii. Convert the differential index to a regular index iii. Positive indices are encoded with GR code with parameter 0, and the corresponding number of bits is estimated. iv. The positive index is encoded with a GR code with parameter 1, and the corresponding number of bits is estimated. v. Calculate the difference index for the previous subband for each object (if the previous subband does not exist, use the data from the previous subframe) vi. Convert differential indexes to positive indexes vii. Positive indices are encoded with GR code with parameter 0, and the corresponding number of bits is estimated. viii. Positive indices are encoded with GR code with parameter 1, and the corresponding number of bits is estimated. b. End for c. For each differential coding mode (subband-related / subframe-related) i. Select the optimal GR parameters for using the mode across all subband data in the current subframe d. End for e. Select the differential coding mode for the current subframe (differential to previous subframe or previous subband) given the shortest code length for the subframe 3. End for

[0191] The GR parameter values ​​tested are 0 and 1.

[0192] Differential coding using GR parameter G for a "prior" vector in subframe i and subband b is shown below. 1. Calculate the difference index for each object relative to the previous vector 2. Convert the differential index to a regular index 3. The positive index is encoded with a GR code with parameter G, and the corresponding number of bits is estimated.

[0193] Next, we show the encoding of the ISM ratio in one isolated object mode. For the first subframe, 1. The ISM ratios of all valid subbands are encoded in the enumeration index, and the total number of bits across all subbands is counted. 2. The first subband is encoded using the enumeration index, and subsequent subbands are differentially encoded with respect to the previous subband using a GR parameter with value 0. The corresponding number of bits is estimated. 3. Of the two methods above, the one that uses the fewest number of bits is selected. 4. One bit is used to indicate the coding method of the first subframe

[0194] A valid subband of a subframe is a subband with a MASA to total energy ratio value less than 1 (or less than a threshold close to 1, eg, 0.98).

[0195] After encoding the ISM ratio data of the first subframe, the ISM ratio values ​​of the separated objects of each subband of the first subframe are checked. If they are all zero, the ISM ratios of the separated objects of the following subframes are not transmitted because they should also be zero for all subbands.

[0196] That is, the following is checked: 1. If (num_objects>2)&&(for all subbands of the first subframe, the ratio index corresponding to the separated object == 0) 1.1.D=2 2.Else 2.1.D=1 3.End if For each subsequent subband, the ISM ratio of OD is transmitted, where O is the number of objects and D is determined above.

[0197] If there are only two objects and the rate index of the separated objects in all subbands of the first subframe is zero, then nothing is transmitted for encoding the ISM rate of the current frame, i.e., O=2 and OD=0.

[0198] The bitstream for one frame in such an embodiment includes the following ISM ratio related data: Vector index of the first subframe of all subbands For each subframe except the first subframe, 1 bit indicating the differential coding mode (for the previous subframe or previous subband) 1 bit indicating GR order (0 or 1) GR encoded differential index

[0199] For the first subband, there is no subband data to look back to, so a differential can be taken relative to the previous subframe data. GR parameters and differential coding flags can be determined for each subframe, and they are valid for all subbands corresponding to that subframe.

[0200] In some embodiments, for the first subframe there may also be a differential mode for the previous subband, i.e., replace 1 above with: 1. The first subframe ISM rate quantization data is encoded with an enumeration index a. For the first subband of the first subframe i. As the enumeration index, encode a vector of O integer values ​​summing up to the value 2^nb-1 (=7). b. For each subband 1 to B-1 (in the first subframe) i. For each object, calculate the difference index to the previous subband ii. Convert the differential index to a regular index iii. Positive indices are encoded with GR code with parameter 0, and the corresponding number of bits is estimated. c. End for

[0201] In some embodiments, the selection may be implemented to be valid for the entire subframe, determined per subband, or determined separately for each subframe and subband.

[0202] Encoding using enumeration indexes is done as specified in GB2217884.2.

[0203] When processing data, there is a special case where all ISM ratios are 0 and do not obey the constant sum constraint shown above. This case corresponds to the object having no audio signal or the absence of an audio signal at all. The information that the object has no audio signal can be inferred from the MASA to total energy ratio. If the MASA to total energy ratio of a TF tile (identified by subband and subframe) is 1, there is no need to transmit the ISM ratio of that TF tile.

[0204] When no audio signal is present, the MASA to total energy ratio can be forced to 1, making it possible to infer degeneracy from the value of the MASA to total energy ratio, which is the case when all values ​​of the ISM ratios are zero.

[0205] In some embodiments, for Mode C encoding (separated object mode), the separated object contributes to the remaining mix, making the encoding more time-stable (there is still something left in the mix after separation). Furthermore, in some embodiments, this can be exploited when considering differential coding.

[0206] To do this, before encoding the ratios of the first O-1 objects in each subband and each subframe, it is determined whether the ratio index corresponding to the separated object is among the indices to be stored or transmitted.

[0207] For example, in the case of four objects, if in a particular subband of a subframe that is not the first subframe, the ISM ratio index value is [3 0 2 2] and the separated object is the second object (ISM ratio index is 0), nothing is changed and the encoding procedure is performed with the first three indices

[0302] .

[0208] However, in an example where the separated object is the last object and the ISM ratio index is [3 1 3 0], the method is configured to swap places between the ISM ratios of the third and fourth objects to obtain the array [3 1 0 3] and then encode the array

[0310] .

[0209] By performing this swapping, the corresponding ISM ratios for the separated objects change little between subframes or subbands, and the resulting differential encoding leads to smaller code lengths on average.

[0210] In some embodiments, it may be possible to reduce the number of ratios transmitted. This can be done, for example, if the ISM ratios of the separated objects in all subbands in the first subframe are zero. In that case, the ISM ratios of the separated objects in subsequent subframes will also be zero and they will not be transmitted. Below is the C code for the swapping and encoding operations: #ifdef ONE_SEP_OBJ_UPDATE rotate = 0; #endif for ( sf = 0;sf < numSf;sf++ ) { for ( band = 0;band < numCodingBands;band++ ) { for ( obj = 0;obj < hMasa->data.nchan_ism;obj++ ) { assert( ( hMasa->data.energy_ratio_ism[sf][band][obj] >= 0 ) && ( hMasa->data.energy_ratio_ism[sf][band][obj] <= 1 ) ); ratio_ism[band][obj] = hMasa->data.energy_ratio_ism[sf][band][obj]; } / * Quantized output ratio * / quantize_ratio_ism_vector( ratio_ism[band], ratio_ism_idx[band], hMasa->data.nchan_ism, hQMetaData->masa_to_total_energy_ratio[sf][band] ); / * Reconfiguration value * / reconstruct_ism_ratios( ratio_ism_idx[band], hMasa->data.nchan_ism, step, hMasa->data.q_energy_ratio_ism[sf][band] ); / * Encode the vector * / } #ifdef ONE_SEP_OBJ_UPDATE if ( ( hMasa->data.nchan_ism > 2) && ( idx_separated_object == hMasa->data.nchan_ism - 1 ) ) { / * Rotate the components * / rotate = 1; for ( band = 0;band < numCodingBands;band++ ) { tmp = ratio_ism_idx[band][hMasa->data.nchan_ism - 1]; ratio_ism_idx[band][hMasa->data.nchan_ism - 1] = ratio_ism_idx[band][0]; ratio_ism_idx[band][0] = tmp; } } #endif / * Encode the data for the current subframe * / encode_ratio_ism_subframe( ratio_ism_idx, hMasa->data.nchan_ism, numCodingBands, sf, ratio_ism_idx_prev_sf, hMetaData, hQMetaData->masa_to_total_energy_ratio[sf] ); / * Calculate the quantized ISM ratio * / / * Save the previous subframe index * / for ( band = 0;band < numCodingBands;band++ ) { mvs2s( ratio_ism_idx[band], ratio_ism_idx_prev_sf[band], hMasa->data.nchan_ism ); } #ifdef ONE_SEP_OBJ_UPDATE if ( rotate ) { for ( band = 0;band < numCodingBands;band++ ) { tmp = ratio_ism_idx[band][hMasa->data.nchan_ism - 1]; ratio_ism_idx[band][hMasa->data.nchan_ism - 1] = ratio_ism_idx[band][0]; ratio_ism_idx[band][0] = tmp; } } #endif }

[0211] At the decoder, after receiving and decoding the O-1 ISM ratio indices, if the separated object is at the end of the list, a swap between the separated object and the first object is performed.

[0212] With reference to FIG. 9, a flow diagram summarizing the operation of the exemplary audio object metadata encoder 111 shown in FIG. 8 is shown.

[0213] The initial operation is to receive / obtain the ISM ratio value, as shown by step 901 in FIG.

[0214] From the ISM ratios, the next operation is to generate a vector from the ISM ratio values, as indicated by step 903 in FIG.

[0215] Once the vectors of ISM ratio values ​​are determined, they may be quantized to generate a quantized vector of ISM ratio values, as indicated by step 905 of FIG.

[0216] If the encoder is in distinct object encoding mode (encoding mode C), it identifies distinct objects in the vector and, if necessary, swaps them so that they are not the last (or excluded vector element), as shown in step 907 of Figure 9.

[0217] From the quantized vector, an index value is then generated that represents the encoded quantized vector, as indicated by step 909 in FIG.

[0218] The encoded ISM vector index values ​​may then be output for inclusion in the bitstream, as indicated by step 911 in FIG.

[0219] The decoder may be configured to decode the ISM values ​​using the opposite process to that described above. Thus, the decoder may be configured to obtain the ISM ratio values ​​from the encoded vector values ​​based on the following operations: 1.For sf=1:num_subframes 1.1. Decode / read ISM ratio index vectors for all subbands 1.2. Save the current subframe data to the previous subframe data 1.3.Reconstructing the ISM Ratio from the ISM Ratio Index 2.End for Decode the ISM rate index for subframe sf. 1.If the first subframe 1.1.For b=1:num_subbands 1.1.1. Read the index of the ISM ratio index vector for subband b 1.1.2. Decode the index into a vector of indices (e.g. according to GB2217884.2) 1.2.End for 1.3.If (num_objects>2)&&(for all subbands of the first subframe, the ratio index corresponding to the separated object == 0) 1.3.1 D=2 1.4 Else 1.4.1 D=1 1.5 End if 1.3.For b=2:num_subbands 1.3.1. Read the bit to determine if it is differential coding 1.3.2 If not differential coding 1.3.2.1. Read the index of the ISM ratio index vector for subband b 1.3.2.2 Decode an index into a vector of indices (e.g., according to GB2217884.2) 1.3.3 Else 1.3.3.1 For i=1:num_objects-D 1.3.3.1.1 Read the GR code and decode it to a positive index (GR order 0) 1.3.3.1.2 Converting a positive index to an integer corresponding to a differential index 1.3.3.1.3. Calculate the ISM ratio index for subband b as the sum of the previous subband ISM ratio index + the decoded difference 1.3.3.2 End for 1.3.4.End else 1.4.End for 2.Else 2.1.Read the differential mode bits 2.2.Reading Golomb Rice Order 2.3.For b=1:num_subbands 2.3.1.For i=1:num_objects-D 2.3.1.1. Read the GR code and decode it to a positive index 2.3.1.2. Convert the positive index to an integer corresponding to the differential index 2.3.2.End for 2.4.End for 2.5.If differential mode with respect to previous subframe 2.5.1.For b=1:num_subbands 2.5.1.1.For i=1:num_objects-D 2.5.1.1.1. Calculate the ISM rate index for subband b as the sum of the ISM rate index of the previous subframe plus the decoded difference 2.5.1.2.End for 2.5.1.3. Calculate the index corresponding to the last object so that the sum of the indices of all objects is a constant K. 2.5.2.End for 2.6.Else 2.6.1. Calculate the ISM ratio index of num_objects-1 for the first subband as the sum of the ISM ratio index of the first subband of the previous subframe + the decoded difference 2.6.2.For b=2:num_subbands 2.6.2.1.For i=1:num_objects-D 2.6.2.1.1. Calculate the ISM ratio index for subband b as the sum of the previous subband ISM ratio index + the decoded difference 2.6.2.2.End for 2.6.2.3. Calculate the index corresponding to the last object so that the sum of the indices of all objects is a constant K. 2.6.3.End for 2.7.End if 2.8. (For separate object encoding mode) If the separated object is at the end of the list, swap the last index with the first index.

[0220] In the above example, the decoder is shown to implement a decoding where the encoding method is configured to not encode an object where the ISM ratio of the separated object in all subbands in the first subframe is zero (because the ISM ratio of the separated object in subsequent subframes is also zero). In an example decoder where this is not implemented, sections 1.1.3 to 1.1.5 can be replaced with the following: 1.1.3 D=1 Alternatively, 1.1.3 through 1.1.5 are deleted and all instances of D are replaced with the value 1.

[0221] 10 is an exemplary electronic device that may be used as any of the apparatus portions of the system as described above. The device may be any suitable electronic device or apparatus. For example, in some embodiments, device 1400 is a mobile device, user equipment, tablet computer, computer, audio playback device, etc. The device may be configured to implement, for example, an encoder / analyzer portion and / or a decoder portion as shown in FIG. 1, or any of the functional blocks as described above.

[0222] In some embodiments, device 1400 includes at least one processor or central processing unit 1407. Processor 1407 can be configured to execute various program code, such as the methods described herein.

[0223] In some embodiments, device 1400 includes at least one memory 1411. In some embodiments, at least one processor 1407 is coupled to memory 1411. Memory 1411 can be any suitable storage means. In some embodiments, memory 1411 includes program code sections for storing program code implementable on processor 1407. Additionally, in some embodiments, memory 1411 can further include a storage data section for storing data, e.g., data that has been processed or to be processed according to embodiments as described herein. The implemented program code stored in the program code sections and the data stored in the storage data section can be retrieved by processor 1407 whenever needed via the memory-processor coupling.

[0224] In some embodiments, device 1400 includes a user interface 1405. User interface 1405, in some embodiments, may be coupled to a processor 1407. In some embodiments, processor 1407 may control the operation of user interface 1405 and receive input from user interface 1405. In some embodiments, user interface 1405 allows a user to input commands into device 1400, for example, via a keypad. In some embodiments, user interface 1405 allows a user to obtain information from device 1400. For example, user interface 1405 may include a display configured to display information from device 1400 to the user. User interface 1405, in some embodiments, may include a touch screen or touch interface that can both allow information to be input into device 1400 and also display information to the user of device 1400. In some embodiments, user interface 1405 may be a user interface for communication.

[0225] In some embodiments, device 1400 includes an input / output port 1409. In some embodiments, input / output port 1409 includes a transceiver. In such embodiments, the transceiver is coupled to processor 1407 and may be configured to enable communication with other apparatuses or electronic devices, for example, via a wireless communication network. The transceiver or any suitable transceiver or transmitter and / or receiver means may, in some embodiments, be configured to communicate with other electronic devices or apparatuses via a wire or wired coupling.

[0226] The transceiver may communicate with the further device via any suitable known communication protocol. For example, in some embodiments, the transceiver may use a suitable radio access architecture based on Long Term Evolution Advanced (LTE-Advanced, LTE-A) or New Radio (NR) (which may also be referred to as 5G), a Universal Mobile Telecommunications System (UMTS) Radio Access Network (UTRAN or E-UTRAN), Long Term Evolution (LTE, same as E-UTRA), a 2G network (a legacy network technology), a wireless local area network (WLAN or Wi-Fi), a system using Worldwide Interoperability for Microwave Access (WiMAX), Bluetooth, Personal Communications Services (PCS), ZigBee, Wideband Code Division Multiple Access (WCDMA), Ultra-Wideband (UWB) technology, a sensor network, a mobile ad hoc network (MANET), a Cellular Internet of Things (IoT) RAN, and an Internet Protocol Multimedia Subsystem (IMS), or any other suitable option and / or any combination thereof.

[0227] The transceiver input / output port 1409 may be configured to receive a signal.

[0228] In some embodiments, device 1400 may be employed as at least a portion of a synthesis device. Input / output port 1409 may be coupled to headphones (which may be headphones with or without head tracking) or the like and loudspeakers.

[0229] Generally, various embodiments of the present invention may be implemented in hardware or special-purpose circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described as block diagrams, flowcharts, or using some other graphical representation, it will be appreciated that these blocks, apparatus, systems, techniques, or methods described herein may be implemented in, by way of non-limiting example, hardware, software, firmware, special-purpose circuits or logic, general-purpose hardware or controller or other computing device, or some combination thereof.

[0230] Embodiments of the present invention may be implemented by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Furthermore, in this regard, it should be noted that any block of logic flow in the diagrams may represent program steps or interconnected logic circuits, blocks, and functions, or combinations of program steps and logic circuits, blocks, and functions. Software may be stored on physical media, such as memory chips or blocks implemented within a processor, magnetic media, such as hard disks or floppy disks, and optical media, such as DVDs and their data variants, CDs.

[0231] The memory may be of any type suitable for the local technology environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The data processor may be of any type suitable for the local technology environment and may include, by way of non-limiting example, one or more of a general purpose computer, a special purpose computer, a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), gate-level circuitry, and a processor based on a multi-core processor architecture.

[0232] Embodiments of the present invention may be implemented in a variety of components, such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available to convert logic-level designs into semiconductor circuit designs ready to be etched into semiconductor substrates.

[0233] Programs, such as those offered by Synopsys, Inc. of Mountain View, Calif., and Cadence Design of San Jose, Calif., use well-established design rules as well as pre-stored libraries of design modules to automatically route conductors and position components on semiconductor chips. Once the design of a semiconductor circuit is complete, the resulting design can be transmitted in a standardized electronic format (e.g., Opus, GDSII, etc.) to a semiconductor manufacturing facility or "fab" for fabrication.

[0234] As used in this application, the term "circuitry" may refer to one or more or all of the following: (a) Hardware-only circuit implementations (e.g., implementations using only analog and / or digital circuits); (b) For example (where applicable), a combination of the following hardware circuitry and software: (i) a combination of analog and / or digital hardware circuitry(s) and software / firmware; (ii) any portion of software-based hardware processor(s) (including digital signal processor(s)), software, and memory(s) that cooperate to cause a device, such as a mobile phone or server, to perform various functions; and Hardware circuitry(s) and / or processor(s), such as microprocessor(s) or portions of microprocessor(s), that require software (e.g., firmware) to operate, but software may not be present if not necessary for operation.

[0235] This definition of circuit applies to all uses of the term in this application, including any claims. As a further example, as used in this application, the term circuit also encompasses implementations of simply a hardware circuit or processor(s), or portions of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuit also encompasses, for example, baseband or processor integrated circuits for mobile devices, or similar integrated circuits in servers, cellular network devices, other computing devices, or other network devices, where applicable to certain claim elements.

[0236] As used herein, the term "non-transitory" is not a limitation regarding the permanence of the data storage (eg, RAM vs. ROM), but rather a limitation of the medium itself (ie, tangible rather than signal).

[0237] As used herein, "at least one of: " and "at least one of " and similar phrases where a list of two or more elements is joined by "and" or "or" mean at least any one of the elements, or at least any two or more of the elements, or at least all of the elements.

[0238] The foregoing description has provided a full and informative description of exemplary embodiments of the present invention, by way of illustrative and non-limiting example. However, various modifications and adaptations may become apparent to those skilled in the art in light of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of this invention will still fall within the scope of the present invention, as defined by the appended claims.

Claims

1. 1. An apparatus for encoding audio object parameters, comprising: identifying one of at least two audio objects in an audio environment to be separately encoded; - obtaining proportion parameters of the at least two audio objects for a time-frequency element of a frame comprising at least one time element and at least one frequency element, the proportion parameters being configured to identify a distribution of a particular audio object within an object portion of the overall audio environment and for a particular time-frequency element; grouping, for said time-frequency components, ratio parameters associated with said at least two audio objects; quantizing the ratio parameters within the group, the quantization of the ratio parameters being configured to generate integer representations of the ratio parameter values ​​that sum to a defined integer value for a particular group; encoding the integer representation of the ratio parameter value within a first group or a first set of groups as an enumeration index; encoding a subsequent group as a difference value relative to the first group or the first set of groups, wherein the means for encoding the subsequent group further comprises: selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group; encoding the selected subset of the integer representations of the ratio parameter values ​​of the subsequent groups as difference values ​​relative to the first group or the first set of groups; the encoding step; and means for performing the steps of the grouping means is adapted to order the ratio parameters within the group such that the selected subset of integer representations of the ratio parameter values ​​includes the integer representations of the ratio parameter values ​​associated with the selected one of the at least two audio objects to be encoded separately.

2. 2. The apparatus of claim 1, wherein the means for selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group is means for selecting all but one of the integer representations of a particular group because the integer representations of the ratio parameter values ​​of the group are values ​​defined for all groups.

3. The means for selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group comprises: determining that the integer representation of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be separately encoded is zero for all frequency components of the first time component; selecting, for a particular group, all but two of the integer representations, wherein said selecting does not include the integer representation of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be separately encoded; The device of claim 1 , which is a means for performing the following:

4. 4. The device according to claim 1, wherein said means for grouping, for a time-frequency component, proportion parameters associated with said at least one audio object are means for generating a vector of said proportion parameters.

5. 5. The apparatus of claim 1, wherein the means for encoding the integer representation of the ratio parameter value in a first group or a first set of groups as an enumeration index is means for generating an integer value based on indexing from the group, the generated integer value representing the ratio parameter of the audio object.

6. The means for quantizing the ratio parameters within the group, wherein the quantization of the ratio parameters is configured to generate integer representations of the ratio parameter values ​​that sum to a defined integer value for a particular group, the means for quantizing comprising: generating a single value by adding elements from said group of ratio parameters; generating said integer representation from said single number by executing an iterative loop from the zeroth iteration up to said single number of iterations and sequentially associating index values ​​with iteration numbers of the iterative loop having valid selections of ratio parameters, said integer value being the highest index value; 6. The apparatus of claim 5, wherein the apparatus is a means for performing the steps of:

7. The means for quantizing the ratio parameters within the group comprises: quantizing the ratio values ​​within the particular selection to obtain quantization index values ​​using minimum nearest neighbor scalar quantization; calculating a reconstructed value of said ratio parameter for said particular selection; calculating an error value based on the difference between the reconstructed ratio value and a particular selection of the ratio parameter value; determining a sum of quantization index values; selecting at least one quantization index value to increment so that a sum of said quantization index values ​​equals a sum of expected indexes; The device according to any one of claims 1 to 6, which is a means for performing the above.

8. The means for selecting at least one quantization index value to increment so that the sum of the quantization index values ​​equals the sum of expected indexes comprises: selecting the at least one quantization index value to increment based on identifying a maximum decrease in the error value when the index value is incremented; or selecting the at least one quantization index value to increment based on identifying a minimum increase in the error value when the index value is incremented; 8. The apparatus of claim 7, wherein the apparatus is a means for performing one of the following:

9. The means for encoding the selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or the first set of groups comprises: for a selected subset of the integer representations of the ratio parameter values ​​of the subsequent group for a particular time element of the frame, determining, for a first entropy coding parameter and a second entropy coding parameter, a number of bits required to entropy code the difference between the quantized frequency components; determining, for the first entropy coding parameter and the second entropy coding parameter, the number of bits required to entropy code the difference between the quantized time elements; selecting, for the particular time element, the first entropy coding parameter or the second entropy coding parameter based on a smaller number of bits required to code the difference within the particular time element of the frame; selecting one of the entropy coding of the difference between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the selected second entropy coding parameter based on a smaller number of bits required to code the difference within the particular time element of the frame; The apparatus according to any one of claims 1 to 8, which is a means for performing the above.

10. 10. The apparatus of claim 9, wherein the means for encoding the selected subset of integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or the first set of groups is means for encoding the selected one of frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the selected second entropy coding parameter.

11. The means for encoding the selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or the first set of groups comprises: For a set of ratio parameter selections for a particular frequency element of the frame, determining, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required to entropy code the quantized difference between the frequency components; determining, for the first entropy coding parameter and the second entropy coding parameter, a number of bits required to entropy code a quantization difference between temporal elements; selecting, for the particular frequency element, the first entropy coding parameter or the second entropy coding parameter based on a smaller number of bits required to code the difference within the particular time element of the frame; selecting one of the entropy coding of the difference between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the selected second entropy coding parameter based on a smaller number of bits required for coding the difference within the particular time element of the frame for the particular frequency element; The apparatus according to any one of claims 1 to 9, which is a means for performing the above.

12. 12. The apparatus of claim 11, wherein the means for encoding the selected subset of integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or the first set of groups is means for encoding the selected one of frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the selected second entropy coding parameter.

13. The means for encoding the selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as difference values ​​relative to the first group or the first set of groups comprises: generating an indicator indicative of the selected first entropy coding parameter or the second entropy coding parameter; generating an indicator indicating the selected one of the entropy coding of the difference between frequency elements or time elements for entropy coding based on the selected first entropy coding parameter or the second entropy coding parameter; The device according to any one of claims 8 to 12, which is a means for performing the above.

14. The apparatus according to any one of claims 8 to 13, wherein the entropy coding is Golomb-Rice entropy coding, the first entropy coding parameter is Golomb-Rice entropy coding order 0, and the second entropy coding parameter is Golomb-Rice entropy coding order 1.

15. 15. The apparatus of claim 8, wherein the means for encoding the selected subset of the integer representations of the ratio parameter values ​​of the subsequent group as differential values ​​relative to the first group or set of groups is means for differentially encoding a selection of the ratio parameters based on a leading indexed time element group of the ratio parameter when there is no leading indexed frequency element group of the ratio parameter.

16. The apparatus of any of claims 1 to 15, wherein the ratio parameter configured to identify a distribution of a particular object within the object portion of the overall audio environment is an ISM ratio.

17. The means for selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group further comprises: determining that the integer representation of the ratio parameter value associated with the selected one of the at least two audio objects to be separately encoded is substantially zero for all frequency elements of the first group or the first set of groups; encoding only the remaining selected subset of the integer representations of the ratio parameter values ​​of the subsequent groups as difference values ​​relative to the first group or the first set of groups; The device according to any one of claims 1 to 16, which is a means for performing the above.

18. 18. The apparatus of claim 1, wherein the first group is a first group of time and frequency elements of the groups, and the first group set is a group of frequency element groups associated with the first time element.

19. The apparatus of any of claims 1 to 18, wherein said means is further means for separately encoding said identified one of said at least two audio objects.

20. 1. An apparatus for decoding audio object parameters, comprising: obtaining a bitstream comprising encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment comprising more than two audio objects, the ratio parameters being configured to identify a distribution of particular objects within the object portions of the overall audio environment and with respect to particular time-frequency elements; identifying at least two audio objects in the audio environment as being separately encoded; decoding an enumeration index within a first group or a first set of groups to generate a decoded integer representation of the ratio parameter value for the first group or the first set of groups; decoding a subsequent group as a difference value relative to the first group or the first set of groups, wherein the means for decoding the subsequent group further comprises: determining a further ratio parameter value for the subsequent group based on a difference between a defined total ratio value and the sum of the ratio values ​​within the group; The apparatus further comprising:

21. The means for decoding a subsequent group as a difference value relative to the first group or the first set of groups further comprises: determining that the ratio parameter value corresponding to one of the at least two audio objects in the separately encoded audio environment is zero for all frequency components of the first time component; determining, for said subsequent group, an additional ratio parameter value having a value of 0, which represents one of said at least two audio objects in a separately encoded audio environment; 21. The apparatus of claim 20, wherein the apparatus is a means for performing the following:

22. 22. Apparatus according to claim 20 or 21, wherein the group is a vector of the ratio parameters.

23. The means for decoding an enumeration index within a first group or a first set of groups to generate a decoded integer representation of the ratio parameter value of the first group or the first set of groups comprises: Obtaining an integer value representing an encoded ratio parameter; converting the integer value to a selection of a ratio parameter based on the indexing of the vector; regenerating at least one further ratio parameter from said selection of said ratio parameters; The device according to any one of claims 20 to 22, which is a means for performing the above.

24. The means for decoding a subsequent group as a difference value relative to the first group or the first set of groups comprises: obtaining a differential indicator identifying a frequency differential or time differential encoding; obtaining an entropy encoding designator that identifies entropy encoding parameters; decoding the selection of the remainder of the ratio parameters for the frame based on the difference indicator and the entropy encoding indicator; The device according to any one of claims 20 to 22, which is a means for performing the above.

25. The apparatus of any of claims 20 to 24, wherein the ratio parameter configured to identify a distribution of a particular object within the object portion of the overall audio environment is an ISM ratio.

26. 1. A method for encoding audio object parameters, comprising: identifying one of at least two audio objects in an audio environment to be separately encoded; - obtaining proportion parameters of the at least two audio objects for a time-frequency element of a frame comprising at least one time element and at least one frequency element, the proportion parameters being configured to identify a distribution of a particular audio object within an object portion of the overall audio environment and for a particular time-frequency element; grouping, for said time-frequency components, ratio parameters associated with said at least two audio objects; quantizing the ratio parameters within the group, the quantization of the ratio parameters being configured to generate integer representations of the ratio parameter values ​​that sum to a defined integer value for a particular group; encoding the integer representation of the ratio parameter value within a first group or a first set of groups as an enumeration index; encoding a subsequent group as a difference value relative to the first group or the first set of groups, wherein encoding the subsequent group further comprises: selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group; encoding the selected subset of the integer representations of the ratio parameter values ​​of the subsequent groups as difference values ​​relative to the first group or the first set of groups; said encoding including: Including, The method, wherein grouping further comprises ordering the ratio parameters within the group such that the selected subset of the integer representations of the ratio parameter values ​​includes the integer representations of the ratio parameter values ​​associated with the selected one of the at least two audio objects to be encoded separately.

27. 27. The method of claim 26, wherein selecting a subset of the integer representations of the ratio parameter values ​​of the subsequent group further comprises selecting all but one of the integer representations of the ratio parameter values ​​of a particular group because the integer representations of the ratio parameter values ​​of the group are values ​​defined for all groups.

28. Selecting the subset of integer representations of the ratio parameter values ​​of the subsequent group may further comprise: determining that the integer representation of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be separately encoded is zero for all frequency components of the first time component; selecting, for a particular group, all but two of the integer representations, wherein said selecting does not include the integer representation of the ratio parameter value corresponding to one of the at least two audio objects in the audio environment to be separately encoded; 27. The method of claim 26, comprising:

29. 1. A method for decoding audio object parameters, comprising: obtaining a bitstream comprising encoded integer representations of ratio parameters for time-frequency elements of a frame comprising at least one time element and at least one frequency element, the integer representations of the ratio parameters being associated with audio objects in an audio environment, the audio environment comprising more than two audio objects, the ratio parameters being configured to identify a distribution of particular objects within the object portions of the overall audio environment and with respect to particular time-frequency elements; identifying at least two audio objects in the audio environment as being separately encoded; decoding an enumeration index within a first group or a first set of groups to generate a decoded integer representation of the ratio parameter value for the first group or the first set of groups; decoding a subsequent group as a difference value relative to the first group or the first set of groups, wherein decoding the subsequent group further comprises determining, for the subsequent group, a further ratio parameter value based on a difference between a defined total ratio value and a sum of the ratio values ​​within the group; The method comprising:

30. Decoding a subsequent group as a difference value relative to the first group or the first set of groups includes: determining that the ratio parameter value corresponding to one of the at least two audio objects in the separately encoded audio environment is zero for all frequency components of the first time component; determining, for said subsequent group, an additional ratio parameter value having a value of 0, which represents one of said at least two audio objects in a separately encoded audio environment; 30. The method of claim 29, comprising:

Images