Parameter quantization and entropy coding for low-latency audio codecs

Abstract

It provides parameter quantization and entropy coding for low-latency audio codecs. [Solution] A method for encoding metadata about an input signal frame by frame, wherein the metadata includes a plurality of parameters that are at least partially interrelated and can be calculated from the input signal. The method includes, for each frame: using a loop process: determining a processing strategy from among a plurality of processing strategies for calculating and quantizing the parameters; calculating and quantizing the parameters based on the determined processing strategy to obtain quantized parameters; and encoding the quantized parameters, performed sequentially and iteratively. In particular, each of the plurality of processing strategies includes a first instruction indicating the ordering related to the calculation and quantization of individual parameters; the processing strategy is determined based on at least one bitrate threshold.

Description

[Technical Field]

[0001] Cross-references to related applications This application claims priority to U.S. Provisional Applications No. 63 / 037,784 and No. 63 / 194,010, filed on 11 June 2020 and 27 May 2021, respectively. Each of those applications is invoked by whole reference.

[0002] Technical field This disclosure relates to the general domain of entropy coding of parameters (side information) for low-latency audio codecs (encoders / decoders), and to a mechanism for achieving a parameter bitrate target by sequentially and iteratively refining the parameter bitrate using a series of quantization and entropy coding techniques. [Background technology]

[0003] When the frame period (frame size) of the audio codec (encoder / decoder) approaches 20 milliseconds or less, the audio essence is updated with a shorter frame size. If we follow an approach that updates both the audio essence and parameters frame by frame, the side information for each frame will also be embedded and transmitted at the same rate.

[0004] However, in this field, it is generally known that side information does not need to be updated very frequently. For example, spatial parameters can generally be calculated and updated every 40ms, for instance. In codecs with a frame period of 40ms or more, this generally means that the parameter update rate is synchronized with the frame rate, and therefore parameters can be encoded independently in each frame. However, in codecs with a short frame period, for example less than 40ms, this effectively means oversampling if all parameters are included in each individual frame.

[0005] Therefore, broadly speaking, the focus of this disclosure is to propose a mechanism that minimizes side information (or sometimes called parameters) as much as possible while maintaining a high frame update rate for the audio essence. [Overview of the project] [Problems that the invention aims to solve]

[0006] In view of the foregoing, the present disclosure broadly provides a method for encoding metadata for an input signal frame by frame, having the characteristics of each independent claim, as well as a corresponding program, a computer-readable storage medium, and an apparatus. [Means for solving the problem]

[0007] According to certain aspects of this disclosure, a method is provided for encoding metadata about an input signal frame by frame. In particular, the metadata may be computed or calculated (e.g., extracted) from an input (audio or video) signal by using a suitable codec (encoder / decoder). Generally, the metadata may be used on the decoder side to reproduce the input signal. The metadata may include several parameters that are at least partially interrelated and can be computed from the input signal. That is, at least some of the parameters of the input signal may be computed (e.g., generated or reproduced) depending on at least some of the other parameters, and as a result, depending on various circumstances, not all of the parameters always have to be simply transmitted.

[0008] In particular, this method may involve, for each frame, sequentially and iteratively performing the following steps using a loop process: determining a processing strategy from a plurality of processing strategies for computing and quantizing parameters; computing and quantizing parameters based on the determined processing strategy to obtain quantized parameters; and encoding the quantized parameters. Since the loop process is generally directed (among other things) towards processing related to quantization, in some cases the loop process may also be called a quantization loop (or simply a loop for short). Similarly, since the processing strategy is generally directed (among other things) towards processing related to quantization, in some cases the processing strategy may also be called a quantization strategy (or, in some cases, interchangeably a quantization scheme). Furthermore, it should be noted that the encoding process may use any suitable encoding procedure, including but not limited to entropy coding (e.g., Huffman coding or arithmetic coding), or without entropy coding (e.g., base2 coding). Depending on various implementations and / or requirements, any other suitable encoding mechanism may be employed.

[0009] As can be understood and recognized by those skilled in the art, multiple processing strategies for calculating and quantizing parameters may be provided in any preferred manner, either predefined or preconfigured. Thus, the processing strategy may also be determined from multiple processing strategies in any preferred manner. For example, a preferred processing strategy may be selected from multiple processing strategies depending on the (current) bitrate requirement, such that the bitrate obtained after calculation, quantization, and encoding (e.g., encoding with or without entropy coding) based on the thus selected processing strategy satisfies the (current) bitrate requirement. It should be noted that the bitrate requirement may change from time to time (e.g., frame to frame), so the processing strategy thus determined may also differ for each frame, or for several frames.

[0010] In particular, each of the processing strategies may include a first instruction indicating the ordering (or sequence) related to the computation and quantization of individual parameters. That is, the first instruction may include sequence information indicating when and in what order individual parameters are computed and quantized. For example (but not limited to), the first instruction may include information indicating that all parameters are computed first before any of them are quantized.

[0011] More specifically, the processing strategy is determined based on at least one bitrate threshold. As will be understood and recognized by those skilled in the art, the bitrate threshold may be predefined or preconfigured, for example, depending on various implementations and / or requirements.

[0012] As configured as described above, the proposed method of this disclosure can be broadly considered to introduce the concept of a sequential, iterative, and stepwise approach to selecting an optimal parameter quantization scheme / strategy that generally seeks the “best” (or most suitable) quantization scheme from among several alternatives. Nevertheless, it should be noted that in this case, the term “best” does not necessarily have to be a quantization scheme with the lowest (resulting) parameter bitrate (i.e., after quantization and possible encoding), but may be considered one that can mitigate the loss of state for the decoder. As will be understood by those skilled in the art, the decoder “state” generally refers to the history of information that the decoder retains from previous frames in order to correctly decode the current frame. For example, in some cases (but not limited to), the encoder side may employ so-called time-differential encoding. However, the use of time-differential encoding can generally present disadvantages. This is mainly due to the fact that it introduces frame-to-frame state, which can be problematic when the audio stream may suffer packet loss during transmission. In this case, both audio and audio-related parameters may be lost during transmission, and therefore any parameters updated by time-delay coding may experience multiple subsequent frames with potential artifacts. In this sense, the state loss mitigation described above refers to an attempt to avoid time-delay coding where possible. This would eliminate the need for the decoder to rely on metadata received in previous frames to decode the metadata of the current frame. Furthermore, when time-delay coding is necessary, it is done in a way that allows the system to recover quickly from packet loss. Specifically, by carefully selecting an appropriate quantization scheme as described in this disclosure, the undesirable behavior associated with packet loss illustrated above can be limited (mitigated) as much as possible.In other words, the present disclosure generally proposes an encoding (encoder side) relaxation related to a sequential iterative selection process for quantization and encoding (with or without entropy encoding). This attempts to minimize the extent to which packet loss artifacts may be introduced, for example, because temporal differential encoding is used.

[0013] In some examples, the processing strategy can be determined such that the (resulting) bitrate of the encoded and quantized parameters is below a (metadata / parameter) bitrate threshold. Thus, the bitrate resulting from quantization and encoding using the determined (e.g., selected) processing strategy is within a (at least one) bitrate threshold, thereby meeting, for example, a bitrate requirement agreed upon in advance or determined in advance by a standardized specification.

[0014] In some examples, each of the plurality of processing strategies may further include a respective second instruction indicating information for performing quantization of the parameters.

[0015] In some examples, the information for performing quantization of the parameters includes a respective quantization range and / or quantization level for a plurality of parameters. For example, the information may be related to a maximum value, a minimum value, the number of quantization levels, or any other suitable value desired for each of the respective parameters (e.g., one for each parameter type). Generally, as would be understood and recognized by those skilled in the art, these quantization-related values / parameters provide or define a generally coarser or finer quantization, and correspondingly, are associated with better or worse spatial reproduction. As would be understood and recognized by those skilled in the art, roughly speaking, some (quantization) parameters are generally considered to be more sensitive to quantization than others, and generally, there may not be an absolute fine / coarse quantization method for all parameters.

[0016] When configured as described above, the plurality of processing strategies may each be regarded as including a first (partial) indication regarding the ordering / sequence related to calculation and quantization; and a second (partial) indication regarding the actual quantization process. By carefully designing the processing strategies (e.g., various combinations of the first and second indications), it is possible to target various bitrate configurations / requirements in an efficient and flexible manner, for example, for various use cases or scenarios. Specifically, in some cases, there may be one processing strategy (e.g., the coarsest quantization strategy among a plurality of quantization strategies) for which it can be considered guaranteed that it is (or equal to) less than the target bitrate threshold.

[0017] In some examples, parameter encoding may involve time and / or frequency difference coding. Broadly speaking, a single metadata parameter may be quantized from a continuous numerical value to an index representing a discrete value. In non-differential coding, the information encoded for that metadata parameter corresponds directly to its index. In particular, the term “non-differential coding” as used in this disclosure may refer to any type of non-time difference coding, non-frequency difference coding, or non-differential coding as appropriate, as understood and recognized by those skilled in the art. In time difference coding, the information encoded is the difference between the index of that metadata parameter from the current frame and the index of the same metadata parameter from the previous frame. As understood and recognized by those skilled in the art, the general concept of time difference coding illustrated above may be further extended, for example, to multiple frequency bands. Thus, metadata parameters may similarly be extended to multiple parameters, for example, each corresponding to a multiple frequency band. Frequency difference coding follows a similar principle, except that the coded difference is the difference between metadata for one frequency band of the current frame and metadata for another frequency band of the current frame (not the difference between the current frame and the previous frame, as in time difference coding). As a simple example (not as an limitation), if a0, a1, a2, and a3 represent parameter indices in four frequency bands of a particular frame, then in an exemplary implementation, the frequency difference indices could be a0, a0-a1, a1-a2, and a2-a3. As will be understood by those skilled in the art, the general idea behind (time and / or frequency) difference coding is that since metadata can typically change slowly from frame to frame or from frequency band to frequency band, even if the original values ​​of the metadata are large, the difference between the metadata and the metadata of the previous frame, or between the metadata and the metadata of another frequency band, is likely to be small. This is advantageous because parameters with statistical distributions that generally tend toward zero can be coded using fewer bits.

[0018] In some examples, the processing strategy determined for the current frame may differ from the processing strategy determined for the previous frame, and thus the encoding of the parameters may involve time-difference coding across different processing strategies. That is, in certain cases where different processing strategies are determined (for example, for different frames of an input signal), the method of this disclosure can still encode the parameters, for example, by engaging in time-difference coding across these different processing strategies.

[0019] As described above, each of the processing strategies may include a first instruction that indicates the ordering (or sequence) related to the calculation and quantization of individual parameters.

[0020] In some examples, the first instruction may include information indicating that all parameters are computed before quantization.

[0021] In some examples, the first instruction may include information indicating that the parameters are calculated individually and then sequentially quantized. In particular, at least one of several parameters may be calculated based on another quantized parameter among the several. Assuming a total of three parameters to be calculated and quantized, not as an limitation but as an example, the first parameter is first calculated (from the input signal) and then quantized; the second parameter is calculated based on the (quantized) first parameter and then quantized itself; and finally, the third parameter is calculated based on the (quantized) first parameter and / or the (quantized) second parameter and then quantized. In one example, the third parameter is calculated based on the quantized first and second parameters.

[0022] In some examples, the first instruction may include information indicating that all parameters are computed before any of the parameters are quantized; in particular, at least one of the parameters is recalculated based on another quantized parameter, and the recalculated parameter is quantized. Continuing with the above assumption of three parameters, all parameters are computed first, then the first and second parameters are quantized; then the third parameter is recalculated, for example, based on the quantized second parameter, and then the third parameter is quantized based on the recalculated value.

[0023] In some examples, the method may further include mapping the indices of quantized parameters from the previous frame to those of the current frame before encoding the quantized parameters. In other words, if a different processing strategy (e.g., a quantization scheme with different quantization levels and / or sequences) is determined (e.g., selected), the (quantized) indices from the previous frame quantized with the different quantization scheme are mapped to the indices of the current frame. In particular, this allows for time-difference coding between frames without relying on sending non-difference frames each time the quantization scheme changes, thereby further improving overall coding efficiency and flexibility.

[0024] In some possible implementations, the index mapping is expressed as: index cur =round(index prev ×(quant_lvl cur -1) / (quant_lvl prev -1)) may be executed based on the index. cur This is the index of the current frame after mapping, and index prev is the index of the previous frame, and quant_lvl cur `quant_lvl` is the quantization level of the current frame. prev This is the quantization level of the previous frame.

[0025] As a simple illustrative example, assume a quantization range from 0 to 2 and 11 previous quantization levels. In the case of uniform quantization, this generally means that each quantization step is 0.2. Further, if there are 21 current quantization levels, each quantization step is 0.1 for uniform quantization. Based on these assumptions, if the quantized value in the previous frame was 0.4, for 11 uniform quantization levels, the previous index index prev = 2 would be obtained. The mapping provides the quantized index of the metadata of the previous frame as if it had been quantized using the quantization levels of the current frame. Thus, in this example, if there are 21 quantization levels in the current frame, the quantized value 0.4 is mapped to index curr = 4. Once the mapped index is calculated, the difference between the indices of the current and previous frames is calculated and this difference is encoded. As will be understood and recognized by those skilled in the art, a similar or analogous approach may be applied to frequency difference coding, if desired.

[0026] Note that the above equations and each example are provided for illustrative purposes only. As will be understood and recognized by those skilled in the art, any other suitable mechanism (such as a look-up table, etc.) may be employed to perform the mapping of the index.

[0027] In some examples, the at least one bitrate threshold may include a target bitrate threshold. Thus, the loop process may involve: quantizing and encoding the parameters in a non-differential and / or frequency-differential manner using an entropy encoder according to a (determined) processing strategy; estimating (e.g., calculating) a first parameter bitrate for the encoded parameters; and terminating the loop process if the first parameter bitrate is less than or equal to the target bitrate threshold. In particular, in some possible implementations, the first parameter bitrate may be estimated (calculated) from the minimum non-differential and frequency-differential encodings encoded by the (trained) entropy encoder. As will be understood and recognized by those skilled in the art, the entropy encoder may be trained in any preferred way, for example, to fit individual encoding schemes. For example, in some possible implementations, training the entropy encoder may involve forming a probabilistic model based on metadata calculated from a large set of input signals. The specific signals chosen to form these models are expected to represent the types of signals that are expected to pass through the system in everyday use. Therefore, metadata from other similar signals should be encoded as efficiently as possible. In short, this training generally concerns adapting an entropy encoder to have maximum efficiency at the expected probability distribution of the parameters.

[0028] In some examples, the loop process may further involve the steps of: quantizing and encoding the parameter in a non-differential manner without entropy according to a processing strategy if the first parameter bitrate is greater than the target bitrate threshold; estimating the second parameter bitrate for the encoded parameter; and terminating the loop process if the second parameter bitrate is less than or equal to the target bitrate threshold.

[0029] In some examples, the loop process may further involve the steps of: quantizing and encoding the parameters in a time-difference manner using a (trained) entropy encoder according to a processing strategy if the second parameter bitrate is greater than the target bitrate threshold; estimating a third parameter bitrate for the encoded parameters; and terminating the loop process if the third parameter bitrate is less than or equal to the target bitrate threshold.

[0030] In some examples, time-difference quantization and encoding may be performed on a subset of parameters in a frequency-interleaved manner with respect to the previous frame. In particular, as will be understood and recognized by those skilled in the art, a frequency-interleaved scheme can generally refer to a case where different frequency bands (e.g., corresponding to different subsets of parameters) are processed (e.g., quantized and encoded) for different frames. In other words, time-difference quantization and encoding of parameters (at least a subset) for the current frame may be performed in a different frequency band (corresponding to the currently processed parameters) than that of the previous frame.

[0031] In some examples, time-difference quantization and encoding may be performed by cycling through several frequency-interleaved time-difference coding schemes such that, for each cycle, different subsets of parameters (corresponding to different sets of frequency bands) are quantized and encoded into time-difference expressions, while the remaining parameters are quantized and encoded non-difference-wise.

[0032] In some examples, the determined processing strategy may be considered as the first processing strategy, and thus the loop process may further involve: determining a second processing strategy from several processing strategies such that, if the third parameter bitrate is greater than the target bitrate threshold, the bitrate obtained by applying the second processing strategy is expected to be smaller than that obtained by using the first processing strategy; and repeating the above steps of the loop process. As will be understood and recognized by those skilled in the art, in such cases, the second processing strategy thus determined (e.g., selected) may simply be considered as a coarser processing strategy than the first processing strategy previously determined (e.g., selected). Thus, the set of possible quantized values / indexes may be smaller in size, which (typically) results in a correspondingly smaller bitrate.

[0033] In some examples, the parameters may be represented by a first number of frequency bands, and the loop process may further involve: if the third parameter bitrate is greater than the target bitrate threshold, reducing the number of frequency bands representing the parameters to a second number smaller than the first number, thereby reducing the total number of parameters to be quantized and encoded; and repeating the above steps of the loop process.

[0034] In some examples, parameters are represented in a first number of frequency bands, and the loop process may involve: if the third parameter bitrate is greater than the target bitrate threshold; reusing (or, in some cases, "freezing") parameters in one or more frequency bands from previous frames in the current frame; and repeating the steps of the loop process described above. As an example, when encoding with a particular encoding scheme, parameters in certain frequency bands (e.g., frequency bands 2, 6, and 10) can be frozen. In a further exemplary example, if all frequency bands are frozen over a period of two frames, the encoder can transmit half of the bands (e.g., even-numbered bands) in frame N and the remaining half (e.g., odd-numbered bands) in frame N+1 (thus reducing the total number of parameters transmitted). This generally means that the decoder obtains all (e.g., 12) updated frequency bands every other frame. In such cases, if one frame is lost, there is generally the option to extrapolate from the last two good frames. When recovering from packet loss, it is possible to interpolate between the bandwidths received in a given frame. Generally, the result of the above freeze process is entropy reduction, which has a slight impact on quality but does not require any changes to the decoder or entropy coding scheme.

[0035] In summary, when reducing the total number of bands, this can be done in at least the following two ways. The first way is to reduce the frequency resolution. Here, instead of using N bands, only M bands (where M < N bands) are used, and the bandwidth of one or more bands in the M-band configuration is larger than that in the N-band configuration. These M bands may be derived from the N bands, for example, adjacent bands can be grouped together as pairs, triples, etc., or other groups with perceptual importance. The second way is to reduce the time resolution. Here, the bandwidths of all N bands can remain exactly the same within the frequency domain, but the bands are frozen over a period of x frames (x > 1). This means that updates to the N bands can be sent over a period of x frames, or in other words, only N / x of the N bands need to be updated and sent to the decoder with each frame.

[0036] In some examples, at least one bitrate threshold may further include a maximum bitrate threshold that is greater than the above-described target bitrate threshold. Thus, the loop process may further be involved in obtaining the minimum values of the first, second, and third parameter bitrates before determining the second processing strategy, or reducing the number of frequency bands, or reusing the parameters; and ending the loop process if the minimum value is below the maximum bitrate threshold.

[0037] If the processing loop terminates at a specific step as described above, this generally means that the final parameter bitrate is the bitrate calculated at that step (i.e., when the processing loop terminates). Furthermore, as mentioned above, to be on the safe side, there may be a (e.g., coarse) quantization strategy that is guaranteed to be smaller than (or equal to) the target bitrate threshold or the maximum bitrate threshold among the given quantization strategies available for quantizing the parameter. Thus, it can be guaranteed that there is always a solution for fitting the parameter bitrate within the target bitrate threshold or the maximum bitrate threshold.

[0038] In some examples, the parameters may include one or more of the following: prediction parameters (sometimes simply referred to as PR parameters), cross-prediction parameters (sometimes simply referred to as C parameters), and decorrelation parameters (sometimes simply referred to as P parameters). As described above, at least some of the parameters are at least partially related and can be calculated based on each other. Of course, as can be understood and recognized by those skilled in the art, there may be any other suitable (type) parameters depending on the various implementations and / or requirements (e.g., the particular codec used).

[0039] As described above, the ordering (or sequence) of parameter calculation and quantization may be indicated by the first instruction of the processing strategy.

[0040] In some examples, the prediction parameters are first calculated and quantized, the cross-prediction parameters are calculated from the quantized prediction parameters and then quantized, and the decorrelation parameters are first calculated from the quantized cross-prediction parameters and the quantized prediction parameters and then quantized.

[0041] In some examples, the parameters (i.e., the predictor parameter, the cross-predictor parameter, and the decorrelation parameter) may be calculated first, then the decorrelation parameter and the predictor parameter may be quantized, and the cross-predictor parameter may be recalculated from the quantized predictor parameter and then quantized.

[0042] In some examples, this method may be applied to metadata encoding of immersive voice and audio services (IVAS) codecs or ambisonics codecs. The ambisonics codec may be a primary ambisonics (FOA) codec or a higher-order ambisonics (HOA) codec. Of course, as will be understood and recognized by those skilled in the art, any other suitable codec may be applied thereto, depending on the various implementations.

[0043] In some examples, the frame size is less than 40ms, and especially less than 20ms.

[0044] According to another aspect of this disclosure, a device is provided that includes a processor and memory coupled to the processor. The processor may be adapted to cause the device to perform all steps of the exemplary methods described through this disclosure.

[0045] Further aspects of this disclosure provide a computer program, which, when executed by a processor, may include instructions that cause the processor to perform all steps of the exemplary methods described through this disclosure.

[0046] From another perspective, a computer-readable storage medium is provided. The computer-readable storage medium may store the aforementioned computer program.

[0047] It will be understood that the features of the apparatus and the steps of the method can be replaced in many ways. In particular, as those skilled in the art will understand, the details of the disclosed method can be implemented by the corresponding apparatus (or system) and vice versa. Furthermore, it will be understood that any of the above statements made concerning the method also apply to the corresponding apparatus (or system) and vice versa. [Brief explanation of the drawing]

[0048] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings. [Figure 1] This is a schematic block diagram of an encoder / decoder ("codec") for encoding and decoding a signal (bitstream) according to one embodiment of the present disclosure. [Figure 2] This flowchart shows an example of a method for encoding metadata for an input signal frame by frame according to one embodiment of the present disclosure. [Figure 3] This flowchart shows an example of a processing loop according to one embodiment of the present disclosure. [Figure 4] This flowchart shows an example of a processing loop according to another embodiment of the present disclosure. [Modes for carrying out the invention]

[0049] The drawings (figures) and the following description relate only to preferred embodiments as examples. It should be noted from the following discussion that alternative embodiments of the structures and methods disclosed herein are readily recognizable as promising alternatives that can be used without departing from the principles of the claims.

[0050] Herein, several embodiments are given in detail. Examples are shown in the accompanying figures. Note that, where practically possible, similar or analogous reference numerals may be used in the figures to indicate similar or analogous functions. The figures illustrate embodiments of the disclosed system (or method) for illustrative purposes only. Those skilled in the art will readily understand from the following description that alternative embodiments of the structures and methods shown herein may be used without departing from the principles described herein.

[0051] Furthermore, in diagrams where connecting elements such as solid or dashed lines or arrows are used to indicate connections, relationships, or associations between two or more other schematic elements, the absence of such connecting elements does not mean that a connection, relationship, or association cannot exist. In other words, some connections, relationships, or associations between elements are not shown in the drawings so as not to obscure the disclosure. Moreover, for the sake of ease of illustration, a single connecting element is used to represent multiple connections, relationships, or associations between elements. For example, if a connecting element represents the communication of signals, data, or instructions, it should be understood by those skilled in the art that such an element represents one or more signal paths that may be required to affect the communication.

[0052] As described above, when the frame period of an audio codec (encoder / decoder) approaches 40 milliseconds, or even 20 milliseconds or less, the audio essence can be updated at short time intervals. However, it is generally known that side information (or metadata / parameters) does not need to be updated so frequently. In other words, in a codec with a short frame period, if parameters are included in every frame (as with the audio signal), it generally means that those parameters will be oversampled. In some implementations, it may be possible to not send metadata in every frame, but only update it every M frames (for example, up to M=4 in some cases). This generally reduces the average metadata bitrate.

[0053] In light of this, broadly speaking, the application of the techniques described herein can be applied to any parameter or side information in audio coding where the temporal correlation of parameters exceeds the codec stride. For example (but not limited to), the frequency-interleaved time-difference entropy coding procedure can be applied to parameters in Immersive Speech and Audio Services (IVAS) codecs, standardized by the Third Generation Partnership Project (3GPP®) for modeling spatial interactions, or in any parametric stereo coding technique that seeks to minimize a codec stride of less than 40 msec. However, as will be understood and recognized by those skilled in the art, while embodiments of the disclosure may be applied to Immersive Primary Ambisonics (FOA) codecs, the approaches described herein are generally applicable to any other suitable audio codecs with small stride or frame size (e.g., Higher-Order Ambisonics (HOA) codecs). Such codecs generally present certain specific challenges in encoding side information in the timely manner described above.

[0054] Referring here to Figure 1, a schematic (simplified) block diagram of an encoder / decoder ("codec") 100 for encoding and decoding a signal (bitstream) according to one embodiment of the present disclosure is shown. In particular, as will be understood by those skilled in the art, the illustrative example in Figure 1 shows a spatial reconstructor (SPAR) primary ambisonics (FOA) codec 100 for encoding and decoding an IVAS bitstream in FOA format. More specifically, as shown in the figure, the FOA codec 100 in Figure 1 is involved in both passive and active prediction, as will be understood and recognized by those skilled in the art.

[0055] Generally speaking, for encoding, an IVAS encoder may include a spatial analysis and downmix unit that receives audio data including, but not limited to, mono signals, stereo signals, binaural signals, spatial audio signals (e.g., multi-channel spatial audio objects), FOA, higher-order ambisonics (HOA), and any other suitable audio data. In some implementations, the spatial analysis and downmix unit may implement complex advanced coupling (CACPL) for analyzing / downmixing stereo / FOA audio signals and / or SPAR for analyzing / downmixing FOA audio signals. In other implementations, the spatial analysis and downmix unit may implement any other suitable format.

[0056] Referring back to Figure 1, the FOA codec 100 may include a SPAR FOA encoder 101, an enhanced voice services (EVS) encoder 105, a SPAR FOA decoder 106, and an EVS decoder 107. The SPAR FOA encoder 101 may be configured to convert the FOA input signal into a set of downmix channels and parameters, which are used to regenerate the input signal in the SPAR FOA decoder 106. Depending on the implementation, the downmix signal may vary from one to four channels, and the parameters (or sometimes called coefficients) may include, but are not limited to, prediction coefficients (PR), cross-prediction coefficients (C), and decorrelation coefficients (P). Note that SPAR is a process used to reconstruct an audio signal from a downmix version of an audio signal using the PR, C, and P parameters. This will be discussed in more detail below.

[0057] Depending on the number of downmix channels, one of the FOA inputs may always be sent untouched (e.g., the W channel as shown in the present example in Figure 1), while one to three other channels (e.g., the Y, Z, and X channels as shown in the present example in Figure 1) may be sent as residuals or entirely parametrically.

[0058] In particular, the prediction parameters may remain the same regardless of the number of downmix channels and can be used to minimize the predictable energy in the residual downmix channels. On the other hand, cross-prediction parameters can be used to further assist in regenerating fully parameterized channels from the residuals. Therefore, these parameters are not required in the case of the 1-channel and 4-channel downmixes. In the former case, there are no residual channels to predict, and in the latter case, there are no parameterized channels to predict. Furthermore, decorrelation parameters can be used to fill in the remaining energy not explained by the prediction and cross-prediction. Here again, the number of decorrelation parameters may depend on the number of downmix channels in each band.

[0059] The example in Figure 1 illustrates a typical embodiment of such a system and shows how these parameters apply on the decoder side. In particular, the exemplary implementation shown in Figure 1 represents a nominal two-channel downmix, where the representation of the W channel (W for passive prediction and W' for active prediction) is sent unmodified to decoder 106 along with a single predicted channel Y'. The cross-prediction coefficient (C) allows at least a portion of the parametric channel to be reconstructed from the residual channel when at least one channel is sent as a residual and at least one is sent parametrically, i.e., in the case of two-channel and three-channel downmixes. Thus, generally speaking, for a two-channel downmix, the C parameter allows a portion of the X and Z channels to be reconstructed from Y', and the remaining channel is reconstructed by a decorrelated version of the W channel, as described in more detail below. In the case of a three-channel downmix, the residual Y' and X' channels are used to reconstruct only Z.

[0060] Notably, as will be understood and recognized by those skilled in the art, in some exemplary implementations, W can be an active channel (or, in other words, have an active prediction; hereafter referred to as W'). For example (but not limited to), an active W channel that allows some kind of mixing of X, Y, and Z channels into the W channel may be defined as follows:

number

[0061] In the exemplary implementation shown in Figure 1, the SPAR FOA encoder 101 may include a (passive or active) predictor unit 102, a remix unit 103, and an extraction / downmix selection unit 104. In particular, the predictor 102 may receive FOA channels (W, Y, Z, X) in 4-channel B format and compute downmix channels (representations of W, Y', Z', X').

[0062] The extraction / downmix selection unit 104 can, for example, extract SPAR FOA metadata from the metadata payload section of the IVAS bitstream. The predictor unit 102 and the remix unit 103 then use the SPAR FOA metadata to generate remixed FOA channels (representations of W, S1', S2', and S3'), which may then be input to the EVS encoder 105 and encoded into an EVS bitstream, which may then be encapsulated into an IVAS bitstream sent to the decoder 106.

[0063] Referring to the SPAR FOA decoder 106, the EVS bitstream is decoded by the EVS decoder 107, resulting in several downmix channels (e.g., N_dmx = 2, where N_dmx is the number of downmix channels). In some implementations, the SPAR FOA decoder 106 may be configured to perform the inverse of the operation performed by the SPAR encoder 101. For example, in the example in Figure 1, the remixed FOA channels (representations of W, S1', S2', S3') can be reconstructed from the two downmix channels using the SPAR FOA spatial metadata. The remixed SPAR FOA channels can then be fed into the inverse mixer 111 to reconstruct the SPAR FOA downmix channels (representations of W, Y', Z', and X'). The predicted SPAR FOA channels can then be fed into the inverse predictor 112 to reconstruct the original unmixed SPAR FOA channels (W, Y, Z, X).

[0064] Note that in this two-channel example, decorrelator blocks 109-1 (dec1) and 109-2 (dec2) may be used to generate a decorrelated version of the W channel using a time-domain or frequency-domain decorrelator. The downmix channel and decorrelated channel may be used in combination with SPAR FOA metadata to parametrically reconstruct the X and Z channels. Block C 108 may refer to multiplying the residual channel by a 2x1 C coefficient matrix, thereby generating two cross-prediction signals that can be added to the parametrically reconstructed channel, as shown in the example in Figure 1. Furthermore, blocks P1 110-1 and P2 110-2 may refer to multiplying the decorrelator output by columns of a 2x2 P coefficient matrix, thereby generating four outputs that can be summed to the parametrically reconstructed channel, as shown in the example in Figure 1.

[0065] As described above, in some implementations, depending on the number of downmix channels, one of the FOA inputs may be sent untouched to the SPAR FOA decoder 106 (e.g., the exemplary W channel), while one to three of the other channels (Y, Z, X) may be sent to the SPAR FOA decoder 106 as residuals or entirely parametrically. The PR coefficient remains the same regardless of the number of downmix channels N_dmx and can be used to minimize the predictable energy in the residual downmix channel. The C coefficient can be used to further assist in reconstructing the fully parameterized channel from the residual. Therefore, the C coefficient may not be necessary in the case of 1 and 4 channel downmixes where there is no residual channel or parameterized channel to predict. The P coefficient is used to fill in the remaining energy not explained by the PR and C coefficients. The number of P coefficients generally depends on the number of downmix channels N in each band.

[0066] In some implementations, the SPAR PR coefficient (passive W only) is calculated as follows: Step 1. Using the prediction matrix composed of prediction coefficients, predict all side signals (Y, Z, X) from the main W signal as follows:

number

number

[0067] Step 2. Remix the W and predicted (Y', Z', X') signals in order from acoustically most significant to least significant. Here, "remix" means rearranging or rearranging the signals according to some methodology.

number

[0068] Step 3. Calculate the covariance of the 4-channel prediction and the downmix after remixing as follows:

number

number

[0069] For examples of 1 to 4-channel WS1S2S3 downmixes, d and u represent the following channels shown in Table 1: [Table 1]

[0070] The main interest for calculating SPAR FOA metadata is R dd , R ud , and R uu It is a quantity.

[0071] Step 4. R dd , R ud and R uu From this quantity, the codec 100 can determine whether it is possible to cross-predict the rest of the channel, which is fully parametric, from the residual channel sent to the decoder. In some possible implementations, the required extra C coefficient may be calculated as follows:

number

[0072] Step 5. Calculate the remaining energy of the parameterized channel that needs to be reconstructed by decorrelators 109-1 and 109-2 as follows:

number

[0073] In some possible implementations, the square root of the matrix is ​​normalized to Res uu This can be taken after setting the off-diagonal elements of the matrix to zero. P may also be a covariance matrix and therefore a Hermitian symmetric matrix. Thus, only parameters from the upper or lower triangle need to be sent to decoder 106. The diagonal elements may be real numbers, and the off-diagonal elements may be complex numbers. In some further possible implementations, the coefficients of P are the diagonal elements P d and off-diagonal element P o It can be further separated. In some implementations, only the diagonal elements of P are computed and sent to the decoder, and these can be computed as follows:

number

[0074] Here, the encoder side may need to quantize these parameters. In particular, given the dependencies between the three parameter types (i.e., PR, C, and P) as described above, the ordering (or sequence) of their calculation and quantization can generally be considered important for audio quality. According to this disclosure, three possible embodiments of a method for achieving this may be as follows:

[0075] 1. All-in-one In this embodiment, the decorrelator is generally not permitted to compensate for quantized prediction errors. More specifically, in the first step, the parameters PR, then C, then P are calculated without quantization as illustrated above. Then, the parameters PR, C, and P are all quantized according to a quantization strategy or scheme (for example, based on a preferred quantization range and / or quantization level, as understood by those skilled in the art).

[0076] 2. Cascade Generally speaking, this particular embodiment allows for accurate and cross-predictive data, and the decorrelator may compensate for errors from quantization. More specifically, in the first step, the parameter PR is calculated and then quantized. Then, from the quantized PR parameter, the parameter C is calculated and then quantized. Finally, from the quantized C parameter, the parameter P is also calculated and then quantized.

[0077] 3. Partial Cascade Generally speaking, this particular embodiment minimizes the P-coefficient, thereby allowing for accurate mutual predictions, but does not allow the inverse correlator to compensate for prediction errors. More specifically, in the first step, parameters PR, C, and P are calculated without quantization, as in the all-in-one embodiment described above, and then the P parameter is quantized. Subsequently, the PR parameter is also quantized. Finally, the C parameter is recalculated from the quantized PR parameter and then quantized.

[0078] In each of the embodiments described above, the downmix (including residuals) can always be calculated using quantized prediction coefficients.

[0079] As will be understood and recognized by those skilled in the art, the quantization process itself can be defined by a preferred (quantization) range. For example, a range [-a, a] may be defined for some parameters (e.g., off-diagonal elements of parameters PR, C, and P), while another range [0, a] may be defined for other parameters. Furthermore, several quantization levels can also be defined that should be uniformly distributed between these endpoints, i.e., parameter types (e.g., PR, C, P) d , P o For each of these, various constraints and step sizes can be constructed or defined. Furthermore, in some implementations, if the parameters are complex numbers, the real and imaginary parts may be quantized within the same / different ranges and number of steps according to the parameter distribution.

[0080] Possible implementations of the quantization process can be defined as follows:

number

[0081] In some possible implementations, it may be desirable to choose an odd number for the quantization level (i.e., qlvl) to ensure that, for example, double-sided parameters, the quantization point is available as 0. This will be understood by those skilled in the art.

[0082] As mentioned above, it may be significant to note that the example in Figure 1 generally illustrates an implementation of passive prediction (i.e., W channel). However, as will be understood and recognized by those skilled in the art, active prediction may be applied in some other possible embodiments. Generally speaking, active W channel can allow some kind of mixing of at least some of the X, Y, and Z channels into the W channel, and such active prediction can typically be used in the case of a one-channel downmix. Thus, in the case of passive prediction, there is generally no mixing of the X, Y, and Z channels into the W channel.

[0083] Figure 2 is a flowchart illustrating an example of a method 200 for encoding metadata for an input signal frame by frame, according to one embodiment of the present disclosure. The method 200 described herein may be applied, for example, to a codec 100 (or any other suitable codec) as shown in Figure 1. The metadata may be computed / calculated (e.g., extracted) from an input (audio or video) signal using a suitable codec (encoder / decoder). Generally speaking, the metadata may be used to assist in the regeneration of the input signal on the decoder side. The metadata may include several parameters that are computed from the input signal and are at least partially interrelated. That is, at least some of the parameters of the input signal may be computed (e.g., generated or regenerated) depending on at least some of the other parameters, and as a result, depending on various circumstances, not all parameters always have to be simply transmitted.

[0084] Method 200 may be performed sequentially and iteratively, for example, by using a loop process (described in detail below) for each frame of the input signal. In particular, Method 200 (more precisely, the loop process) begins in step S210 by calculating parameters and determining a processing strategy from several processing strategies for quantization.

[0085] Once a processing strategy is determined (for example, selected) in step S210, the loop process proceeds to step S220, where it calculates and quantizes parameters based on the determined processing strategy and obtains quantized parameters.

[0086] Subsequently, in step S230, the (quantized) parameters are encoded accordingly, and then the (resulting) bitrate is estimated (e.g., calculated) from the encoded parameters, and in step S240, a determination is made based on the estimated bitrate together with at least one target bitrate threshold (e.g., a predefined or preconfigured).

[0087] If the bitrate threshold is met, for example, if the estimated bitrate is less than or equal to the bitrate threshold, method 200 terminates the processing loop. Otherwise, the loop returns to step S210 and continues through steps S210 to S240. In particular, upon re-entering the loop, a new processing strategy may be determined to satisfy the bitrate threshold target.

[0088] As will be understood and recognized by those skilled in the art, multiple processing strategies for calculating and quantizing parameters can be provided in any preferred manner, for example, predefined or preconfigured. Thus, the processing strategy can also be determined from multiple processing strategies in any preferred manner. For example, a preferred processing strategy may be selected from multiple processing strategies depending on the (current) bitrate requirement, such that the bitrate resulting after performing calculation, quantization, and encoding (e.g., with or without entropy coding) based on the thus selected processing strategy satisfies the (current) bitrate requirement.

[0089] Since loop processes are generally directed towards (particularly) quantization-related operations, in some cases a loop process may also be called a quantization loop (or simply a loop). Similarly, since processing strategies are generally directed towards (particularly) quantization-related operations, processing strategies may also be called quantization strategies (or, in some other cases, interchangeably as quantization schemes). Furthermore, it should be noted that the encoding process can use any suitable encoding procedure, including but not limited to entropy encoding or non-entropy encoding (e.g., base2 encoding). Of course, depending on various implementations and / or requirements, any other suitable encoding mechanism may be employed.

[0090] Specifically, each of the processing strategies may include a first instruction indicating the ordering (or sequence) related to the computation and quantization of individual parameters. That is, the first instruction may include sequence information indicating when and in what order the individual parameters are computed and quantized. For example (but not limited to), the first instruction may include information indicating that all parameters are computed before any of the parameters are quantized.

[0091] Here, the loop process will be explained in more detail with reference to the examples shown in Figures 3 and 4. As shown above, in codecs with short strides or frame updates, parameters can be oversampled if they were all included in every frame. Therefore, the main focus of this disclosure is to propose a mechanism that minimizes side information as much as possible while preserving a short frame update rate for audio essence and parameters.

[0092] To address the above issues, and in particular to evaluate the expansion of side information, broadly speaking, the inventors of this disclosure propose a mechanism that broadly incorporates time-difference estimates for parameters in some (frequency) bands together with non-difference estimates for parameters in other (frequency) bands. The proposed approach interleaves which bands are time-difference encoded and which are non-difference encoded, so that all bands are periodically refreshed in non-difference calculations without the need for complete parameter updates. The core concept is that as the frame size decreases, the inter-frame correlation of parameters increases, and therefore the encoding gain from time-difference encoding of parameters can increase.

[0093] In addition to frequency interleaving in time-difference coding, the concept of a sequential, iterative, and stepwise approach to selecting the optimal parameter quantization scheme is also introduced, which searches for the "best" (or most optimal) quantization scheme from multiple alternatives. In this case, the terms "best" or "most optimal" do not necessarily refer to the quantization scheme with the lowest parameter bitrate, but rather to one that relaxes the decoder's state.

[0094] For example, the use of time-diffusion encoding can generally present drawbacks. This is primarily due to the introduction of frame-to-frame state, which can be problematic when the audio stream may suffer packet loss during transmission. In this case, both audio and parameters may be lost, and any parameters updated by time-diffusion encoding may experience multiple subsequent frames with potential artifacts. Decoder mitigation of the aforementioned problem is not generally addressed in this disclosure. Instead, this problem is generally addressed (mitigated) by selecting a suitable quantization scheme that will limit this behavior as much as possible. Broadly speaking, encoding (encoder-side) mitigation generally involves a sequential, iterative selection process for quantization and entropy coding, which attempts to minimize the extent to which artifacts resulting from packet loss can be introduced due to the use of time-diffusion encoding.

[0095] Now, referring back to the figures, Figure 3 is a schematic flowchart illustrating an example of a processing loop 300 according to one embodiment of the present disclosure. The processing loop 300 begins in step S310, in which a first bitrate (hereinafter referred to as b1) is calculated (or estimated). In some possible implementations, the entropy of the non-differential and / or frequency-differential quantized parameters is estimated for all frames. In some other possible implementations, the first bitrate b1 may be calculated as the minimum of non-differential and frequency-differential coding schemes encoded by a (trained) entropy encoder (e.g., Huffman coding or arithmetic coding).

[0096] In step S320, the first bitrate b1 is compared with the target bitrate (hereinafter referred to as t). If the estimated parameter bitrate b1 is within (or less than) the target bitrate t, the processing loop terminates. As a result, the parameter is encoded so that any additional available bits are supplied to the audio encoder to increase the bitrate of the audio essence.

[0097] If step S320 fails (i.e., the estimated bitrate b1 is greater than the target bitrate t), then in step S330, a second bitrate of the quantized parameter (hereinafter referred to as b2) is calculated. In some possible implementations, the second bitrate b2 may be calculated in a non-differential manner without entropy coding (for example, by using base2 coding).

[0098] Next, in step S340, the second bitrate b2 is compared with the target bitrate t. If the second bitrate b2 is within (or less than) the target bitrate t, the processing loop terminates.

[0099] Otherwise, in step S350, a third bitrate of the parameter (hereinafter referred to as b3) is calculated. In some possible implementations, the third bitrate b3 may be calculated by time-difference coding using a (trained) entropy encoder. In some further possible implementations, a subset of the parameter values ​​in the current frame may be quantized and then subtracted from the quantized parameter values ​​in the previous frame to calculate the difference in quantized parameter values ​​and entropy.

[0100] In step S360, if the calculated bitrate b3 is less than or equal to the threshold t, the processing loop terminates, the parameters are encoded at the supplied bitrate, and any extra bits are supplied for use in encoding the audio.

[0101] Otherwise, various measures may be implemented in step S370 to ultimately satisfy the target bitrate threshold t.

[0102] For example, in some possible implementations, a second, coarser processing strategy (quantization strategy) may be selected from among several processing strategies. In such cases, as will be understood and recognized by those skilled in the art, the quantization process may include several levels of quantization with increasing coarseness, such as fine, medium, coarse, and very coarse quantization strategies. After determining (e.g., selecting) a coarser quantization strategy, the processing loop repeats steps S310 to S360.

[0103] In some other possible implementations, the step of reducing the number of frequency bands may be performed in S370. Then, the above steps (i.e., steps S310-S360) may be repeated with the reduced bandwidth configuration. This generally reduces the total number of parameters to quantize and can give a lower bitrate for (at least) some frames.

[0104] Alternatively or additionally, some further implementations may also perform the step of freezing (i.e., reusing) parameters in the bandwidth from the previous frame. This essentially prevents the parameters from changing over time, thereby giving reduced entropy for time-difference entropy coding. For example, when encoding with coding scheme 4a, as shown in Table 2 (described in detail below), parameters in frequency bands 2, 6, and 10 may be frozen. This typically reduces entropy, makes no changes to the decoder or entropy coding scheme, and has only a slight impact on quality. It should be noted that the examples of 2, 6, and 10 above are merely illustrative examples, and there can be many bandwidth configurations that can be frozen over multiple frames, as will be understood and recognized by those skilled in the art. For example, if all frequency bands are frozen over a period of two frames, the encoder can transmit half the bandwidth in frame N and the remaining half in frame N+1 (thus reducing the total number of parameters transmitted), which generally means that the decoder gets all (e.g., 12) updated frequency bands every other frame. In such cases, if one frame is lost, there is generally an option to extrapolate from the last two good frames. When recovering from packet loss, it is possible to interpolate the bandwidth received in a given frame.

[0105] In particular, if the loop terminates at step x, the final parameter bitrate is the bitrate calculated at that step x.

[0106] Furthermore, in some implementations, it may be possible (or even desirable) to consider designing the bitrate b3 with the coarsest quantization strategy (of a given set of quantization strategies available for quantizing the parameter) so that it is guaranteed to be smaller than the target bitrate threshold t. In such cases, it may be guaranteed that there is always a solution to fit the parameter bitrate within the target bitrate t.

[0107] Figure 4 is a schematic flowchart illustrating an example of a processing loop 400 according to another embodiment of the present disclosure. In particular, the same or similar reference numerals in loop 400 in Figure 4 generally indicate the same or similar elements in loop 300, as shown in Figure 3, so a description of the repetition may be omitted for the sake of brevity.

[0108] In particular, the processing loop in Figure 4 may be especially suitable when two bitrate thresholds (represented as a target bitrate threshold t1 and a maximum bitrate threshold t2) are used, unlike the single target bitrate threshold scenario shown in Figure 3. Roughly speaking, the target bitrate threshold t or t1 may be considered a good target or goal to achieve, while the maximum bitrate threshold t2 may simply be considered a "hard" threshold that should not be exceeded.

[0109] More specifically, steps S410 to S470 are the same as the steps in Figure 3 (i.e., steps S310 to S370), and a detailed explanation of their repetition may be omitted for brevity. However, instead of switching directly to step S470 if the condition in S460 is not met, an additional step S461 is inserted by calculating a fourth bitrate (b4) as the minimum of the bitrates b1, b2, and b3. Then, in step S462, the fourth bitrate b4 is compared with the maximum bitrate threshold t2.

[0110] If the fourth bitrate b4 is less than or equal to the maximum bitrate threshold t2, the processing loop 400 terminates. Otherwise, the processing loop 400 proceeds to step S470 (which is essentially the same as step S370 in Figure 4), and then repeats steps S410 through S462.

[0111] Similar to Figure 3, if the loop terminates at step x, the final parameter bitrate is the bitrate calculated at that step x.

[0112] Furthermore, in some implementations, it may be possible (or even desirable) to consider designing the bitrate b3 using the coarsest quantization strategy (among several given quantization strategies available for quantizing the parameters) so that it is guaranteed to be less than the maximum bitrate threshold t2. In such cases, it may be guaranteed that there always exists a solution that keeps the parameter bitrate within the maximum bitrate t2.

[0113] In summary, steps S310, S330, and S350 in Figure 3, and the corresponding steps S410, S430, and S450 in Figure 4, generally do not affect audio quality. However, step S461 in Figure 4 degrades quality by affecting both the audio bitrate and the parameter bitrate. Furthermore, any of the possible techniques described in step S370 in Figure 3 and step S470 in Figure 4 (e.g., transitioning to coarser quantization, bandwidth reduction by reducing frequency resolution, bandwidth reduction by reducing temporal resolution, etc.) fundamentally negatively impact quality. Therefore, the steps in the examples in Figures 3 and 4 are ordered to minimize quality degradation or to address constraints in other areas. Broadly speaking, the methods described herein tend to select one or more of the exemplary techniques described above to strike a balance between metadata bitrate reduction and perceived quality.

[0114] There are also additional considerations that go into the specific ordering of the above steps and, potentially, the reasons for the two target parameters and bitrates (i.e., t1 and t2).

[0115] In particular, step-by-step ordering allows a procedure to terminate when constraints are met. This generally reduces the computational load when calculations are performed serially, because typically, not all available steps are followed.

[0116] Furthermore, ordering also allows for implicit preferences among choices. For example, ordering non-differential entropy coding as the first step generally implies that this choice is preferable if the constraints are met. This is an encoder relaxation to minimize the states for improving quality under the packet loss condition.

[0117] Furthermore, the possibility of using two targets (t1 and t2) generally allows for the ability to trade off audio bitrate and parameter bitrate with greater control.

[0118] Here, we will explain in more detail the concept of interleaving to achieve time-difference coding.

[0119] Table 2 shows several possible implementations for managing interleaving in time-difference entropy coding. [Table 2]

[0120] In this particular example, five configurations are generally proposed for metadata bitstream coding, each consisting of 12 (frequency) bands. More specifically, bands designated by 0 are coded non-differentially, and bands designated by 1 are coded time-differentially (i.e., the parameters are quantized and subtracted from the quantization parameters of the previous frame).

[0121] As described in this example, the parameter bitrate for each frame is first evaluated by non-differential (i.e., base) encoding by quantizing the parameters (see, for example, step S410 or S510). Then, in step S450 or S550, a time-difference encoding scheme is selected based on the encoding scheme of the previous frame (if so requested).

[0122] Table 3 below shows an example of the mapping from the encoding scheme of the previous frame to the time-difference encoding scheme of the current frame. [Table 3]

[0123] In particular, in this example, the term “base” used in Table 3 generally refers to a non-differential coding scheme. Thus, as can be seen from Table 3, time-differential coding always cycles through 4a-4d (and back again). It is possible to continue the cycle without ever requiring the implementation of non-differential coding. In this particular example, the maximum memory or “state” of the codec is the current frame and three past frames (i.e., a total of four frames). Of course, as will be understood and recognized by those skilled in the art, the number of configurations (five) and (twelve) (frequency) bands, etc., are merely used as examples for illustrative purposes, and any other suitable number can be used depending on various implementations and / or requirements. Similar or comparable arguments apply to switching between coding schemes, as shown in Table 3, in which any suitable technique can be employed.

[0124] In particular, when different quantization schemes are selected, the indices from previous frames quantized with the different scheme may first be mapped to the indices of the current frame. Generally speaking, the mapping step may be necessary to allow time-difference coding of parameters when, for example, the number of quantization levels changes from one frame to the next, thereby allowing time-difference coding between frames without resorting to the need to send a non-difference frame each time the quantization scheme changes.

[0125] As a possible example, index mapping can be performed based on the following formula:

number

[0126] As a simple example, let's set the quantization range from 0 to 2 and the previous quantization level to 11. In the case of uniform quantization, this generally means that each quantization step is 0.2. Furthermore, if the current quantization level is 21, then each quantization step will be 0.1 in uniform quantization. Based on these assumptions, if the quantized value in the previous frame was 0.4, then across 11 uniform quantization levels, the previous index will be as follows: prev You will get = 2. The mapping provides a quantized index of the metadata from the previous frame as if it were quantized using the quantization level of the current frame. So in this example, if the quantization level in the current frame is 21, the quantized value 0.4 will be index curr This is mapped to =4. Once the mapped index is calculated, the difference between the index of the current frame and the index of the previous frame is calculated, and this difference is encoded. As will be understood and recognized by those skilled in the art, a similar or similar approach may be applied to frequency difference coding if necessary.

[0127] Of course, depending on the various implementations and / or requirements, any other suitable mapping method (for example, by using a lookup table) may be adopted.

[0128] Furthermore, as described above, a single metadata parameter can be quantized from a continuous numerical value to an index representing a discrete value. In non-differential coding, the information encoded for that metadata parameter directly corresponds to its index. In time-differential coding, the information encoded is the difference between the index of that metadata parameter from the current frame and the index of the same metadata parameter from the previous frame. As will be understood and recognized by those skilled in the art, the above general concept of time-differential coding can be further extended, for example, to multiple frequency bands. Thus, metadata parameters can similarly be extended, as appropriate, to multiple parameters corresponding to multiple frequency bands, for example. Frequency-differential coding follows a similar principle, except that the encoded difference is the difference between the metadata of one frequency band of the current frame and the metadata of the other frequency band of the current frame (not the difference between the current frame and the previous frame, as in time-differential coding). As a simple example (not as an limitation), if a0, a1, a2, and a3 represent the parameter indices in four frequency bands of a particular frame, then in an exemplary implementation, the frequency difference indices could be a0, a0-a1, a1-a2, and a2-a3. As will be understood by those skilled in the art, the general idea behind (time and / or frequency) difference coding is that since metadata can typically change slowly from frame to frame or from frequency band to frequency band, even if the original values ​​of the metadata were large, the difference between the metadata and the metadata of the previous frame, or between the metadata and the metadata of another frequency band, is likely to be small. This is advantageous because, generally, parameters with statistical distributions that tend towards zero can be coded using fewer bits. Thus, even if some exemplary implementations refer to time difference coding temporarily or simply, those skilled in the art will understand that frequency difference coding can also be applied to it (possibly with minor preferred adaptations).

[0129] Some further possible examples of the present disclosure relate to a process for processing an input audio signal represented by subbands to generate a downmixed signal and associated metadata, which can be performed by one or more processors. This process may include determining a downmix matrix and associated metadata for each subband; and remixing each of the subbands according to the downmix matrix to generate the downmixed signal. One or more quantization strategies and one or more coding strategies can be used to encode the metadata, given a target and / or maximum metadata bitrate limit.

[0130] In some implementations, the process can include non-differential entropy coding for all subbands. This process can further include frequency-differential entropy coding for all subbands.

[0131] This process may further include combining time-difference encoding and frequency interleaving of quantized parameters corresponding to selected subbands for a low-latency audio codec, as detailed above. This process may further include non-entropy coding of subband metadata. The process iteratively repeats the steps of finding a suitable coding strategy that satisfies bitrate and audio quality requirements, and also reducing decoder states. This process may further include reducing frequency resolution by reducing the number of subbands to which spatial metadata is encoded, for example, from 12 to 6. This process may also include reducing temporal resolution by time-fixing (or freezing) one or more subband metadata so that the subband metadata does not need to be transmitted. This process may include using multiple quantization strategies, each strategy being a combination of quantization levels for various spatial metadata parameters. This process may further include selecting among these quantization strategies to ensure that the bitrate target is met. This process may include iteratively repeating the steps of finding a suitable quantization scheme that satisfies bitrate and audio quality requirements. The successive iterative method focuses on obtaining a desired metadata bitrate with a desired quantization scheme, minimum computational complexity, and reduced decoder state. If the desired quantization level does not fit within the desired bitrate range, it falls back to a (e.g., coarser) quantization scheme, ensuring minimal impact on audio quality.

[0132] In some implementations, mapping indices from previous frames quantized to a different number of levels to those of the current frame allows for inter-frame time-difference coding without relying on the need to send non-differential frames each time a different quantization level is required.

[0133] In various implementations, quantization (the conversion of continuous values ​​to discrete indices for encoding) may involve manipulating the order of successive metadata coefficient calculations and quantization to determine the best values ​​for the coefficients according to current needs.

[0134] A computing device implementing the above-described technology may have the following exemplary architecture. Other architectures are also possible, including architectures with more or fewer components. In some implementations, the exemplary architecture includes one or more processors (e.g., dual-core Intel® Xeon® processors), one or more output devices (e.g., LCDs), one or more network interfaces, one or more input devices (e.g., mouse, keyboard, touch-sensitive display), and one or more computer-readable media (e.g., RAM, ROM, SDRAM, hard disk, optical disc, flash memory). These components can communicate and exchange data through one or more communication channels (e.g., buses) which can utilize various hardware and software to facilitate the transfer of data and control signals between components.

[0135] The term “computer-readable medium” refers to a medium involved in providing instructions to a processor for execution, including but not limited to non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory), and transmission media. Transmission media include, but are not limited to, coaxial cables, copper wires, and optical fibers.

[0136] The computer-readable medium may further include an operating system (e.g., the Linux® operating system), a network communications module, an audio interface manager, an audio processing manager, and a live content distributor. The operating system may be multi-user, multi-processing, multi-tasking, multi-threaded, real-time, etc. The operating system performs basic tasks including, but not limited to, recognizing inputs from the network interface 706 and / or device 708 and providing outputs to them; tracking and managing files and directories on the computer-readable medium (e.g., memory or storage device); controlling peripheral devices; and managing traffic on one or more of the communication channels. The network communications module includes various components for establishing and maintaining network connectivity (e.g., software for implementing communication protocols such as TCP / IP and HTTP).

[0137] The architecture can be implemented using parallel processing or peer-to-peer infrastructure, or in a single device with one or more processors. The software may include multiple software components or a single code body.

[0138] The features described above can be advantageously implemented in one or more computer programs executable on a programmable system, which includes a data storage system, at least one input device, and at least one output device, coupled to at least one programmable processor that receives data and instructions from and transmits data and instructions to it. A computer program is a set of instructions that can be used directly or indirectly within a computer to perform some activity or to produce some result. A computer program can be written in any form of programming language (e.g., Objective-C, Java), including languages ​​that are compiled or interpreted, and it can be deployed in any form, including as a standalone program or as a module, component, subroutine, browser-based web application, or other unit suitable for use in a computing environment.

[0139] Suitable processors for executing instruction programs include, for example, both general-purpose and dedicated microprocessors, and one of the sole processors or multiple processors or cores of any type of computer. Generally, a processor receives instructions and data from read-only memory or random-access memory or both. Essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer also includes, or is operationally coupled to, one or more mass storage devices for storing data files. Such devices include magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Suitable storage devices for materially embodying computer program instructions and data include, for example, semiconductor memory such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and all forms of non-volatile memory, including CD-ROM and DVD-ROM disks. Processors and memory can be supplemented by or incorporated into ASICs (Application-Specific Integrated Circuits).

[0140] To provide user interaction, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or retinal display device for displaying information to the user. The computer may have a touch surface input device (e.g., a touchscreen) or a keyboard and a pointing device such as a mouse or trackball that allows the user to provide input to the computer. The computer may have a voice input device for receiving voice commands from the user.

[0141] These features can be implemented in a computer system with backend components, such as a data server, or middleware components, such as an application server or internet server, or frontend components, such as a client computer with a graphical user interface or internet browser, or any combination thereof. The components of the system can be connected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include, for example, computers and networks that form a LAN, WAN, and the internet.

[0142] A computing system can include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The client-server relationship arises thanks to computer programs running on each computer that have a client-server relationship with each other. In some embodiments, the server sends data (e.g., an HTML page) to the client device (for example, to display data and receive user input from a user interacting with the client device). Data generated on the client device (e.g., the results of user interaction) can be received by the server from the client device.

[0143] One or more computer systems can be configured to perform specific actions by installing software, firmware, hardware, or a combination thereof on the system that causes the system to perform those actions while in operation. One or more computer programs can be configured to perform specific actions by including instructions that cause a data processing device to perform an action when executed by that device.

[0144] This specification includes many specific implementation details, which should not be interpreted as limitations on the scope or claims of any particular invention, but rather as descriptions of features specific to specific embodiments of a particular invention. Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any preferred subcombination. Furthermore, features are described above as acting in certain combinations, and may even be initially described in the claims as such, but one or more features from a claimed combination may, in some cases, be extracted from the combination, and the claimed combination may be directed towards subcombinations or variations of subcombinations.

[0145] Similarly, while the drawings show operations in a specific order, this should not be understood as requiring that such operations be performed in a specific order or sequentially, or that all exemplified operations be performed, in order to achieve the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged in multiple software products.

[0146] Unless otherwise specified, as will be apparent from the following discussion, any discussion throughout this disclosure using terms such as “processing,” “computing,” “calculating,” “determining,” and “analyzing” is understood to refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulates and / or transforms data, expressed as physical quantities, such as electronic quantities, into other data, also expressed as physical quantities.

[0147] Throughout this disclosure, any reference to “one exemplary embodiment,” “several exemplary embodiments,” or “a certain exemplary embodiment” means that any particular feature, structure, or characteristic described in relation to that exemplary embodiment is included in at least one exemplary embodiment of this disclosure. Therefore, the occurrence of the phrases “in one exemplary embodiment,” “in several exemplary embodiments,” or “in a certain exemplary embodiment” throughout this disclosure does not necessarily refer to the same exemplary embodiment. Furthermore, any particular feature, structure, or characteristic may be combined in any preferred manner in one or more exemplary embodiments, as will be apparent to those skilled in the art from this disclosure.

[0148] Where used herein, unless otherwise specified, the use of ordinal adjectives such as “first,” “second,” “third,” etc., to describe common objects simply indicates that different instances of similar objects are being referred to, and is not intended to imply that the objects described in this way must be in a given sequence, temporally, spatially, in rank, or in any other way.

[0149] In the claims and descriptions herein, the terms having, comprising, or including are open terms that include at least the listed elements / features but do not exclude others. Therefore, when used in the claims, the term having should not be interpreted as being limited to the listed means, elements, or steps. For example, an apparatus including A and B should not be limited to an apparatus consisting only of elements A and B. As used herein, the terms including and comprising are open terms that include at least the listed elements / features but do not exclude others. Therefore, including is synonymous with having and means having.

[0150] In the above description of the exemplary embodiments of this disclosure, it should be understood that, for the purpose of improving the flow of the disclosure and aiding in the understanding of one or more of the various aspects of the invention, various features of the disclosure may be grouped together in a single exemplary embodiment, drawing, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claims require more features than are expressly described in each claim. Rather, as reflected in the following claims, the aspects of the invention are fewer than all the features of the single exemplary embodiment disclosed above. Thus, the claims following this paper are expressly incorporated herein, and each claim stands alone as a separate exemplary embodiment of this disclosure.

[0151] Furthermore, while some exemplary embodiments described herein include some features included in other exemplary embodiments but not others, as will be understood by those skilled in the art, combinations of features from different exemplary embodiments are within the scope of this disclosure and are intended to constitute different exemplary embodiments. For example, in the following claims, any of the exemplary embodiments described in the claims may be used in any combination.

[0152] Numerous specific details are described in the descriptions provided herein. However, it is understood that exemplary embodiments of this disclosure may be carried out without these specific details. On the other hand, well-known methods, structures, and techniques are not described in detail so as not to confuse the understanding of this paper.

[0153] Therefore, while the best mode of this disclosure is described, those skilled in the art will recognize that other further modifications may be made without departing from the spirit of the disclosure and will be entitled to request all such changes and modifications that fall within the scope of this disclosure. For example, each of the formulas given above simply represents a procedure that may be used. Functions may be added or removed from the block diagram, and operations may be swapped between function blocks. Steps may be added or removed from the methods described within the scope of this disclosure.

[0154] Various aspects and implementations of this disclosure can also be understood from the following enumerated example embodiments (EEEs) that are not claims.

[0155] [EEE1] A method for processing an input audio signal represented by subbands to generate a downmixed signal and associated metadata, the method being: For each subband, determine the downmix matrix and associated metadata; This includes remixing each of the subbands according to the downmix matrix to generate the downmixed signal. method. [EEE2] A method of EEE1, wherein the metadata is encoded using one or more quantization strategies and one or more coding strategies, given a target and / or maximum metadata bitrate limit. [EEE3] A method for EEE2 that includes non-differential entropy coding of all subbands. [EEE4] A method of EEE3, comprising combining time-difference encoding and frequency interleaving of quantized parameters corresponding to selected subbands for a low-latency audio codec. [EEE5] A method of EEE4, including non-entropy coding of subband metadata. [EEE6] The method of EEE5, wherein steps 3) to 5) are iterated sequentially to find a suitable encoding strategy that meets the requirements for bitrate and audio quality, and to reduce the decoder state. [EEE7] A method of EEE6, which includes reducing the number of bandwidths transmitted by combining metadata in subbands. [EEE8] A method of EEE7, comprising fixing one or more subband metadata for a period of time so that subband metadata does not need to be transmitted. [EEE9] A method of EEE8, which includes using multiple quantization levels for a given metadata to ensure that the bitrate target is met. [EEE10] The method according to EEE9, wherein the steps of EEE3 to EEE9 are sequentially repeated to find a suitable quantization scheme to meet bitrate and audio quality requirements. [EEE11] The method of EEE3 or 9, wherein the mapping of indices from previous frames quantized to a different number of levels to those of the current frame allows for inter-frame time-difference coding without relying on the need to send non-differential frames each time a different quantization level is required. [EEE12] The method according to any one of EEE1 to 11, wherein the quantization includes determining the best values ​​for the coefficients according to current needs by manipulating the order of calculation and quantization of successive metadata coefficients. [EEE13] With one or more processors; The system includes a non-temporary computer-readable medium that, when executed by one or more processors, stores instructions that cause one or more processors to perform the operations described in any one of EEE1 to 12. system. [EEE14] A non-temporary computer-readable medium that, when executed by one or more processors, stores instructions that cause the one or more processors to perform the operations described in any one of EEE1 to 12.

[0156] Several aspects are described below. [Aspect 1] A method for encoding metadata about an input signal frame by frame, wherein the metadata includes a plurality of parameters that are at least partially interrelated and can be calculated from the input signal, and the method applies to each frame: Using a loop process: The steps include determining a processing strategy from among multiple processing strategies for calculating and quantizing the aforementioned parameters; The steps include: calculating the parameters based on the determined processing strategy, quantizing them, and obtaining the quantized parameters; The steps include encoding the quantized parameters and This includes performing the following sequentially and repeatedly: Each of the aforementioned processing strategies includes a first instruction that indicates the ordering related to the calculation and quantization of individual parameters; The processing strategy is determined based on at least one bitrate threshold. method. [Aspect 2] The method according to embodiment 1, wherein the processing strategy is determined such that the bitrate of the encoded quantized parameters is less than or equal to a bitrate threshold. [Aspect 3] The method according to embodiment 1 or 2, wherein each of the plurality of processing strategies further includes a second instruction indicating information for performing the quantization of the parameters. [Aspect 4] The method according to embodiment 3, wherein the information for performing the quantization of the parameters includes the respective quantization range and / or quantization level for the plurality of parameters. [Aspect 5] The method according to any one of embodiments 1 to 4, wherein the encoding of the parameters includes time and / or frequency difference coding. [Aspect 6] The method according to any one of embodiments 1 to 5, wherein the processing strategy determined for the current frame differs from the processing strategy determined for the previous frame, and the encoding of the parameters includes time-difference coding across the different processing strategies. [Aspect 7] The method according to any one of embodiments 1 to 6, wherein the first instruction includes information indicating that all parameters are computed before quantization. [Aspect 8] The method according to any one of embodiments 1 to 6, wherein the first instruction includes information indicating that the parameters are calculated individually and then sequentially quantized; and at least one of the plurality of parameters is calculated based on another quantized parameter of the plurality of parameters. [Aspect 9] The method according to any one of embodiments 1 to 6, wherein the first instruction includes information indicating that all parameters are computed before any parameter is quantized; at least one of the parameters is recalculated based on another quantized parameter, and the recalculated parameter is quantized. [Aspect 10] This method involves encoding the quantized parameters before: This further includes mapping the indices of quantized parameters from the previous frame to those of the current frame. The method described in Embodiment 6 or any one of Embodiments 7 to 9 when referring to Embodiment 6. [Aspect 11] The at least one bitrate threshold includes a target bitrate threshold, and the loop process is: The processing strategy involves the steps of: quantizing and encoding the parameters using an entropy encoder in a non-differential and / or frequency-differential manner; The steps include: estimating the first parameter bitrate for the encoded parameters; The process includes the step of terminating the loop process if the first parameter bitrate is less than or equal to the target bitrate threshold, The method according to any one of embodiments 1 to 10. [Aspect 12] The aforementioned loop process is: If the first parameter bitrate is greater than the target bitrate threshold: The steps include: quantizing and encoding the parameters in a non-differential manner without entropy according to the processing strategy described above; The steps involve estimating a second parameter, the bitrate, for the encoded parameters; The process further includes the step of terminating the loop process if the second parameter bitrate is less than or equal to the target bitrate threshold, The method according to embodiment 11. [Aspect 13] The aforementioned loop process is: If the second parameter bitrate is greater than the target bitrate threshold: The process involves the steps of quantizing and encoding the parameters using the entropy encoder in a time-difference manner according to the processing strategy described above; The steps involve estimating a third parameter, the bitrate, for the encoded parameters; The process further includes the step of terminating the loop process if the third parameter bitrate is less than or equal to the target bitrate threshold, The method described in Embodiment 12. [Aspect 14] The method according to embodiment 13, wherein the time-difference quantization and encoding are performed on a subset of the parameters in a manner that is frequency-interleaved with respect to the previous frame. [Aspect 15] The method according to aspect 13 or 14, wherein the time-difference quantization and encoding is performed by cycling through several frequency-interleaved time-difference coding schemes such that, for each cycle, a different subset of the parameters is quantized and encoded into a time-difference expression, while the remaining parameters are quantized and encoded non-difference-wise. [Aspect 16] The aforementioned determined processing strategy is the first processing strategy, and the loop process is: If the third parameter bitrate is greater than the target bitrate threshold: The second processing strategy is determined from the plurality of processing strategies such that the bitrate obtained by applying the second processing strategy is expected to be lower than that obtained by using the first processing strategy; Further comprising repeating the steps of the loop process according to embodiments 11 to 13, The method described in any one of embodiments 13 to 15. [Aspect 17] The parameter is expressed in a first number of frequency bands, and the loop process is: If the third parameter bitrate is greater than the target bitrate threshold: The number of frequency bands representing the aforementioned parameters is reduced to a second number smaller than the first number, thereby reducing the total number of parameters to be quantized and encoded; Further comprising repeating the steps of the loop process according to embodiments 11 to 13, The method described in any one of embodiments 13 to 15. [Aspect 18] The parameter is expressed in a first number of frequency bands, and the loop process is: If the third parameter bitrate is greater than the target bitrate threshold: In the current frame, reuse parameters in one or more frequency bands from the previous frame; Further comprising repeating the steps of the loop process according to embodiments 11 to 13, The method described in any one of embodiments 13 to 15. [Aspect 19] The at least one bitrate threshold further includes a maximum bitrate threshold that is greater than the target bitrate threshold, and the loop process is: Before determining the second processing strategy, reducing the number of frequency bands, or reusing the parameters, Obtain the minimum value among the first, second, and third parameter bitrates; If the minimum value is less than or equal to the maximum bitrate threshold, the loop process is further terminated. The method described in any one of embodiments 16 to 18. [Aspect 20] The method according to any one of embodiments 1 to 19, wherein the parameters include one or more of the prediction parameters, cross-prediction parameters, and decorrelation parameters. [Aspect 21] The method according to Embodiment 20, as referenced to Embodiment 8, wherein the prediction parameter is first calculated and quantized, the cross-prediction parameter is calculated from the quantized prediction parameter and then quantized, and the decorrelation parameter is calculated from the quantized cross-prediction parameter and the quantized prediction parameter and then quantized. [Aspect 22] The method according to embodiment 20, as referenced to embodiment 9, wherein the aforementioned parameters are first calculated, then the decorrelated parameters and the prediction parameters are quantized, and the cross-prediction parameters are recalculated from the quantized prediction parameters and then quantized. [Aspect 23] The method according to any one of embodiments 1 to 22, wherein the method is applied to metadata encoding of an Immersive Voice and Audio Services (IVAS) codec or an Ambisonics codec. [Aspect 24] The method according to any one of embodiments 1 to 23, wherein the frame size is less than 40ms, and in particular 20ms or less. [Aspect 25] A device having a processor and a memory coupled to the processor, wherein the processor is adapted to cause the device to perform the method described in any one of embodiments 1 to 24. [Aspect 26] A program having instructions that, when executed by a processor, cause the processor to perform the method described in any one of the embodiments 1 to 24. [Aspect 27] A computer-readable storage medium storing the program described in aspect 26.

Claims

1. A method for encoding metadata about an input signal frame by frame, wherein the metadata includes a plurality of parameters that are at least partially interrelated and can be calculated from the input signal, and the method for each frame: Using a loop process: The steps include: selecting a quantization strategy from among several quantization strategies for calculating and quantizing the aforementioned parameters; The steps include: calculating the parameters based on the selected quantization strategy, quantizing them, and obtaining the quantized parameters; The steps include encoding the quantized parameters and This includes performing the following sequentially and repeatedly: Each of the aforementioned quantization strategies includes a first instruction that indicates the ordering related to the calculation and quantization of individual parameters; The quantization strategy is selected based on the bitrate of the encoded quantized parameters and the target bitrate threshold. The aforementioned loop process is: The steps include: using an entropy encoder to quantize and encode the parameters in a non-differential and / or frequency-differential manner; The steps include: estimating the first parameter bitrate for the encoded parameters; If the first parameter bitrate is less than or equal to the target bitrate threshold, the loop process is terminated; If the first parameter bitrate is greater than the target bitrate threshold: The steps include: quantizing and encoding the parameters in a non-difference manner without entropy; The steps include: estimating a second parameter bitrate for the encoded parameters; If the second parameter bitrate is less than or equal to the target bitrate threshold, the loop process is terminated; If the second parameter bitrate is greater than the target bitrate threshold: The steps include: quantizing and encoding the parameters using the entropy encoder in a time-difference manner; The steps include: estimating a third parameter bitrate for the encoded parameters; The process includes the step of terminating the loop process if the third parameter bitrate is less than or equal to the target bitrate threshold, method.

2. A device having a processor and a memory coupled to the processor, wherein the processor is adapted to cause the device to perform the method described in claim 1.

3. A non-temporary computer-readable medium having, when executed by one or more processors, instructions causing the one or more processors to perform the method according to claim 1.

Images