Time-Domain Gain Modeling in the QMF Domain

Abstract

A method for processing audio is provided. The method includes determining a Modulation Filter Bank (MFB) domain wideband gain for fading an audio signal according to a time-domain target gain such that application of the wideband gain in the MFB domain emulates application of the target gain in the time domain. Determining the wideband gain includes calculating the wideband gain using the target gain, an MFB analysis prototype filter, and an MFB synthesis prototype filter. Corresponding devices, programs, and computer-readable storage media are also provided.

Description

[Technical field]

[0001] REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 330,428, filed April 13, 2022, and U.S. Provisional Application No. 63 / 490,722, filed March 16, 2023, the contents of both of which are incorporated by reference in their entireties.

[0002] Technical Field This disclosure relates to techniques for audio processing. In particular, this disclosure describes modeling of time-domain gain in a Modulated Filter Bank (MFB) domain (e.g., a Quadrature Mirror Filter (QMF) domain) and crossfading of audio signals in the MFB domain, where a target time-domain crossfade gain function must be realized as accurately as possible. [Background technology]

[0003] In audio coding, the input to the encoder can be two or more audio channels that undergo a lossless transformation, which varies across time frames. When switching from the previous time frame to the current time frame, the transformation parameters of the transformed signals are smoothly cross-faded in the time domain before encoding. This cross-fading needs to be mirrored on the decoder side. Traditionally, the cross-fading is performed in the time domain after parametric decoding in the QMF domain. However, the time domain cross-fade introduces additional processing delays and impedes audio processing of the decoded signal in the QMF domain.

[0004] Therefore, there is a need for a technique that enables efficient audio processing, particularly on the decoder side. Furthermore, there is a need for a technique that enables cross-fading of audio signals in the MFB domain (e.g., QMF domain) and efficiently determines appropriate MFB domain wideband gains, particularly on the decoder side. Summary of the Invention [Problem to be solved by the invention]

[0005] In view of this need, the present disclosure provides methods and devices for processing audio, as well as corresponding programs and computer-readable storage media, having the features of the respective independent claims. [Means for solving the problem]

[0006] One aspect of the present disclosure relates to a method for processing audio. The method may include determining a Modulation Filter Bank (MFB) domain wideband gain for fading an audio signal according to a time-domain target gain such that application of the wideband gain in the MFB domain emulates application of the target gain in the time domain. In the method, determining the wideband gain may include calculating the wideband gain using the target gain, an MFB analysis prototype filter, and an MFB synthesis prototype filter. The target gain may relate to a target gain function that is a function of time (or sample number, or other suitable time index).

[0007] The proposed method is structured as defined above and enables MFB domain processing of the gain applied audio signal by applying wideband gain in the MFB domain, reducing the overall delay and computational complexity of the audio processing chain.

[0008] In some embodiments, for each MFB analysis time slot of a plurality of MFB analysis time slots, a respective wideband gain may be calculated.

[0009] In some embodiments, calculating the wideband gain may include optimizing the wideband gain by calculating a least squares solution.

[0010] In some embodiments, determining the wideband gain may include, for each MFB analysis time slot of the multiple MFB analysis time slots and for each frequency band of the multiple frequency bands, determining a respective MFB analysis signal based on the input training signal and the MFB analysis prototype filter. Determining the wideband gain may further include, for each MFB analysis time slot of the multiple MFB analysis time slots, determining a respective MFB synthesis signal based on the MFB analysis signal and the MFB synthesis prototype filter in the respective MFB analysis time slot. Determining the wideband gain may still further include calculating a wideband gain over the MFB analysis time slot based on the MFB synthesis signal and the target gain.

[0011] Thus, the optimum wideband gain can be calculated accurately and efficiently.

[0012] In some embodiments, calculating the wideband gain may include optimizing the wideband gain by calculating a least squares solution.

[0013] In some embodiments, the least squares solution may minimize an error between samples of the first audio signal and samples of the second audio signal. The first audio signal may be obtainable by MFB analysis of a training signal, followed by MFB synthesis, overlap-add, and application of a target gain. Alternatively, the first audio signal may be obtainable by application of a target gain and delaying only the processing delays of the MFB analysis and MFB synthesis. The second audio signal may be obtainable by applying, in each MFB analysis time slot, a respective (to be determined) wideband gain to the respective MFB synthesis signal and summing up the contributions from all MFB analysis time slots. In this method, the first and second audio signals may be time-domain audio signals.

[0014] In some embodiments, the least squares solution is a transformation matrix T that depends on the multiple MFB synthesis signals. 1, and the goal vector t 1 It may be a solution of an objective function based on

[0015] In some embodiments, the transformation matrix T 1 teeth, JPEG2025513061000002.jpg548, where K is the number of MFB analysis time slots and n indicates the sample number. 1 teeth, The image may be given by JPEG2025513061000003.jpg531, where x 2 (n) is the time-domain signal that can be obtained by MFB analysis of the training signal, followed by MFB synthesis and overlap-add, and D P is the delay. The least squares solution is 1 G=t 1 where G is JPEG2025513061000004.jpg533 is the wideband gain vector given by T indicates transposition. D P may be the processing delay of the MFB analysis, followed by MFB synthesis and overlap-add.

[0016] In some embodiments, the least squares solution for the wideband gain vector G is: JPEG2025513061000005.jpg831, where □ -1 indicates the reciprocal. In some embodiments, the training signal may be a random signal or a DC signal. The random signal may be, for example, a white noise signal.

[0017] In some embodiments, the step of calculating the wideband gain is performed iteratively, with each iteration after the first iteration being calculated using a respective modified training signal or a respective different training signal. The step of calculating the wideband gain in each iteration after the first iteration may be further based on an average of the results of at least one previous iteration. For example, the final result of the wideband gain may be determined by an average of the results of all the iterations.

[0018] By repeating / iterating the process of determining the wideband gain for different input training signals, better accuracy of the proposed method for a wide range of real-world audio signals can be achieved.

[0019] In some embodiments, determining the wideband gain may include determining an MFB interpolation prototype filter based on the MFB analysis prototype filter and the MFB synthesis prototype filter. Determining the wideband gain may further include calculating a wideband gain over the MFB analysis time slot based on the MFB interpolation prototype filter and the target gain.

[0020] Thus, the optimum wideband gain can be calculated accurately and efficiently.

[0021] In some embodiments, the MFB interpolation prototype filter may be determined as a product of one of the MFB analysis prototype filter and the MFB synthesis prototype filter and a mirrored (e.g., time-mirrored) and shifted version of the other of the MFB analysis prototype filter and the MFB synthesis prototype filter. The shifted version may be shifted, for example, according to the effective length of the MFB interpolation prototype filter.

[0022] In some embodiments, calculating the wideband gain may include optimizing the wideband gain by calculating a least squares solution.

[0023] In some embodiments, The least squares solution is the transformation matrix T 2 , and the goal vector t 2 It may be a solution of an objective function based on

[0024] In some embodiments, the least squares solution is a transformation matrix T that depends on the MFB interpolation prototype filter. 2 , and the goal vector t 2 It may be a solution of an objective function based on

[0025] In some embodiments, the transformation matrix T 2 is a matrix of shifted versions of the MFB interpolation prototype filter, where each version may be associated with a particular MFB analysis time slot.

[0026] In some embodiments, the transformation matrix T 2 teeth, The image may be given by JPEG2025513061000006.jpg564, where p i is the MFB interpolation prototype filter, K is the number of MFB analysis time slots, n indicates the sample number, and S is the slot length of the MFB analysis time slot. 2 teeth, JPEG2025513061000007.jpg546, where g is the target gain. The least squares solution is given by the equation T 2 G=t 2 G can be solved by JPEG2025513061000008.jpg533 is the wideband gain vector given by T indicates transposition.

[0027] In some embodiments, the least squares solution for the wideband gain vector G is: This can be given by JPEG2025513061000009.jpg831. -1 indicates the reciprocal. In some embodiments, the MFB interpolation prototype filter p i (n) is JPEG2025513061000010.jpg536, where p A is the MFB analytical prototype filter, and p S is the MFB synthesis prototype filter, and D+1 is the MFB interpolation prototype filter p i is the effective length.

[0028] In some embodiments, the method may further include determining a set of MFB analysis time slots by identifying a non-constant gain function interval of the target gain encapsulated by the time samples and determining associated time slots based on the non-constant gain function interval.

[0029] In some embodiments, the MFB domain may be a quadrature mirror filter (QMF) domain.

[0030] In some embodiments, the method may include applying the determined wideband gain in the MFB region.

[0031] In some embodiments, the method may include generating a time-domain wideband signal using the determined wideband gain.

[0032] In some embodiments, the method may further include limiting (eg, mapping or clipping) the determined wideband gain to a predetermined range, for example a range greater than or equal to 0 and less than or equal to 1.

[0033] In some embodiments, the method may further include a step of decoding the transformed signal in the MFB domain, including a step of fading the audio signal related to the current parameter set and / or fading the audio signal related to the previous parameter set using a wideband gain for each MFB analysis time slot. This may correspond to, for example, cross-fading the audio signals described above. Specifically, this may correspond to cross-fading the previous parameters and the current parameters (parameter set) to achieve a cross-fade of the signals related to the previous parameters and the current parameters.

[0034] According to another aspect, there is provided an apparatus for processing audio. The apparatus may include a processor and a memory, coupled to the processor, for storing instructions for the processor. The processor may be configured to perform all steps of the methods according to the preceding aspects and embodiments thereof.

[0035] According to another aspect, a computer program is described which, when executed by a processor, may comprise instructions for performing the methods or method steps outlined throughout this disclosure.

[0036] According to yet another aspect, a computer readable storage medium is described. The storage medium may store a computer program adapted to be executed on a processor and, when executed on the processor, for performing the methods or method steps outlined throughout this disclosure.

[0037] It should be noted that the methods and systems, including preferred embodiments thereof, outlined in this disclosure may be used alone or in combination with other methods and systems disclosed herein. Furthermore, all aspects of the methods and systems outlined in this disclosure may be combined in any manner. In particular, the features of the claims may be combined with each other in any manner.

[0038] It will be understood that apparatus features and method steps can be interchanged in many ways. In particular, details of the disclosed methods can be implemented by corresponding apparatus, and vice versa, as will be understood by those skilled in the art. Furthermore, it will be understood that any statements made above with respect to methods (and, e.g., steps thereof) apply equally to corresponding apparatus (and, e.g., blocks, stages, units thereof), and vice versa. [Brief description of the drawings]

[0039] The invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0040] [Figure 1] FIG. 1 is a block diagram that illustrates a schematic example of a basic operation for applying gain in the time domain.

[0041] [Diagram 2] FIG. 2 is a block diagram that illustrates a schematic example of an alternative operation of MFB analysis followed by MFB synthesis and application of delayed time domain gains.

[0042] [Diagram 3] FIG. 3 is a block diagram that illustrates a schematic example of a calculation for applying a gain in the MFB domain according to an embodiment of the present disclosure.

[0043] [Figure 4] FIG. 4 is a flow chart that diagrammatically illustrates a method for processing audio according to an embodiment of the present disclosure.

[0044] [Diagram 5] FIG. 5 is a flow chart that diagrammatically illustrates an example of an implementation of the steps in the method of FIG. 4 according to an embodiment of the present disclosure.

[0045] [Figure 6] FIG. 6 is a diagram illustrating an example of a transformation matrix based on a DC input training signal according to an embodiment of the present disclosure.

[0046] [Figure 7] FIG. 7 is a flow chart that diagrammatically illustrates another example of an implementation of the steps in the method of FIG. 4 according to an embodiment of the present disclosure.

[0047] [Figure 8] FIG. 8 is a diagram illustrating an example of a transformation matrix based on an interpolation prototype filter according to an embodiment of the present disclosure.

[0048] [Figure 9] FIG. 9 illustrates an example of a time domain gain function to be modeled.

[0049] [Figure 10] FIG. 10 is a block diagram of an example of an apparatus for performing a method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0050] This disclosure describes methods for optimally (eg, in a least squares error sense) computing MFB (eg, QMF) domain wideband gains to model any desired time-domain gain.

[0051] Here, MFB refers to a real-valued or (preferably) complex-valued, subsampled M-channel (or frequency band) modulated filter bank. Using this filter bank as an analysis filter bank, a time-domain input signal is transformed to the MFB domain (e.g., QMF domain) via MFB analysis (e.g., QMF analysis) to generate an MFB domain signal (e.g., QMF domain signal). After optional signal processing in the MFB domain, the (possibly processed) MFB domain signal is transformed back to the time domain via MFB synthesis using an appropriate synthesis filter bank (e.g., inverse filter bank).

[0052] An example of an MFB is a QMF filter bank. The prototype filters of the analysis and synthesis filter banks can be different and can be symmetric or asymmetric. The subsampling factor (or stride) S is typically equal to M (critically sampled relative to the number of complex data) or smaller than M (oversampled). Typical values ​​for M for perceptual audio coding and processing at a 48 kHz sample rate are, for example, 60, 64, or 77. The overall analysis / synthesis system delay D depends on the nature of the prototype filters. With subsampling (S>1), the processing delay D P D P = D - S + 1. It is assumed that the MFB (e.g., QMF filter bank) has near-perfect reconstruction properties and aliasing errors due to filter bank domain processing are well suppressed.

[0053] FIG. 1 shows a schematic example of a basic operation of applying a gain (e.g., a cross-fade gain) in the time domain. This operation may be performed, for example, at the encoder side to cross-fade between different sets of parameters (e.g., transformation parameters). It is understood that n denotes a sample number or any other suitable time index. An input signal x(n), 10, to which a time-domain gain g(n), 20, is applied, is input to a multiplication block 160, where the time-domain gain g(n) is applied, for example, on a sample-by-sample basis. The multiplication block 160 outputs an output signal y(n), 170, which is a faded (and delayed) version of the input signal x(n), faded by the time-domain gain g(n).

[0054] FIG. 2 illustrates diagrammatically an example of a corresponding operation that may be performed, for example, on the decoder side. An input signal x(n), 10, is input to an MFB analysis block 40 (filter bank analysis, FB-A) for transformation into the MFB domain, generating an MFB domain signal u(c,k), 45, where c∈0,...,M-1 denotes a frequency band or frequency channel and k denotes an MFB time slot (e.g., a QMF time slot). If necessary, parametric encoding (decoding) or other MFB domain processing may be applied to the MFB domain signal u(c,k). The MFB domain signal u(c,k) is then input to an MFB synthesis block 50 (filter bank synthesis, FB-S) for transformation back into the time domain, generating a time domain signal x(c,k). 2 (n), 55, to generate the MFB analysis and synthesis processing delay D P The delay time domain gain g(nD P ), 25, to apply the time domain signal x 2 (n) is input to multiplication block 260, where the (delay) time domain gain g(nD P ) is applied, for example, on a sample-by-sample basis. The multiplication block 260 produces an output signal y(n) that is a faded (and delayed) version of the input signal x(n) faded by the time-domain gain g(n). 2 Output (n), 270,.

[0055] As mentioned above, applying a (delayed) time domain gain after the MFB analysis, possibly MFB domain processing, and MFB synthesis typically introduces additional delays in the audio processing chain (e.g., the decoding chain). That is, if further processing in the MFB domain is desired for the crossfaded audio signal, another MFB analysis-synthesis stage may be required following the time domain crossfade, thus adding the additional processing delay D of the MFB analysis and synthesis to the overall delay. P2. Moreover, when cross-fading two audio signals, the architecture of FIG. 2 may require running two instances of the MFB synthesis block 50 and then cross-fading in the time domain, which adds significantly to the computational overhead. To address these issues, the present disclosure proposes applying a gain in the MFB domain. This is illustrated in FIG. 3. FIG. 3 shows an example of the proposed operation with application of a wideband gain G per time slot k in the MFB domain (generally the filter bank domain, e.g., the QMF domain).

[0056] The operation shown in Fig. 3 may be performed, for example, at the decoder side. At the decoder, the transmitted / received audio signal may be decoded and subjected to MFB analysis (e.g., QMF analysis). For signal reconstruction, the transmitted parameters are applied in the MFB domain, and the processed MFB domain signals are combined. To cross-fade from the previous frame parameters to the current frame parameters, MFB wideband slot gains may be applied to the parameters. For example, in one application, there are 16 time slots per frame, and there may be 16 wideband gains, but to cover the desired cross-fade time range, a smaller number of those gains, for example, only three, may be optimally calculated as described below.

[0057] In FIG. 3 , an input signal x(n), 10, is input to an MFB analysis block 40 (filter bank analysis, FB-A) for conversion to the MFB domain, generating an MFB domain signal u(c,k), 45,. An MFB domain wideband gain G(k), 30, is then applied for each time slot. If desired, parametric encoding (decoding) or other MFB domain processing may be applied to the MFB domain signal u(c,k), 45, before or after gain application. The gain-applied MFB domain signal is then input to an MFB synthesis block 50 (filter bank synthesis, FB-S) for conversion to the time domain, generating a time domain signal y(n), which is a faded (delayed) version of the input signal x(n) faded by application of the MFB domain wideband gain G(k), 30,. 3 (n), 370, where the application of an MFB-domain wideband gain G(k), 30, is intended to mimic (or emulate, implement) the application of a predefined time-domain (target) gain g(n).

[0058] This disclosure proposes two methods for calculating the optimal wideband gain G(k) used in the example processing chain of FIG.

[0059] As a general notation, K is adopted to represent the number of filter bank analysis time slots involved. Then, the column vector G, which holds the unknown (i.e., to be calculated) wideband gains G(k), can be defined as follows:

number

number

[0060] Performance is the error e(n)=y 3 (n)-y 2 This error definition purposely excludes the reconstruction error of the filter bank and focuses on modeling the gain function. Referring to Figure 2, assuming the reconstruction properties of the target MFB are near perfect, the following relationship holds:

number

[0061] In accordance with the above, an example of an audio processing method 400 according to an embodiment of the present disclosure is shown in the flow chart of Fig. 4. The method 400 includes steps S410 and S420, of which step S420 is optional. Furthermore, the method 400 refers to the MFB domain. An example of the MFB domain is the QMF domain.

[0062] In step S410, a Modulation Filter Bank (MFB) domain wideband gain is determined to fade the audio signal according to the time domain target gain, where the application of the wideband gain in the MFB domain should emulate (i.e., mimic or implement) the application of the target gain in the time domain. In other words, the wideband gain may be determined to minimize the error e(n) defined above, e.g., in a least-squares sense. The (time domain) target gain may relate to a target gain function that is a function of time (or number of samples, or any other suitable time index).

[0063] The wideband gain determination step may include calculating the wideband gain using the target gain, the MFB analysis prototype filter, and the MFB synthesis prototype filter. For each MFB analysis time slot of the multiple MFB analysis time slots k, a respective wideband gain G(k) may be calculated. A detailed example is described below.

[0064] Additionally, determining (eg, calculating, computing) the wideband gain may include optimizing the wideband gain by computing a least-squares solution. In some embodiments, the determined wideband gains may be limited (e.g., mapped or clipped) to a predetermined target range before they are applied to any audio signal. For example, the determined wideband gains may be limited to a range greater than or equal to zero (0.0) and less than or equal to one (1.0).

[0065] In step S420, the transformed signal is decoded in the MFB domain, which includes fading the audio signal related to the current parameter set and / or fading the audio signal related to the previous parameter set using a wideband gain for each MFB analysis time slot. The fading step may correspond to, for example, cross-fading the audio signal. In particular, the fading step may correspond to cross-fading the previous and current parameters to achieve a cross-fade of the signals related to the previous and current parameters.

[0066] The decoding step in this step may include applying the determined wideband gain in the MFB domain. Additionally or alternatively, the decoding step may include generating a time-domain wideband signal using the determined wideband gain. An exemplary method for determining the wideband gain will now be described in more detail.

[0067] Method 1: Training signals According to a first exemplary method, optimal wideband gains are calculated for a "training" signal and then generally applied.

[0068] The MFB analytic signal (e.g., QMF analytic signal) in the frequency band c corresponding to the time slot k and the input training signal x(n) is expressed as a real analytic prototype p A and modulator m for band c c Using this, it can be calculated as follows:

number

number

[0069] The MFB synthesis signal (e.g., QMF synthesis signal) w related to the analytic signal u in time slot k k is the finite-length analytical prototype filter p S Using this, it can be calculated as follows:

number

[0070] The final synthesis signal x without filter bank processing 2 (for example, as shown in FIG. 2) can be calculated as follows:

number

[0071] The final output signal y with the wideband gain G(k) applied 3 (for example, as shown in FIG. 3) can be calculated as follows:

number

[0072] According to the above definition, the wideband gain determination step S410 of the method 400 may proceed via an example method 500 shown in the flowchart of Figure 5. The method 500 includes steps S510 to S530.

[0073] In step S510, for each time slot of the plurality of time slots k (MFB analysis time slots) and for each frequency band of the plurality of frequency bands c, an input training signal x(n) and an MFB analysis prototype filter p A Based on this, the respective MFB analysis signals u(c,k) are determined.

[0074] In step S520, for each MFB analysis time slot of the plurality of MFB analysis time slots k, an MFB analysis signal u(c,k) and an MFB synthesis prototype filter p S Based on the above, each MFB synthesis signal w k is determined.

[0075] Next, in step S530, the MFB synthesis signal w k and the target gain g(n), a wideband gain G(k) is calculated over the MFB analysis time slot k.

[0076] An example of the computational details of step S530, which involves optimizing the wideband gain by computing a least squares solution, will now be described.

[0077] Let K be the number of analysis time slots involved, and let T 1 can be defined as follows:

number

number

number

[0078] The following equation can then be solved in a least squares sense to determine the wideband gain:

number

number

[0079] Following the above example, the least squares solution calculated in step S530 of method 500 may be a solution for a first audio signal (e.g., audio signal y 2 (n)) and a second audio signal (e.g., the audio signal y 3 The first audio signal may be obtained by MFB analysis of the training signal x(n), followed by MFB synthesis, overlap-add, and application of a (delayed) target gain g(n). Alternatively, the first audio signal may be obtained by application of a target gain and delaying only the processing delays of the MFB analysis and MFB synthesis. The second audio signal may be obtained by applying a respective wideband gain G(k) to a respective MFB synthesis signal w(n) in each MFB analysis time slot k. k and by summing up the contributions from all MFB analysis time slots k. It will be appreciated that the first and second audio signals may be time-domain audio signals.

[0080] Further in keeping with the above example, the least squares solution (e.g., the solution defined in equation (13) above) can be used to find the multiple MFB synthesis signals w k The transformation matrix T depends on 1 , and the target vector t which depends on the target gain g(n) 1 (e.g., the objective function defined in equation (12) above).

[0081] In one embodiment, the transformation matrix T 1 As mentioned above, JPEG2025513061000024.jpg548, where K is the number of MFB analysis time slots and n is the sample number. Furthermore, the target vector t 1 teeth, The image may be given by JPEG2025513061000025.jpg531, where x 2 (n) is the time-domain signal that can be obtained by MFB analysis of the training signal x(n), followed by MFB synthesis and overlap-add, and D P is the delay (e.g., the processing delay of the MFB analysis, followed by the MFB synthesis and overlap-add). The least-squares solution is 1 G=t 1 where G is JPEG2025513061000026.jpg533 is the wideband gain vector given by T Specifically, the least squares solution for the wideband gain vector G is JPEG2025513061000027.jpg831, where □ -1 indicates the reciprocal.

[0082] As a variation of the above, the first audio signal may be obtained by applying the target gain and then delaying it by the MFB delay. In this case, the optimized wideband gain may also aim to minimize the MFB reconstruction error in addition to the objective of the actual gain. In this case, the target vector t in Eq. (11) 1can be replaced by:

number

[0083] A white noise random input signal x(n) can be shown to yield very good modeling results, but other input training signals, such as a constant signal (e.g., a DC signal), can also be used. The latter is a synthetic signal w k are shifted versions of each other. This is illustrated in the diagram of Figure 6. Figure 6 shows an example transformation matrix T for a constant input, S=60, and three time slots. 1 6, the horizontal axis indicates the time slot index, and the vertical axis indicates the time sample. However, the error of a constant input may generally be larger than the error of a random input. In line with these findings, the input training signal used above can be, for example, a random signal (eg, white noise) or a DC signal.

[0084] The performance of the proposed method can be further improved by repeating the method several times (eg, typically two or three times) and averaging the resulting optimal wideband gains.

[0085] Thus, the calculation of the wideband gain through the steps of method 500 may be performed iteratively. Each iteration after the first iteration is calculated using a respective modified or different training signal. The calculation of the wideband gain in each iteration after the first iteration may be further based on an average (e.g., weighted average, arithmetic average, etc.) of the results from at least one previous iteration. For example, the final result for the wideband gain may be determined by averaging over the results of all iterations.

[0086] Method 2: Interpolation Prototype According to a second exemplary method, an optimal wideband gain may be calculated using an MFB interpolation prototype filter (e.g., a QMF interpolation prototype filter) based on an MFB analysis prototype filter (e.g., a QMF analysis prototype filter) and an MFB synthesis prototype filter (e.g., a QMF synthesis prototype filter).

[0087] Thus, the wideband gain determination step S410 of method 400 may proceed via an exemplary method 700 shown in the flowchart of Figure 7. Method 700 includes steps S710 and S720.

[0088] In step S710, an MFB interpolation prototype filter is determined based on the MFB analysis prototype filter and the MFB synthesis prototype filter.

[0089] Then, in step S720, a wideband gain is calculated over the MFB analysis time slot based on the MFB interpolation prototype filter and the target gain.

[0090] This method has the advantage that it does not require training data (training signals) and its results depend only on the analysis and synthesis prototype filters, but it may ignore some aliasing products in the filter bank processing.

[0091] Next, an example of the computational details in steps S710 and S720, including optimizing the wideband gain by computing a least squares solution, will be described. In particular, the output of the analysis-synthesis filter bank operation, x 2 It can be shown that (n) (eg, as shown in FIG. 2) can be approximated by the following equation:

number

number

number

[0092] That is, the MFB interpolation prototype filter p i may be determined (e.g., calculated) (e.g., in step S710) as a product of one of the MFB analysis prototype filter and the MFB synthesis prototype filter and a mirrored (e.g., time-mirrored) and shifted version of the other of the MFB analysis prototype filter and the MFB synthesis prototype filter. The mirrored and shifted version may be shifted according to the effective length of the MFB interpolation prototype filter. For example, the MFB interpolation prototype filter p i (n) is the MFB synthesis filter p S (n) and MFB analysis filter p A A mirrored and shifted version of (n) p A As a product with (Dn), JPEG2025513061000032.jpg536, where D+1 is the MFB interpolation prototype filter p i is the effective length.

[0093] Let K be the number of analysis time slots involved, and then we define the transformation matrix T as follows: 2 can be defined.

number

number

number

number

number

[0094] Following the above example, the least squares solution calculated in step S720 of method 700 is 2 , and the target vector t which depends on the target gain g(n) 2 Furthermore, the least squares solution can be a solution of the objective function based on the MFB interpolation prototype filter p i (n)-dependent transformation matrix T 2 , and the target vector t which depends on the target gain g(n) 2 Specifically, the transformation matrix T 2 are the MFB interpolation prototype filters p i It may be a matrix of a shifted version of

[0095] For example, along the lines above, the transformation matrix T 2 teeth, JPEG2025513061000038.jpg564, where p i is the MFB interpolation prototype filter, K is the number of MFB analysis time slots, n indicates the sample number, and S is the slot length of the MFB analysis time slot. 2 teeth, JPEG2025513061000039.jpg546, where g is the target gain. The least squares solution is then given by the equation T 2 G=t 2 where G is JPEG2025513061000040.jpg533, where □ T denotes the transpose. A least squares solution for the wideband gain vector G is, for example, The image can be given by JPEG2025513061000041.jpg831, where □ -1 indicates the reciprocal. FIG. 8 shows an example transformation matrix T based on the interpolation prototype for S=60 and three time slots. 2 1 is a diagram showing a time slot index and a vertical axis shows time samples.

[0096] Using any of the above methods, the MFB domain wideband gains may be calculated, for example, at initialization in the decoder. Alternatively, the MFB domain wideband gains may be calculated separately from the decoder and the pre-calculated wideband gains may be stored in the decoder, for example in the form of one or more look-up tables. In yet another implementation, the wideband gains may be calculated at the encoder side and transmitted to the decoder side, for example together with the encoded audio signal.

[0097] Determining the relevant MFB time slot One method for determining the relevant time slot for gain modeling is to 0 and n 1The aim may be to identify a non-constant gain function interval of the target gain, encapsulated by 0 and k 1 (Here, the total number of time slots, K, is K=k 1 -k 0 +1) can be calculated as follows:

number

number

[0098] The target gain function g(nD P ) and the transformation matrices involved in the optimization are For the second example method, It can be restricted according to JPEG2025513061000045.jpg530.

[0099] In line with the above, the above method (e.g., method 400, method 500, or method 700) may include a step of: 0 and n 1 ) to determine a set of (related) MFB analysis time slots by identifying a non-constant gain function interval of the target gain encapsulated (e.g., delimited, bounded) by the non-constant gain function interval, and determining associated time slots based on the non-constant gain function interval.

[0100] An example of a gain function corresponding to a time domain target gain for modeling the MFB domain wideband gain (per MFB time slot) is shown in the diagram of Figure 9, where the horizontal axis indicates time samples and the vertical axis indicates gain values. The gain values ​​are normalized to values ​​between 0 and 1. Non-constant gain function sections can be identified in this diagram as rising segments between constant gain function sections.

[0101] Apparatus for implementing the methods according to the present disclosure Finally, the present disclosure also relates to an apparatus (e.g., a computer-implemented apparatus) for performing the methods and techniques described throughout the present disclosure. FIG. 10 illustrates an example of such an apparatus 1000. In particular, the apparatus 1000 comprises a processor 1010 and a memory 1020 coupled to the processor 1010. The memory 1020 may store instructions for the processor 1010. The processor 1010 may also receive appropriate input data (e.g., audio input, time-domain target gain, etc.), among others, depending on the use case and / or implementation. The processor 1010 may be adapted to perform the methods / techniques described throughout the present disclosure (e.g., method 400 of FIG. 4, method 500 of FIG. 5, and / or method 700 of FIG. 7) and generate corresponding output data 1040 (e.g., MBF-domain wideband gain or cross-faded audio signal), depending on the use case and / or implementation.

[0102] The present disclosure also relates to corresponding computer programs, computer program products, and computer readable storage media storing such computer programs or computer program products.

[0103] Technical Advantages The techniques described herein can be applied to efficiently decode signals transformed in the QMF domain that may need to be processed later. The input to the encoder can be two or more audio channels that undergo a reversible transformation, where the transformation varies across time frames. When switching from the transformation parameters of the previous time frame to the transformation parameters of the current time frame, the transformed signals are smoothly cross-faded in the time domain before encoding.

[0104] At the decoder, the encoder process is reversed in the QMF domain by cross-fading the current and previous frame parameter sets (transformed to the QMF domain and possibly combined with other processing) using time-varying wideband gains for each QMF analysis time slot. This disclosure describes, among other things, how these QMF domain wideband gains can be advantageously computed.

[0105] interpretation Aspects of the systems described herein may be implemented in a suitable computer-based sound processing network environment (e.g., a server or cloud environment) for processing digital or digitized audio files. Part of an adaptive audio system may include one or more networks with any desired number of individual machines, including one or more routers (not shown) capable of buffering and routing data transmitted between computers. Such networks may be built on a variety of different network protocols and may be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

[0106] One or more of the components, blocks, processes, or other functional components may be implemented via a computer program that controls the execution of a processor-based computing device of the system. Furthermore, the various functions described herein may be described in terms of their operations, register transfers, logic components, and / or other characteristics using any number of combinations of hardware, firmware, and / or as data and / or instructions embodied in various machine-readable or computer-readable media. Computer-readable media on which such formatted data and / or instructions may be embodied include, but are not limited to, physical (non-transitory) non-volatile storage media in various forms, such as optical, magnetic, or semiconductor storage media.

[0107] In particular, it should be understood that the embodiments may include hardware, software, and electronic components or modules that, for purposes of discussion, may be illustrated and described as if the majority of the components were implemented solely in hardware. However, those skilled in the art will recognize, based on reading the detailed description herein, that in at least one embodiment, the electronic-based aspects may be implemented in software (e.g., stored on a non-transitory computer-readable medium) executable by one or more electronic processors, such as microprocessors and / or application-specific integrated circuits ("ASICs"). Thus, it should be noted that the above embodiments may be implemented utilizing multiple hardware and software-based devices, as well as multiple different structural components. For example, the systems, encoders, decoders, or blocks described with respect to Figures 1, 2, 3, and / or 19 above may include one or more electronic processors, one or more computer-readable media modules, one or more input / output interfaces, and various connections (e.g., a system bus) connecting the various components.

[0108] Although one or more implementations have been described by way of example and with reference to specific embodiments, it is to be understood that the one or more implementations are not limited to the embodiments of the present disclosure. On the contrary, it is intended to cover various modifications and similar arrangements as will be apparent to those skilled in the art. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements. It is also to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "comprising," or "having" and their derivatives is intended to encompass the items listed thereafter and equivalents thereof, as well as additional items. Unless otherwise noted, the terms "mounted," "connected," "supported," and "coupled," along with their derivatives, are used broadly to encompass both direct and indirect mounting, connecting, supporting, and coupling.

[0109] Enumerated Example Embodiments Various aspects and implementations of the present disclosure can be understood from the following enumerated example embodiments (EEE), which are not claimed.

[0110] EEE1. A method of processing audio, the method comprising determining a Modulation Filter Bank (MFB) domain wideband gain for fading an audio signal in accordance with a time domain target gain such that application of the wideband gain in the MFB domain emulates application of a target gain in the time domain; determining the wideband gain includes calculating the wideband gain using the target gain, an MFB analysis prototype filter, and an MFB synthesis prototype filter; method.

[0111] EEE2. The method of EEE1, wherein a respective wideband gain is calculated for each MFB analysis time slot of a plurality of MFB analysis time slots.

[0112] EEE3. The method of any one of the preceding EEE, wherein the step of calculating said wideband gain comprises optimizing said wideband gain by calculating a least squares solution.

[0113] EEE4. The step of determining a wideband gain comprises: determining, for each MFB analysis time slot of a plurality of MFB analysis time slots and for each frequency band of a plurality of frequency bands, a respective MFB analysis signal based on an input training signal and the MFB analysis prototype filter; determining, for each MFB analysis time slot of the plurality of MFB analysis time slots, a respective MFB synthesis signal based on the MFB analysis signal in the respective MFB analysis time slot and the MFB synthesis prototype filter; calculating the wideband gain over an MFB analysis time slot based on the MFB synthesis signal and the target gain; Including, The method described in EEE1.

[0114] EEE5. The step of calculating the wideband gain comprises optimizing the wideband gain by calculating a least squares solution. The method described in EEE4.

[0115] EEE6. The least squares solution minimizes an error between samples of a first audio signal and samples of a second audio signal, the first audio signal being obtainable by MFB analysis of the training signal, followed by MFB synthesis, overlap-adding and application of the target gain, or by application of the target gain and delaying only the processing delays of MFB analysis and MFB synthesis, and the second audio signal being obtainable by applying, in each MFB analysis time slot, a respective wideband gain to a respective MFB synthesis signal and summing up the contributions from all MFB analysis time slots. The method described in EEE5.

[0116] EEE7. The least squares solution is a transformation matrix T that depends on the multiple MFB synthesis signals. 1 , and a target vector t 1 is the solution of the objective function based on The method described in EEE5 or EEE6.

[0117] EEE8. The transformation matrix T 1 teeth, JPEG2025513061000046.jpg548, where K is the number of MFB analysis time slots, n indicates the sample number, and the target vector t 1 teeth, JPEG2025513061000047.jpg531, where x 2 (n) is a time-domain signal obtainable by MFB analysis of the training signal, followed by MFB synthesis and overlap-add; D P is the delay, The least squares solution is given by the formula T 1 G=t 1 where G is JPEG2025513061000048.jpg533 is the wideband gain vector given by T denotes the transposition, The method described in EEE7.

[0118] EEE9. The least squares solution for the wideband gain vector G is JPEG2025513061000049.jpg831, where -1 denotes the reciprocal, The method described in EEE8.

[0119] EEE10. The method according to any one of EEE4 to EEE9, wherein the training signal is a random signal or a DC signal.

[0120] EEE11. The step of calculating the wideband gain is performed iteratively, each iteration after the first iteration being calculated using a respective modified training signal or a respective different training signal. The method according to any one of EEE4 to EEE10.

[0121] EEE12. Determining said wideband gain comprises: determining an MFB interpolation prototype filter based on the MFB analysis prototype filter and the MFB synthesis prototype filter; calculating the wideband gain over an MFB analysis time slot based on the MFB interpolation prototype filter and the target gain; Including, The method described in EEE1.

[0122] EEE13. The MFB interpolation prototype filter is determined as a product of one of the MFB analysis prototype filter and the MFB synthesis prototype filter and a mirrored (e.g., time-mirrored) and shifted version of the other of the MFB analysis prototype filter and the MFB synthesis prototype filter. The method described in EEE12.

[0123] EEE14. The step of calculating the wideband gain comprises optimizing the wideband gain by calculating a least squares solution. The method according to any one of claims 8 to 12.

[0124] EEE15. The least squares solution is 2 , and a target vector t 2 is the solution of the objective function based on Method according to EEE14.

[0125] EEE16. The least squares solution is a transformation matrix T that depends on the MFB interpolation prototype filter. 2 , and a target vector t 2 is the solution of the objective function based on Method according to EEE14.

[0126] EEE17. The transformation matrix T 2 is a matrix of shifted versions of the MFB interpolation prototype filter, each version corresponding to a particular MFB analysis time slot; The method according to claim 8, further comprising:

[0127] EEE18. The transformation matrix T 2 teeth, JPEG2025513061000050.jpg564, where p i is the MFB interpolation prototype filter, K is the number of MFB analysis time slots, n indicates the sample number, S is the slot length of the MFB analysis time slot, and the target vector t 2 teeth, JPEG2025513061000051.jpg546, where g is the target gain, The least squares solution is given by the formula T 2 G=t 2 where G is JPEG2025513061000052.jpg533 is the wideband gain vector given by T denotes the transposition, The method according to any one of EEE15 to EEE17.

[0128] EEE19. The least squares solution for the wideband gain vector G is Given by JPEG2025513061000053.jpg831, -1 denotes the reciprocal, Method according to EEE18.

[0129] EEE20. The MFB interpolation prototype filter p i (n) is JPEG2025513061000054.jpg536, where p A is the MFB analysis prototype filter, and p S is the MFB synthesis prototype filter, and D+1 is the MFB interpolation prototype filter p i is the effective length of The method according to claim 8 or 9.

[0130] EEE21. Determining a set of MFB analysis time slots by identifying non-constant gain function intervals of the target gain encapsulated by time samples and determining associated time slots based on the non-constant gain function intervals. The method of any one of the preceding EEE, further comprising:

[0131] EEE22. The MFB domain is a quadrature mirror filter (QMF) domain. The method according to any one of the preceding EEE.

[0132] EEE23. Applying the determined wideband gain in the MFB region The method of any one of the preceding EEE, including:

[0133] EEE24. Generating a time domain wideband signal using said determined wideband gain. The method of any one of the preceding EEE, including:

[0134] EEE25. Limiting the determined wideband gain to a range of 0 to 1. The method of any one of the preceding EEE, including:

[0135] EEE26. Decoding the transformed signal in the MFB domain, comprising fading an audio signal related to a current parameter set and / or fading an audio signal related to a previous parameter set using the wideband gain for each MFB analysis time slot. The method of any one of the preceding EEE, further comprising:

[0136] EEE27. An apparatus comprising: a processor; and a memory coupled to the processor for storing instructions for the processor, the processor adapted to perform a method according to any one of EEE1 to EEE26.

[0137] EEE28. A program comprising instructions which, when executed by a processor, cause the processor to perform a method according to any one of EEE1 to EEE26.

[0138] A computer-readable storage medium storing a program according to EEE29.EEE28.

[0139] EEE30. A method for processing audio, comprising: calculating a wideband gain for a training signal, the wideband gain comprising: calculating a Modulated Filter Bank (MFB) analysis signal and an MFB synthesis signal related to the MFB analysis signal; decoding the transformed signal in the MFB domain, comprising cross-fading a current and a previous parameter set using the wideband gain for each MFB analysis time slot; The method includes:

[0140] EEE31. The method of EEE30, wherein said training signal is a random signal or a DC signal.

[0141] EEE32. Determining MFB analysis time slots by identifying non-constant gain function intervals encapsulated by time samples and determining associated time slots based on said non-constant gain function. The method of EEE30, comprising:

[0142] EEE33. The step of calculating the wideband gain is performed iteratively, each iteration after the first iteration being calculated using a respective modified training signal and an average of the results of at least one previous iteration. Method according to EEE30.

[0143] EEE34. A method for processing audio, comprising: calculating a wideband gain, the calculating a wideband gain comprising: calculating an interpolation prototype filter based on the one or more filter bank prototype filters; approximating a wideband gain of a Modulated Filter Bank (MFB) analysis signal and a MFB synthesis signal over a time slot based on the interpolated prototype filter; decoding the transformed signal in the MFB domain, comprising cross-fading a current and a previous parameter set using the wideband gain for each MFB analysis time slot; The method includes:

[0144] EEE35. Applying the wideband gain to a filter bank domain The method according to any one of EEE30 to EEE34, comprising:

[0145] EEE36. Generating the Target Time-Domain Wideband Signal Using the Wideband Gain The method according to any one of EEE30 to EEE34, comprising:

[0146] EEE37. The step of calculating the wideband gain comprises optimizing a gain function by calculating a least squares solution. The method according to any one of claims 8 to 10.

[0147] EEE38. The least squares solution is a solution of an objective function based on a transformation matrix T and a target vector t that depends on a target gain function. Method according to EEE37.

[0148] EEE39. The matrix T is a matrix of shifted versions of the interpolated prototypes or filter bank synthesis data associated with a particular analysis time slot. Method according to EEE38.

[0149] A system including one or more processors configured to perform the operations recited in any one of EEE40.EEE30 to EEE39.

[0150] EEE41. A computer program product configured to cause one or more processors to perform the operations set forth in any one of EEE30 to EEE39.

Claims

1. A method for processing audio, the method comprising the step of determining a broadband gain in the modulation filter bank (MFB) region to fade an audio signal according to a time-domain target gain, such that the application of a broadband gain in the MFB region emulates the application of a target gain in the time domain. The step of determining the broadband gain includes the step of calculating the broadband gain using the target gain, the MFB analysis prototype filter, and the MFB synthesis prototype filter. method.

2. The method according to claim 1, wherein the broadband gain is calculated for each of the multiple MFB analysis time slots.

3. The method according to claim 1 or 2, wherein the step of calculating the broadband gain includes a step of optimizing the broadband gain by calculating a least-squares solution.

4. The step of determining the broadband gain is: A step of determining each MFB analysis signal for each MFB analysis time slot of a plurality of MFB analysis time slots and for each frequency band of a plurality of frequency bands, based on the input training signal and the MFB analysis prototype filter, A step of determining the MFB synthesis signal for each of the plurality of MFB analysis time slots based on the MFB analysis signal and the MFB synthesis prototype filter in each of the MFB analysis time slots, A step of calculating the broadband gain over an MFB analysis time slot based on the MFB composite signal and the target gain, including, The method according to claim 1.

5. The step of calculating the broadband gain includes a step of optimizing the broadband gain by calculating the least squares solution. The method according to claim 4.

6. The least-squares solution minimizes the error between a sample of the first audio signal and a sample of the second audio signal, the first audio signal being obtainable by MFB analysis of the training signal, followed by MFB synthesis, overlap summation, and application of the target gain, or by application of the target gain and a delay equal to the processing delay of the MFB analysis and MFB synthesis, and the second audio signal being obtainable by applying the respective broadband gain to the respective MFB synthesis signal in each MFB analysis time slot and summing the contributions from all MFB analysis time slots. The method according to claim 5.

7. The least-squares solution is a transformation matrix T that depends on the multiple MFB composite signals. １ , and the target vector t which depends on the target gain. １ The solution to the objective function based on , The method according to claim 5 or 6.

8. The transformation matrix T １ teeth, The target vector t is given by, where K is the number of MFB analysis time slots, n is the sample number, and t is the target vector. １ teeth, Given by, where x ２ (n) is a time-domain signal obtainable by MFB analysis of the training signal, followed by MFB synthesis and overlap addition, D Ｐ This is a delay, The least squares solution is given by equation T １ G = t １ Solving this, here G is is the broadband gain vector given by, □ Ｔ indicates transpose, The method according to claim 7.

9. The least-squares solution for the broadband gain vector G is: It is given by, where, □ －１ This indicates the reciprocal. The method according to claim 8.

10. The method according to claim 44 or 5, wherein the training signal is a random signal or a constant signal.

11. The process of calculating the broadband gain is performed repeatedly, with each subsequent iteration being calculated using the respective modified training signal or each different training signal. The method according to claim 44 or 5.

12. The determination of the broadband gain is as follows: A step of determining an MFB interpolation prototype filter based on the MFB analysis prototype filter and the MFB synthesis prototype filter, A step of calculating the broadband gain over an MFB analysis time slot based on the MFB interpolation prototype filter and the target gain, including, The method according to claim 1.

13. The MFB interpolation prototype filter is determined as the product of one of the MFB analysis prototype filter and the MFB synthesis prototype filter and a mirrored and shifted version of the other of the MFB analysis prototype filter and the MFB synthesis prototype filter. The method according to claim 12.

14. The step of calculating the broadband gain includes a step of optimizing the broadband gain by calculating the least squares solution. The method according to claim 12 or 13.

15. The least squares solution is obtained by the transformation matrix T ２ , and the target vector t which depends on the target gain. ２ The solution to the objective function based on , The method according to claim 14.

16. The least-squares solution is a transformation matrix T that depends on the MFB interpolation prototype filter. ２ , and the target vector t which depends on the target gain. ２ The solution to the objective function based on , The method according to claim 14.

17. The transformation matrix T ２ This is a matrix of shifted versions of the MFB interpolation prototype filter, where each version is associated with a specific MFB analysis time slot. The method according to claim 15.

18. The transformation matrix T ２ teeth, Given by, where p ｉ is the MFB interpolation prototype filter, K is the number of MFB analysis time slots, n is the sample number, S is the slot length of the MFB analysis time slot, and t is the target vector ２ teeth, This is given by, where g is the target gain, The least squares solution is given by equation T ２ G = t ２ Solving this, here G is This is a broadband gain vector given by □ Ｔ This indicates transpose. The method according to claim 15.

19. The least squares solution for the broadband gain vector G is, Given by, □ －１ This indicates the reciprocal. The method according to claim 18.

20. The MFB interpolation prototype filter p ｉ (n) is, Given by, where p Ａ This is the MFB analysis prototype filter, and p Ｓ p is the MFB synthesis prototype filter, and D+1 is the MFB interpolation prototype filter p ｉ The effective length is, The method according to claim 18.

21. The process of determining a set of MFB analysis time slots by identifying a non-constant gain function interval of the target gain encapsulated by a time sample, and determining a time slot associated with the non-constant gain function interval. The method according to claim 1 or 2, further encompassing the method according to claim 1 or 2.

22. The aforementioned MFB region is an orthogonal mirror filter (QMF) region. The method according to claim 1 or 2.

23. Steps to apply the broadband gain determined in the MFB region. The method according to claim 1 or 2, comprising:

24. The process of generating a time-domain broadband signal using the determined broadband gain. The method according to claim 1 or 2, comprising:

25. A step of limiting the determined broadband gain to a range of 0 or more and 1 or less. The method according to claim 1 or 2, comprising:

26. A step of decoding the signal converted in the MFB region, comprising the steps of using the broadband gain to fade the audio signal related to the current parameter set and / or fade the audio signal related to the previous parameter set for each MFB analysis time slot, The method according to claim 1 or 2, further encompassing the method according to claim 1 or 2.

27. An apparatus comprising a processor and a memory coupled to the processor for storing instructions for the processor, wherein the processor is adapted to carry out the method according to claim 1 or 2.

28. A program comprising instructions that, when executed by a processor, cause the processor to perform the method according to claim 1 or 2.

29. A computer-readable storage medium storing the program described in claim 28.

Images