DOWNMIXER AND METHOD FOR DOWNMIXING AT LEAST TWO CHANNELS AND MULTI-CHANNEL ENCODER AND DECODER - Patent application

Abstract

A downmixer is provided for downmixing at least two channels of a multi-channel signal (12) having two or more channels. The downmixer comprises a processor (10) for calculating a partial downmix signal (14) from at least two channels, a complementary signal calculator (20) for calculating a complementary signal (22) from a multi-channel signal (12), the complementary signal (22) being different from the partial downmix signal (14), and an adder (30) for adding the partial downmix signal (14) and the complementary signal (22) to obtain a downmix signal (40) of the multi-channel signal.

Description

[Technical field]

[0001] The present invention relates to audio processing, and in particular to processing multi-channel audio signals containing two or more audio channels. [Background technology]

[0002] Reducing the number of channels is essential to achieve multi-channel coding at low bit rates. For example, parametric stereo coding schemes are based on an appropriate mono downmix from the left and right input channels. The resulting mono signal is then encoded and transmitted by a mono codec together with side information describing the auditory scene in a parametric form. The side information usually consists of several spatial parameters per frequency subband. They can be, for example, Inter-Channel Level Difference (ILD), which measures the level difference (or balance) between channels; It may also include Inter-Channel Time Difference (ITD) or Inter-Channel Phase Difference (IPD), which describe the time or phase difference between channels, respectively.

[0003] However, the downmix process is prone to producing signal cancellation and coloration due to inter-channel phase misalignment, which leads to undesirable quality degradation. As an example, if the channels are coherent and close to out of phase, the downmix signal is likely to exhibit perceptible spectral biases, such as the characteristics of a comb filter.

[0004] The downmix operation is m[n] = w1l[n] + w2r[n] This can be done in the time domain by simply summing the left and right channels, as represented by: where l[n] and r[n] are the left and right channels, n is the time index, and w1[n] and w2[n] are the weights that determine the mixing. If the weights are constant over time, we speak of passive downmixing, which has the disadvantage that regardless of the input signal, the quality of the resulting downmix signal depends heavily on the input signal characteristics. Adapting the weights over time can reduce this problem to some extent.

[0005] However, to solve the main problem, active downmixing is usually performed in the frequency domain, for example using the Short-Term Fourier Transform (STFT), so that the weights can be made dependent on the frequency index k and the time index n, and can be better adapted to the signal characteristics. The downmix signal is then M[k, n] = W1[k, n]L[k, n] + W2[k, n]R[k, n] where M[k,n], L[k,n] and R[k,n] are the STFT components of the downmix signal, left channel and right channel at frequency index k and time index n, respectively. The weights W1[k,n] and W2[k,n] can be adaptively adjusted in time and frequency. It aims to preserve the average energy or amplitude of the two input channels by minimizing the spectral bias caused by the comb filtering effect.

[0006] The most straightforward method for active downmixing is to equalize the energy of the downmix signal to yield the average energy of the two input channels for each frequency bin or subband [1]. The downmix signal as shown in Figure 7b is then M[k] = W[k](L[k] + R[k]) where:

[0007]

number

[0008] Such a straightforward solution has several drawbacks. First, the downmix signal is undefined when the two channels have equal amplitude phase-inverted time-frequency components (ILD=0db and IPD=π). This singularity arises as a result of the denominator becoming zero in this case. The output of a simple active downmixing is unpredictable in this case. This behavior is shown for various inter-channel level differences in Fig. 7a, where the phase is plotted as a function of IPD.

[0009] For ILD = 0 dB, the sum of the two channels is discontinuous at IPD = π, resulting in steps of π radians. For other conditions, the phase evolves regularly and continuously modulo 2π.

[0010] The second nature of the problem comes from the significant difference in normalization gain to achieve such energy equalization. In fact, the normalization gain can vary significantly from frame to frame and between adjacent frequency subbands, which leads to unnatural coloration and block effects in the downmix signal. The use of synthesis windows for the STFT and overlap-add methods results in a smooth transition between the processed audio frames. However, large changes in normalization gain between consecutive frames can still lead to audible transition artifacts. Moreover, this significant equalization can also lead to audible artifacts due to aliasing from the frequency response sidelobes of the analysis window of the block transform.

[0011] Alternatively, active downmix can be achieved by performing phase alignment of the two channels before calculating the sum signal [2-4]. Since the two channels are already in phase before summing them, the energy equalization to be done on the new sum signal is then limited. In [2], the phase of the left channel is used as a reference to align the phase of the two channels. If the phase of the left channel is not well aligned (e.g., zero or low level noise channel), the downmix signal is directly affected. In [3], this important issue is solved by taking the phase of the sum signal as a reference before rotation. Note that the singularity problem at ILD = 0 dB and IPD = π is not addressed. For this reason, [4] modifies the method by using a wideband phase difference parameter to improve stability in such cases. Nevertheless, none of these methods considered the second nature of the problem related to instability. Phase rotation of the channels can also lead to unnatural mixing of the input channels, which can produce severe instability and blocking effects, especially when large changes occur in the processing over time and frequency.

[0012] Finally, there are more developed techniques such as [5] and [6], which are based on the observation that signal cancellation during downmixing occurs only for time-frequency components that are coherent between the two channels. In [5], the coherent components are filtered out before summing the incoherent parts of the input channels. In [6], phase alignment is only calculated for the coherent components before summing the channels. Moreover, the phase alignment is normalized over time and frequency to avoid stability and discontinuity problems. Both techniques are computationally demanding, since in [5] the filter coefficients need to be identified at every frame, and in [6] the covariance matrix between the channels has to be calculated. Summary of the Invention [Problem to be solved by the invention]

[0013] It is an object of the present invention to provide an improved concept for downmixing or multi-channel processing. [Means for solving the problem]

[0014] This object is achieved by a downmixer according to claim 1, a method for downmixing according to claim 13, a multi-channel encoder according to claim 14, a method for multi-channel encoding according to claim 15, an audio processing system according to claim 16, a method for processing an audio signal according to claim 17 or a computer program according to claim 18.

[0015] The invention is based on the finding that a downmixer for downmixing at least two channels of a multi-channel signal having two or more channels not only performs an addition of the at least two channels to calculate a downmix signal from the at least two channels, but also comprises a complementary signal calculator for calculating a complementary signal from the multi-channel signal, the complementary signal being different from the partial downmix signal. Furthermore, the downmixer comprises an adder for adding the partial downmix signal and the complementary signal to obtain the downmix signal of the multi-channel signal. This procedure is advantageous because the complementary signal, which is different from the partial downmix signal, fills any time-domain or spectral-domain holes in the downmix signal that may arise due to a certain phase constellation of the at least two channels. In particular, when the two channels are in phase, then typically no problems should arise when a direct addition of the two channels is performed. However, when the two channels are out of phase, then the addition of these two channels results in a signal with a very low energy, even approaching zero energy. However, due to the fact that the complementary signal is now added to the partial downmix signal, the final resulting downmix signal still has significant energy or at least does not exhibit such significant energy fluctuations.

[0016] The present invention is advantageous because it introduces a procedure for downmixing two or more channels that aims to minimize the typical signal cancellation and instability observed in conventional downmixing.

[0017] Furthermore, the embodiments are advantageous because they represent a low-complexity procedure that has the potential to minimize typical problems from multi-channel downmixing.

[0018] A preferred embodiment relies on a controlled energy or amplitude equalization of the sum signal, which is mixed with a complementary signal, also derived from the input signal, but different from the partial downmix signal. The energy equalization of the sum signal is controlled to avoid problems at singularities, but also to minimize significant signal impairments due to large gain variations. Preferably, the complementary signal is present to compensate for the remaining energy loss or to compensate at least a part of this remaining energy loss.

[0019] In one embodiment, the processor is configured to calculate the partial downmix signal such that a predefined energy- or amplitude-related relationship between the at least two channels and the partial downmix channel is satisfied when the at least two channels are in phase, and such that an energy loss is produced in the partial downmix signal when the at least two channels are out of phase. In this embodiment, the complementary signal calculator is configured to calculate the complementary signal such that an energy loss of the partial downmix signal is partially or fully compensated by adding the partial downmix signal and the complementary signal together.

[0020] In one embodiment, the complementary signal calculator is configured for calculating the complementary signal such that it has a coherence index of 0.7 with respect to the partial downmix signal, where a coherence index of 0.0 indicates complete incoherence and a coherence index of 1 indicates complete coherence. It is therefore ensured that the partial downmix signal on the one hand and the complementary signal on the other hand are sufficiently different from each other.

[0021] Preferably, downmixing, as performed in conventional passive or active downmixing techniques, generates a sum signal of two channels, such as L + R. A gain applied to this sum signal, later called W1, aims to equalize the energy of the sum channel to match the average energy or average amplitude of the input channels. However, in contrast to conventional active downmixing techniques, the W1 value is limited to avoid instability problems and to avoid the energy relationship being restored based on a corrupted sum signal.

[0022] The second mixing is performed with a complementary signal, chosen so that its energy does not vanish when L and R are out of phase. The weighting factor W2 compensates for the energy equalization due to the limitation introduced in the W1 value.

[0023] Preferred embodiments are discussed below with reference to the accompanying drawings. [Brief description of the drawings]

[0024] [Figure 1] FIG. 2 is a block diagram of a downmixer according to one embodiment; [Figure 2a] 13 is a flow diagram illustrating an energy loss compensation feature. [Figure 2b] FIG. 2 is a block diagram illustrating one embodiment of a complementary signal calculator. [Diagram 3] FIG. 2 is a schematic block diagram illustrating a downmixer operating in the spectral domain and having a summer output connected to different alternative or cumulative processing elements. [Figure 4] FIG. 4 illustrates a preferred procedure implemented by a processor for processing a partial downmix signal. [Diagram 5] FIG. 2 illustrates a block diagram of a multi-channel encoder in one embodiment. [Figure 6] FIG. 1 illustrates a block diagram of a multi-channel decoder. [Figure 7a]FIG. 1 illustrates a singularity of a sum component according to the prior art; [Figure 7b] FIG. 7b illustrates an equation for calculating the downmix in the prior art example of FIG. 7a. [Figure 8a] FIG. 2 illustrates an energy relationship for downmixing according to one embodiment. [Figure 8b] FIG. 8b illustrates equations for the embodiment of FIG. 8a. [Figure 8c] FIG. 13 illustrates an alternative equation with coarser frequency resolution of the weighting coefficients. [Figure 8d] FIG. 8b illustrates the downmix phase for the embodiment of FIG. 8a. [Figure 9a] FIG. 13 illustrates a gain limit diagram for a sum signal in a further embodiment. [Figure 9b] FIG. 9b illustrates an equation for calculating the downmix signal M for the embodiment of FIG. 9a. [Figure 9c] FIG. 9b illustrates a manipulation function for calculating weighting coefficients that are manipulated for the calculation of the sum signal of the embodiment of FIG. 9a. [Figure 9d] FIG. 9C illustrates the calculation of the weighting factor W2 for the calculation of the complementary signal for the embodiment of FIGS. 9a-9c. [Figure 9e] FIG. 9C illustrates the energy relationships of the downmixing of FIGS. 9a to 9d. [Figure 9f] FIG. 10 illustrates the gain W2 for the embodiment of FIGS. 9a to 9e. [Figure 10a] FIG. 13 illustrates the downmix energy for a further embodiment. [Figure 10b] FIG. 10b illustrates an equation for the calculation of the downmix signal and the first weighting factor W1 for the embodiment of FIG. 10a. [Figure 10c] FIG. 13 illustrates a procedure for calculating second or complementary signal weighting coefficients for the embodiment of FIGS. 10a-b. [Figure 10d]FIG. 10c illustrates equations for parameters p and q of the embodiment of FIG. [Figure 10e] FIG. 13 illustrates the gain W2 as a function of ILD and IPD of downmixing for the embodiment illustrated in FIGS. 10a to 10d. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] FIG. 1 illustrates a downmixer for downmixing at least two channels of a multichannel signal 12 having two or more channels. In particular, the multichannel signal can be only a stereo signal having a left channel L and a right channel R, or the multichannel signal can even have three or more channels. The channels can also contain or consist of audio objects. The downmixer comprises a processor 10 for calculating a partial downmix signal 14 from at least two channels from the multichannel signal 12. Furthermore, the downmixer comprises a complementary signal calculator 20 for calculating a complementary signal from the multichannel signal 12, the complementary signal 22 output by the block 20 being different from the partial downmix signal 14 output by the block 10. In addition, the downmixer comprises an adder 30 for adding the partial downmix signal and the complementary signal to obtain a downmix signal 40 of the multichannel signal 12. In general, the downmix signal 40 has only a single channel or, alternatively, has more than one channel. Generally, however, the downmix signal has fewer channels than are contained in the multi-channel signal 12. Hence, when the multi-channel signal has, for example, five channels, the downmix signal may have four channels, three channels, two channels or a single channel. A downmix signal having one or two channels is preferred over a downmix signal having more than two channels. In case of a two-channel signal as the multi-channel signal 12, the downmix signal 40 only has a single channel.

[0026] In one embodiment, the processor 10 is configured to calculate the partial downmix signal 14 such that a predefined energy- or amplitude-related relationship between the at least two channels and the partial downmix signal is satisfied when the at least two channels are in phase, and such that an energy loss is produced in the partial downmix signal for the at least two channels when the at least two channels are out of phase. An embodiment and example of a predefined relationship is that the amplitude of the downmix signal is in a certain relationship to the amplitude of the input signal, or for example the subband-wise energy of the downmix signal is in a predefined relationship to the energy of the input signal. One particularly interesting relationship is that the energy of the downmix signal over the whole bandwidth or within a subband is equal to the average energy of the two downmix signals or more than two downmix signals. Hence, the relationship may be energy- or amplitude-related. Furthermore, the complementary signal calculator 20 of FIG. 1 is configured to calculate the complementary signal 22 such that the energy loss of the partial downmix signal, as exemplified by 14 in FIG. 1, is partially or fully compensated by adding the partial downmix signal 14 and the complementary signal 22 in the adder 30 of FIG. 1 to obtain the downmix signal.

[0027] In general, the embodiments are based on controlled energy or amplitude equalization of a sum signal that is mixed with a complementary signal that is also derived from an input channel.

[0028] The embodiment is based on a controlled energy or amplitude equalization of the sum signal, which is mixed with a complementary signal, which is also derived from the input channel. The energy equalization of the sum signal is controlled to avoid problems at singularities, but also to greatly minimize signal impairments due to large gain variations. The complementary signal is there to compensate for the remaining energy loss, or at least part of it. The general form of the new downmix is: M[k, n] = W1[k, n](L[k, n] + R[k, n]) + W2[k, n]S[k, n] where the complementary signals S[k, n] should ideally be as orthogonal as possible to the sum signal, but in practice S[k, n] = L[k, n] or S[k, n] = R[k, n] or S[k, n] = L[k, n] - R[k, n] You can choose:

[0029] In either case, downmixing, as performed in conventional passive or active downmixing techniques, first generates a sum channel L + R. The gain W1[k, n] aims to equalize the energy of the sum channel to match the average energy or average amplitude of the input channels. However, unlike conventional active downmixing techniques, W1[k, n] is limited to avoid instability problems and to avoid that the energy relationship is restored based on a corrupted sum signal.

[0030] The second mixing is performed with a complementary signal, chosen so that its energy does not vanish when L[k,n] and R[k,n] are out of phase. W2[k,n] compensates for the energy equalization due to the limitation introduced in W1[k,n].

[0031] As illustrated, the complementary signal calculator 20 is configured to calculate the complementary signal such that the complementary signal is different from the partial downmix signal. In terms of quantity, it is preferred that the coherence index of the complementary signal is less than 0.7 with respect to the partial downmix signal. On this scale, a coherence index of 0.0 indicates complete incoherence and a coherence index of 1.0 indicates complete coherence. Hence, a coherence index less than 0.7 has proven useful so that the partial downmix signal and the complementary signal are sufficiently different from each other. However, a coherence index less than 0.5 and even less than 0.3 is more preferred.

[0032] Fig. 2a illustrates the procedure performed by the processor. In particular, as illustrated in item 50 of Fig. 2a, the processor calculates a partial downmix signal having energy loss for at least two channels representing the input to the processor. Furthermore, a complementary signal calculator 52 calculates the complementary signal 22 of Fig. 1 for partially or completely compensating for the energy loss.

[0033] In an embodiment illustrated in Fig. 2b, the complementary signal calculator comprises a complementary signal selector or complementary signal determiner 23, a weighting coefficient calculator 24 and a weighter 25 for finally obtaining the complementary signal 22. In particular, the complementary signal selector or complementary signal determiner 23 is configured to use one signal of a group of signals consisting of a first channel, such as L, a second channel, such as R, and a difference between the first and second channels, as indicated as LR in Fig. 2b, for calculating the complementary signal. Alternatively, the difference can also be RL. The further signal used by the complementary signal selector 23 can be a further channel of the multi-channel signal, i.e. a channel that is not selected by the processor for calculating the partial downmix signal. This channel can be, for example, a center channel or any other additional channel including a surround channel or an object. In other embodiments, the signal used by the complementary signal selector is the de-correlated first channel, the de-correlated second channel, the de-correlated further channel or even the de-correlated partial downmix signal as calculated by the processor 10. In a preferred embodiment, however, either a first channel such as L or a second channel such as R, or more preferably, the difference between the left channel and the right channel or the difference between the right channel and the left channel, is preferred to calculate the complementary signal.

[0034] The output of the complementary signal selector 23 is input to a weighting coefficient calculator 24, which additionally typically receives two or more signals to be combined by the processor 10, and the weighting coefficient calculator calculates weights W2, illustrated at 26. Those weights, together with the signals used and determined by the complementary signal selector 23, are input to a weighter 25, which then weights the corresponding signals output from block 23 using the weighting coefficients from block 24 to ultimately obtain the complementary signal 22.

[0035] The weighting factors can only be time-dependent, such that for a block or frame in time, a single weighting factor W2 is calculated. In other embodiments, however, it is preferred to use time- and frequency-dependent weighting factors W2, such that for a block or frame of the complementary signal, not only a single weighting factor for this time block is available, but a set of weighting factors W2 for a set of different frequency values ​​or spectral bins of the signal generated or selected by block 23.

[0036] A corresponding embodiment for time and frequency dependent weighting coefficients for use by the complementary signal calculator 20 as well as for use by the processor 10 is illustrated in FIG.

[0037] In particular, Fig. 3 illustrates a downmixer in a preferred embodiment comprising a time-to-spectral converter 60 for converting time-domain input channels into frequency-domain input channels, each frequency-domain input channel having a series of spectra. Each spectrum has a distinct time index n, and within each spectrum, a certain frequency index k refers to the frequency component uniquely associated with the frequency index. Thus, in one example, when a block has 512 spectral values, then the frequency index k ranges from 0 to 511 to uniquely identify each of the 512 different frequency indexes.

[0038] The time-to-spectral transformer 60 is configured to apply an FFT, and preferably an overlapping FFT, such that the series of spectra obtained by block 60 relate to overlapping blocks of input channels. However, non-overlapping spectral transform algorithms and other transforms apart from FFTs or the like, such as DCTs, may be used as well.

[0039] In particular, the processor 10 of FIG. 1 comprises a first weighting coefficient calculator 15 for calculating a weight W1 for an individual spectral index k or a weighting coefficient W1 for a sub-band b, the sub-band being wider in frequency than one spectral value and typically including two or more spectral values.

[0040] The complementary signal calculator 20 of Figure 1 comprises a second weighting factor calculator which calculates the weighting factor W2. Hence, item 24 may be configured similarly to item 24 of Figure 2b.

[0041] Furthermore, the processor 10 of Fig. 1 for calculating the partial downmix signal comprises a downmix weighter 16 which receives as input a weighting factor W1 and outputs a partial downmix signal 14 which is forwarded to the adder 30. Furthermore, the embodiment illustrated in Fig. 3 additionally comprises a weighter 25 already mentioned in relation to Fig. 2b which receives as input a second weighting factor W2.

[0042] The adder 30 outputs a downmix signal 40. The downmix signal 40 can be used in several different occasions. One way to use the downmix signal 40 is to input it to a frequency domain downmix encoder 64 illustrated in Fig. 3, which outputs an encoded downmix signal. An alternative procedure is to insert a frequency domain representation of the downmix signal 40 into a spectrum-to-time converter 62 to obtain a time domain downmix signal at the output of block 62. A further embodiment is to feed the downmix signal 40 to a further downmix processor 66, which generates a downmix channel to be transmitted, a downmix channel to be stored, or a downmix channel with some equalization, gain change, etc.

[0043] In an embodiment, the processor 10 is configured to calculate a time or frequency dependent weighting factor W1 as illustrated by block 15 in Fig. 3 in order to weight the sum of the at least two channels according to a predefined energy or amplitude relationship between the at least two channels and the sum signal of the at least two channels. Further, after this procedure, also illustrated in item 70 in Fig. 4, the processor is configured to compare the weighting factor W1 calculated for a frequency index k and a time index n or for a spectral subband b and a time index n with a predefined threshold as illustrated in block 72 in Fig. 4. This comparison is preferably performed for each spectral index k or for each subband index b or for each time index n, preferably for one spectral index k or b and for each time index n. When the calculated weighting factor is in a first relationship to the predefined threshold, such as being below the threshold as illustrated in 73, the weighting factor W1 calculated at that time is used as illustrated in 74 in Fig. 4. However, when the calculated weighting factors are in a second relationship to the predefined threshold, different from the first relationship to the predefined threshold, such as being above the threshold as shown at 75, the predefined threshold is used instead of the calculated weighting factors for calculating the partial downmix signal, for example in block 16 of FIG. 3. This is a "hard" limit of W1. In another embodiment, a kind of "soft limit" is performed. In this embodiment, the modified weighting factors are derived using a modification function such that the modified weighting factors are closer to the predefined threshold than the calculated weighting factors.

[0044] The embodiment in Figures 8a-8d uses hard constraints, while the embodiment in Figures 9a-9f and the embodiment in Figures 10a-10e use soft constraints, ie varying functions.

[0045] In a further embodiment, the procedure in Fig. 4 is performed with respect to blocks 70 and 76, but without the comparison with a threshold as discussed with respect to block 72. After the calculation in block 70, the modified weighting coefficients are derived using the above described modification function of block 76, the modification function being such that the modified weighting coefficients result in an energy of the partial downmix signal that is smaller than the energy of the defined energy relation. Preferably, the modification function applied without a specific comparison is such that for high values ​​of W1, it limits the manipulated or modified weighting coefficients to a certain limit, or only has a very small increase, such as a log or ln function, or is not limited to a certain value but only has a very slow increase so that stability problems as discussed before are substantially avoided or at least reduced.

[0046] In a preferred embodiment illustrated in Figures 8a to 8d, the downmix is M[k, n] = W1[k, n](L[k, n] + R[k, n]) + W2[k, n]L[k, n] where:

[0047]

number

[0048]

number

[0049] In the above equation, A is a real-valued constant preferably equal to the square root of 2, but A can also have different values ​​between 0.5 and 5. Depending on the application, even values ​​different from those mentioned above can be used as well.

[0050] if |L[k, n] + R[k, n]| ≦ |L[k, n]| + |R[k, n]| Then, W1[k, n] and W2[k, n] are always positive, and W1[k, n] is

[0051]

number

[0052] Or limited to, for example, 0.5.

[0053] The mixing gain can be calculated bin-wise for each index k of the STFT as described in the previous formula, or band-wise for each non-overlapping subband that aggregates a set of indices b of the STFT. The gain is calculated according to the following equation:

[0054]

number

[0055]

number

[0056] It is calculated based on:

[0057] Since energy conservation during equalization is not a hard constraint, the energy of the resulting downmix signal varies compared to the average energy of the input channels. The energy relationship depends on the ILD and IPD as illustrated in Fig. 8a.

[0058] In contrast to simple active downmixing methods, which preserve a constant relationship between the output energy and the average energies of the input channels, the new downmix signal does not show any singularities, as illustrated in Fig. 8d. Indeed, in Fig. 7a a jump of magnitude π (180°) is observable at IP = π and ILD = 0 dB, while in Fig. 8d the jump is 2π (360°), which corresponds to a continuous change in the unwrapped phase domain.

[0059] Listening test results confirm that the new downmix method produces significantly less instability and impairments than conventional active downmixing for a wide range of stereo signals.

[0060] In this context, Fig. 8a illustrates along the x-axis the inter-channel level difference in dB between the first left channel and the first right channel. Furthermore, the downmix energy is shown along the y-axis on a relative scale between 0 and 1.4, the parameter being the inter-channel phase difference IPD. In particular, the energy of the resulting downmix signal values ​​varies in particular depending on the phase between the channels, and for a phase of π (180°), i.e. for out-of-phase situations, the energy change appears to be in good shape, at least for positive inter-channel level differences. Fig. 8b illustrates an equation for calculating the downmix signal M, and it also becomes clear that as a complementary signal, the left channel is selected. Fig. 8c illustrates weighting coefficients W1 and W2 for individual spectral indices as well as for a set of indices from the STFT, i.e. for subbands where at least two spectral values ​​k are added together to obtain a subband.

[0061] Compared to the prior art illustrated in Figures 7a and 7b, Figure 8d no longer contains any specificity when compared to Figure 7a.

[0062] 9a to 9f illustrate a further embodiment in which the downmix is ​​calculated using the difference between the left signal L and the right signal R as the basis for the complementary signal. In particular in this embodiment: M[k, n] = W1[k, n](L[k, n] + R[k, n]) + W2[k, n](L[k, n] - R[k, n]) However, the set of gains W1[k, n] and W2[k, n] is calculated such that the energy relationship between the downmixed signal and the input channels persists under all conditions.

[0063] First, the gain W1[k, n] is calculated to equalize the energy up to a given limit, where A is again

[0064]

number

[0065] is a real-valued number that is equal to or different from this value,

[0066]

number

[0067]

number

[0068] As a result, the gain of the sum signal W1[k, n] is restricted to the range [0, 1] as shown in Figure 9a. In the equation for x, an alternative implementation is to use a denominator without a square root.

[0069] If the two channels have an IPD greater than π / 2, W1 can no longer compensate for the loss of energy, which then comes from the gain W2. W2 is given by the quadratic equation:

[0070]

number

[0071] It is calculated as one of the roots of

[0072] The root of the equation is

[0073]

number

[0074] where:

[0075]

number

[0076]

number

[0077] One of the two roots may then be selected. For both roots, the energy relationship is preserved for all conditions, as shown in Figure 9e.

[0078] If the two channels have an IPD greater than π / 2, W1 can no longer compensate for the loss of energy, which then comes from the gain W2. W2 is given by the quadratic equation:

[0079]

number

[0080] It is calculated as one of the roots of

[0081] The root of the equation is

[0082]

number

[0083] where:

[0084]

number

[0085]

number

[0086] One of the two roots may then be selected. For both roots, the energy relationship is preserved for all conditions, as shown in Fig. 9f.

[0087] Preferably, the root with the smallest absolute value is adaptively selected for W2[k, n]. Such an adaptive selection would result in a switch from one root to another for ILD=0 dB, which again may create a discontinuity.

[0088] In contrast to the state of the art, this approach resolves the comb filtering effects and spectral biases of the downmix without introducing any singularities. It preserves the energy relations in all conditions, but introduces more instabilities compared to the preferred embodiment.

[0089] Fig. 9a therefore illustrates a comparison of the gain limit obtained by the coefficient W1 of the sum signal in the calculation of the partial downmix signal of this embodiment. In particular, the straight line is the situation before normalization or modification of the values ​​as previously discussed with respect to block 76 of Fig. 4, and the other line, which approaches a value of 1 due to the modification function as a function of the weighting coefficient W1. It becomes clear that the influence of the modification function occurs for values ​​above 0.5, but the deviations only become really visible for values ​​W1 of about 0.8 and above.

[0090] FIG. 9b illustrates the equations implemented by the block diagram of FIG. 1 for this embodiment.

[0091] Furthermore, Fig. 9c illustrates how the value W1 is calculated, and thus Fig. 9a illustrates the functional situation of Fig. 9c. Finally, Fig. 9d illustrates the calculation of W2, the weighting coefficient used by the complementary signal generator 20 of Fig. 1.

[0092] FIG. 9e illustrates that the downmix energy is always the same and equal to 1 for all phase differences between the first and second channels and for all level differences ALD between the first and second channels.

[0093] However, FIG. 9f shows that the equation for p and the equation for q illustrated in FIG. 9d have denominators that can be zero, which is a problem with the E M This illustrates the discontinuity incurred by the calculation of the rules of the equation for .

[0094] 10a-10e illustrate a further embodiment which can be seen as a comparison between two previously mentioned alternatives.

[0095] Downmixing is M = W1[k](L[k] + R[k]) + W2[k](L[k] - R[k]) where:

[0096]

number

[0097]

number

[0098] In the equation for x, an alternative implementation is to use a denominator without a square root.

[0099] In this case, the quadratic equation to be solved is:

[0100]

number

[0101] It is.

[0102] This time, the gain W2 is not taken strictly as one of the roots of the quadratic equation, but rather as

[0103]

number

[0104] where:

[0105]

number

[0106]

number

[0107] As a result, the energy relationship is not preserved all the time as shown in Fig. 10a, whereas the gain W2 does not show any discontinuity in Fig. 10e and the instability problem is reduced compared to the second embodiment.

[0108] Hence, Fig. 10a illustrates the energy relationships of this embodiment illustrated by Figs. 10a-10e, where once again the downmix energy is illustrated on the y-axis and the inter-channel level difference is illustrated on the x-axis. Fig. 10b illustrates the procedure taken to calculate the first weighting factor W1 as illustrated with respect to the equation applied by Fig. 1 and block 76. Furthermore, Fig. 10c illustrates an alternative calculation of W2 with respect to the embodiment of Figs. 9a-9f. In particular, p follows an absolute value function that appears when comparing Fig. 10c with the similar equation in Fig. 9d.

[0109] FIG. 10d then once again shows the calculation of p and q, which roughly corresponds to the equations at the bottom of FIG. 9d.

[0110] FIG. 10e illustrates the energy relationship of this new downmixing according to the embodiment illustrated in FIGS. 10a-d, where only gain W2 appears to approach a maximum value of 0.5.

[0111] Although the previous description and certain figures provide detailed equations, it should be noted that advantages are already obtained even when the equations are not calculated exactly, but when the equations are calculated, the results are altered. In particular, the functionality of the first weighting coefficient calculator 15 and the second weighting coefficient calculator 24 of FIG. 3 is performed such that the first weighting coefficient or the second weighting coefficient has a value that is within ±20% of the value determined based on the equations given above. In a preferred embodiment, the weighting coefficient is determined to have a value that is within ±10% of the value determined by the above equations. In a more preferred embodiment, the deviation is only ±1%, and in the most preferred embodiment, the result of the equations is taken exactly. However, as stated, advantages of the present invention are obtained even when a deviation of ±20% from the above equations is applied.

[0112] Fig. 5 illustrates an embodiment of a multi-channel encoder, in which an inventive downmixer as previously discussed with respect to Figs. 1-4, 8a-10e may be used. In particular, the multi-channel encoder comprises a parameter calculator 82 for calculating multi-channel parameters 84 from at least two channels of a multi-channel signal 12 having two or more channels. Furthermore, the multi-channel encoder comprises a downmixer 80, which may be implemented as previously discussed and provides one or more downmix channels 40. Both the multi-channel parameters 84 and the one or more downmix channels 40 are input to an output interface 86 for outputting an encoded multi-channel signal comprising one or more downmix channels and / or multi-channel parameters. Alternatively, the output interface may be configured for storing or transmitting the encoded multi-channel signal to a multi-channel decoder, for example as illustrated in Fig. 6. The multi-channel decoder illustrated in Fig. 6 receives as input an encoded multi-channel signal 88. This signal is input to an input interface 90 which outputs, on the one hand, multi-channel parameters 92 and, on the other hand, one or more downmix channels 94. Both data items, i.e. multi-channel parameters 92 and downmix channels 94, are input to a multi-channel reconstructor 96, which reconstructs at its output an approximation of the original input channels and generally outputs output channels which may contain or consist of an output audio object or anything similar thereto as indicated by reference numeral 98.In particular, the multi-channel encoder in Figure 5 and the multi-channel decoder in Figure 6 together represent an audio processing system, where the multi-channel encoder is operable as discussed with respect to Figure 5, and the multi-channel decoder is implemented, for example, as illustrated in Figure 6, and configured for decoding the encoded multi-channel signal to obtain a reconstructed audio signal, generally illustrated at 98 in Figure 6. Thus, the procedures illustrated with respect to Figures 5 and 6 additionally represent a method of processing an audio signal including a method of multi-channel encoding and a corresponding method of multi-channel decoding.

[0113] An audio signal encoded in an inventive manner may be stored on a digital or non-transitory storage medium or transmitted over a transmission medium, such as a wireless transmission medium or a wired transmission medium, such as the Internet.

[0114] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent descriptions of corresponding methods, where a block or device corresponds to a method step or feature of a method step. Analogously, aspects described in the context of a method step also represent descriptions of a corresponding block or item or feature of a corresponding apparatus.

[0115] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. Implementation may be performed using a digital storage medium having electronically readable control signals stored thereon, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory, which cooperates with (or is capable of cooperating with) a programmable computer system in which the respective methods are performed.

[0116] Some embodiments according to the invention comprise a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to cause one of the methods described herein to be performed.

[0117] In general, embodiments of the present invention may be implemented as a computer program product having program code operable to perform one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0118] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium.

[0119] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0120] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium, or a computer readable medium) containing, having recorded thereon, the computer program for performing one of the methods described herein.

[0121] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, The data stream or the sequence of signals may for example be adapted to be transferred via a data communication connection, for example via the Internet.

[0122] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0123] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0124] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0125] The above-described embodiments merely serve to illustrate the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore the intention to be limited only by the scope of the appended claims, and not by the specific details presented throughout the description and explanation of the embodiments herein.

[0126] <Additional Notes> Example 1 A downmixer for downmixing at least two channels of a multi-channel signal (12) having two or more channels, comprising: a processor (10) for computing a partial downmix signal (14) from said at least two channels; a complementary signal calculator (20) for calculating a complementary signal from said multi-channel signal (12), said complementary signal (22) being different from said partial downmix signal (14); A downmixer for downmixing at least two channels, comprising an adder (30) for adding said partial downmix signal (14) and said complementary signal (22) to obtain a downmix signal (40) of said multi-channel signal. Example 2 the processor (10) is configured to calculate (50) the partial downmix signal (14) such that a predefined energy or amplitude relationship between the at least two channels of the multi-channel signal (12) and the partial downmix channel is satisfied when the at least two channels are in phase, and such that an energy loss is produced in the partial downmix signal with respect to the at least two channels when the at least two channels are out of phase, 2. The downmixer of claim 1, wherein the complementary signal calculator is configured to calculate (52) the complementary signal such that the energy or amplitude loss of the partial downmix signal (14) is partially or fully compensated for by the addition of the partial downmix signal (14) and the complementary signal (22) in the adder (30). Example 3 3. The downmixer of claim 1 or 2, wherein the complementary signal calculator (20) is configured to calculate the complementary signal (22) such that the complementary signal has a coherence index of less than 0.7 with respect to the partial downmix signal (14), a coherence index of 0.0 indicating perfect incoherence and a coherence index of 1.0 indicating perfect coherence. Example 4 4. The downmixer according to any one of embodiments 1 to 3, wherein the complementary signal calculator (20) is configured to use one signal of a group of signals comprising the first channel of the at least two channels, the second channel of the at least two channels, the difference between the first channel and the second channel, the difference between the second channel and the first channel, the further channel of the multi-channel signal when the multi-channel signal has more than the at least two channels, or a decorrelated first channel, a decorrelated second channel, a decorrelated further channel, a decorrelated difference involving the first channel and the second channel or a decorrelated partial downmix signal (14). Example 5 The processor (10) calculating (70) time or frequency dependent weighting coefficients for weighting the sum of the at least two channels according to a predefined energy or amplitude relationship between the at least two channels and a sum signal of the at least two channels; and comparing the calculated weighting factor with a defined threshold (72); and using (74) the calculated weighting factors for calculating the partial downmix signal (14) when the calculated weighting factors are in a first relationship to a predefined threshold value; or using (76) the predefined threshold value instead of the calculated weighting factors for calculating the partial downmix signal (14) when the calculated weighting factors are in a second relationship to the predefined threshold value that is different from the first relationship; or A downmixer as described in any one of embodiments 1 to 4, configured for deriving modified weighting coefficients using a modification function (76) when the calculated weighting coefficients are in a second relationship to the defined threshold different from the first relationship, the modification function being configured for deriving modified weighting coefficients such that the modified weighting coefficients are closer to the defined threshold than the calculated weighting coefficients. Example 6 The processor (10) calculating (70) time or frequency dependent weighting coefficients for weighting the sum of the at least two channels according to a predefined energy or amplitude relationship between the at least two channels and a sum signal of the at least two channels; and A downmixer described in any one of embodiments 1 to 5, configured for deriving modified weighting coefficients using a modification function, the modification function being such that the modified weighting coefficients result in an energy of the partial downmix signal that is smaller than the energy as defined by the defined energy relationship. Example 7 The processor (10) is configured to weight (16) the sum signal of the at least two channels using a time or frequency dependent weighting factor W1, the weighting factor being determined by the following equation for frequency bin k and time index n:

number

number

number

number

number

number

number

number

number

number

number

number

[0127] References [1] US 7,343,281 B2, “PROCESSING OF MULTI-CHANNEL SIGNALS”, Koninklijke Philips Electronics N.V., Eindhoven (NL) [2] Samsudin, E. Kurniawati, Ng Boon Poh, F. Sattar, and S. George, “A Stereo to Mono Downmixing Scheme for MPEG-4 Parametric Stereo Encoder,” in IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 5, 2006, pp. 529-532. [3] T. M. N. Hoang, S. Ragot, B. Kovesi, and P. Scalart, “Parametric Stereo Extension of ITU-T G. 722 Based on a New Downmixing Scheme,” IEEE International Workshop on Multimedia Signal Processing (MMSP) (2010). [4] W. Wu, L. Miao, Y. Lang, and D. Virette, “Parametric Stereo Coding Scheme with a New Downmix Method and Whole Band Inter Channel Time / Phase Differences,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, pp. 556-560. [5] Alexander Adami, Emanuel AP Habets, Jurgen Herre, “DOWN-MIXING USING COHERENCE SUPPRESSION”, 2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) [6] Vilkamo, Juha; Kuntz, Achim; Fug, Simone, “Reduction of Spectral Artifacts in Multichannel Downmixing with Adaptive Phase Alignment”, AES August 22, 2014 [Explanation of symbols]

[0128] 10 Processors 12 Multi-channel signals 14 Partial downmix signal 15 First Weighting Factor Calculator 16 Downmix Weighter 20 Complementary signal calculator, complementary signal generator 22 Complementary Signals 23 Complementary signal determiner, complementary signal selector 24 Second Weighting Factor Calculator 25 Weighting 30 Adder 40 Downmix signal, Downmix channel 60 Time-to-Spectral Converter 62 Spectral-to-Time Converter 64 Frequency Domain Downmix Encoder 66 Downmix Processor 80 Downmixer 82 Parameter Calculator 84 Multi-Channel Parameters 86 Output Interface 88 encoded multi-channel signals 90 Input Interface 92 Multi-Channel Parameters 94 downmix channels 96 Multichannel Reconstructor 98 Reconstructed Audio Signals

Claims

1. A downmixer for downmixing at least two channels of a multi-channel signal (12) having two or more channels, comprising: a processor (10) for calculating a partial downmix signal (14) from said at least two channels using a summation of said two or more channels, said partial downmix signal (14) having an energy loss with respect to said two or more channels; a complementary signal calculator (20) for calculating a complementary signal (22) from the multi-channel signal (12) such that, when the complementary signal (22) is added to the partial downmix signal (14), it partially or completely compensates for the energy loss, the complementary signal (22) being different from the partial downmix signal (14); an adder (30) for adding the partial downmix signal (14) and the complementary signal (22) to obtain a downmix signal (40) of the multi-channel signal, the downmix signal (40) having reduced or no energy loss with respect to the two or more channels; A down mixer comprising:

2. 2. The downmixer of claim 1, wherein the complementary signal calculator (20) is configured to calculate the complementary signal (22) such that the complementary signal (22) has a coherence index of less than 0.7 with respect to the partial downmix signal (14), a coherence index of 0.0 indicating perfect incoherence and a coherence index of 1.0 indicating perfect coherence.

3. 3. The downmixer of claim 1 or 2, wherein the complementary signal calculator (20) is configured to use a first channel of the at least two channels or a second channel of the at least two channels to calculate the complementary signal (22).

4. A downmixer as described in claim 1 or 2, wherein the complementary signal calculator (20) is configured to use the difference between a first channel and a second channel to calculate the complementary signal (22).

5. A downmixer as described in claim 1 or 2, wherein the complementary signal calculator (20) is configured to use further channels of the multi-channel signal when the multi-channel signal has more than the at least two channels to calculate the complementary signal (22).

6. A downmixer as described in claim 1 or 2, wherein the complementary signal calculator (20) is configured to use a de-correlated first channel, a de-correlated second channel, a de-correlated further channel, or a de-correlated difference involving the first channel or the second channel to calculate the complementary signal (22).

7. A downmixer as claimed in claim 1 or 2, wherein the complementary signal calculator (20) is configured to use a decorrelated partial downmix signal for calculating the complementary signal (22).

8. A method for downmixing at least two channels of a multi-channel signal (12) having two or more channels, comprising: - calculating a partial downmix signal (14) from said at least two channels using a summation of said two or more channels, said partial downmix signal (14) having an energy loss with respect to said two or more channels; - calculating a complementary signal (22) from the multi-channel signal (12) such that, when added to the partial downmix signal (14), the complementary signal (22) partially or completely compensates for the energy loss, the complementary signal (22) being different from the partial downmix signal (14); adding said partial downmix signal (14) and said complementary signal (22) to obtain a downmix signal (40) of said multi-channel signal, said downmix signal (40) having reduced or no energy loss with respect to said two or more channels; A method comprising:

9. a parameter calculator (82) for calculating multi-channel parameters (84) from at least two channels of a multi-channel signal having two or more channels; A downmixer (80) according to any one of claims 1 to 7; an output interface (86) for outputting or storing one or more downmix signals (40) and / or an encoded multi-channel signal including said multi-channel parameters (84); A multi-channel encoder comprising:

10. 1. A method for encoding a multi-channel signal, comprising the steps of: Calculating multi-channel parameters (84) from at least two channels of a multi-channel signal having two or more channels; downmixing according to the method of claim 8; outputting or storing an encoded multi-channel signal (88) comprising one or more downmix signals (40) and said multi-channel parameters (84); A method comprising:

11. A multi-channel encoder according to claim 9 for generating an encoded multi-channel signal (88); a multi-channel decoder for decoding the encoded multi-channel signal (88) to obtain a reconstructed audio signal (98); An audio processing system comprising:

12. 1. A method for processing an audio signal, comprising the steps of: A method for encoding a multi-channel signal as claimed in claim 10, multi-channel decoding the encoded multi-channel signal to obtain a reconstructed audio signal (98); A method comprising:

13. 13. A computer program for carrying out the method of any one of claims 8, 10 or 12 when running on a computer or processor.

Images