Optimization process for reducing stereo audio signal channels
By determining half of the phase difference between the two channels of the stereo signal for phase-corrected filtering, combined with frame-by-frame decision-making and filter adaptation, the sound coloration and artifact problems of stereo signal downmixing are solved, and high-quality mono signal generation is achieved.
Patent Information
- Application Number
- CN202480024741.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-23
- Filing Date
- 2024-04-10
- Publication Date
- 2025-12-05
AI Technical Summary
Existing technologies suffer from sound coloration, reverberation enhancement, and loss of sound source intelligibility when downmixing stereo signals to mono signals. Furthermore, existing methods are prone to producing audible artifacts and time delays during frequency domain filtering.
By determining half of the phase difference between the two channels of the stereo signal for phase-corrected filtering, and combining frame-by-frame decision-making and filter adaptation, a suitable downmixing mode is selected, including phase-corrected filtering and time difference index filtering, to avoid spectral variations and audible artifacts.
It effectively reduces coloration and reverberation, avoids spectral variations and audible artifacts, adapts to the characteristics of stereo signals, and improves downmixing quality.
Smart Images

Figure CN121079736A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the general field of audio signal processing. More specifically, it relates to downmixing multi-channel audio signals. Particular interest is focused on downmixing stereo signals to mono signals.
[0002] This type of processing is generally applicable to the field of audio technology, and more specifically to the field of audio encoding and decoding, whether in the encoding or decoding steps. Background Technology
[0003] Downmixing refers to deriving a signal y consisting of a smaller number (M) of channels from the combination of N channels of a multi-channel signal x. In practice, this involves defining functions. , so that:
[0004] ,in,
[0005] In this invention, we focus on the special case where M = 1 and N = 2; this is then referred to as mono downmixing. In the case of N = 2, it is called stereo content or stereo signal, where channel 1 ( ) is called the left channel or channel L, and channel 2 ( The right channel (or channel R) is referred to as the right channel. The signal y obtained after downmixing is called a mono signal (monophonic or mono), which will be denoted as M below.
[0006] These stereo signals can be captured using stereo microphone formats. A certain number of microphone formats can be used to create this type of content.
[0007] Perhaps the most popular is the XY system, which uses two cardioid microphones with an azimuth difference between 90° and 135°. The MS (center-side) system is another widely used system, consisting of a cardioid microphone and a second microphone (called a "figure-eight" microphone) oriented at 90°. By combining (and / differentializing) these two microphones, left and right stereo content channels can be created. These systems are called coincident systems, meaning there is no delay between the microphone pickups, and spatial perception is achieved through the sound intensity difference between the left and right channels.
[0008] Another class of formats (called phase stereo formats) uses microphones that are far apart from each other, so that a phase difference is created between the channels of a sound source that is located closer to one of the microphones. The most famous is the AB format, which uses 2 omnidirectional microphones, spaced a few centimeters to a few meters apart. Another widely used format is the ORTF format, which uses both the phase difference created by a 17 cm spacing between the microphones, and the amplitude difference created by a cardioid directivity (the microphones are at an angle of about 90°). Binaural formats refer to the placement of omnidirectional microphones in the ear canals of an artificial head: this type of device makes it possible to natively create content that can be listened to through headphones.
[0009] The simplest way to produce a downmix signal by downmixing is a method called passive downmixing. This method consists in averaging the left and right channels of a stereo signal, so that:
[0010]
[0011] Or, in the frequency domain:
[0012]
[0013] where, is the fast Fourier transform (FFT) of , f is the index of the frequency, and k is the index of the frame:
[0014]
[0015] where k is the index of a frame of size L, and N is the size of the FFT, and is a sine window or a Hann window or other apodization window adapted to the frame size. may be the signal itself or its zero-padded version: for example, in the case where the frame size L is smaller than the size N of the FFT, may be:
[0016]
[0017] This very simple method works well in the case where the right and left channels are in phase. However, in the case where the microphones do not coincide and there is a phase difference, this mixing creates a comb filter, which manifests as a coloring of the original signal. This is due to the fact that, depending on the distance between the microphones, the position of the sound source relative to the pair of microphones, and the frequency, the left and right channels of the same sound source can be in phase (as Figure 1 depicted in a), or in anti-phase (as Figure 1The left and right signals either add up (constructive interference) or cancel out (destructive interference), resulting in a downmix signal Level variations and coloration effects occur. Coloration results from comb-filter effects: the effects of constructive and destructive interference are manifested in different frequency bands, changing the balance of the downmixed signal with respect to the original signal, and thus its tonality.
[0018] To correct this defect of level intensity variation with frequency, the downmix of the e-AAC+ codec is level-corrected , which preserves the energy level of the downmix signal in each frequency band comparable to the original signal:
[0019] where
[0020]
[0021] This approach makes it possible to compensate to some extent for the intensity drop resulting from downmixing. However, when the signal is close to antiphase, this compensation leads to over-amplification of the signal: in practice, the correction factor is therefore capped (at twice the value).
[0022] With regard to the same approach, Figure 7 a method is described for downmixing a stereo channel to a mono channel, integrated in the public candidate version IVAS codec, available at the following address:
[0023] https: / / forge.3gpp.org / rep / ivas-codec-pc / ivas-codec / - / tree / main.
[0024] The signal x(m) with 2 input channels is de-interleaved (block 701) to obtain the left and right channels. Next, a frequency analysis is performed using windowing and Fourier transform (blocks 702 and 703) in order to obtain the spectra of the 2 channels, and the cross-correlation is estimated based on the phase spectrum (block 704), then the time difference between the channels (here labeled ITD, although it is strictly the ICTD between the 2 channels) is determined by searching for the peak (block 705); block 705 also provides the correlation level corresponding to the ITD.
[0025] The downmix factor is then determined as follows (block 706):
[0026] if (itd!= 0):
[0027] if itd > 0, then
[0028] if itd < 0, then
[0029] else
[0030] g = 0.5
[0031] The two channels are combined into a mono signal in block 706 by the gain g (where the value of the gain is smoothed sample by sample at the beginning of each frame). Next, the energies of the left and right channels and the energy of the mono signal are determined (blocks 708, 709, 710). In blocks 713 and 713, the left and right channels are re-injected (added) to the mono signal resulting from block 706 according to the energy compensation determined in block 711 and with the respective scale factors defined in blocks 712 and 713.
[0032] In addition to the signal coloring due to the downmix (related to the incomplete correction of the comb filtering), these methods have the problem of excessive reverberation related to the summation of the different signals. In particular, it is unrealistic to apply the same delay compensation for all frequencies, since the delay is related to the various sound sources that make up the content. In practice, this leads to additional coloring in addition to a potential loss of intelligibility of the sound sources, which cannot be corrected by the level compensation.
[0033] Other downmix methods seek to circumvent the aforementioned drawbacks. The principle of these methods is to perform a phase correction on the left and right signals before summing them. A method is proposed in the document by Samsudin, E. Kurniawati, N. Boon Poh, F. Sattar, S. George entitled "A stereo to mono downmixing scheme for MPEG-4 parametric stereo encoder" (Proceedings of ICASSP, 2006), which consists in applying a phase shift φ in the frequency domain to one of the channels (usually the right channel) in order to phase correct it with respect to the left channel, so that:
[0034]
[0035] The ideal phase shift is given by the so-called IPD (Inter-channel Phase Difference):
[0036]
[0037] where is the conjugate of and ∠ indicates the phase of the complex operand.
[0038] The phase shift is computed in the frequency domain: this makes it possible to phase correct the spectral components of various sound sources - a sound source with its own IPD can dominate in a frequency band f, while another sound source with a different IPD can dominate in another frequency f'.
[0039] In the case of the MPEG4 codec described in the Samsudin document cited above, the IPD applied is estimated by averaging over the Bark bands, rather than computing the IPD for each band, which proves to be significantly noisy and varies greatly from one frame to another.
[0040] In published patent application WO 2017103418, a downmix method is proposed that combines the method proposed by Samsudin with passive downmixing. Notably, this patent application proposes an ISD (inter-spectral distance) index that makes it possible to select, on a per-band basis, the method that is best suited to phase correcting the signal:
[0041]
[0042] This index makes it possible to measure whether the signal is in-phase (ISD < 1), that is to say that there is a phase difference or IPD in the interval , or anti-phase (ISD > 1), that is to say that the IPD is in the interval . When the signal is in anti-phase, a high ISD value is equivalent to the levels between the left and right channels being comparable, and phase correction is essential to avoid coloration and dropout. Conversely, an ISD value close to 1 indicates that one channel dominates over the other; in the latter case, phase correction is irrelevant, since the downmix is roughly equivalent to the dominant signal. However, if the right channel dominates, it is important to avoid modifying the phase of the right channel. Therefore, in this patent application, the following downmix selection mechanism is proposed:
[0043]
[0044] The passive downmix proposed in the aforementioned patent application can sometimes lead to an increase in reverberation when the ISD is low.
[0045] Moreover, the method proposed in this patent application requires filtering in the frequency domain, the downmix signal being based on a reconstruction of the short-time inverse Fourier transform (STFT) of the signal M(f, k). This type of filtering generates a circular convolution, which results in audible artefacts: echoes, pre-echoes and noise. In order to mask these artefacts, an implementation using frame overlap combined with a suitable windowing (analysis window and / or synthesis window) can be used, thus ensuring the filtered signal reconstruction. Generally, this type of overlap is not compatible with the operation of audio encoders working with adjacent frames (i.e. without overlap, N = L); in addition, this type of reconstruction results in a delay (the usual overlap amount is generally half a frame), which is not acceptable for encoders having strict constraints in terms of latency.
[0046] Another overlap-add (OLA) implementation is also possible: this method requires a step of zero-padding of the signal and the filter in order to avoid circular convolution. However, tests have shown that, for such filters with a phase that varies very quickly from one frame to another, the OLA method does not completely mask the artefacts at the transition of certain frames, so that the artefacts can still be heard. SUMMARY
[0047] The present invention aims to improve the prior art.
[0048] To this end, the invention relates to a method for downmixing a stereo signal to obtain a mono signal, the method comprising, for a frame of the stereo signal, selecting between:
[0049] - a first downmix mode comprising a phase-corrected filtering in each channel of the stereo signal by an angle defined by half of the phase difference determined between the two channels of the stereo signal, and
[0050] - a second downmix mode comprising a phase-corrected filtering in one channel of the stereo signal by an angle defined by the phase difference determined between the two channels of the stereo signal,
[0051] The selection of the first downmix mode or the second downmix mode depends on the value of a phase indicator determined for each frame.
[0052] Thus, by shifting the two channels of the stereo signal according to the first downmix mode, it is possible to limit or even prevent the increase in reverberation. Moreover, compared to the Samsudin method described above which modifies the single right channel by the phase difference, shifting by half of the phase difference determined between the two channels of the stereo signal greatly limits the colouration of the modified left and right signals spectrum.
[0053] By selecting between the two downmix modes, it is possible to adapt the downmix to the channels of the stereo signal.
[0054] In the case where the intensity of the right channel is dominant (the phase indicator ISD is close to 1) and the downmix is mainly performed, it is advantageous to select the first downmix mode: its spectral variations will be reduced, practically unhearable.
[0055] The present application thus improves the prior art by providing a downmix method selection adapted to the case of signal in-phase or volume imbalance.
[0056] In one embodiment, the method further comprises a step of determining, for each frequency f, a phase indicator measuring the degree of phase opposition between the frequency components of the channels of the stereo signal, then determining a phase indicator per frame of the stereo signal, the phase indicator being computed based on the percentage of frequencies for which the phase indicator per frequency exceeds a first threshold.
[0057] In one embodiment, the first downmix mode is applied to the current frame if the phase indicator per frame is less than a second threshold, and the second downmix mode is applied to the current frame if the phase indicator per frame is greater than the second threshold.
[0058] This frame-by-frame decision thus makes it possible to avoid large phase shifts between frequencies, and other audible artefacts that can occur when deciding per frequency as in the prior art method.
[0059] In another embodiment, a time difference indicator between the stereo channels is determined for each frame of the stereo signal, and a third downmix operation is implemented when this indicator is higher than a threshold.
[0060] Thus, in this embodiment, another criterion is used to select another downmix method, called the third operation.
[0061] In one particular embodiment, the time difference indicator between the stereo channels determined for each frame of the stereo signal is taken into account, and the phase indicator per frame is compared to the second threshold to determine whether the first or the second downmix operation is applied to the frame when this indicator is less than a threshold.
[0062] The comparison of the time difference indicator is then performed before the comparison of the phase indicator with another threshold. By these different comparisons, the downmix operation can be best adapted to the characteristics of the stereo signal.
[0063] In one embodiment, the phase difference determined between the two channels of the stereo signal is smoothed over time via a forgetting factor.
[0064] In particular, the phase difference can vary rapidly over time. Therefore, the filters used for various downmix operations can vary very rapidly from frame to frame, which creates discontinuities that are sometimes audible during the inter-frame transition. Thus, smoothing the phase difference can limit this effect.
[0065] In one embodiment, the downmix operation corresponds to filtering the channels using filters, the method comprising a preliminary phase of defining the filters by at least one truncation and windowing operation, before applying to the stereo signal.
[0066] This adaptation of the filters makes it possible to avoid the circular convolution and time delay that can occur in the prior art methods.
[0067] In one embodiment, the preliminary phase of defining the filters comprises the following adaptation steps:
[0068] - obtaining the impulse responses of the filters corresponding to the downmix operation;
[0069] - truncating a portion of the impulse responses;
[0070] - weighting the remaining portion by applying a weighting window;
[0071] - normalizing the impulse responses obtained by the windowing to obtain adapted filters to be applied to a frame of the stereo signal.
[0072] Advantageously, the downmix operation with adapted filters is applied frame by frame by a cross-fade step that takes into account the impulse responses of the filters used by the previous frame of the current frame.
[0073] This cross-fade transition can avoid audible artefacts when the filters differ greatly between two consecutive frames, for example when a new sound source appears.
[0074] The invention is directed to a downmix device comprising processing circuitry for implementing the steps of the downmix method as described above.
[0075] The invention relates to a computer program comprising instructions for implementing the downmix method as described above when executed by a processor.
[0076] Finally, the invention relates to a storage medium that can be read by a processor and that stores a computer program comprising instructions for executing the downmix method described above. BRIEF DESCRIPTION OF DRAWINGS
[0077] Other features and advantages of the present application will become more apparent from the following description, given by way of non-limiting example only, of a particular embodiment thereof and with reference to the annexed drawings, in which:
[0078] [ Figure 1 a] shows the channels of a mono signal as described above;
[0079] [ Figure 1 b] shows the channels of an inverted stereo signal as described above;
[0080] [ Figure 2 ] shows, in block diagram form, a downmix sequence in an embodiment of the application;
[0081] [ Figure 3 ] shows an embodiment of filter computation and selection of downmix operation for downmix;
[0082] [ Figure 4 ] shows an embodiment of the adaptation phase of the downmix filter;
[0083] [ Figure 5 ] shows an embodiment of a transition when applying downmix during a transition between two frames of a stereo signal;
[0084] [ Figure 6 ] shows another embodiment of filter computation and selection of downmix operation for downmix;
[0085] [ Figure 7 ] shows the existing downmix method described above;
[0086] [ Figure 8 ] shows an example of a structural embodiment of a downmix device according to an embodiment of the application. DETAILED DESCRIPTION
[0087] Figure 2 An example of a sequence for processing a stereo audio signal is shown, the processing operation including a downmix operation.
[0088] At the input of this processing sequence, a stereo signal x composed of two channels (x1(n) and x2(n)), also called right and left channels, is first divided into frames of L samples (x1(k, l) and x2(k, l), k being the frame index and n the sample index) in 201. A FFT (standing for Fast Fourier Transform) is applied in block 202 in order to obtain the frequency domain signals (X1(k, f) and X2(k, f), f being the frequency index). Then, in 203, a downmix filter to be applied to the signal frame is selected and determined. Reference will be made to the following figures for a detailed description of the downmix filter selection and computation. Figure 3Describe this step. The filter thus determined is adjusted (or adapted) in 204 to make it causal and optimized, thereby reducing processing complexity.
[0089] refer to Figure 4 Describe this adaptation phase.
[0090] Once the filter is determined and adapted for the current frame, it is applied to that current frame in step 205. To avoid audible artifacts caused by filter changes between two frames, the previous frame is considered. The filter is used to perform crossfade in and out.
[0091] Reference Figure 5 Describe this step.
[0092] Figure 3 Described Figure 1 Detailed embodiment of middle frame 203. In this processing frame, the first or second downmixing operation to be applied to the current frame of the stereo signal is selected and determined.
[0093] The first downmixing method T1 is defined here by simultaneously shifting the left and right channels of the stereo signal by an angle, which is limited by half the phase difference (IPD) determined between the two channels of the stereo signal. Therefore, the resulting downmixed signal is as follows:
[0094]
[0095] This method avoids the excessive reverberation degradation caused by conventional downmixing (averaging the two signals) or the comb-filter degradation that occurs when there is a time shift between the two channels.
[0096] This type of downmixing, as defined herein, can be considered in the frequency domain as filtering according to the following equation:
[0097]
[0098] in, and .
[0099] The second downmixing method T2 is, for example, defined as the method proposed in the Samsudin paper cited above.
[0100] Then, in the second method, and .
[0101] The stereo signal channels (X1(k, f) and X2(k, f)) in the frequency domain are used on the one hand to calculate a phase indicator (which measures the degree of phase opposition between the stereo signal channels) in 301 and on the other hand to calculate the inter-channel phase difference (IPD) in 303.
[0102] The phase indicator is for example defined by the ISD (Inter-Spectral Distance) indicator as defined above.
[0103] In the embodiments described here, the ISD is determined from the stereo signal frame so as to be able to decide frame by frame whether to select the first downmix or the second downmix.
[0104] This frame by frame decision makes it possible to avoid large phase shifts between frequencies and other audible artefacts that can occur when deciding per frequency as in the above cited patent application.
[0105] In order to apply the frame by frame decision, a decision criterion is defined in 302 from the phase indicator ISD according to the following equation:
[0106]
[0107] This criterion measures the percentage of frequencies for which the criterion supports a downmix with phase correction (i.e. when is applied). When exceeds a certain threshold (set for example to 70%), it is decided to apply the first downmix with phase correction as defined above.
[0108] Then, in 305, the following downmix selection decision is made between T1 and T2:
[0109]
[0110] In 303, the inter-channel phase difference IPD between the two channels of the stereo signal is obtained by where ∠ indicates the phase of a complex operand.
[0111] However, a frame by frame decision can sometimes not be sufficient because the IPD calculated according to the above equation can vary very rapidly over time. In addition, the filter
[0112] may vary very rapidly from one frame to another, which can create discontinuities that are sometimes audible during the inter-frame transition. In order to mitigate this effect, a smoothed version of the IPD over time can be used to calculate the filter . One way of smoothing the IPD is to apply a first order IIR low pass filter in 304 so that:
[0113]
[0114] forgetting factor The forgetting factor is chosen depending on the situation. It can be advantageous to choose a high forgetting factor at lower frequencies (typically below 5 kHz) because this makes it possible to preserve the phase coherence of the low frequency part of the spectrum from frame to frame, the speech signal spectrum being more stable, especially in its low frequency part, and the human ear being particularly phase sensitive to this part of the spectrum. In contrast, it is advantageous to choose a low forgetting factor at higher frequencies: at higher frequencies (typically above 5 kHz), the phase continuity of the speech signal is less apparent and the human ear is less sensitive to the phase discontinuity of this part of the spectrum.
[0115] The forgetting factor can be chosen in the following form:
[0116]
[0117] , , and The values of the parameters are determined experimentally. In the case of an adjacent frame length L = 20 ms, the following set of parameters can be chosen:
[0118] , , ,
[0119] Figure 4 The block 204 in Figure 2 is illustrated in detail. For example, the filter defined in the downmixing methods T1 and T2 described above is adjusted or indeed adapted to avoid the appearance of a circular convolution.
[0120] In a first step, the impulse response of the filter is calculated in 401 so that:
[0121]
[0122] where the filter has a size N, which corresponds to the dimension of the FFT.
[0123] The impulse response thus defined is generally non-causal. One conventional technique for making it causal is to rotate it by L / 2 samples, so that the response is centered on L / 2. The drawback is that the filter thus reconstructed then has a time delay of L / 2. In order to avoid this time delay, it is chosen to truncate the filter in 402, keeping only a part of the first half of the impulse response , as follows:
[0124]
[0125] The advantage of this filter is that the maximum value (absolute value) usually appears on its first sample, which guarantees zero delay. In terms of frequency response, its phase is similar to that of the optimal phase-corrected filter. Their phases are approximately equal.
[0126] The choice of P depends on several criteria: a P value close to N / 2 guarantees near-optimal phase correction, and a lower value limits the computational cost of filtering. A lower P value also smooths the filter's phase. This is an advantage for filters defined by the aforementioned processing operation T2, where the phase changes very rapidly with frequency: these changes can produce extremely high group delays, which are perceptible to the human ear. A lower P value allows for the removal of all or part of these artifacts.
[0127] Such a truncated filter (Especially when P < N / 2 or even P ≪ N / 2) the impulse response tail does not approach 0. This produces an audible clicking discontinuity at the transition between two frames. Therefore, to avoid these audible artifacts, a window is used in 403. right Weighting is performed so that:
[0128]
[0129] in, It is monotonically decreasing, and its coefficients approach 0 as n approaches P. This allows for reduction of sample energy at the end of the filter and avoids discontinuities during the transition between consecutive frame filters.
[0130] The window can be a triangular half-window, or it can actually be a Hanning half-window:
[0131]
[0132] Due to truncation and windowing, the filter... Energy relative to the optimal filter Energy reduction: As a pure phase-shifting filter, the latter, by its construction, has a 1 / 2 norm. Therefore, the energy of such a truncated and windowed filter can be normalized in 404 such that:
[0133]
[0134] in, It is the 2-norm of the vector. Figure 5 Detailed description Figure 1Box 205 in the diagram shows that the OLS (overlap-store) implementation of the filter is performed in the time domain or the frequency domain. The choice of domain is based on complexity, which depends on the relative size of the filter and the size of the frames to be processed.
[0135] This section describes an OLS implementation in the time domain, where the size of adjacent frames is L.
[0136] First, in 501, the current frame is concatenated with P-1 samples from the previous frame (storage principle):
[0137]
[0138] In the 503, each channel is filtered through its own phase-shifting filter:
[0139]
[0140] Then, in 505, a mono submix is generated by summing the phase corrections of the two channels:
[0141]
[0142] In practice, the filters between two consecutive frames can be significantly different, for example, when a new sound source is introduced. In such cases... and The transitions between them may become audible. To avoid these artifacts, crossfade steps can be applied. To perform this crossfade, the filter... Multiple samples X applied to the current frame:
[0143]
[0144] This is due to Figure 5 The execution of box 502 in the middle.
[0145] The number of samples required for crossfade-in and crossfade-out must be determined experimentally based on factors such as filter size and sampling frequency.
[0146] Next, in frame 504, a mono submix is generated based on the filter from the previous frame. :
[0147]
[0148] In 506, the signal generated by the undermixing Based on cross-fade-in and fade-out, it is generated according to the following equation:
[0149]
[0150] in, is a monotonically decreasing function, where and (ε being a value close to 0 or zero). Typically, the Hanning half-window can be chosen:
[0151]
[0152] It should be noted that, when integrated with many other downmix methods as in Figure 6 , these methods are not always expressible by filtering operations. Figure 7 This is the case for the method T3 or any other method whose gain depends on the signal instantaneous level in
[0153] Figure 6 A downmix device 600 in the sense of the invention is illustrated.
[0154] In this case, one solution is to compute the downmix for each method for the current frame k and to transition between downmixes by cross-fading. More specifically, in the case of a transition from method T1 (or T2) to method T3, the downmix can be done between the mono downmix of method 2 (i.e. at the output of T1 (or T2)) and the signal of method T3. It can also be done between the output of T1 (or T2) and the signal d(n). In the latter case, the energy compensation (adapt_gain) in method T3 will be applied continuously to all methods. The decision to switch from one method to another can be driven by criteria other than ISD, for example, parameters already computed by method T3 (data parameters in Figure 6 ), such as ITD - for example by choosing to apply method T3 when ITD is very large - or any other criterion or combination of criteria.
[0155] Figure 8 A downmix device 800 in the sense of the invention is illustrated.
[0156] The device 800 comprises processing circuitry, typically comprising:
[0157] - a memory MEM1 for storing instruction data of a computer program in the sense of the invention;
[0158] - an interface INT1 for receiving a stereo audio signal x;
[0159] - a processor PROC1 for receiving the signal and processing the signal by executing the computer program instructions stored in the memory MEM1, so as to perform a downmix; in particular, the processor can control the processing modules as described with reference to Figures 2 to 5 ; and
[0160] - a communication interface COM 1 for transmitting the down-mix signal y generated by the down-mix to another processing module, for example an encoding or decoding module of an audio signal encoder or decoder.
[0161] Of course, this Figure 8 An example of a structural embodiment of a down-mix device in the sense of the application is shown. As explained above, Figures 2 to 7 A functional embodiment of the device is described in detail.
[0162] This type of down-mix is applied for example in audio encoding, for example when the remote device does not have the capability to render stereo at its terminal. In this case, it is not necessary to transmit a stereo signal, which makes it possible to save bandwidth. This type of method can operate during a point-to-point session if one of the participants has made a stereo or binaural recording. This type of method can also occur during a multi-party call: the conference bridge device spatializes the scene by creating a stereo scene, but not all participants must have the capability to render stereo. The scene is then down-mixed to mono for these participants.
[0163] The application can also be applied to audio decoding: in this case, the stereo content is down-mixed by the user's renderer (rendering module) in order to adapt to the user's limited capabilities (for example, only one loudspeaker).
Claims
1. A method for downmixing a stereo signal to obtain a mono signal, the method comprising, for a frame of the stereo signal, selecting (305) between: - a first downmix mode (Tl) comprising a phase-corrected filtering in each channel of the stereo signal by an angle defined by half of a phase difference determined (303, 304) between the two channels of the stereo signal, and - a second downmix mode (T2) comprising a phase-corrected filtering in one channel of the stereo signal by an angle defined by the phase difference determined (303, 304) between the two channels of the stereo signal, the selection of the first or second downmix mode depending on a value of a phase indicator determined (302) per frame.
2. The method of claim 1, wherein, The method further comprises a step of determining, for each frequency f ( ) a phase indicator measuring a degree of phase opposition between frequency components of the channels of the stereo signal, then determining a phase indicator for each frame of the stereo signal ( ) calculated based on a percentage of frequencies for which the phase indicator exceeds a first threshold.
3. The method of claim 2, wherein, If the phase index of each frame is less than the second threshold ( If the phase index of each frame is greater than the second threshold, then the first downmixing mode is applied to the current frame, and if the phase index of each frame is greater than the second threshold, then the second downmixing mode is applied to the current frame.
4. The method of any one of the preceding claims, wherein, determining a time difference indicator between the stereo channels for each frame of the stereo signal, a third downmix operation being implemented when the indicator is higher than a threshold.
5. The method of any one of the preceding claims, wherein, determining a time difference indicator between the stereo channels for each frame of the stereo signal, and wherein, when the indicator is lower than a threshold, a phase indicator per frame is compared to the second threshold to determine whether the first or second downmix operation is applied to the frame.
6. The method of any one of claims 1 to 5, wherein, smoothing the phase difference determined between the two channels of the stereo signal over time via a forgetting factor.
7. The method of any one of the preceding claims, wherein, The downmix operation corresponds to filtering the channels using filters, the method comprising a preliminary phase of defining the filters by at least one truncation and windowing operation before being applied to the stereo signal.
8. The method of claim 7, wherein, The preliminary phase of defining the filters comprises the following adaptation steps: - obtaining impulse responses of the filters corresponding to the downmix operation; - truncating a portion of the impulse responses; - weighting the remaining portion by applying a weighting window; - normalizing the impulse responses obtained by the windowing to obtain adapted filters to be applied to a frame of the stereo signal.
9. The method of claim 8, wherein, The downmix operation with adapted filters is applied frame by frame by a cross-fade step taking into account the impulse responses of the filters used by the previous frame of the current frame.
10. A downmix device comprising processing circuitry for implementing the steps of the downmix method as claimed in one of claims 1 to 9.
11. A storage medium readable by a processor, the storage medium storing a computer program comprising instructions for executing the method as claimed in one of claims 1 to 9.
Citation Information
Patent Citations
Adaptive channel-reduction processing for encoding a multi-channel audio signal
WO2017103418A1
