Method and device for multi-channel comfort noise injection into a decoded sound signal - Patents.com

Abstract

A method and device for injecting multi-channel comfort noise into a decoded multi-channel sound signal is implemented in a multi-channel sound decoder, where a background noise in a decoded mono downmix signal is estimated, and comfort noise for each of a plurality of channels of the decoded multi-channel sound signal is calculated in response to the estimated background noise, and the calculated comfort noise is injected into each channel of the decoded multi-channel sound signal.

Description

[Technical field]

[0001] The present disclosure relates to audio coding, and in particular, but not exclusively, to audio codecs, and in particular, but not exclusively, to methods and devices for multi-channel comfort noise injection into a decoded audio signal in a decoder of a stereo audio codec.

[0002] In this disclosure and the accompanying claims: - The term "sound" may relate to voice, acoustics and other sounds. - The term "stereo" is an abbreviation of "stereophonic." - The term "mono" is an abbreviation of "monophonic." [Background technology]

[0003] Historically, conventional telephones have been implemented with handsets that have only one transducer to output sound to only one of the user's ears. In the last decade, users have begun to use portable handsets with headphones that receive sound at both of the user's ears, primarily for listening to music, and occasionally for listening to voice. Nevertheless, when portable handsets are used to send and receive conversational voice, the content is still mono, but when a headset is used, it is transmitted to both of the user's ears.

[0004] With the latest 3GPP (Third Generation Partnership Project) speech coding standard, designated Enhanced Voice Services (EVS), described in reference [1], the entire contents of which are incorporated herein by reference, the quality of coded sounds, e.g., voice and / or acoustics, transmitted and received via portable handsets has improved significantly. The next logical step is to transmit stereo information such that the receiver gets as close as possible to the real acoustic scene captured at the other end of the communication link.

[0005] Efficient stereo coding techniques have been developed and are used for low bit rates. As a non-limiting example, so-called parametric stereo coding constitutes one efficient technique for low bit rate stereo coding.

[0006] Parametric stereo encodes two, left and right channels as a mono signal using a common mono codec and a certain amount of stereo side information (corresponding to the stereo parameters) that represents the stereo image. The two input, left and right channels are downmixed to a mono signal, for example by adding the left and right channels and dividing the sum by two. The stereo parameters are usually calculated in the transform domain, for example the Discrete Fourier Transform (DFT) domain, and are related to so-called binaural or inter-channel cues. Binaural cues (references [2] and [3], the contents of which are incorporated herein by reference in their entirety) include Interaural Level Difference (ILD), Interaural Time Difference (ITD), and Interaural Correlation (IC). Depending on the signal characteristics, the stereo scene configuration, etc., some or all binaural cues are coded and transmitted to the decoder. Information about which binaural cues are coded and transmitted is usually sent as signaling information that is part of the stereo side information. The binaural cues may also be quantized (coded) using the same or different coding techniques, which result in a variable number of bits being used. In addition to the quantized binaural cues, the stereo side information may include, usually at medium to high bit rates, a quantized residual signal resulting from downmixing, which is obtained, for example, by calculating the difference between the left and right channels and dividing the difference by two. The binaural cues, the residual signal, and the signaling information may be coded using an entropy coding technique, for example an arithmetic encoder. Additional information about arithmetic encoders may be found, for example, in reference [1]. In general, parametric stereo coding is most efficient at medium to low bit rates.

[0007] Moreover, in the last few years, sound generation, recording, representation, encoding, transmission, and reproduction have been transformed into improved interactive and immersive experiences for listeners. An immersive experience can be described as, for example, being deeply immersed or absorbed in a sound scene while sounds come from all directions. In immersive sound (also called 3D (three-dimensional) sound), the sound image is reproduced in all three dimensions around the listener, taking into account a wide range of sound characteristics, such as accuracy of sound quality, directionality, reverberation, transparency, and (auditory) spaciousness. Immersive sound is generated for a specific sound playback or reproduction system, such as a loudspeaker-based system, an integrated reproduction system (sound bar), or headphones. The interactivity of the sound reproduction system may then include, for example, the ability to adjust sound levels, change the position of the sound, or select a different language for reproduction.

[0008] Recently, 3GPP (Third Generation Partnership Project) has started working on developing a 3D sound codec for immersive services called IVAS (Immersive Voice and Audio Services), based on the EVS codec (see Reference [4], the entire contents of which are incorporated herein by reference). Summary of the Invention [Means for solving the problem]

[0009] The present disclosure relates to a method, implemented in a multi-channel sound decoder, for injecting multi-channel comfort noise into a decoded multi-channel sound signal, comprising the steps of estimating background noise in a decoded mono downmix signal, calculating a comfort noise for each of a plurality of channels of the decoded multi-channel sound signal in response to the estimated background noise, and injecting the calculated comfort noise into the respective channel of the decoded multi-channel sound signal.

[0010] The present disclosure also relates to a device implemented in a multi-channel sound decoder for injecting comfort noise into a decoded multi-channel sound signal, the device comprising: an estimator of background noise in the decoded mono downmix signal; and a comfort noise injector for calculating a comfort noise for each of a plurality of channels of the decoded multi-channel sound signal depending on the estimated background noise, and for injecting the calculated comfort noise into the respective channel of the decoded multi-channel sound signal.

[0011] The above and other objects, advantages and features of methods and devices for multi-channel comfort noise injection will become more apparent upon reading the following non-limiting description of illustrative embodiments thereof, given by way of example only with reference to the accompanying drawings. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic block diagram illustrating simultaneously a parametric stereo decoder and a corresponding parametric stereo decoding method, including a device for multi-channel comfort noise injection and a method for multi-channel comfort noise injection; [Diagram 2]2 is a schematic diagram illustrating a converter of a mono downmix signal to the frequency domain and simultaneously the operation of converting the mono downmix signal to the frequency domain; [Diagram 3] 1 is a graph showing power spectrum compression. [Figure 4] 11 is a schematic flow chart showing an initialization procedure for a background noise estimation calculation. [Diagram 5] FIG. 2 is a simplified block diagram of an exemplary configuration of hardware components forming the parametric stereo decoder and decoding methods described above, including devices and methods for multi-channel comfort noise injection. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present disclosure generally relates to multi-channel, e.g. stereo, comfort noise injection techniques in sound decoders.

[0014] The stereo comfort noise injection technique is described with reference to a parametric stereo sound decoder of the IVAS coding framework, referred to throughout this disclosure as the IVAS codec (or IVAS sound codec), purely as a non-limiting example, however, it is within the scope of this disclosure to incorporate such multi-channel comfort noise injection techniques into other types of multi-channel sound decoders and codecs.

[0015] 1. Introduction Mobile communication scenarios involving stereo signal capture may use low-bitrate parametric stereo encoding, for example as described in references [2] or [3]. In low-bitrate parametric stereo encoders, a single transmission channel is typically used to transmit a mono downmix sound signal. The downmixing process is designed to extract signals from the main direction of the incoming sound. The quality of the representation of the mono downmix signal is determined to a large extent by the underlying core codec. Due to the limitations of the available bit budget, the quality of the decoded mono downmix signal is often not very good, especially in the presence of background noise as described in reference [5], the entire contents of which are incorporated herein by reference. As a non-limiting example, in the case of a CELP-based core codec, the available bit budget is distributed between the encoding of various components such as the spectral envelope of the excitation signal, the adaptive codebook, the fixed codebook, the gain of the adaptive codebook, and the fixed codebook gain. In active segments of a noisy speech signal, the amount of bits allocated to the encoding of the fixed codebook is not sufficient for a transparent representation of it. Spectral holes may be observed in some frequency regions in the spectrogram of the synthesized sound signal, for example between formants. When listening to the synthesized sound signal, the background noise is perceived as intermittent, thereby degrading the performance of the parametric stereo encoder.

[0016] The technical effect of the method and device according to the present disclosure for stereo comfort noise injection into a decoded sound signal in a decoder of a sound codec, in particular, but not limited to, a parametric stereo decoder, is to reduce the adverse effect of insufficient background noise representation in the codec. The decoded sound signal is analyzed during inactive segments where background noise is assumed to be present without speech. A long-term estimate of the spectral envelope of the background noise is calculated and stored in the memory of the decoder. A synthetically created copy of the background noise is then generated in active segments of the decoded sound signal and injected into the decoded sound signal. The method and device for stereo comfort noise injection according to the present disclosure differs from the so-called "comfort noise addition" applied, for example, in the EVS codec (reference [1]). The differences include, among others, at least the following aspects: The estimation of the background noise spectral envelope in the parametric stereo decoder is performed using infinite impulse response (IIR) filtering combined with adaptive boosting of the obtained and filtered spectrum in frequency sections using massive averaging. - Stereo comfort noise generation and injection is performed in the up-mixed stereo signal, separately for the left and right channel.

[0017] The disclosed method and device for stereo comfort noise injection may be part of a parametric stereo decoder of an IVAS sound codec.

[0018] 2. Parametric Stereo Decoder FIG. 1 is a schematic block diagram illustrating simultaneously a parametric stereo decoder 100 and a corresponding parametric stereo decoding method 150, including a device for and a method for stereo comfort noise injection.

[0019] As already mentioned, the stereo comfort noise injection device and method will be described with reference to a parametric stereo decoder in the IVAS sound codec, purely as a non-limiting example.

[0020] 2.1 Demultiplexer 1, the parametric stereo decoding method 150 includes an operation 151 of receiving a bitstream from a parametric stereo encoder of the IVAS sound codec. To perform the operation 151, the parametric stereo decoder 100 includes a demultiplexer 101.

[0021] The demultiplexer 101 recovers from the received bitstream (a) an encoded mono downmix signal 131, e.g. in the time domain, and (b) encoded stereo parameters 132, such as the above-mentioned ILD, ITD and / or IC binaural cues, and possibly the above-mentioned quantized residual signal resulting from the downmixing.

[0022] 2.2 Core Decoder 1 comprises an operation 152 of core decoding the encoded mono downmix signal 131. To perform the operation 152, the parametric stereo decoder 100 comprises a core decoder 102.

[0023] By way of a non-limiting example, the core decoder 102 may be a CELP (Code-Excited Linear Prediction) based core codec, which then uses CELP techniques to obtain, in the time domain, a decoded mono downmix signal 133 from the received encoded mono downmix signal 131.

[0024] It is within the scope of this disclosure to use other types of core decoder techniques, such as Algebraic Code-Excited Linear Prediction (ACELP), Transform-Coded eXcitation (TCX), or Generic audio Signal Coder (GSC).

[0025] Additional information about CELP, ACELP, TCX, and GSC decoders can be found, for example, in reference [1].

[0026] 2.3 Stereo Parameter Decoder 1, the parametric stereo decoding method 150 comprises an operation 160 of decoding the encoded stereo parameters 132 from the demultiplexer 101 to obtain decoded stereo parameters 145. To perform the operation 160, the parametric stereo decoder 100 comprises a decoder 110 of the stereo parameters.

[0027] Naturally, the stereo parameter decoder 110 uses a decoding technique that corresponds to the technique used to encode the stereo parameters 132 .

[0028] For example, if the above-mentioned binaural cues, residual signal, and signaling information are encoded using an entropy coding technique, such as arithmetic coding, the decoder 110 uses a corresponding entropy / arithmetic decoding technique to restore these binaural cues, residual signal, and signaling information.

[0029] 2.4 Frequency Conversion 1, the parametric stereo decoding method 150 includes an operation 154 of frequency transforming the decoded mono downmix signal 133. To perform the operation 154, the parametric stereo decoder 100 comprises a frequency transform calculator 104.

[0030] The calculator 104 converts the time-domain decoded mono downmix signal 133 into a frequency-domain mono downmix signal 135. For that purpose, the calculator 104 uses a frequency transform such as the Discrete Fourier Transform (DFT) or the Discrete Cosine Transform (DCT).

[0031] 2.5 Stereo Upmixing The parametric stereo decoding method 150 comprises an operation 155 of stereo upmixing the frequency domain mono downmix signal 135 from the frequency transform calculator 104 and the decoded stereo parameters 145 from the stereo parameter decoder 110 to generate a frequency domain left channel 136 and a right channel 137 of a decoded stereo sound signal. To perform the operation 154, the parametric stereo decoder 100 comprises a stereo upmixer 105.

[0032] An example of stereo upmixing the frequency domain mono downmix signal 135 from the frequency conversion calculator 104 and the decoded stereo parameters 145 from the stereo parameter decoder 110 to generate a frequency domain left channel 136 and a frequency domain right channel 137 is described, for example, in References [2], [3], and [6], the contents of which are incorporated herein by reference in their entireties.

[0033] 2.6 Inverse Frequency Transform The parametric stereo decoding method 150 includes an operation 157 of inverse frequency transforming the upmixed frequency domain left channel 138 and right channel 139. To perform operation 157, the parametric stereo decoder 100 comprises an inverse frequency transform calculator 107.

[0034] In particular, calculator 107 inversely transforms frequency domain left channel 138 and right channel 139 into time domain left channel 140 and right channel 141. For example, if calculator 104 uses a discrete Fourier transform, calculator 107 uses an inverse discrete Fourier transform. If calculator 104 uses a DCT transform, calculator 107 uses an inverse DCT transform.

[0035] Additional information regarding parametric stereo encoders and decoders can be found, for example, in references [2], [3], and [6].

[0036] 3. Stereo Comfort Noise Injection As described herein below, the parametric stereo decoding method 150 of FIG. 1 includes a stereo comfort noise injection method, and the parametric stereo decoder 100 of FIG. 1 includes a stereo comfort noise injection device.

[0037] 3.1 Background Noise Estimation 1, the stereo comfort noise injection method of the parametric stereo decoding method 150 includes an operation of background noise estimation 153. To perform the operation 153, the stereo comfort noise injection device of the parametric stereo decoder 100 comprises a background noise estimator 103.

[0038] The background noise estimator 103 of the parametric stereo decoder 100 of Fig. 1 estimates the background noise envelope, for example by analyzing the decoded mono downmix signal 133 during speech inactivity. The background noise envelope estimation process is performed on short frames, typically having a duration of 15-30 ms. Frames of a given duration, each containing a given number of subframes and containing a given number of consecutive sound signal samples, are used to process sound signals in the field of sound signal coding, and more information about such frames can be found, for example, in reference [1].

[0039] The information about voice inactivity is calculated in a parametric stereo encoder (not shown) of the IVAS sound codec using a Voice Activity Detection (VAD) algorithm similar to that used in the EVS codec (reference [1]) and is represented by a binary VAD flag f VAD Alternatively, the binary VAD flag f VAD may be coded as part of the encoder type parameter, for example as described in the EVS codec [1]. The encoder type parameter in the EVS codec is selected from the following set of signal classes: INACTIVE, UNVOICED, VOICED, GENERIC, TRANSITION, and AUDIO. When the decoded encoder type parameter is INACTIVE, the VAD flag f VAD is '0'. In all other cases, the VAD flag is '1'. The binary VAD flag f VAD If f is transmitted in the bitstream and cannot be inferred from the encoder type parameters, it can be explicitly calculated in the background noise estimator 103 by running a VAD algorithm on the decoded mono downmix signal 133. VAD may be expressed, for example, using the following relationship (1):

[0040]

number

[0041] where n is the index of the sample of the decoded mono downmix signal 133 and N is the total number of samples in the current frame (the length of the current frame).d (n), n = 0,...,N-1.

[0042] Estimation of the background noise envelope by analyzing the mono downmix signal 133 decoded during speech inactivity is described later in this specification in sections 3.1.1 to 3.1.5.

[0043] 3.1.1 Power spectrum compression The background noise estimator 103 transforms the decoded mono downmix signal 133 into the frequency domain using a DFT transform. The DFT transform process 200 is shown in a schematic diagram in Figure 2. The input to the DFT transform 201 includes the current frame 202 and the previous frame 203 of the decoded mono downmix signal 133. Thus, the length of the DFT transform is 2N.

[0044] To reduce the effect of spectral leakage at frame boundaries, the decoded mono downmix signal 133 is first multiplied with a tapered window, e.g., a normalized sine window 204. The raw sine window w s (n) can be expressed using the following relationship (2):

[0045]

number

[0046] Sine window w s (n) is normalized, for example, using the following relationship (3): sn (n)).

[0047]

number

[0048] Decoded mono downmix signal 133(md (n)) is, for example, calculated by using the following relation (4) to calculate the normalized sine window w sn (n) is windowed (m w (n)). m w (n)=m d (n)w sn (n), n = 0, …, 2N-1(4) (4)

[0049] The windowed, decoded mono downmix signal m w (n) is then transmitted using a DFT transform 201, for example using relation (5) below.

[0050]

number

[0051] Since the input decoded mono downmix signal 133 is real, its spectrum (see 205 in Fig. 2) is symmetric and only the first half, i.e. N spectral bins (k), are taken into account when calculating the power spectrum of the decoded mono downmix signal 133. This can be expressed using the following relationship (6):

[0052]

number

[0053] As can be seen from relation (6), the power spectrum of the decoded mono downmix signal 133 (see 206 in FIG. 2) is normalized (1 / N 2 ).

[0054] The normalized power spectrum P(k) is compressed in the frequency domain by compacting the frequency bins into frequency bands. As a non-limiting example, assume that the decoded mono downmix signal 133 is sampled at a sampling frequency of 16 kHz and the length of a frame is 20 ms. The total number of samples in every frame is N=320, and the length of the FFT (Fast Fourier Transform used to calculate the DFT) transform is 2N=640. The total number of frequency bands is denoted as B. The process 300 of compacting the spectral bins into frequency bands is shown in FIG. 3 for an exemplary case of N=320. In this example, 320 bins 301 of the normalized power spectrum P(k) ranging from 0 Hz to 8 kHz are compressed into B=61 frequency bands 302.

[0055] The human auditory system is more responsive to spectral components at low frequencies. Thus, in the example partitioning scheme of FIG. 3, a single bin partition of f BIN = 950Hz. The index corresponding to this frequency is k BIN In this illustrative example, the last frequency index of the bin-wise partition is k BIN = 38. At low frequencies, k BIN Until then, no spectral compression is performed and the bin-wise power spectrum is simply copied to a band-wise (compressed) power spectrum. This can be expressed, for example, using the following relationship (7): N(k)=P(k), k=0,…,k BIN (7)

[0056] k BIN For frequencies higher than k, the background noise estimator 103 compresses the bin-wise power spectrum using spectral averaging of the frequency bins of the power spectrum P(k) in the corresponding frequency band. This is done by first finding the average N 0 This is done by calculating (b).

[0057]

number

[0058] where b represents the frequency band and the range 〈k low (b),k high (b)〉 identifies a set of frequency bins in the bth frequency band, of which k low (b) is the lowest frequency bin, k high (b) is the highest frequency bin. In the exemplary case where the number of frequency bins N=320, the allocation of frequency bins to frequency bands is defined in Table 1, where k mid (b) represents the mid-frequency bin of frequency band b.

[0059] [Table 1]

[0060] 3.1.2 Compensating for dispersion losses The above-described spectral averaging of relation (8) tends to reduce the variance of the background noise. To compensate for the loss of variance, the background noise estimator 103 adds random Gaussian noise to the average power spectrum. This is done as follows: First, the background noise estimator 103 calculates the variance σ(b) of the random Gaussian noise in each frequency band b, for example using the following relation (9):

[0061]

number

[0062] The random Gaussian noise generated by the background noise estimator 103 has zero mean and a variance calculated in each frequency band using equation (9). The generated random Gaussian noise is

[0063]

number

[0064] The addition of the generated random Gaussian noise N(b) to the compressed power spectrum can therefore be expressed using relationship (10):

[0065]

number

[0066] 10 -5 There is a limited value of the compressed power spectrum below . The addition of random Gaussian noise to the average power spectrum is performed only after an initialization procedure, which is described later in this disclosure.

[0067] 3.1.3 Spectral smoothing The background noise estimator 103 smoothes the compressed power spectrum N(b) in the frequency domain using nonlinear IIR filtering. The IIR filtering operation is performed using the VAD flag f VAD As a general rule, the smoothing is stronger during the inactive segments and weaker during the active segments of the decoded stereo sound signal. The smoothed compressed power spectrum depends on

[0068]

number

[0069] , b=0,…,B-1.

[0070] In the inactive segments of the decoded stereo signal, the VAD flag f VAD When is '0', IIR smoothing is performed, for example, using the following relationship (11):

[0071]

number

[0072] where the index m in parentheses has been added to indicate the current frame. In the first line of relation (11), a fast downward update of the compressed power spectrum is performed on a single bin section with a forgetting factor α of 0.8. In the second line of relation (11), only a slow upward update is performed for all bands of the compressed power spectrum with a factor α of 1.05. The third line of relation (11) represents the default IIR filter configuration with a forgetting factor α of 0.95 for all cases other than those described by the conditions in lines 1 and 2.

[0073] In the active segment of the decoded stereo signal, the VAD flag f VAD When is equal to '1', the background noise estimator 103 performs IIR smoothing only in some selected frequency bands. The smoothing operation is performed using an IIR filter with a forgetting factor proportional to the ratio of the total energy of the compressed power spectrum to the total energy of the smoothed compressed power spectrum.

[0074] Total energy of the compressed power spectrum E N can be calculated, for example, using the following relation (12):

[0075]

number

[0076] Total energy of the smoothed compressed power spectrum

[0077]

number

[0078] can be calculated, for example, using the following relation (13):

[0079]

number

[0080] Total energy of the compressed power spectrum E N and the total energy of the smoothed compressed power spectrum

[0081]

number

[0082] Relative to enr can be calculated, for example, using the following relation (14):

[0083]

number

[0084] where ε is a small constant value added to avoid division by zero, e.g., ε=10. -7 It is.

[0085] Energy ratio r enr If is lower than 0.5, the total energy E of the compressed power spectrum N is the total energy of the smoothed compressed power spectrum

[0086]

number

[0087] In this case, the smoothed compressed power spectrum of the current frame m is

[0088]

number

[0089] is updated, for example, using the following relation (15):

[0090]

number

[0091] Therefore, in all bands where a significant energy drop is detected in the current frame, the smoothed compressed power spectrum

[0092]

number

[0093] The energy of is the energy ratio r enr It updates fairly quickly, proportional to

[0094] Energy ratio r enr If is greater than or equal to 0.5, the smoothed compressed power spectrum

[0095]

number

[0096] is updated only in the frequency band above 2275 Hz, which corresponds to b≧50 in this exemplary embodiment. First, the background noise estimator 103 calculates the smoothed compressed power spectrum

[0097]

number

[0098] Calculate the short-term average of

[0099]

number

[0100] However, for b=50,…,B-1

[0101]

number

[0102] The short-term smoothed compressed power spectrum is r enr The background noise estimator 103 is updated every frame, regardless of the value of r enr Smoothed compressed power spectrum of the frame, ≧0.5

[0103]

number

[0104] Update.

[0105]

number

[0106] Again, only downward updates (energy drops are detected in the current frame) are allowed, but the updates are enr <0.5 is slower than the case

[0107] The smoothed, compressed power spectrum described in this section 3.1.3.

[0108]

number

[0109] The updates of are changed during the initialization procedure described in the next section of this disclosure.

[0110] 3.1.4 Initialization procedure The background noise estimation computation 153 requires a proper initialization. Figure 4 is a schematic flow chart illustrating the initialization procedure of the background noise estimation computation 153. During such initialization procedure 400, the background noise estimator 103 uses a successive IIR filter to generate a smoothed compressed power spectrum.

[0111]

number

[0112] Update.

[0113] The background noise estimator 103 calculates the smoothed compressed power spectrum

[0114]

number

[0115] A counter c of consecutive inactive frames is updated. CNI (f VAD Use = "0" for the counter c CNI is initialized to 0 (block 401 in FIG. 4) at the start of the initialization procedure 400 (block 402 in FIG. 4). The background noise estimator 103 also uses a binary flag f CNI Use the binary flag f CNI A counter c is also initialized to 0 at the start of the initialization procedure 400 (block 401 in FIG. 4). CNI and flag f CNI is updated using a simple state machine as illustrated in FIG.

[0116] Referring to FIG. 4, the initialization procedure 400 includes, for each frame, the following sub-operations: - binary flag f CNIis set to "1" (sub-operation 404), the initialization procedure 400 is completed and terminated (sub-operation 411). - binary flag f CNI is set to "0" (sub-operation 404), and the binary VAD flag f VAD If is set to "1" (sub-operation 405), indicating an active frame, the counter c CNI is reset to 0 (sub-operation 406 ) and the initialization procedure 400 returns to sub-operation 404 . - binary flag f CNI is set to "0" (sub-operation 404), and the binary VAD flag f VAD If is set to "0" (sub-operation 405), indicating an inactive frame, the background noise estimator 103 uses a continuous IIR filter to calculate the smoothed compressed power spectrum

[0117]

number

[0118] (sub-operation 403). - Smoothed and compressed power spectrum in sub-operation 403

[0119]

number

[0120] Following the update of the counter c CNI is the function of the parameter c MAX (sub-operation 408). - The comparison in sub-operation 408 is done with counter c CNI is the parameter c MAX If it is less than the counter c CNI is incremented by "1" (sub-operation 409), and the initialization procedure 400 returns to sub-operation 404. - The comparison in sub-operation 408 is done with counter c CNI is the parameter c MAXBinary flag f if it indicates equal to or greater than CNI is set to "1" (sub-operation 410), and the initialization procedure 400 is completed and terminated (sub-operation 411).

[0121] As can be seen, the initialization procedure 400 generates a smoothed compressed power spectrum for a given number of consecutive inactive frames.

[0122]

number

[0123] is updated. This is completed after the parameter c MAX As a non-limiting example, the parameter c MAX is set to 5. The parameter c MAX Setting σ to a higher value may result in an initialization procedure 400 of the background noise estimation computation 153 that is more stable but requires a longer period of time to complete the initialization.

[0124]

number

[0125] It is not wise to extend the initialization period too long, since it is used for stereo comfort noise injection and during discontinuous transmission (DTX). More information on DTX operation can be found, for example, in reference [1].

[0126] During the initialization procedure 400, the background noise estimator 103 calculates a smoothed compressed power spectrum using, for example, a continuous IIR filter using the following relationship (18):

[0127]

number

[0128] (sub-operation 403).

[0129]

number

[0130] where [m] is the frame index, for b=0,…,B-1

[0131]

number

[0132] In this way, the forgetting factor α=1 / (c CNI +1) is the counter c CNI and hence the smoothed compressed power spectrum

[0133]

number

[0134] is proportional to the number of updated inactive frames. Using this initialization procedure 400, the smoothed compressed power spectrum

[0135]

number

[0136] contains meaningful spectral information about the background noise. For example, if it occurs that DTX activity is detected at the decoder before the initialization procedure is completed, the smoothed compressed power spectrum

[0137]

number

[0138] It is still possible to use as an estimate of the background noise.

[0139] 3.1.5 Power spectrum expansion Similar to the power spectrum compression shown in FIG. 3 and described in Section 3.1.1, the background noise estimator 103 generates a smoothed compressed power spectrum

[0140]

number

[0141] At low frequencies, k BIN Until then, no expansion is performed and the band-wise compressed power spectrum is copied to the bin-wise (expanded) power spectrum, for example using the following relation (19):

[0142]

number

[0143] k BIN For frequencies higher than , the background noise estimator 103 expands the band-wise compressed power spectrum using linear interpolation in the logarithmic domain as described in reference [1]. To that end, the background noise estimator 103 uses, for example, the following relationship (20) to calculate the multiplicative increase β mult Calculate the minutes first.

[0144]

number

[0145] where b identifies the frequency band, and k mid (b) identifies the middle bin of the bth band. The expanded power spectrum is then divided into bins for all b=k BIN Calculated for +1,…,B-1.

[0146]

number

[0147] In relations (20) and (21), the frame index [m] is omitted for simplicity.

[0148] Expanded power spectrum

[0149]

number

[0150] Since σ i is calculated according to relations (19) and (21) in the inactive frames, it represents an estimate of the background noise in the decoded mono downmix signal 133.

[0151] 3.2 Stereo comfort noise injection 1, the parametric stereo decoding method 150 includes an operation 156 of injecting comfort noise from the stereo upmixer 105 into the left channel 136 and the right channel 137. To perform operation 156, the parametric stereo decoder 100 includes a stereo comfort noise injector 106.

[0152] The stereo comfort noise injection (CNI) technique of operation 156 is based on the comfort noise addition (CNA) technique originally developed and integrated in the 3GPP EVS codec (reference [1]). The purpose of the CNA in the EVS codec is to compensate for the loss of energy resulting from the ACELP-based encoding of a noisy speech signal (reference [5]). The loss of energy is especially noticeable at low bit rates when the number of available bits in the ACELP encoder is insufficient to encode the fixed contribution of the excitation (fixed codebook index and gain). As a result, the energy of the decoded signal in the spectral valleys between the speech formants is lower than that of the original signal. This leads to the undesirable effect of "noise attenuation", which is perceived unpleasantly by the listener. The addition of random noise with an appropriate level and spectral shape covers the spectral valleys, thereby raising the noise floor and resulting in the uninterrupted perception of the background noise. In the EVS decoder, comfort noise is generated and added to the decoded signal in the frequency domain.

[0153] It is possible to generate and inject comfort noise into the decoded mono downmix signal 133 of the parametric stereo decoder 100. However, the decoded mono downmix signal 133 is converted into the left channel 136 and the right channel 137 during the stereo upmixing operation 155. This may result in an undesired spatial unmasking effect, since the spatial characteristics of the dominant sound and the spatial characteristics of the ambient (background) noise, represented by the decoded mono downmix signal 133, may be quite different. To avoid this problem, the comfort noise is generated after the stereo upmixing operation 155 and injected separately into the left channel 136 and the right channel 137. The spatial characteristics of the background noise are estimated directly in the decoder during the inactive segments.

[0154] 3.2.1 Estimation of background noise spatial characteristics at the decoder Assuming the decoder 100 is operating in a non-DTX mode of operation, the spatial characteristics of the background noise are determined by the VAD flag f set to "0". VAD The spatial parameters may be estimated during inactive segments of the decoded stereo sound signal signaled by: . The main spatial parameter is inter-channel coherence (ICC). Since the estimation of the ICC parameters involves a transformation of the decoded stereo signal (left and right channels) into the frequency domain, it is very complicated to calculate such an ICC parameter. A reasonable approximation of the ICC parameter is the inter-channel correlation (IC) parameter, which may be calculated in the time domain. The IC parameter may be calculated by the stereo comfort noise injector 106, for example, using the following relationship (22):

[0155]

number

[0156] where l(n) and r(n) are the left and right channels of the time-domain decoded stereo sound signal, respectively, calculated from the frequency-domain left channel 136 and right channel 137 using an inverse frequency transform to that used in calculator 104, N is the number of samples in the current frame, [m] is the frame index, and the index LR refers to left (L) and right (R) to indicate that the parameter IC relates to the correlation between the left and right channels.

[0157] The second spatial parameter estimated in the decoder 100 is the inter-channel level difference (ILD). The stereo comfort noise injector 106 estimates the ratio c of the energy of the left channel l(n) to the energy of the right channel r(n) of the decoded stereo sound signal in the current frame using, for example, the following relationship (23): LRThe parameter ILD may be calculated by expressing:

[0158]

number

[0159] The ILD parameter is then calculated, for example, using the following relationship (24):

[0160]

number

[0161] Since both spatial parameters, IC and ILD, are calculated from the same, single frame, their variations are large. Therefore, the stereo comfort noise injector 106 smoothes the IC and ILD spatial parameters using IIR filtering. The smoothed inter-channel correlation (IC) parameter is, for example, given by the following relationship (25):

[0162]

number

[0163] and a smoothed inter-channel level difference (ILD) parameter may be calculated using, for example, the following relationship (26):

[0164]

number

[0165] During the initialization procedure 400 of FIG. CNI When =0, the stereo comfort noise injector 106 sets the smoothed IC and ILD parameters to their instantaneous values, as follows:

[0166]

number

[0167]

number

[0168] and

[0169]

number

[0170] The initial value is "0".

[0171] 3.2.2 Stereo comfort noise generation and injection The stereo comfort noise injector 106 generates and injects stereo comfort noise in the frequency domain. In the following non-limiting example of an implementation: The complex spectrum of the left channel 136 of the decoded stereo sound signal in the frequency domain is denoted as L(k), where k=0, . . . , M−1, and M is the length of the FFT transform used in the frequency conversion operation 154. The complex spectrum of the right channel 137 of the decoded stereo sound signal in the frequency domain is denoted as R(k), where k=0,...,M-1.

[0172] According to the previous non-limiting implementation example, the decoded mono downmix signal is sampled at 16 kHz and the background noise is estimated in the frequency range of 0-8000 Hz. For a normal background noise injection in the upmix domain (left channel 136 and right channel 137), the sampling rate of the left channel 136 and the right channel 137 is at least 16 kHz. As a non-limiting example, it is assumed that the left channel 136 and the right channel 137 of the decoded stereo sound signal are sampled at 32 kHz for a number of samples per frame M=640. This corresponds to an FFT length of 20 ms, which is also the frame length in the parametric stereo decoder 100. For example, the frequency resolution of the background noise spectrum P is 25 Hz, while the frequency resolution of the spectrum of the left channel 136 and the right channel 137 of the decoded stereo sound signal is 50 Hz. The frequency resolution mismatch can be resolved during the stereo comfort noise generation by averaging the level of the background noise in two adjacent spectral bins, as will be explained in the following description.

[0173] The stereo comfort noise injector 106 generates two random signals with Gaussian probability density functions (PDF), for example using the following relationship (28): G 1 (k)~N(0,1) G 2 (k)~N(0,1) (28) where k=0,...,M-1, M is the number of samples per frame. Two random signals G1(k) and G2(k) are mixed together to create the left and right channels of stereo comfort noise. The mix is ​​designed to match the spatial characteristics of the estimated background noise, represented by the smoothed inter-channel correlation (IC) parameter described in relation (25) and the smoothed inter-channel level difference (ILD) parameter described in relation (26). The stereo comfort noise injector 106 calculates the mix coefficient γ, for example, using the following relation (29):

[0174]

number

[0175] The spectral envelope of the stereo comfort noise (comfort noise for left and right channels) is controlled by the expanded power spectrum (estimated background noise in the decoded mono downmix signal 133), calculated in relations (19) and (21). Also, the frequency resolution of the expanded power spectrum is reduced by a factor of “2”.

[0176] Expanded power spectrum

[0177]

number

[0178] The minimum and maximum levels in each pair of adjacent frequency bins of are expressed, for example, using the following relationship (30):

[0179]

number

[0180] where N is the number of frequency bins and k is the frequency bin index.

[0181] The stereo comfort noise injector 106 then performs a frequency resolution reduction, for example using the following relationship (31):

[0182]

number

[0183] Therefore, by relation (31), the level of comfort noise for injection in the left channel 136 and the right channel 137 in the frequency domain is determined by the spread power spectrum at the adjacent frequency pins.

[0184]

number

[0185] Maximum of

[0186]

number

[0187] and the minimum

[0188]

number

[0189] If the ratio of exceeds the threshold of 1.2, the expanded power spectrum

[0190]

number

[0191] In one situation, the stereo comfort noise level is set to the minimum level in two adjacent frequency bins of 100 kHz. This prevents excessive comfort noise injection in signals with a strong tilt of the estimated background noise. In another situation, the level of the stereo comfort noise is set to the average level over two adjacent frequency bins.

[0192] The stereo comfort noise injector 106 may, for example, use the following relationship (32) to determine the coefficient N / 2 and the global gain g scale The scaling factor r calculated using scaleScale the level of the stereo comfort noise using (k).

[0193]

number

[0194] where N is the number of frequency bins, k is the frequency bin index, and g scale is the global gain, which is explained later in this disclosure.

[0195] The mixture of two random signals using a Gaussian PDF can be described, for example, by the following pair of equations (33):

[0196]

number

[0197] However, N L (k) and N R (k) is the generated comfort noise signal for injection into the left 136 and right 137 channels, respectively. In equation (33), the generated comfort noise signal N L (k) and N R The stereo comfort noise injector 106 finally injects the generated comfort noise signals N(k) into the left 136 (L(k)) and right 137 (R(k)) channels of the decoded stereo sound signal, using, for example, the following relationship (34): L (k) and N R Inject (k). L(k)=L(k)+N L (k), where k=0,…,N / 2-1 R(k)=R(k)+N R (k), where k=0,…,N / 2-1 (34)

[0198] 3.2.3 Using the Decoded Spatial Parameters In the case of a parametric stereo encoder as described in [6], it is possible to code and transmit the IC / ICC and ILD parameters in the bitstream. The transmitted IC / ICC and ILD parameters are then used by the stereo comfort noise injector 106 instead of the parameters estimated in section 3.2.1. Typically in a parametric stereo encoder, the parameters IC / ICC and ILD are calculated and encoded in the frequency domain for each critical band.

[0199] The decoded IC / ICC and ILD parameters can be shown, for example, as follows:

[0200]

number

[0201] where the subscript PS stands for parametric stereo, and B PS Let b denote the number of frequency bands b used by the parametric stereo encoder, and the maximum frequency of the parametric stereo encoder can be expressed as the last index of the last frequency band as follows: k max_PS =max(k(B PS -1)) (36)

[0202] Similarly, the mixing coefficients γ shown in relation (29) may be calculated for each frequency band using the decoded stereo parameters IC / ICC and ILD, for example using the following relation (37):

[0203]

number

[0204] however

[0205]

number

[0206] is the decoded inter-channel coherence parameter in the bth band, as defined in relation (35),

[0207]

number

[0208] is the decoded inter-channel level difference parameter in the bth band, defined in equation (35).

[0209] The stereo comfort noise injector 106 then performs the blending process, for example using the following relationship (38):

[0210]

number

[0211] where γ(b k ) is the bth frequency bin that contains the kth frequency bin. k Therefore, in the frequency bins that belong to the same frequency band, the comfort noise signal N L (k) and N R In generating (k), a single value of the mix coefficient is used, one for each frequency band. L (k) and N R (k) is min(k max_PS , N / 2−1)) are generated.

[0212] The stereo comfort noise injector 106 may, for example, again use relationship (33) to inject the generated comfort noise signals N(k) into the left 136 (L(k)) and right 137 (R(k)) channels of the decoded stereo sound signal. L(k) and N R Inject (k).

[0213] 3.2.4 DTX mode When the IVAS sound codec operates in DTX mode, the background noise estimation described in section 3.1 is not performed. Instead, information about the spectral envelope of the background noise is decoded from the Silence Insertion Descriptor (SID) frame and converted to a power spectrum representation. This can be done in different ways depending on the SID / DTX scheme used by the codec. For example, the TD-CNG or FD-CNG techniques from the EVS codec (reference [1]) may be used since they both contain information about the background noise envelope.

[0214] The IC / ICC and ILD spatial parameters may also be transmitted as part of the SID frame, in which case the decoded spatial parameters are used in stereo comfort noise generation and injection as described in Section 3.2.3.

[0215] 3.2.5 Soft VAD parameters To prevent abrupt changes in the level of the injected stereo comfort noise, the stereo comfort noise injector 106 applies a fade-in-fade-out strategy to the noise injection. For that purpose, soft VAD parameters are used, which are determined by a binary VAD flag f VAD This is achieved by smoothing the

[0216]

number

[0217] however

[0218]

number

[0219] represents the soft VAD parameters, and f VAD represents the unsmoothed binary VAD flag, and [m] is the frame index.

[0220] From the relationship (39), it can be seen that the soft VAD parameters are bounded to the range from 0 to 1. The soft VAD parameters are the VAD flag f VAD rises faster when going from 0 to 1, and rises less fast when going from 1 to 0. Thus, the fade-out period is longer than the fade-in period.

[0221] During the initialization procedure 400 of FIG. CNI When =0, the soft VAD parameter is set to "0".

[0222]

number

[0223]

number

[0224] The initial value of is 0.

[0225] 3.2.6 Global Gain Control The level of the stereo comfort noise is determined by the global gain g used in relation (32). scale The stereo comfort noise injector 106 is globally controlled using a global gain g scale is initialized to “0”, and the global gain g scale Update.

[0226]

number

[0227] however

[0228]

number

[0229] are the soft VAD parameters calculated in equation (39). During the initialization period, f CNI When = 0, the global gain g scale is reset to "0". Therefore, the global gain g scale are the soft VAD parameters

[0230]

number

[0231] , thereby applying a fade-in / fade-out effect to the injected stereo comfort noise.

[0232] 4. Exemplary Configurations of Hardware Components FIG. 5 is a simplified block diagram of an exemplary configuration of hardware components forming the above-described parametric stereo decoder including a device for stereo comfort noise injection.

[0233] The parametric stereo decoder including the device for stereo comfort noise injection may be implemented as part of a mobile terminal, as part of a portable media player, or in any similar device. The parametric stereo decoder including the device for stereo comfort noise injection (identified in FIG. 5 as 500) comprises an input unit 502, an output unit 504, a processor 506, and a memory 508.

[0234] The input unit 502 is configured to receive a bitstream (FIG. 1) from a parametric stereo encoder (not shown). The output unit 504 is configured to feed the left channel 140 and the right channel 141 (FIG. 1). The input unit 502 and the output unit 504 may be implemented in a common module, for example a serial input / output device.

[0235] The processor 506 is operatively connected to the input 502, the output 504, and the memory 508. The processor 506 is implemented as one or more processors for executing code instructions supporting the functionality of the various elements and operations of the parametric stereo decoders and decoding methods described above, including the devices and methods for stereo comfort noise injection shown in the accompanying figures and / or described in this disclosure.

[0236] The memory 508 may comprise non-transitory memory for storing code instructions executable by the processor 506, in particular processor-readable memory that stores non-transitory instructions that, when executed, cause the processor to perform elements and operations of parametric stereo decoders and decoding methods, including devices and methods for stereo comfort noise injection. The memory 508 may also comprise random access memory or buffers for storing intermediate processed data from various functions performed by the processor 506.

[0237] Those skilled in the art will appreciate that the description of the parametric stereo decoders and decoding methods, including devices and methods for stereo comfort noise injection, are merely illustrative and are not intended to be limiting in any manner. Other embodiments will readily occur to those skilled in the art having the benefit of this disclosure. Furthermore, the disclosed parametric stereo decoders and decoding methods, including devices and methods for stereo comfort noise injection, can be customized to provide useful solutions to existing needs and problems of encoding and decoding sound, e.g., stereo sound.

[0238] For clarity, not all of the routine features of an implementation of a parametric stereo decoder and decoding method, including a device and method for stereo comfort noise injection, are shown and described. It will of course be appreciated that in the development of any such actual implementation of a parametric stereo decoder and decoding method, including a device and method for stereo comfort noise injection, numerous implementation-specific decisions may need to be made to achieve the developer's particular goals, such as complying with application, system, network, and business-related constraints, and that these particular goals will vary from implementation to implementation and from developer to developer. Furthermore, it will be appreciated that the development effort will be complex and time-consuming, but will nevertheless be a routine undertaking for those skilled in the art of sound processing having the benefit of this disclosure.

[0239] According to the present disclosure, the elements, processing operations, and / or data structures described herein may be implemented using various types of operating systems, computing platforms, network devices, computer programs, and / or general-purpose machines. In addition, those skilled in the art will recognize that devices of a less general-purpose nature, such as hardware devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc., may also be used. When a method including a series of operations and sub-operations is implemented by a processor, computer, or machine, and the operations or sub-operations may be stored as a series of non-transitory code instructions readable by the processor, computer, or machine, they may be stored in a tangible and / or non-transitory medium.

[0240] The elements and processing operations of the parametric stereo decoders and decoding methods, including the devices and methods for stereo comfort noise injection described herein, may include software, firmware, hardware, or any combination of software, firmware, or hardware suitable for the purposes described herein.

[0241] In parametric stereo decoders and decoding methods, including devices and methods for stereo comfort noise injection, various processing operations and sub-operations may be performed in various orders, and some of the processing operations and sub-operations may be optional.

[0242] Although the present disclosure has been described above with respect to its non-limiting, exemplary embodiments, these embodiments can be freely modified within the scope of the appended claims without departing from the spirit and essence of the present disclosure.

[0243] 5. References This disclosure refers to the following references, the entire contents of which are incorporated herein by reference: (References) [1] 3GPP TS 26.445, v.16.1.0, “Codec for Enhanced Voice Services (EVS); Detailed Algorithmic Description”, July 2020. [2] E. Schuijers, W. Oomen, B. den Brinker, and J. Breebaart, “Advances in parametric coding for high-quality audio,” in Proc. 114th AES Convention, Amsterdam, The Netherlands, Mar. 2003, Preprint 5852. [3] F. Baumgarte, C. Faller, “Binaural cue coding - Part I: Psychoacoustic fundamentals and design principles,” IEEE Trans. Speech Audio Processing, vol. 11, pp. 509-519, Nov. 2003. [4] 3GPP SA4 contribution S4-170749, “New WID on EVS Codec Extension for Immersive Voice and Audio Services”, SA4 meeting #94, June 26-30, 2017, http: / / www.3gpp.org / ftp / tsg_sa / WG4_CODEC / TSGS4_94 / Docs / S4-170749.zip [5] R. Hagen and E. Ekudden, “An 8 kbit / s ACELP coder with improved background noise performance,” 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258), Phoenix, AZ, USA, 1999, pp. 25-28 vol.1, doi: 10.1109 / ICASSP.1999.758053. [6] J. Breebaart, S. van de Par, A. Kohlrausch, “Parametric Coding of Stereo Audio.” EURASIP Journal of Advanced Signal Processing 2005, 561917 (2005). https: / / doi.org / 10.1155 / ASP.2005.1305

Symbol Explanation

[0244] 100 Parametric Stereo Decoder 101 Demultiplexer 102 Core Decoder 103 Background Noise Estimator 104 Frequency Conversion Calculator 105 Stereo Up Mixer 106 Stereo Comfort Noise Injector 107 Inverse Frequency Transform Calculator 110 Stereo Parameter Decoder 131 Encoded mono downmix signal 132 encoded stereo parameters 133 Decoded Mono Downmix Signal 135 Frequency domain mono downmix signal 136 Left Channel 137 Right Channel 138 Left Channel 139 Right Channel 140 Left Channel 141 Right Channel 145 decoded stereo parameters 500 Parametric Stereo Decoder 502 Input section 504 Output section 506 Processor 508 Memory

Claims

1. 1. A device implemented in a multi-channel sound decoder for injecting multi-channel comfort noise into a decoded multi-channel sound signal, comprising: an estimator of background noise in the decoded mono downmix signal; a multi-channel comfort noise injector for calculating a comfort noise for each of a plurality of channels of the decoded multi-channel sound signal in response to the estimated background noise and for injecting the calculated comfort noise into the respective channel of the decoded multi-channel sound signal; A device comprising:

2. The device of claim 1 , wherein the background noise estimator estimates a background noise envelope by analyzing the decoded mono downmix signal during speech inactivity.

3. The device of claim 1 or 2, wherein the background noise estimator is configured to calculate a power spectrum of the decoded mono downmix signal and to compress the power spectrum of the decoded mono downmix signal.

4. 4. The device of claim 3, wherein the background noise estimator is configured to calculate a frequency transform of the decoded mono downmix signal and to calculate the power spectrum of the decoded mono downmix signal using the frequency transform of the decoded mono downmix signal.

5. 5. The device of claim 4, wherein to calculate the frequency transform of the decoded mono downmix signal, the background noise estimator windowizes the decoded mono downmix signal and applies the frequency transform to the windowed decoded mono downmix signal.

6. 6. The device of claim 3, wherein the background noise estimator is configured to normalize the power spectrum of the decoded mono downmix signal and to compress the normalized power spectrum.

7. 7. The device of claim 3, wherein the background noise estimator compresses the power spectrum of the decoded mono downmix signal by consolidating frequency bins of the power spectrum into frequency bands above a given frequency.

8. 8. The device of claim 7, wherein the background noise estimator does not perform a compression of the power spectrum but converts frequency bins to respective frequency bands below the given frequency.

9. 9. The device of claim 7 or 8, wherein at frequencies higher than the given frequency, the background noise estimator condenses the frequency bins of the power spectrum into frequency bands using spectral averaging of the frequency bins of the power spectrum in each frequency band, and wherein to spectrally average the frequency bins of the power spectrum in each frequency band, the background noise estimator calculates the variance of the frequency bins of the power spectrum in each frequency band.

10. 10. The device of claim 3, wherein the background noise estimator adds random Gaussian noise to the compressed power spectrum to compensate for loss of variance of the estimated background noise.

11. 11. The device of claim 10, wherein the background noise estimator is configured to calculate a variance of the random Gaussian noise in each frequency band using the power spectrum of the decoded mono downmix signal to generate random Gaussian noise with zero mean and calculated random Gaussian noise variance.

12. 12. The device of claim 3, wherein the background noise estimator smoothes the compressed power spectrum using an infinite impulse response (IIR) filter.

13. 13. The device of claim 12, wherein the IIR filter is responsive to a voice activity detection (VAD) flag in a current frame such that the smoothing of the compressed power spectrum is stronger during inactive segments of the decoded multi-channel sound signal and weaker during active segments of the decoded multi-channel sound signal.

14. The device according to claim 12 or 13, wherein the background noise estimator comprises a continuous IIR filter for updating the smoothed compressed power spectrum over a number of consecutive inactive frames.

15. A device described in any one of claims 12 to 14, wherein for a given value of a voice activity detection (VAD) flag and a given value of the ratio of the total energy of the compressed power spectrum to the total energy of the smoothed compressed power spectrum, the background noise estimator updates the smoothed compressed power spectrum in the current frame in a frequency band above a given frequency.

16. 16. The device of claim 12, wherein the background noise estimator expands the smoothed compressed power spectrum.

17. 17. The device of claim 16, wherein the background noise estimator does not perform a widening of the smoothed compressed power spectrum up to a given frequency.

18. 18. The device of claim 16 or 17, wherein the background noise estimator expands the smoothed compressed power spectrum using linear interpolation with multiplicative increase at frequencies higher than the determined frequency.

19. 19. The device of claim 16, wherein the injector of comfort noise uses the extended power spectrum to control a spectral envelope of stereo comfort noise.

20. 20. The device of claim 19, wherein the comfort noise injector performs a reduction in frequency resolution by setting a level of comfort noise to a minimum level in two adjacent frequency bins of the expanded power spectrum if a ratio between a maximum level and a minimum level of comfort noise in the two adjacent frequency bins exceeds a given threshold.

21. A device as described in claim 19 or 20, wherein if the ratio between minimum and maximum levels does not exceed a certain threshold, the comfort noise injector performs a reduction in frequency resolution by setting the level of comfort noise to the average value of the minimum and maximum levels of comfort noise in two adjacent frequency bins of the expanded power spectrum.

22. 22. The device of claim 20 or 21, wherein the injector of comfort noise is configured to scale the level of comfort noise for injection into each channel of the decoded multi-channel sound signal using a scaling factor.

23. 23. The device of claim 22, wherein the injector of comfort noise calculates the scaling factor using a number of frequency bins divided by two and a global gain.

24. 24. The device of claim 23, wherein the comfort noise injector calculates the global gain by: (a) smoothing a binary voice activity detection (VAD) flag to generate a soft VAD parameter bounded to a range of 0 to 1; and (b) generating the global gain as a function of the soft VAD parameter.

25. 24. The device of claim 23, wherein the comfort noise injector generates the comfort noise for each channel of the decoded multi-channel sound signal as a function of a scaling factor, spatial parameters in a current frame of the decoded multi-channel sound signal, and a random signal.

26. 1. A method implemented in a multi-channel sound decoder for injecting multi-channel comfort noise into a decoded multi-channel sound signal, comprising: estimating the background noise in the decoded mono downmix signal; calculating a comfort noise for each of a plurality of channels of the decoded multi-channel sound signal in response to the estimated background noise, and injecting the calculated comfort noise into the respective channel of the decoded multi-channel sound signal. A method comprising:

27. 27. The method of claim 26, wherein estimating background noise comprises estimating a background noise envelope by analyzing the decoded mono downmix signal during speech inactivity.

28. 28. The method of claim 26 or 27, wherein the step of estimating background noise comprises the steps of: calculating a power spectrum of the decoded mono downmix signal; and compressing the power spectrum of the decoded mono downmix signal.

29. 29. The method of claim 28, wherein estimating background noise comprises calculating a frequency transform of the decoded mono downmix signal and calculating the power spectrum of the decoded mono downmix signal using the frequency transform of the decoded mono downmix signal.

30. 30. The method of claim 29, wherein estimating background noise comprises windowing the decoded mono downmix signal and applying the frequency transform to the windowed decoded mono downmix signal to calculate the frequency transform of the decoded mono downmix signal.

31. 31. The method according to claim 28, wherein the step of estimating background noise comprises the steps of normalizing the power spectrum of the decoded mono downmix signal and compressing the normalized power spectrum.

32. 32. The method according to claim 28, wherein estimating background noise comprises consolidating frequency bins of a power spectrum of the decoded mono downmix signal into frequency bands above a given frequency to compress the power spectrum of the decoded mono downmix signal.

33. 33. The method of claim 32, wherein estimating background noise does not perform compression of the power spectrum below the given frequency, but includes transforming frequency bins into respective frequency bands.

34. 34. The method of any one of claims 28 to 33, wherein estimating background noise comprises adding random Gaussian noise to the compressed power spectrum to compensate for loss of variance of the estimated background noise.

35. 35. The method of claim 34, wherein estimating background noise comprises calculating a variance of the random Gaussian noise to generate random Gaussian noise having zero mean and the calculated random Gaussian noise variance.

36. 36. The method of any one of claims 28 to 35, wherein estimating the background noise comprises smoothing the compressed power spectrum using infinite impulse response (IIR) filtering.

37. 37. The method of claim 36, wherein the IIR filtering is responsive to a voice activity detection (VAD) flag in a current frame such that the smoothing of the compressed power spectrum is stronger during inactive segments of the decoded multi-channel sound signal and weaker during active segments of the decoded multi-channel sound signal.

38. 38. The method of claim 36 or 37, wherein estimating the background noise comprises using a continuous IIR filter to update the smoothed compressed power spectrum over a number of consecutive inactive frames.

39. 39. The method of any one of claims 36 to 38, wherein estimating background noise comprises expanding the smoothed compressed power spectrum.

40. 40. The method of claim 39, wherein estimating background noise comprises not performing widening of the smoothed compressed power spectrum up to a given frequency.

41. 41. The method of claim 39 or 40, wherein estimating the background noise comprises expanding the smoothed compressed power spectrum using linear interpolation with multiplicative increases at frequencies higher than the determined frequency.

42. 42. The method of any one of claims 39 to 41, wherein calculating and injecting multi-channel comfort noise comprises using the extended power spectrum to control a spectral envelope of stereo comfort noise.

43. 43. The method of claim 42, wherein the step of calculating and injecting multi-channel comfort noise comprises the step of performing a reduction in frequency resolution by setting a level of comfort noise to a minimum level in two adjacent frequency bins of the expanded power spectrum if a ratio between a maximum level and a minimum level of comfort noise in the two adjacent frequency bins exceeds a given threshold.

44. A method as described in claim 42 or 43, wherein if the ratio between minimum and maximum levels does not exceed a certain threshold, the step of calculating and injecting multi-channel comfort noise includes a step of performing a reduction in frequency resolution by setting the level of comfort noise to the average value of the minimum and maximum levels of comfort noise in two adjacent frequency bins of the expanded power spectrum.

45. 45. The method of claim 26, wherein the step of calculating and injecting multi-channel comfort noise comprises generating the comfort noise for each channel of the decoded multi-channel sound signal as a function of a random signal, a scaling factor, a mixing coefficient for mixing the random signals together to create a channel of the multi-channel comfort noise, and Inter-Channel Correlation (IC) and Inter-Channel Level Difference (ILD) spatial parameters in a current frame of the decoded multi-channel sound signal.

Images