Harmonic transposition

Abstract

To relate to transposing signals in time and / or frequency and in particular to coding of audio signals.SOLUTION: More particular, the present invention relates to high frequency reconstruction (HFR) methods including a frequency domain harmonic transposer. A method and system for generating a transposed output signal from an input signal using a transposition factor T is described. The system comprises an analysis window of length La, extracting a frame of the input signal, and an analysis transformation unit of order M transforming the samples into M complex coefficients. The system further comprises a nonlinear processing unit altering the phase of the complex coefficients by using the transposition factor T, a synthesis transformation unit of order M transforming the altered coefficients into M altered samples, and a synthesis window of length Ls, generating a frame of the output signal.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to transposing signals in frequency and / or stretching / compressing signals in time, in particular for encoding audio signals. In other words, the present invention relates to modifying the time scale and / or the frequency scale. More specifically, the present invention relates to high frequency reconstruction (HFR) including a frequency domain harmonic transposer. [Background technology]

[0002] HFR techniques such as Spectral Band Replication (SBR) techniques can significantly improve the coding efficiency of traditional perceptual audio codecs. In combination with MPEG-4 Advanced Audio Coding (AAC), HFR technology forms a very efficient audio codec. It is already used in the XM Satellite Radio system and Digital Radio Mondiale, and is standardized within 3GPP, DVD Forum, etc. The combination of AAC and SBR is called aacPlus. It is part of the MPEG-4 standard, where it is referred to as the High Efficiency AAC Profile. In general, HFR technology can be combined with any perceptual audio codec in a forward and backward compatible manner, thus offering the possibility of upgrading already established broadcasting systems such as MPEG-2 Layer 2 used in the Eureka DAB system. HFR conversion techniques can also be combined with audio codecs to enable wide bandwidth audio at very low bit rates.

[0003] The basic idea behind HRF is the observation that there is usually a strong correlation between the characteristics of the high frequency range of a signal and the characteristics of the low frequency range of the same signal. Hence, a good approximation to the representation of the original input high frequency range of a signal can be achieved by transforming the signal from the low frequency range to the high frequency range.

[0004] The conversion concept was established in WO98 / 57436 as a way to regenerate high frequency bands from lower frequency bands of an audio signal. Substantial savings in bit rate can be obtained by using this concept in audio coding and / or speech coding. In the following, reference is made to audio coding, but it should be noted that the methods and systems described are equally applicable in speech coding and in unified speech and audio coding.

[0005] In an HFR-based audio coding system, a low-bandwidth signal is presented to a core waveform coder, and higher frequencies are regenerated at the decoder side using a transformation of the low-bandwidth signal and additional side information. The side information is typically encoded at a very low bit rate and describes a target spectral shape. Due to the low bit rate and narrow bandwidth of the core coding signal, it becomes increasingly important to reproduce or synthesize the high band, i.e. the high frequency range of the audio signal, with perceptually pleasing characteristics.

[0006] In the prior art there are several methods for harmonic frequency reconstruction methods, for example using harmonic transposition or time-stretching. One method is based on the phase vocoder, which operates on the principle of performing a frequency analysis with a sufficiently high frequency resolution. Before resynthesizing the signals, a signal modification is performed in the frequency domain. The signal modification may be a time-stretching or transposition operation.

[0007] One of the underlying problems that exists with these methods is the conflicting constraints of an intended high frequency resolution to obtain a high quality conversion for stationary sounds, and the time response of the system for transient or striking sounds. In other words, while a high frequency resolution is beneficial for the conversion of stationary signals, such a high frequency resolution typically requires a large window size, which becomes detrimental when dealing with the transient parts of the signal. One approach to address this issue can be to adaptively vary the window of the converter as a function of the input signal characteristics, for example by using window switching. Typically, for the stationary parts of the signal, a long window is useful to achieve a high frequency resolution. On the other hand, for the transient parts of the signal, a short window is used to implement a good transient response of the converter, i.e., a good time resolution. However, this approach has the drawback that signal analysis measures, such as transient detection, must be incorporated into the conversion system. Such signal analysis measures often include a decision step that triggers a switch of the signal processing, for example a decision about the presence of a transient signal. Moreover, such measures typically affect the reliability of the system and may introduce signal artifacts when switching the signal processing, for example when switching the window size.

[0008] The present invention solves the above-mentioned problems regarding the transient performance of harmonic conversion without the need for window switching. Moreover, improved harmonic conversion is achieved without significant added complexity. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] EP0940015B1 / WO98 / 57436 Summary of the Invention [Problem to be solved by the invention]

[0010] The present invention relates to the problem of improved transient performance for harmonic inversion and various improvements to known methods for harmonic inversion. Furthermore, the present invention explains how the added complexity can be kept to a minimum while maintaining the proposed improvements. Among other things, the invention may have at least one of the following aspects: Oversampling in frequency by a factor which is a function of the commutation factor at the commutator operating point; · Appropriate selection of decomposition and composition window combinations; and · Ensuring time alignment of different converted signals when such signals are combined. [Means for solving the problem]

[0011] According to one aspect of the invention, a system is described for generating a transposed output signal from an input signal using a transposition factor T. The transposed output signal may be a time-stretched and / or frequency-shifted version of the input signal. Relative to the input signal, the transposed output signal may be stretched in time by the transposition factor T. Alternatively, the frequency components of the transposed output signal may be shifted up by the transposition factor T.

[0012] The system may include a decomposition window of length L for extracting L samples of the input signal. Typically, the L samples of the input signal are samples of the input signal in the time domain, for example an audio signal. The extracted L samples are referred to as a frame of the input signal. The system further comprises a decomposition transform unit of order M=F×L for transforming the L time domain samples into M complex coefficients, where F is a frequency oversampling factor. The M complex coefficients are typically coefficients in the frequency domain. The decomposition transform may be a Fourier transform, a fast Fourier transform, a discrete Fourier transform, a wavelet transform or a decomposition stage of a (possibly modulated) filter bank. The oversampling factor F is based on or a function of a transformation factor T.

[0013] The oversampling operation may be referred to as zero-padding the decomposition window with additional (F-1) × L zeros. It can also be seen as choosing the size M of the decomposition transform to be a factor F times larger than the size of the decomposition window.

[0014] The system may also comprise a nonlinear processing unit for modifying the phase of the complex coefficients by using a transfer factor T. The phase modification may include multiplying the phase of the complex coefficients by the transfer factor T. Furthermore, the system may comprise a synthesis transform unit of order M for transforming the modified coefficients into M modified sample values, and a synthesis window of length L for generating the output signal. The synthesis transform may be an inverse Fourier transform, an inverse fast Fourier transform, an inverse discrete Fourier transform, an inverse wavelet transform or a synthesis stage of a (possibly) modulated filter bank. Typically, the decomposition transform and the synthesis transform are related to each other to achieve perfect reconstruction of the input signal, for example when the transfer factor T=1.

[0015] According to another aspect of the invention, the oversampling factor F is proportional to the transposition factor T. In particular, the oversampling factor F may be equal to or greater than (T+1) / 2. This selection of the oversampling factor F ensures that undesired signal artifacts that may be caused by transposition, such as pre-echoes and post-echoes, are rejected by the synthesis window.

[0016] It should be noted that in a more general form, the length of the analysis window may be La and the length of the synthesis window may be Ls. Also, in such a case, it may be beneficial to select the order M of the transform unit based on the transformation order T, i.e. as a function of the transformation order T. Furthermore, it may be beneficial to select M to be larger than the average length of the analysis and synthesis windows, i.e. larger than (La+Ls) / 2. In an embodiment, the difference between the order M of the transform unit and the average window length is proportional to (T-1). In a further embodiment, M is selected to be equal to or larger than (TLa+Ls) / 2. It should be noted that the case where the lengths of the analysis and synthesis windows are equal, i.e. La=Ls=L, is a special case of the above general case. For the general case, the oversampling factor is

number

[0017] In other words, the decomposition window may extract or isolate L or more generally La samples of the input signal, for example by multiplying a set of L samples of the input signal by a non-zero window coefficient. Such a set of L samples may be referred to as an input signal frame or a frame of the input signal. The decomposition stride unit shifts the decomposition window along the input signal, thereby selecting different frames of the input signal, i.e. generating a sequence of frames of the input signal. The sample distance between the successive frames is given by the decomposition stride. Similarly, the synthesis stride unit shifts the synthesis window and / or the frames of the output signal, i.e. generating a sequence of shifted frames of the output signal. The sample distance between the successive frames of the output signal is given by the synthesis stride. The output signal may be determined by superimposing the sequence of frames of the output signal and adding the samples that coincide in time.

[0018] According to a further aspect of the invention, the synthesis stride is T times the decomposition stride. In such a case, the output signal corresponds to the input signal time-stretched by a transposition factor T. In other words, by choosing the synthesis stride to be T times larger than the decomposition stride, a time shift or time stretch of the output signal with respect to the input signal can be obtained. This time shift is of order T.

[0019] In other words, the above-mentioned system may be described as follows: using a decomposition window unit, a decomposition transform unit and a decomposition stride unit with a decomposition stride Sa, a suite or sequence of sets of M complex coefficients may be determined from an input signal. The decomposition stride defines the number of sample values ​​by which the decomposition window is moved forward along the input signal. Since the elapsed time between two successive sample values ​​is given by the sampling rate, the decomposition stride also defines the elapsed time between two frames of the input signal. As a result, the elapsed time between two successive sets of M complex coefficients is also given by the decomposition stride Sa.

[0020] After passing through a non-linear processing unit where the phase of the complex coefficients can be changed, for example, by a factor T of the conversion factor, a suite or sequence of M sets of complex coefficients may be reconverted into the time domain. Each set of M changed complex coefficients may be converted into M changed sample values using a synthesis conversion unit. In a subsequent overlapping and adding operation involving a synthesis stride unit with a synthesis window unit and a synthesis stride Ss, a suite of sets of M changed sample values may be overlapped and added to form an output signal. In this overlapping and adding operation, successive sets of M changed sample values may be shifted by only Ss sample values relative to each other, then multiplied by the synthesis window, and then added to produce an output signal. As a result, if the synthesis stride Ss is T times the decomposition stride Sa, the signal may be time-stretched by a factor T.

[0021] According to a further aspect of the present invention, the synthesis window is derived from the decomposition window and the synthesis stride. In particular, the synthesis window may be given by the following formula.

[0022]

Equation

[0023] According to another aspect of the invention, the system further comprises a contraction unit for performing a rate conversion of the output signal, for example by a transposition order T, thereby resulting in a transposed output signal. By choosing the synthesis stride to be T times the decomposition stride, a time-stretched output signal can be obtained as outlined above. If the sampling rate of the time-stretched signal is increased by a factor T, or if the time-stretched signal is downsampled by a factor T, a transposed output signal can be generated that corresponds to a frequency-shifted version of the input signal by the transposition factor T. The downsampling operation may comprise the step of selecting only a subset of the sample values ​​of the output signal. Typically, only every Tth sample value of the output signal is kept. Alternatively, the sampling rate may be increased by a factor T, i.e. the sampling rate is interpreted as T times higher. In other words, resampling or sampling rate conversion means that the sampling rate is changed to a higher or lower value. Downsampling means rate conversion to a lower value.

[0024] According to a further aspect of the invention, the system may generate a second output signal from the input signal. The system may include a second nonlinear processing unit for modifying the phase of the complex coefficients by using a second transposition factor T2, and a second synthesis stride unit for shifting a synthesis window and / or a frame of the second output signal by a second synthesis stride. The modification of the phase may include multiplying the phase by a factor T2. The frames of the second output signal may be generated from the frames of the input signal by modifying the phase of the complex coefficients by using the second transposition factor, converting the second modified coefficients into M second modified sample values, and applying the synthesis window. The second output signal may be generated in the overlap-and-add unit by applying the second synthesis stride to a sequence of frames of the second output signal.

[0025] The second output signal may be shrunk in a second shrinking unit, which performs a rate conversion of the second output signal, for example by a second transformation order T2. This results in a second transformed output signal. In summary, the first transformed output signal may be generated using a first transformation factor T, and the second transformed output signal may be generated using a second transformation factor T2. These two transformed output signals may then be merged in a combination unit to produce an overall transformed output signal. The merging operation may include adding the two transformed output signals. Such generation and combination of multiple transformed output signals may be beneficial to obtain a good approximation of the high-frequency signal components to be synthesized. It should be noted that any number of transformed output signals may be generated using multiple transformation orders. The multiple transformed output signals may then be merged, for example added, in the combination unit to produce an overall transformed output signal.

[0026] It may be beneficial for the combining unit to weight the first and second transformed output signals prior to merging. The weighting may be performed such that the energy or energy per bandwidth of the first and second transformed output signals corresponds to the energy or energy per bandwidth of the input signals, respectively.

[0027] According to a further aspect of the invention, the system may comprise an alignment unit for applying a time offset to the first and second transformed output signals before entering the combination unit. Such a time offset may comprise a shift in the time domain of the two transformed output signals relative to each other. The time offset may be a function of the transformation order and / or the window length. In particular, the time offset may be (T-2)L / 4 may be determined as:

[0028] According to another aspect of the invention, the above conversion system may be incorporated into a system for decoding a received multimedia signal including an audio signal. The decoding system may have a conversion unit corresponding to the system outlined above. Here, the input signal is typically a low frequency component of the audio signal and the output signal is a high frequency component of the audio signal. In other words, the input signal is typically a low pass signal with a certain bandwidth and the output signal is typically a band pass signal with a higher bandwidth. Furthermore, the decoding system may have a core decoder for decoding the low frequency component of the audio signal from the received bitstream. Such a core decoder may be based on a coding scheme such as Dolby E, Dolby Digital or AAC. In particular, such a decoding system may be a set-top box for decoding a received multimedia signal including an audio signal and other signals such as video.

[0029] It should be noted that the present invention also describes a method for transforming an input signal by a transformation factor T. The method corresponds to the system outlined above and may include any combination of the above-mentioned aspects. It may include the steps of extracting samples of the input signal using a decomposition window of length L and selecting an oversampling factor F as a function of the transformation factor T. It may further include the steps of transforming the L samples from the time domain to the frequency domain resulting in F×L complex coefficients and modifying the phase of the complex coefficients using the transformation factor T. In a further step, the method may transform the F×L modified complex coefficients to the time domain resulting in F×L modified samples and may generate an output signal using a synthesis window of length L. It should be noted that the method may also be adapted to general lengths of the decomposition and synthesis windows, i.e. to general La and Ls as outlined above.

[0030] According to a further aspect of the invention, the method may comprise shifting the decomposition window by a decomposition stride of Sa samples along the input signal and / or shifting the synthesis window and / or the frame of the output signal by a synthesis stride of Ss samples. By selecting the synthesis stride to be T times the decomposition stride, the output signal may be time-stretched by a factor T with respect to the input signal. When performing an additional step of performing a rate conversion of the output signal by a transposition order T, a transposed output signal may be obtained. Such a transposed output signal may comprise frequency components that are shifted up by a factor T with respect to corresponding frequency components of the input signal.

[0031] The method may further comprise steps for generating a second output signal. This may be implemented by modifying the phase of the complex coefficients by using a second transposition factor T2. The second output signal may be generated using the second transposition factor T2 and the second synthesis stride by shifting the synthesis window and / or the frame of the second output signal by the second synthesis stride. The second transposed output signal may be generated by performing a rate conversion of the second output signal by the second transposition order T2. Finally, a merged or overall transposed output signal including high frequency signal components generated by two or more transpositions with different transposition factors can be obtained by merging the first and second transposed output signals.

[0032] According to another aspect of the invention, the invention describes a software program adapted for execution on a processor and for performing the method steps of the invention when executed on a computing device. The invention also describes a storage medium having a software program adapted for execution on a processor and for performing the method steps of the invention when executed on a computing device. Furthermore, the invention describes a computer program product comprising executable instructions for performing the method of the invention when executed on a computer.

[0033] According to a further aspect, another method and system for transforming an input signal by a transformation factor T is described. The method and system may be used stand-alone or in combination with the methods and systems outlined above. Any of the features outlined herein may be applied to this method / system and vice versa.

[0034] The method may include extracting a frame of sample values ​​of the input signal using a decomposition window of length L. The frame of the input signal may then be transformed from the time domain to the frequency domain to produce M complex coefficients. The phase of the complex coefficients may be modified using a transposition factor T, and the M modified complex coefficients may be transformed to the time domain to produce M modified sample values. Finally, the frame of the output signal may be generated using a synthesis window of length L. The method and system may use decomposition and synthesis windows that differ from each other. The decomposition and synthesis windows may differ in terms of their shape, length, number of coefficients that define the window, and / or values ​​of the coefficients that define the window. By doing this, additional degrees of freedom in the selection of the decomposition and synthesis windows may be obtained and aliasing of the transformed output signal may be reduced or eliminated.

[0035] According to another aspect, the decomposition window and the synthesis window are bi-orthogonal with respect to each other. s (n) may be given by the following formula:

[0036]

number

[0037]

number

[0038] According to a further aspect, the decomposition window may be selected such that its z-transform has dual zeros on the unit circle. Preferably, the z-transform of the decomposition window only has dual zeros on the unit circle. For example, the decomposition window may be a squared sine window. In another example, a decomposition window of length L may be determined by convolving two sine windows of length L to yield a squared sine window of length 2L-1. In a further step, zeros may be appended to the squared sine window to yield a base window of length 2L. Finally, the base window may be resampled using linear interpolation, thereby yielding an even symmetric window of length L as the decomposition window.

[0039] The methods and systems described herein may be implemented as software, firmware and / or hardware. Certain components may be implemented as software running on, for example, a digital signal processor or a microprocessor. Other components may be implemented as, for example, hardware and / or application specific integrated circuits. The signals encountered in the methods and systems described herein may be stored on media such as random access memory or optical storage media. The signals may be transmitted over a network such as an airwave network, a satellite network, a wireless network or a wired network, for example the Internet. A typical device using the methods and systems described herein is a set-top box or other customer premises equipment that decodes audio signals. On the encoding side, the methods and systems may be used in broadcast stations, for example in video or television head-end systems.

[0040] It should be noted that the embodiments and aspects of the present invention described herein may be combined in any manner. In particular, it should be noted that the aspects outlined for the system are also applicable to the corresponding methods encompassed by the present invention. Furthermore, it should be noted that the disclosure of the present invention also covers claim combinations other than those explicitly given by reference in the dependent claims. That is, the claims and their technical features may be combined in any order and in any manner.

[0041] The present invention will now be described, by way of illustrative examples and not intended to limit the scope or spirit of the invention, with reference to the accompanying drawings. [Brief description of the drawings]

[0042] [Figure 1] FIG. 2 shows a Dirac at a particular position appearing in the decomposition and synthesis windows of a harmonic converter. [Diagram 2] FIG. 1 shows a diagram illustrating Diracs at different positions appearing in the decomposition and synthesis windows of a harmonic converter. [Diagram 3] FIG. 3 shows a Dirac diagram for the position of FIG. 2 that appears in accordance with the present invention. [Figure 4] FIG. 13 illustrates the operation of an HFR enhanced audio decoder. [Diagram 5] FIG. 1 illustrates the operation of a harmonic converter using several orders. [Figure 6] FIG. 1 is a diagram illustrating the operation of a frequency domain (FD) harmonic converter. [Figure 7] FIG. 2 illustrates a series of decomposition synthesis windows. [Figure 8] FIG. 1 illustrates decomposition and synthesis windows at different strides. [Figure 9] FIG. 13 illustrates the effect of resampling on the synthesis stride of a window. [Figure 10]FIG. 2 illustrates an embodiment of an encoder that uses the enhanced harmonic conversion scheme outlined in this document. [Figure 11] FIG. 2 shows an embodiment of a decoder that uses the improved harmonic transformation scheme outlined in this document. [Figure 12] FIG. 12 shows an embodiment of the conversion unit shown in FIGS. 10 and 11. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] The embodiments described below are merely illustrative of the principles of the present invention for improved harmonic conversion. It is understood that modifications and variations to the configurations and details described herein will be apparent to others skilled in the art. Therefore, it is intended that the present invention be limited only by the scope of the appended claims and not by the specific details presented by the description and explanation of the embodiments described herein.

[0044] In the following, the principles of harmonic transposition in the frequency domain and the proposed improvements taught by the present invention are outlined. The key element of harmonic transposition is the time stretching by an integer transposition factor T, which preserves the frequency of the sinusoids. In other words, harmonic transposition is based on time-stretching the underlying signal by a factor T. The time stretching is performed in such a way that the frequencies of the sinusoids that make up the input signal are preserved. Such time stretching can be performed using a phase vocoder, which is a vocoder that is implemented by dividing the input signal by a decomposition window v a (n) and composite window v s Such a decomposition / synthesis transform is also called the short-time Fourier transform (STFT).

[0045] A short-time Fourier transform is performed on the time-domain input signal to obtain a series of overlapped spectral frames. An appropriate decomposition / synthesis window, e.g., Gaussian, Cosine, Hamming, Hann, Rectangular, Bartlett, Blackman, etc., should be selected to minimize possible side-band effects. The time delay at which each spectral frame is picked up from the input signal is called the hop size or stride. The STFT of the input signal is called the decomposition stage, which leads to a frequency-domain representation of the input signal. The frequency-domain representation contains multiple subband signals, where each subband signal represents a frequency component of the input signal.

[0046] The frequency domain representation of the input signal may then be processed in the desired manner. For the purpose of time stretching the input signal, each subband signal may be time stretched, for example by delaying the subband signal samples. This may be achieved by using a synthesis hop size larger than the decomposition hop size. The time domain signal may be reconstructed by performing an inverse (fast) Fourier transform on all frames and then accumulating the frames successively. This operation of the synthesis stage is called an overlap-add operation. The resulting output signal is a time-stretched version of the input signal, containing the same frequency components as the input signal. In other words, the resulting output signal has the same spectral composition as the input signal, but is slower than the input signal, i.e. its progression is stretched in time.

[0047] A transposition to a higher frequency can then be obtained in a subsequent step or in an integrated manner through downsampling of the stretched signal. As a result, the transposed signal has the time length of the initial signal but with its frequency components shifted upwards by a predefined transposition factor.

[0048] Mathematically, a phase vocoder can be written as follows: An input signal x(t) is sampled at a sampling rate R to produce a discrete input signal x(n). During the decomposition stage, a particular decomposition time t is generated for a set of values ​​k.a k The STFT is determined for the input signal x(n) at a k =kΔt a where Δt a is the decomposition Hopf factor or decomposition stride. a k In each of the decomposition windows v a (t) is t a k The focus is on v a (tt a k ) This windowed portion of the input signal x(n) is called a frame. The result is an STFT representation of the input signal x(n), which can be expressed as

[0049]

number

[0050] The synthesis stage is typically s k =kΔt s The synthetic time t is uniformly distributed according to s k where Δt sis the synthesis Hopf factor or synthesis stride. At each of these synthesis times, the short-time signal y k (n) is the synthesis time t s k In this case, X(t a k ,Ω m ) s k ,Ω m However, typically the STFT subband signals are modified, for example time stretched and / or phase and / or amplitude modulated, so that the decomposition subband signals X(t a k ,Ω m ) is the composite subband signal Y(t s k ,Ω m ) In a preferred embodiment, the STFT subband signals are phase modulated, i.e. the phase of the STFT subband signals is modified. k (n) It can be expressed as follows:

[0051]

number

[0052]

number

[0053] In the following, we outline the implementation of time stretching in the frequency domain. A good starting point for describing aspects of a time stretcher is to consider the case T=1, i.e. the transfer factor T is equal to 1 and no stretching is performed. The decomposition time stride Δt of the DFT filter bank is a and the composite time stride Δt s is equal, i.e., Δt a = Δt s = Δt, the combined effect of decomposition and subsequent synthesis is a function of period Δt.

number

number

[0054] For T>1, i.e., for a transition factor greater than 1, the time stretching reduces the composite stride by Δt s = Δt, while stride Δt a= Δt / T. In other words, time stretching by a factor T can be obtained by applying a Hopf factor or stride in the decomposition window that is T times smaller than the Hopf factor or stride in the synthesis stage. As can be seen from the formula given above, using a synthesis stride T times larger than the decomposition stride results in a large time-domain synthesis signal y k (n) by an interval T times larger in the overlap-and-add operation, which ultimately leads to a time expansion of the output signal y(n).

[0055] It should be noted that time expansion by a factor T may further involve a phase multiplication by a factor T between the decomposition and the synthesis. In other words, time expansion by a factor T includes a phase multiplication by a factor T of the subband signals.

[0056] In the following, we outline how the above time stretching operation can be translated into a harmonic transposition operation. Pitch-scale modification or harmonic transposition can be obtained by performing a sample rate conversion of the time-stretched output signal y(n). To perform a harmonic transposition by a factor T, an output signal y(n) that is a time-stretched version of the input signal x(n) by a factor T may be obtained using the phase vocoding method described above. The harmonic transposition may then be obtained by downsampling the output signal y(n) by a factor T or by converting the sampling rate from R to TR. In other words, instead of interpreting the output signal y(n) as having the same sampling rate as the input signal x(n) but with a duration T times longer, the output signal y(n) may be interpreted as having the same duration but with a sampling rate T times longer. The subsequent downsampling by T may then be interpreted as making the output sampling rate equal to the input sampling rate so that the signals can finally be added together.

[0057] Assume that the input signal x(n) is sinusoidal and the symmetric decomposition window va Assuming that x(n) is the input signal, the phase vocoder-based time stretching method described above works perfectly for odd T, yielding a time-stretched version of the input signal x(n) with the same frequency. Combined with subsequent downsampling, we obtain a sine wave y(n) with a frequency T times the frequency of the input signal x(n).

[0058] For even T, the time stretching / harmonic transformation method outlined above becomes a closer approximation. a This is because the negative side lobes of the frequency response of y(n) are reproduced with different fidelity by the phase multiplication. The negative side lobes typically result from the fact that most practical windows (or prototype filters) have a number of discrete zeros located on the unit circle, resulting in a phase shift of 180 degrees. When multiplying the phase angle with an even transposition factor, the phase shift is typically transformed to 0 (or rather a multiple of 360) degrees, depending on the transposition factor used. In other words, when using even transposition factors, the phase shift disappears. This typically leads to aliasing in the transposed output signal y(n). A particularly unfavourable scenario can arise when a sine wave is located at a frequency corresponding to the top of the first side lobe of the analysis filter. Depending on the rejection of this lobe in the magnitude response, the aliasing will be more or less audible in the output signal. For even factors T, reducing the overall stride Δt typically improves the performance of the time stretcher, at the expense of a higher computational complexity.

[0059] US Pat. No. 6,399,433, entitled "Source Coding Enhancement Using Spectral Band Replication," which is incorporated herein by reference, describes a method on how to avoid aliasing resulting from harmonic transposition when using even transposition factors. This method, called relative phase locking, evaluates the relative phase difference between adjacent channels to determine whether a sine wave is phase-inverted in any channel. Detection is performed by using equation (32) in US Pat. No. 6,399,433. Channels detected as phase-inverted are corrected after their phase angles are multiplied by the actual transposition factor.

[0060] In the following, a new method is described to avoid aliasing when using even and / or odd transformation factors T. Contrary to the relative phase locking methods in the patent literature, the method does not require phase angle detection and correction. A new solution to the above problem makes use of non-identical decomposition and synthesis transform windows. In the case of perfect reconstruction (PR), this corresponds to a biorthogonal transform / filter bank instead of an orthogonal one.

[0061] A certain decomposition window v a To obtain a biorthogonal transform for a given (n), we use a synthesis window v s (n) is

number

number

[0062] However, in the following, another sequence w(n) is introduced. w(n) is the sequence of the synthesis window v s (n) is the decomposition window v a is a measure of how far the sequence w(n) deviates from (n), i.e. how different the biorthogonal transform is from the orthogonal transform. w(n)=v s (n) / v a (n) 0≦n <L is given by:

[0063] Then, the condition for perfect reconstruction is

number

[0064]

number

[0065]

number

[0066] In order to obtain a decomposition / synthesis window pair that suppresses aliasing for even transfer factors, several embodiments are outlined below. According to a first embodiment, the window or prototype filter is made long enough to attenuate the level of the first side lobes in the frequency response below a certain "aliasing" level. The decomposition window stride Δt a is in this case only a (small) fraction of the window length L. This typically leads to a smearing of transient components, e.g. in impact signals.

[0067] According to the second embodiment, the decomposition window v a (n) is chosen to have dual zeros on the unit circle. The phase response resulting from the dual zeros is a 360 degree phase shift. These phase shifts are preserved when the phase angles are multiplied by transfer factors, whether the transfer factor is odd or even. A proper smooth decomposition filter v with dual zeros on the unit circle is a When (n) is obtained, the synthesis window is obtained from the equations outlined above.

[0068] In the second embodiment, the decomposition filter / window v a (n) is the "squared sine window", i.e., the sine window v(n) = sin{(π / L)(n+0.5)} 0≦n <L of

number

[0069] Overall, we have outlined how pairs of analysis and synthesis windows can be selected such that aliasing in the transformed output signal can be avoided or significantly reduced. This method is particularly important when using even transformation factors.

[0070] Another aspect to consider in the context of vocoder-based harmonic commutators is phase unwrapping. Although careful attention must be paid to the phase unwrapping issue in general-phase vocoders, it should be noted that harmonic commutators have an unambiguously defined phase behavior when an integer transposition factor T is used. Thus, in preferred embodiments, the transposition order T is integer-valued. Otherwise, phase unwrapping techniques can be applied, where phase unwrapping is the process of estimating the instantaneous frequency of nearby sinusoids in each channel using the phase increment between two successive frames.

[0071] Yet another aspect to consider when dealing with the transformation of acoustic and / or voice signals is the processing of stationary and / or transient signal sections. Typically, to be able to transform stationary acoustic signals without intermodulation artifacts, the frequency resolution of the DFT filter bank needs to be high, and therefore the windows are long compared to the input signal x(n), especially the transient components in the acoustic and / or voice signal. As a consequence, the transformer has a poor transient response. However, as will be discussed below, this problem can be solved by modifying the window design, the transform size and the time stride parameters. Thus, unlike many state-of-the-art methods for phase vocoder transient response improvement, the proposed solution does not rely on any signal adaptive operations such as transient component detection.

[0072] In the following, the harmonic conversion of a transient signal using a vocoder is outlined. As a starting point, we consider a prototype transient signal, a discrete-time Dirac pulse at time t=t0.

number

[0073]

number

[0074] This shows that the operation of phase multiplication of the decomposition subband signals by a factor T leads to the desired time shift of the Dirac pulse, i.e. of the transient input signal. It should be noted that for more realistic transient signals with two or more non-zero sample values, a further operation of time expansion of the decomposition subband signals by a factor T should be performed. In other words, different hop sizes should be used on the decomposition and synthesis sides.

[0075] However, it should be noted that the above considerations are for decomposition / synthesis stages with infinitely long decomposition and synthesis windows. In fact, a theoretical commutator with a window of infinite duration gives the correct stretching of the Dirac pulse δ(t-t0). For windowed decompositions of finite duration, the situation is complicated by the fact that each decomposition block should be interpreted as one periodic interval of a periodic signal with period equal to the size of the DFT.

[0076] This is illustrated in Figure 1. Figure 1 shows the decomposition and synthesis 100 of a Dirac pulse δ(t-t0). The top of Figure 1 shows the input to the decomposition stage 110, and the bottom of Figure 1 shows the output of the synthesis stage 120. The top and bottom graphs represent the time domain. The stylized decomposition window 111 and the synthesis window 121 are depicted as triangular (Bartlett) windows. The input pulse δ(t-t0) 112 at time t=t0 is depicted as a vertical arrow in the top graph 110. The DFT transform block is assumed to be of size M=L, i.e. the size of the DFT transform is chosen to be equal to the size of the window. Phase multiplication of the subband signals by a factor T results in a DFT decomposition of the Dirac pulse δ(t-Tt0) at t=Tt0, but partitioned into a train of Dirac pulses with period L. This is due to the finite length of the applied window and the Fourier transform. A segmented pulse train with period L is depicted by the dashed arrows 123, 124 in the graph below.

[0077] In a real-world system where the decomposition window and the synthesis window are of finite length, the pulse train actually contains only a few pulses (depending on the transformation factor). There is one main pulse, i.e., the desired term, and a few pre-pulses and a few post-pulses, i.e., the undesired terms. The pre-pulses and post-pulses occur because the DFT is periodic (with period L). When a pulse is located within the decomposition window and is folded (wrapped) when its complex phase is multiplied by T (i.e., the pulse is shifted beyond the end of the window and returns to the beginning), the undesired pulses appear. The undesired pulses may or may not have the same polarity as the input pulse, depending on their position in the decomposition window and the transformation factor.

[0078] This can be seen mathematically when using a DFT of length L centered at t = 0 to transform a Dirac pulse δ(t - t0) located in the interval -L / 2 ≤ t0 < L / 2.

[0079]

Number

Number

[0080] In the example of Figure 1, the synthesis window multiplication uses the finite window v s (n) 121. The finite synthesis window 121 picks up the desired pulse δ(t - Tt0) at t = Tt0 depicted as the solid arrow 122 and cancels the other contributions shown as the dashed arrows 123, 124.

[0081] As the decomposition and synthesis stages move along the time axis according to the Hopf factor or time stride Δt, the pulses δ(t−t0) 112 will have different positions relative to the center of their respective decomposition windows 111. As outlined above, the operation to achieve time stretching consists in moving the pulses 112 by T times their position relative to the center of the window. As long as this position is within the window 121, this time stretching operation guarantees that all contributions add up to a single time-stretched composite pulse δ(t−Tt0) at t=Tt0.

[0082] However, a problem arises for the situation in Fig. 2, where the pulse δ(t-t0) 212 has now moved further out towards the edge of the DFT block. Fig. 2 shows a decomposition / synthesis configuration 200 similar to that in Fig. 1. The top graph 210 shows the input to the decomposition stage and the decomposition window 211, while the bottom graph 220 shows the output of the synthesis stage and the synthesis window 221. When we time-stretch the input Dirac pulse 212 by a factor T, the time-stretched Dirac pulse 222, i.e. δ(t-Tt0), is outside the synthesis window 221. At the same time, another Dirac pulse 224 in the pulse train, i.e. δ(t-Tt0+L) at time t=Tt0-L, is picked up by the synthesis window. In other words, the input Dirac pulse 212 is not delayed to a time T times later, but rather brought forward to a time that is before the input Dirac pulse 212. The net effect on the audio signal is the occurrence of a pre-echo at a time distance on the scale of the longer commutator window, i.e. at a time t=Tt0-L that is L-(T-1)t0 earlier than the input Dirac pulse 212.

[0083] The principle of the solution proposed by the invention is described with reference to Fig. 3. Fig. 3 shows a decomposition / synthesis scenario 300 similar to Fig. 2. The upper graph 310 shows the input to the decomposition stage with the decomposition window 311, and the lower graph 320 shows the output of the synthesis stage with the synthesis window 321. The basic idea of ​​the invention is to adapt the DFT size to avoid pre-echoes. This can be achieved by setting the DFT size M such that the synthesis window does not pick up unwanted Dirac pulse images from the resulting pulse train. The size of the DFT transform 301 is increased to M=FL, where L is the length of the window function 302 and the factor F is the oversampling factor in the frequency domain. In other words, the size of the DFT transform 301 is chosen to be larger than the window size 302. In particular, the size of the DFT transform 301 may be chosen to be larger than the window size 302 of the synthesis window. Due to the increased length 301 of the DFT transform, the period of the pulse train including the Dirac pulses 322, 324 is FL. By choosing a large enough value of F, i.e. by choosing a large enough frequency domain oversampling factor, the undesired contribution to the pulse stretching can be eliminated. This is illustrated in Fig. 3. The Dirac pulse 324 at time t = Tt0-FL is outside the synthesis window 321. Therefore, the Dirac pulse 324 is not picked up by the synthesis window 321 and, as a result, pre-echoes are avoided.

[0084] It should be noted that in a preferred embodiment, the synthesis and decomposition windows have equal "normal" lengths. However, when using implicit resampling of the output signal by dropping or inserting samples in frequency bands of a transform or filter bank, the synthesis window size will typically differ from the decomposition size, depending on the resampling or transformation factor.

[0085] The minimum value of F, i.e., the minimum frequency-domain oversampling factor, can be deduced from Fig. 3. The condition for not picking up the undesired Dirac pulse image can be formulated as follows: for any input pulse δ(t - t0) at position t = t0 < L / 2, i.e., for any input pulse contained within the decomposition window 311, the undesired image δ(t - Tt0 + FL) at time t = Tt0 - FL must be located to the left of the left end of the synthesis window at t = -L / 2. Equivalently, the condition T(L / 2) - FL ≤ -L / 2 must be satisfied. This leads to the rule F ≥ (T + 1) / 2 (3) as follows.

[0086] As can be seen from formula (3), the minimum frequency-domain oversampling factor F is a function of the transformation / time stretching factor T. More specifically, the minimum frequency-domain oversampling factor F is proportional to the transformation / time stretching factor T.

[0087] By repeating the above line of thought for the case where the decomposition and synthesis windows have different lengths, a more general formula can be obtained. Let L A and L S be the lengths of the decomposition window and the synthesis window respectively, and let M be the DFT size used. Then, the rule for extending formula (3) is M ≥ (TL A + L S ) / 2 (4) as follows.

[0088] That this rule is actually an extension of (3) can be verified by substituting M = FL and L A = L S - L into (4) and dividing both sides of the resulting equation by L.

[0089] The above analysis has been carried out for a rather special model of transient signals, i.e. Dirac pulses. However, the idea can be extended to show that when using the above time stretching scheme, an input signal with an almost flat spectral envelope and zero outside the time interval [a,b] is stretched to a small output signal outside the interval [Ta,Tb]. The disappearance of pre-echo in stretched signals when respecting the above rules for selecting an appropriate frequency domain oversampling factor can also be checked by examining spectrograms of real acoustic and / or speech signals. A more quantitative analysis reveals that the pre-echo is reduced even when using frequency domain oversampling factors that are somewhat inferior to the value imposed by the condition of formula (3). This is due to the fact that for a typical window function v s This is due to the fact that (n) is smaller near the edges, thereby attenuating undesired pre-echoes that are located near the edges of the window function.

[0090] In summary, the present invention teaches a new way to improve the transient response of a frequency response harmonic converter or time stretcher by introducing an oversampled transform where the amount of oversampling is a function of the chosen transform factor.

[0091] In the following, the application of the inventive harmonic transformation in audio decoders is described in more detail. A typical use case for a harmonic transformer is in audio / speech codec systems that use so-called bandwidth extension or high frequency regeneration (HFR). It should be noted that although reference is made to audio coding, the described method and system are equally applicable in speech coding and in unified speech and audio coding.

[0092] In such HFR systems, a commutator may be used to generate high-frequency signal components from low-frequency signal components provided by a so-called core decoder. The envelope of the high-frequency components may be shaped in time and frequency based on the side information conveyed in the bitstream.

[0093] FIG. 4 illustrates the operation of an HFR enhanced audio decoder. The core audio decoder 401 outputs a low bandwidth audio signal, which is input to an upsampler 404. The upsampler 404 may be required to generate the final audio output contribution at the desired full sampling rate. Such upsampling is required for dual rate systems, where the bandwidth limited core audio codec operates at half the external audio sampling rate, while the HFR portion is processed at the full sampling frequency. As a result, for single rate systems, this upsampler 404 is omitted. The low bandwidth output of 401 is also sent to a transformer or transformation unit 402, which outputs a transformed signal, i.e. a signal containing the desired high frequency range. This transformed signal may be shaped in time and frequency by an envelope adjuster 403. The final audio output is the sum of the low bandwidth core signal and the envelope adjusted transformed signal.

[0094] As outlined in the context of FIG. 4, the core decoder output signal may be upsampled by a factor of 2 as a pre-processing step in the transposition unit 402. Transposition by a factor T results in a signal that, in the case of time expansion, is T times as long as the untransposed signal. Downsampling or rate conversion of the time-stretched signal is then performed to achieve the desired pitch-shifting or frequency transposition to a T times higher frequency. As mentioned above, this operation may be achieved through the use of different analysis and synthesis strides in the phase vocoder.

[0095] The global transposition order can be obtained in different ways. A first possibility is to upsample the decoder output signal by a factor 2 at the transposition input, as pointed out above. In such a case, the time-stretched signal needs to be downsampled by a factor T to obtain the desired output signal frequency-transposed by a factor T. A second possibility is to omit said pre-processing step and to perform a time-stretching operation directly on the output signal of the core decoder. In such a case, in order to keep the global upsampling factor 2 and to achieve a frequency transposition by a factor T, the transposed signal has to be downsampled by a factor T / 2. In other words, when performing a downsampling of the output signal of the transpositioner 402 by T / 2 instead of T, the upsampling of the core decoder signal may be omitted. However, it should be noted that the core signal still needs to be upsampled before combining it with the transposed signal in the upsampler 404.

[0096] It should also be noted that the commutator 402 may use several different integer commutation factors to generate the high frequency components. This is illustrated in Fig. 5. Fig. 5 shows the operation of a harmonic commutator 501, which corresponds to the commutator 402 of Fig. 4, having several commutators with different commutation orders or commutation factors T. If the signal to be commutated is a harmonic commutator with several commutators with different commutation orders T=2, 3, ..., T max Each of the converters 501-2, 501-3, ..., 501-T has max Typically, the bank of conversion orders T max = 3 is sufficient for most audio coding applications. maxThe contributions are summed in 502 to provide a combined commutator output. In a first embodiment, this summing operation may involve adding together the individual contributions. In another embodiment, the contributions are weighted using different weights so that the effect of adding multiple contributions at certain frequencies is mitigated. For example, a third order contribution may be added with a lower gain than a second order contribution. Finally, the summing unit 502 may selectively add these contributions depending on the output frequency. For example, a second order commutation may be used for a first lower target frequency unit and a third order commutation may be used for a second higher target frequency unit.

[0097] FIG. 6 illustrates the operation of one of the individual blocks of 501, i.e. one of the harmonic converters 501-T of conversion order T. The decomposition stride unit 601 selects a sequence of frames of the input signal to be converted. These frames are overlapped, e.g. multiplied, with a decomposition window in the decomposition window unit 602. It is noted that the operations of selecting the frames of the input signal and multiplying the samples of the input signal with the decomposition window function may be performed in unique steps, e.g. by using a window function that is shifted along the input signal by the decomposition stride. In the decomposition transform unit 603, the windowed frames of the input signal are transformed into the frequency domain. The decomposition transform unit 603 may for example perform a DFT. The size of the DFT is selected to be F times larger than the size L of the decomposition window, thereby generating M=F×L complex frequency domain coefficients. These complex coefficients are modified in the nonlinear processing unit 604, e.g. by multiplying their phase by a conversion factor T. The complex coefficients of the sequence of complex frequency domain signals, i.e. the sequence of frames of the input signal, may be viewed as subband signals. The combination of the decomposition stride unit 601, the decomposition window unit 602 and the decomposition transform unit 603 may be viewed as a combined decomposition stage or a decomposition filter bank.

[0098] The modified coefficients or modified subband signals are transformed back to the time domain using a synthesis transform unit 605. For each set of transformed complex coefficients, this gives a frame of modified samples, i.e. a set of M modified samples. Using a synthesis window unit 606, L samples may be extracted from each set of modified samples, thereby giving a frame of an output signal. Overall, a sequence of frames of the output signal may be generated for a sequence of frames of the input signal. The frames of this sequence are shifted relative to each other by a synthesis stride in a synthesis stride unit 607. The synthesis stride may be T times larger than the decomposition stride. The output signal is generated in an overlap-add unit 608, where the shifted frames of the output signal are overlapped and samples at the same time are added together. By passing through the above system, the input signal may be time-stretched by a factor T. That is, the output signal may be a time-stretched version of the input signal.

[0099] Finally, the output signal may be contracted in time using the contraction unit 609. The contraction unit 609 may perform a sampling rate conversion of order T, i.e., increase the sampling rate of the output signal by a factor T while keeping the number of sample values ​​unchanged. This gives a transformed output signal with the same time length as the input signal but with frequency components shifted up by a factor T with respect to the input signal. The combination unit 609 may also perform a downsampling operation by a factor T, i.e., keep only every Tth sample value and discard the other samples. This downsampling operation may be achieved by a low-pass filter operation. If the overall sampling rate remains unchanged, the transformed output signal has frequency components shifted up by a factor T with respect to the frequency components of the input signal.

[0100] It should be noted that the contraction unit 609 may perform a combination of rate conversion and downsampling. As an example, the sampling rate may be increased by a factor of 2. At the same time, the signal may be downsampled by a factor T / 2. Overall, such a combination of rate conversion and downsampling also leads to an output signal that is a harmonic transformation of the input signal by a factor T. In general, it may be stated that the contraction unit 609 performs a combination of rate conversion and / or downsampling to give a harmonic transformation with a transformation order T. This is particularly useful when performing a harmonic transformation of a low bandwidth output of the core audio decoder 401. As outlined above, such a low bandwidth output may have been downsampled by a factor of 2 in the encoder and may therefore require upsampling in the upsampling unit 404 before merging with the reconstructed high frequency components. Nevertheless, it may be useful to perform a harmonic transformation in the transformation unit 402 using the "not upsampled" low bandwidth output to reduce the amount of computation. In such a case, the contraction unit 609 of the transformation unit 402 may perform a rate conversion of order 2, thereby implicitly performing the required upsampling operation of the high frequency components. As a result, the transformed output signal of order T is downsampled in the contraction unit 609 by a factor T / 2.

[0101] In the case of multiple parallel transformers with different transform orders as shown in FIG. 5, some transform or filter bank operations are performed by different transformers 501-2, 501-3, ..., 501-T. max , may be shared between the DFT and synthesis filter banks. The sharing of filter bank operations may preferably be done on the decomposition to obtain a more efficient implementation of the transformation unit 402. A preferred way to resample the outputs from the different transformers may be to discard DFT bins or subband channels prior to the synthesis stage. In this way, the resampling filters may be omitted and the computational effort may be reduced when performing an inverse DFT / synthesis filter bank of smaller size.

[0102] As just mentioned, the decomposition window may be common to signals of different transfer factors. When using a common decomposition window, an example of the stride of the window 700 applied to a low-band signal is illustrated in Figure 7. Figure 7 shows the difference in the decomposition hop factor or decomposition time stride Δt a 7 shows the strides of decomposition windows 701, 702, 703 and 704 which are displaced relative to each other by

[0103] An example of the window stride applied to a low-band signal, e.g. the output signal of the core decoder, is illustrated in Fig. 8(a). The stride by which the decomposition window of length L is moved for each decomposition transform is Δt a Each such decomposition transform and windowed portion of the input signal is also called a frame. The decomposition transform converts a frame of input samples into a set of complex FFT coefficients. After the decomposition transform, the complex FFT coefficients may be transformed from Cartesian to polar coordinates. The suite of FFT coefficients for the subsequent frame constitutes the decomposition subband signal. The transformation factors used are T=2,3,…,T max For each of the FFT coefficients, the phase angle is multiplied by the respective transformation factor T and transformed back to Cartesian coordinates.

[0104] Thus, for each transfer factor T, there is a different set of complex FFT coefficients that represent a particular frame. In other words, for transfer factors T=2, 3, …, T max For each of the subbands Y(t s k ,Ω m ) are generated.

[0105] In the synthesis stage, the synthesis stride of the synthesis window, Δt sis determined as a function of the transformation order T used in each transformer. As outlined above, the time expansion operation also involves a time stretching of the subband signals, i.e. a time stretching of the suite of frames. This operation is performed by dividing the decomposition stride Δt by a factor T. a The synthetic hop factor or synthetic stride Δt is increased s As a result, the composite stride Δt for a commutator of order T sT is Δt sT =TΔt a (b) and (c) in Fig. 8 show the synthesis stride Δt sT Here, Δt s2 =2Δt a , Δt s3 =3Δt a It is.

[0106] FIG. 8 also shows a reference time t that has been “stretched” by factors T=2 and T=3 in FIGS. 8(b) and 8(c), respectively, relative to FIG. 8(a). r However, in the output, this reference time t r need to be aligned for two transposition factors. To align the outputs, the third order transposed signal, i.e. Fig. 8c, needs to be downsampled or rate converted by a factor 3 / 2. This downsampling leads to a harmonic transposition for the second order transposed signal. Fig. 9 shows the effect of such resampling on the synthesis stride of the window for T=3. If the decomposed signal is the output signal of the core decoder that is not upsampled, then the signal in Fig. 8b is effectively frequency transposed by a factor of 2 and the signal in Fig. 8c is effectively frequency transposed by a factor of 3.

[0107] In the following, we address the aspect of time alignment of transposition sequences of different transposition factors when using a common decomposition window, in other words, the aspect of aligning the output signals of frequency transpositioners using different transposition orders. When using the method outlined above, the Dirac function δ(t-t0) is time stretched, i.e. moved along the time axis, by the amount of time given by the applied transposition factor T. To convert the time stretching operation into a frequency shifting operation, a decimation or downsampling with the same transposition factor T is performed. When such a decimation by a transposition factor or transposition order T is performed on the time stretched Dirac function δ(t-Tt0), the downsampled Dirac pulse is time aligned with respect to the zero reference time 710 in the center of the first decomposition window 701. This is illustrated in FIG. 7.

[0108] However, when using different decomposition orders T, the decimation leads to different offsets with respect to the zero reference, unless the zero reference is aligned to the "zero" time of the input signal. As a result, a time offset adjustment of the decimated decomposition signals needs to be performed before they can be summed in the summation unit 502. As an example, a first decomposition unit of order T=3 and a second decomposition unit of order T=4 are assumed. Furthermore, it is assumed that the output signal of the core decoder is not upsampled. The decomposition units then decompose the third-order time-expanded signal by a factor of 3 / 2 and the fourth-order time-expanded signal by a factor of 2. The second-order time-expanded signal, i.e. T=2, is interpreted at the end as having a higher sampling frequency compared to the input signal, i.e. twice as high, which effectively pitch-shifts the output signal by a factor of 2.

[0109] It can be shown that in order to align the transformed and downsampled signals, a time offset of (T-2)L / 4 needs to be added to the transformed signal before decimation. That is, for third and fourth order transformations, an offset of L / 4 and L / 2 needs to be applied, respectively. To verify this with a concrete example, let us assume that the zero reference for a second order time-stretched signal corresponds to the time or sample value L / 2, i.e., the zero reference 710 in FIG. 7, since no decimation is used. For a third order time-stretched signal, the reference shifts to (L / 2)(2 / 3)=L / 3 due to the downsampling by a factor of 3 / 2. If a time offset according to the above rule is added before decimation, the reference shifts to ((L / 2)+(L / 4))(2 / 3)=L / 2. This means that the reference of the downsampled transformed signal is aligned with the zero reference 710. Similarly, for a fourth order transposition without offset, the zero reference corresponds to (L / 2)(1 / 2)=L / 4, but when using the proposed offset, the reference shifts to ((L / 2)+(L / 2))(1 / 2)=L / 2, which is also aligned with the second order zero reference 710, i.e., the zero reference for the transposed signal using T=2.

[0110] Another aspect to be considered when using several transposition orders simultaneously concerns the gain applied to transposition sequences with different transposition factors. In other words, one may address the aspect of combining the output signals of transpositioners with different transposition orders. When selecting the gain of the transposed signal, there are two principles, which can be considered under different theoretical approaches. In one option, the transposed signal is energy-conserving, i.e. the total energy in the low-band signal, which is then transposed and constitutes the high-band signal transposed T times, is conserved. In this case, the energy per bandwidth should be reduced by the transposition factor T, since the signal has been stretched in frequency by the same amount T. However, a sine wave with energy in an infinitesimal bandwidth will retain its energy after transposition. This is due to the fact that a sine wave is shifted in frequency when transposing, i.e. the duration in frequency (i.e. the bandwidth) is not changed by the frequency transposition operation, in the same way that a Dirac pulse is shifted in time by a transpositioner during time stretching, i.e. the duration of the pulse is not changed by the time stretching operation. That is, even if the energy per bandwidth is reduced by a factor of T, a sine wave still has all its energy at one point in frequency, so the energy per point is conserved.

[0111] Another option when choosing the gain of the transposed signal is to preserve the energy per bandwidth after transposition: in this case, broadband white noise and transient signals will have a flat frequency response after transposition, while the energy of sine waves will increase by a factor T.

[0112] A further aspect of the present invention is the selection of the analysis and synthesis phase vocoder windows when using a common analysis window. a (n) and v s It is beneficial to choose (n) carefully. To allow perfect reconstruction, the synthesis window v s Not only should (n) obey formula 2 above, but also the decomposition window v a(n) should also have sufficient sidelobe level rejection. Otherwise, undesired "aliasing" terms will typically become audible as interference with the main term for frequency-varying sine waves. Such undesired "aliasing" terms can also appear for stationary sine waves in the case of even transfer factors as mentioned above. The present invention proposes the use of a sine wave window because of its good sidelobe rejection ratio. Thus, the decomposition window is v a (n) = sin{(π / L)(n + 0.5)} 0≦n <L (4) It is proposed that:

[0113] Composite hop size Δt s If is not a divisor of the decomposition window length L, i.e., the decomposition window length L is not divisible by the synthesis hop size, the synthesis window v s (n) is the decomposition window v a (n) or given by formula (2) above. For example, L=1024, Δt s = 384, then 1024 / 384 = 2.667 is not an integer. It should be noted that it is also possible to choose a biorthogonal decomposition and synthesis window pair as outlined above. This can be beneficial for reducing aliasing in the output signal, especially when using even transformation orders T.

[0114] In the following, reference is made to Fig. 10 and Fig. 11, which show an exemplary encoder 1000 and an exemplary decoder 1100, respectively, for Unified Speech and Audio Coding (USAC). The general structure of the USAC encoder 1000 and decoder 1100 is described as follows: First, there may be a common pre-processing / post-processing consisting of an MPEGS (MPEG Surround) functional unit for handling stereo or multi-channel processing and an enhanced Spectral Band Replication (eSBR) unit 1001 and 1101 for handling parametric representation of higher audio frequencies in the input signal. eSBR may utilize the harmonic transformation method outlined in this paper. There are two branches, one consisting of a modified Advanced Audio Coding (AAC) tool path and the other consisting of a linear predictive coding (LP or LPC domain) based path. This latter features a frequency domain or time domain representation of the LPC residual. All transmitted spectra for both AAC and LPC may be represented in the MDCT domain and then quantized and arithmetically coded. The time domain representation may use the ACELP excitation coding scheme.

[0115] The enhanced spectral band replication (eSBR) unit 1001 of the encoder 1000 may include a high-frequency reconstruction system as outlined herein. In some embodiments, the eSBR unit 1001 may include a transformation unit as outlined in the context of Figures 4, 5 and 6. Encoded data related to harmonic transformation, such as the transformation order used, the amount of frequency domain oversampling required or the gain used, may be derived in the encoder 1000 and merged with other encoded information in a bitstream multiplexer and forwarded to the corresponding decoder 1100 as an encoded audio stream.

[0116] The decoder 1100 shown in FIG. 11 also comprises an enhanced Spectral Bandwidth Replication (eSBR) unit 1101, which receives the encoded audio bitstream or encoded signal from the encoder 1000 and uses the methods outlined herein to generate a high frequency component or high band of the signal, which is merged with the decoded low frequency component or low band to produce the decoded signal. The eSBR unit 1101 may comprise various components outlined herein, in particular a transformation unit as outlined in the context of FIG. 4, FIG. 5 and FIG. 6. The eSBR unit 1101 may use information about the high frequency components provided by the encoder 1000 via the bitstream to perform high frequency reconstruction. Such information may be the spectral envelope of the original high frequency components, the transformation order used, the amount of frequency domain oversampling required or the gain used to generate the synthesized subband signals and thus the high frequency components of the decoded signal.

[0117] Furthermore, Figures 10 and 11 show possible additional components of the USAC encoder / decoder, such as:

[0118] A bitstream payload demuxer tool, which separates the bitstream payload into parts for each tool and gives each tool the bitstream payload information that pertains to that tool.

[0119] A scale factor noiseless decoding tool that receives information from the bitstream payload demuxer, parses it, and decodes Huffman and DPCM encoded scale factors.

[0120] A Spectral Noiseless Decoding Tool that receives information from the Bitstream Payload Demultiplexer, parses it, decodes the arithmetically coded data, and reconstructs the quantized spectrum.

[0121] An inverse quantization tool, which receives the quantized values ​​for the spectrum and converts the integer values ​​into an unscaled reconstructed spectrum. This quantizer is preferably a companding quantizer, whose companding factor depends on the selected core coding mode.

[0122] A noise filling tool, which is used to fill spectral gaps in the decoded spectrum, which appear when spectral values ​​are quantized to 0 due to, for example, strong constraints on bit demand in the encoder.

[0123] A rescaling tool, which converts integer representations of scale factors to real-world values ​​and multiplies the unscaled, dequantized spectrum by the associated scale factor.

[0124] · M / S tools as described in ISO / IEC14496-3.

[0125] · Temporal noise shaping (TNS) tools as described in ISO / IEC14496-3.

[0126] A filterbank / block switching tool, which applies the inverse of the frequency mapping performed in the encoder. For the filterbank tool the inverse modified discrete cosine transform (IMDCT) is preferably used.

[0127] · Time-warped filter bank / block switching tool. This replaces the regular filter bank / block switching tool when time-warp mode is enabled. The filter bank is preferably the same as for the regular filter bank (IMDCT), and in addition the windowed time domain samples are mapped from the warped time domain to the linear time domain by time-varying resampling.

[0128] MPEG Surround (MPEGS) tools, which generate multiple signals from one or more input signals by applying a sophisticated upmix procedure to the input signals controlled by appropriate spatial parameters. In the USAC context, MPEGS is preferably used to code multi-channel signals by transmitting parametric side information together with the transmitted downmixed signal.

[0129] A signal classifier tool, which analyses the original input signal and generates from it control information that triggers the selection of various coding modes. The analysis of the input signal is typically implementation dependent and attempts to choose the optimal core coding mode for a given input signal frame. The output of the signal classifier may optionally also be used to influence the behavior of other tools, e.g. MPEG Surround, Enhanced SBR, Time-warping filter banks, etc.

[0130] The LPC Filter tool generates a time-domain signal from the excitation domain signal by filtering the reconstructed excitation signal through a linear predictive synthesis filter.

[0131] · ACELP tools, which provide an efficient way to represent time-domain excitation signals by combining long-term predictors (adaptive codewords) with pulse-like sequences (innovation codewords).

[0132] Figure 12 shows one embodiment of the eSBR unit shown in Figures 10 and 11. The eSBR unit 1200 is described below in the context of a decoder, and the input to the eSBR unit 1200 is the low frequency component, also known as the low band, of the signal.

[0133] In Fig. 12, the low frequency components 1213 are input to a QMF filter bank to generate QMF frequency bands. These QMF frequency bands should not be confused with the decomposition subbands outlined in this paper. The QMF frequency bands are used for the purpose of manipulating and merging the low and high frequency components of the signal in the frequency domain, not in the time domain. The low frequency components 1214 are input to a transformation unit 1204, which corresponds to the system for high frequency reconstruction outlined in this paper. The transformation unit 1204 generates the high frequency components 1212, also known as the high band of the signal, which are transformed to the frequency domain by the QMF filter bank 1203. Both the QMF transformed low frequency components and the QMF transformed high frequency components are input to a manipulation and merging unit 1205. This unit 1205 may perform an envelope adjustment of the high frequency components and combines the adjusted high and low frequency components. The combined output signal is transformed back to the time domain by the inverse QMF filter bank 1201.

[0134] Typically, the QMF filter bank 1202 has 32 QMF frequency bands. In such a case, the low frequency components 1213 are of bandwidth f s / 4, where f s / 2 is the sampling frequency of the signal 1213. The high frequency component 1212 has a bandwidth of f s / 2 and is filtered through a QMF bank 1203 having 64 QMF frequency bands.

[0135] In this paper, a method for harmonic inversion has been outlined. This harmonic inversion method is particularly suitable for the inversion of transient signals. The method involves a combination of frequency domain oversampling and harmonic inversion using a vocoder. The inversion operation depends on a combination of the decomposition window, the decomposition window stride, the inversion size, the synthesis window, the synthesis window stride, and on the phase adjustment of the decomposed signals. By using this method, undesirable effects such as pre-echo and post-echo can be avoided. Furthermore, the method does not use signal analysis measures such as transient signal detection, which typically introduce signal distortions due to discontinuities in the signal processing. Furthermore, the proposed method has a reduced computational complexity. The harmonic inversion method according to the present invention can be further improved by appropriate selection of the decomposition / synthesis window, gain values ​​and / or time alignment.

[0136] Several aspects will be described. [Aspect 1] A system for generating an output signal from an input signal using a transfer factor T, comprising: an analysis window unit for applying an analysis window of length La, thereby extracting frames of said input signal; A decomposition transformation unit of order M that transforms the sample values ​​into M complex coefficients; a nonlinear processing unit for modifying the phase of said complex coefficients by using a transformation factor T; · a synthesis transformation unit of order M for transforming the modified coefficients into M modified sample values; a synthesis window unit for applying a synthesis window of length Ls to the M modified sample values, thereby generating a frame of the output signal, M is based on the conversion factor T, system. [Aspect 2] 2. The system of embodiment 1, wherein a difference between M and an average length of the analysis window and the synthesis window is proportional to (T-1). [Aspect 3] 3. The system of embodiment 2, wherein M is greater than or equal to (TLa+Ls) / 2. Aspect 4 the decomposition transformation unit performs one of a Fourier transform, a fast Fourier transform, a discrete Fourier transform, and a wavelet transform; the synthesis transformation unit performs a corresponding inverse transformation; The system according to any one of embodiments 1 to 3. Aspect 5 a decomposition stride unit for shifting the decomposition window along the input signal by a decomposition stride of Sa samples; a synthesis stride unit for shifting the sequence of frames of said output signal by a synthesis stride of Ss sample values; a convolution-add unit for convolution-adding the sequence of shifted frames from the synthesis stride unit, thereby generating the output signal; 5. The system of any one of embodiments 1 to 4. Aspect 6 the synthesis stride is T times the decomposition stride; the output signal corresponds to the input signal time-stretched by a transformation factor T; The system of embodiment 5. Aspect 7 7. The system of any one of aspects 5 or 6, wherein the synthesis window is derived from the decomposition window and the decomposition stride. Aspect 8 The synthesis window is expressed by the formula

number

number

number

Claims

1. 1. An audio signal processing apparatus for transforming an input audio signal by a transformation factor T to generate an output audio signal, the audio signal processing apparatus comprising: extracting a frame of L time-domain samples of the input audio signal using an analysis window of length L having a function v(n)=sin((π / L)(n+0.5), where 0≦n<L; transforming the L time domain samples into M complex frequency domain coefficients; modifying the phase of the complex frequency domain coefficients using the transformation factor T; converting the modified frequency domain coefficients into M modified time domain samples; and generating a frame of L time-domain output samples of the output audio signal from the M modified time-domain samples using a synthesis window; M=F*L, where F is a frequency-domain oversampling factor determined in response to frequency-domain oversampling information received in the encoded bitstream; The frame of L time-domain output samples of the output audio signal includes a plurality of high-frequency components not present in the frame of L time-domain samples of the input audio signal, at least one of the high-frequency components being generated using a transformation factor T and at least one other of the high-frequency components being generated using a second transformation factor T 2 and T is generated using 2 is not equal to Audio signal processing device.

2. 2. The audio signal processing apparatus according to claim 1, wherein the oversampling factor F is equal to or greater than (T+1) / 2, and the transfer factor T is an integer greater than 1.

3. 2. The apparatus of claim 1, wherein the modifying the phase comprises multiplying the phase by a transformation factor T.

4. 2. The apparatus of claim 1, wherein the analysis window has a length L with zero padding with an additional (F-1)*L zeros.

5. The one or more components further comprise: shifting the decomposition window along the input audio signal by a decomposition stride to result in a sequence of frames of the input audio signal; shifting the sequence of frames of L time-domain output samples by a synthesis stride; and overlapping and adding the sequence of shifted frames of L time-domain output samples to generate the output signal.

2. The audio signal processing apparatus according to claim 1.

6. 6. The audio signal processing apparatus of claim 5, wherein the one or more components are further configured to increase a sampling rate of the output signal by a transformation order T, resulting in a transformed output signal.

7. 7. The audio signal processing apparatus of claim 6, wherein the synthesis stride is T times the decomposition stride.

8. 1. A method performed by an audio signal processing apparatus for transforming an input audio signal by a transformation factor T to generate an output audio signal, the method comprising: extracting a frame of L time-domain samples of the input audio signal using an analysis window of length L having a function v(n)=sin((π / L)(n+0.5), where 0≦n<L; transforming the L time domain samples into M complex frequency domain coefficients; modifying the phase of the complex frequency domain coefficients using the transformation factor T; converting the modified frequency domain coefficients into M modified time domain samples; generating a frame of L time-domain output samples of the output audio signal from the M modified time-domain samples using a synthesis window; M=F*L, where F is a frequency-domain oversampling factor determined in response to frequency-domain oversampling information received in the encoded bitstream; The frame of L time-domain output samples of the output audio signal includes a plurality of high-frequency components not present in the frame of L time-domain samples of the input audio signal, at least one of the high-frequency components being generated using a transformation factor T and at least one other of the high-frequency components being generated using a second transformation factor T 2 and T is generated using 2 is not equal to method.

9. 9. The method of claim 8, wherein transforming the L time-domain sample values ​​into M complex frequency-domain coefficients comprises performing one of a Fourier transform, a fast Fourier transform, a discrete Fourier transform, and a wavelet transform.

10. 9. The method of claim 8, wherein the oversampling factor F is greater than or equal to (T+1) / 2 and the transfer factor T is an integer greater than 1.

11. The method of claim 8 , wherein the input audio signal comprises low frequency components of an audio signal.

12. 10. A non-transitory computer readable medium having instructions for execution in an audio signal processing device, the instructions, when executed by the audio signal processing device, causing the audio signal processing device to perform the method of claim 8.

13. A computer program product comprising executable instructions for performing the method of claim 8 when executed on a computer.

Images