Encoder, decoder, encoding method and decoding method for long-term prediction of tone signals in the frequency domain for audio coding

Abstract

An encoder, decoder, encoding method, and decoding method for long-term prediction of a tone signal in the frequency domain for speech coding are provided. According to an embodiment, an encoder (100) is provided for encoding a current frame of an audio signal in response to one or more previous frames of the audio signal. The one or more previous frames precede the current frame, and each of the current frame and the one or more previous frames includes one or more harmonic components of the audio signal, and each of the current frame and the one or more previous frames includes a plurality of spectral coefficients in the frequency domain or the transform domain. To generate an encoded version of the current frame, the encoder (100) determines estimates of two harmonic parameters for each of the one or more harmonic components of an earliest frame of the one or more previous frames. Furthermore, the encoder (100) determines estimates of two harmonic parameters for each of the one or more harmonic components of the earliest frame using a first group of three or more spectral coefficients of the plurality of spectral coefficients of each of the one or more previous frames of the audio signal.

Description

[Technical field]

[0001] The present invention relates to speech signal coding, speech signal processing and speech signal decoding, and in particular to an apparatus and method for frequency domain long-term prediction of a tonal signal for speech coding. [Background technology]

[0002] In the field of speech coding, prediction is used to remove redundancy in speech signals. By subtracting predicted data from the original data and quantizing and coding the residual, which usually exhibits lower entropy, the bit rate for transmitting and storing speech signals can be reduced [1]. Long-Term Prediction (LTP) is a type of prediction technique that aims to remove periodic components in speech signals [2].

[0003] The MPEG-2 Advanced Audio Coding (AAC) standard uses the Modified Discrete Cosine Transform (MDCT) as a time-frequency transform and a perceptual audio coder with backward-adaptive LTP [3].

[0004] 4 is a diagram showing the configuration of a transform-type perceptual speech coder using backward-adaptive LTP. The speech coder in FIG. 4 includes an MDCT section 410, a psychoacoustic model section 420, a pitch estimation section 430, a long-term prediction section 440, a quantizer 450, and a quantizer reconstruction section 460.

[0005] As shown in Figure 4, the prediction unit receives a reconstructed MDCT frame as input. To perform conventional time-domain long-term prediction (TDLTP), the MDCT coefficients of the reconstructed signal need to be transformed into the time domain first. Then, the predicted time-domain segment is transformed back to the MDCT domain for residual calculation.

[0006] The MDCT uses overlapped analysis windows to mitigate the effects of blocking and provides perfect reconstruction through an overlap-add (OLA) procedure during the synthesis stage of the inverse transform [4]. Because alias-free reconstruction of the second half of the current frame requires the first half of the future frame [4], the prediction delay must be chosen carefully [2].

[0007] If only fully reconstructed samples in the buffer are used for prediction, there may be a delay of an integer number of pitch periods between the previous pitch lag selected and the predicted pitch lag. Due to the non-stationarity of the speech signal, longer delays may cause the prediction to become less stable. For signals with high fundamental frequencies, the pitch period is short, so the negative effect of this additional delay on the prediction may be more noticeable.

[0008] The concept of Frequency Domain Prediction (FDP), operating directly in the MDCT domain, was proposed in [5] (see also

[13] ). In that method, each harmonic component of a tonal signal is treated separately during prediction. The prediction of a bin in the current frame is obtained by computing the sinusoidal progression of its spectral neighbors in the previous frame. Summary of the Invention [Problem to be solved by the invention]

[0009] However, if the frequency resolution of these MDCT coefficients is relatively low with respect to the fundamental frequency of the harmonic signal, the harmonic components may overlap each other significantly on the bins, leading to poor performance of the frequency domain approach.

[0010] The object of the present invention is to provide an improved concept for the coding, processing and decoding of audio signals. This object is solved by an encoder according to claim 1, by a decoder according to claim 23, by an apparatus according to claim 45, by a method according to claim 52, by a method according to claim 53, by a method according to claim 54 and by a computer program according to claim 55. [Means for solving the problem]

[0011] According to an embodiment, an encoder is provided for encoding a current frame of an audio signal in response to one or more previous frames of the audio signal. The one or more previous frames precede the current frame, and each of the current frame and the one or more previous frames comprises one or more harmonic components of the audio signal, and each of the current frame and the one or more previous frames comprises a plurality of spectral coefficients in a frequency domain or a transform domain. To generate an encoded version of the current frame, the encoder determines an estimate of two harmonic parameters for each of the one or more harmonic components of an earlier frame of the one or more previous frames. Furthermore, the encoder determines an estimate of two harmonic parameters for each of the one or more harmonic components of the earlier frame using a first group of three or more spectral coefficients of the plurality of spectral coefficients of each of the one or more previous frames of the audio signal.

[0012] Further, a decoder for reconstructing a current frame of an audio signal according to an embodiment is provided, where one or more previous frames of the audio signal precede the current frame, where each of the current frame and the one or more previous frames comprises one or more harmonic components of the audio signal, where each of the current frame and the one or more previous frames comprises a plurality of spectral coefficients in the frequency domain or the transform domain. The decoder receives an encoding of the current frame. The decoder determines an estimate of two harmonic parameters for each of the one or more harmonic components of an earliest frame of the one or more previous frames. The two harmonic parameters for each of the one or more harmonic components of the earliest frame depend on a first group of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames of the audio signal. Further, the decoder reconstructs the current frame in response to the encoding of the current frame and in response to the estimate of the two harmonic parameters for each of the one or more harmonic components of the earliest frame.

[0013] Further, an apparatus for frame loss concealment according to an embodiment is provided. One or more previous frames of an audio signal precede a current frame of the audio signal. Each of the current frame and the one or more previous frames includes one or more harmonic components of the audio signal, and each of the current frame and the one or more previous frames includes a plurality of spectral coefficients in a frequency domain or a transform domain. The apparatus determines an estimate of two harmonic parameters for each of the one or more harmonic components of an earliest frame of the one or more previous frames, and the two harmonic parameters for each of the one or more harmonic components of the earliest frame depend on a first group of three or more spectral coefficients of the plurality of reconstructed (reconstructed) spectral coefficients for each of the one or more previous frames of the audio signal. If the apparatus does not receive the current frame or if the current frame is received by the apparatus in a corrupted state, the apparatus reconstructs the current frame according to the estimate of the two harmonic parameters for each of the one or more harmonic components of the earliest frame.

[0014] Further, a method for encoding a current frame of an audio signal depending on one or more previous frames of the audio signal according to an embodiment is provided, the one or more previous frames preceding the current frame. Each of the current frame and the one or more previous frames comprises one or more harmonic components of the audio signal. Each of the current frame and the one or more previous frames comprises a plurality of spectral coefficients in the frequency domain or in the transform domain. To generate an encoding of the current frame, the method comprises determining an estimate of two harmonic parameters for each of the one or more harmonic components of an earliest frame of the one or more previous frames. Determining the estimate of the two harmonic parameters for each of the one or more harmonic components of the earliest frame is performed using a first group of three or more spectral coefficients of the plurality of spectral coefficients of each of the one or more previous frames of the audio signal.

[0015] Further, a method of reconstructing a current frame of an audio signal according to an embodiment is provided. One or more previous frames of the audio signal precede the current frame. Each of the current frame and the one or more previous frames comprises one or more harmonic components of the audio signal. Each of the current frame and the one or more previous frames comprises a plurality of spectral coefficients in a frequency domain or a transform domain. The method includes receiving an encoding of the current frame. The method further includes determining an estimate of two harmonic parameters for each of the one or more harmonic components of an earliest frame of the one or more previous frames, the two harmonic parameters for each of the one or more harmonic components of the earliest frame being dependent on a first group of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames of the audio signal. The method further includes reconstructing the current frame in response to the encoding of the current frame and in response to the estimate of the two harmonic parameters for each of the one or more harmonic components of the earliest frame.

[0016] Further, a method for frame loss concealment according to an embodiment is provided. One or more previous frames of an audio signal precede a current frame of the audio signal, the current frame and the one or more previous frames each include one or more harmonic components of the audio signal, and the current frame and the one or more previous frames each include a plurality of spectral coefficients in a frequency domain or a transform domain. The method includes determining an estimate of two harmonic parameters for each of the one or more harmonic components of an earliest frame of the one or more previous frames, the two harmonic parameters for each of the one or more harmonic components of the earliest frame being dependent on a first group of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames of the audio signal. Furthermore, the method includes reconstructing the current frame according to the two harmonic parameters for each of the one or more harmonic components of the earliest frame if the current frame is not received or if the current frame is received due to a corrupted condition.

[0017] Furthermore, a computer program according to an embodiment is provided for performing one of the above mentioned methods when the computer program is executed by a computer or signal processing device.

[0018] Traditionally, Long-Term Prediction (LTP) is used to predict signals with certain periodicity in the time domain. For transform coding with backward adaptation in speech coders, the decoder unit generally has only frequency coefficients at hand, and therefore an inverse transform is required before prediction. The embodiments provide a Frequency Domain Least Mean Square Prediction (FDLMSP) concept that operates directly in the Modified Discrete Cosine Transform (MDCT) domain and significantly reduces the bit rate for speech coding, even under very low frequency resolution, for example. Thus, some embodiments may be employed in transform codecs to increase coding efficiency, especially in low-delay speech coding scenarios.

[0019] Some embodiments provide a Frequency Domain Least Mean Square Prediction (FDLMSP) concept that performs LTP directly in the MDCT domain. However, instead of making predictions for each bin individually, this new concept uses a real-valued linear equation system to model the harmonic content of the tonal signal in the transform domain. The prediction is performed after solving the linear equation system with the Least Mean Squares (LMS). Then, based on the nature of the phase progression of the harmonics, the parameters of the harmonics are used to predict the current frame. It should be noted that this prediction concept can also be applied to other real-valued linear transforms or filter banks, such as different types of Discrete Cosine Transforms (DCTs) and Polyphase Quadrature Filters (PQFs) [6].

[0020] In the following, we present the signal model, explain the harmonic content estimation and prediction process in detail, describe experiments to evaluate the FDLMSP concept in comparison with TDLTP and FDP, and present and discuss the results. [Brief description of the drawings]

[0021] Hereinafter, an embodiment of the present invention will be described in more detail with reference to the drawings. [Figure 1] FIG. 1 illustrates an encoder for encoding a current frame of an audio signal depending on one or more previous frames of the audio signal according to an embodiment. [Diagram 2] FIG. 2 illustrates a decoder for decoding an encoding of a current frame of an audio signal according to an embodiment. [Diagram 3] FIG. 3 is a diagram for explaining the system according to the embodiment. [Figure 4] FIG. 4 shows the structure of a transform perceptual audio coder with backward-adaptive LTP. [Diagram 5] FIG. 5 shows a diagram illustrating the bit rate saved with single note prediction using three prediction concepts with different prediction bandwidths and MDCT lengths. [Figure 6] FIG. 6 shows the bit rate savings in four different working modes for six different items with bandwidth limited to 4 kHz and MDCT frame lengths of 64 and 512. [Figure 7] FIG. 7 illustrates an apparatus for frame loss concealment according to an embodiment. [Figure 8] FIG. 8 is a schematic block diagram of an encoder for encoding an audio signal of the FDP prediction concept according to an embodiment. [Figure 9] FIG. 9 is a schematic block diagram of a decoder 201 for decoding the coded signal 120 of the FDP prediction concept according to an example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] FIG. 1 illustrates an encoder 100 for encoding a current frame of an audio signal depending on one or more previous frames of the audio signal, according to an embodiment.

[0023] The one or more previous frames precede the current frame, and each of the current frame and the one or more previous frames includes one or more harmonic components of the audio signal, and each of the current frame and the one or more previous frames includes a plurality of spectral coefficients in a frequency domain or a transform domain.

[0024] To generate an encoded version of the current frame, the encoder 100 determines an estimate of two harmonic parameters for each of the one or more harmonic components of an earlier-most frame of the one or more previous frames. Additionally, the encoder 100 determines an estimate of two harmonic parameters for each of the one or more harmonic components of the earlier-most frame using a first group of three or more spectral coefficients of the plurality of spectral coefficients of each of the one or more previous frames of the speech signal.

[0025] The earliest frame can be, for example, the earliest frame relative to the current frame.

[0026] The earliest frame may be, for example, the immediately previous frame (which may be referred to as the immediately previous frame). The immediately previous frame may be, for example, the one that immediately precedes the current frame.

[0027] The current frame includes one or more harmonic components of the speech signal. Each of the one or more previous frames may include one or more harmonic components of the speech signal. The fundamental frequencies of the one or more harmonic components in the current frame and the one or more previous frames are assumed to be the same.

[0028] According to an embodiment, the encoder 100 may be configured to estimate two harmonic parameters for each of the one or more harmonic components of the most recent frame without using a second group of one or more further spectral coefficients of each of the plurality of spectral coefficients of the one or more previous frames.

[0029] According to an embodiment, the encoder 100 may be configured to determine a gain factor and a residual as a function of fundamental frequencies of one or more harmonic components of the current frame and one or more previous frames as an encoding of the current frame, and as a function of estimates of two harmonic parameters for each of the one or more harmonic components of the most previous frame. The encoder 100 may be configured to generate an encoding of the current frame, such that the encoding of the current frame includes a gain factor and a residual signal.

[0030] In an embodiment, the encoder 100 may be configured to determine an estimate of the two harmonic parameters for each of the one or more harmonic components of the current frame, for example, in response to an estimate of the two harmonic parameters for each of the one or more harmonic components of the most recent frame, and in response to the fundamental frequencies of the one or more harmonic components of the current frame and one or more previous frames. The fundamental frequencies may, for example, be assumed to be invariant across the current frame and one or more previous frames.

[0031] According to an embodiment, the two harmonic parameters for each of the one or more harmonic components are a first parameter for a cosine subcomponent and a second parameter for a sine subcomponent for each of the one or more harmonic components.

[0032] In an embodiment, the encoder 100 may be configured to estimate two harmonic parameters for each of the one or more harmonic components of the most previous frame, for example, by solving a system of linear equations including at least three equations, each of the at least three equations depending on a first group of spectral coefficients consisting of three or more spectral coefficients of each of the plurality of spectral coefficients of the one or more previous frames.

[0033] According to an embodiment, the encoder 100 may be configured to solve a system of linear equations using, for example, a least mean squares algorithm.

[0034] TIFF0007638990000001.tif68170

[0035] In some embodiments, r≧1.

[0036] TIFF0007638990000002.tif222163TIFF0007638990000003.tif33158

[0037] TIFF0007638990000004.tif86170

[0038] In an embodiment, the encoder 100 may, for example, encode the fundamental frequency of the harmonic components, the window function, the gain factor, and the residual signal.

[0039] According to an embodiment, the encoder 100 may be configured to determine, for example, a number of the one or more harmonic components of the most recent frame and a fundamental frequency of the one or more harmonic components of the most recent frame before estimating two harmonic parameters for each of the one or more harmonic components of the most recent frame using a first group of three or more spectral coefficients of each of the plurality of spectral coefficients of the one or more previous frames of the audio signal.

[0040] According to an embodiment, the encoder 100 may be configured to, for example, determine one or more groups of harmonic components from the one or more harmonic components and apply a prediction of the audio signal to the one or more groups of harmonic components. The encoder 100 may be configured to, for example, encode an order for each of the one or more groups of harmonic components of the most recent frame.

[0041] TIFF0007638990000005.tif145163

[0042] According to an embodiment, the encoder 100 may be configured to determine a residual signal, for example, depending on a plurality of spectral coefficients of the current frame in the frequency domain or in the transform domain and depending on estimates of two harmonic parameters for each of one or more harmonic components of the current frame, and the encoder 100 may be configured to, for example, encode the residual signal.

[0043] In an embodiment, the encoder 100 may be configured to determine a spectral prediction of one or more of the spectral coefficients of the current frame, for example, in response to an estimate of two harmonic parameters for each of one or more harmonic components of the current frame. The encoder 100 may be configured to determine a residual signal and a gain factor, for example, in response to the spectral coefficients of the current frame in the frequency domain or the transform domain and in response to a spectral prediction of three or more of the spectral coefficients of the current frame. Here, the encoder 100 may be configured to generate an encoding of the current frame, such that the encoding of the current frame includes the residual signal and the gain factor.

[0044] TIFF0007638990000006.tif64170

[0045] FIG. 2 illustrates a decoder 200 for reconstructing a current frame of an audio signal according to an embodiment.

[0046] One or more previous frames of the audio signal precede the current frame, and each of the current frame and the one or more previous frames includes one or more harmonic components of the audio signal, and each of the current frame and the one or more previous frames includes a plurality of spectral coefficients in a frequency domain or a transform domain.

[0047] The decoder 200 receives an encoding of the current frame.

[0048] Additionally, the decoder 200 determines an estimate of two harmonic parameters for each of the one or more harmonic components of an earliest frame of the one or more previous frames, the two harmonic parameters for each of the one or more harmonic components of the earliest frame being dependent on a first group of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames of the speech signal.

[0049] Furthermore, the decoder 200 reconstructs the current frame in response to having encoded the current frame and in response to estimates of the two harmonic parameters for each of the one or more harmonic components of the most recent frame.

[0050] The earliest frame can be, for example, the earliest frame relative to the current frame.

[0051] The earliest frame may be, for example, the immediately previous frame (which may be referred to as the immediately previous frame). The immediately previous frame may be, for example, the one that immediately precedes the current frame.

[0052] The current frame includes one or more harmonic components of the speech signal. Each of the one or more previous frames may include one or more harmonic components of the speech signal. The fundamental frequencies of the one or more harmonic components in the current frame and the one or more previous frames are assumed to be the same.

[0053] According to an embodiment, the two harmonic parameters for each of the one or more harmonic components of the most recent frame are independent of a second group of one or more further spectral coefficients of the plurality of spectral coefficients of the one or more previous frames.

[0054] In an embodiment, the decoder 200 may, for example, determine an estimate of two harmonic parameters for each of one or more harmonic components of the current frame in response to an estimate of two harmonic parameters for each of one or more harmonic components of the most previous frame, and in response to fundamental frequencies of the one or more harmonic components of the current frame and one or more previous frames.

[0055] According to an embodiment, the decoder 100 may be configured to receive an encoding of a current frame, for example including a gain factor and a residual signal. The decoder 200 may be configured to reconstruct the current frame, for example depending on the gain factor, depending on the residual signal and depending on the fundamental frequencies of one or more harmonic components of the current frame and one or more previous frames. The fundamental frequencies may for example be assumed to be unchanged across the current frame and one or more previous frames.

[0056] According to an embodiment, the two harmonic parameters for each of the one or more harmonic components are a first parameter for a cosine subcomponent and a second parameter for a sine subcomponent for each of the one or more harmonic components.

[0057] In an embodiment, the two harmonic parameters for each of the one or more harmonic components of the most previous frame depend on a system of linear equations including at least three equations, each of the at least three equations depending on a first group of spectral coefficients consisting of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames.

[0058] According to one embodiment, the system of linear equations can be solved using a least mean squares algorithm.

[0059] TIFF0007638990000007.tif69170

[0060] In some embodiments, r≧1.

[0061] TIFF0007638990000008.tif199170

[0062] TIFF0007638990000009.tif104160

[0063] TIFF0007638990000010.tif78170

[0064] In an embodiment, the decoder 200 may be configured to receive, for example, a fundamental frequency of a harmonic component, a window function, a gain factor, and a residual signal, and may be configured to reconstruct a current frame, for example, according to the fundamental frequency of one or more harmonic components of the most recent frame, according to an order of the harmonic components, according to a window function, according to a gain factor, and according to the residual signal.

[0065] Only the fundamental frequency, the order of the harmonic components, the window function, the gain factor, and the residual need to be transmitted. The decoder 200 can, for example, calculate U based on this received information and perform harmonic parameter estimation and current frame prediction. The decoder can then reconstruct the current frame, for example, by adding the transmitted residual spectrum to the predicted spectrum scaled by the transmitted gain factor.

[0066] According to an embodiment, the decoder 200 may be configured to receive, for example, a number of one or more harmonic components of the most recent frame and a fundamental frequency of the one or more harmonic components of the most recent frame. The decoder 200 may be configured to decode an encoding of the current frame, for example, depending on the number of one or more harmonic components of the most recent frame and depending on the fundamental frequency of the one or more harmonic components of the current frame and one or more previous frames.

[0067] According to an embodiment, the decoder 200 decodes an encoding of a current frame according to one or more groups of harmonic components, the decoder 200 applying a prediction of the audio signal in the one or more groups of harmonic components.

[0068] According to an embodiment, the decoder 200 may be configured, for example, to determine two harmonic parameters for each of one or more harmonic components of the current frame in dependence on two harmonic parameters for each of one of the one or more harmonic components of the most recent frame.

[0069] TIFF0007638990000011.tif178164

[0070] According to an embodiment, the decoder 200 may be configured, for example, to receive a residual signal, which depends on a number of spectral coefficients of the current frame in the frequency domain or in the transform domain, and which depends on estimates of two harmonic parameters for each of one or more harmonic components of the current frame.

[0071] In an embodiment, the decoder 200 may be configured to determine a spectral prediction of one or more of the plurality of spectral coefficients of the current frame, for example in response to estimates of two harmonic parameters for each of the one or more harmonic components of the current frame, and the decoder 200 may be configured to determine a current frame of the audio signal, for example in response to the spectral prediction of the current frame, in response to the residual signal and in response to a gain factor.

[0072] TIFF0007638990000012.tif46170

[0073] FIG. 3 is a diagram illustrating a system according to an embodiment.

[0074] The system comprises an encoder 100 according to one of the above described embodiments for encoding a current frame of an audio signal.

[0075] Furthermore, the system comprises a decoder 200 according to one of the embodiments described above for decoding the encoding of the current frame of the audio signal.

[0076] FIG. 7 illustrates an apparatus 700 for frame loss concealment according to an embodiment.

[0077] One or more previous frames of the audio signal precede a current frame of the audio signal, the current frame and the one or more previous frames each include one or more harmonic components of the audio signal, and the current frame and the one or more previous frames each include a plurality of spectral coefficients in a frequency domain or a transform domain.

[0078] The apparatus 700 determines an estimate of two harmonic parameters for each of the one or more harmonic components of an earlier frame of the one or more previous frames, wherein the two harmonic parameters for each of the one or more harmonic components of the earlier frame are dependent on a first group of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames of the audio signal.

[0079] If the device 700 does not receive the current frame, or if the current frame is received by the device 700 in a corrupted state, the device 700 reconstructs the current frame according to estimates of two harmonic parameters for each of the one or more harmonic components of the most recent frame.

[0080] The earliest frame can be, for example, the earliest frame relative to the current frame.

[0081] The earliest frame may be, for example, the immediately previous frame (which may be referred to as the immediately previous frame). The immediately previous frame may be, for example, the one that immediately precedes the current frame.

[0082] The current frame includes one or more harmonic components of the audio signal. Each of the one or more previous frames may include one or more harmonic components of the audio signal. The fundamental frequencies of the one or more harmonic components in the current frame and the one or more previous frames may be assumed to be the same.

[0083] According to an embodiment, the device 700 may be configured to receive, for example, the number of one or more harmonic components of the most recent frame, and may decode an encoding of the current frame, for example, depending on the number of one or more harmonic components of the most recent frame and depending on fundamental frequencies of the one or more harmonic components of the current frame and the one or more previous frames.

[0084] In an embodiment, to reconstruct the current frame, the apparatus 700 may be configured, for example, to determine an estimate of two harmonic parameters for each of one or more harmonic components of the current frame in response to an estimate of two harmonic parameters for each of one or more harmonic components of the most previous frame.

[0085] TIFF0007638990000013.tif175164

[0086] According to an embodiment, the apparatus 700 may be configured to determine a spectral prediction of three or more spectral coefficients of the plurality of spectral coefficients of the current frame, for example in response to an estimate of two harmonic parameters for each of the one or more harmonic components of the current frame.

[0087] In the following, preferred embodiments are provided.

[0088] First, the signal model is described.

[0089] TIFF0007638990000014.tif73170

[0090] TIFF0007638990000015.tif93170

[0091] TIFF0007638990000016.tif75151

[0092] TIFF0007638990000017.tif120169

[0093] In the following, harmonic estimation and prediction are described.

[0094] Based on the above assumed signal model according to equations (3)-(8), and adding the assumption that the frequencies of harmonic components do not change abruptly between adjacent frames, the proposed FDLMSP approach can be divided into three steps. For example, to predict the m-th frame, we first estimate the frequency information of all harmonic components contained in the m-th frame. This frequency information will later be transmitted as part of the side information to aid prediction in the decoder 200. Then, the parameters of each harmonic component in the m-1-th frame (h=[1,...,H], where a h , b h ) is estimated using only the previous frame.

[0095] Finally, the mth frame is predicted based on the estimated harmonic parameters. Then, the residual spectrum is calculated and further processed (e.g., quantized and transmitted). The pitch information of each frame can be obtained by the pitch estimator.

[0096] First, harmonic estimation will be described in detail.

[0097] TIFF0007638990000018.tif61170

[0098] TIFF0007638990000019.tif133170

[0099] TIFF0007638990000020.tif59170

[0100] TIFF0007638990000021.tif176170

[0101] If the number of parameters to be estimated exceeds the number of MDCT bins spanned by the harmonics, an underdetermined system of linear equations will result. This is avoided by stacking the matrix U vertically and the vector X horizontally with the corresponding values ​​from earlier frames. However, no extra delay is incurred since (most of) the previous frames are already in the buffer. Conversely, this extension makes the proposed method applicable to scenarios with extremely low frequency resolution with densely populated harmonic components. It is also possible to apply a scaling factor to the number of previous frames employed to guarantee an overdetermined system of linear equations. This also improves the robustness of the prediction concept against noise in the signal.

[0102] Now, the details of the prediction will be explained.

[0103] TIFF0007638990000022.tif111170

[0104] TIFF0007638990000023.tif53170

[0105] In bins where no prediction is made, the predicted value is set to zero.

[0106] However, due to non-stationarity of the signal, the amplitudes of the harmonics may vary slightly between successive frames. To account for the amplitude variations, a gain factor is introduced and transmitted to the decoder 200 as part of the side information.

[0107] TIFF0007638990000024.tif29166

[0108] Below we provide an evaluation of the above concepts presented.

[0109] To evaluate the performance of the proposed FDLMSP concept, a Python coder environment was constructed according to Fig. 4. The proposed concept is implemented according to the above description with r = 2. For comparison, TDLTP and FDP are reimplemented according to references [2], [5]. The aim is to use experiments to evaluate the three prediction concepts in three aspects: (i) their performance with respect to different frequency resolutions of the MDCT coefficients, (ii) their sensitivity to the inharmonicity of the test material [7], and (iii) their overall performance and capabilities compared to each other in the same coding scenario. Inharmonicity of a tone usually means that its higher harmonics are no longer equidistant. As higher band harmonics are perceptually less important [8], the impact of this factor by using different prediction bandwidths was evaluated.

[0110] In the experiments, a sampling frequency of 16 kHz and MDCT frame lengths of 64, 128, 256, and 512 were used. Predictions were performed with limited bandwidths of 1 kHz, 2 kHz, 4 kHz, and 8 kHz. The analysis window was chosen as a sine window, which satisfies the constraint for perfect reconstruction [9]. This method can also handle asymmetric windows when switching to different frame lengths. To improve the accuracy of the harmonic estimation, the F(ω) function is calculated by interpolating the transfer function of the analysis window. In TDLTP, a three-tap prediction filter is calculated for each frame based on the autocorrelation concept using the perfectly reconstructed data and the original time-domain signal. When searching for the previous perfectly reconstructed pitch lag from the buffer data, the possibility that the pitch lag is not an integer multiple of the sampling interval is also taken into account. The number of temporal or spectral neighboring bins in FDP is limited to two.

[0111] The YIN algorithm

[10] is used for pitch estimation. The search range of fo is set to [20,...,1000]Hz, with a harmonic threshold of 0.25. A perceptual model based on an infinite impulse response IIR filter bank proposed in

[11] is used to calculate the masking threshold for quantization. A finer pitch search (±0.5Hz, step size 0.02Hz) around the YIN estimate and an optimal gain coefficient search in [0.5,...,2], step size 0.01 are performed simultaneously in each frame by minimizing the perceptual entropy (PE)

[12] of the (quantized) residual after quantization (an entropy approximation of the residual spectrum after quantization taking into account the perceptual model).

[0112] The encoder has four operating modes: FDLMSP, TDLTP, FDP, and Adaptive MDCT LTP (AMLTP). In AMLTP mode, the encoder switches between different prediction concepts on a frame-by-frame basis, with the PE minimization as the criterion. In all four operating modes, no prediction is performed in a frame if the PE of the residual spectrum is higher than the original signal spectrum.

[0113] In each mode, the coder was tested on six different materials: three single tones of duration 1-2 seconds: a bass tone (f0 around 50Hz), a harpsichord tone (f0 around 88Hz), and a pitch pipe tone (f0 around 290Hz). These test materials have a relatively regular harmonic structure and slowly varying time envelopes. The coder was also tested on more complex test materials: a trumpet piece (~5 seconds, f0 varies between 300Hz and 700Hz), a female vocal (~10 seconds, f0 varies between 200Hz and 300Hz), and a male speech (~8 seconds, f0 varies between 100Hz and 220Hz). These three test materials have large envelope variations, pitches change faster over time, and harmonic structures that are less regular. During the experiments, it was found that the bass tone had a stronger second harmonic than the first harmonic, which consistently led to incorrect pitch estimates. Therefore, in order to estimate the pitch correctly, the f0 search range of this bass note in the YIN pitch estimator was adjusted.

[0114] The average PE of the quantized residual spectrum and the quantized spectrum of the original signal was estimated. Based on the estimated PE, the bitrate (BS) [bits / s] that can be saved when transmitting the signal by applying the prediction was calculated (bitrate consumption of side information is not taken into account). First, the comparison was limited to one-tone prediction in order to consider the behavior of each concept and make reasonable inferences and analyses. Then, the performance of the four modes was compared with the same parameter configuration.

[0115] FIG. 5 shows the bit rate savings for one-tone prediction using three prediction concepts with different prediction bandwidths and MDCT lengths.

[0116] First, we will explain the prior art FDP prediction concept, which is explained in more detail in [5] and

[13] (WO 2016 142357 A1, published in September 2016).

[0117] Fig. 8 is a schematic block diagram of an encoder 101 for encoding an audio signal 102 in an FDP prediction concept according to an example. The encoder 101 is configured to encode the audio signal 102 in a transform or filter bank domain 104 (e.g., frequency domain or spectral domain), the encoder 101 being configured to determine spectral coefficients 106_t0_f1 to 106_t0_f6 of the audio signal 102 for a current frame 108_t0 and spectral coefficients 106_t-1_f1 to 106_t-1_f6 of the audio signal for at least one previous frame 108_t-1. Furthermore, the encoder 101 is configured to selectively apply predictive coding to a plurality of individual spectral coefficients 106_t0_f2 or a group of spectral coefficients 106_t0_f4 and 106_t0_f5, the encoder 101 being configured to determine an interval value. Here, the encoder 101 is adapted to select, based on an interval value, a plurality of individual spectral coefficients 106_t0_f2 or a group of spectral coefficients 106_t0_f4 and 106_t0_f5 to which predictive coding is applied.

[0118] In other words, the encoder 101 is configured to selectively apply predictive coding to a plurality of individual spectral coefficients 106_t0_f2 or a group of spectral coefficients 106_t0_f4, 106_t0_f5 selected based on a single spacing value transmitted as side information.

[0119] This interval value may correspond to a frequency (e.g., the fundamental frequency of a harmonic timbre (of the audio signal 102)) that defines the center of all groups of spectral coefficients to which the prediction is applied together with its integer multiples. A first group may be centered on this frequency, a second group may be centered on two times this frequency, a third group may be centered on three times this frequency, and so on. Knowing these center frequencies allows the computation of prediction coefficients for predicting the corresponding sinusoidal signal components (such as the fundamental and overtones of a harmonic signal). This avoids the need for complex and error-prone reverse adaptation of the prediction coefficients.

[0120] In an example, the encoder 101 may be configured to determine one spacing value per frame.

[0121] In an embodiment, a group of multiple individual spectral coefficients 106_t0_f2 or spectral coefficients 106_t0_f4 and 106_t0_f5 may be separated by at least one spectral coefficient 106_t0_f3.

[0122] In an example, the encoder 101 may be configured to apply predictive coding to a plurality of individual spectral coefficients separated by at least one spectral coefficient, e.g., two individual spectral coefficients separated by at least one spectral coefficient. Furthermore, the encoder 101 may be configured to apply predictive coding to a plurality of groups of spectral coefficients (each of the groups including at least two spectral coefficients) separated by at least one spectral coefficient, e.g., two groups of spectral coefficients separated by at least one spectral coefficient. Furthermore, the encoder 101 may be configured to apply predictive coding to a plurality of individual spectral coefficients and / or groups of spectral coefficients separated by at least one spectral coefficient, e.g., at least one individual spectral coefficient and at least one group of spectral coefficients separated by at least one spectral coefficient.

[0123] In the example shown in Fig. 8, the encoder 101 is configured to determine six spectral coefficients 106_t0_f1 to 106_t0_f6 of the current frame 108_t0 and six spectral coefficients 106_t-1_f1 to 106_t-1_f6 of the (most) previous frame 108_t-1. Thereby, the encoder 101 is configured to selectively apply predictive coding to an individual second spectral coefficient 106_t0_f2 of the current frame and to a group of spectral coefficients consisting of a fourth spectral coefficient 106_t0_f4 and a fifth spectral coefficient 106_t0_f5 of the current frame 108_t0. As can be seen, the individual second spectral coefficient 106_t0_f2 and the group of spectral coefficients consisting of the fourth and fifth spectral coefficients 106_t0_f4 and 106_t0_f5 are separated from each other by a third spectral coefficient 106_t0_f3.

[0124] It should be noted that the term "selectively" as used herein means applying predictive coding to (only) selected spectral coefficients. In other words, predictive coding is not necessarily applied to all spectral coefficients, but rather only to selected individual spectral coefficients or groups of spectral coefficients, and only to selected individual spectral coefficients and / or groups of spectral coefficients that are separable from each other by at least one spectral coefficient. In other words, predictive coding may be disabled for at least one spectral coefficient from which a selected number of individual spectral coefficients or groups of spectral coefficients are separated.

[0125] In an example, the encoder 101 may be configured to selectively apply predictive coding to a plurality of individual spectral coefficients 106_t0_f2 or a group of spectral coefficients 106_t0_f4 and 106_t0_f5 of a current frame 108_t0 based on at least a corresponding plurality of individual spectral coefficients 106_t-1_f2 or a group of spectral coefficients 106_t-1_f4 and 106_t-1_f5 of a previous frame 108_t-1.

[0126] For example, the encoder 101 may be configured to predictively encode a plurality of individual spectral coefficients 106_t0_f2 or a group of predicted spectral coefficients 110_t0_f2 and 110_t0_f5 of the current frame 108_t0 by encoding a prediction error between the plurality of predicted individual spectral coefficients 110_t0_f2 or the group of predicted spectral coefficients 110_t0_f4 and 110_t0_f5 of the current frame 108_t0 and the plurality of individual spectral coefficients 106_t0_f2 or the group of spectral coefficients 106_t0_f4 and 106_t0_f5 of the current frame (or a quantized version thereof).

[0127] In FIG. 8, the encoder 101 encodes the individual spectral coefficient 106_t0_f2 and the group of spectral coefficients consisting of the spectral coefficients 106_t0_f4 and 106_t0_f5 by encoding the prediction error between the predicted individual spectral coefficient 110_t0_f2 of the current frame 108_t0 and the individual spectral coefficient 106_t0_f2 of the current frame 108_t0, and the prediction error between the group of predicted spectral coefficients 110_t0_f4 and 110_t0_f5 of the current frame and the group of spectral coefficients 106_t0_f4 and 106_t0_f5 of the current frame.

[0128] In other words, the second spectral coefficient 106_t0_f2 is coded by coding the prediction error (or difference) between the predicted second spectral coefficient 110_t0_f2 and the (actual or determined) second spectral coefficient 106_t0_f2. The fourth spectral coefficient 106_t0_f4 is coded by coding the prediction error (or difference) between the predicted fourth spectral coefficient 110_t0_f4 and the (actual or determined) fourth spectral coefficient 106_t0_f4. The fifth spectral coefficient 106_t0_f5 is coded by coding the prediction error (or difference) between the predicted fifth spectral coefficient 110_t0_f5 and the (actual or determined) fifth spectral coefficient 106_t0_f5.

[0129] In one example, the encoder 101 can be configured to determine a group of predicted individual spectral coefficients 110_t0_f2 or predicted spectral coefficients 110_t0_f4, 110_t0_f5 of the current frame 108_t0 by a corresponding actual version of a group of individual spectral coefficients 106_t-1_f2 or spectral coefficients 106_t-1_f4, 106_t-1_f5 (of the previous frame 108_t-1).

[0130] That is, in the above-mentioned decision process, the encoder 101 can directly use a number of actual individual spectral coefficients 106_t-1_f2 or a group of actual spectral coefficients 106_t-1_f4, 106_t-1_f5 of the previous frame 108_t-1, where 106_t-1_f2, 106_t-1_f4, 106_t-1_f5 represent the original, not yet quantized spectral coefficients or groups of spectral coefficients, respectively, which are obtained by the encoder 101 so that the encoder can operate in the transform domain or the filter bank domain 104.

[0131] For example, the encoder 101 may be configured to determine a second predicted spectral coefficient 110_t0_f2 of the current frame 108_t0 based on a corresponding, not yet quantized version of the second spectral coefficient 106_t-1_f2 of the previous frame 108_t-1, to determine a predicted fourth spectral coefficient 110_t0_f4 of the current frame 108_t0 based on a corresponding, not yet quantized version of the fourth spectral coefficient 106_t-1_f4 of the previous frame 108_t-1, and to determine a predicted fifth spectral coefficient 110_t0_f5 of the current frame 108_t0 based on a corresponding, not yet quantized version of the fifth spectral coefficient 106_t-1_f5 of the previous frame.

[0132] This approach allows the predictive coding and decoding scheme to exhibit a kind of harmonic shaping of the quantization noise, an example of which will be described below with respect to Fig. 11, since the corresponding decoder can only adopt the transmitted quantized version of the individual spectral coefficients 106_t-1_f2 or groups of spectral coefficients 106_t-1_f4 and 106_t-1_f5 of the previous frame 108_t-1 for predictive decoding in the above-mentioned decision step.

[0133] Although such harmonic noise shaping may be subjectively advantageous for predictive coding, e.g. as conventionally done by long-term prediction (LTP) in the time domain, it may also be undesirable in some cases, since it may result in an unnecessary excessive amount of tonality being introduced into the decoded audio signal. For this reason, an alternative predictive coding scheme is described below, which is fully synchronous with the corresponding decoding, and therefore only exploits all possible prediction gains, and does not result in quantization noise shaping. According to this alternative coding example, the encoder 101 may be configured to determine a group of a plurality of predicted individual spectral coefficients 110_t0_f2 or a group of predicted spectral coefficients 110_t0_f4, 110_t0_f5 of the current frame 108_t0 using a corresponding quantized version of a group of a plurality of individual spectral coefficients 106_t-1_f2 or a group of spectral coefficients 106_t-1_f4, 106_t-1_f5 of the previous frame 108_t-1.

[0134] For example, the encoder 101 may be configured to determine a second predicted spectral coefficient 110_t0_f2 of the current frame 108_t0 based on a corresponding quantized version of the second spectral coefficient 106_t-1_f2 of the previous frame 108_t-1, to determine a predicted fourth spectral coefficient 110_t0_f4 of the current frame 108_t0 based on a corresponding quantized version of the fourth spectral coefficient 106_t-1_f4 of the previous frame 108_t-1, and to determine a predicted fifth spectral coefficient 110_t0_f5 of the current frame 108_t0 based on a corresponding quantized version of the fifth spectral coefficient 106_t-1_f5 of the previous frame.

[0135] Further, the encoder 101 may be configured to derive prediction coefficients 112_f2, 114_f2, 112_f4, 114_f4, 112_f5 and 114_f5 from the interval values, and a plurality of individual spectral coefficients 106_t-1_f2 and 106_t-2_f2 or spectral coefficients 106_t-1_f4, 106_t-2_f4, 106_t-1_f5 of at least two previous frames 108_t-1 and 108_t-2. , and 106_t-2_f5, and using the derived prediction coefficients 112_f2, 114_f2, 112_f4, 114_f4, 112_f5 and 114_f5, a plurality of predicted individual spectral coefficients 110_t0_f2 or a group of predicted spectral coefficients 110_t0_f4, _t0_f5 for the current frame 108_t0 can be configured to be calculated.

[0136] For example, the encoder 101 can be configured to derive prediction coefficients 112_f2 and 114_f2 for the second spectral coefficient 106_t0_f2 from the interval value, derive prediction coefficients 112_f4 and 114_f4 for the fourth spectral coefficient 106_t0_f4 from the interval value, and derive prediction coefficients 112_f5 and 114_f5 for the fifth spectral coefficient 106_t0_f5 from the interval value.

[0137] For example, the derivation of the prediction coefficients can be done as follows: If the spacing value corresponds to a frequency f0 or its coded version, then the center frequency of the Kth group of spectral coefficients for which prediction is possible is fc=K*f0. If the sampling frequency is fs and the transform hop size (shift between successive frames) is N, then the ideal prediction coefficients for the Kth group, assuming a sinusoidal signal of frequency fc, are: p1=2*cos(N*2*pi*fc / fs) and p2=-1

[0138] For example, if both spectral coefficients 106_t0_f4 and 106_t0_f5 are in this group, the prediction coefficients are: 112_f4=112_f5=2*cos(N*2*pi*fc / fs) and 114_f4=114_f5=-1

[0139] For stability reasons, a damping factor d can be introduced to modify the prediction coefficients. 112_f4'=112_f5'=d*2*cos(N*2*pi*fc / fs), 114_f4'=114_f5'=d 2

[0140] Since the spacing values ​​are transmitted in the coded audio signal 120, the decoder can derive exactly the same prediction coefficients 212_f4=212_f5=2*cos(N*2*pi*fc / fs) and 114_f4=114_f5=-1. If damping factors are used, the coefficients can be modified accordingly.

[0141] As shown in Fig. 8, the encoder 101 may be configured to provide an encoded audio signal 120. Thereby, the encoder 101 may be configured to include in the encoded audio signal 120 a quantized version of the prediction error for a plurality of individual spectral coefficients 106_t0_f2 or for a group of spectral coefficients 106_t0_f4 and 106_t0_f5 to which predictive coding is applied. Furthermore, the encoder 101 may also be configured to not include in the encoded audio signal 120 the prediction coefficients 112_f2 to 114_f5.

[0142] In this way, the encoder 101 can use only the prediction coefficients 112_f2 to 114_f5 for calculating a group of a plurality of predicted (or predicted) individual spectral coefficients 110_t0_f2 or predicted spectral coefficients 110_t0_f4, 110_t0_f5 and therefrom for calculating the prediction error between the predicted individual spectral coefficients 110_t0_f2 or the group of predicted spectral coefficients 110_t0_f4 and 110_t0_f5 and the individual spectral coefficients 106_t0_f2 or the group of predicted spectral coefficients 110_t0_f4 and 110_t0_f5 of the current frame. However, the encoder 101 would not provide either the individual spectral coefficient 106_t0_f4 (or a quantized version thereof) or the group of spectral coefficients 106_t0_f4 and 106_t0_f5 (or a quantized version thereof) and the prediction coefficients 112_f2 to 114_f5 in the coded audio signal 120. Thus, an example decoder, described below with respect to Figure 11, may derive the prediction coefficients 112_f2 to 114_f5 for computing a number of predicted individual spectral coefficients or a group of predicted spectral coefficients for the current frame from the interval value.

[0143] In other words, the encoder 101 may be configured to provide an encoded audio signal 120 comprising a quantized version of the prediction error instead of a quantized version of the group of individual spectral coefficients 106_t0_f2 or the group of spectral coefficients 106_t0_f4 and 106_t0_f5 to which predictive coding is applied.

[0144] Furthermore, the encoder 101 may be configured to provide an encoded audio signal 102 comprising a quantized version of the spectral coefficient 106_t0_f3 from which a plurality of individual spectral coefficients 106_t0_f2 or a group of spectral coefficients 106_t0_f4 and 106_t0_f5 have been separated, such that there are alternating spectral coefficients 106_t0_f2 or groups of spectral coefficients 106_t0_f4 and 106_t0_f5 for which a quantized version of the prediction error is included in the encoded audio signal 120 and spectral coefficients 106_t0_f3 or groups of spectral coefficients for which a quantized version is provided without using predictive coding.

[0145] In an example, the encoder 101 may be further configured to entropy code a quantized version of the prediction error and a quantized version of the plurality of individual spectral coefficients 106_t0_f2 or the spectral coefficients 106_t0_f3 from which the groups of spectral coefficients 106_t0_f4 and 106_t0_f5 have been separated, and to include the entropy coded version in the encoded audio signal 120 (rather than a non-entropy coded version thereof).

[0146] In an embodiment, the encoder 101 may be configured to select, for predictive coding, groups 116_1 to 116_6 of spectral coefficients (or individual spectral coefficients) that are spectrally arranged according to a harmonic grid defined by spacing values, whereby the harmonic grid defined by spacing values ​​describes a periodic spectral distribution (equidistant spacing) of harmonics in the audio signal 102. In other words, the harmonic grid defined by spacing values ​​may be a sequence of spacing values ​​that describes equidistant spacing of harmonics of the audio signal.

[0147] Further, for predictive coding, the encoder 101 can be configured to select spectral coefficients (e.g., only those spectral coefficients) where the spectral index is equal to or within a (e.g., predetermined or variable) range of multiple spectral indices derived based on the interval value.

[0148] From the interval value, an index (or number) of a spectral coefficient representing a harmonic of the audio signal 102 can be derived. For example, assuming that the fourth spectral coefficient 106_t0_f4 represents the instantaneous fundamental frequency of the audio signal 102 and the interval value is 5, a spectral coefficient having index 9 can be derived based on the interval value. The spectral coefficient having index 9 derived in this way, i.e., the ninth spectral coefficient 106_t0_f9, represents the second harmonic. Similarly, spectral coefficients having indexes 14, 19, 24, and 29 can be derived, representing the third to sixth harmonics 124_3 to 124_6. However, not only the spectral coefficients having indexes equal to the multiple spectral indexes derived based on the interval value, but also the spectral coefficients having indexes within a predetermined range around the multiple spectral indexes derived based on the interval value can be predictively coded.

[0149] Furthermore, the encoder 101 may be configured to select the groups 116_1 to 116_6 of spectral coefficients (or individual spectral coefficients) to which predictive coding is applied, such that they alternate periodically with a tolerance of ±1 spectral coefficient between the groups 116_1 to 116_6 of spectral coefficients (or individual spectral coefficients) to which predictive coding is applied and the spectral coefficients to which the groups of spectral coefficients (or individual spectral coefficients) to which predictive coding is applied are separated. A tolerance of ±1 spectral coefficient may be required when the distance between two harmonics of the audio signal 102 is not equal to an integer interval value (an integer in terms of index or number of spectral coefficients) but a fraction or multiple thereof.

[0150] In other words, the audio signal 102 may include at least two harmonic signal components 124_1-124_6, and the encoder 101 may be configured to selectively apply predictive coding to a plurality of groups 116_1-116_6 of spectral coefficients (or individual spectral coefficients) representative of the at least two harmonic signal components 124_1-124_6 or of a spectral environment surrounding the at least two harmonic signal components 124_1-124_6 of the audio signal 102. The spectral environment surrounding the at least two harmonic signal components 124_1-124_6 may be, for example, ±1, 2, 3, 4 or 5 spectral components.

[0151] Thereby, the encoder 101 can be configured to not apply predictive coding to those groups 118_1 to 118_5 of spectral coefficients (or individual spectral coefficients) that do not represent the at least two harmonic signal components 124_1 to 124_6 or the spectral environment of the at least two harmonic signal components 124_1 to 124_6 of the audio signal 102. In other words, the encoder 101 can be configured to not apply predictive coding to those groups 118_1 to 118_5 of spectral coefficients (or individual spectral coefficients) that belong to non-harmonic background noise between the signal harmonics 124_1 to 124_6.

[0152] Furthermore, the encoder 101 can be configured to determine a harmonic spacing value indicative of a spectral spacing between the at least two harmonic signal components 124_1 to 124_6 of the audio signal 102, the harmonic spacing value being indicative of a plurality of individual spectral coefficients or groups of spectral coefficients representing the at least two harmonic signal components 124_1 to 124_6 of the audio signal 102.

[0153] Furthermore, the encoder 101 may be configured to provide the encoded audio signal 120 such that the encoded audio signal 120 includes interval values ​​(e.g., one interval value per frame) or (alternatively) parameters from which the interval values ​​can be directly derived.

[0154] The embodiment addresses the above two problems of the FDP method by introducing a harmonic spacing value into the FDP process, which is signaled from the encoder (transmitter) 101 to the respective decoder (receiver) so that both can operate in perfect synchronization. The harmonic spacing value serves as an indication of the instantaneous fundamental frequency (or pitch) of one or more spectra associated with the frame to be encoded, and can identify which spectral bins (spectral coefficients) have to be predicted. More specifically, only the spectral coefficients around harmonic signal components located at integer multiples of the fundamental pitch (with respect to their index) (as defined by the harmonic spacing value) shall be considered for prediction.

[0155] Fig. 9 is a schematic block diagram of a decoder 201 for decoding an encoded signal 120 of an FDP prediction concept according to an embodiment. The decoder 201 is configured for decoding the encoded audio signal 120 in a transform or filter bank domain 204, the decoder 201 is configured for analysing the encoded audio signal 120 to obtain coded spectral coefficients 206_t0_f1 to 206_t0_f6 of the audio signal for a current frame 208_t0 and coded spectral coefficients 206_t-1_f0 to 206_t-1_f6 for at least one previous frame 208_t-1, the decoder 201 is configured for selectively applying predictive decoding to a plurality of individual coded spectral coefficients or groups of coded spectral coefficients separated by at least one coded spectral coefficient.

[0156] In an example, the decoder 201 may be configured to apply predictive decoding to a plurality of individual coded spectral coefficients separated by at least one coded spectral coefficient, e.g., two individual coded spectral coefficients separated by at least one coded spectral coefficient. Furthermore, the decoder 201 may be configured to apply predictive decoding to a plurality of groups of coded spectral coefficients (each of the groups including at least two coded spectral coefficients) separated by at least one coded spectral coefficient, e.g., two groups of coded spectral coefficients separated by at least one coded spectral coefficient. Furthermore, the decoder 201 may be configured to apply predictive decoding to a plurality of individual coded spectral coefficients and / or groups of coded spectral coefficients separated by at least one coded spectral coefficient, e.g., at least one individual coded spectral coefficient and at least one group of coded spectral coefficients separated by at least one coded spectral coefficient.

[0157] In the example shown in Fig. 9, the decoder 201 is configured to determine six coded spectral coefficients 206_t0_f1 to 206_t0_f6 for the current frame 208_t0 and six coded spectral coefficients 206_t-1_f1 to 206_t-1_f6 for the previous frame 208_t-1, whereby the decoder 201 is configured to selectively apply predictive decoding to an individual second coded spectral coefficient 206_t0_f2 of the current frame and to a group of coded spectral coefficients consisting of the fourth and fifth coded spectral coefficients 206_t0_f4 and 206_t0_f5 of the current frame 208_t0. As can be seen, the groups of coded spectral coefficients consisting of the individual second coded spectral coefficient 206_t0_f2 and the fourth and fifth coded spectral coefficients 206_t0_f4 and 206_t0_f5 are separated from each other by the third coded spectral coefficient 206_t0_f3.

[0158] It should be noted that the term "selectively" as used herein refers to applying predictive decoding (only) to selected coded spectral coefficients. In other words, predictive decoding is not applied to all coded spectral coefficients, but only to selected individual coded spectral coefficients or groups of coded spectral coefficients, the selected individual coded spectral coefficients and / or groups of coded spectral coefficients being separated from each other by at least one coded spectral coefficient. In other words, predictive decoding is not applied to at least one coded spectral coefficient separating a selected number of individual coded spectral coefficients or groups of coded spectral coefficients.

[0159] In an example, the decoder 201 may be configured to not apply predictive decoding to the individual coded spectral coefficient 206_t0_f2 or to at least one coded spectral coefficient 206_t0_f3 separating the groups of spectral coefficients 206_t0_f4 and 206_t0_f5.

[0160] The decoder 201 may be configured to entropy decode the coded spectral coefficients to obtain quantized prediction errors for the spectral coefficients 206_t0_f2, 2016_t0_f4 and 206_t0_f5 to which predictive decoding is to be applied and a quantized spectral coefficient 206_t0_f3 for at least one spectral coefficient to which predictive decoding is not applied, whereby the decoder 201 may be configured to apply the quantized prediction errors to a plurality of predicted individual spectral coefficients 210_t0_f2 or a group of predicted spectral coefficients 210_t0_f4 and 210_t0_f5 to obtain, for the current frame 208_t0, decoded spectral coefficients associated to the coded spectral coefficients 206_t0_f2, 206_t0_f4 and 206_t0_f5 to which predictive decoding is applied.

[0161] For example, the decoder 201 may be configured to obtain a second quantized prediction error for the second quantized spectral coefficient 206_t0_f2, apply the second quantized prediction error to the predicted second spectral coefficient 210_t0_f2, and obtain a second decoded spectral coefficient associated with the second coded spectral coefficient 206_t0_f2. Here, the decoder 201 may be configured to obtain a fourth quantized prediction error for the fourth quantized spectral coefficient 206_t0_f4, and apply the fourth quantized prediction error to the predicted fourth spectral coefficient 210_t0_f4, and obtain a fourth decoded spectral coefficient associated with the fourth coded spectral coefficient 206_t0_f4. Here, the decoder 201 may be configured to obtain a fifth quantized prediction error for the fifth quantized spectral coefficient 206_t0_f5, apply the fifth quantized prediction error to the predicted fifth spectral coefficient 210_t0_f5, and obtain a fifth decoded spectral coefficient associated with the fifth encoded spectral coefficient 206_t0_f5.

[0162] Furthermore, the decoder 201 may be configured to determine a plurality of predicted individual spectral coefficients 210_t0_f2 or a group of predicted spectral coefficients 210_t0_f4 and 210_t0_f5 for the current frame 208_t0 based on a corresponding plurality of individual encoded spectral coefficients 206_t-1_f2 (e.g. using a plurality of previously decoded spectral coefficients associated with the plurality of individual encoded spectral coefficients 206_t-1_f2) or based on a group of encoded spectral coefficients 206_t-1_f4 and 206_t-1_f5 (e.g. using a group of previously decoded spectral coefficients associated with the group of encoded spectral coefficients 206_t-1_f4 and 206_t-1_f5) of the previous frame 208_t-1.

[0163] For example, the decoder 201 may be configured to determine a second predicted spectral coefficient 210_t0_f2 of the current frame 208_t0 using a previously decoded (quantized) second spectral coefficient associated with the second coded spectral coefficient 206_t-1_f2 of the previous frame 208_t-1, to determine a fourth predicted spectral coefficient 210_t0_f4 of the current frame 208_t0 using a previously decoded (quantized) fourth spectral coefficient associated with the fourth coded spectral coefficient 206_t-1_f4 of the previous frame 208_t-1, and to determine a fifth predicted spectral coefficient 210_t0_f5 of the current frame 208_t0 using a previously decoded (quantized) fifth spectral coefficient associated with the fifth coded spectral coefficient 206_t-1_f5 of the previous frame 208_t-1.

[0164] Further, the decoder 201 can be configured to derive prediction coefficients from the interval values, and the decoder 201 can be configured to calculate a plurality of predicted individual spectral coefficients 210_t0_f2 or a group of predicted spectral coefficients 210_t0_f4 and 210_t0_f5 for the current frame 208_t0 using a corresponding plurality of previously decoded individual spectral coefficients or groups of previously decoded spectral coefficients of the at least two previous frames 208_t-1 and 208_t-2 and using the derived prediction coefficients.

[0165] For example, the decoder 201 may be configured to derive prediction coefficients 212_f2 and 214_f2 for the second coded spectral coefficient 206_t0_f2 from the interval value, derive prediction coefficients 212_f4 and 214_f4 for the fourth coded spectral coefficient 206_t0_f4 from the interval value, and derive prediction coefficients 212_f5 and 214_f5 for the fifth coded spectral coefficient 206_t0_f5 from the interval value.

[0166] It should be noted that the decoder 201 may be configured to decode the encoded audio signal 120 in order to obtain a quantized prediction error instead of a plurality of individual quantized spectral coefficients or groups of quantized spectral coefficients for which predictive decoding is applied.

[0167] Furthermore, the decoder 201 may be configured to decode the encoded audio signal 120 to obtain quantized spectral coefficients separating a plurality of individual spectral coefficients or groups of spectral coefficients, such that there is an alternation between the coded spectral coefficients 206_t0_f2 or a group of coded spectral coefficients 206_t0_f4, 206_t0_f5 from which the quantized prediction error is obtained and the coded spectral coefficients 206_t0_f3 or a group of coded spectral coefficients from which the quantized spectral coefficient is obtained.

[0168] The decoder 201 may be configured to provide a decoded audio signal 220 using decoded spectral coefficients associated with the coded spectral coefficients 206_t0_f2, 206_t0_f4 and 206_t0_f5 to which predictive decoding is applied, and using entropy decoded spectral coefficients associated with the coded spectral coefficients 206_t0_f1, 206_t0_f3 and 206_t0_f6 to which predictive decoding is not applied.

[0169] In an example, the decoder 201 may be configured to obtain an interval value, and the decoder 201 may be configured to select, based on the interval value, a plurality of individual encoded spectral coefficients 206_t0_f2 or a group of encoded spectral coefficients 206_t0_f4 and 206_t0_f5 to which predictive decoding is applied.

[0170] As already mentioned above with respect to the description of the corresponding encoder 101, the spacing value can for example be the spacing (or distance) between two characteristic frequencies of the audio signal. Furthermore, the spacing value can be an integer value of a spectral coefficient (or an index of a spectral coefficient) that approximates the spacing between two characteristic frequencies of the audio signal. Of course, the spacing value can also be a fraction or a multiple of the integer value of the spectral coefficient describing the spacing between two characteristic frequencies of the audio signal.

[0171] The decoder 201 may be configured to select, for predictive decoding, individual spectral coefficients or groups of spectral coefficients that are spectrally arranged according to a harmonic grid defined by spacing values. The harmonic grid defined by spacing values ​​may describe a periodic spectral distribution (equidistant spacing) of harmonics in the audio signal 102. In other words, the harmonic grid defined by spacing values ​​may be a sequence of spacing values ​​that describe equidistant spacing of harmonics of the audio signal 102.

[0172] Furthermore, the decoder 201 may be configured to select, for predictive decoding, spectral coefficients (e.g., only those spectral coefficients) whose spectral indexes are equal to or within a range (e.g., a predetermined range or a variable range) around a plurality of spectral indexes derived based on the interval value, whereby the decoder 201 may be configured to set the width of the range depending on the interval value.

[0173] In an example, the encoded audio signal may include the interval value or a coded version thereof (e.g. a parameter from which the interval value can be directly derived), and the decoder 201 may be configured to extract the interval value or a coded version thereof from the encoded audio signal to obtain the interval value.

[0174] Alternatively, the decoder 201 may be configured to determine the interval values ​​itself, i.e. the encoded audio signal does not include the interval values, in which case the decoder 201 may be configured to determine the instantaneous fundamental frequency (of the encoded audio signal 120 representing the audio signal 102) and derive the interval values ​​from the instantaneous fundamental frequency or a fraction or multiple thereof.

[0175] In an example, the decoder 201 may be configured to select a plurality of individual spectral coefficients or groups of spectral coefficients to which predictive decoding is applied such that there is a periodic alternation, with a tolerance of ±1 spectral coefficient, between the plurality of individual spectral coefficients or groups of spectral coefficients to which predictive decoding is applied and the spectral coefficients separating the plurality of individual spectral coefficients or groups of spectral coefficients to which predictive decoding is applied.

[0176] In an example, the audio signal 102 represented by the encoded audio signal 120 comprises at least two harmonic signal components and the decoder 201 is configured to selectively apply predictive decoding to those plurality of individual coded spectral coefficients 206_t0_f2 or a group of coded spectral coefficients 206_t0_f4 and 206_t0_f5 representing the at least two harmonic signal components or the spectral environment around the at least two harmonic signal components of the audio signal 102. The spectral environment around the at least two harmonic signal components may for example be ±1, 2, 3, 4 or 5 spectral components.

[0177] Thereby, the decoder 201 can be configured to identify at least two harmonic signal components and to selectively apply predictive decoding to a group of those plurality of individual encoded spectral coefficients 206_t0_f2 or encoded spectral coefficients 206_t0_f4 and 206_t0_f5 that are associated with the identified harmonic signal component (e.g., represent or surround the identified harmonic signal component).

[0178] Alternatively, the encoded audio signal 120 may include information (e.g., spacing values) identifying at least two harmonic signal components. In that case, the decoder 201 may be configured to selectively apply predictive decoding to those multiple individual encoded spectral coefficients 206_t0_f2 or the group of encoded spectral coefficients 206_t0_f4 and 206_t0_f5 that are associated with (e.g., represent or surround) the identified harmonic signal component.

[0179] In both of the above alternatives, the decoder 201 may be configured to not apply predictive decoding to those plurality of individual coded spectral coefficients 206_t0_f3, 206_t0_f1, 206_t0_f6 or groups of coded spectral coefficients that do not represent the spectral environment of the at least two harmonic signal components or the at least two harmonic signal components of the audio signal 102.

[0180] In other words, the decoder 201 can be configured not to apply predictive decoding to those individual coded spectral coefficients 206_t0_f3, 206_t0_f1, 206_t0_f6 or groups of coded spectral coefficients that belong to non-harmonic background noise between signal harmonics of the audio signal 102.

[0181] The idea of ​​a particular embodiment is now to provide an encoder and a decoder having two different modes of operation.

[0182] According to an embodiment, the encoder 100 may be operable, for example, in a first mode and may be operable, for example, in at least one of a second mode, a third mode and a fourth mode.

[0183] When the encoder 100 is in a first mode, the encoder 100 may be configured to encode the current frame by, for example, determining an estimate of two harmonic parameters for each of one or more harmonic components of the most previous frame using a first group of three or more spectral coefficients of each of the plurality of spectral coefficients of one or more previous frames of the audio signal.

[0184] When the encoder 100 is in the second mode, the encoder 100 may be configured to, for example, encode the audio signal in a transform domain or a filter bank domain, and the encoder may be configured to, for example, determine a plurality of spectral coefficients 106_t0_f1:106_t0_f6; 106_t-1_f1:106_t-1_f6 of the audio signal 102 for a current frame 108_t0 and at least a previous frame 108_t-1, and the encoder 100 may, for example, determine a plurality of individual spectral coefficients 106_t0_f1:106_t0_f6; 106_t-1_f1:106_t-1_f6 of the audio signal 102 for a current frame 108_t0 and at least a previous frame 108_t-1. The encoder 100 may be configured to selectively apply predictive coding to a group of individual spectral coefficients 106_t0_f2 or spectral coefficients 106_t0_f4, 106_t0_f5, for example, and the encoder 100 may be configured to, for example, determine a spacing value, and the encoder 100 may be configured to, for example, select a group of a plurality of individual spectral coefficients 106_t0_f2 or spectral coefficients 106_t0_f4, 106_t0_f5 to which predictive coding may be applied, for example based on the spacing value.

[0185] In an embodiment, in each of the first, second, third and fourth modes, the encoder 100 may be configured to, for example, refine the fundamental frequency to obtain a refined fundamental frequency and adapt the gain factors on a frame-by-frame basis according to a minimization criterion to obtain adapted gain factors. Furthermore, the encoder 100 may be configured to, for example, encode the refined fundamental frequency and the adapted gain factors instead of the original fundamental frequency and gain factors.

[0186] In an embodiment, the encoder 100 may be configured to set itself to the first mode or to at least one of the second mode, the third mode and the fourth mode, for example, depending on a current frame of the audio signal. The encoder 100 may be configured to encode whether the current frame was encoded in the first mode, the second mode, the third mode or the fourth mode.

[0187] With regard to the decoder, according to an embodiment, the decoder 200 may be operable, for example, in a first mode, and may be operable, for example, in at least one of a second mode, a third mode, and a fourth mode.

[0188] When the decoder 200 is in a first mode, the decoder 200 may, for example, be configured to determine an estimate of two harmonic parameters for each of the one or more harmonic components of the most recent frame, where the two harmonic parameters for each of the one or more harmonic components of the most recent frame depend on a first group of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames of the audio signal, and the decoder 200 may, for example, be configured to decode an encoding of the current frame in response to the estimate of the two harmonic parameters for each of the one or more harmonic components of the most recent frame.

[0189] When the decoder 200 is in the second mode, the decoder 200 may be configured to analyze the encoding of the audio signal 120 to obtain, for example, the coded spectral coefficients 206_t0_f1:206_t0_f6; 206_t-1_f1:206_t-1_f6 of the audio signal 120 for the current frame 208_t0 and at least the previous frame 208_t-1, and the decoder 200 may, for example, analyze the coding of the audio signal 120 to obtain, for example, the coded spectral coefficients 206_t0_f2 or the coded spectral coefficients 206_t0_f3 and 206_t-1_f4 and 206_t-1_f5 and 206_t-1_f6 of the audio signal 120 for the current frame 208_t0 and at least the previous frame 208_t-1. The decoder 200 may be configured to selectively apply predictive decoding to a group of individually coded spectral coefficients 206_t0_f2 or 206_t0_f4, 206_t0_f5, and the decoder 200 may be configured, for example, to obtain a spacing value, and the decoder 200 may be configured, for example, to select a plurality of individually coded spectral coefficients 206_t0_f2 or a group of coded spectral coefficients 206_t0_f4, 206_t0_f5 to which predictive decoding may be applied based on the spacing value.

[0190] When the decoder 200 is in the third mode, the decoder 200 may be configured to decode the audio signal by employing, for example, Time Domain Long-term Prediction.

[0191] When the decoder 200 is in the fourth mode, the decoder 200 may, for example, employ Adaptive Modified Discrete Cosine Transform Long-Term Prediction to decode the audio signal, and when the decoder 200 employs Adaptive Modified Discrete Cosine Transform Long-Term Prediction, the decoder 200 may be configured to select, for example, either Time Domain Long-Term Prediction or Frequency Domain Prediction or Frequency Domain Least Mean Square Prediction as a prediction method on a frame-by-frame basis depending on a minimization criterion.

[0192] According to an embodiment, in each of the first, second, third and fourth modes, the decoder 200 may be configured to decode the audio signal in response to a refined fundamental frequency, for example determined on a frame basis, and in response to an adapted gain factor.

[0193] In an embodiment, decoder 200 may, for example, receive and decode an encoding that includes an indication of whether a current frame was encoded in a first mode, a second mode, a third mode, or a fourth mode, and decoder 200 may, for example, configure itself to the first mode, the second mode, the third mode, or the fourth mode in response to the indication.

[0194] In Fig. 5, it can be seen that as the frame length increases, the BS of all three concepts drops significantly for pipe notes, since the redundancy of the original signal has been largely removed by the transform itself. FDP's performance drops significantly for low-pitched bass notes, due to the large overlap of MDCT coefficient harmonics. TDLTP's performance is generally good. However, for large frame lengths, a large delay is required to find the matching previous pitch period, which reduces the performance. FDLMSP provides relatively good and stable performance for different notes and different frame lengths. Also in Fig. 5, the BS drops when the prediction bandwidth is increased to 8 kHz, which is due to the presence of inharmonicity in the high frequency band of sounds. Since the inharmonicity depends on the spectral characteristics of the individual sound material, the bitrate consumption can be pre-calculated and compared band by band to obtain a higher coding efficiency. A prediction decision can then be made and signaled in each frame as side information.

[0195] Figure 6 shows the bit rates saved in four different working modes for six different items with bandwidth limited to 4 kHz and MDCT frame lengths of 64 and 512.

[0196] As shown in Fig. 6, FDLMSP outperforms TDLTP and FDP in many scenarios and generally performs well. AMLTP performs the best, choosing between FDLMSP and TDLTP in most cases, indicating that combining FDLMSP with TDLTP can significantly enhance BS.

[0197] A new approach for LTP in the MDCT domain is presented. This new approach models each MDCT frame as a hypothesis of harmonic content and estimates the parameters of all harmonic content from the previous frame using the LMS concept. Then, prediction is made based on the estimated harmonic parameters. This approach provides competitive performance compared to similar concepts and can also be jointly used to increase the efficiency of speech coding.

[0198] The above concepts can be employed to analyze the influence of the accuracy of the pitch information on the prediction, for example by using different pitch estimation algorithms or by applying different quantization step sizes. The above concepts can also be employed to determine the pitch information of an audio signal on a frame-by-frame basis using a minimization criterion or for refinement processing. The influence of inharmonicity and other complex signal characteristics on the prediction may, for example, be taken into account. The above concepts can be employed for error concealment, for example.

[0199] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of the corresponding method, where a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of the corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0200] Depending on the particular implementation requirements, embodiments of the present invention can be implemented in hardware or in software, or at least partly in hardware or at least partly in software. The implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM or FLASH memory, having electronically readable control signals stored thereon, which cooperates (or can cooperate) with a programmable computer system such that the respective method is performed. The digital storage medium may therefore be computer readable.

[0201] Some embodiments according to the invention include a data carrier having an electronically readable control signal, the data carrier being capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0202] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product is run on a computer. The program code can, for example, be stored on a machine-readable carrier.

[0203] A further embodiment comprises the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0204] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0205] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transitory.

[0206] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can for example be adapted to be transmitted via a data communication connection, for example the Internet.

[0207] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0208] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0209] Further embodiments according to the invention include an apparatus or system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0210] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0211] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0212] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0213] The above-described embodiments merely exemplify the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims, and not by the specific details presented in the description and explanation of the embodiments herein.

[0214] References [1] Jurgen Herre and Sascha Dick, "Psychoacoustic models for perceptual audio coding a tutorial review," Applied Sciences, vol. 9, pp. 2854, ITT 2019. [2] Juha Ojanpera, Mauri Vaananen, and Lin Yin, "Long Term Predictor for Transf orm Domain Perceptual Audio Coding," in Audio Engineering Society Convention 107, Sep 1999. [3] Hendrik Fuchs, "Improving mpeg audio coding by backward adaptive linear ste reo prediction," in Audio Engineering Society Convention 99, Oct 1995. [4] J. Princen, A. Johnson, and A. Bradley, "Subband / transform coding using fil ter bank designs based on time domain aliasing cancellation," in ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing, April 1987, vol. 12, pp. 2161-2164. [5] Christian Helmrich, Efficient Perceptual Audio Coding Using Cosine and Sine Modulated Lapped Transforms, doctoral thesis, Friedrich-Alexander-Universit at Erlangen-Nurnberg (FAU), 2017, Chapter 3.3: Frequency-Domain Prediction w ith Very Low Complexity. [6] J. Rothweiler, "Polyphase quadrature filters-a new subband coding technique ," in ICASSP '83. IEEE International C01iference on Acoustics, Speech, and S ignal Processing, April 1983, vol. 8, pp. 1280--1283. [7] Albrecht Schneider and Klaus Frieler, "Perception of harmonic and inharmoni c sounds: Results from ear models;· in Computer Music Modeling and Retrieva l. Genesis of Meaning in Sound and Music, Solvi Ystad, Richard Kronland-Mart inet, and Kristoffer Jensen, Eds., Berlin, Heidelberg, 2009, pp. 18-44, Spri nger Berlin Heidelberg. [8] Hugo Fast! and Eberhard Zwicker, Psychoacoustics: Facts and Models, Springe r-Verlag, Berlin, Heidelberg, 2006, Chapter 7.2: Just-Noticeable Changes in Frequency. [9] John P. Princen and Alan Bernard Bradley, "Analysis / synthesis filter bank d esign based on time domain aliasing cancellation," IEEE Transactions on Acou stics, Speech, and Signal Processing, vol. 34, no. 5, pp. 1153-1161, October 1986.

[10] Alain de Cheveign and Hideki Kawahara, "Yin, a fundamental frequency e stimator for speech and music;· The Journal of the Acoustical Society of Am erica, vol. 111, pp. 1917-30, 05 2002.

[11] Armin Taghipour, Psychoacoustics of detection of tonality and asymmetr y of masking: implementation of tonality estimation methods in a psychoacous tic model for perceptual audio coding, doctoral thesis, Friedrich-Alexander- Universitat Erlangen-Nurnberg (FAU), 2016, Chapter 4: The Psychoacoustic mod el.

[12] J. D. Johnston, "Estimation of perceptual entropy using noise masking criteria," in ICASSP-88? International Conference on Acoustics, Speech, an d Signal Processing, April 1988, pp. 2524--2527 vol.5.

[13] WO 2016 142357A1, published September 2016.

Claims

1. 1. An encoder (100) for generating an encoding of one or more harmonic components of a current frame of an audio signal in response to one or more harmonic components of each frame of one or more previous frames of the audio signal, the one or more previous frames preceding the current frame, the current frame and the one or more previous frames each comprising a plurality of spectral coefficients in a frequency domain or a transform domain; To generate an encoded version of the one or more harmonic components of the current frame, the encoder (100) determines, for each of the one or more harmonic components of an earlier frame of the one or more previous frames, an estimate of two harmonic parameters that depend on that harmonic component of the one or more harmonic components, the encoder (100) determining the estimate of the two harmonic parameters for each of the one or more harmonic components of the earlier frame using a first group of three or more spectral coefficients of the plurality of spectral coefficients of each of the one or more previous frames of the audio signal, the encoder (100) determines a gain factor and a residual signal as an encoding of the one or more harmonic components of the current frame in response to fundamental frequencies of the one or more harmonic components of the current frame and the one or more previous frames and in response to the estimates of the two harmonic parameters for each of the one or more harmonic components of the most previous frame; the encoder (100) generates an encoding of the one or more harmonic components of the current frame such that the encoding of the one or more harmonic components of the current frame includes the gain factor and the residual signal; the two harmonic parameters for each of the one or more harmonic components being a first parameter for a cosine subcomponent and a second parameter for a sine subcomponent of each of the one or more harmonic components; The encoder (100) estimates the two harmonic parameters for each of the one or more harmonic components of the most previous frame by solving a linear equation system including at least three equations, each of the at least three equations depending on the first group of spectral coefficients consisting of the three or more spectral coefficients of the plurality of spectral coefficients of each of the one or more previous frames.

2. 2. The encoder of claim 1, wherein the encoder estimates the two harmonic parameters for each of the one or more harmonic components of the most recent frame without using a second group of one or more other spectral coefficients of each of the plurality of spectral coefficients of the one or more previous frames.

3. 3. The encoder (100) of claim 1 or 2, wherein the encoder (100) determines an estimate of the two harmonic parameters for each of the one or more harmonic components of the current frame depending on the estimate of the two harmonic parameters for each of the one or more harmonic components of the most previous frame and depending on the fundamental frequencies of the one or more harmonic components of the current frame and the one or more previous frames.

4. The encoder (100) of any of claims 1 to 3, wherein the encoder (100) solves the system of linear equations using a least mean squares algorithm.

5.

6. The encoder (100) of claim 5, wherein r≧1.

7.

8.

9. The encoder (100) of claim 8, wherein the encoder (100) encodes a fundamental frequency of a harmonic component, a window function, the gain factor, and the residual signal.

10. 10. The encoder of claim 9, further comprising: a first group of three or more spectral coefficients of each of the plurality of spectral coefficients of the one or more previous frames of the audio signal before estimating the two harmonic parameters for each of the one or more harmonic components of the most recent frame using a first group of three or more spectral coefficients of each of the plurality of spectral coefficients of the one or more previous frames of the audio signal.

11. The encoder (100) of claim 10, wherein the encoder (100) determines one or more groups of harmonic components from the one or more harmonic components and applies a prediction of the speech signal to the one or more groups of harmonic components, wherein the encoder (100) encodes an order for each of the one or more groups of harmonic components of the most recent frame.

12. The encoder (100) of claim 11, wherein the encoder (100) determines the two harmonic parameters for each of the one or more harmonic components of the current frame in response to the two harmonic parameters for each of the one or more harmonic components of the previous frame.

13.

14. the encoder (100) determines the residual signal in response to the plurality of spectral coefficients of the current frame in the frequency domain or the transform domain and in response to the estimates of the two harmonic parameters for each of one or more harmonic components of the current frame; and The encoder (100) encodes the residual signal. An encoder (100) according to any of claims 11 to 13.

15. determining a spectral prediction of one or more of the plurality of spectral coefficients of the current frame in response to the estimates of the two harmonic parameters for each of the one or more harmonic components of the current frame; and the encoder (100) determines the residual signal and a gain factor in response to the plurality of spectral coefficients of the current frame in the frequency domain or the transform domain and in response to the spectral prediction of the three or more of the plurality of spectral coefficients of the current frame, and the encoder (100) encodes the order for each of the group of one or more harmonic components of the most recent frame. The encoder (100) of claim 14.

16.

17. 1. A decoder (200) for reconstructing one or more harmonic components of a current frame of an audio signal in dependence on one or more harmonic components of each of one or more previous frames of the audio signal, the one or more previous frames of the audio signal preceding the current frame, the current frame and each of the one or more previous frames comprising a plurality of spectral coefficients in a frequency domain or a transform domain; The decoder (200) receives an encoding of one or more harmonic components of a current frame of the audio signal that is dependent on one or more harmonic components of each of one or more previous frames of the audio signal; the decoder (200) determines an estimate of two harmonic parameters for each of the one or more harmonic components of an earlier frame of the one or more previous frames, the two harmonic parameters for each of the one or more harmonic components of the earlier frame being dependent on a first group of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames of the audio signal; 1. A decoder (200) for reconstructing the one or more harmonic components of the current frame of the speech signal in response to an encoding of the one or more harmonic components of the current frame and in response to the estimates of the two harmonic parameters for each of the one or more harmonic components of the previous frame, the decoder comprising: The decoder (200) receives an encoding of the one or more harmonic components of the current frame, the encoding including a gain factor and a residual signal; the decoder (200) reconstructs the current frame in response to the gain factor, in response to the residual signal, and in response to fundamental frequencies of the one or more harmonic components of the current frame and one or more previous frames; the two harmonic parameters for each of the one or more harmonic components being a first parameter for a cosine subcomponent and a second parameter for a sine subcomponent of each of the one or more harmonic components; the two harmonic parameters for each of the one or more harmonic components of the most previous frame depend on a system of linear equations including at least three equations, each of the at least three equations depending on a first group of spectral coefficients consisting of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames; A decoder (200).

18. 20. The decoder (200) of claim 17, wherein the two harmonic parameters for each of the one or more harmonic components of the most recent frame are independent of a second group of one or more other spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames.

19. The decoder (200) of claim 17 or 18, wherein the decoder (200) determines an estimate of the two harmonic parameters for each of the one or more harmonic components of the current frame depending on the estimate of the two harmonic parameters for each of the one or more harmonic components of the most previous frame and depending on the fundamental frequencies of the one or more harmonic components of the current frame and the one or more previous frames.

20. Decoder (200) according to any of claims 17 to 19, wherein the system of linear equations is solvable using a least mean squares algorithm.

21.

22. 22. The decoder (200) of claim 21, wherein r>1.

23.

24.

25. The decoder (200) receives a fundamental frequency of a harmonic component, a window function, the gain factor, and the residual signal; The decoder (200) according to any one of claims 17 to 24, wherein the decoder (200) reconstructs the current frame in response to the fundamental frequency of the one or more harmonic components of the most recent frame, in response to the window function, in response to the gain factor, and in response to the residual signal.

26. the decoder (200) receives the number of the one or more harmonic components of the most recent frame; and 26. The decoder (200) of claim 25, wherein the decoder (200) decodes an encoding of the one or more harmonic components of the current frame depending on the number of the one or more harmonic components of the most recent frame.

27. the decoder (200) decodes an encoding of the one or more harmonic components of the current frame in response to a group of one or more harmonic components; 27. The decoder (200) of claim 26, wherein the decoder (200) applies a prediction of the audio signal to the group of one or more harmonic components.

28. 28. A decoder (200) as claimed in any one of claims 17 to 27, wherein the decoder (200) determines the two harmonic parameters for each of one or more harmonic components of the current frame depending on the two harmonic parameters for each of the one or more harmonic components of the previous frame.

29.

30. 30. The decoder (200) of any of claims 17 to 29, wherein the decoder (200) receives the residual signal, the residual signal depending on the plurality of spectral coefficients of the current frame in the frequency domain or the transform domain, the residual signal depending on the estimates of the two harmonic parameters for each of one or more harmonic components of the current frame.

31. 31. The decoder of claim 30, wherein the decoder determines a spectral prediction of one or more of the plurality of spectral coefficients of the current frame in response to the estimates of the two harmonic parameters for each of the one or more harmonic components of the current frame, and the decoder determines the current frame of the audio signal in response to the spectral prediction of the current frame, in response to the residual signal, and in response to a gain factor.

32.

33.

34. the apparatus (700) receiving the number of the one or more harmonic components of the most recent frame; 34. The apparatus (700) of claim 33, wherein the apparatus (700) decodes the encoding of the one or more harmonic components of the current frame in response to the number of the one or more harmonic components of the most previous frame and in response to fundamental frequencies of the one or more harmonic components of the current frame and the one or more previous frames.

35. The apparatus (700) of claim 33 or 34, further comprising: a processor (100) for processing the first and second harmonic components of the current frame, the processor (100) being configured to estimate a first harmonic component of the current frame based on the first and second harmonic components of the current frame;

36. 1. A system comprising: An encoder (100) according to any one of claims 1 to 16 for encoding the one or more harmonic components of a current frame of an audio signal; A decoder (200) according to any of claims 17 to 32 for decoding an encoding of the one or more harmonic components of the current frame of the audio signal; A system comprising:

37. 1. A method for generating an encoding of one or more harmonic components of a current frame of an audio signal in response to one or more harmonic components of each of one or more previous frames of the audio signal, the one or more previous frames preceding the current frame, the current frame and each of the one or more previous frames comprising a plurality of spectral coefficients in a frequency domain or a transform domain; The method includes determining estimates of two harmonic parameters for each of the one or more harmonic components of an earlier frame of the one or more previous frames to generate an encoding of the one or more harmonic components of the current frame; determining, for each of the one or more harmonic components of the most previous frame, the estimates of the two harmonic parameters dependent on that harmonic component of the one or more harmonic components is performed using a first group of three or more spectral coefficients of the plurality of spectral coefficients of each of the one or more previous frames of the audio signal; The method includes determining a gain factor and a residual signal as an encoding of the one or more harmonic components of the current frame in response to fundamental frequencies of the one or more harmonic components of the current frame and the one or more previous frames and in response to the estimate of the two harmonic parameters for each of the one or more harmonic components of the most previous frame; generating an encoding of the one or more harmonic components of the current frame is performed such that the encoding of the one or more harmonic components of the current frame includes the gain factor and the residual signal; the two harmonic parameters for each of the one or more harmonic components being a first parameter for a cosine subcomponent and a second parameter for a sine subcomponent of each of the one or more harmonic components; the step of estimating the two harmonic parameters for each of the one or more harmonic components of the most previous frame is performed by solving a system of linear equations including at least three equations, each of the at least three equations depending on the first group of three or more spectral coefficients of the plurality of spectral coefficients of each of the one or more previous frames. method.

38. 1. A method for reconstructing one or more harmonic components of a current frame of an audio signal in response to one or more harmonic components of each of one or more previous frames of the audio signal, the one or more previous frames preceding the current frame, the current frame and each of the one or more previous frames comprising a plurality of spectral coefficients in a frequency domain or a transform domain; The method comprises receiving an encoding of one or more harmonic components of a current frame of an audio signal that is dependent on one or more harmonic components of each of one or more previous frames of the audio signal; The method comprises determining, for each of the one or more harmonic components of an earlier frame of the one or more previous frames, an estimate of two harmonic parameters dependent on that harmonic component of the one or more harmonic components, wherein the two harmonic parameters for each of the one or more harmonic components of the earlier frame are dependent on a first group of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames of the audio signal; The method comprises the steps of reconstructing the one or more harmonic components of the current frame of the audio signal in response to an encoding of the one or more harmonic components of the current frame and in response to the estimates of the two harmonic parameters for each of the one or more harmonic components of the most previous frame, The method includes receiving an encoding of the one or more harmonic components of the current frame including a gain factor and a residual signal; reconstructing the current frame is performed in response to the gain factor, in response to the residual signal, and in response to fundamental frequencies of the one or more harmonic components of the current frame and one or more previous frames; the two harmonic parameters for each of the one or more harmonic components being a first parameter for a cosine subcomponent and a second parameter for a sine subcomponent of each of the one or more harmonic components; the two harmonic parameters for each of the one or more harmonic components of the most previous frame depend on a system of linear equations including at least three equations, each of the at least three equations depending on a first group of spectral coefficients consisting of three or more spectral coefficients of the plurality of reconstructed spectral coefficients for each of the one or more previous frames; method.

39.

40. 40. A computer program for carrying out the method according to any of claims 37 to 39, when the computer program is run by a computer or signal processing device.

Images