Multi-lag formats for audio coding

Abstract

A method for encoding an audio signal is described, the method comprising the steps of generating a plurality of sub-band audio signals based on the audio signal, determining a spectral envelope of the audio signal, and for each sub-band audio signal, determining autocorrelation information for the sub-band audio signal based on an autocorrelation function of the sub-band audio signal, and generating a coded representation of the audio signal, the coded representation comprising a representation of the spectral envelope of the audio signal and a representation of the autocorrelation information for the plurality of sub-band audio signals. Further, methods for decoding the audio signal from the coded representation are described, as well as corresponding encoders, decoders, computer programs and computer-readable media.

Description

[Technical field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. Provisional Patent Application No. 62 / 889,118, filed August 20, 2019, and European Patent Application No. 19192552.8, filed August 20, 2019, the entire contents of each of which are incorporated by reference.

[0002] [Technical field] This disclosure relates generally to methods for encoding an audio signal into a coded representation and for decoding the audio signal from the coded representation.

[0003] Although certain embodiments are described herein with particular reference to that disclosure, it will be recognized that the disclosure is not limited to such fields of use, but is applicable in a broader context. [Background technology]

[0004] Any discussion of background technology throughout this disclosure should in no way be taken as an admission that the technology is widely known or constitutes part of the common general knowledge in the art.

[0005] In high-quality audio coding systems, it is common to have the largest portion of information describe the detailed waveform characteristics of the signal. Smaller portions of information are used to describe more statistically defined features, such as the energy in frequency bands, or control data intended to shape the quantization noise according to known joint masking properties of hearing (e.g., side information in MDCT-based waveform coders that conveys the quantization step size and range information necessary to accurately dequantize the data representing the waveform in the decoder). However, these high-quality audio coding systems require a relatively large amount of data to code the audio content, i.e., they have a relatively low coding efficiency.

[0006] What is needed is an audio coding method and apparatus that can code audio data with improved coding efficiency. Summary of the Invention

[0007] The present disclosure provides a method for encoding an audio signal, a method for decoding an audio signal, an encoder, a decoder, a computer program, and a computer-readable storage medium.

[0008] According to a first aspect of the present disclosure, a method of encoding an audio signal is provided. The encoding may be performed for each of a plurality of consecutive portions (e.g., groups of samples, segments, frames) of the audio signal. In some implementations, the portions may overlap each other. An encoded representation may be generated for each such portion. The method may include generating a plurality of sub-band audio signals based on the audio signal. Generating the plurality of sub-band audio signals based on the audio signal may include a spectral decomposition of the audio signal, which may be performed by a filter bank of bandpass filters (BPFs). The frequency resolution of the filter bank may be related to the frequency resolution of the human auditory system. For example, the BPF may be a complex-valued BPF. Alternatively, generating the plurality of sub-band audio signals based on the audio signal may include spectrally and / or temporally flattening the audio signal, optionally windowing the flattened audio signal by a window function, and spectrally decomposing the resulting signal into a plurality of sub-band audio signals. The method may further include determining a spectral envelope of the audio signal. The method may further include, for each subband audio signal, determining autocorrelation information for the subband audio signal based on an autocorrelation function (ACF) of the subband audio signal. The method may also include generating an encoded representation of the audio signal, the encoded representation including a representation of a spectral envelope of the audio signal and a representation of the autocorrelation information for the plurality of subband audio signals. For example, the encoded representation may relate to a portion of the bitstream. In some implementations, the encoded representation may further include waveform information related to a waveform of the audio signal and / or one or more waveforms of the subband audio signals. The method may further include outputting the encoded representation.

[0009] When configured as above, the proposed method provides a coded representation of the audio signal that has very high coding efficiency (i.e. requires a very low bit rate to code the audio) but at the same time contains adequate information to achieve very good sound quality after reconstruction. This is done by providing, in addition to the spectral envelope, also autocorrelation information for multiple subbands of the audio signal. Notably, two values ​​per subband, i.e. one lag value and one autocorrelation value, have been proven to be sufficient to achieve high sound quality.

[0010] In some embodiments, the autocorrelation information for a given subband audio signal may comprise a lag value for the respective subband audio signal and / or an autocorrelation value for the respective subband audio signal. Preferably, the autocorrelation information may comprise both a lag value for the respective subband audio signal and an autocorrelation value for the respective subband audio signal, where the lag value may correspond to a delay value (e.g. abscissa) at which the autocorrelation function reaches a local maximum and the autocorrelation value may correspond to this local maximum (e.g. ordinate).

[0011] In some embodiments, the spectral envelope may be determined at a first update rate and the autocorrelation information for the plurality of subband audio signals may be determined at a second update rate. In this case, the first update rate and the second update rate may be different from each other. The update rate may also be referred to as a sampling rate. In such an embodiment, the first update rate may be higher than the second update rate. Furthermore, different update rates may be applied to different subbands, i.e. the update rates of the autocorrelation information for different subband audio signals may be different from each other.

[0012] By reducing the update rate of the autocorrelation information compared to the update rate of the spectral envelope, the coding efficiency of the proposed method can be further improved without affecting the sound quality of the reconstructed audio signal.

[0013] In some embodiments, generating the plurality of subband audio signals may include applying a spectral and / or temporal flattening to the audio signal. Generating the plurality of subband audio signals may further include windowing the flattened audio signal with a window function. Also, generating the plurality of subband audio signals may further include spectrally decomposing the windowed flattened audio signal into the plurality of subband audio signals. In this case, for example, spectrally and / or temporally flattening the audio signal may include generating a perceptually weighted LPC residual of the audio signal.

[0014] In some embodiments, generating a plurality of sub-band audio signals may include spectrally decomposing the audio signal. Then, determining an autocorrelation function for a given sub-band audio signal may include determining a sub-band envelope of the sub-band audio signal. Determining the autocorrelation function may further include envelope flattening the sub-band audio signal based on the sub-band envelope. The sub-band envelope may be determined by taking a magnitude value of the windowed sub-band audio signal. Determining the autocorrelation function may include windowing the envelope-flattened sub-band audio signal with a window function. Also, determining the autocorrelation function may further include determining (e.g., calculating) an autocorrelation function of the envelope-flattened windowed sub-band audio signal. The autocorrelation function may be determined for a real-valued (envelope-flattened windowed) sub-band signal.

[0015] Another aspect of the present disclosure relates to a method for decoding an audio signal from an encoded representation of the audio signal. The encoded representation may include a representation of a spectral envelope of the audio signal and a representation of autocorrelation information for each of a plurality of sub-band audio signals (or ones generated from the audio signal) of the audio signal. The autocorrelation information for a given sub-band audio signal may be based on an autocorrelation function of the sub-band audio signal. The method may include receiving an encoded representation of the audio signal. The method may further include extracting the spectral envelope and the autocorrelation information(s) from the encoded representation of the audio signal. The method may also include determining a reconstructed audio signal based on the spectral envelope and the autocorrelation information. The reconstructed audio signal may be determined such that an autocorrelation function for each of a plurality of sub-band audio signals (or ones generated from the reconstructed audio signal) of the reconstructed audio signal satisfies a condition derived from the autocorrelation information for a corresponding sub-band audio signal (or ones generated from the audio signal) of the audio signal. For example, the reconstructed audio signal may be determined such that, for each sub-band audio signal of the reconstructed audio signal, a value of an autocorrelation function of the sub-band audio signal (or one generated from the reconstructed audio signal) of the reconstructed audio signal at a lag value (e.g. delay value) indicated by the autocorrelation information for the corresponding sub-band audio signal (or one generated from the reconstructed audio signal) of the audio signal substantially matches the autocorrelation value indicated by the autocorrelation information for the corresponding sub-band audio signal of the audio signal. This may mean that the decoder can determine the autocorrelation function of the sub-band audio signal in the same way as is done by the encoder. This may include any, some or all of flattening, windowing and normalization.In some implementations, the reconstructed audio signal may be determined such that autocorrelation information for each of a plurality of subband signals (or generated from the reconstructed subband audio signals) of the reconstructed subband audio signal substantially matches autocorrelation information for a corresponding subband audio signal (or generated from the audio signal) of the audio signal. For example, the reconstructed audio signal may be determined such that for each subband audio signal (or generated from the reconstructed audio signal) of the reconstructed audio signal, autocorrelation values ​​and lag values ​​(e.g. delay values) of an autocorrelation function of the subband signal of the reconstructed audio signal substantially match autocorrelation values ​​and lag values ​​indicated by the autocorrelation information for the corresponding subband audio signal (or generated from the audio signal) of the audio signal. This may mean that the decoder is able to determine the autocorrelation information (i.e. lag values ​​and autocorrelation values) for each subband signal of the reconstructed audio signal in the same way as is done by the encoder. Here, the term substantially matches may mean, for example, matches up to a predefined margin. In those implementations in which the encoded representation includes waveform information, the reconstructed audio signal may be determined further based on the waveform information. The subband audio signal may, for example, be obtained by spectral decomposition of the applicable audio signal (i.e. the original audio signal on the encoder side or the reconstructed audio signal on the decoder side) or by flattening and windowing the applicable audio signal followed by spectral decomposition.

[0016] The decoder may therefore be said to operate according to a synthesis by analysis approach, which attempts to find a reconstructed audio signal z that satisfies at least one condition derived from a coded representation h(x) of the encoded audio signal or whose coded representation h(z) substantially matches a coded representation h(x) of the original audio signal x, where h is the coding map used by the encoder.

[0017]

number

[0018] In some embodiments, the reconstructed audio signal may be determined in an iterative procedure starting from an initial candidate for the reconstructed audio signal and generating a respective intermediate reconstructed audio signal at each iteration. At each iteration, an update map may be applied to the intermediate reconstructed audio signal to obtain the intermediate reconstructed audio signal for the next iteration. The update map may be configured such that the autocorrelation function of the subband audio signal (or generated from the intermediate reconstruction) of the intermediate reconstruction of the audio signal approaches a condition derived from autocorrelation information for the corresponding subband audio signal (or generated from the audio signal) of the audio signal and / or such that the difference between the measured signal power of the subband audio signal (or generated from the reconstructed audio signal) of the reconstructed audio signal and the signal power of the corresponding subband audio signal (or generated from the audio signal) of the audio signal as indicated by the spectral envelope is reduced from one iteration to the next. When both the autocorrelation information and the spectral envelope are taken into account, an appropriate difference metric for the degree to which the condition is fulfilled and the difference between the signal powers for the subband audio signals may be defined. In some implementations, the update map may be configured such that the difference between the coded representation of the intermediate reconstructed audio signal and the coded representation of the audio signal becomes successively smaller from one iteration to the next. To this end, a suitable difference metric for the coded representation (including the spectral envelope and / or autocorrelation information) may be defined and used. The autocorrelation functions of the sub-band audio signals of the intermediate reconstructed audio signal (or generated from the intermediate reconstructed audio signal) may be determined in the same way as the encoder does for the sub-band audio signals of the audio signal (or generated from the audio signal). Similarly, the coded representation of the intermediate reconstructed audio signal may be the coded representation obtained if the intermediate reconstructed audio signal was subjected to the same coding technique that resulted in the coded representation of the audio signal.

[0019] Such an iterative method allows for a simple yet efficient implementation of the synthesis approach according to the above analysis.

[0020] In some embodiments, determining the reconstructed audio signal based on the spectral envelope and the autocorrelation information may include applying a machine learning based generative model that receives as input the spectral envelope of the audio signal and the autocorrelation information for each of the multiple sub-band audio signals of the audio signal, and generates and outputs the reconstructed audio signal. In those implementations in which the encoded representation includes waveform information, the machine learning based generative model may further receive the waveform information as input. This means that the machine learning based generative model may also be conditioned / trained using the waveform information.

[0021] Such machine learning based methods allow a very efficient implementation of the analysis-by-synthesis techniques described above, achieving a reconstructed audio signal that is perceptually very close to the original audio signal.

[0022] Another aspect of the present disclosure relates to an encoder for encoding an audio signal, the encoder may include a processor and a memory coupled to the processor, the processor adapted to perform the steps of any one of the encoding methods described throughout the present disclosure.

[0023] Another aspect of the present disclosure relates to a decoder for decoding an audio signal from an encoded representation of the audio signal, the decoder may include a processor and a memory coupled to the processor, the processor adapted to perform the steps of any one of the decoding methods described throughout the present disclosure.

[0024] Another aspect relates to a computer program comprising instructions that, when executed, cause a computer to perform the steps of any of the methods described throughout this disclosure.

[0025] Another aspect of the present disclosure relates to a computer readable storage medium having stored thereon a computer program according to the above aspect. [Brief description of the drawings]

[0026] Example embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which: [Figure 1] FIG. 2 is a block diagram that illustrates a schematic example of an encoder according to an embodiment of the present disclosure. [Diagram 2] 1 is a flowchart illustrating an example of an encoding method according to an embodiment of the present disclosure. [Diagram 3] 3 shows a schematic example of a waveform that may be present in the framework of the encoding method of FIG. 2; [Figure 4] FIG. 1 is a block diagram that illustrates generally an example of an analysis-by-synthesis approach for determining a decoding function. [Diagram 5] 1 is a flowchart illustrating an example of a decoding method according to an embodiment of the present disclosure. [Figure 6] 6 is a flow chart illustrating example steps in the decoding method of FIG. 5. [Figure 7] FIG. 13 is a block diagram illustrating a schematic diagram of another example of an encoder according to an embodiment of the present disclosure. [Figure 8] FIG. 2 is a block diagram that illustrates a schematic example of a decoder according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] [Introduction] High-quality audio coding systems generally require a relatively large amount of data to code audio content, i.e., have a relatively low coding efficiency. Although the development of tools such as noise filling and high-frequency restoration shows that waveform description data can be partially replaced by a smaller set of control data, no high-quality audio codec relies primarily on perceptually relevant features. However, increased computing power and recent advances in the field of machine learning have increased the feasibility of decoding audio primarily from any encoder format. This disclosure proposes examples of such encoder formats.

[0028] Broadly speaking, the present disclosure proposes an encoding format based on sub-band envelopes and additional information provided by auditory resolution. The additional information comprises a single autocorrelation value and a single lag value per sub-band (and per update step). The envelopes can be calculated at a first update rate and the additional information can be sampled at a second update rate. Decoding of the encoding format can proceed using a synthesis by analysis approach, which can be implemented, for example, by iterative or machine learning based techniques.

[0029] [Encoding] The encoding format (encoded representation) proposed in this disclosure may be called a multi-lag format since it provides one lag per subband (and update step). Figure 1 is a block diagram that illustrates a schematic example of an encoder 100 for generating the encoding format according to an embodiment of the present disclosure.

[0030] The encoder 100 receives a target sound 10 corresponding to an audio signal to be encoded. The audio signal 10 may include multiple consecutive or partially overlapping parts (e.g. groups of samples, segments, frames, etc.) that are processed by the encoder. The audio signal 10 is spectrally decomposed into multiple subband audio signals 20 in corresponding frequency subbands by a filter bank 15. The filter bank 15 may be a filter bank of bandpass filters (BPF), for example it may be a complex-valued BPF. In audio, a filter bank of BPFs with a frequency resolution related to the human auditory system is naturally used.

[0031] The spectral envelope 30 of the audio signal 10 is extracted in an envelope extraction block 25. For each subband, a power is measured at a given time step as a basic model of the auditory envelope or excitation pattern for the cochlea resulting from the input sound signal, thereby determining the spectral envelope 30 of the audio signal 10. That is, the spectral envelope 30 may be determined based on a plurality of subband audio signals 20, for example by measuring (for example estimating, calculating) a respective signal power for each of the plurality of subband audio signals 20. However, the spectral envelope 30 may be determined by any suitable alternative tool, for example a Linear Predictive Coding (LPC) description. In particular, in some implementations, the spectral envelope may be determined from the audio signal before the spectral decomposition by the filter bank 15.

[0032] Optionally, the extracted spectral envelope 30 may be downsampled in a downsampling block 35, and the downsampled spectral envelope 40 (or the spectral envelope 30) is output as part of the encoding format or encoded representation of the audio signal 10 (where applicable).

[0033] A reconstructed signal reconstructed from the spectral envelope only may still lack sound quality. To address this issue, the present disclosure proposes to include a single value (i.e., ordinate and abscissa) of the autocorrelation function of the (possibly envelope-flattened) signal per subband, which results in dramatically improved sound quality. To this end, the subband audio signals 20 are optionally flattened (envelope-flattened) in the divider 45 and input to the autocorrelation block 55. The autocorrelation block 55 determines an autocorrelation function (ACF) of its input signal and outputs a respective autocorrelation information 50 for each of the subband audio signals 20 (i.e., for each of the subbands) based on the ACF of the respective subband audio signal 20. The autocorrelation information 50 for a given subband includes (e.g., consists of) a representation 50 of a lag value T and an autocorrelation value ρ(T). That is, for each subband, one value of lag T and the corresponding (possibly normalized) autocorrelation value (ACF value) ρ(T) are output (e.g., transmitted) as autocorrelation information 50 that is part of the coded representation, where the lag value T corresponds to the lag value at which the ACF reaches a local maximum, and the autocorrelation value ρ(T) corresponds to this local maximum. In other words, the autocorrelation information for a given subband may include the lag value (i.e., abscissa) and the autocorrelation value (i.e., ordinate) of the local maximum of the ACF.

[0034] Thus, the encoded representation of the audio signal includes a spectral envelope of the audio signal and autocorrelation information for each of the subbands. The autocorrelation information for a given subband includes a representation of a lag value T and an autocorrelation value ρ(T). The encoded representation corresponds to an output of the encoder. In some implementations, the encoded representation may further include waveform information relating to the waveform of the audio signal and / or one or more waveforms of the subband audio signals.

[0035] The above procedure defines a coding function (or coding map) h that maps the input audio signal to its coded representation.

[0036] As mentioned above, the spectral envelope and the autocorrelation information for the subband audio signals may be determined and output at different update rates (sample rates). For example, the spectral envelope may be determined at a first update rate and the autocorrelation information for the multiple subband audio signals may be determined at a second update rate different from the first update rate. The representation of the spectral envelope (for all subbands) and the representation of the autocorrelation information may be written to the bitstream at the respective update rates (sample rates). In this case, the coded representation may relate to a part of the bitstream output by the encoder. In this regard, it should be noted that at each time point, the current spectral envelope and the current set of autocorrelation information (one per subband) are defined by the bitstream and may be the coded representation. Alternatively, the representation of the spectral envelope (for all subbands) and the representation of the autocorrelation information may be updated at the respective update rates in the respective output units of the encoder. In this case, each output unit (e.g., an encoded frame) of the encoder corresponds to an instance of the coded representation. The representation of the spectral envelope and the autocorrelation information may be the same between a series of consecutive output units depending on the respective update rates.

[0037] Preferably, the first update rate is higher than the second update rate. In one example, the first update rate R1 may be R1=1 / (2.5ms) and the second update rate R2 may be R2=1 / (20ms), so that an updated representation of the spectral envelope is output every 2.5ms, but an updated representation of the autocorrelation information is output every 20ms. For a portion (e.g., a frame) of an audio signal, the spectral envelope may be determined every nth portion (e.g., every subportion), but the autocorrelation information may be determined every mth portion (m>n).

[0038] The coded representation may be output as a sequence of frames of a certain frame length. Among other factors, the frame length may depend on the first and / or second update rate. Considering a frame having a length of a first period L1 (e.g., 2.5 ms) corresponding to a first update rate R1 (e.g., 1 / (2.5 ms)) via L1=1 / R1, this frame includes one representation of the spectral envelope and one set of autocorrelation information representations (one per subband audio signal). For first and second update rates of 1 / (2.5 ms) and 1 / (20 ms), respectively, the autocorrelation information is the same for eight consecutive frames of the coded representation, respectively. In general, assuming that R1 and R2 are appropriately selected to have an integer ratio, the autocorrelation information is the same for R1 / R2 consecutive frames of the coded representation. On the other hand, if we consider a frame having a length of a second period L2 (e.g., 20 ms) corresponding to a second update rate R2 (e.g., 1 / (20 ms)) via L2=1 / R2, then this frame contains one set of representations of the autocorrelation information and R1 / R2 (e.g., 8) representations of the spectral envelope.

[0039] In some implementations, different update rates may also be applied to different subbands, i.e. autocorrelation information for different subband audio signals may be generated and output at different update rates.

[0040] 2 is a flow chart illustrating an example of an encoding method 200 according to an embodiment of the present disclosure, which may be implemented by the encoder 100 described above and receives an audio signal as input.

[0041] In step S210, a plurality of subband audio signals are generated based on the audio signal. This may involve spectrally decomposing the audio signal, in which case this step may be performed according to the operation of the filter bank 15 described above. Alternatively, it may involve spectrally and / or temporally flattening the audio signal, optionally windowing the flattened audio signal by a window function, and spectrally decomposing the resulting signal into a plurality of subband audio signals.

[0042] In step S220, the spectral envelope of the audio signal is determined (e.g. calculated). This step may be performed in accordance with the operation of envelope extraction block 25 described above.

[0043] In step S230, for each subband audio signal, autocorrelation information for the subband audio signal is determined based on the ACF of the subband audio signal. This step may be performed in accordance with the operation of autocorrelation block 55 above.

[0044] In step S240, an encoded representation of the audio signal is generated, the encoded representation including a representation of the spectral envelope of the audio signal and a representation of autocorrelation information for each of the multiple subband audio signals.

[0045] Next, example implementation details of the steps of method 200 are described.

[0046] For example, as mentioned above, generating a plurality of subband audio signals may include (or may mean) spectrally decomposing an audio signal, for example using a filter bank. In this case, determining an autocorrelation function for a given subband audio signal may include determining a subband envelope of the subband audio signal. The subband envelope may be determined by taking a magnitude value of the subband audio signal. The ACF itself may be calculated on the real-valued (windowed after envelope flattening) subband signal.

[0047] Assuming that the subband filter response is complex-valued by a Fourier transform supported essentially at positive frequencies, the subband signal becomes complex-valued. The subband envelope can then be determined by taking the magnitude of the complex-valued subband signal. This subband envelope has the same number of samples as the subband signal and can still oscillate to some extent. Optionally, the subband envelope can be downsampled, for example by computing a triangular window weighted sum of squares of the envelope in a segment of a certain length (e.g., 5 ms long, 2.5 ms leading edge, 2.5 ms trailing edge) for every half-shift of a certain length (e.g., 2.5 ms) along the signal, and then taking the square root of this sequence to obtain the downsampled subband envelope. This may be said to correspond to the definition of the "rms envelope". The triangular window can be normalized so that a constant envelope of value 1 gives a sequence of ones. Other methods for determining the subband envelope are feasible as well, such as half-wave rectification followed by low-pass filtering in the case of real-valued subband signals. In either case, the subband envelope can be said to convey information about the energy in the subband signal (at the selected update rate).

[0048] The subband audio signal may then be envelope flattened based on the subband envelope. For example, a new full sample rate envelope signal may be created by linearly interpolating the downsampled values ​​and dividing the original (complex-valued) subband signal by this linearly interpolated envelope to arrive at the fine-structure signal (carrier) from which the ACF data is calculated.

[0049] The envelope-flattened sub-band audio signal may then be windowed by a suitable window function. Finally, an ACF of the windowed envelope-flattened sub-band audio signal is determined (e.g., calculated). In some implementations, determining the ACF for the given sub-band audio signal may further include normalizing the ACF of the windowed envelope-flattened sub-band audio signal by an autocorrelation function of the window function.

[0050] In Figure 3, the upper curve 310 shows the real-valued windowed envelope-flattened subband signal used to compute the ACF, and the lower solid curve 320 shows the real-valued complex ACF.

[0051] The main idea here is to find the greatest local maximum of the ACF of the subband signal among the local maxima above the absolute value of the ACF of the absolute value of the impulse response of the (complex-valued) subband filter (i.e. the corresponding BPF of the filter bank). For the ACF of the complex-valued subband signal, the real value of the ACF may be considered at this point. It may be necessary to find the greatest local maximum above the ACF of the absolute value of the impulse response in order to avoid picking lags that are related to the center frequency of the subband and not to the characteristics of the input signal. As a final adjustment, the maximum may be divided by the value of the ACF of the window function used for the subband ACF window (e.g. assuming that the ACF of the subband signal itself is normalized so that the autocorrelation value of zero delay is normalized to 1). This results in a better use of the interval between 0 and 1, where ρ(T)=1 is the maximum tonality.

[0052] Thus, determining the autocorrelation information for a given subband audio signal based on the ACF of the subband audio signal may further include comparing the ACF of the subband audio signal with the ACF of the absolute value of the impulse response of the respective bandpass filter associated with the subband audio signal. The ACF of the absolute value of the impulse response of the respective bandpass filter associated with the subband audio signal is indicated by a solid curve 330 in the lower part of FIG. 3. The autocorrelation information is then determined based on the highest local maxima of the ACF of the subband signal above the ACF of the absolute value of the impulse response of the respective bandpass filter associated with the subband audio signal. In the lower part of FIG. 3, the local maxima of the ACF are indicated by crosses and the selected highest local maxima of the ACF of the subband signal above the ACF of the absolute value of the impulse response of the respective bandpass are indicated by circles. Optionally, the selected local maxima of the ACF may be normalized by the value of the ACF of the window function ACF (e.g. assuming that the ACF itself is normalized such that the autocorrelation value of zero lag is normalized to 1). The highest selected local maximum after normalization of the ACF is indicated by an asterisk at the bottom of FIG. 3, and the dashed curve 340 shows the ACF of the window function.

[0053] The autocorrelation information determined at this stage may include the autocorrelation value and the delay value (ie the ordinate and abscissa) of the selected (normalized) highest local maximum of the ACF of the subband audio signal.

[0054] A similar coding format may be defined in the framework of an LPC-based vocoder. Also in this case, autocorrelation information is extracted from subband signals affected by at least some degree of spectral and / or temporal flattening. Different from the above example, this is done by creating a (perceptually weighted) LPC residual, windowing it and decomposing it into subbands to obtain multiple subband audio signals. This is followed by calculating the ACF and extracting the lag and autocorrelation values ​​of each subband audio signal.

[0055] For example, generating the plurality of subband audio signals may include applying a spectral and / or temporal flattening to the audio signal (e.g., by generating a perceptually weighted LPC residual from the audio signal using an LPC filter). This may be followed by windowing the flattened audio signal with a window function and spectrally decomposing the windowed flattened audio signal into the plurality of subband audio signals. As described above, the result of the temporal and / or spectral flattening may correspond to a perceptually weighted LPC residual, which is then subjected to windowing and spectral decomposition into subbands. The perceptually weighted LPC residual may, for example, be a pink LPC residual.

[0056] [Decryption] This disclosure relates to audio decoding based on an analysis-by-synthesis approach. At the most abstract level, it is assumed that a coding map h from a signal to a perceptually motivated domain is given, such that the original audio signal x is represented by y=h(x). In the best case, a simple distortion measure such as least squares in the perceptual domain will be a good predictor of the subjective difference measured by a population of listeners.

[0057] One remaining problem is to design a decoder q that maps from y (both the encoded and decoded versions) to an audio signal z = d(y). For this, we can use the concept of analysis-by-synthesis, which involves "finding the waveform that comes closest to producing a given image". The goal is that z and x sound the same, so that the decoder should solve the inverse problem h(z) = y = h(x). For the construction of the map, d should approximate the left reciprocal of h,

[0058]

number

[0059] FIG. 4 is a block diagram that illustrates in schematic form an example of an analysis-by-synthesis for determining a decoding function (or decoding map) d given an encoding function (or encoding map) h. An original audio signal x, 410 is subjected to an encoding map h, 415, resulting in an encoded representation y, 420, where y=h(x). The encoded representation y may be defined in the perceptual domain. The objective is to find a decoding function (decoding mapping) d, 425 that maps the encoded representation y to a reconstructed audio signal z, 430, with the property that applying the encoding mapping h, 435 to the reconstructed audio signal z results in an encoded representation h(z), 440 that substantially matches the encoded representation y=h(x). Here, "substantially matches" may mean, for example, "matches up to a certain margin". In other words, given an encoding map h, the ...

[0060]

number

[0061] 5 is a flow chart illustrating an example of a decoding method 500 according to an analysis-by-synthesis approach according to an embodiment of the present disclosure. The method 500 is a method for decoding an audio signal from an encoded representation of an (original) audio signal. It is assumed that the encoded representation includes a representation of a spectral envelope of the original audio signal and a representation of autocorrelation information for each of a number of sub-band audio signals of the original audio signal. The autocorrelation information for a given sub-band audio signal is based on the ACF of the sub-band audio signal.

[0062] In step S510, an encoded representation of an audio signal is received.

[0063] In step S520, the spectral envelope and autocorrelation information is extracted from the encoded representation of the audio signal.

[0064] In step S530, the reconstructed audio signal is determined based on the spectral envelope and the autocorrelation information, where the reconstructed audio signal is determined such that an autocorrelation function of each of the plurality of subband signals of the reconstructed subband audio signal (substantially) satisfies a condition derived from the autocorrelation information for the corresponding subband audio signal of the audio signal. This condition may for example be that, for each subband audio signal of the reconstructed audio signal, at a lag value (e.g. delay value) indicated by the autocorrelation information for the corresponding subband audio signal of the audio signal, the value of the ACF of the subband audio signal of the reconstructed audio signal substantially matches the autocorrelation value indicated by the autocorrelation information for the corresponding subband audio signal of the audio signal. This may mean that the decoder can determine the ACF of the subband audio signal in the same way as is done by the encoder. This may include any, some or all of flattening, windowing and normalization. In one implementation, the reconstructed audio signal may be determined such that, for each subband audio signal of the reconstructed audio signal, autocorrelation values ​​and lag values ​​(e.g. delay values) of the ACF of the subband audio signal of the reconstructed audio signal substantially match the autocorrelation values ​​and lag values ​​indicated by the autocorrelation information of the corresponding subband audio signal of the original audio signal. This may mean that the decoder can determine the autocorrelation information for each subband signal of the reconstructed audio signal in the same way as is done by the encoder. In those implementations where the encoded representation also includes waveform information, the reconstructed audio signal may be determined further based on the waveform information. The subband audio signals of the reconstructed audio signal may be generated in the same way as is done by the encoder. For example, this may include a spectral decomposition, or a sequence of flattening, windowing and spectral decomposition.

[0065] Preferably, the determination of the reconstructed audio signal in step S530 also takes into account the spectral envelope of the original audio signal. The reconstructed audio signal may then be further determined such that, for each sub-band audio signal of the reconstructed sub-band audio signal, a measured (e.g. estimated or calculated) signal power of the sub-band audio signal of the reconstructed audio signal substantially matches a signal power for the corresponding sub-band audio signal of the original audio signal as indicated by the spectral envelope.

[0066] As can be seen from the above, the proposed method 500 can be said to result from an analysis-by-synthesis approach in that it tries to find a reconstructed audio signal z that (substantially) satisfies at least one condition derived from a coded representation y=h(x) of the original audio signal x, where h is the coding map used by the encoder. In some implementations, the proposed method can even be said to operate according to an analysis-by-synthesis approach in that it tries to find a reconstructed audio signal z whose coded representation h(z) substantially matches the coded representation y=h(x) of the original audio signal x. In other words, the decoding method:

[0067]

number

[0068] [Implementation example 1: Parametric synthesis or signal repetition] The inverse problem h(z)=y is n ) is h(z n-1 ) so that z is closer to y than n-1 Update map to fix Z n =f(z n-1,y). For example, the starting point of the iteration (i.e., the initial candidate for the reconstructed audio signal) may be a random noise signal (e.g., white noise) or may be determined based on the coded representation of the audio signal (e.g., as a manually created first guess). In the latter case, the initial candidate for the reconstructed audio signal may relate to a trained guess made based on spectral envelope and / or autocorrelation information for multiple subband audio signals. In those implementations where the coded representation includes waveform information, the trained guess may be made further based on the waveform information.

[0069] More specifically, the reconstructed audio signal in this implementation is determined in an iterative procedure starting from an initial candidate for the reconstructed audio signal and generating a respective intermediate reconstructed audio signal at each iteration. At each iteration, an update map is applied to the intermediate reconstructed audio signal to obtain an intermediate reconstructed audio signal for the next iteration. The update map is selected such that the difference between the encoded representation of the intermediate reconstructed audio signal and the encoded representation of the original audio signal becomes successively smaller from one iteration to the next. To this end, a suitable difference metric for the encoded representation (e.g., spectral envelope, autocorrelation information) may be defined and used to evaluate the difference. The encoded representation of the intermediate reconstructed audio signal may be an encoded representation obtained if the intermediate reconstructed audio signal were subjected to the same encoding scheme that results in the encoded representation of the audio signal.

[0070] If the procedure searches for a reconstructed audio signal satisfying at least one condition derived from the autocorrelation information(s), the update map may be selected such that the autocorrelation functions of the subband audio signals of intermediate reconstructions of the audio signal satisfy the respective conditions derived from the autocorrelation information for the corresponding subband audio signals of the audio signal and / or such that the difference between the measured signal powers of the subband audio signals of the reconstructed audio signal and the signal powers for the corresponding subband audio signals of the audio signal as indicated by the spectral envelope is reduced from one iteration to the next. If both the autocorrelation information and the spectral envelope are taken into account, an appropriate difference metric may be defined for the extent to which the condition is satisfied and the difference between the signal powers for the subband audio signals.

[0071] [Implementation example 2: Generative model based on machine learning] Another option, made possible by modern machine learning methods, is to train a machine learning based generative model (or generative model for short) for audio x conditioned on data y. That is, given a large set of examples of (x,y) where y=h(x), a parametric conditional distribution p(x|y) from y to x is trained. The decoding algorithm may then consist of sampling from the distribution z~p(x|y).

[0072] This option has been found to be particularly advantageous when h(x) is a speech vocoder and p(x|y) is defined by a sequential generative model sample Recurrent Neural Network (RNN). However, other generative models, such as a variational autoencoder or a generative adversarial model, are equally relevant for this task. Thus, without intending to be limiting, the machine learning based generative model can be one of a recurrent neural network, a variational autoencoder, or a generative adversarial model (e.g., a Generative Adversarial Network (GAN)).

[0073] In this example implementation, determining the reconstructed audio signal based on the spectral envelope and the autocorrelation information includes applying a machine learning based generative model that receives as input the spectral envelope of the audio signal and the autocorrelation information for each of a plurality of subband audio signals of the audio signal, and generates and outputs the reconstructed audio signal. The encoded representation also includes waveform information. In these implementations, the machine learning based generative model may further receive the waveform information as input.

[0074] As described above, the machine learning based generative model may include a parametric conditional distribution p(x|y) that associates an encoded representation y of the audio signal and a corresponding audio signal x with a respective probability p. Determining the reconstructed audio signal may then include sampling from the parametric conditional distribution p(x|y) for the encoded representation of the audio signal.

[0075] In a training phase, prior to decoding, the machine learning based generative model may be conditioned / trained on a dataset of multiple audio signals and corresponding encoded representations of the audio signals. If the encoded representations also include waveform information, the machine learning based generative model may also be conditioned / trained using the waveform information.

[0076] Figure 6 is a flow chart illustrating an example implementation 600 of step S530 in the decoding method 500 of Figure 5. In particular, the implementation 600 relates to a subband-by-subband implementation of step S530.

[0077] In step 610, a plurality of reconstructed subband audio signals are determined based on the spectral envelope and the autocorrelation information, where the plurality of reconstructed subband audio signals are determined such that, for each reconstructed subband audio signal, an autocorrelation function of the reconstructed subband audio signal satisfies a condition derived from the autocorrelation information for a corresponding subband audio signal of the audio signal. In some implementations, the plurality of reconstructed subband audio signals are determined such that, for each reconstructed subband audio signal, the autocorrelation information for the reconstructed subband audio signal substantially matches the autocorrelation information for the corresponding subband audio signal.

[0078] Preferably, the determination of the plurality of reconstructed subband audio signals in step S610 also takes into account the spectral envelope of the original audio signal. The plurality of reconstructed subband audio signals are then further determined, for each reconstructed subband audio signal, such that a measured (e.g. estimated, calculated) signal power of the reconstructed subband audio signal substantially matches a signal power for the corresponding subband audio signal as indicated by the spectral envelope.

[0079] In step S620, a reconstructed audio signal is determined based on the multiple reconstructed subband audio signals by spectral synthesis.

[0080] The above implementation examples 1 and 2 may also be applied to the implementation of each subband of step S530. In implementation example 1, each reconstructed subband audio signal may be determined in an iterative procedure, starting from an initial candidate for the reconstructed subband audio signal and generating a respective intermediate reconstructed subband audio signal in each iteration. In each iteration, an update map may be applied to the intermediate reconstructed subband audio signal to obtain the intermediate reconstructed subband audio signal for the next iteration, such that the difference between the autocorrelation information for the intermediate reconstructed subband audio signal and the autocorrelation information for the corresponding subband audio signal becomes successively smaller from one iteration to the next iteration, or such that the reconstructed subband audio signal better satisfies each condition derived from the autocorrelation information for each corresponding subband audio signal of the audio signal.

[0081] Also, the spectral envelope may be taken into account at this point. That is, the update map may be such that the (joint) differences between the respective signal powers of the subband audio signals and the (joint) differences between the respective items of autocorrelation information become successively smaller. This may imply the definition of an appropriate difference metric for evaluating the (joint) differences. Otherwise, explanations similar to those given above for implementation example 1 may apply in this case.

[0082] When applying Implementation Example 2 to a per-subband implementation of step S530, determining the plurality of reconstructed subband audio signals based on the spectral envelope and the autocorrelation information may include applying a machine learning based generative model that receives as input the spectral envelope of the audio signal and the autocorrelation information for each of the plurality of subband audio signals of the audio signal, and generates and outputs the plurality of reconstructed subband audio signals. Otherwise, similar explanations as given above for Implementation Example 2 may be applied in this case.

[0083] The present disclosure further relates to an encoder for encoding an audio signal, capable of and adapted to perform the encoding methods described throughout the present disclosure. An example of such an encoder 700 is shown generally in Fig. 7 in the form of a block diagram. The encoder 700 comprises a processor 710 and a memory 720 coupled to the processor 710. The processor 710 is adapted to perform the method steps of any one of the encoding methods described throughout the present disclosure. To this end, the memory 720 may comprise respective instructions for the processor 710 to execute. The encoder 700 may further comprise an interface 730 for receiving an input audio signal 740 to be encoded and / or for outputting an encoded representation 750 of the audio signal.

[0084] The present disclosure further relates to a decoder for decoding an audio signal from an encoded representation of the audio signal, capable of and adapted to perform the decoding methods described throughout the present disclosure. An example of such a decoder 800 is shown generally in FIG. 8 in the form of a block diagram. The decoder 800 comprises a processor 810 and a memory 820 coupled to the processor 810. The processor 810 is adapted to perform the method steps of any one of the decoding methods described throughout the present disclosure. To this end, the memory 820 may comprise respective instructions for the processor 810 to execute. The encoder 800 may further comprise an interface 830 for receiving an input encoded representation 840 of the audio signal to be decoded and / or for outputting a decoded (i.e. reconstructed) audio signal 850.

[0085] The present disclosure further relates to a computer program comprising instructions that, when executed, cause a computer to perform the encoding or decoding methods described throughout this disclosure.

[0086] Finally, the present disclosure also relates to a computer readable storage medium storing the above computer program.

[0087] [interpretation] Unless otherwise indicated, and as will be apparent from the discussion that follows, discussions throughout this disclosure using terms such as "processing," "calculating," "operating," "determining," "analyzing," and the like will be recognized to refer to operations and / or processes of a computer or computer system or similar electronic computing device that manipulates and / or transforms data represented as physical quantities, such as electronic quantities, into other data similarly represented as physical quantities.

[0088] Similarly, the term "processor" may refer to any device or part of a device that processes electronic data, e.g., from a register and / or memory, and converts it into other electronic data, which may be stored, e.g., in a register and / or memory. A "computer" or "computing platform" may include one or more processors.

[0089] The methods described herein, in one exemplary embodiment, can be executed by one or more processors that accept computer-readable (also referred to as machine-readable) code that includes a set of instructions that, when executed by the one or more processors, perform at least one of the methods described herein. Any processor capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken is included. Thus, one example is a typical processing system that includes one or more processors. Each processor may include one or more of a CPU, a graphics processing unit, and a programmable DSP unit. The processing system may further include a memory subsystem including a main RAM and / or static RAM and / or ROM. A bus subsystem for communicating between the components may be included. Additionally, the processing system may be a distributed processing system with processors coupled by a network. If the processing system requires a display, a display such as, for example, a liquid crystal display (LCD) or a cathode ray tube (CRT) display may be included. If manual data entry is required, the processing system also includes an input device such as one or more of an alphanumeric input unit such as a keyboard, a pointing control device such as a mouse, and the like. The processing system may also include a storage system, such as a disk drive unit. The processing system in some configurations may also include a sound output device and a network interface device. Thus, the memory subsystem includes a computer readable carrier that carries computer readable code (e.g., software) that includes a set of instructions that, when executed by one or more processors, causes the execution of one or more of the methods described herein. It is noted that when a method includes several elements, e.g., several steps, no order of such elements is implied unless specifically stated.The software may reside on a hard disk or may reside, completely or at least partially, within the RAM and / or the processor during its execution by the computer system. Thus, the memory and the processor also constitute a computer readable carrier carrying computer readable code. Moreover, the computer readable carrier may form or be included in a computer program product.

[0090] In alternative exemplary embodiments, one or more processors may operate as stand-alone devices or may be connected, e.g., networked, to other processors; in a networked deployment, one or more processors may operate in the capacity of a server or user machine in a server-user network environment, or as a peer machine in a peer-to-peer or distributed network environment. One or more processors may form a personal computer (PC), tablet PC, Personal Digital Assistant (PDA), mobile phone, web appliance, network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the machine.

[0091] It is noted that the term "machine" is also intended to include any collection of machines that individually or collectively execute a set (or sets) of instructions for performing any one or more of the methodologies discussed herein.

[0092] Thus, one exemplary embodiment of each of the methods described herein is in the form of a computer readable carrier carrying a set of instructions, e.g., a computer program for execution on one or more processors, e.g., one or more processors that are part of a web server configuration. Thus, as will be appreciated by those skilled in the art, exemplary embodiments of the present disclosure may be embodied as a method, an apparatus such as a special purpose device, an apparatus such as a data processing system, or a computer readable carrier, e.g., a computer program product. The computer readable carrier carries computer readable code that, when executed on one or more processors, causes the processor to perform the method. Thus, aspects of the present disclosure may take the form of a method, an entirely hardware exemplary embodiment, an entirely software exemplary embodiment, or an exemplary embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a carrier (e.g., a computer program product on a computer readable storage medium) carrying computer readable program code embodied in a medium.

[0093] The software may further be transmitted or received over a network via a network interface device. In an exemplary embodiment, the carrier is a single medium, but the term "carrier" should be interpreted to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of instructions. The term "carrier" should also be interpreted to include any medium that can store, encode or carry a set of instructions for execution by one or more processors, causing one or more processors to perform any one or more of the methodologies of the present disclosure. The carrier may take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical disks, magnetic disks, and magneto-optical disks. Volatile media include dynamic memory, such as main memory. Transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up a bus subsystem. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications. For example, the term "carrier" includes, but is not limited to, solid-state memories, computer products embodied in optical and magnetic media, media carrying a propagated signal detectable by at least one processor or one or more processors and representing a set of instructions that, when executed, perform a method, and transmission media in a network carrying a propagated signal detectable by at least one processor of one or more processors and representing a set of instructions.

[0094] It is understood that the steps of the described method are performed, in one exemplary embodiment, by a suitable processor (or processors) of a processing (e.g., computer) system executing instructions (computer readable code) stored in storage. It is also understood that the present disclosure is not limited to any particular implementation or programming technique, and that the present disclosure may be implemented using any suitable technique for implementing the functions described herein. The present disclosure is not limited to any particular programming language or operating system.

[0095] Throughout this disclosure, references to "one exemplary embodiment," "some exemplary embodiments," or "exemplary embodiments" mean that a particular feature, structure, or characteristic described in connection with an exemplary embodiment is included in at least one exemplary embodiment of this disclosure. Thus, the appearances of the phrases "in one exemplary embodiment," "some exemplary embodiments," or "in exemplary embodiments" in various places throughout this disclosure are not necessarily all referring to the same exemplary embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more exemplary embodiments, as would be apparent to one of ordinary skill in the art from this disclosure.

[0096] As used herein, the use of ordinal adjectives such as "first," "second," "third," etc. to describe a common object indicates merely that different instances of a similar object are being referred to, and is not intended to imply that the objects so described must be in a given order in time, space, ranking, or otherwise.

[0097] In the following claims and the description herein, any one of the terms "comprising", "comprised of" or "which comprises" is an open term meaning to include at least an element / feature but not to exclude other elements. Thus, when used in the claims, the term "comprising" should not be interpreted as being limited to the means or elements or steps listed. For example, an apparatus including A and B should not be limited to a device consisting of only elements A and B. As used herein, any one of the terms "including" or "which includes" is also an open term meaning to include at least an element / feature related to the term but not to exclude other terms. Thus, "including" is synonymous with "comprising" and means the same thing.

[0098] In the above description of exemplary embodiments of the disclosure, it should be recognized that various features of the disclosure are sometimes grouped together in a single exemplary embodiment, figure, or description thereof for the purpose of streamlining the disclosure and facilitating understanding of one or more of the various inventive aspects. However, this method of disclosure should not be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single above-disclosed exemplary embodiment. Accordingly, the claims following the description are expressly incorporated herein, with each claim standing on its own as a separate exemplary embodiment of the present disclosure.

[0099] Furthermore, although some exemplary embodiments described herein include some features and not other features included in other exemplary embodiments, it is understood by those skilled in the art that combinations of features of different exemplary embodiments are within the scope of the present disclosure and are meant to form different exemplary embodiments. For example, in the following claims, any of the exemplary embodiments recited in the claims can be used in any combination.

[0100] In the description provided herein, numerous specific details are set forth. However, it is understood that exemplary embodiments of the present disclosure may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0101] Thus, while what is believed to be the best mode of the disclosure has been described, those skilled in the art will recognize that other and further modifications may be made without departing from the spirit of the disclosure, and it is intended to claim all such changes and modifications that fall within the scope of the disclosure. For example, the above formulas merely represent procedures that may be used. Functions may be added or deleted from the block diagrams, and operations may be interchanged between functional blocks. Steps may be added or deleted to methods that are within the scope of the disclosure.

[0102] Various aspects and embodiments of the present disclosure can be appreciated from the following exemplary embodiments (EEE):

[0103] EEE1. A method for encoding an audio signal, comprising the steps of: generating a plurality of subband audio signals based on the audio signal; determining a spectral envelope of an audio signal; determining, for each subband audio signal, autocorrelation information for the subband audio signal based on an autocorrelation function of the subband audio signal; generating an encoded representation of the audio signal, the encoded representation including a representation of a spectral envelope of the audio signal and a representation of autocorrelation information for a plurality of subband audio signals; The method includes:

[0104] EEE2. The method of EEE1, wherein the spectral envelope is determined based on a plurality of subband audio signals.

[0105] EEE3. The method of EEE1 or EEE2, wherein the autocorrelation information for a given subband audio signal includes a lag value for the respective subband audio signal and / or an autocorrelation value for the respective subband audio signal.

[0106] EEE4. The method according to EEE3, wherein the lag value corresponds to the delay value at which the autocorrelation function reaches a local maximum, and the autocorrelation value corresponds to this local maximum.

[0107] EEE5. The method of any one of EEE1 to 4, wherein the spectral envelope is determined at a first update rate and the autocorrelation information for the plurality of subband audio signals is determined at a second update rate, the first update rate and the second update rate being different from each other.

[0108] EEE6. The method of EEE5, wherein the first update rate is higher than the second update rate.

[0109] EEE7. Generating a plurality of sub-band audio signals applying spectral and / or temporal flattening to the audio signal; Window the flattened audio signal, 7. The method of any one of EEE1 to 6, comprising spectrally decomposing the windowed flattened audio signal into a plurality of subband audio signals.

[0110] EEE8. Generating a plurality of subband audio signals includes spectrally decomposing the audio signal; Determining the autocorrelation function for a given subband audio signal comprises: determining a subband envelope of the subband audio signal; envelope flattening the subband audio signal based on the subband envelope; Windowing the envelope-flattened subband audio signal with a window function; 7. The method of any one of EEE1 to 6, comprising determining an autocorrelation function of the windowed envelope-flattened sub-band audio signal.

[0111] EEE9. Determining an autocorrelation function for a given subband audio signal comprises: The method according to any one of claims 8 to 10, further comprising normalizing the autocorrelation function of the windowed envelope-flattened sub-band audio signal by the autocorrelation function of the window function.

[0112] EEE10. Determining an autocorrelation function for a subband audio signal based on an autocorrelation function of a given subband audio signal, comprising: comparing the autocorrelation function of the subband audio signal with the autocorrelation function of the absolute value of the impulse response of each bandpass filter associated with the subband audio signal; The method according to any one of EEE1 to 9, comprising determining autocorrelation information based on the highest local maximum of the autocorrelation function of the subband signal that lies above the autocorrelation function of the absolute value of the impulse response of a respective bandpass filter associated with the subband audio signal.

[0113] EEE11. The method of any one of EEE1 to EEE10, wherein determining the spectral envelope comprises measuring a signal power for each of a plurality of subband audio signals.

[0114] EEE12. A method of decoding an audio signal from an encoded representation of the audio signal, the encoded representation comprising a representation of a spectral envelope of the audio signal and a representation of autocorrelation information for each of a plurality of subband audio signals generated from the audio signal, the autocorrelation information for a given subband audio signal being based on an autocorrelation function of the subband audio signal, the method comprising: receiving an encoded representation of an audio signal; extracting spectral envelope and autocorrelation information from the encoded representation of the audio signal; determining a reconstructed audio signal based on the spectral envelope and the autocorrelation information; Including, A method in which the reconstructed audio signal is determined such that an autocorrelation function for each of a plurality of subband audio signals generated from the reconstructed audio signal satisfies a condition derived from autocorrelation information for a corresponding subband audio signal generated from the audio signal.

[0115] EEE13. The method according to EEE12, wherein the reconstructed audio signal is further determined such that, for each sub-band audio signal of the reconstructed audio signal, a measured signal power of the sub-band audio signal of the reconstructed audio signal substantially matches a signal power for a corresponding sub-band audio signal of the audio signal as indicated by the spectral envelope.

[0116] EEE14. The reconstructed audio signal is determined in an iterative procedure starting from an initial candidate for the reconstructed audio signal and generating a respective intermediate reconstructed audio signal at each iteration; The method according to EEE12 or 13, wherein in each iteration an update map is applied to the intermediate reconstructed audio signal to obtain an intermediate reconstructed audio signal for the next iteration, such that a difference between the intermediate reconstructed audio signal and the coded representation of the audio signal becomes successively smaller from one iteration to another.

[0117] EEE15. The method of EEE14, wherein initial candidates for the reconstructed audio signal are determined based on an encoded representation of the audio signal.

[0118] EEE16. The method of EEE14, wherein an initial candidate for the reconstructed audio signal is white noise.

[0119] EEE17. The method according to EEE12 or 13, wherein determining a reconstructed audio signal based on the spectral envelope and the autocorrelation information comprises applying a machine learning based generative model that receives as input the spectral envelope of the audio signal and the autocorrelation information for each of a plurality of sub-band audio signals of the audio signal, and generates and outputs the reconstructed audio signal.

[0120] EEE18. The machine learning based generative model includes a coded representation of an audio signal and a parametric conditional distribution relating a corresponding audio signal to a respective probability, The method according to EEE17, wherein determining the reconstructed audio signal comprises sampling from a parametric conditional distribution over the encoded representation of the audio signal.

[0121] EEE19. The method according to EEE17 or 18, further comprising, during a training phase, training a machine learning based generative model on a dataset of a plurality of audio signals and corresponding encoded representations of the audio signals.

[0122] EEE20. The method of any one of EEE17 to 19, wherein the machine learning based generative model is one of a recurrent neural network, a variational autoencoder, or an adversarial generative model.

[0123] EEE21. Determining a reconstructed audio signal based on spectral envelope and autocorrelation information determining a plurality of reconstructed subband audio signals based on the spectral envelope and the autocorrelation information; determining a reconstructed audio signal based on the plurality of reconstructed subband audio signals by spectral synthesis; The method according to EEE12, wherein the plurality of reconstructed subband audio signals are determined such that for each reconstructed subband audio signal an autocorrelation function of the reconstructed subband audio signal satisfies a condition derived from autocorrelation information for a corresponding subband audio signal.

[0124] EEE22. The method of EEE21, wherein the plurality of reconstructed subband audio signals are further determined such that, for each reconstructed subband audio signal, a measured signal power of the reconstructed subband audio signal substantially matches a signal power for a corresponding subband audio signal as indicated by the spectral envelope.

[0125] EEE23. Each reconstructed subband audio signal is determined in an iterative procedure starting from an initial candidate for the reconstructed subband audio signal and generating a respective intermediate reconstructed subband audio signal in each iteration; The method according to EEE21 or 22, wherein in each iteration an update map is applied to an intermediate reconstructed subband audio signal to obtain an intermediate reconstructed subband audio signal for the next iteration, such that a difference between autocorrelation information for the intermediate reconstructed subband audio signal and autocorrelation information for the corresponding subband audio signal becomes successively smaller from one iteration to another.

[0126] EEE24. The method according to EEE21 or 22, wherein determining a plurality of reconstructed subband audio signals based on the spectral envelope and the autocorrelation information comprises applying a machine learning based generative model that receives as input the spectral envelope of the audio signal and the autocorrelation information for each of a plurality of subband audio signals of the audio signal, and generates and outputs a plurality of reconstructed subband audio signals.

[0127] EEE25. An encoder for encoding an audio signal, comprising: a processor; and a memory coupled to the processor, the processor adapted to perform the steps of the method according to any one of EEE1 to 11.

[0128] EEE26. A decoder for decoding an audio signal from an encoded representation of the audio signal, comprising a processor and a memory coupled to the processor, the processor being adapted to perform the steps of the methods according to any one of EEE12 to 24.

[0129] EEE27. A computer program comprising instructions which, when executed, cause a computer to carry out the method of any one of EEE1 to EEE24.

[0130] A computer readable storage medium storing a computer program according to any one of claims 1 to 8, wherein said computer program is a program code.

Claims

1. 1. A method for encoding an audio signal, comprising the steps of: generating a plurality of subband audio signals based on the audio signal; determining a spectral envelope of the audio signal; - for each subband audio signal, determining autocorrelation information for the subband audio signal based on an autocorrelation function of the subband audio signal, the autocorrelation information comprising an autocorrelation value for the subband audio signal; encoding the spectral envelope of the audio signal and the autocorrelation information for the plurality of subband audio signals into an encoded representation of the audio signal; Including, A method, wherein the autocorrelation information for a given subband audio signal further comprises a lag value for each subband audio signal.

2. The method of claim 1 , further comprising the step of outputting a bitstream defining the encoded representation.

3. The method according to claim 1 or 2, wherein the spectral envelope is determined based on the plurality of subband audio signals.

4. 4. The method of claim 1, wherein the lag value corresponds to a delay value at which the autocorrelation function reaches a local maximum, and the autocorrelation value corresponds to the local maximum.

5. the spectral envelope is determined at a first update rate and the autocorrelation information for the plurality of subband audio signals is determined at a second update rate; The method of claim 1 , wherein the first update rate and the second update rate are different from each other.

6. The method of claim 5 , wherein the first update rate is higher than the second update rate.

7. generating the plurality of subband audio signals, applying a spectral and / or temporal flattening to the audio signal; windowing the flattened audio signal; 7. The method of claim 1, further comprising: spectrally decomposing the windowed, flattened audio signal into the plurality of sub-band audio signals.

8. generating the plurality of subband audio signals includes spectrally decomposing the audio signal; Determining the autocorrelation function for a given subband audio signal comprises: determining a subband envelope of said subband audio signal; envelope flattening the subband audio signal based on the subband envelope; windowing the envelope-flattened sub-band audio signal with a window function; 7. A method according to any one of the preceding claims, comprising determining the autocorrelation function of the windowed envelope-flattened sub-band audio signal.

9. Determining the autocorrelation function for a given subband audio signal comprises: The method of claim 8 , further comprising: normalizing the autocorrelation function of the windowed envelope-flattened sub-band audio signal by an autocorrelation function of the window function.

10. Determining the autocorrelation function for the subband audio signal based on the autocorrelation function of the given subband audio signal comprises: comparing the autocorrelation function of the subband audio signal with an autocorrelation function of the absolute value of the impulse response of each bandpass filter associated with the subband audio signal; 10. The method of claim 1, further comprising determining the autocorrelation information based on the highest local maximum of the autocorrelation function of the subband audio signal that lies above the autocorrelation function of the absolute value of the impulse response of a respective bandpass filter associated with the subband audio signal.

11. The method of claim 1 , wherein determining the spectral envelope comprises measuring a signal power for each of the plurality of sub-band audio signals.

12. 1. A method for decoding an audio signal from an encoded representation of the audio signal, the encoded representation comprising a spectral envelope of the audio signal and autocorrelation information for each of a number of subband audio signals generated from the audio signal, the autocorrelation information for a given subband audio signal being based on an autocorrelation function of the subband audio signal, the method comprising: receiving the encoded representation of the audio signal; extracting the spectral envelope and the autocorrelation information from the encoded representation of the audio signal; determining a reconstructed audio signal based on the spectral envelope and the autocorrelation information; Including, A method according to claim 1, wherein the autocorrelation information for a given subband audio signal comprises an autocorrelation value for the subband audio signal and a lag value for each subband audio signal.

13. 13. The method of claim 12, wherein the reconstructed audio signal is determined such that the autocorrelation information for each of a plurality of subband signals generated from the reconstructed audio signal satisfies a condition derived from the autocorrelation information for a corresponding subband audio signal generated from the audio signal.

14. 14. The method of claim 13, wherein the reconstructed audio signal is determined such that autocorrelation information for each of the plurality of subband signals of the reconstructed audio signal matches the autocorrelation information for a corresponding subband audio signal of the audio signal to a predetermined margin.

15. 14. The method according to claim 12 or 13, wherein the reconstructed audio signal is determined such that, for each sub-band audio signal of the reconstructed audio signal, at the lag value indicated by the autocorrelation information for the corresponding sub-band audio signal of the audio signal, a value of the autocorrelation function of the sub-band audio signal of the reconstructed audio signal matches, to a predetermined margin, the autocorrelation value indicated by the autocorrelation information for the corresponding sub-band audio signal of the audio signal.

16. 16. The method according to any one of claims 12 to 15, wherein the reconstructed audio signal is further determined such that, for each sub-band audio signal of the reconstructed audio signal, a measured signal power of the sub-band audio signal of the reconstructed audio signal matches, to a predetermined margin, a signal power for a corresponding sub-band audio signal of the audio signal as indicated by the spectral envelope.

17. the reconstructed audio signal is determined in an iterative procedure starting from an initial candidate for the reconstructed audio signal and generating a respective intermediate reconstructed audio signal at each iteration, 17. The method of claim 12, wherein in each iteration an update map is applied to the intermediate reconstructed audio signal to obtain the intermediate reconstructed audio signal for a next iteration such that a difference between the intermediate reconstructed audio signal and the encoded representation of the audio signal becomes successively smaller from one iteration to another.

18. The method of claim 17 , wherein the initial candidates for the reconstructed audio signal are determined based on the encoded representation of the audio signal.

19. The method of claim 17 , wherein the initial candidate for the reconstructed audio signal is white noise.

20. 17. The method of claim 12, wherein determining the reconstructed audio signal based on the spectral envelope and the autocorrelation information comprises applying a machine learning based generative model that receives as input the spectral envelope of the audio signal and the autocorrelation information for each of the plurality of subband audio signals of the audio signal, and generates and outputs the reconstructed audio signal.

21. the machine learning based generative model includes a coded representation of an audio signal and a parametric conditional distribution relating a corresponding audio signal to a respective probability; The method of claim 20, wherein determining the reconstructed audio signal comprises sampling from the parametric conditional distribution over the encoded representation of the audio signal.

22. 22. The method of claim 20 or 21, further comprising, during a training phase, training the machine learning based generative model on a dataset of a plurality of audio signals and corresponding encoded representations of the audio signals.

23. 23. The method of any one of claims 20 to 22, wherein the machine learning based generative model is one of a recurrent neural network, a variational autoencoder, or a generative adversarial model.

24. Determining the reconstructed audio signal based on the spectral envelope and the autocorrelation information comprises: determining a plurality of reconstructed subband audio signals based on the spectral envelope and the autocorrelation information; determining a reconstructed audio signal based on the plurality of reconstructed subband audio signals by spectral synthesis; 14. The method of claim 13, wherein the plurality of reconstructed subband audio signals are determined such that, for each reconstructed subband audio signal, the autocorrelation function of the reconstructed subband audio signal satisfies a condition derived from the autocorrelation information for a corresponding subband audio signal of the audio signal.

25. 25. The method of claim 24, wherein the plurality of reconstructed subband audio signals are determined such that autocorrelation information for each reconstructed subband audio signal matches the autocorrelation information for a corresponding subband audio signal of the audio signal to a predetermined margin.

26. 25. The method of claim 24, wherein the plurality of reconstructed subband audio signals are determined such that, for each reconstructed subband audio signal, at the lag value indicated by the autocorrelation information for a corresponding subband audio signal of the audio signal, a value of the autocorrelation function of the reconstructed subband audio signal matches, to a predetermined margin, an autocorrelation value indicated by the autocorrelation information for the corresponding subband audio signal of the audio signal.

27. 27. The method of claim 24, wherein the plurality of reconstructed subband audio signals are further determined such that, for each reconstructed subband audio signal, a measured signal power of the reconstructed subband audio signal matches a signal power for a corresponding subband audio signal as indicated by the spectral envelope to within a predetermined margin.

28. each reconstructed subband audio signal is determined in an iterative procedure starting from an initial candidate for said reconstructed subband audio signal and generating a respective intermediate reconstructed subband audio signal in each iteration, 28. The method of claim 24, wherein in each iteration an update map is applied to the intermediate reconstructed subband audio signal to obtain the intermediate reconstructed subband audio signal for a next iteration such that a difference between the autocorrelation information for the intermediate reconstructed subband audio signal and the autocorrelation information for a corresponding subband audio signal becomes successively smaller from one iteration to another.

29. 28. The method of claim 24, wherein determining the plurality of reconstructed subband audio signals based on the spectral envelope and the autocorrelation information comprises applying a machine learning based generative model that receives as input the spectral envelope of the audio signal and the autocorrelation information for each of the plurality of subband audio signals of the audio signal, and generates and outputs the plurality of reconstructed subband audio signals.

30. 1. An encoder for encoding an audio signal, comprising: An encoder comprising a processor and a memory coupled to said processor, said processor being adapted to perform the steps of the method according to any one of claims 1 to 11.

31. 1. A decoder for decoding an audio signal from an encoded representation of said audio signal, comprising: A decoder comprising a processor and a memory coupled to said processor, said processor adapted to perform the steps of the method according to any one of claims 12 to 29.

32. A computer program comprising instructions which, when executed, cause a computer to carry out a method according to any one of claims 1 to 29.

33. 33. A computer readable storage medium having stored thereon the computer program of claim 32.

Images