Method and apparatus for improving spectral gap filling in spectral time manner using filtering in audio coding
By using spectrum tilt information, noise level information and filter intensity adjustment in audio decoder and encoder, combining prediction hysteresis information and high-frequency energy difference value, the problem of lack of flexibility in spectrum gap filling in the prior art is solved, and the problem of inflexibility in spectrum gap filling in the prior art is realized. Efficient decoding and encoding of audio information.
Patent Information
- Application Number
- CN202280092535.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-01-28
- Filing Date
- 2022-12-23
- Publication Date
- 2024-11-01
AI Technical Summary
The existing audio encoding methods lack flexibility when filling the spectrum gap, resulting in a difficult trade-off between auditory impressions and signaling workload.
By using spectrum tilt information, noise level information and filter intensity adjustment in audio decoder and encoder, the spectrum gap is filled with spectrum time, and combining predicted hysteresis information and high-frequency energy difference value, the improved reconstruction of the spectrum envelope is achieved. .
It effectively reduces the signaling and representation workload when filling the spectrum gap, improves the decoding quality and encoding efficiency of audio information, and enhances the reconstruction ability of the spectrum envelope.
Smart Images

Figure CN118891673A_ABST
Abstract
Description
Technical Field
[0001] Embodiments according to the invention relate to a method and a device for improving spectral gap filling in a spectro-temporal manner using filtering in audio coding.
[0002] Embodiments according to the invention relate to a method and a device for improving spectral gap filling in a spectro-temporal manner using different noise filling methods in audio coding.
[0003] Embodiments according to the invention relate to a method and a device for improving spectral gap filling in a spectro-temporal manner using tilting in audio coding.
[0004] Embodiments according to the invention relate to a decoder, an encoder and a method for audio coding using spectral tilt information.
[0005] Further embodiments according to the invention relate to decoders, encoders and methods for audio coding using predicted lag information and / or using lag values and gain values and / or high frequency energy values or high frequency energy difference values.
[0006] Other embodiments according to the invention relate to decoders and methods using filter strength adaptation.
[0007] Further embodiments according to the invention relate to methods and devices for improving spectral gap filling in a spectro-temporal manner in audio coding. Background Art
[0008] Conventional audio coding methods include techniques for filling zero quantized portions of a spectral range with random spectral values. As an example, a perceptual noise substitution (PNS) decoder may insert pseudo-random values into zero quantized bands, scaled so that the inserted signal energy matches a signaled target energy. However, for such methods, many bits may need to be reserved for signaling the zero quantized band energy. Furthermore, only all-zero quantized spectral bands may be replaced, so such methods may lack flexibility.
[0009] Other noise filling methods may allow replacing zero-quantized spectral coefficients after decoding with pseudo-random values above a certain "noise filling start frequency", however, this may require larger signaling overhead, especially when many frequency bands are zero-quantized.
[0010] Therefore, there is a need for concepts which produce a better compromise between the auditory impression based on the encoded audio information and the signaling effort for transmitting the encoded audio information.
[0011] This is achieved by the subject-matter of the independent claims of the present application.
[0012] Other embodiments according to the invention are defined by the subject matter of the dependent claims of the present application. Summary of the invention
[0013] Hereinafter, embodiments of the present invention are structurally explained according to various inventive aspects. However, it should be noted that the following structure is for explanatory purposes in order to facilitate understanding of the present invention.
[0014] Therefore, it should be noted that any features, functionalities and / or details of an embodiment according to any aspect of the present invention may be used and / or incorporated, alone or in combination, with any other embodiment according to the same and / or another aspect of the present invention.
[0015] In addition, some inventive embodiments will be explained in the context of a decoder, and other inventive embodiments will be explained in the context of an encoder. It should be noted that the features, functionality, and details explained in the context of a decoder may similarly be implemented in, added to, or used with a corresponding encoder, either individually or in combination. Vice versa, features, functionality, and details as disclosed for the inventive encoder may be incorporated into a corresponding decoder. Therefore, it should be noted that a decoder and a corresponding encoder (or vice versa) may be based on similar and / or equivalent inventive concepts, and may therefore include corresponding advantages.
[0016] In addition, other inventive aspects will be explained in the context of the method.It should be noted that any one of the features, functionality and / or details explained in the context of any one in encoder and / or decoder of the present invention may be incorporated into any one in the inventive method individually or in combination, or may be used together with any one in the inventive method, or may be added to any one in the inventive method.In addition, the method according to an embodiment of the present invention may be based on the same or similar or similar considerations and / or conceptions as corresponding encoder and / or decoder.Therefore, these methods may include the same or similar or similar features and advantages.
[0017] According to the above explanation, for the sake of brevity and simplicity, some features, functionalities and details may be explained or disclosed in the context of an embodiment according to a particular aspect, or an encoder rather than a decoder or vice versa, or according to a method. Therefore, in addition, it should be emphasized that any features, functionalities and / or details of an embodiment may be incorporated into any other embodiment according to the present invention, or used together with any other embodiment according to the present invention, or added to any other embodiment according to the present invention, either alone or in combination.
[0018] Aspect 1
[0019] An embodiment according to the first aspect of the invention comprises an audio decoder for providing decoded audio information based on encoded audio information, wherein the audio decoder is configured to derive spectral tilt information, such as T', from the encoded audio information. sf Furthermore, the audio decoder is configured to use a filling value (eg a gap filling coefficient; eg a noise value for noise filling; eg a gap filling value for smart gap filling) in order to fill spectral holes of the decoded set of spectral values.
[0020] Furthermore, the audio decoder is configured to apply, for example in a multiplicative manner, a frequency variable scaling whose spectral tilt is determined by spectral tilt information to such filling values, for example to noise samples replacing zero quantized samples, wherein, as an example, the spectral tilt of the frequency variable scaling is determined by the spectral tilt information.
[0021] The inventors have recognized that the quality of the decoded audio information can be improved using frequency variable scaling of fill values for filling spectral holes of a decoded set of spectral values. Therefore, an audio decoder according to an embodiment of the present invention can be configured to derive spectral tilt information from the encoded audio information based on a determinable frequency variable scaling.
[0022] For example, a main idea of embodiments according to the first aspect of the invention is to calculate and, for example, signal at a low bit rate, for example in the logarithmic intensity domain, a difference curve between a (for example, true) spectral envelope of a frame (and / or a subframe) (for example, its input signal envelope) and a masking envelope of the frame (and / or a subframe) (for example, its noise shaping envelope). Since the masking envelope can be transmitted to a decoder, for example, the additional transmission of the difference can allow reconstruction of the (for example, true) spectral envelope from the masking envelope and the difference curve in a spectral hole filling procedure (for example, in a gap or noise filling decoding procedure). Therefore, according to the invention, the difference curve can be characterized by spectral tilt information. Using the corresponding inventive decoder, good accuracy and / or quality of the audio information can be achieved, for example, with a few side information bits.
[0023] The spectrum tilt information may be, for example, frame-by-frame and / or subframe-by-subframe spectrum tilt information. As an example, the spectrum tilt information may include a tilt index, such as t sf As an example, the information T' can be determined based on the tilt index sf , where T' sf A frequency dependent term (eg f) may be multiplied, for example, in order to apply a frequency variable scaling to the padding value. Optionally, explicit transmission of the target energy in the zero quantized non-overlapping frequency range may not be transmitted, thus keeping the signaling effort at a low level.
[0024] The inventors recognized that using spectral tilting, the spectral envelope of audio information can be recovered from a masking envelope of the audio information (e.g., a noise shaping envelope, e.g., a masking envelope corresponding to or associated with scaling values or scaling factors of frames and / or subframes) with only a few additional signaling bits.
[0025] According to further embodiments according to the first aspect of the invention, the audio decoder is configured to derive noise level information (eg L sf ), and the audio decoder is configured to use this noise level information, for example in addition to the frequency variable scaling, in order to obtain a filling value.
[0026] For example, noise level information (e.g., L sf ) can be derived or reconstructed from the noise level index, for example, N-bit noise level index 0≤l sf <2 N For example, the noise level information and / or the noise level index may be transmitted from the corresponding encoder to the decoder. As an example, the noise level information and / or the noise level index may, for example, include information about spectral tilt information (e.g., other information about the difference curve), such as an offset, e.g. sf In other words, the decoder may be configured to derive information about the spectral tilt information (eg, ie, further information about the difference curve) from the noise level information and / or from the noise level index.
[0027] The inventors have realised that using noise level information for decoding may allow determining an improved filling value, eg allowing a good reconstruction of the encoded audio signal.
[0028] According to other embodiments of the first aspect of the invention, the audio decoder is configured to apply the frequency variable scaling so that the frequency variable scaling describes a linear decrease in intensity (e.g., a fill value) with increasing frequency on a logarithmic intensity scale, e.g., within a tolerance of + / -3dB or + / -2dB or + / -1dB.
[0029] The inventors have realized that using a logarithmic intensity scale or by a linear decrease in intensity with increasing frequency, an improved reconstruction of the spectral envelope of the audio information can be achieved. As an example, the effect of a pre-emphasis tilt applied during the calculation of the masking envelope of the audio information can be compensated so that the spectral envelope can be at least approximately restored.
[0030] According to further embodiments according to the first aspect of the invention, the spectral tilt information describes the spectral tilt in the logarithmic domain, eg wherein the spectral tilt (eg the spectral tilt information) may eg be used in the logarithmic domain and / or the linear domain.
[0031] It should be noted that embodiments according to the present invention are not limited to spectral tilt information in the logarithmic domain. Spectral tilt information can be used, for example, in the logarithmic domain and / or the linear domain. Use in the logarithmic domain can allow the calculation of (e.g., spectrally tilted) fill values at low computational cost.
[0032] According to other embodiments of the first aspect of the present invention, the spectral tilt information describes a line function with spectral tilt in the logarithmic domain.
[0033] The inventors have realised that a function of this form in the logarithmic domain with a spectral tilt allows efficient decoding of audio information with good accuracy.
[0034] According to further embodiments according to the first aspect of the invention, the audio decoder is configured to obtain the scaling value for the frequency variable scaling in the logarithmic domain, and the audio decoder is configured to obtain the scaling value for the frequency variable scaling, for example using an exponential function (for example using an exponential function with base 10; for example using a function of the form 10 x A function of ( ) converts the scaling value used for frequency variable scaling from the logarithmic domain to the linear domain.
[0035] According to an embodiment of the present invention, the computational domain (e.g., logarithmic domain or linear domain) can be, for example, changed or adapted to different processing steps. The inventors have realized that such switching or changing of domains can improve the flexibility of the audio coding concept of the present invention. In addition, the computational cost can be reduced by performing different processing steps in the corresponding suitable domain.
[0036] According to other embodiments of the first aspect of the present invention, the audio decoder is configured to determine the value of the tilt information based on the tilt value (eg, T′) of the audio decoder. sf ) and a frequency value (eg, f, such as a frequency value describing the frequency or a frequency value describing a frequency offset relative to a reference value) to obtain a scaling value for frequency variable scaling.
[0037] As an example, the tilt value may be scaled, for example, by a constant (eg, an additional constant) in order to maintain noise level information (eg, L sf The inventors have realized that the scaling value for frequency variable scaling can be obtained with low computational effort, for example using the product of the tilt value and the frequency value.
[0038] According to further embodiments according to the first aspect of the invention, the audio decoder is configured to obtain a plurality of scaling values for frequency variable scaling associated with different frequency bands, eg such that scaling values are associated with different frequency bands.
[0039] The inventors have realised that using scaling values associated with different frequency bands the decoding of audio information may be improved, eg in terms of complexity or flexibility.
[0040] According to other embodiments of the first aspect of the present invention, the audio decoder is configured to obtain a scaling value for frequency variable scaling using the start frequency of the corresponding frequency band or using the center frequency of the corresponding frequency band; wherein, for example, the scaling value associated with the first frequency band is obtained by multiplying the (e.g., lower) start frequency of the first frequency band by a tilt value, and wherein, for example, the scaling value associated with the second frequency band is obtained by multiplying the (e.g., lower) start frequency of the second frequency band by a tilt value; or wherein, for example, the scaling value associated with the first frequency band is obtained by multiplying the center frequency of the first frequency band by a tilt value, and wherein, for example, the scaling value associated with the second frequency band is obtained by multiplying the center frequency of the second frequency band by a tilt value.
[0041] It should be noted that embodiments according to the present invention are not limited to a specific selection of frequency representations of the respective frequency bands. As explained before, a starting frequency and / or a center frequency may be used. However, other (e.g., application specific) advantageous selections of frequency band information may be implemented. Therefore, the inventive concept according to an embodiment may provide high flexibility.
[0042] According to other embodiments of the first aspect of the present invention, the audio decoder is configured to obtain a scaling value for frequency variable scaling using a start frequency interval index of the corresponding frequency band or using a center frequency interval index of the corresponding frequency band; wherein, for example, the scaling value associated with the first frequency band is obtained by multiplying the (e.g., lower) start frequency interval index of the first frequency band by a tilt value, and wherein, for example, the scaling value associated with the second frequency band is obtained by multiplying the (e.g., lower) start frequency interval index of the second frequency band by a tilt value; or wherein, for example, the scaling value associated with the first frequency band is obtained by multiplying the center frequency interval index of the first frequency band by a tilt value, and wherein, for example, the scaling value associated with the second frequency band is obtained by multiplying the center frequency interval index of the second frequency band by a tilt value.
[0043] The inventors have recognized that using frequency bin indices (eg, instead of frequency values) may allow for reduced computational costs.
[0044] According to other embodiments of the first aspect of the invention, the audio decoder is configured to use the noise intensity information (eg, L sf ) to obtain the filling value; for example, using a frequency-independent noise scaling value, which frequency-independent noise scaling value can be derived, for example, from the encoded audio information; the encoded audio information can be derived, for example, from l sf Export.
[0045] Thus, the audio decoder may for example be configured to determine or obtain a fill value using noise level information and / or noise intensity information. Optionally, the decoder may for example be configured to derive noise intensity information. In some applications, the noise level information may for example be equal to the noise intensity information.
[0046] According to other embodiments of the present invention, the audio decoder is configured to use the noise value, the frequency-independent noise scaling value (eg L sf ) is compared with the frequency-variable noise scaling value determined by taking the spectrum tilt into account (e.g., 10 T ' sf*f ) to obtain the filling value; wherein the noise value is a random noise value or a pseudo-random noise value, for example, having a predetermined amplitude or having an amplitude within a predetermined amplitude range.
[0047] The inventors have recognized that adapting noise values using frequency-independent noise scaling values and frequency-variable noise scaling values based on spectral tilting can improve the quality of decoded audio information with only limited impact on the complexity of the corresponding decoder. The frequency-variable noise scaling values can allow shaping (e.g., tilting with respect to frequency) the masking envelope of the audio information so as to better approximate the spectral envelope of the originally encoded audio information.
[0048] According to further embodiments of the first aspect of the invention, the audio decoder is configured to apply a masking envelope based scaling, e.g. to non-zero decoded spectrum values and to filling values, e.g. so that in practice the masking envelope is applied to the complete spectrum, optionally including the filling values.
[0049] The inventors realised that, for example, application of the inventive scaling may improve decoded audio information when not only padding values but also other decoded spectral values are affected by the scaling.Thus, the decoded spectrum of the audio information may, for example, be adapted, for example tilted depending on frequency.
[0050] Further embodiments according to the first aspect of the invention include an audio encoder for providing encoded audio information based on input audio information, wherein the audio encoder is configured to encode a plurality of quantized spectral values, and wherein the audio encoder is configured to determine spectral tilt information (e.g., spectral tilt information depicting a line function with spectral tilt in a logarithmic domain, e.g., wherein the spectral tilt (e.g., spectral tilt information) can be used, e.g., in the logarithmic domain and / or in the linear domain) based on spectral energy information of, e.g., a spectral envelope and masking envelope information, e.g., such that the spectral tilt information describes an average frequency variation of a difference between the spectral energy and the masking envelope. Furthermore, the audio encoder is configured to encode the spectral tilt information.
[0051] As explained above, the spectral tilt information may describe the shape difference between the spectral energy of the audio information and the masking envelope used to encode the audio information. This shape difference may be expressed, for example, in the form of a frequency-dependent tilt (in the frequency-amplitude plane). Thus, the spectral tilt information may be transmitted to a corresponding decoder and the spectral tilt information may, for example, be used as a correction factor to adapt the transmitted masking envelope in order to better reconstruct the spectral envelope of the audio information.
[0052] According to other embodiments of the first aspect of the invention, the audio encoder is configured to determine the spectral tilt information such that the spectral tilt information describes a difference between spectral energy information (e.g. a "true spectral envelope" or a smoothed (e.g. in frequency direction) version of the spectral values) and masking envelope information (e.g. expressed as a scaling factor or one or more prediction coefficients) that varies with frequency, such as such that the tilt information describes an average value of the frequency variation, or, for example, such that the tilt information describes the tilt of a (e.g. linear) regression line of the difference between the spectral energy information and the masking envelope information with frequency.
[0053] As explained before, the idea according to an embodiment of the invention is to calculate and signal (e.g. in the logarithmic intensity domain) at a low bit rate the frequency variations between the (e.g. true) spectral energy of a frame (and / or a subframe) (e.g. the spectral envelope, e.g. the input signal envelope thereof) and the masking envelope of the frame (and / or a subframe), e.g. a difference curve. This information may be transmitted using spectral tilt information. Thus, as an example, by providing the masking envelope and the spectral tilt information and therefore information about the difference curve, a reconstruction of the spectral energy of the audio information may be performed with good accuracy and with a low signaling effort. This may in particular improve the reconstruction of zero-quantized spectral coefficients, since the noise-filled or spectral gap-filled coefficients (in the corresponding decoder) may be adapted or corrected, e.g. using the spectral tilt information, thus reducing the difference between the "original" spectrum and the reconstructed or decoded spectrum of the audio information.
[0054] According to other embodiments according to the first aspect of the invention, the spectral tilt information describes a line function with a spectral tilt in the logarithmic domain. The inventors have realized that this may allow signaling correction information for the masking envelope, better approximating the original spectrum of the audio information with fewer signaling bits and good accuracy.
[0055] According to further embodiments of the first aspect of the invention, the audio encoder is configured to determine spectral tilt information in the logarithmic domain, for example using a logarithmic (e.g. frequency-dependent) representation of the spectral energy information and for example using a logarithmic (e.g. frequency-dependent) representation of the masked envelope information.
[0056] As explained previously, the inventors have realised that the determination of spectral tilt information in the logarithmic domain can be performed computationally efficiently.
[0057] According to other embodiments of the first aspect of the invention, the audio encoder is configured to determine spectral tilt information based on a difference between a logarithmic representation of a spectral envelope (e.g., varying with frequency) that may constitute spectral energy information and a logarithmic representation of a masked envelope (e.g., varying with frequency) that may constitute masked envelope information (e.g., a difference curve (f) = true spectral envelope (f) - masked envelope (f), e.g., a frequency-dependent difference).
[0058] The inventors have realised that the determination of spectral tilt information in the logarithmic domain may be performed, for example, as a difference operation which is simple and can be implemented efficiently.
[0059] According to further embodiments of the first aspect of the invention, the audio encoder is configured to obtain spectral tilt information using linear regression, wherein the spectral tilt information may, for example, be regression coefficients obtained by linear regression of the evolution of the difference between a (true) spectral envelope and a masking envelope with frequency, for example in the logarithmic intensity domain.
[0060] The inventors have realized that linear regression may allow approximation of a correction term or a difference term or a (e.g. monotonic) difference curve between a (e.g. true) spectral envelope and a masking envelope with limited complexity and good approximation results. Based on the correction term or the difference term or the e.g. monotonic difference curve, spectral tilt information may be obtained, for example. Optionally, the correction term or the difference term or the e.g. monotonic difference curve may be spectral tilt information.
[0061] According to other embodiments of the first aspect of the present invention, the audio encoder is configured to obtain the spectral tilt information based on a spectral band-by-spectral band (e.g., summed) energy value or a spectral band-by-spectral band root mean square value representing the energy of the spectral values in multiple corresponding spectral bands and based on a spectral band-by-spectral band (e.g., summed) energy value or a spectral band-by-spectral band root mean square value representing, for example, an energy level of a masking threshold in multiple corresponding spectral bands.
[0062] The inventors have realized that the use of a representation of per-spectral band energy values (e.g., sum of squares) or root mean square (RMS) values for obtaining spectral tilt information may allow the computational complexity to be kept low. However, embodiments are not limited to the use of such representations, so per-transform coefficient values may also be used.
[0063] According to further embodiments of the first aspect of the invention, the audio encoder is configured to determine separate spectral tilt information, such as separate spectral tilt values, for different audio frames and / or for different audio subframes.
[0064] The inventors recognized that frame-by-frame or sub-frame-by-sub-frame spectral tilt information may allow determining effective correction information, such as spectral tilt to be transmitted to a corresponding decoder in order to improve the fitting of a decoded spectrum of the audio information to the original spectrum of the audio information.
[0065] According to further embodiments according to the first aspect of the invention, the audio encoder is configured to determine a difference value (e.g., 0) representing the difference between the spectral energy information and the masking envelope information in the form of a single value over a frequency range comprising a plurality of spectral intervals (e.g., over a frequency band, or even over a plurality of spectral bands, or even over all frequency bands). sf or T sf , for example, the offset value O sf ; or for example, the tilt value T sf , for example quantize to the value t sf , and / or values which may be transmitted, for example, to a noise filling decoder, and / or which may be used, for example, in a negated form, in a noise filling encoder). Furthermore, the audio encoder is configured to obtain noise level information, which may, for example, describe the noise level on a plurality of spectral bands or even on all frequency bands depending on the difference value, for example, sf .
[0066] For example, O sf may be an offset, which may not actually be needed or may not need to be encoded (but may optionally be used). sf can be quantized (e.g., t sf ) and a value that can be transmitted, and which can be used (e.g., in negated form) for example in a noise filling encoder (and / or in a noise filling decoder).
[0067] The inventors have recognized that using a single difference value may provide a good compromise between (e.g., signaling or transmission) complexity and reconstruction accuracy of the audio information. As an example, tilt information may be determined that may describe the tilt of a masking envelope in frequency relative to the original spectrum of the audio information. Thus, a decoder-side correction of zero-quantized spectral coefficients based on padding values, e.g., adapted according to the masking envelope and corrected using the tilt information, may allow reconstruction of valid audio information.
[0068] According to other embodiments of the first aspect of the invention, the audio encoder is configured to use linear regression, for example, using the linear regression mentioned above to obtain a difference value (e.g., Osf or Tsf, for example, an offset value Osf; or for example a tilt value Tsf, for example, quantized, for example, to a value tsf, and / or which can, for example, be transmitted to, for example, a noise filling decoder, and / or which can, for example, be used in, for example, a negated form, in a noise filling encoder).
[0069] The inventors have recognized that, for example, in many applications, the difference between the original (e.g., "real") audio signal spectral envelope and the masking envelope may include a substantially linear (e.g., in the logarithmic frequency domain) characteristic. In other words, the intensity difference between the real spectral envelope and the masking envelope may vary monotonically with frequency. For example, in the logarithmic intensity domain (e.g., logarithm to base 10) and in gap or noise-filled spectral regions, the monotonic difference curve may resemble a straight line most of the time. Thus, using linear regression may allow the corresponding difference values to be approximated with low computational cost and good accuracy.
[0070] According to further embodiments of the first aspect of the invention, the audio encoder is configured to encode the spectral tilt information using three bits.
[0071] This may allow a good trade-off between the number of signaling bits and the accuracy of the spectral tilt information.
[0072] According to further embodiments according to the first aspect of the invention, the audio encoder is configured to encode the spectral tilt information such that the encoded spectral tilt information always represents a negative spectral tilt, eg decreasing with increasing frequency.
[0073] The inventors realized that a negative spectral tilt may, for example, allow for a good adaptation or correction or improvement of the reconstructed audio information.As an example, correction of padding values with negative spectral tilt information may compensate for undesirable effects of pre-emphasis.
[0074] According to further embodiments according to the first aspect of the invention, the audio encoder is configured to perform the following functionality for one or more frames or subframes sf (e.g., audio frames or audio subframes):
[0075] 1. Calculate the energy value or RMS value E per spectral band from the input (e.g. uncoded) spectrum sf (f);
[0076] 2. Set one or more values E sf (f) Convert to the logarithmic domain and from the value E sf (f) or, for example, subtracting a plurality of values E from their logarithmic versions sf (f) or, for example, its logarithmic version of the overall mean, to obtain a zero mean E' sf (f);
[0077] 3. From zero mean value E' sf Calculate, quantize and dequantize the mask envelope M sf ;
[0078] 4. From M sf Reconstruct the energy value or RMS value of each spectrum band, and from M sf Derive the logarithm (or, e.g., logarithmization) and zero mean M' sf(f);
[0079] 5. In paired spectral bands E' sf With M' sf Perform linear regression between sf and offset O sf ;
[0080] 6. From T sf Quantize and dequantize the tilt index t sf ;
[0081] 7. From t sf Reconstruct the tilt value to obtain the decoded tilt T' sf , and use -T' sf *f is used to calculate the noise level index I sf .
[0082] The inventors have realised that the above described functionality may allow for efficient encoding of audio information.
[0083] Other embodiments according to the first aspect of the invention include a method for providing decoded audio information based on encoded audio information, the method comprising: deriving spectral tilt information, such as T', from the encoded audio information. sf ; and using a filling value (e.g., a gap filling coefficient; e.g., a noise value for noise filling; e.g., a gap filling value for smart gap filling) to fill spectral holes of the decoded set of spectral values. The method further comprises applying, e.g., multiplicatively, a frequency variable scaling in which the spectral tilt is determined by the spectral tilt information to the filling value, e.g., to the noise samples replacing the zero quantized samples.
[0084] Other embodiments according to the first aspect of the invention include a method for providing encoded audio information based on input audio information, the method comprising: encoding a plurality of quantized spectral values; and determining spectral tilt information based on spectral energy information (e.g., spectral envelope) and masking envelope information, such that the spectral tilt information describes an average frequency variation of a difference between the spectral energy and the masking envelope. The method further comprises encoding the spectral tilt information.
[0085] Further embodiments according to the first aspect of the invention comprise a computer program for performing any of the above methods when the computer program is run on a computer.
[0086] Aspect 2
[0087] Embodiments according to the second aspect of the invention comprise an audio decoder for providing decoded audio information based on encoded audio information, wherein the audio decoder is configured to fill spectral holes of a set of decoded spectral values, e.g. using replacement of spectral coefficients quantized to zero based on corresponding filling values. Furthermore, the audio decoder is configured to obtain prediction lag information, e.g. a frequency domain long prediction lag value p, e.g. from a bit stream or from the encoded audio information. sf For example, prediction lag information indicating a prediction period in the frequency direction, such as a spectrum (LTP, for example, long-term prediction) distance value p sf .
[0088] Furthermore, the audio decoder is configured to select a first spectrum filling method (eg “noise filling” + FD LTP) depending on the prediction lag information, for example if p sf is not zero) and one or more other spectrum filling methods (e.g., the second spectrum filling method or the third spectrum filling method, such as "noise filling" without FD-LTP; such as "gap filling", such as if p sf is zero), in which the first spectrum filling method uses frequency filtering or frequency prediction (e.g., TNS or LTP (e.g., filtering in which spectral values associated with a first frequency have an influence on spectral values associated with a second frequency) to obtain filling values for filling the spectrum holes, and in which the one or more other spectrum filling methods, frequency filtering and frequency prediction are not used (e.g., neither frequency filtering nor frequency prediction is used) to obtain filling values for filling the spectrum holes.
[0089] The idea of an embodiment according to the second aspect of the invention is to adaptively switch between a first spectrum filling method (e.g. a noise filling solution) and a second (or multiple second, e.g. second and third) spectrum filling method (e.g. a gap filling solution), e.g. based on signal characteristics of a (e.g. a sub)frame. In addition, the first spectrum filling method may include frequency filtering or frequency prediction, such as frequency domain long-term prediction (FD-LTP), and the second spectrum filling method may not include frequency filtering and frequency prediction. Thus, the decoder may, for example, switch between different methods for generating "artificial" spectral content for filling zero quantized spectral coefficients.
[0090] The inventors have realized that switching between different filling methods may improve the reconstructed (e.g., decoded) audio signal. Furthermore, the inventors have found that prediction lag information may allow controlling the adaptation of different filling methods with only limited impact on the signaling representation complexity. As an example, a decoder according to an embodiment may be configured to depend on the prediction lag information, for example, on the FD-LTP lag value p sfAnd switch or select between noise filling with FD-LTP and tonality-based gap filling without FD-LTP (e.g., similar to IGF in EVS) or noise filling without FD-LTP (e.g., similar to noise filling in EVS or MPEG-D).
[0091] As another optional feature, the prediction hysteresis information may, for example, comprise only integer values in order to reduce computational complexity.
[0092] Hence, the inventive coding concept according to the second aspect of the present invention may provide good flexibility, eg when switching or selecting spectral hole filling methods, in order to achieve better coding efficiency for the audio information.
[0093] The prediction lag information may comprise information about the relationship between (e.g. zero quantized) spectral coefficients of different frequency blocks. The prediction lag information may comprise information about periodicity or about a reduction of the spectral coefficients. Thus, the prediction lag information may for example be an indicator whether the relationship between (e.g. zero quantized) spectral coefficients is sufficient or suitable in order to reconstruct or approximate the spectral coefficients depending on the corresponding related spectral coefficients. In this case, for example, a good auditory impression may be achieved, but bits may be saved.
[0094] In particular, as an example, the inventors have experimentally found that, for example, applause-like, rain-like and low-frequency LF male speech signals may benefit from an improved reconstruction of the high-frequency HF fine time signal envelope during spectral hole filling (e.g. gap or noise filling) at the decoder side. For such signals, which may for example be detected and classified as "long-term transients" (e.g. classified using prediction lag information), the fine time structure of a specific (e.g. sub-)frame may be parameterized by prediction lag information, e.g. frequency domain long-term prediction (FD-LTP) information. Similar to conventional LTP pitch and gain information obtained, e.g. in the time domain (TD), as an example, the prediction lag information lag and gain values (e.g. FD-LTP lag and gain values) may be obtained (e.g. directly) in the transform domain of the audio codec. The choice of spectral hole filling to be applied in the decoder may depend on said prediction lag information (e.g. FD-LTP lag p or p) transmitted in the audio bitstream. sf ) value and is signaled to the decoder.
[0095] According to other embodiments of the second aspect of the present invention, the audio decoder is configured to predict the lag information (eg, the prediction lag value, eg, the quantized FD LTP lag value, eg, the long-term prediction lag value; eg, p sf ) is non-zero, (e.g. selectively) using the first spectrum filling method. Alternatively or, for example, in addition, the audio decoder is configured to predict the lag information (e.g., the predicted lag value; e.g., the quantized FD LTP lag value; e.g., p sf) is greater than zero, and (e.g. selectively) uses the first spectrum filling method. In addition, the audio decoder is configured to, for example, use one of the one or more other spectrum filling methods (e.g. selectively) when the prediction lag information is zero or the prediction lag information is less than or equal to zero.
[0096] The inventors have realized that the prediction lag information may allow a simple distinction of implementation situations. As an example, in the case where the relationship between the (e.g. zero quantization) spectral coefficients brings advantages for the reconstruction (e.g. better reconstruction of the time signal), the prediction lag information may be non-zero or greater than zero. Otherwise, the decoder may, for example, use a second spectral filling, which may be associated with a small dependency between the spectral coefficients, e.g. in the case where the prediction lag information is zero.
[0097] According to further embodiments of the second aspect of the invention, the audio decoder is configured to use an encoded representation (eg a quantized and encoded representation) of the prediction lag value comprised in the encoded audio information in order to obtain the prediction lag value.
[0098] According to other embodiments of the second aspect of the present invention, when the first spectrum filling method is used, the audio decoder is configured to use prediction or filtering (eg using the calculation rule d*c(i)+G' sf *c(iP' sf )) determine the (e.g. final) fill value (e.g. for c(i) replacement, e.g. ), so that a given fill value (e.g. associated with a given frequency (e.g. with a given frequency interval) ) is dependent on different frequencies (eg, different frequency intervals, eg, with a frequency interval index iP' sf different frequency intervals, such as the spectral distance P' from a given frequency or a given frequency interval sf Or the spectrum distance d sf Another spectral value (e.g., c(iP')) associated with a frequency or frequency interval sf )or ) and obtain it.
[0099] Furthermore, when the first spectrum filling method is used, the audio decoder is configured to depend on different frequencies (eg, different frequency intervals; eg, different frequency intervals with a frequency interval index iP′). sf Adapting the filtering strength (e.g., weighting of the spectral values associated with different frequencies, e.g., by selectively setting the filtering strength to G') based on the encoded or quantized spectral values associated with different frequency intervals (e.g., initially determined by the encoded representation of the individual spectral values in the encoded audio information, e.g., determined by the spectral values before applying the noise filling, e.g., determined by the spectral values directly after arithmetic decoding).sf or 1 / 2G' sf ).
[0100] As explained previously, for example where the prediction lag information is non-zero, a fill value associated with a given frequency may be determined or obtained or calculated based on or using spectral values associated with different frequencies, and thus, as an example, indicate the transient nature of the signal. Furthermore, the inventors have recognized that the decoding and / or reconstruction of audio information may be improved by adapting the filtering strength depending on the encoded or quantized spectral values associated with different frequencies.
[0101] As an example, in case a first spectral filling method is selected, for example in case noise filling with FD-LTP is selected (for example, if the prediction lag information is non-zero, as an example, if the FD-LTP lag is non-zero), applying a long-term predictive filter in the spectral domain (for example, the MDCT domain) of the audio transform codec may be performed during the decoder-side noise filling routine (for example, depending on whether the "current" encoded FD coefficient is zero and on whether the corresponding "previous" encoded FD coefficient located at a certain distance from the current coefficient (specified by the transmitted prediction lag information, for example, specified by the transmitted FD-LTP lag) is zero).
[0102] As an example, an infinite impulse response (IIR) LTP type filter may be used for filtering.
[0103] According to other embodiments of the second aspect of the present invention, the filtering strength is determined, for example, by sf )'s effect on a given fill value.
[0104] The inventors have recognised that adapting the effect of another spectral value on a given padding value based on the filtering strength may improve the quality of the decoded audio information.
[0105] According to further embodiments of the second aspect of the invention, the audio decoder is configured to adapt the filtering strength depending on spectral values associated with different frequencies (eg initially determined by an encoded representation of the respective spectral values in the encoded audio information).
[0106] The inventors have realised that using a value represented by the encoded representation for adapting the filter strength allows using or exploiting information provided by the encoded representation rather than a filtered version thereof (which may for example be alternating). It has been found that using such a criterion the selection of the filter strength is more reliable than using a criterion that depends on a value that has been pre-processed on the decoder side.
[0107] According to further embodiments of the second aspect of the invention, the audio decoder is configured to adapt the filtering strength depending on spectral values associated with different frequencies before applying the noise filling.
[0108] The inventors realized that using spectral values associated with different frequencies before noise filling may allow adapting the filter strength based on information whether the spectral value is quantized to zero or not.
[0109] According to further embodiments of the second aspect of the invention, the audio decoder is configured to adapt the filtering strength depending on whether spectral values associated with the different frequencies (or values) are quantized to zero or not.
[0110] According to further embodiments according to the second aspect of the invention, the audio decoder is configured to adapt the filtering strength depending on whether noise filling is applied to spectral values associated with different frequencies (or values).
[0111] The inventors have realised that using such criteria, for example, filtering strength adaptation may be performed based on information whether a corresponding spectral value has been quantised to zero, in addition to whether noise filling is attempted or performed for the corresponding frequency of the spectral value. This may include using a flag.
[0112] According to other embodiments of the second aspect of the invention, the audio decoder is configured to, for the spectral values for which noise filling is applied (eg for the position i>=P′ sf Each noise-filled zero-quantized spectral coefficient at c) selectively applies filtering in the frequency direction or prediction in the frequency direction.
[0113] The inventors have realized that, as an example, zero-quantized spectral values may be approximated or estimated based on or using filtering or prediction in the frequency direction.Thus, a dependency of spectral values in the frequency direction may be exploited, for example, among different spectral values.
[0114] According to further embodiments according to the second aspect of the invention, the audio decoder is configured to apply prediction or filtering in order to determine a given (e.g. final) filling value based on a random or pseudo-random noise value (e.g. c(i)), e.g.
[0115] The inventors have recognised that random or pseudo-random noise values may be adapted, e.g. using prediction or filtering, in order to compute (e.g. final) filling values that may provide a good approximation of zero-quantised spectral values for a (e.g. original), e.g. input spectrum of the audio information.
[0116] According to further embodiments of the second aspect of the invention, the audio decoder is configured to perform a weighted combination of a noise value associated with a given frequency and a noise value associated with another frequency in order to obtain a given (e.g. final) filling value, e.g.
[0117] As an example, the audio decoder may be configured to perform the combination d*c(i)+G' sf *c(iP'sf ), where the weight d of the noise value c(i) is associated with a given frequency, and the weight of the noise value G' sf Associated with another frequency, or perform the combination d*c(i)+1 / 2*G' sf *c(iP' sf ), where the weight d of the noise value c(i) is associated with a given frequency, and the weight of the noise value is 1 / 2*G' sf Associated with another frequency.
[0118] Alternatively, the audio decoder is configured to perform a weighted combination of a noise value associated with a given frequency and a filling value associated with another frequency in order to obtain a given (eg final) filling value, e.g.
[0119] Furthermore, the audio decoder is configured to adjust the weight (eg G′) given to the noise value associated with the other frequency depending on whether noise filling has been applied to the spectral value associated with the other frequency. sf or 1 / 2*G' sf ) or a weight given to a fill value associated with another frequency (e.g. G' sf or 1 / 2*G' sf ).
[0120] As explained above, the inventors recognize that the (e.g. final) filling value can be calculated, for example, using different frequency-dependent amounts of a noise value associated with a given frequency and / or associated with another frequency and / or a filling value associated with another frequency. Thus, the inventive concept can allow determining or obtaining or calculating a (e.g. final) filling value with good flexibility, so that, depending on the specific situation, a filling value can be obtained that is well or even optimally suitable for the reconstruction of the (e.g. original) audio information spectrum. For example, the selection of the corresponding amount to be used to obtain the (e.g. final) filling value can be performed based on the prediction lag information. Furthermore, the inventors recognize that the adaptation or adjustment of the corresponding weights of the corresponding noise value or filling value associated with another frequency can improve the determination of the (e.g. final) filling value, and therefore improve the reconstruction of the audio information.
[0121] According to further embodiments of the second aspect of the invention, the audio decoder is configured to perform a weighted combination of a noise value associated with a given frequency and a noise value associated with another frequency in order to obtain a given (e.g. final) filling value, e.g.
[0122] As an example, the audio decoder may be configured to perform the combination d*c(i)+G' sf *c(iP' sf ), where the weight d of the noise value c(i) is associated with a given frequency, and the weight of the noise value G'sf Associated with another frequency, or perform the combination d*c(i)+1 / 2*G' sf *c(iP' sf ), where the weight d of the noise value c(i) is associated with a given frequency, and the weight of the spectrum (or noise) value is 1 / 2*G' sf Associated with another frequency.
[0123] Alternatively, the audio decoder is configured to perform a weighted combination of a noise value associated with a given frequency and a filling value associated with another frequency in order to obtain a given (eg final) filling value, e.g.
[0124] Furthermore, the audio decoder is configured to adjust a weight (eg G′) given to a noise value associated with another frequency or given to a spectral value associated with another frequency depending on whether noise filling has been applied for the spectral value associated with another frequency. sf or 1 / 2*G' sf ) or adjust the weight (eg, G') given to a fill value associated with another frequency or given to a spectral value associated with another frequency sf or 1 / 2*G' sf ).
[0125] According to further embodiments according to the second aspect of the invention, the audio decoder is configured to determine a spectral distance, e.g. P′, between a filling value associated with a given frequency and another spectral value associated with a different frequency based on encoded information describing the spectral distance included in the encoded representation of the audio information (e.g. an encoded value). sf (For example, based on P' sf The spectrum distance d sf ).
[0126] As an example, fill values (e.g., noise samples, such as those replacing zero quantized samples) may be used. ) is filtered so that the filtering strength depends on the spectral distance d from i sf The quantized value c(id sf ). In the case of using FD-LTP, d sf Can be equal to P' sf .
[0127] The inventors have realised that the spectral distance may be used, for example, in order to improve the determination of the spectral filling value.
[0128] According to further embodiments of the second aspect of the invention, the audio decoder is configured to generate a gain signal based on gain information (eg gain values, eg g) included in the encoded representation of the audio information. sf) determines a weight (e.g., d) applied to a noise value associated with a given frequency, wherein the weight applied to the noise value associated with a given frequency is a positive value, such as in a range between 0.5 and 1.
[0129] The inventors have appreciated that such determination and application of weights may, for example, allow the noise value associated with a given frequency to be adjusted so as to better approximate the original spectral envelope of the audio information.
[0130] According to further embodiments of the second aspect of the invention, the audio decoder is configured to, depending on gain information (eg gain values, eg g) included in the encoded representation of the audio information, sf ) determines the noise value (e.g., c(iP') applied to another frequency sf )) or a weight applied to a fill value associated with another frequency (e.g. G sf =(-1) Ssf *(3+2*g sf ) / 8 or 1 / 2*G sf ), where the weight applied to the noise value associated with a given frequency is, for example, a positive or negative value, for example having an absolute value between 0.25 and 0.75.
[0131] The inventors have realised that a corresponding noise value or a corresponding filling value associated with another frequency may be adapted, for example, with a weight determined depending on the gain information. This may allow shaping of the noise or filling value to improve its match with the corresponding spectral value of the originally encoded audio information.
[0132] According to further embodiments of the second aspect of the invention, the audio decoder is configured to determine, depending on sign information (eg sign values, eg S ) included in the encoded representation of the audio information. sf ) and determine the noise value (eg, c(iP') applied to another frequency sf )f) or a weight applied to a fill value associated with another frequency (e.g. G sf =(-1) Ssf *(3+2*g sf ) / 8 or 1 / 2*G sf ).
[0133] The inventors have recognized that, for example, using sign information, such as 1-bit information, can, for example, improve weight determination. As an example, the sign information can allow adaptation of the (e.g. final) fill value relative to a phase relationship of a noise value and / or fill value associated with another frequency on which it may be based.
[0134] According to other embodiments of the second aspect of the invention, the audio decoder is configured to sf) is obtained using noise filling (e.g. at spectral position iP' sf The coefficient c(iP' sf ) in the case of being marked as noise-filled zero-quantized spectral coefficients in the previous processing) according to Determines the given fill value And in the coefficient c(iP' sf ) is not obtained using noise filling (e.g. at spectral position iP' sf The coefficient c(iP' sf ) in the case where it is not marked as noise-filled zero-quantized spectral coefficients in the previous processing) according to Determines the given fill value c(i) represents the spectral coefficient with spectral index i obtained using noise filling, d represents the attenuation coefficient, G' sf represents a weight based on a gain value included in the encoded audio representation; and c(iP' sf ) represents a spectrum index iP' sf spectral coefficients (which may be obtained, for example, using noise, or which may be obtained, for example, without using noise filling, and which may be obtained, for example, using prediction or filtering), where P' sf are prediction parameters or filter parameters based on prediction parameter information included in the encoded audio representation.
[0135] The inventors have realised that using the above equation, an effective fill value can be determined.
[0136] According to other embodiments of the second aspect of the present invention, the audio decoder is configured to s f=p sf +B obtains prediction parameters or filtering parameters P' sf , where p sf is a lag index included in the encoded audio representation, and wherein B is a constant, wherein B may be, for example, equal to the lag index used to encode p sf The number of bits, where p sf For example, 0 and 2 B -1. Additionally or alternatively, the audio decoder is configured to sf =(-1) Ssf *(3+2*g sf ) / 8 to obtain the weight G' sf , where S sf is a binary value included in the encoded representation, for example in encoded form, and wherein g sf is a binary value included in the encoded representation, for example in encoded form. Alternatively or additionally, the audio decoder is configured according to d=(7.5-g sf) / 8 to obtain the attenuation coefficient d, where g sf is a binary value included in the encoded representation, for example in encoded form.
[0137] As an example, the constant B may be selected depending on whether a given frame has more than one subframe. The inventors have realized that using the above equation, a good trade-off between signaling effort, complexity and efficiency of decoding may be achieved.
[0138] According to other embodiments of the second aspect of the invention, the audio decoder is configured to mark noise filled zero quantized spectral coefficients, and the audio decoder is configured to selectively use a reduced filtering strength, e.g. 1 / 2*G', applied to spectral coefficients not marked as noise filled zero quantized spectral coefficients. sf .
[0139] The inventors have realised that a reduction in the filtering strength for unlabelled spectral coefficients may, for example, improve the reconstruction or approximation of the original spectrum of the audio information.
[0140] According to further embodiments of the second aspect of the invention, the audio decoder is configured to switch between a second spectrum filling method (e.g., “noise filling”, in which random or pseudo-random filling values are used to fill the spectral holes, e.g., without frequency filtering and without frequency prediction in order to obtain the filling values) and a third spectrum filling method (e.g., “gap filling”, in which filling values obtained by using replication of non-zero spectral coefficients are used to fill the spectral holes) depending on the prediction lag information and / or on the tonality of the audio information. Optionally, the tonality may be determined, e.g., depending on the presence of tonality information and / or on the tonality information and / or on the HPF data. As a further optional feature, the second spectrum filling method and the third spectrum filling method are, e.g., “one or more other spectrum filling methods”.
[0141] The inventors have recognized that the decoding of the audio information can be improved, for example, by switching between the use of random or pseudo-random fill values and the replication of the non-zero spectrum, for example within a frequency distance (e.g., within a frequency distance determined by the prediction lag information). Furthermore, the inventors have recognized that such switching can be performed based on the prediction lag information and / or depending on the tonality of the audio information.
[0142] As an example, classification of audio information or, for example, a subframe sf as "tonal" may be performed based on prior art audio tonality data, such as by classifying the sf as "tonal" if audio tonality data is present (e.g., TD-LTP / HPF data is non-zero). Or, as another example, the sf may be classified as "tonal" only if a TD-LTP / HPF gain value is transmitted and is at a maximum value.
[0143] According to other embodiments of the second aspect of the present invention, the audio decoder is configured to obtain tonality information based on the encoded audio information, for example, a tonality value that quantitatively describes the tonality of the audio content of the encoded audio information, or a tonality flag that indicates whether the audio content of the encoded audio information is tonal, so as to obtain the temporal (audio tonality) tonality information j on a frame-by-frame or sub-frame (e.g., audio frame-by-audio sub-frame) basis from the bit stream. sf Furthermore, the audio decoder is configured to switch between a second spectrum filling method (e.g. “noise filling”, e.g. based on random or pseudo-random noise values) and a third spectrum filling method (e.g. “gap filling”) depending on or in dependence on the tonality information.
[0144] The inventors have realised that the signalling effort may be improved if the decoder is configured to derive tonality information from or based on the encoded audio information.
[0145] According to further embodiments of the second aspect of the invention, the audio decoder is configured to obtain prediction lag information, such as a frequency domain long-term prediction lag value p, for example from a bit stream or from encoded audio information. sf , for example, the spectrum (LTP, such as long-term prediction) distance value p sf .
[0146] According to further embodiments of the second aspect of the invention, the audio decoder is configured to judge (e.g. determine or decide) whether the audio information is tonal depending on tonality information included in the encoded audio representation and extractable by the audio decoder from the encoded audio information and / or on information indicating whether the tonality information is included in the encoded audio information (e.g. a flag) and / or on a filter gain value and / or on a prediction gain value (e.g. a TD-LTP gain value) and / or on a time domain postfilter gain value (HPF gain value, e.g. a harmonic postfilter gain value).
[0147] As explained above, embodiments according to the invention are not limited to a specific evaluation of the tonal properties of the audio information.Thus, the inventive decoder may comprise a good flexibility for checking the tonal information.
[0148] According to further embodiments according to the second aspect of the invention, the audio decoder is configured to apply a high frequency noise gain adjustment for filling of spectral holes in higher frequency regions below a (eg higher) noise filling end frequency.
[0149] As an example, using high frequency noise gain adjustment, the spectral energy of the filling values used to fill the spectral holes may be adjusted to allow for better reconstruction of the (eg original) audio input spectrum.
[0150] According to further embodiments of the second aspect of the invention, the audio decoder is configured to obtain high frequency energy information, such as high frequency energy difference values, based on the encoded audio information, e.g. using decoding of encoded high frequency energy information values included in the encoded audio information.
[0151] As an example, the high frequency energy information, such as the HF energy value (or, for example, the difference in the case of differential entropy coding), may represent, for example, the original energy of the spectral coefficients (quantized to zero) of the audio information slightly below the noise filling end frequency (e.g., in the frequency range of 8kHz to 10kHz) in a spectro-temporal manner, such as the original RMS energy. For example, the high frequency energy information may be quantized, such as a scaling factor in AAC, e.g., logarithmically in steps of 1.51dB.
[0152] Using the high frequency energy information, the audio information may be efficiently decoded and / or reconstructed. Optionally, gain adjustment may be performed based on the high frequency energy information.
[0153] According to other embodiments of the second aspect of the invention, the audio decoder is configured to depend on the high frequency energy value (eg EHF sf , which may for example be included in an encoded audio representation in an encoded form), depending on a global gain value (e.g. GG sf , which may for example be included in an encoded audio representation in an encoded form) and depends on (e.g. wideband) noise level information (which may for example be associated with a frequency region wider than the frequency region associated with the high frequency energy values, such as L sf , which may be included in the encoded audio representation in an encoded form) to obtain a high frequency energy difference value (e.g. nrFac sf ). Furthermore, the audio decoder is configured to apply the high frequency energy difference value to obtain one or more noise filling values.
[0154] As an example, in order to minimize the signaling overhead required to convey information about the high frequency energy value to the decoder, the information about the energy value can be transmitted as a difference value relative to the product of a global gain value (e.g., the global gain of the core encoder) and noise level information (e.g., the noise level), for example as a "noise gain normalized" value, such as GG sf *L sf This may be achieved, for example, by transmitting a rounded scaled result of the logarithm of the ratio between the high frequency energy value and the product of the global gain value and the noise level information.
[0155] The inventors have realised that based on the product of the gain value and the noise level information, a HF energy value can be obtained which allows obtaining a noise filling value for filling spectral holes which may provide a good reconstruction of the (eg original) input audio spectrum.
[0156] According to other embodiments of the second aspect of the present invention, the audio decoder is configured to selectively multiply one or more intermediate noise filling values associated with frequencies in a higher frequency region lower than (e.g., higher) the noise filling end frequency by the high-frequency energy difference value, for example, so that the noise values in the lower frequency region lower than the higher frequency region are not affected by the high-frequency energy difference value.
[0157] Using this method, the noise intensity in the higher frequency region can be adjusted, for example, based on the high frequency energy difference value which can be encoded in the bit stream. In this way, for example, the auditory impression can be improved.
[0158] According to further embodiments of the second aspect of the invention, the audio decoder is configured to apply the high frequency noise gain adjustment selectively to spectral values for which noise filling is performed, while leaving spectral values for which noise filling is not performed unaffected, for example.
[0159] The inventors have realised that a good compromise between computational effort and optimization effort can be achieved by, for example, performing gain adjustments only on the spectral values for which noise filling is performed.
[0160] According to further embodiments of the second aspect of the invention, the audio decoder is configured to (e.g. selectively) apply high frequency noise gain adjustment in the frequency range between 8 kHz and 10 kHz, e.g. based on a single common high frequency energy value or based on a single common high frequency energy difference value.
[0161] The inventors have realised that applying a high frequency noise gain adjustment within the above mentioned frequency range may provide a good trade-off between additional complexity and quality of the decoded audio information.
[0162] According to other embodiments of the second aspect of the present invention, the high-frequency energy value or the high-frequency energy difference value represents the (e.g. original (e.g. RMS)) energy of multiple (e.g. spectro-temporally normalized) spectral coefficients (quantized to zero) at a frequency lower than and, for example, adjacent to the noise filling end frequency or in a frequency region lower than and, for example, adjacent to the noise filling end frequency.
[0163] Therefore, the noise in the higher frequency region may be adjusted to be close to the (eg original) eg true) intensity.
[0164] Further embodiments according to the second aspect of the invention comprise an audio encoder for providing encoded audio information based on input audio information, wherein the audio encoder is configured to encode a plurality of quantized spectral values, and wherein the audio encoder is configured to obtain (e.g. determine) a lag value, e.g. a FD-LTP lag, e.g. a lag value P sf, which defines characteristics of a filtering operation (e.g. in the frequency direction) or a prediction operation (e.g. in the frequency direction) to be performed by the audio decoder to derive one or more filling values for filling the spectral holes. Furthermore, the audio encoder is configured to obtain (e.g. determine) a gain value (e.g. G sf ), the gain value defines the characteristics of the filtering operation or prediction operation to be performed by the audio decoder to derive one or more filling values for filling the spectral holes. In addition, if the gain value (e.g. G sf ) is less than a threshold value (e.g., β) or if the absolute value of the gain value is less than a threshold value, the audio encoder is configured to (e.g., selectively) set or, for example, change the lag value to zero, thereby obtaining a modified lag value (e.g., p sf =0); and the audio encoder is configured to encode the determined lag value or the modified lag value, wherein, for example, if the gain value is modified, the modified lag value is encoded using, for example, 3 or 4 bits.
[0165] For example, as explained above, decoder-side filling of spectral holes of a decoded set of spectral values may be performed based on predicted lag information. The predicted lag information may correspond to, for example may be or may include, a lag value or a modified lag value, or may be determined using the lag value or the modified lag value. Thus, based on such lag information, decoding and / or reconstruction of the audio information may, for example, be performed efficiently.
[0166] The inventors have recognized that the hysteresis value may be determined based on a gain value associated with a filtering operation or a prediction operation to be performed by the audio decoder to derive one or more filling values for filling the spectral holes. In short, and as an example, if the gain value is not significant, the hysteresis value may be set to zero. This adaptation may result in a modified hysteresis value.
[0167] For example, in case of low or insignificant gain values, there may be only a weak relationship between spectral coefficients from different frequency bands, and thus hysteresis information that may correspond to the correlation between these values in the frequency direction may not be available, or may not be suitable for spectral value reconstruction, e.g. because of the low gain-effect correlation. Thus, by setting the hysteresis value to zero in case of too small a gain, bit rate may be saved.
[0168] According to further embodiments of the second aspect of the invention, the audio encoder is configured to use a method for applying, for example, a delay B <p'<B+2 B The autocorrelation information of a set of spectral values at (eg applied to a spectrum normalized in a spectrotemporal manner) determines the lag value and the gain value, where, for example, the lag value (eg P sf ) is determined depending on the position of the peak of the autocorrelation function obtained based on the set of spectral values.
[0169] As an example, the autocorrelation information may be normalized autocorrelation information. In general, lag values (or modified lag values), gain values and / or symbol indices or corresponding indices for filtering and / or prediction of spectral coefficients may be calculated, for example, in a domain normalized in a spectrotemporal manner, such as utilized prior to quantization of transform coefficients.
[0170] According to further embodiments according to the second aspect of the invention, the audio encoder is configured to selectively encode the gain value if the encoded lag value (eg the lag value or the modified lag value) is non-zero.
[0171] As an example, vice versa, as explained before, if the gain value is significant and the lag value is non-zero, prediction or filtering of spectral coefficients (e.g. noise filling or gap filling spectral values) may be performed, and therefore, as an example, only in such a case may signaling bits for the gain value and the lag value be provided.
[0172] According to further embodiments of the second aspect of the invention, the audio encoder is configured to selectively encode a high frequency energy value describing, for example, the energy in an upper part of the spectrum of the input audio information or a preprocessed version thereof when the encoded lag value is zero.
[0173] As an example, in cases where filtering or prediction of spectral coefficients may not be performed due to unfavorable gain and / or lag values, high frequency energy values may be provided, eg, in order to perform noise filling or gap filling with corresponding spectral energies.
[0174] According to further embodiments of the second aspect of the invention, the audio encoder is configured to selectively encode a gain value or a high frequency energy value describing the energy in an upper part of the spectrum (e.g. the input audio information or a preprocessed version thereof) depending on the encoded lag value.
[0175] As an example, by using selective encoding of gain values or high frequency energy values, the signaling effort may be reduced.
[0176] According to further embodiments of the second aspect of the invention, the audio encoder is configured to encode the gain value and the high-frequency energy value using the same number of bits, wherein, for example, the gain value is encoded using one bit for the sign and one bit for the magnitude, and wherein, for example, the high-frequency energy value is encoded using 2 bits.
[0177] The inventors have realised that using the same number of bits to encode the gain value and the high frequency energy value may provide an interchangeable encoding such that decisions on what to encode may be taken with respect to the lag value without having to adapt the number of bits to be encoded.
[0178] According to further embodiments of the second aspect of the invention, the audio encoder is configured to determine individual lag values and / or individual gain values for different audio frames and / or for different audio subframes.
[0179] The inventors have recognised that frame-by-frame and / or sub-frame-by-sub-frame lag values and / or gain values may improve the encoding of audio information.
[0180] According to further embodiments of the second aspect of the invention, the audio encoder is configured to obtain lag values and / or gain values in a transform domain, e.g. using a set of spectral values (e.g. using an analysis of periodicity in the set of spectral values in the frequency direction).
[0181] The inventors have realised that the determining or obtaining of such information may be performed in a computationally efficient manner.
[0182] According to further embodiments according to the second aspect of the invention, the audio encoder is configured to perform long term transient detection and selectively set the lag value to zero if, for example, an audio frame or an audio subframe (specified by sf) is found not to be long term transient.
[0183] Since the hysteresis value may be, for example, an indicator of transients for a frame or subframe, the value may be set to zero if no transients are detected. Thus, the encoder may further temporarily suspend filtering or prediction of zero-quantized spectral values in the decoder based on transient detection if no transients are detected for a frame or subframe.
[0184] Other embodiments according to the present invention include an audio encoder for providing encoded audio information based on input audio information, wherein the audio encoder is configured to encode multiple quantized spectral values, and wherein the audio encoder is configured to encode high-frequency energy values or high-frequency energy difference values. In addition, the high-frequency energy values or high-frequency energy difference values represent the (e.g. original (e.g. RMS)) energy of multiple (e.g. normalized in a spectral-temporal manner) spectral coefficients (quantized to zero) at a frequency lower than and, for example, adjacent to a noise filling end frequency or in a frequency region lower than and, for example, adjacent to a noise filling end frequency.
[0185] As explained previously, the high-frequency energy values (or differences, for example in the case of differential entropy coding) may represent the original energy, e.g. the original RMS energy, of the spectral coefficients (quantized to zero) normalized in a spectro-temporal manner slightly below the noise filling end frequency (e.g. in the frequency range of 8kHz to 10kHz).
[0186] In order to minimize the signaling overhead to communicate the high frequency energy value to the decoder, the energy value may be transmitted, for example, as a difference relative to the product of the global gain and the noise level, for example, as a "noise gain normalized" value. This may be achieved, for example, by transmitting a rounded scaled result of the logarithm of the ratio between the high frequency energy value and the product of the global gain and the noise level.
[0187] As an example, based on the high frequency energy values or high frequency energy difference values, in the decoder, zero quantized spectral coefficients may be reconstructed, e.g. using gap filling, such that the energy of the zero quantized coefficients (e.g., the original audio signal) is at least approximately derived.
[0188] According to further embodiments of the second aspect of the invention, the audio encoder is configured to quantize the high frequency energy values or the high frequency energy difference values in a logarithmic manner.
[0189] The inventors have realised that logarithmic quantisation can be performed in a computationally efficient manner.
[0190] According to other embodiments of the second aspect of the invention, the audio encoder is configured to encode a high-frequency energy difference value relative to the product of a global gain encoded by the audio encoder and a noise level encoded by the audio encoder, which describes the energy of multiple (e.g., spectro-temporally normalized) spectral coefficients (quantized to zero) at frequencies below and, for example, adjacent to a noise filling end frequency or in a frequency region below and, for example, adjacent to a noise filling end frequency.
[0191] The inventors have recognized that encoding of high frequency energy difference magnitudes may, for example, minimize signaling overhead.
[0192] According to further embodiments of the second aspect of the invention, the audio encoder is configured to obtain a rounded scaling result of the logarithm of the ratio between the high-frequency energy value and the product of the global gain and noise information so as to encode the high-frequency energy value, for example in the form of a high-frequency energy difference value.
[0193] The inventors have realised that a rounded scaled result can be obtained in a computationally efficient manner.
[0194] According to other embodiments of the second aspect of the invention, the audio encoder is configured to sf =1+round(Δ*log2(EHF sf / (GG sf *L sf )) Determine a quantized high frequency energy difference value, where EHF is a high frequency energy value, such as the original RMS energy of the HF spectrum value quantized to zero, where GG sf is the global gain, where L sf is the noise level, and Δ is a constant.
[0195] The inventors have realised that use of the above formula may allow for efficient determination of the magnitude of the quantised high frequency energy difference.
[0196] Further embodiments according to the invention include a method for providing decoded audio information based on encoded audio information, the method comprising: filling spectral holes of a set of decoded spectral values, e.g. using replacement of spectral coefficients quantized to zero based on corresponding filling values; and obtaining prediction lag information, e.g. a frequency domain long-term prediction lag value p, e.g. from a bit stream or from the encoded audio information. sf , such as prediction lag information indicating a prediction period in the frequency direction, such as a spectrum (LTP, such as long-term prediction) distance value psf. In addition, the method includes switching between a first spectrum filling method (such as "noise filling"+FD LTP) and one or more other spectrum filling methods (such as a second spectrum filling method or a third spectrum filling method, such as "gap filling") depending on the prediction lag information, in which the first spectrum filling method, frequency filtering or frequency prediction (such as TNS or LTP, such as filtering in which the spectrum value associated with the first frequency has an influence on the spectrum value associated with the second frequency) is used to obtain the filling value for filling the spectrum hole, and in the one or more other spectrum filling methods, frequency filtering is not used and frequency prediction is not used to obtain the filling value for filling the spectrum hole.
[0197] Other embodiments according to the present invention include a method for providing encoded audio information based on input audio information, the method comprising: encoding a plurality of quantized spectral values; and obtaining (e.g., determining) a lag value (e.g., an FD-LTP lag, e.g., a lag value P sf ), which defines characteristics of a filtering operation (e.g. in the frequency direction) or a prediction operation (e.g. in the frequency direction) to be performed by the audio decoder for deriving one or more filling values for filling the spectral holes. The method further comprises: obtaining (e.g. determining) a gain value (e.g. G sf ), which defines the characteristics of a filtering operation or a prediction operation to be performed by the audio decoder for deriving one or more filling values for filling the spectral holes; and at this gain value (e.g. G sf ) is less than a threshold value (e.g., β) or when the absolute value of the gain value is less than a threshold value, the hysteresis value is (e.g., selectively) set or, for example, changed to zero, thereby obtaining a modified hysteresis value (e.g., p sf =0). Furthermore, the method comprises encoding the determined hysteresis value or the modified hysteresis value, wherein for example if the gain value is modified, the modified hysteresis value is encoded using, for example, 3 or 4 bits.
[0198] Other embodiments according to the invention comprise a computer program for performing any of the methods explained above when the computer program is run on a computer.
[0199] According to other embodiments of the present invention, the audio decoder is configured to perform a weighted combination of a noise value associated with a given frequency and a spectral value associated with another frequency (e.g., a noise value or a filling value, or a processed or unprocessed encoded value), or a weighted combination of a filling value associated with a given frequency and a spectral value associated with another frequency (e.g., a noise value or a filling value, or a processed or unprocessed encoded value) to obtain a given filling value. In addition, the audio decoder is configured to adjust the weight given to the spectral value associated with another frequency depending on whether noise filling has been applied for the spectral value associated with the other frequency.
[0200] Other embodiments include an audio decoder for providing decoded audio information based on encoded audio information, wherein the audio decoder is configured to fill spectrum holes of a decoded set of spectrum values. In addition, the audio decoder is configured to: obtain prediction gain information; and switch between a first spectrum filling method and one or more other spectrum filling methods depending on the prediction gain information, in which the filling values for filling the spectrum holes are obtained using frequency filtering or frequency prediction, and in which the filling values for filling the spectrum holes are obtained without frequency filtering and without frequency prediction.
[0201] Other embodiments according to the present invention include an audio encoder for providing encoded audio information based on input audio information, wherein the audio encoder is configured to encode multiple quantized spectral values. In addition, the audio encoder is configured to obtain a gain value, which defines the characteristics of a filtering operation or a prediction operation to be performed by an audio decoder for deriving one or more filling values for filling a spectral hole. In addition, the audio encoder is configured to encode a gain value, and if the quantized gain value or the encoded gain value is non-zero, a hysteresis value is selectively encoded, which defines the characteristics of the filtering operation or the prediction operation to be performed by the audio decoder for deriving one or more filling values for filling a spectral hole. Alternatively or in addition, the audio encoder is configured to selectively encode a hysteresis value when the gain value is greater than or equal to a threshold value, which defines the characteristics of a filtering operation or a prediction operation to be performed by the audio decoder for deriving one or more filling values for filling a spectral hole.
[0202] Other embodiments according to the present invention include a method for providing decoded audio information based on encoded audio information, the method comprising: filling spectral holes of a decoded spectral value set; obtaining prediction gain information; and switching between a first spectral filling method and one or more other spectral filling methods depending on the prediction gain information, in which the first spectral filling method, frequency filtering or frequency prediction is used to obtain filling values for filling the spectral holes, and in which the one or more other spectral filling methods, frequency filtering and frequency prediction are not used to obtain filling values for filling the spectral holes.
[0203] Other embodiments according to the present invention include a method for providing encoded audio information based on input audio information, the method comprising: encoding multiple quantized spectral values; obtaining a gain value, which defines the characteristics of a filtering operation or a prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes; encoding the gain value; if the quantized gain value or the encoded gain value is non-zero, selectively encoding a lag value, which defines the characteristics of a filtering operation or a prediction operation to be performed by the audio decoder for deriving one or more filling values for filling spectral holes; or if the gain value is greater than or equal to a threshold, selectively encoding a lag value, which defines the characteristics of a filtering operation or a prediction operation to be performed by the audio decoder for deriving one or more filling values for filling spectral holes.
[0204] Aspect 3
[0205] Further embodiments according to the third aspect of the invention comprise an audio decoder for providing a decoded audio representation based on an encoded audio representation, wherein the audio decoder is configured to fill spectral holes of a set of decoded spectral values with corresponding filling values (e.g. using replacement of spectral coefficients quantized to zero based on the corresponding filling values). Furthermore, the audio decoder is configured to use prediction or filtering (e.g. using the calculation rule d*c(i)+G' sf *c(iP' sf )) determine the (e.g. final) fill value (e.g. for c(i) replacement, e.g. ), so that a given fill value (e.g. associated with a given frequency (e.g. with a given frequency interval) ) is dependent on different frequencies (eg, different frequency intervals, eg, with a frequency interval index iP' sf Different frequency intervals, such as the spectral distance P' from a given frequency or a given frequency interval sf Or the spectrum distance d sf Another spectral value (e.g., c(iP')) associated with a frequency or frequency interval sf )or ) and obtain it.
[0206] Furthermore, the audio decoder is configured to depend on different frequencies (eg, different frequency intervals; eg, different frequencies with a frequency interval index iP′). sf The filter strength is adapted (e.g. the weighting of the spectral values associated with different frequencies, e.g. by selectively setting the filter strength to G') based on the encoded or quantized spectral values associated with different frequency intervals of the audio signal (e.g. as (initially) determined by the encoded representation of the respective spectral values in the encoded audio information, e.g. determined by the spectral values before applying the noise filling, e.g. determined by the spectral values directly after arithmetic decoding). sf or 1 / 2G' sf ).
[0207] The inventors have realised that fill values may be determined or calculated or obtained based on other spectral values associated with different frequencies using prediction or filtering. In brief, and as an example, correlation or dependency of spectral coefficients of spectral values of different frequencies (e.g. different frequency bands) may be exploited.
[0208] Thus, the encoding effort may be reduced and / or the auditory impression may be improved, for example, by exploiting such correlations.Thus, using prediction coefficients and / or filter coefficients, filler values may be determined with a reduced amount of bits that need to be transmitted, while still providing a good representation of the originally encoded audio signal.
[0209] Furthermore, the inventors have realised that decoding of an encoded audio representation may be improved, for example, by adapting the filtering strength depending on the encoded or quantized spectral values associated with different frequencies.
[0210] As explained previously, the fill value associated with a given frequency may be determined or obtained or calculated based on or using spectral values associated with different frequencies, for example where the prediction lag information is non-zero and therefore (as an example) indicates transient nature of the signal.
[0211] As an example, in case the first spectrum filling method is selected, e.g. in case noise filling with FD-LTP is selected (e.g. if the prediction lag information is non-zero, as an example if the FD-LTP lag is non-zero), applying a long-term predictive filter in the spectral domain (e.g. MDCT domain) of the audio transform codec may be performed during the decoder-side noise filling routine (e.g. depending on whether the "current" encoded FD coefficient is zero and depending on whether the corresponding "previous" encoded FD coefficient located at a certain distance from the current coefficient (specified by the transmitted prediction lag information, e.g. specified by the transmitted FD-LTP lag) is zero). As an example, an infinite impulse response (IIR) LTP type filter may be used for filtering.
[0212] For example, if the spectral values associated with different frequencies are relatively large, e.g. non-zero, the filter strength may be reduced. Thus, the impact of large spectral values associated with different frequencies may be reduced by selectively adapting the filter strength. Thus, it may be avoided that fill values or noise values adopt excessively large values.
[0213] According to other embodiments of the third aspect of the present invention, the filtering strength is determined, for example, by sf )'s effect on a given fill value.
[0214] Thus, as an example, the filtering strength may represent a weighting factor of another spectral value.The inventors have recognized that such an adaptability of the influence or, as an example, the weighting of another spectral value may improve the decoded audio information.
[0215] According to further embodiments of the third aspect of the invention, the audio decoder is configured to adapt the filtering strength depending on spectral values associated with different frequencies, eg initially determined by an encoded representation of respective spectral values in the encoded audio information.
[0216] The inventors have realised that using a value represented by the encoded representation for adapting the filter strength allows using or exploiting information provided by the encoded representation rather than a filtered version thereof (which may for example be alternating). It has been found that using such a criterion the selection of the filter strength is more reliable than using a criterion that depends on a value that has been pre-processed on the decoder side.
[0217] According to further embodiments of the third aspect of the invention, the audio decoder is configured to adapt the filtering strength depending on spectral values associated with different frequencies before applying the noise filling.
[0218] The inventors realized that using spectral values associated with different frequencies before noise filling may allow adapting the filter strength based on information whether the spectral value is quantized to zero or not.
[0219] According to further embodiments of the third aspect of the invention, the audio decoder is configured to adapt the filtering strength depending on whether spectral values associated with different frequencies (or values) are quantized to zero or not.
[0220] The inventors have realised that, for example, different filtering strengths may be applied to spectral values that are quantised to zero compared to spectral values that are not quantised to zero. This may improve the accuracy of the reconstructed spectrum.
[0221] According to further embodiments according to the third aspect of the invention, the audio decoder is configured to adapt the filtering strength depending on whether noise filling is applied to spectral values associated with different frequencies (or values).
[0222] The inventors have realised that using such criteria, for example, filtering strength adaptation may be performed based on information whether a corresponding spectral value has been quantised to zero, in addition to whether noise filling is attempted or performed for the corresponding frequency of the spectral value. This may include using a flag.
[0223] According to other embodiments of the third aspect of the invention, the audio decoder is configured to, for the spectral values for which noise filling is applied (eg for the spectral values at positions i>=P′ sf Each noise-filled zero-quantized spectral coefficient at c) selectively applies filtering in the frequency direction or prediction in the frequency direction.
[0224] As explained above, the inventors have realized that, as an example, zero-quantized spectral values may be approximated or estimated based on or using filtering or prediction in the frequency direction.Thus, a dependency of spectral values in the frequency direction may be exploited, for example, among different spectral values.
[0225] According to further embodiments according to the third aspect of the invention, the audio decoder is configured to apply prediction or filtering in order to determine a given (e.g. final) fill value (e.g. ).
[0226] As explained previously, the inventors recognized that random or pseudo-random noise values may be adapted, for example using prediction or filtering, in order to calculate (e.g. final) filling values that provide a good approximation of zero-quantized spectral values for a (e.g. original) e.g. input spectrum for the audio information.
[0227] According to other embodiments of the third aspect of the invention, the audio decoder may be configured to perform a weighted combination of a noise value associated with a given frequency and a noise value associated with another frequency (eg, a combination d*c(i)+G' sf *c(iP' sf ), where the weight d of the noise value c(i) is associated with a given frequency, and the weight of the noise value G' sf Associated with another frequency, or perform the combination d*c(i)+1 / 2*G' sf *c(iP' sf ), where the weight d of the noise value c(i) is associated with a given frequency, and the weight of the noise value is 1 / 2*G' sf associated with another frequency), or a weighted combination of a noise value associated with a given frequency and a fill value associated with another frequency, in order to obtain a given (eg, final) fill value, e.g. Furthermore, the audio decoder is configured to adjust the weight (eg G′) given to the noise value associated with the other frequency depending on whether noise filling has been applied to the spectral value associated with the other frequency. sf or 1 / 2*G'sf ) or a weight given to a fill value associated with another frequency (e.g. G' sf or 1 / 2*G' sf ).
[0228] As explained above, the inventors recognize that the (e.g. final) filling value can be calculated, for example, using different frequency-dependent amounts of a noise value associated with a given frequency or associated with another frequency and / or a filling value associated with another frequency. Thus, the inventive concept can allow determining or obtaining or calculating a (e.g. final) filling value with good flexibility, so that, depending on the specific situation, a filling value that is well or even optimally suitable for the reconstruction of the (e.g. original) audio information spectrum can be obtained. For example, the selection of the corresponding amount to be used to obtain the (e.g. final) filling value can be performed based on the prediction lag information. Furthermore, the inventors recognize that the adaptation or adjustment of the corresponding weights of the corresponding noise value or filling value associated with another frequency can improve the determination of the (e.g. final) filling value, and therefore improve the reconstruction of the audio information.
[0229] According to further embodiments according to the third aspect of the invention, the audio decoder is configured to determine a spectral distance, e.g. P′, between a filling value associated with a given frequency and another spectral value associated with a different frequency based on encoded information describing the spectral distance included in the encoded representation of the audio information (e.g. an encoded value). sf .
[0230] Based on the spectral distance, the decoder can, for example, decide whether to use prediction or filtering for the determination of the filling value. The distance can be associated with the prediction lag information and / or the prediction lag value explained above. In addition, parameters of the corresponding prediction or filtering, such as the filter order, can be determined or set or obtained based on the distance. The inventors have realized that the spectral distance can be used in order to improve the determination of the spectral filling value.
[0231] According to further embodiments of the third aspect of the invention, the audio decoder is configured to generate a gain signal based on gain information (eg gain values, eg g) included in the encoded representation of the audio information. sf ) determines a weight (e.g., d) applied to the noise value associated with a given frequency, where, as an example, the weight applied to the noise value associated with the given frequency is a positive value, such as in the range between 0.5 and 1.
[0232] As explained above, the inventors have realised that, for example, the corresponding noise value associated with a given frequency can be adapted with a weight determined depending on the gain information. This can allow shaping of the noise value to improve its match with the corresponding spectral value of the originally encoded audio information.
[0233] According to further embodiments of the third aspect of the invention, the audio decoder is configured to, depending on gain information (eg gain values, eg g) included in the encoded representation of the audio information, sf ) determines the noise value (e.g., c(iP') applied to another frequency sf )) or a weight applied to a fill value associated with another frequency (e.g., G sf =(-1) Ssf *(3+2*g sf ) / 8, or 1 / 2*G sf ), where the weight applied to the noise value associated with a given frequency is, for example, a positive or negative value, for example having an absolute value between 0.25 and 0.75.
[0234] As explained above, the inventors have realized that, for example, a corresponding noise value or a corresponding filler value associated with another frequency can be adapted with a weight determined depending on the gain information. This can allow shaping of the noise or filler value to improve its match with the corresponding spectral value of the originally encoded audio information.
[0235] According to further embodiments of the third aspect of the invention, the audio decoder is configured to, depending on the symbol information (eg symbol value, eg S sf ) to determine the noise value (e.g., c(iP') applied to another frequency sf )) or a weight of a fill value associated with another frequency, such as G sf =(-1) Ssf *(3+2*g sf ) / 8 or 1 / 2*G sf .
[0236] As explained above, the inventors have realized that, for example, using sign information, such as 1-bit information, can, for example, improve weight determination. As an example, the sign information can allow adaptation of the (e.g. final) fill value relative to the phase relationship of the noise value and / or fill value associated with another frequency on which it can be based.
[0237] According to other embodiments of the third aspect of the invention, the audio decoder is configured to sf ) is obtained using noise filling (e.g. at spectral position iP' sf The coefficient c(iP' sf ) in the case of being marked as noise-filled zero-quantized spectral coefficients in the previous processing) according to Determines the given fill value And in the coefficient c(iP' sf ) is not obtained using noise filling (e.g. at spectral position iP'sf The coefficient c(iP' sf ) in the case where it is not marked as noise-filled zero-quantized spectral coefficients in the previous processing) according to Determines the given fill value c(i) represents the spectral coefficient with spectral index i obtained using noise filling, d represents the attenuation coefficient, G' sf represents a weight based on a gain value included in the encoded audio representation; c(iP' sf ) represents a spectrum index iP′ which can be obtained, for example, using noise or can be obtained, for example, without using noise filling, and can be obtained, for example, using prediction or filtering sf The spectral coefficients of P' sf are prediction parameters or filter parameters based on prediction parameter information included in the encoded audio representation.
[0238] The inventors have realised that using the above equation, an effective fill value can be determined.
[0239] According to other embodiments of the third aspect of the present invention, the audio decoder is configured to sf =p sf +B obtains prediction parameters or filtering parameters P' sf , where p sf is a lag index included in the encoded audio representation, and wherein B is a constant, wherein B may be, for example, equal to the lag index used to encode p sf The number of bits, where p sf For example, 0 and 2 B -1. Alternatively or additionally, the audio decoder is configured to sf =(-1) Ssf *(3+2*g sf ) / 8 to obtain the weight G' sf , where S sf is a binary value included in the encoded representation, for example in encoded form, and wherein g sf is a binary value included in the encoded representation, for example in encoded form. Alternatively or additionally, the audio decoder is configured to decode the audio signal according to d=(7.5-g sf ) / 8 to obtain the attenuation coefficient d, where g sf is a binary value included in the encoded representation, for example in encoded form.
[0240] As an example, the constant B may be selected depending on whether a given frame has more than one subframe. The inventors have realized that using the above equation, a good trade-off between signaling effort, complexity and efficiency of decoding may be achieved.
[0241] According to other embodiments of the third aspect of the invention, the audio decoder is configured to mark noise-filled zero-quantized spectral coefficients, and the audio decoder is configured to selectively use a reduced filtering strength, e.g. 1 / 2*G', applied to spectral coefficients not marked as noise-filled zero-quantized spectral coefficients. sf .
[0242] The inventors have realised that a reduction in the filtering strength for unlabelled spectral coefficients may, for example, improve the reconstruction or approximation of the original spectrum of the audio information.
[0243] According to other embodiments of the third aspect of the present invention, the audio decoder is configured to perform the following processing for a plurality of subframes (sf):
[0244] 1. Set P' sf =p sf +B,G' sf =(-1) Ssf *(3+2*g sf ) / 8 and d=(7.5-g sf ) / 8, where, for example, p sf >0;
[0245] 2. For example, use l sf , for example, performing (eg, conventional) noise filling using random or pseudo-random noise values to replace spectral coefficients that are zero, wherein the noise intensity may be determined, for example, by a noise intensity value I sf determining and marking, for example, all or a plurality of noise filled zero quantized spectral coefficients;
[0246] 3. For example, in order of increasing i, at position i>=P' sf To fill multiple or even each noise-filled zero-quantized spectral coefficient c at a position, perform:
[0247] 4. If the position iP' is marked in step 2 sf The coefficient c at the point is d*c(i)+G' sf *c(iP' sf ) replaces c(i); otherwise
[0248] 5. Use d*c(i)+1 / 2*G' sf *c(iP' sf ) replaces c(i).
[0249] The inventors have realized that using the above steps a good trade-off between signaling effort, complexity and effectiveness of decoding can be achieved.
[0250] Further embodiments according to the third aspect of the invention comprise an audio decoder for providing a decoded audio representation based on an encoded audio representation, wherein the audio decoder is configured to determine processed spectral values (e.g. spectral coefficients) using prediction or filtering (e.g. using temporal noise shaping (TNS) and / or using frequency domain noise shaping (FD-LTP)) such that a given processed spectral value associated with a given frequency (e.g. with a given frequency interval) is dependent on another spectral value (e.g. c(iP')). sf ),or It is associated with different frequencies, for example, with different frequency intervals, for example, with a frequency interval index iP' sf For example, it is associated with a spectral distance P' from a given frequency or a given frequency interval. sf Or the spectrum distance d sf obtained by associating with the frequency or frequency interval of the
[0251] Furthermore, the audio decoder is configured to depend on different frequencies (eg, different frequency intervals; eg, different frequencies with a frequency interval index iP′). sf The filter strength is adapted (e.g. a weighting of the spectral values associated with different frequencies, e.g. by selectively setting the filter strength to G') based on coded or quantized or signaled spectral values associated with different frequency intervals of the audio signal (e.g. as (initially) determined by an coded representation of individual spectral values in the coded audio information, e.g. determined by spectral values before applying noise filling, e.g. determined by spectral values directly after arithmetic decoding). sf or 1 / 2G' sf ).
[0252] The inventors have realized that processed spectral values may be determined or calculated or obtained based on other spectral values associated with different frequencies, for example using prediction or filtering. As explained previously, correlations or dependencies of spectral coefficients of spectral values of different frequencies (e.g. different frequency bands) may be exploited, for example, not only for fill-in values, but also for processed spectral values.
[0253] Thus, the encoding effort may be reduced, for example, by exploiting such correlations.Thus, using prediction coefficients and / or filter coefficients, spectral values may be determined with a reduced amount of bits that need to be transmitted, while still providing a good representation of the originally encoded audio signal.
[0254] Furthermore, the inventors have realised that decoding of an encoded audio representation may be improved by adapting the filtering strength depending on the encoded or quantized spectral values associated with different frequencies.
[0255] According to other embodiments of the third aspect of the invention, the audio decoder is configured to adapt the filtering strength to (e.g. selectively) reduce the contribution (e.g. weighted in prediction or filtering) of non-zero quantized and possibly previously processed (e.g., previous TNS synthesis filtering, e.g., lower frequency) spectral coefficients included in the prediction or filtering, when compared to the contribution (e.g., weighted in prediction or filtering) of zero quantized (and possibly previously processed, e.g., previous TNS synthesis filtering, e.g., lower frequency) spectral coefficients included in the prediction or filtering.
[0256] According to other embodiments of the third aspect of the present invention, the audio decoder is configured to generate a filter when a current spectral coefficient (e.g., a spectral coefficient c(i) at a current spectral position specified by a spectral index I, e.g., a spectral coefficient c(i) at a current spectral position before applying filtering, e.g., a transmitted current spectral coefficient or an encoded current spectral coefficient or a quantized current spectral coefficient) is zero (e.g., quantized to zero) and a previous spectral coefficient (e.g., a spectral coefficient c(i) is equal to or greater than zero). sf ), for example, represented as "another spectral value"), for example, when it has not been encoded as zero or quantized to zero at the audio encoder side, selectively adapting (for example, reducing) the filtering strength (for example, temporal noise shaping filtering, for example, frequency domain long-term prediction, for example, based on the comparison between the unfiltered current spectral coefficient (for example, c(i)) and the filtered or unfiltered previous spectral coefficient (for example, c(iP' sf )) weighted combination (e.g. d*c(i)+att*G' sf *c(iP' sf ), where, for example, d is the weight of the current unfiltered spectral coefficient, att is an attenuation factor describing the adaptation of the filtering strength, G' sf is the normal weight of the previous spectral coefficient, filtered or unfiltered, and P' sf describing the spectral distance between the current spectral coefficient and the previous spectral coefficient) provides a filtered current spectral coefficient).
[0257] According to further embodiments of the third aspect of the invention, the audio decoder is configured to selectively reduce the filter strength to a value between 0.25 and 0.75, or preferably to a value between 0.4 and 0.6, or preferably to a value of 0.5, in order to adapt the filter strength.
[0258] According to other embodiments of the third aspect of the present invention, the audio decoder is configured to, when a current spectral coefficient (e.g., c(i)) is encoded or quantized to zero, determine the current spectral coefficient based on a plurality of spectral coefficients (e.g., d(i)). sfa plurality of previous (e.g. coded or quantized or signaled) spectral coefficients (e.g. a plurality of previous spectral coefficients) by using, for example, a common downscaling factor (which takes into account the plurality of (e.g. d sf ) previous spectral coefficients) to scale down the filter coefficients or prediction coefficients of the filter (e.g., by scaling down the filter coefficients or filter coefficients, e.g., c(i-1) to c(id sf )) to selectively reduce the filtering strength.
[0259] According to further embodiments of the third aspect of the invention, the audio decoder is configured to, in case a current spectral coefficient (e.g. c(i)) is encoded or quantized or signaled to be zero and all previous spectral coefficients (e.g. c(i-1) to c(i-d) considered in filtering are sf -1))(except for a previous spectral coefficient considered in filtering, e.g. c(id sf ), for example when a spectral coefficient considered in the filtering with a maximum spectral distance from a current spectral coefficient) is encoded or quantized or signaled as zero, the filtering strength is selectively reduced (wherein the filtering strength may, for example, be defined by a plurality of filtering weights, where in case of the reduction of the filtering strength the filtering weights may, for example, be selectively scaled down using a common scaling factor (which may, for example, be equal to 1 / 2)), otherwise, an unreduced filtering strength is used.
[0260] According to other embodiments of the third aspect of the invention, the audio decoder is configured to use filtering or prediction depending on the frequency spectrum index id sf a plurality of (eg, encoded or quantized or signaled or filtered or predicted) previous spectral coefficients (eg, c(i-1) to c(id sf )) to obtain the filtered current spectral coefficient (e.g., c(i)) with spectral index i.
[0261] Furthermore, the audio decoder is configured to selectively reduce the filtering strength in the following cases: in (eg, in and only in) a frequency spectrum having the spectrum index id sf In the case where one or more (eg, signaled) spectral coefficients or all spectral coefficients from +1 to i have been quantized or encoded or signaled to be zero, and in the case where the spectral index id sf The spectral coefficients have not been quantized or coded or signaled as zero, where for example d sf Equal to the filter order or prediction order.
[0262] According to another embodiment of the third aspect of the present invention, sfThe filter coefficients associated with spectral coefficients of spectral indices between +1 and i-1 are equal to zero.
[0263] According to other embodiments of the third aspect of the invention, the audio decoder is configured to use the encoded or quantized or signaled spectral coefficients for determining the filtering strength, e.g. before noise filling. In addition, the audio decoder is configured to use the pre-processed spectral coefficients as input for filtering or prediction, e.g. after applying noise filling and / or after applying frequency domain long-term prediction.
[0264] Other embodiments according to the third aspect of the invention include a method for providing a decoded audio representation based on an encoded audio representation, the method comprising filling spectral holes of a set of decoded spectral values with corresponding filling values (e.g. using replacement of spectral coefficients quantized to zero based on the corresponding filling values). The method further comprises using prediction or filtering (e.g. using the calculation rule d*c(i)+G' sf *c(iP' sf )) determine the (e.g. final) fill value (e.g. for c(i) replacement, e.g. ), so that a given fill value (e.g. associated with a given frequency (e.g. with a given frequency interval) ) is dependent on different frequencies (eg, different frequency intervals, eg, with a frequency interval index iP' sf different frequency intervals, such as the spectral distance P' from a given frequency or a given frequency interval sf Or the spectrum distance d sf Another spectral value (e.g., c(iP')) associated with a frequency or frequency interval sf )or In addition, the method includes depending on different frequencies (eg, different frequency intervals; eg, with a frequency interval index iP' sf The filter strength (e.g. the weighting of the spectral values associated with different frequencies, e.g. by selectively setting the filter strength to G') is adapted based on the encoded or quantized spectral values associated with different frequency intervals of the audio signal (e.g. as (initially) determined by the encoded representation of the respective spectral values in the encoded audio information, e.g. determined by the spectral values before applying the noise filling, e.g. determined by the spectral values directly after arithmetic decoding). sf or 1 / 2G' sf ).
[0265] Further embodiments according to the third aspect of the invention comprise a method for providing a decoded audio representation based on an encoded audio representation, the method comprising determining processed spectral values (e.g. spectral coefficients) using prediction or filtering such that a given processed spectral value associated with a given frequency (e.g. with a given frequency interval) is dependent on a value associated with a different frequency (e.g. with a different frequency interval, e.g. with a frequency interval index iP′ sf different frequency intervals, such as the spectral distance P' from a given frequency or a given frequency interval sf Or the spectrum distance d sf Another spectral value (e.g., c(iP')) associated with a frequency or frequency interval sf )or In addition, the method includes depending on different frequencies (eg, different frequency intervals; eg, with a frequency interval index iP' sf The filter strength (e.g. the weighting of the spectral values associated with different frequencies, e.g. by selectively setting the filter strength to G') is adapted based on the encoded or quantized spectral values associated with different frequency intervals of the audio signal (e.g. as (initially) determined by the encoded representation of the respective spectral values in the encoded audio information, e.g. determined by the spectral values before applying the noise filling, e.g. determined by the spectral values directly after arithmetic decoding). sf or 1 / 2G' sf ).
[0266] Other embodiments according to the third aspect of the present invention include a computer program for performing any one of the above methods when the computer program is run on a computer. BRIEF DESCRIPTION OF THE DRAWINGS
[0267] The drawings are not necessarily to scale, instead emphasis is generally placed on illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following figures, in which:
[0268] Figure 1 A schematic diagram showing an audio decoder for providing decoded audio information based on encoded audio information according to an embodiment of the first aspect of the invention;
[0269] Figure 2 shows a schematic example of a spectrum envelope according to conventional concepts;
[0270] Figure 3 illustrative example showing a spectral envelope (intensity as a function of frequency) according to the first aspect of the invention;
[0271] Figure 4 A schematic diagram showing an audio decoder according to an embodiment of the first aspect of the invention with additional optional features;
[0272] Figure 5 A schematic diagram showing an audio encoder for providing encoded audio information based on input audio information according to an embodiment of the first aspect of the invention;
[0273] Figure 6 A schematic diagram showing an audio encoder according to an embodiment of the first aspect of the invention with additional optional features;
[0274] Figure 7 An example showing the functionality of an encoder according to an embodiment of the first aspect of the invention;
[0275] Figure 8 A schematic diagram showing an audio decoder for providing decoded audio information based on encoded audio information according to an embodiment of the second aspect of the invention;
[0276] Fig. 9 A schematic diagram showing a first spectrum filling method unit according to an embodiment of the second aspect of the present invention.
[0277] Fig.10 A schematic diagram showing an audio decoder according to an embodiment of the second aspect of the invention with additional optional features;
[0278] Fig.11 A schematic diagram showing an audio encoder for providing encoded audio information based on input audio information according to an embodiment of the second aspect of the invention;
[0279] Fig.12 A schematic diagram showing another audio encoder for providing encoded audio information based on input audio information according to an embodiment of the second aspect of the invention is shown.
[0280] Fig.13 A schematic diagram showing an audio decoder for providing decoded audio information based on encoded audio information according to an embodiment of the third aspect of the invention;
[0281] Fig.14 A schematic diagram showing an audio decoder according to an embodiment of the third aspect of the invention with additional optional features;
[0282] Fig.15 An example showing the functionality of a decoder according to an embodiment of the third aspect of the invention;
[0283] Fig.16 A schematic diagram showing an audio decoder according to an embodiment of the third aspect of the invention with additional optional features;
[0284] Fig.17A block diagram showing a method for providing decoded audio information based on encoded audio information according to an embodiment of the first aspect of the invention;
[0285] Fig.18 A block diagram showing a method for providing encoded audio information based on input audio information according to an embodiment of the first aspect of the invention;
[0286] Fig.19 A block diagram showing a method for providing decoded audio information based on encoded audio information according to an embodiment of the second aspect of the invention;
[0287] Fig. 20 A block diagram showing a method for providing encoded audio information based on input audio information according to an embodiment of the second aspect of the invention;
[0288] Fig.21 A block diagram showing a first method for providing decoded audio information based on encoded audio information according to an embodiment of the third aspect of the invention;
[0289] Fig. 22 A block diagram showing a second method for providing decoded audio information based on encoded audio information according to an embodiment of the third aspect of the invention;
[0290] Fig.23 An example graph showing the time domain effects of FD-LTP filtering of a pseudorandom noise spectrum subjected to an inverse transform according to an embodiment of the present invention;
[0291] Fig.24 shows an illustrative example of a reduction in filtering strength according to an embodiment of the present invention; and
[0292] Fig.25 A schematic diagram showing an example of adaptive filtering according to an embodiment of the present invention. DETAILED DESCRIPTION
[0293] In the following description, one or more identical or equivalent components having the same or equivalent functionality are denoted by the same or equivalent reference numerals even if they appear in different drawings.
[0294] In the following description, a number of details are set forth to provide a more thorough explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other cases, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obfuscating embodiments of the present invention. In addition, unless otherwise specifically noted, the features of the different embodiments described herein below may be combined with each other.
[0295] Figure 1A schematic diagram showing an audio decoder for providing decoded audio information based on encoded audio information according to an embodiment of the first aspect of the invention is shown. Figure 1 An audio decoder 100 is shown, which has a spectrum tilt information derivation unit 110, a frequency variable scaling unit 120 and a spectrum hole filling unit 130. Optionally, as Figure 1 As shown in the example of , the decoder 100 may include a decoding unit 140.
[0296] The decoder 100 may be provided with encoded audio information 102. Based on or using the encoded audio information, the spectral tilt information derivation unit 110 may be configured to derive or determine or calculate spectral tilt information 112.
[0297] Optionally, the decoder 100 may be configured to decode the encoded audio information 102 or a part of the encoded audio information using an optional decoding unit 140 in order to obtain a set of decoded spectral values 142. However, it should be noted that the set of decoded spectral values 142 may also be provided from an external device.
[0298] Using the frequency variable scaling unit, the decoder 100 may be configured to apply a frequency variable scaling where the spectrum tilt is determined by the spectrum tilt information 120 to the filling value 122. The filling value 122 may be, for example, a gap filling coefficient or a noise value for noise filling or a gap filling value for smart gap filling (e.g., a spectrum value from a different frequency or frequency band). Therefore, the frequency variable scaling unit may provide the scaled filling value 124 to the spectrum hole filling unit 130.
[0299] Using the spectral hole filling unit 130 , the decoder 100 may be configured to use the modified filling values 122 , 124 in order to fill spectral holes of the decoded set of spectral values 142 .
[0300] Based on the spectral hole filling, decoded audio information 104 may be provided.
[0301] It should be noted that the frequency variable scaling may be performed, for example, after filling the spectral holes of the decoded set of spectral values 142, wherein these holes may have been filled with the unmodified filling values 122. The scaling may then optionally be applied to the modified set of spectral values (e.g. the decoded set of spectral values 142 filled with the filling values 122). As an example, the decoded audio information may thus be provided based on the frequency variable scaling unit 120, wherein the frequency variable scaling unit may receive its input from the spectral hole filling unit 130.
[0302] As explained before, the adaptation of the spectral envelope of the decoded spectral values (eg, decoded spectral coefficients) may allow for a better reconstruction or approximation of the original spectral envelope of the audio information.
[0303] This will be Figure 2 and Figure 3 is further explained in the context of .
[0304] Figure 2 A schematic example of a spectral envelope (intensity as a function of frequency) according to conventional concepts is shown. An example of an original spectral envelope is shown with line 210, which for example represents original spectral values, such as original spectral coefficients representing audio information or a frame or subframe of audio information. Dashed line 220 is offset downwards for better visibility of all curves and represents an example of a masking envelope, such as a masking threshold (e.g., a noise shaping envelope), which can be associated with a scaling factor. Line 230 can show an example of a reconstructed noise envelope according to conventional concepts. According to the masking envelope 220, noise filling can be performed for a signal portion with frequencies between a noise filling start frequency 240 and a noise filling end frequency 250. As shown Figure 2 As shown in , the reconstructed envelope 230 exceeds the original spectral envelope 210 at high frequencies, thus possibly causing audible noise after decoding, while it remains significantly below the original spectral envelope 120 at lower frequencies, thus possibly causing insufficient gap-filling energy and audible spectral holes. It can be seen that the distance between the masking envelope 210 and the effective reconstructed noise envelope 230 may be constant (thin double arrows), and therefore, does not exactly follow the original spectral envelope 210. For example, this may be caused, for example, inter alia, by pre-emphasis tilting during the calculation of the masking envelope.
[0305] Using the embodiments according to the present invention, for example Figure 1 The decoder shown in , can be implemented as Figure 3 The reconstructed noise envelope 310 shown in FIG. Figure 3 A schematic example showing a spectral envelope (intensity as a function of frequency) according to the first aspect of the invention.
[0306] Therefore, using the spectrum tilt information, e.g. Figure 1 , the reconstructed noise envelope 310 may be corrected to reduce the difference between the original signal envelope 210 and the reconstructed envelope 310 .
[0307] about Figure 2 and Figure 3 It should be noted that the masking envelope 220 may be, for example, an approximation or interpolation based on multiple scaling factors for different frequency bands.
[0308] Figure 4 A schematic diagram showing an audio decoder according to an embodiment according to the first aspect of the invention with additional optional features. Figure 4 An audio decoder 400 is shown, which comprises a spectral tilt information derivation unit 410, a frequency variable scaling unit 420, a spectral hole filling unit 430 and an optional decoding unit 440, whose functionality is based on the Figure 1 An audio decoder 100 is provided.
[0309] As an optional feature, the decoder 400 comprises a noise information derivation unit 450. The noise information derivation unit 450 may be configured to derive the noise information 450 from the encoded audio information 402. The noise information may be or may include, for example, noise level information (e.g., L sf ) and / or noise intensity information.
[0310] Furthermore, the decoder 400 may optionally include a filling value obtaining unit 460, which may be configured to use the noise information 450 (e.g., noise level information and / or noise intensity information) to obtain or determine or calculate the filling value 422. Therefore, the filling value 422 may be a noise filling value, wherein the energy of the corresponding noise filling value may be set according to the noise level information.
[0311] Optionally, the frequency variable scaling unit 420 may be configured to apply the frequency variable scaling such that the frequency variable scaling describes a linear decrease in intensity as the frequency increases on a logarithmic intensity scale.
[0312] As another optional feature, the spectral tilt information 412 may describe the spectral tilt in the logarithmic domain.
[0313] As another optional feature, the decoder 400 may include a scaling value obtaining unit 470. The scaling value obtaining unit may, for example, be configured to obtain a scaling value 472 for frequency variable scaling. The decoder 400, such as the scaling value obtaining unit 470 of the decoder 400, may determine or obtain or derive the scaling value 472 in the logarithmic domain. However, it should be noted that the conversion from the logarithmic domain to the linear domain may be performed for any value (e.g., a spectral value or a mathematical operation). Therefore, as an example, the scaling value 472 is used for frequency variable scaling from the logarithmic domain to the linear domain.
[0314] As an example, scaling value 472 may be derived or obtained or calculated, for example, based on or using or depending on a product of tilt value 474 based on tilt information 412 and frequency information 476, such as a frequency value.
[0315] As an example, the tilt value 474 may be provided, for example, by a spectral tilt information derivation unit, for example based on the spectral tilt information 412, or for example directly from the encoded audio information 402. In some embodiments, the spectral tilt information 412 may, for example, be the tilt value 474. For example, the frequency information may be a frequency value or a frequency index, which describes or provides information about the frequency of the spectral values or coefficients to be scaled.
[0316] As an example, the frequency variable scaling unit 420 may be provided with information of the spectral tilt information 412 via a scaling value 472 based on a tilt value 474 .
[0317] Optionally, the scaling value obtaining unit 470 may be configured to obtain a plurality of scaling values for frequency variable scaling associated with different frequency bands.
[0318] As another optional feature, the frequency information 476 may, for example, include the start frequency center frequency of the corresponding frequency bands, and the spectrum values (e.g., spectrum coefficients, such as noise values or gap filling values (e.g., filling values)) of these corresponding frequency bands are to be scaled. Therefore, the scaling value obtaining unit may be configured to obtain the scaling value 472 using the start frequency of the corresponding frequency band or using the center frequency of the corresponding frequency band.
[0319] Similarly, the frequency information 476 may include a start frequency interval index or a center frequency interval index of a corresponding frequency band for obtaining the scaling value 472 .
[0320] As another optional feature, the scaling value 472 may, for example, include a frequency-independent noise scaling value and / or a frequency-variable noise scaling value, wherein the frequency-variable noise scaling value may be determined based on a tilt value 474 (e.g., a spectral tilt). Optionally, the decoder 400, such as the frequency-variable scaling unit 420 of the decoder 200, may be configured to obtain a padding value (e.g., a scaled padding value 424) using a noise value (the noise information may, for example, include a noise value), a multiplication of the frequency-independent noise scaling value and the frequency-variable noise scaling value. The noise value may, for example, be a random noise value or a pseudo-random noise value and may be determined by the noise information derivation unit 450.
[0321] As another optional feature, the audio decoder 400 may be configured to apply a masking envelope based scaling to the decoded spectral values 442 and the padding values 222 .
[0322] In general, the audio decoder may, for example, be configured to obtain the masking envelope from the encoded audio information. The masking envelope may, for example, be associated with a scaling factor, the masking envelope may, for example, be an interpolation of the scaling factor.
[0323] Thus, scaling (e.g., based on a masking envelope) can, for example, be applied to the complete spectrum, e.g., decoded spectral values that have not been quantized to zero, and spectral values that have been quantized to zero and filled with a filling value (or, for example, first scaling the filling value and then filling it in the spectral "hole").
[0324] Optionally, the spectral tilt information derivation unit 410 may, for example, be configured to obtain the spectral envelope from the encoded audio information 402 and may be configured to provide information about the spectral envelope using the spectral tilt information which may be used to adapt the complete spectrum.
[0325] Figure 5 A schematic diagram showing an audio encoder for providing encoded audio information based on input audio information according to an embodiment of the first aspect of the invention is shown. Figure 5An encoder 500 is shown comprising an optional encoding unit 510. For example, the encoding unit may be configured to encode a plurality of quantized spectral values 512. The encoder 500 further comprises an optional spectral tilt information determination unit 520. The spectral tilt information determination unit 520 may be configured to determine spectral tilt information 522 based on spectral energy information 524 and masking envelope information 526. Optionally, the masking envelope information 526 may be provided by a processing unit 530, for example based on input audio information. As an example, the masking envelope may be calculated depending on the input audio information. As another example, a fixed masking envelope may be used. For example, the spectral tilt information may describe an average variation of the difference between the spectral energy of the input audio signal and the masking envelope.
[0326] As an optional example, the encoder 500 may include a processing unit 530, which may be configured to provide spectral energy information 524 (e.g., spectral energy) and quantized spectral values 512 to the spectral tilt information determination unit 520 and the encoding unit 510, respectively, based on the input audio information 502 (e.g., input audio data).
[0327] In addition, the encoding unit 510 may receive the spectral tilt information 522 and may be configured to encode the spectral tilt information. As an example, an encoder, such as the encoding unit 510 of the encoder 500, may be configured to provide the encoded audio information 504, such as an encoded representation including the quantized spectral values 512 and the encoded representation of the spectral tilt information 522.
[0328] Figure 6 A schematic diagram showing an audio encoder according to an embodiment according to the first aspect of the invention with additional optional features. Figure 6 An encoder 600 is shown, which includes a spectral tilt information determination unit 620, an optional processing unit 630 and an encoding unit 610 (and corresponding input / output signals), as shown in Figure 5 explained in the context of .
[0329] As another optional feature, the spectral tilt information determination unit 620 may optionally be configured to determine the spectral tilt information 622 such that the spectral tilt information describes a frequency variation of the difference between the spectral energy information 624 and the masking envelope information 626 as a function of frequency.
[0330] Furthermore, the spectral tilt information 622 may, for example, describe a line function with a spectral tilt in the logarithmic domain. As explained before, the line function may allow adjusting the tilt of the reconstructed spectrum to better approximate the original spectrum of the input audio information 602.
[0331] As another optional feature, the spectral tilt information determination unit may, for example, be configured to determine the spectral tilt information in a logarithmic domain.
[0332] As a further optional feature, the spectral tilt information determination unit 620 may be configured to determine the spectral tilt information 622 based on a difference between a logarithmic representation of the spectral envelope and a logarithmic representation of the masking envelope. Thus, the spectral energy information 624 may include information about the spectral envelope of the input audio information 602, e.g. in logarithmic form, and the masking envelope information 626 may, e.g., include the masking envelope, e.g. in logarithmic form, e.g. including a scaling factor.
[0333] Again, it should be noted that, in general, the encoder may perform any calculation in the logarithmic and / or linear domain. Thus, values and or calculations may be transformed in one domain or the other.
[0334] As another optional feature, the spectral tilt information determination unit 620 may, for example, be configured to use linear regression to obtain the spectral tilt information 622. The inventors have realized that using linear regression a computationally cheap calculation with good accuracy for the tilt information may be performed.
[0335] As another optional feature, the spectral tilt information may, for example, be configured to obtain the spectral tilt information based on a spectral band-by-spectral band energy value or a spectral band-by-spectral band root mean square value representing the energy of the spectral values in multiple corresponding spectral bands and based on a spectral band-by-spectral band energy value or a spectral band-by-spectral band root mean square value representing a masking threshold in multiple corresponding spectral bands.
[0336] As an example, the spectral energy information 624 may therefore include a spectral band-by-spectral band root mean square value representing the energy of the spectral values in multiple corresponding spectral bands, and the masking envelope information 626 may, for example, include a spectral band-by-spectral band energy value or a spectral band-by-spectral band root mean square value representing the masking threshold in multiple corresponding spectral bands.
[0337] Optionally, processing unit 630 may be configured to provide said information.
[0338] As another optional feature, the spectral tilt information determination unit 620 may be configured to determine separate spectral tilt information 622 for different audio frames and / or for different audio subframes.
[0339] As another optional example, the encoder 600 may for example include a difference determiner 640. The difference determiner 640 may for example be configured to determine a difference value 642 representing the difference between the spectral energy information 624 and the masking envelope information 626 in the form of a single value over a frequency range including a plurality of spectral bins.
[0340] Furthermore, the encoder 600 may optionally include a noise level information obtaining unit 650 , which may be configured to obtain or determine or calculate noise level information 652 depending on or based on the difference value 642 .
[0341] As another optional feature, the encoding unit 610 may, for example, receive noise level information 652, and may be configured to encode the noise level information in the encoded audio information.
[0342] Optionally, the difference determiner 640 may be configured to obtain the difference 642 using, for example, linear regression.
[0343] As another optional feature, the encoding unit 610 may, for example, be configured to encode the spectral tilt information 622 using three bits.
[0344] Furthermore, the encoding unit 610 may, for example, be configured to encode the spectral tilt information 622 such that the encoded spectral tilt information always indicates a negative spectral tilt.
[0345] Figure 7 An encoder according to an embodiment of the first aspect of the present invention (eg, Figure 5 The encoder 500 shown in or Figure 6 An example of the functionality of the encoder 600 shown in FIG. 10 is shown in FIG. 10 . Thus, the encoder of the present invention may be configured to perform the following steps:
[0346] 1. Calculate the energy value or RMS value E of each spectrum band from the input spectrum 602 sf (f); (710)
[0347] 2. Set one or more values E sf (f) Convert to the logarithmic domain and from the value E sf (f) Subtract multiple values E sf (f) to obtain the zero mean E' sf (f); (720)
[0348] 3. From zero mean value E' sf Calculate, quantize and dequantize the mask envelope M sf ; (730)
[0349] 4. From M sf Reconstruct the energy value or RMS value of each spectrum band, and from M sf Derive the logarithm and zero mean M' sf (f); (740)
[0350] 5. In paired spectral bands E' sf With M' sf Perform linear regression between sf and offset O sf ; (750)
[0351] 6. From T sf Quantize and dequantize the tilt index T sf ; (760)
[0352] 7. From t sf Reconstruct the tilt value to obtain the decoded tilt T' sf , and use -T' sf *f is used to calculate the noise level index I sf (770).
[0353] Figure 8 A schematic diagram showing an audio decoder for providing decoded audio information based on encoded audio information according to an embodiment of the second aspect of the invention is shown. Figure 8 The decoder 800 is shown comprising a spectral hole filling unit 810, which is configured to fill spectral holes of a decoded set of spectral values 812. The result of the hole filling may be the decoded audio information 802, for example.
[0354] As another optional feature, the decoder 600 may include a prediction lag information obtaining unit 820. The prediction lag information obtaining unit 820 may be configured to obtain or determine or calculate prediction lag information 822. As another optional feature, the prediction lag information obtaining unit 820 may receive encoded audio information 804 that may be used to determine the prediction lag information 822.
[0355] As a further optional feature, the decoder 800 comprises a decoding unit 830. The decoding unit 830 may be configured to provide a set 812 of decoded spectral values based on the encoded audio information 804.
[0356] Furthermore, the decoder 800 may include a first spectrum filling method unit 840 and a second spectrum filling method unit 850 (optionally, the decoder 800 may include a plurality of second (e.g., other) spectrum filling method units or the second spectrum filling method unit may be configured to provide the functionality of a plurality of other spectrum filling methods). The respective spectrum filling method unit may, for example, be configured to provide a filling value 814 to the spectrum hole filling unit 810 in order to fill the spectrum holes.
[0357] Based on the predicted hysteresis information 822 , switching (using switch 860 ) may be performed between a first spectrum filling method unit 840 and a second spectrum filling method unit 850 (or eg a plurality of other spectrum filling method units) to provide the filling value 814 to the spectrum hole filling unit 810 .
[0358] Using the first spectrum filling method, frequency filtering or frequency prediction may be used to obtain filling values for filling spectrum holes. Using the second spectrum filling method, filling values for filling spectrum holes may be obtained without frequency filtering and without frequency prediction.
[0359] Optionally, the decoder 800 may be configured, for example, to use a first spectrum filling method when the prediction lag information 822 is non-zero, or to use the first spectrum filling method when the prediction lag information 822 is greater than zero, and to use a second spectrum filling method (e.g., one of one or more other spectrum filling methods) in other cases.
[0360] As another optional feature, the prediction lag information obtaining unit may, for example, be configured to use an encoded representation of the prediction lag value included in the encoded audio information 804 to obtain the prediction lag information 822, such as the prediction lag value.
[0361] Fig. 9 A schematic diagram showing a first spectrum filling method unit according to an embodiment of the second aspect of the present invention. Fig. 9 Displayable Figure 8 Schematic diagram of details of the first spectrum filling method unit 840. Fig. 9 A prediction or filtering unit 910 and a filter strength adaptation unit 920 are shown.
[0362] For example, the audio decoder of the present invention may be configured to depend on another spectral value 914 associated with a different frequency, for example determined using the prediction or filtering unit 910 to obtain a filling value 912 associated with a given frequency. The prediction or filtering unit 910 may therefore be configured to use or apply prediction or filtering.
[0363] Furthermore, the audio decoder of the present invention may be configured to adapt the filter strength information 922, such as the filter strength, for example using the filter strength adaptation unit 920, depending on the encoded or quantized spectral values 924 associated with different frequencies. As an example, the spectral values 914 may be used, for example, instead to adapt the filter strength.
[0364] Optionally, the filter strength information 922 may, for example, include a filter strength, wherein the filter strength determines the influence of the further spectral value 914 on the fill value 912 .
[0365] Optionally, for example using a filter strength adaptation unit 920, the decoder of the present invention can be configured to adapt filter strength information, such as filter strength, depending on spectral values 924 associated with different frequencies (as determined by the encoded representation of individual spectral values in the encoded audio information).
[0366] Optionally, an adaptation of the filter information 922, eg the filter strength, depending on the spectral values 924 associated with different frequencies may be performed, eg before applying the noise filling.
[0367] As another optional feature, the filter strength adaptation unit may be configured to adapt the filter information 922, such as the filter strength, depending on whether the spectral values 924 or 914 associated with different frequencies (or values) are or have been quantized to zero. Thus, the filter strength may be adjusted depending on the masking envelope of, for example, a scaling factor. The inventors have recognized that zero-quantized spectral values may be filtered in different ways in order to improve the decoding of the audio information.
[0368] Optionally, the decoder of the present invention may be configured to adapt (eg, using a filter strength adaptation unit 920) the filter strength depending on whether noise filling is or has been applied to spectral values 924 or 914 associated with different frequencies (or values).
[0369] Optionally, the decoder of the present invention may be configured to selectively apply filtering in the frequency direction or prediction in the frequency direction, for example using the prediction or filtering unit 910 for the spectral values 924 or 914 to which noise filling is applied.
[0370] Optionally, the decoder of the present invention can be configured to, for example, use a prediction or filtering unit 910 to apply prediction or filtering so as to determine a given fill value 912 based on a random or pseudo-random noise value. Optional noise value information 916, for example, including a random or pseudo-random noise value, can therefore be provided to the prediction or filtering unit 910. Optional noise value information 916 can include random and / or pseudo-random noise values. These values can, for example, be provided by a noise generator (not shown). Therefore, the decoder can optionally include a noise generator. In addition, the noise value information can include a noise generator signal, such as a random and / or pseudo-random noise value.
[0371] As another optional feature, the spectral value associated with the different frequencies 924 or 914 may be, for example, a filling value associated with another frequency. Optionally, the spectral value associated with the different frequencies 924 or 914 may be, for example, a noise value associated with another frequency. The inventors recognize that the filling value 912 may be determined, for example, using a weighted combination of a noise value associated with a given frequency and a noise value associated with another frequency or a weighted combination of a noise value associated with a given frequency and a filling value associated with another frequency. Therefore, the prediction and / or filtering unit 910 may be configured to perform one or two weighted combinations in order to obtain a given filling value 912.
[0372] Furthermore, the decoder of the present invention may include a weight adjustment unit 930. For example, the weight adjustment unit may be configured to adjust the weight given to the noise value associated with another frequency or the weight given to the filling value associated with another frequency depending on whether noise filling has been applied to the spectral value 924 or 914 associated with another frequency.
[0373] Optionally, the audio decoder of the present invention may include a spectral distance determination unit 940, which may be configured to determine the spectral distance between a fill value 912 associated with a given frequency and another spectral value 924 or 914 associated with a different frequency based on encoded information describing the spectral distance included in the encoded representation 804 of the audio information.
[0374] Optionally, the weight adjustment unit 930 may receive the encoded representation of the audio information 804. The weight adjustment unit 930 may be configured to determine weight information 932, such as a weight, to apply to a noise value associated with a given frequency based on gain information included in the encoded representation of the audio information 804.
[0375] As another optional feature, the weight adjustment unit 930 may be configured to determine weight information 932, such as a weight to be applied to a noise value associated with another frequency or to a fill value associated with another frequency, depending on gain information included in the encoded representation 804 of the audio information.
[0376] As another optional feature, the weight adjustment unit 930 may be configured to determine weight information 932, such as a weight to be applied to a noise value associated with another frequency or to a fill value associated with another frequency, depending on symbol information included in the encoded representation of the audio information.
[0377] Optionally, the prediction or filtering unit 910 may be configured, for example, to sf ) (e.g., spectral values associated with different frequencies 924 or 914) are obtained using noise filling according to Determines the given fill value (e.g. fill value 912), and in the coefficient c(iP' sf ) is not obtained using noise filling. Determines the given fill value Where c(i) represents the spectral coefficient obtained using noise filling and having spectral index i; where d represents the attenuation coefficient, where G' sf (eg, weight information 932) represents a weight based on a gain value included in the encoded audio representation 804; and wherein c(iP' sf ) represents a spectrum index iP' sf The spectral coefficients of P's f are prediction parameters or filter parameters based on prediction parameter information included in the encoded audio representation.
[0378] As another optional feature, the decoder of the present invention can be configured tosf =p sf +B obtains prediction parameters or filtering parameters P' sf , where p sf is a lag index included in the encoded audio representation 804, and wherein B is a constant and / or wherein the audio decoder is configured to calculate the lag index according to G' sf =(-1) Ssf *(3+2*g sf ) / 8 to obtain the weight G' sf , where S sf is a binary value included in the encoded representation, and where g sf is a binary value included in the encoded representation; and / or wherein the audio decoder is configured according to d=(7.5-g sf ) / 8 to obtain the attenuation coefficient d, where g sf is the binary value included in the encoded representation.
[0379] Optionally, the filter strength adaptation unit may for example be configured to selectively use a reduced filter strength applied to spectral coefficients not marked as noise filled zero quantized spectral coefficients. Optionally, the decoder may comprise a marking unit for marking noise filled zero quantized spectral coefficients (not shown).
[0380] Fig.10 A schematic diagram showing an audio decoder according to an embodiment of the second aspect of the invention with additional optional features. Fig.10 A decoder 1000 is shown, which includes a spectrum hole filling unit 1010, a prediction hysteresis information obtaining unit 1020, a decoding unit 1030, a first spectrum filling method unit 1040, a second spectrum filling method unit 1050, and a switch 1060. The functionalities of these components may be similar or similar to, for example, Figure 8 and Fig. 9 corresponding components.
[0381] As a further optional feature, the decoder 1000 comprises a third spectrum filling method unit 1070. For example, the decoder 1000 may be configured to switch between the second spectrum filling method (e.g. using the spectrum filling method unit 1050, wherein random or pseudo-random filling values are used to fill the spectrum holes (e.g. providing the corresponding filling values 1014 to the spectrum hole filling unit 1010)) and the third spectrum filling method 1070 (wherein filling values 1014 obtained using replication of non-zero spectral coefficients are used to fill the spectrum holes) depending on the prediction lag information and / or depending on the tonality information 1082 (e.g. tonality) of the audio information.
[0382] As another optional feature, the decoder 1000 may include a tonality information obtaining unit 1080 , which may be configured to obtain tonality information 1082 based on the encoded audio information 1004 .
[0383] Optionally, the tonality information obtaining unit 1080 may be configured to determine whether the audio information is tonal, e.g., depending on the tonality information included in the encoded audio representation 1004 and / or depending on the information indicating whether the tonality information is included in the encoded audio information and / or depending on the filter gain value and / or depending on the prediction gain value and / or depending on the time domain post-filter gain value. The tonality information obtaining unit 1080 may therefore be configured to determine or extract the corresponding information for determining from the encoded audio information 1004. Thus, the tonality information obtaining unit 1080 may receive the encoded audio information and / or at least one of, e.g., the information indicating whether the tonality information is included in the encoded audio information, the filter gain value, the prediction gain value and / or the time domain post-filter gain value.
[0384] As another optional feature, the spectrum hole filling unit 1010 may be configured to apply high frequency noise gain adjustment for filling spectrum holes in higher frequency regions below the noise filling end frequency.
[0385] As an example, the decoding unit 1030 may be configured to obtain the high frequency energy information 1032 based on the encoded audio information 1004 .
[0386] As another optional feature, the decoder 1000 (eg, the decoding unit 1030) may be configured to obtain the high-frequency energy difference value depending on the high-frequency energy value, depending on the global gain value, and depending on the noise level information. Fig.10 In the example of , the HF energy information 1032 may include a high frequency energy difference value. Furthermore, the high frequency energy value, the global gain value and / or the noise level information may be included in the encoded audio information 1004, for example, in an encoded form.
[0387] Optionally, the audio decoder may be configured to apply the high frequency energy delta value to obtain one or more noise filling values. As an example, according to an embodiment, the filling value 1014 may be a noise filling value, and the spectrum hole filling unit 1010 may be configured to apply the high frequency energy delta value provided by the decoding unit to adapt the filling value 1014 to "fill" the noise filling value into the decoded spectrum value set.
[0388] As another optional feature, the audio decoder 1000 (e.g., the spectral hole filling unit) may be configured to selectively multiply one or more intermediate noise filling values (e.g., filling value 1014) associated with frequencies in a higher frequency region below the noise filling end frequency by the high frequency energy difference value.
[0389] Optionally, the audio decoder 1000 (eg, the spectral hole filling unit 1010) may be configured to selectively apply the high frequency noise gain adjustment to the spectral values for which the noise filling is performed. As an example, the high frequency noise gain adjustment may be applied in the frequency range between 8 kHz and 10 kHz.
[0390] Optionally, the high frequency energy value or the high frequency energy difference value may represent the energy of a plurality of spectral coefficients quantized to zero at a frequency lower than the noise filling end frequency or in a frequency region lower than the noise filling end frequency.
[0391] Fig.11 A schematic diagram showing an audio encoder for providing encoded audio information based on input audio information according to an embodiment of the second aspect of the invention is shown. Fig.11 An encoder 1100 is shown comprising an optional encoding unit 1110. The encoding unit 1110 may be configured to encode a plurality of quantized spectral values 1112.
[0392] Furthermore, the encoder 1100 comprises a lag value obtaining unit 1120 which may be configured to obtain a lag value 1122 defining characteristics of a filtering operation or a prediction operation to be performed by the audio decoder to derive one or more filling values for filling the spectral holes.
[0393] Furthermore, the encoder 1100 comprises a gain value obtaining unit 1130 which may be configured to obtain a gain value 1132 defining characteristics of a filtering operation or a prediction operation to be performed by the audio decoder to derive one or more filling values for filling the spectral holes.
[0394] The encoder 1100 may further include a hysteresis value modification unit 1140 which may be configured to set the hysteresis value 1122 to zero if the gain value 1132 is less than a threshold or if the absolute value of the gain value is less than a threshold, thereby obtaining a modified hysteresis value 1142 .
[0395] The encoding unit 1110 may be configured to encode the determined hysteresis value 1122 or the modified hysteresis value 1142 .
[0396] Thus, as an option, the quantized spectral values 1112 and the (modified) lag values 1122 / 1142 may be encoded in the encoded audio information 1102 using the encoding unit 1110 .
[0397] Furthermore, the encoder xx may comprise an optional processing unit 1150 for providing quantized spectral values 1112 to the encoding unit 1110 based on the input audio information 1104 .
[0398] As an example, the hysteresis value 1122 and the gain value 1132 may be determined or calculated using or based on autocorrelation information applied to a set of spectral values 1152, which may, for example, be associated with the spectral value 1112. For example, the gain value 1132 may be determined depending on a peak of an autocorrelation function obtained based on the set of spectral values. As an example, the processing unit 1150 may be configured to provide the set of spectral values 1152 to the hysteresis value obtaining unit 1120 and the gain value obtaining unit 1130. The spectral value 1152 may, for example, be quantized and may, for example, be equal to the quantized spectral value 1112.
[0399] Optionally, the encoding unit 1110 may be configured to encode the gain value 1132 if the encoded lag value is non-zero. The lag value 1122 / 1142 may include information about a dependency or correlation between spectral coefficients (e.g., spectral values), for example, on different frequency bands. In the event that such a correlation exists, the lag value 1122 may be non-zero, and thus the dependency may be characterized by the gain value 1132.
[0400] In general, in short, and as an example, the hysteresis value 1122 may describe the distance in the frequency domain between a spectrum value having a given frequency and another spectrum value having a different frequency. For example, a gain value may describe or quantify the correlation between spectrum values. Thus, one spectrum value may be determined by another spectrum value and gain and hysteresis information. Therefore, the transmission of the second spectrum value may not be necessary if the hysteresis and gain information are known.
[0401] As another optional feature, the processing unit 1150 may, for example, be configured to determine or calculate a high frequency (HF) energy value 1154. The HF energy value 1154 may include information for adjusting a HF gap filling range.
[0402] Optionally, the encoding unit 1110 may be configured to selectively encode a high frequency energy value 1154, which may describe the energy in the upper part of the spectrum (e.g., the quantized spectral value) in case the encoded lag value is zero. As an example, a lag value 1122 / 1142 of zero may indicate that no correlation between spectral coefficients may be used to encode the spectral value. Thus, instead of encoding the lag and gain values, the HF energy value may be encoded to perform gap filling, e.g., so that in the decoder the spectral energy in the gap filling range is adapted according to the HF energy value 1154.
[0403] As another optional feature, the encoding unit 1110 may be configured to selectively encode the gain value 1132 or the high frequency energy value 1154 depending on the encoded lag value.
[0404] Furthermore, the encoding unit 1110 may optionally be configured to use the same number of bits to encode the gain value 1132 and the high frequency energy value 1154. Thus, the encoding scheme (e.g., the number of bits reserved for particular information in the encoded bit stream) may remain constant in either case, or in other words, remain constant regardless of whether the gain value or the high frequency energy value is encoded.
[0405] As another optional feature, the encoder 1100 may optionally be configured to determine separate lag values 1122 / 1142 and / or separate gain values 1132 for different audio frames and / or for different audio subframes.
[0406] Additionally, the hysteresis values 1122 / 1142 and / or the gain value 1132 may be determined or calculated in the transform domain.
[0407] Optionally, the hysteresis value obtaining unit 1120 may be configured to perform long-term transient detection and selectively set the hysteresis value 1122 to zero if the audio frame or the audio subframe is found not to be a long-term transient.
[0408] Fig.12 A schematic diagram showing another audio encoder for providing encoded audio information based on input audio information according to an embodiment of the second aspect of the invention is shown. Fig.12 An encoder 1200 is shown, which includes an optional processing unit 1210 and an optional encoding unit 1220.
[0409] The encoding unit 1220 may be configured to encode a plurality of quantized spectral values 1222. Furthermore, the encoding unit 1220 may be configured to encode high frequency energy values or high frequency energy difference values 1224. Thus, the encoder 1200 may provide encoded audio information 1202 comprising encoded representations of quantized spectral values and / or HF energy (difference) values.
[0410] The processing unit 1210 may be configured to provide the quantized spectral value 1222 based on the input audio information 1204. Furthermore, the processing unit 1210 may be configured to provide the HF energy (difference) value 1224 using the input audio information 1204.
[0411] The high frequency energy value or the high frequency energy difference value may represent the energy of a plurality of spectral coefficients quantized to zero at a frequency lower than the noise filling end frequency or in a frequency region lower than the noise filling end frequency. As an example, the HF energy value (or difference, e.g., in the case of differential entropy coding) 1224 may represent the raw RMS energy of the spectral coefficients normalized in a spectro-temporal manner slightly below the noise filling end frequency quantized to zero (e.g., in the frequency range of 8 kHz to 10 kHz).
[0412] Optionally, the processing unit 1210 may be further configured to logarithmically quantize the high-frequency energy value or the high-frequency energy difference value (as an example), thereby providing a quantized representation of the high-frequency energy value or the high-frequency energy difference value to the encoding unit 1220.
[0413] As another optional feature, the processing unit 1210 may be configured to convert, for example, GG sf The global gain 1212 and / or noise information 1214 (e.g., noise level, e.g., L sf ) is provided to an encoding unit 1220 (which may be determined based on the input audio information 1204). The encoding unit 1220 may optionally be configured to encode a high frequency energy difference value relative to the product of the global gain 1212 and the noise level 1214, which may optionally describe the energy of a plurality of spectral coefficients quantized to zero at a frequency below the noise filling end frequency or in a frequency region below the noise filling end frequency.
[0414] As another optional feature, the encoding unit 1220 may, for example, be configured to obtain a rounded scaling result of the logarithm of the ratio between the high-frequency energy value and the product of the global gain 1212 and the noise information 1214 (e.g., in the form of a noise value) in order to encode the high-frequency energy value.
[0415] Optionally, the processing unit 1210 may be configured to determine the quantized high frequency energy difference value according to the following formula:
[0416] E sf =1+round(Δ*log2(EHF sf / (GG sf *L sf )),
[0417] Where EHF is the high frequency energy value, GG sf is the global gain 1212, where L sf is the noise level 1214, and where Δ is a constant.
[0418] Fig.13 A schematic diagram showing an audio decoder for providing decoded audio information based on encoded audio information according to an embodiment of the third aspect of the invention is shown. Fig.13 An audio decoder 1300 is shown comprising an optional spectral hole filling unit 1310 which may be configured to fill spectral holes of a decoded set of spectral values 1312 with corresponding filling values 1314 .
[0419] The decoder 1300 may further comprise an optional prediction or filtering unit 1320 which may be configured to determine the respective filling value 1314 using prediction or filtering such that a given filling value 1314 associated with a given frequency is obtained dependent on another spectral value 1322 associated with a different frequency.
[0420] Furthermore, the decoder 1300 may optionally include a filtering strength adaptation unit 1330. The filtering strength adaptation unit 1330 may provide filtering strength information 1332 (e.g., information about the filtering strength) to the prediction or filtering unit 1320. The filtering strength adaptation unit 1330 may be configured to adapt the filtering strength using the spectral values 1322 associated with different frequencies provided to the prediction or filtering unit 1320 depending on the encoded or quantized spectral values 1334 (optionally, for example, as an alternative).
[0421] As an optional feature, the decoder 1330 may include a decoding unit 1340 that may be configured to provide a decoded set of spectral values 1312 to the spectral hole filling unit 1310 using or based on the encoded audio information 1302. Optionally, the decoding unit 1300 may provide spectral values 1322 associated with different frequencies (e.g., determined from the encoded audio information 1302) to the prediction or filtering unit 1320 and / or the filtering strength adaptation unit 1330.
[0422] Optionally, the filtering strength determines (e.g., as part of the filtering strength information 1332) an influence of another spectral value 1322 on a given filling value 1314. Furthermore, the filtering strength adaptation unit 1330 may, for example, be configured to adapt the filtering strength depending on the spectral values 1334 associated with different frequencies as determined by the encoded representation of the respective spectral values in the encoded audio information 1302. Optionally, the filtering strength may be adapted depending on the spectral values 1334 associated with different frequencies before applying noise filling and / or depending on whether the spectral values 1334 associated with different frequencies (or values) are quantized to zero and / or depending on whether noise filling is applied to the spectral values 1334 associated with different frequencies (or values).
[0423] As another optional feature, the prediction or filtering unit 1320 may be configured for selectively applying filtering in the frequency direction or prediction in the frequency direction for the spectral values to which the noise filling is applied.
[0424] Fig.14 A schematic diagram showing an audio decoder according to an embodiment of the third aspect of the invention with additional optional features. Fig.14Decoder 1400 is shown, which includes an optional decoding unit 1410, a spectral hole filling unit 1420, a prediction or filtering unit 1430, and a filter strength adaptation unit 1440. These components may include the same Fig.13 The same or similar or like functionality and corresponding input and / or output signals as explained in the context of .
[0425] In addition, as another optional feature, the audio decoder 1400, for example the prediction or filtering unit 1440 of the audio decoder 1400, can be configured to apply prediction or filtering in order to determine the given filling value 1432 based on a random or pseudo-random noise value. Therefore, the noise value information 1436 (e.g., including the random or pseudo-random noise value) can be provided to the prediction or filtering unit 1430.
[0426] Optionally, the noise value information 1436 may, for example, include a noise value associated with a given frequency and / or a noise value associated with another frequency. As another optional feature, the spectrum value 1454 or 1434 associated with different frequencies may, for example, be a filler value associated with another frequency. Thus, optionally, the decoder 1400 may include a means for providing the noise value information 1436 and or filler values associated with different frequencies (e.g., so in this case, optionally not provided by the decoding unit 1410). For example, the decoding unit 1410 may provide information that the spectrum value is quantized to zero, which may be replaced based on the noise value information or based on filler values with different frequencies (e.g., as explained below).
[0427] As another optional feature, the audio decoder 1400, for example its prediction or filtering unit 1430, can be configured to perform a weighted combination of a noise value associated with a given frequency and a noise value associated with another frequency, or a weighted combination of a noise value associated with a given frequency and a filling value associated with another frequency, so as to obtain a given filling value 1432.
[0428] Furthermore, the decoder 1400 may include a weight adjustment unit 1450, which may be configured to adjust the weight given to the noise value associated with the other frequency or the weight given to the filling value associated with the other frequency, depending on whether noise filling has been applied to the spectral value associated with the other frequency 1454 or 1434. Thus, weight information 1452, for example including information about the respective weights, may be provided from the weight adjustment unit 1450 to the prediction or filtering unit 1430.
[0429] As another optional feature, the decoder 1400 may include a spectral distance determination unit 1460, which may be configured to determine the spectral distance between a fill value 1430 associated with a given frequency and another spectral value 1454 or 1434 associated with a different frequency based on encoded information describing the spectral distance included in the encoded representation 1402 of the audio information.
[0430] Optionally, the audio decoder 1400, e.g., the weight adjustment unit 1450, may be configured to determine a weight (wherein the weight information 1452 may include a weight) to be applied to a noise value associated with a given frequency (wherein the noise value information 1436 may include a noise value) based on gain information included in the encoded representation 1402 of the audio information.
[0431] As another optional feature, the audio decoder 1400, e.g. the weight adjustment unit 1450, may be configured to determine a weight (wherein the weight information 1452 may include a weight) depending on gain information included in an encoded representation of the audio information, which weight is applied to a noise value associated with another frequency (wherein the noise value information 1436 may include a noise value) or to a fill value associated with another frequency.
[0432] Thus, the encoded representation 1402 of the audio information may include an encoded representation of the gain information, which may be decoded by the decoding unit 1410 and provided to the weight adjustment unit 1450, for example. As an example, adjustment information 1454, optionally including the gain information, may be provided to the weight adjustment unit 1450.
[0433] Optionally, the audio decoder, eg the weight adjustment unit 1450, may be configured to determine a weight to be applied to a noise value associated with another frequency or to a filling value associated with another frequency depending on sign information included in the encoded representation 1402 of the audio information.
[0434] Thus, the encoded representation 1402 of the audio information may include an encoded representation of the sign information that may be decoded, for example, by the decoding unit 1410 and provided to the weight adjustment unit 1450 , for example, in the adjustment information 1454 .
[0435] As an example, an audio decoder, such as the prediction or filtering unit 1430, may be configured to calculate the coefficients c(iP' sf )1454 or 1434 is obtained by using noise filling. Determines the given fill value And in the coefficient c(iP' sf )1454 or 1434 is not obtained by using noise filling. Determines the given fill value Where c(i) represents the spectral coefficient obtained using noise filling and having spectral index i; where d represents the attenuation coefficient, where G' sf represents a weight 1452 based on a gain value 1454 included in the encoded audio representation 1402; and wherein c(iP' sf ) represents a spectrum index iP' sf The spectral coefficients of P' sf are prediction parameters or filter parameters based on prediction parameter information included in the encoded audio representation.
[0436] As another optional feature, the audio decoder 1400, for example the decoding unit 1410, may be configured to generate a decoded signal according to P'. sf =p sf +B obtains prediction parameters or filtering parameters P' sf , where p sf is the lag index included in the encoded audio representation, and where B is a constant.
[0437] Alternatively or additionally, the audio decoder, such as the weight adjustment unit 1450, may be configured to adjust the weight according to G' sf =(-1) Ssf *(3+2*g sf ) / 8 to obtain the weight G' sf (The weight information may include weight G' sf ), where S sf is a binary value included in the encoded representation, and where g sf is the binary value included in the encoded representation.
[0438] Alternatively or additionally, the audio decoder, such as the decoding unit 1410, may be configured to decode the audio signal according to d=(7.5-g sf ) / 8 to obtain the attenuation coefficient d, where g sf is the binary value included in the encoded representation.
[0439] If optionally in Fig.14 As shown in FIG. 1 , the decoding unit 1410 may, for example, include prediction parameters or filter parameters P 'sf The parameter information 1412 of the constant B and / or the attenuation coefficient d is provided to the prediction or filtering unit 1430. However, it should be noted that the decoding unit 1410 may optionally only provide the decoded spectral value set 1422 to the spectral hole filling unit 1420, and the decoder 1400 may include one or more dedicated obtaining and / or calculating and / or determining units for providing the corresponding information, for example based on the encoded audio information 1402.
[0440] As another optional feature, the audio decoder 1400 may be configured to mark noise-filled zero-quantized spectral coefficients and selectively use a reduced filtering strength applied to unmarked spectral coefficients.
[0441] Fig.15 A decoder according to an embodiment of the third aspect of the present invention (eg, Fig.14 The decoder 1400 shown in or Fig.13 An example of the functionality of the decoder 1300 shown in FIG. 10 is shown in FIG. 10 . Therefore, the decoder of the present invention can be configured to perform the following steps:
[0442] 1. Set P' sf =p sf +B, G' sf =(-1) Ssf *(3+2*g sf ) / 8 and d=(7.5-g sf ) / 8(1501);
[0443] 2. Perform noise filling and mark the noise-filled zero quantized spectral coefficients (1502);
[0444] 3. For position i>=P' sf The multiple noise filled zero quantized spectrum coefficients c at the locations are processed (1503):
[0445] 4. If the position iP' is marked in step 2 sf The coefficient c at the point is d*c(i)+G' sf *c(iP' sf ) replaces c(i); otherwise
[0446] 5. Use d*c(i)+1 / 2*G' sf *c(iP' sf ) replaces c(i).
[0447] Fig.16 A schematic diagram showing an audio decoder according to an embodiment of the third aspect of the invention with additional optional features. Fig.16 A decoder 1600 is shown comprising an optional prediction or filtering unit 1610 which may be configured to determine a processed spectral value 1612 using prediction or filtering such that a given processed spectral value 1612 associated with a given frequency is obtained depending on another spectral value 1614 associated with a different frequency.
[0448] As an optional feature, the decoder 1600 may include a decoding unit 1620 which may be configured to provide spectral values 1614 associated with different frequencies to the prediction or filtering unit 1610 based on the encoded audio representation 1602 .
[0449] As another optional feature, the decoder 1600 may include a filter strength adaptation unit 1630, which may be configured to adapt the filter strength depending on the encoded or quantized spectral values 1634 (e.g., optionally alternatively, 1614 associated with different frequencies). Thus, the filter strength adaptation unit may provide filter strength information 1632 to the prediction or filtering unit 1610. Optionally, the spectral values 1634 may be provided by the decoding unit based on the encoded audio representation.
[0450] Optionally, the filter strength adaptation unit 1630 may be configured to adapt the filter strength to reduce the contribution of non-zero quantized spectral coefficients included in the prediction or filtering.
[0451] Fig.17 A block diagram of a method for providing decoded audio information based on encoded audio information according to an embodiment of the first aspect of the invention is shown. The method 1700 comprises: deriving 1710 spectral tilt information from the encoded audio information; using 1720 filling values in order to fill spectral holes of a decoded set of spectral values; and applying 1730 frequency variable scaling to the filling values, the spectral tilt of the frequency variable scaling being determined by the spectral tilt information.
[0452] Fig.18 A block diagram of a method for providing encoded audio information based on input audio information according to an embodiment of the first aspect of the present invention is shown. The method 1800 comprises: encoding 1810 a plurality of quantized spectral values; determining 1820 spectral tilt information based on spectral energy information and masking envelope information; and encoding 1830 the spectral tilt information.
[0453] Fig.19 A block diagram of a method for providing decoded audio information based on encoded audio information according to an embodiment of the second aspect of the invention is shown. The method 1900 comprises: filling 1910 spectral holes of a decoded set of spectral values; obtaining 1920 prediction lag information; and switching 1930 between a first spectral filling method and one or more other spectral filling methods depending on the prediction lag information, in which the filling values for filling the spectral holes are obtained using frequency filtering or frequency prediction, and in which the filling values for filling the spectral holes are obtained without frequency filtering and without frequency prediction.
[0454] Fig. 20A block diagram of a method for providing encoded audio information based on input audio information according to an embodiment of the second aspect of the invention is shown. The method 2000 comprises: encoding 2001 a plurality of quantized spectral values; obtaining 2002 a hysteresis value, which defines the characteristics of a filtering operation or a prediction operation to be performed by an audio decoder to derive one or more filling values for filling spectral holes; obtaining 2003 a gain value, which defines the characteristics of a filtering operation or a prediction operation to be performed by an audio decoder to derive one or more filling values for filling spectral holes; setting 2004 the hysteresis value to zero if the gain value is less than a threshold value or if the absolute value of the gain value is less than a threshold value, thereby obtaining a modified hysteresis value; and encoding 2005 the determined hysteresis value or the modified hysteresis value.
[0455] Fig.21 A block diagram of a first method for providing decoded audio information based on encoded audio information according to an embodiment of the third aspect of the invention is shown. The method 2100 comprises: filling 2101 spectral holes of a decoded set of spectral values using corresponding filling values; determining 2102 the filling values using prediction or filtering so that a given filling value associated with a given frequency is obtained depending on another spectral value associated with a different frequency; and adapting 2103 the filtering strength depending on the encoded or quantized spectral values associated with the different frequencies.
[0456] Fig. 22 A block diagram of a second method for providing decoded audio information based on encoded audio information according to an embodiment of the third aspect of the invention is shown. The method 2200 comprises: determining 2201 a processed spectral value using prediction or filtering so that a given processed spectral value associated with a given frequency is obtained depending on another spectral value associated with a different frequency; and adapting 2202 a filtering strength depending on the encoded or quantized spectral value associated with the different frequencies.
[0457] In the following, other embodiments of the present invention are disclosed. In addition, the embodiments according to the present invention as discussed above will be explained in different words. It should be noted that any features, functionalities and details as discussed above can be optionally used or incorporated with any one of the embodiments explained below, and vice versa.
[0458] In addition, it should be noted that, hereinafter, some embodiments according to the present invention are explained in a structured manner according to different inventive aspects. However, the following structuring in each aspect may, for example, be different from the structuring explained previously, so as to highlight the different features, functionality and details of the embodiment and the corresponding combinability of such features, functionality and details. As an example, aspect 1 as explained previously may correspond to aspect 1 below. Aspect 2 as explained previously may, for example, correspond to aspects 2, 3 and 4 below. Aspect 3 as explained previously may, for example, correspond to aspects 2, 3 and 4 below. However, these are only examples, and it should be noted again that any features, functionality and details according to any embodiment may be incorporated into any other embodiment or used together with any other embodiment, for example, without regard to the classification to different aspects. This classification may, for example, be used only to provide examples of clusters of embodiments, so that those skilled in the art may form a better understanding of the present invention.
[0459] In the following, different inventive embodiments and aspects will be described in the “Method and apparatus for improving spectral gap filling in a spectro-temporal manner in audio coding”, for example in the section “Introduction, conventional solutions”, for example in the section “Deficiencies of conventional solutions”, for example in the section “Content of the invention”, for example in the section “Description of the invention”, for example in the section “Application of FD-LTP adaptive filtering aspects to temporal noise shaping filtering” and for example in the section “Proposal for three [e.g., partially] non-dependent decoders according to embodiments of the invention”.
[0460] Furthermore, other embodiments are defined by the following claims.
[0461] It should be noted that any embodiment as defined by the claims may be supplemented by any of the details (features and functionalities) described in the above-mentioned sections, respectively, and by any of the details (features and functionalities) described in the context of any other embodiment as disclosed herein.
[0462] Furthermore, any of the embodiments described in the above-mentioned section and the previously discussed embodiments may be used alone and may also be supplemented by any of the features in another section or from any other embodiment or by any features included in the claims.
[0463] Also, it should be noted that the individual aspects described herein can be used alone or in combination. Thus, details can be added to each of the individual aspects without adding details to another of these aspects.
[0464] Furthermore, the features and functionality disclosed herein in relation to the methods may also be used in a device (configured to perform such functionality). In addition, any features and functionality disclosed herein with respect to the devices may also be used in the corresponding methods. In other words, the methods disclosed herein may be supplemented by any of the features and functionality described with respect to the devices.
[0465] Also, any of the features and functionalities described herein may be implemented in hardware or software, or using a combination of hardware and software, as will be described in "Implementation Alternatives".
[0466] Implementation Alternatives :
[0467] Although some aspects have been described or will be described in the context of a device, it is apparent that these aspects also represent descriptions of corresponding methods, where a block or device corresponds to a method step or a feature of a method step. Similarly, the aspects described in the context of a method step also represent descriptions of corresponding blocks or items or features of a corresponding device. Some or all of the method steps may be performed by (or using) a hardware device (e.g., a microprocessor, a programmable computer, or an electronic circuit). In some embodiments, one or more of the most important method steps may be performed by such a device.
[0468] The encoded audio signal of the present invention may be stored on a digital storage medium or may be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
[0469] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or in software. Implementation may be performed using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, on which is stored electronically readable control signals that cooperate (or are capable of cooperating) with a programmable computer system to cause the corresponding method to be performed. Thus, the digital storage medium may be computer readable.
[0470] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
[0471] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.
[0472] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
[0473] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0474] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitory.
[0475] Therefore, a further embodiment of the inventive method is a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection (eg, via the Internet).
[0476] A further embodiment comprises processing means (for example, a computer or a programmable logic device) configured or adapted to perform one of the methods described herein.
[0477] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
[0478] Another embodiment according to the invention comprises an apparatus or system configured to transmit (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, comprise a file server for transmitting the computer program to the receiver.
[0479] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, the field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.
[0480] The devices described herein may be implemented using a hardware device or using a computer or using a combination of a hardware device and a computer.
[0481] The devices described herein or any component of a device described herein may be implemented at least partially in hardware and / or in software.
[0482] The methods described herein may be performed using a hardware device or using a computer or using a combination of a hardware device and a computer.
[0483] Any component of the methods described herein or the apparatus described herein may be performed at least in part by hardware and / or by software.
[0484] Method and apparatus for improving spectral gap filling in a spectrotemporal manner in audio coding
[0485] 1. Introduction, General Solutions
[0486] The present invention relates to a way of improving, for example, the computation of spectral envelopes (e.g. as applied in modern audio transform codecs) in a perceptually improved manner, and / or a way of improving the reconstruction of spectral and / or temporal fine structure of spectral regions quantized to zero in an encoder. Additionally, or in other words, as an example, the present invention relates to a spectral envelope representing a time-varying and / or frequency-varying masking threshold used, for example, during spectral quantization in conventional audio codecs, wherein, as an example, each spectrum may, for example, be divided by an associated masking threshold before quantization, and, for example, be multiplied by an associated masking threshold after quantization, as an example, thereby obtaining, for example, a spectral shaping of the quantization distortion according to the masking threshold. The computation of such a spectral envelope may, for example, conventionally involve applying a certain spectral tilt (also as an example or, for example, often referred to as "pre-emphasis") to the envelope data before quantization, for example in order to ensure, for example, during coding bit allocation, that the coding SNR at low frequencies is higher than the coding SNR at high frequencies, and thereby, as an example, ensure a higher audio quality. See for example 3GPP TS 26.445, Enhanced Voice Service (EVS) Detailed Algorithm Description or PCT / EP2018 / 080137 from 2018. In addition, the present invention relates to spectral replacement or "filling" of spectral gaps (zero quantized frequency coefficients after encoding) caused by coarse quantization at relatively low target bit rates.
[0487] It has been found that low frequency (LF) spectral content is typically encoded sufficiently accurately, e.g., by the above-described methods, e.g., because the LF SNR is, e.g., relatively high, e.g., due to the application of a spectral tilt on the actual spectral envelope during the calculation of the masking envelope. However, it has also been found that at low coding bit rates, e.g., a larger portion of the high frequency (HF) spectral range may, e.g., be likely to be quantized to zero, e.g., resulting in HF spectral gaps. There are methods that may, e.g., fill these zero quantized portions with random spectral values (e.g., such as perceptual noise substitution (PNS) in MPEG-4 Advanced Audio Coding (AAC) and / or frequency domain noise filling in MPEG-D extension HE-AAC and 3GPP EVS), but these methods may exhibit certain drawbacks, e.g., in terms of flexibility and / or signaling, e.g., as described below.
[0488] 2. Deficiencies of Conventional Solutions
[0489] The PNS method can signal the target energy of a spectral band to a decoder, which has been quantized to zero, e.g., in an encoder, and the PNS decoder can insert a pseudo-random value into the zero-quantized band, e.g., scaled such that the inserted signal energy matches the signaled target energy. Although this scheme can, e.g., quite accurately (e.g., at low bitrates) preserve spectral energy (and thus, as an example, preserve the original spectral envelope), it may tend to require many bits, e.g., for the signaling of the zero-quantized band energy, which can be counterproductive. Additionally, it can be, e.g., relatively inflexible, since only fully zero-quantized spectral bands can, e.g., be replaced - replacement may not be performed in a band where at least one spectral coefficient is non-zero (e.g., after quantization).
[0490] The noise filling method used in MPEG-D extended HE-AAC and 3GPP EVS can, e.g., improve the PNS design by allowing, e.g., the replacement of zero-quantized spectral coefficients with pseudo-random values, e.g., above a certain "noise filling start frequency" after decoding (e.g., even when a spectral band is not fully quantized to zero in the encoder). However, the MPEG-D codec can still signal per-band target energy data for all fully zero-quantized bands, thus, as an example, increasing the signaling overhead, e.g., especially when many bands are zero-quantized. The noise filling method in 3GPP EVS can avoid the transmission of such per-band energies and, instead, can utilize, e.g., only the globally transmitted spectral noise level / and / or a predefined spectral tilt t.
[0491] By Figure 2 and Figure 3 Looking more closely at the EVS noise filling method reveals that in EVS, the spectral envelope reconstructed in the zero-quantized spectral region may not be directly given by the spectral envelope of the original signal (i.e., the signal envelope; thick black solid curve), but rather, e.g., by a scaled version of the masking threshold (i.e., the inverse of the normalized envelope used before quantization, e.g., for spectral shaping of quantization distortion, e.g., as described in the introduction; thick black dashed curve). This is not unexpected: the decoding result in the zero-quantized spectral region can be the product of the inserted pseudo-random value, the globally transmitted spectral noise level 0 < L < 1, and the transmitted masking envelope - no representation of the true spectral envelope can be transmitted from the encoder to the decoder.
[0492] Figure 2Spectral envelopes targeted by an EVS gap filling (or, e.g., noise filling) algorithm, e.g., in the absence of any spectral tilt compensation, are compared, or the comparison may be displayed, for example. It should be noted that the masking threshold dashed curve has been shifted downward for better visibility of all curves. It can be seen that, e.g., due to the frequency-invariant level l, the distance between the masking envelope and the effective reconstructed noise envelope (grey solid curve) may be constant (thin double arrows), and, e.g., therefore, may not or not exactly follow the original spectral envelope. In fact, the reconstructed noise envelope may exceed the original spectral envelope at high frequencies, e.g., thus, may cause audible noise after decoding, while it may, e.g., remain significantly lower than the original spectral envelope at lower frequencies, e.g., thus, may cause insufficient gap filling energy and / or audible spectral holes. The spectral tilt applied, e.g., during the calculation of the masking envelope (i.e., the noise shaping envelope, e.g., as explained), may, e.g., therefore, be used in the spectral envelope reconstructed in the zero-quantized spectral region.
[0493] Figure 3 An example of, for example, desired spectral gap filling behavior is illustrated, or an illustration of the example may be shown, for example. Here, the distance between the noise shaping envelope and the effective reconstructed envelope in the zero-quantized spectral region (grey curve) is not constant, but rather slopes downwards towards higher frequencies (the length of the thin double arrow decreases with frequency). The slope, which may be applied multiplicatively, for example in a frequency-dependent manner / in a frequency-dependent manner, may attempt to compensate for a pre-emphasis slope applied, for example, during the calculation of the masking envelope, for example in order to recover or at least approximate the true spectral envelope of the input signal (e.g. during gap filling). In EVS, this slope t (see above) may be a predefined constant, but it may be observed, for example, that the optimal value of t for each frame or transform may, for example, vary considerably, for example due to quantization of the masking envelope (e.g., by low-rate vector quantization in EVS and derived codecs) and / or some input signal dependency. For example, a signal-adaptive, frame-by-frame or transform-by-transform signaled t will or may, for example, therefore be, for example, more desirable than a constant value of, for example, t.
[0494] Furthermore, it would or may be desirable to allow, for example, adjustment of the fine spectral and / or temporal envelope of the substituted pseudo-random gap filling values during decoding, for example, to better match the input signal envelope. 3. Summary of the invention
[0496] By way of example, to address the aforementioned disadvantages of the prior art, e.g., relatively large gap filling energies signaling overhead and / or insufficiently accurate spectral envelope recovery during audio decoding, e.g. via gap filling, e.g. combined with insufficiently accurate reconstruction of a fine spectro-temporal envelope, e.g. in the spectral regions generated via gap filling, the following method for improving gap filling is proposed, e.g., according to an embodiment of the present invention:
[0497] 1. Transmitting a frame-by-frame and / or sub-frame-by-sub-frame spectral tilt correction t, e.g. in an audio transform codec, e.g. applying spectral gap filling, without explicitly transmitting a target energy in a zero quantization band (where, e.g., "band" indicates a specific non-overlapping frequency range). This aspect of the embodiment or examples of e.g. this aspect are described in section 4.1. The benefit may be, e.g., improved recovery of the true spectral envelope, e.g. from a transmitted masking envelope (i.e., a noise shaping envelope), and / or improved recovery of the true spectral envelope, e.g., independent of the quantization of the transmitted masking envelope (i.e., a noise shaping envelope), e.g. (e.g., with very little additional signaling).
[0498] 2. Signal adaptive selection, e.g. on a per-frame and / or per-subframe basis, e.g. between different methods, e.g. for generating "artificial" spectral content for use, e.g. during gap filling, wherein the selection is signaled to the audio transform decoder as an example, e.g. via a frequency domain long-term prediction (FD-LTP) lag parameter. This aspect or examples of this aspect, e.g. according to an embodiment, are described in section 4.2. The general idea according to an embodiment is to select, e.g. depending on the FD-LTP lag value, between: a) noise filling with FD-LTP; b) tonality-based gap filling, e.g. without FD-LTP, e.g. similar to IGF in EVS (prior art method); and c) e.g. conventional noise filling, e.g. without FD-LTP, e.g. similar to noise filling in EVS or MPEG-D.
[0499] 3. When, e.g. in aspect 2 above, noise filling with FD-LTP is selected (where, as an example, the FD-LTP lag is non-zero), applying a long-term predictive filter, e.g. in the spectral domain (e.g. MDCT domain) of, e.g., an audio transform codec, may be performed, e.g. during a decoder-side noise filling routine (e.g. depending on whether the "current" encoded FD coefficient is zero, and e.g. depending on whether the corresponding "previous" encoded FD coefficient located at a certain distance from the current coefficient (specified by the transmitted FD-LTP lag) is zero). This aspect or examples of this aspect according to an embodiment are described in section 4.3.
[0500] 4. When, e.g. in aspect 2 above, gap filling without FD-LTP is selected (where e.g. the FD-LTP lag is zero), application of a signal adaptive (e.g. copying and / or tonality-based) spectral gap filling procedure may be performed, e.g. similar to the Intelligent Gap Filling (IGF) method used in e.g. 3GPP EVS and MPEG-H Audio. Copying may indicate reconstruction of zero quantized FD coefficients from lower frequency non-zero quantized FD coefficients, and tonality-based may e.g. mean that the copying procedure may be guided by transmitted (sub)frame-by-frame (e.g. frame-by-frame and / or sub-frame-by-sub-frame) "audio tonality data", e.g. known from conventional solutions (here, time-domain LTP or HPF lag). This (e.g. final) aspect or examples of this aspect according to an embodiment are described in section 4.4.
[0501] As an example, these aspects are designed for the 3GPP IVAS codec, but may be equally applicable (or, eg, in a similar manner) to other codecs, such as EVS.
[0502] 4. Description of Embodiments of the Invention
[0503] For example, one or even a preferred implementation (e.g., embodiment) of aspects of the present invention may, for example, require the following, for example, for use in an audio codec. As an example, embodiments according to the present invention may include components in order to satisfy the following prerequisites. Therefore, embodiments according to the present invention may include the following features:
[0504] Prerequisite a: N-bit noise level index (e.g. 0≤l) is transmitted in, for example, each frame and / or subframe sf. sf <2 N ), as an example, to derive the noise level, such as L sf < 1. For example, in EVS, IVAS and / or other codecs, N=3 may be selected, for example, in general.
[0505] Prerequisite b: A noise shaping envelope (ie, as an example, a masking envelope) is transmitted, which is derived as an example in the encoder, for example by obtaining the spectral band-by-spectral energy or the RMS value of the input spectrum.
[0506] Prerequisite c: Some frame and / or subframe audio tonality information is transmitted to the decoder, e.g. via time domain long term prediction (TD-LTP) and / or harmonic post filtering (e.g. HPF) lag and gain. If this information is present, the (sub)frame can e.g. be considered as tonality.
[0507] It should also be noted that in the following description, the terms frame and subframe are used interchangeably.
[0508] 4.1. Preferred embodiment: For example according to aspect 1 (as an example, adaptive tilt correction)
[0509] The general idea behind the transmission tilt correction value according to an embodiment may for example be to calculate and / or signal at a low bit rate (e.g. in the logarithmic intensity domain) the (e.g. true) spectral envelope of a subframe (i.e. as an example its input signal envelope, Figure 2 and Figure 3 The black solid curve in ) and the masking envelope of the subframe (as an example, that is, the noise shaping envelope, Figure 2 and Figure 3 Since the masked envelope may be transmitted to the decoder (e.g. according to prerequisite b), the additional transmission of the difference may, for example, allow (e.g. in a gap and / or noise filling decoding procedure) to reconstruct the (e.g. true) spectral envelope, e.g. from the masked envelope and / or the tilt-related difference curve, e.g. with better accuracy and / or with fewer side information bits than conventional solutions.
[0510] 4.1.1. Slant line calculation and encoding in encoder (optional)
[0511] Figure 3 It is shown that the intensity difference between the (e.g. true) spectral envelope and the masking envelope may vary monotonically (e.g. with frequency). In the logarithmic intensity domain (e.g. logarithm with base 10) and / or in gap and / or noise filled spectral regions (e.g. between two thin vertical arrows), it is found that the monotonic difference curve, for example, resembles a straight line most of the time. Therefore, for example, according to an embodiment it is proposed to calculate and parameterize the difference curve, for example by, for example, simple linear regression, for example in the logarithmic intensity domain (see, for example, https: / / en.m.wikipedia.org / wiki / Simple_linear_regression for a description), for example via or using the following function
[0512] Difference curve (f) = true spectrum envelope (f) - masking envelope (f) = T·f + O,
[0513] Where f is the desired frequency (or, e.g., equivalently, the offset of the transform coefficients), T is the tilt (or slope) value, O is the intensity offset, and both envelopes are in the logarithmic domain. For example, in order to minimize the computational complexity of T and O, both envelopes may, e.g., preferably be represented by per-spectral band energies (e.g., sums of squares) and / or root mean square (RMS) values (e.g., rather than per-transform coefficient values) (e.g., according to prerequisite b, as an example, i.e., the number of values of f may, e.g., be less than the number of transform coefficients). The encoder-side calculation and / or encoding / decoding of T and O may, e.g., then proceed as follows:
[0514] For each (sub)frame sf, do:
[0515] 1. Calculate the per-spectral band energy and / or RMS value E, e.g., from the input (i.e., uncoded) spectrum for sf sf (f)
[0516] 2. For example, all E sf (f) Transform to the logarithmic domain; from all E sf (f) minus its overall mean => (eg, providing) a zero mean value E' sf
[0517] 3. From E' sf Calculate, quantize and / or dequantize the mask envelope M sf , for example, as described in the prior art
[0518] 4. From M sf Reconstruct spectral band-by-spectral band energy and / or RMS value => (eg provide) logarithmic and zero mean value M' sf (f)
[0519] 5. For example, for all pairs of spectral band values E' sf With M' sf For example, a simple linear regression is performed between => (for example, providing) T sf and O sf
[0520] 6. From T sf Quantize (e.g. to 3 bits) and dequantize the tilt index t sf (For example, quantized T sf Values: [1,2,...7,8] / -5)
[0521] 7. From t sf Reconstructed tilt value => (eg, provide) decoded tilt T' sf ; In calculating the noise level index l sf Use -T' during sf ·f.
[0522] It should be noted that with proper choice of certain constants, for example during step 7, O sf The value of can be, for example, sf is taken into account in the calculation of (as an example, i.e., it can be calculated by l sf self-compensation). Thus, for example, it may not be necessary to quantize and signal 0 sf to the decoder, thereby, for example, causing the method rate to be very low (for example, only 3 bits can or must be transmitted). sf ).
[0523] It should be further noted that T' sf It can be, for example, still in the logarithmic domain, as an example, that is, T' sff can be an additive product, for example in the logarithmic domain. Therefore, in deriving l sf Eliminating this logarithmic domain product would mean that the linear domain product (e.g., 10 T ' sf·f ), such as in the case of calculations performed in the linear domain.
[0524] 4.1.2. Slanted reconstruction and application in decoder (optional)
[0525] Gap and / or noise filling in the decoder (e.g. using l sf ), the encoder side step 7 may be applied, for example, in reverse, for example as follows:
[0526] For each (sub)frame sf, do:
[0527] 1. For example, according to the prior art, from the transmitted l sf Reconstruct the final noise level L sf ; See also section 4.5 as an example
[0528] 2. From t sf Reconstructed tilt value => (eg, provide) decoded tilt T' sf ; For example, at the (eg final) noise level L sf Use T' during the multiplication sf ·f.
[0529] As an example, in other words, for example, when L sf When multiplied to the resulting gap filling factor, it may be possible, for example, to also multiply by the tilt correction product T' in a frequency offset (f) dependent manner. sf · Linear domain equivalents of f (e.g., 10 T ' sf·f , as described above). In order to maintain (e.g. on average) L sf The value range of T' can be scaled by a constant, for example sf .
[0530] In case of a band-wise segmentation of the transform spectrum (e.g. as described above), the frequency offset f for each spectral band may for example represent, for example: a) the start frequency of the band (or, e.g., equivalently, the offset of the first transform coefficient associated with the band), or b) the center frequency of the band (or, e.g., equivalently, the offset of the first transform coefficient in the band plus half the width of the band, e.g. in the number of transform coefficients). It was found that both options lead to almost the same accuracy of the method.
[0531] 4.2. Preferred embodiment: For example according to aspect 2 (as an example, adaptive gap filling selection)
[0532] Prior art, e.g. as described in section 2, provides e.g. at least two different approaches to reconstruct zero quantized spectral regions in audio transform coding: simple noise filling (or PNS), e.g. using pseudo-randomly generated transform coefficient values, and smarter gap filling (or e.g. spectral band replication, SBR), e.g. applying copies (copy-up or copy-over) from non-zero quantized spectral coefficients. The general idea behind according to embodiments and e.g. this aspect of embodiments according to the invention or examples of this aspect is to provide means for switching between noise filling and gap filling solutions (e.g. based on signal characteristics of (sub)frames), the former with optional improved fine temporal shaping, e.g. as follows.
[0533] 4.2.1. “Long-term transient” detection and FD-LTP encoding (optional)
[0534] It has been found experimentally that, for example, in particular, applause-like, rain-like and / or LF male speech signals may benefit, for example, from an improved reconstruction of the fine temporal signal envelope of the HF, for example during gap and / or noise filling at the decoder side. For these signals detected and / or classified (e.g. as "long-term transients"), for example, the fine temporal structure of a particular (sub)frame sf may be parameterized, for example, by frequency-domain long-term prediction (FD-LTP) information. Analogous to conventional LTP pitch and gain information obtained in the time domain (TD), the FD-LTP lag and / or gain values may, for example, be obtained directly in the transform domain of the audio codec; a detailed description follows section 4.3. For example, the choice of noise and / or gap filling to be applied in the decoder may, for example, be made and / or signaled to the decoder depending on the value of said FD-LTP lag p, for example transmitted in the audio bitstream, for example as follows:
[0535] For each (sub)frame sf, do:
[0536] 1. Perform a "long-term transient" test; if sf is not a long-term transient, set p sf = 0 and terminate; otherwise:
[0537] 2. Calculation of FD-LTP lag P sf and / or gain G sf ; If |G sf |<β, where 0<β<1 is the threshold, then set p sf = 0 and stop; otherwise:
[0538] 3. For example, from the lag P sf Get the quantized (e.g. to 3 or 4 bits) FD-LTP lag index p sf , for example as described in section 4.3 below
[0539] 4. If psf is 0, then a quantized (e.g. to 2 bits) HF energy value is calculated, for example, for adjusting the HF gap filling region; otherwise:
[0540] 5. From G sf Get the quantized (e.g. to 2 bits) FD-LTP gain g sf and / or symbols sf , for example as described below in Section 4.3.
[0541] Examples according to embodiments for calculating HF energy values (e.g. for adjusting the HF gap filling range) are described in detail in section 4.4. It should be noted that "long term transients" detection may be performed, e.g. conventionally as in prior art audio encoders, e.g. by comparing (e.g. for each subframe) a calculated instantaneous (and e.g. possibly temporally smoothed) spectral and / or temporal flatness measure with a predefined threshold and e.g. classifying the SF as "long term transient" (e.g. in case the temporal flatness is below and the spectral flatness is above the threshold).
[0542] 4.2.2. Filling type selection and parameter reading in decoder (optional)
[0543] With the quantized FD-LTP lag value p sf (For example, with FD-LTP g sf and its symbol s sf In case the audio tonality information (TD-LTP or HPF data, prerequisite c) is transmitted in the bitstream together with the audio tonality information, the decoder can choose which type of spectrum filling to apply (e.g. gap filling or noise filling with or without FD-LTP filtering), for example as follows:
[0544] For each (sub)frame sf, do:
[0545] 1. If the quantized lag p sf >0, then read the quantized gain g sf and symbol s sf And select type: noise filling + FD-LTP; otherwise:
[0546] 2. Read the HF energy value and check the existence of audio tonality data; if sf is tonal, select type: gap filling; otherwise:
[0547] 3. Select type: Traditional noise filling without FD-LTP, as described in the prior art (eg EVS, MPEG-D).
[0548] Examples according to embodiments for operating FD-LTP augmented noise filling and tonality-based gap filling are described below. It should be noted that the classification of sf as "tonal" in step 2 may be based, for example, on prior art audio tonality data, for example by classifying sf as "tonal" if audio tonality data is present (i.e., TD-LTP / HPF data is non-zero, as an example). Alternatively, sf may be classified as "tonal" for example only if a TD-LTP / HPF gain value is transmitted and is at a maximum.
[0549] 4.3. Preferred embodiments: For example according to aspect 3 (noise filling with FD-LTP as an example)
[0550] As mentioned or explained in section 4.2.1, for example, the temporal fine structure of the encoded audio signal can be more accurately reconstructed, for example, by FD-LTP filtering, for example, during a noise filling procedure on the decoder side. Therefore, when, in the (sub)frame-by-frame procedure in section 4.2.2, the type noise filling + FD-LTP has been selected (or, as an example, can be selected according to an embodiment), as an example, an infinite impulse response (IIR) LTP-type filter can be applied according to this aspect, for example, to the pseudo-random noise coefficients generated during noise filling on the decoder side, thereby generating (as an example) a fine temporal shaped noise filled signal.
[0551] 4.3.1. FD-LTP calculation in encoder (optional)
[0552] The decision whether to apply noise filling with FD-LTP filtering may be based on FD predictor parameters that may be determined, for example, in the encoder. These predictor parameters (lag index p sf , gain index g sf and / or symbol indexes sf ) may e.g. preferably be computed in a spectro-temporally normalized domain e.g. utilized before quantization of transform coefficients, as an example, i.e. with respect to (if applicable) a TNS analysis filtered transform vector, which e.g. has been perceptually normalized e.g. by a noise shaping envelope (i.e. divided by the noise shaping envelope, as an example). It should be noted that the TNS analysis filtering may e.g. effectively remove the coarse temporal envelope of the subframe, whereas the perceptual normalization may e.g. remove the coarse spectral envelope, e.g. leaving only the fine temporal envelope (e.g. to be parameterized by FD-LTP) and / or the fine spectral envelope (which may be expected to be negligible for said "long term transient" signal). For example, the FD-LTP parameter calculation may e.g. be applied similarly to the conventional TD-LTP and / or HPF calculation:
[0553] As an example, for each (sub)frame sf classified as "long term transient", do:
[0554] 1. On the spectrum normalized in a spectro-temporal manner at lag B <p'<B+2 B Normalized autocorrelation is calculated at
[0555] 2. Find the p' that maximizes the normalized autocorrelation value. => (eg, provide) a value G with autocorr. (eg, autocorrelation) sf P sf
[0556] 3. If |G sf |<β, then set p sf = 0 and stop (see, for example, also Section 4.2.1; 0<β<1 can be a threshold, such as 1 / 4); otherwise, set p sf =P sf -B
[0557] 4. If |G sf |<1 / 2, then set g sf =0; otherwise, set g sf = 1. Finally, if G sf ≥0, then set s sf =0; otherwise, set s sf =1 (see, for example, also section 4.2.1).
[0558] It should be noted that, for example, to reduce computational complexity, only integer spectral lags may be calculated, for example, for FD-LTP. Furthermore, all calculations may be applied only in the HF gap filling region. As an example, the constant B is described in section 4.5.
[0559] 4.3.2. FD-LTP application in decoder (optional)
[0560] After, for example, all steps in section 4.2.2 have been performed and the type noise filling+FD-LTP has been selected, the three FD-LTP parameters may, for example, be decoded and conventional noise filling and (for example subsequently) FD-LTP filtering may, for example, be applied:
[0561] As an example, for each (sub)frame sf of the selected type noise filling+FD-LTP, do:
[0562] 1. Set P' sf =p sf +B,G' sf =(-1) Ssf ·(3+2·g sf ) / 8, and d=(7.5-g sf ) / 8. It should be noted that here, it can be guaranteed that p sf >0
[0563] 2. Perform regular noise filling (i.e. use lsf ); Marking, for example, all noise-filled zero-quantized spectral coefficients
[0564] 3. For (e.g. sorted by increasing i) at position i ≥ P' sf Each noise-filled zero-quantized spectral coefficient c at is performed:
[0565] 4. If the mark in step 2 is at position iP' sf The coefficient c at the point is d·c(i)+G' sf ·c(iP' sf ) replaces c(i); otherwise:
[0566] 5. Use d·c(i)+1 / 2·G' sf ·c(iP' sf ) replaces c(i).
[0567] Fig.23 1 illustrates the time domain effects of FD-LTP filtering of a pseudo-random noise spectrum that has been subjected to an inverse transform (i.e., a frequency-to-time transform, e.g., using an inverse MDCT) (or an illustration that can show an example of this). It shows that, for example, the dependence of p sf and / or sf The choice of, for example, the number and position of the shaped peaks can be varied.
[0568] Note 1:
[0569] It should be noted that, for example, decoding steps 4 and 5 may effectively limit the contribution of low-frequency non-zero quantized spectral coefficients, e.g. to a given replaced zero-quantized spectral coefficient, e.g. during FD-LTP filtering. For example, the same approach may be applied during FD-LPC filtering (e.g. temporal noise shaping (TNS) synthesis filtering), e.g. to reduce the likelihood of audible clicks, e.g. in low bitrate audio coding. In particular, when filtering a given zero-quantized spectral coefficient (e.g. as part of a vector) using, e.g. a TNS synthesis filter, the contribution of the non-zero quantized (and, e.g., possibly previously TNS synthesis filtered) lower frequency spectral coefficients included in the filtering operation (as an example, i.e. scaled by non-zero filter weights) may be limited, e.g. by attenuating their (e.g. filter output) values (e.g. as attenuated by 1 / 2 as in step 5, e.g. when those equal values are used during the filtering operation).
[0570] 4.4. Preferred embodiment: For example according to aspect 4 (as an example, gap filling based on tonality)
[0571] Aspect 3 or embodiments according to aspect 3 address the need for more accurate fine temporal noise shaping. The need for more accurate fine spectral noise shaping, e.g., in particular for highly pitched and / or harmonic audio signals (e.g., such as speech or isolated musical instruments, like acoustic or electric guitar, harpsichord, trumpet) is addressed by the following tonality-based spectral gap filling method according to further embodiments of the present invention, which method may, for example, be similar to the IGF scheme in e.g. 3GPP EVS. Three main differences between the following proposal and the IGF technique may be: a) the dependence on audio tonality parameters (e.g., in particular, TD-LTP or HPF parameters), and / or b) the tonality-based gap filling is applied at lower frequencies (as an example, i.e., in the HF spectral region that is usually targeted by noise filling), and / or c) the gap filling may be performed, e.g., via l sf and / or tilted lines enable LF spectrum shaping using only one HF energy value (or eg, difference).
[0572] 4.4.1. Tonality-based gap filling in encoders (example)
[0573] As an example, when step 4 is performed in the (sub)frame-by-frame procedure in section 4.2.1 (i.e., selecting tonality-based gap filling and disabling FD-LTP, as an example), the harmonic-continuous gap replacement may be performed, for example, according to et al. (Integral Band-wise Parametric Coder, 2021) is applied, for example with the notable exception that this method is used exclusively on the spectro-temporally normalized spectrum in the relevant HF gap filling region. This region may, for example, be the spectral range between a typical noise filling start frequency (e.g., 2kHz) and a noise filling end frequency (e.g., 10kHz), where the latter may, for example, be equal to the conventional IGF start frequency in the case of ultra-wideband and / or full-band coding. It should be noted that, as an example, conventional IGF processing for audio bandwidth extension (ABE) may still be applied above 10kHz, i.e., other IGF-related whitening / flattening / energy data may be calculated for the IGF ABE region.
[0574] The HF energy value (or for example the difference (e.g. in the case of differential entropy coding)) may represent the raw RMS energy of the spectral coefficients normalized in a spectro-temporal manner, e.g. slightly below the noise filling end frequency, which may have been quantized to zero (e.g. in the frequency range of 8 kHz to 10 kHz). For example, the energy value may preferably be quantized, like a scaling factor in AAC, as an example, logarithmically in steps of 1.51 dB. In addition to the fine spectral envelope, the tonality-based gap filling may for example thus also accurately reconstruct the coarse HF noise spectral envelope. It should be noted that, for example, in order to minimize the signaling overhead required for example to communicate the HF energy value to a decoder, the energy value may for example be transmitted as a difference relative to the product of the global gain and noise level of the core encoder, as an example, i.e. as a "noise gain normalized" value. Preferably, this may for example be achieved by transmitting, for example, a rounded scaling result of the logarithm of the ratio between the HF energy value and the product of the global gain and noise level, e.g. according to the following formula:
[0575] For example, ehf sf =1+round(Δ·log2(EHF sf / (GG sf ·L sf )),
[0576] where ehf sf is the quantified HF energy value (or difference), EHF sf is the above HF original RMS energy, GG sf is the global gain, L sf is the (e.g. previous) noise level, and Δ is a constant scalar (e.g. Δ=2). To ensure that ehf sf constant bit consumption, it may be necessary to restrict it to a range of values (e.g., for 2 bits, 0 ≤ ehf sf <4).
[0577] 4.4.2. Tonality-based gap filling in the decoder (optional)
[0578] As an example, for a gap fill without FD-LTP being signaled (ie, where p sf =0), the above “noise gain normalized” HF energy difference (as an example, ie, for example, the ratio EHF sf / (GG sf ·L sf )) can be reconstructed in the decoder, for example, according to the following formula:
[0579] For example, nrgFac sf =EHF sf / (GG sf ·L sf)=2 (ehfsf-1) / Δ ,
[0580] where ehf sf and L sf are the transmitted quantized HF energy difference and the decoded noise level, respectively, and GG sf Denotes the reconstructed global gain value as used for gain normalization in the encoder. Note that the value Δ may be chosen as in the encoder, and "-1" (and "1+" in section 4.4.1) may be omitted.
[0581] Since the product GG sf ·L sf The replaced zero-quantized spectral coefficients may have been multiplied onto them, e.g. by a prior art decoder, and thus all the resulting gap-filling spectral coefficients replaced by zero-quantized coded coefficients in the spectral region (e.g. slightly below the noise filling end frequency (e.g. in the 8 kHz to 10 kHz frequency range, e.g. as described above) may be simply multiplied by nrgFac sf The desired HF spectral energy restoration of the present invention is achieved (e.g. before applying an inverse (i.e., frequency to time) transform such as, for example, IMDCT to the reconstructed spectral coefficient vector). In this way, the original RMS energy of the gapped / noise-filled spectral values (e.g., gapped and / or noise-filled spectral values) slightly below the noise filling end frequency can, for example, be closely reconstructed.
[0582] As an example, the remainder of the decoder-side gap filling operation (eg, or ie, application of tonality-based filling or conventional noise filling) may, for example, depend on the presence of the previously mentioned audio tonality data. As an example:
[0583] If sf is tonal (ie, TD-LTP / HPF data is non-zero, as an example), then replication and / or tonality-based gap filling is applied, e.g. et al. described in European Patent EP21185666 (Integral Band-wise Parametric Coder, 2021).
[0584] • Otherwise (ie, as an example, sf is not tonal, TD-LTP / HPF data is zero or does not exist), apply conventional noise filling.
[0585] 4.5. Preferred Embodiment: Parameter Signaling
[0586] Side information that may be needed, for example, to signal subframe-by-subframe l, t and FD-LTP lag and gain or HF energy delta data may preferably have a fixed bit length. This may simplify the bit allocation before spectrum quantization in the encoder. The following example according to an embodiment of the transmission design (consumed fixed 12 bits per subframe) works well in practice:
[0587] Signaling syntax:
[0588] ---------------------------------------------------------------------
[0589] If a given frame has more than one subframe (e.g., 2 subframes), then:
[0590] -Set B=3
[0591] - Transmit other control data with {number of subframes} bits (see below)
[0592] otherwise:
[0593] - Set B = 4
[0594] For each subframe in a given frame 0≤sf<{number of subframes}, perform:
[0595] - For example, according to the prior art, the noise level index l is transmitted with 3 bits, for example sf
[0596] - For example, according to the present invention, the spectrum tilt index t is transmitted, for example, with 3 bits sf
[0597] According to the present invention, the FD-LTP lag index p is transmitted with B bits sf
[0598] If the transmitted FD-LTP lag is non-zero (i.e., FD-LTP is used), then:
[0599] According to the present invention, for example, the FD-LTP gain g is transmitted with 1 bit sf
[0600] - According to the present invention, for example, the FD-LTP symbol s is transmitted with 1 bit sf
[0601] otherwise:
[0602] According to the invention, the HF energy value is transmitted, for example, using 2 bits.
[0603] --------------------------------------------------------------------
[0604] It should be noted that when a frame is divided into 2 subframes, e.g., two TCX-10 transforms instead of one TCX-20 transform in EVS or IVAS, the FD-LTP lag value may be transmitted using 3 bits per subframe instead of 4 bits, e.g. due to the reduced transform length, which may, e.g., effectively save two bits for the affected frame. However, two bits may, e.g., be used for (and e.g., are preferably used for) other bit allocation control data, e.g., a 2-bit index, e.g., which defines how the bit budget available for encoding spectral coefficients and gap filling data is distributed among the two subframes. In this case, the sum of the signaling overhead of the present invention and, e.g., (if applicable) 2-bit subframe bit distribution information may remain at a constant 12 bits per subframe, e.g., independent of the choice of the number of subframes. This may simplify the encoder-side bit allocation and / or quantization steps.
[0605] For example, in summary, we discuss and / or clarify l sf and spectral time flattening material (e.g., according to an embodiment of the invention), which may be incorporated from conventional solutions:
[0606] - Subframe-by-subframe 3-bit noise level index l sf It can be signaled, for example, as in EVS or PCT / EP2018 / 080137, as an example, that is, the final noise level L sf It can be reconstructed, for example, as follows: sf = l sf 3 / 32, or as an alternative example, L sf =(l sf +1 / 2)·3 / 32.
[0607] - The spectral whitening flag may be used, for example, to distinguish between moderate spectral flattening and strong spectral flattening (e.g., if TD-LTP or HPF data is not available) and / or no spectral flattening and moderate spectral flattening of the copied spectral content (e.g., if such data is available). For a description of such spectral whitening content and how to obtain it, see, for example, the EVS or ISO / IEC 23008-3 (MPEG-H) audio standards, in particular the IGF decoding description.
[0608] - The temporal flattening flag may for example be used to signal the activation of a TNS-like filtering that replicates the spectral content, for example in order to flatten its temporal envelope. Again, for a description of this technique see for example the EVS or ISO / IEC 23008-3 (MPEG-H) audio standards, in particular the IGF decoding algorithm.
[0609] Aspect 2 described in section 4.2 for example introduces, inter alia, a signal adaptive selection on a per-frame and / or per-subframe basis between different methods for generating "artificial" spectral gap filling content, wherein the selection is signaled, for example, to the audio transform decoder via a frequency domain long-term prediction (FD-LTP) parameter. In particular, this FD-LTP parameter preferably constitutes a transform domain "lag" parameter p optionally transmitted in the audio bitstream for the frame or subframe sf. sf It should be obvious to those skilled in the art that the “hysteresis” parameter p sf Alternatively, the choice of spectral gap filling method may for example instead depend on different FD-LTP parameters, ie or for example the FD-LTP "gain" parameter g sf .
[0610] As an example, more precisely, the parameterization of the (eg absolute) gain value may be chosen such that the quantized gain value g sf = 0 means that the activated FD-LTP is deactivated (for example, this is because the effective decoded gain G sf may or will be zero). In that case, the FD-LTP lag p sf and / or symbols sf Data need not be transmitted (instead, for example HF energy values may be transmitted), and the choice of whether to apply noise filling with FD-LTP post-processing (e.g. instead of conventional noise or gap filling without FD-LTP) may depend on for example gain rather than hysteresis parameters.
[0611] For example, the decoder side step 1 as described in Section 4.2.2 will then be or can be written, for example, as follows (note that p sf With g sf Exchange of):
[0612] 1. If the quantized gain g sf >0, then read the quantized hysteresis p sf and symbol s sf And select type: noise filling + FD-LTP;….
[0613] Similarly, on the encoder side, for example as described in Section 4.2.1, it will or may be necessary to use g sf Exchange for example all p sf , and vice versa. Another change would or could be, for example, to adjust the number of bits used for signaling the HF energy value, e.g. from the described 2 bits to 4 or 5 bits, e.g. in order to match the sum of bits used for signaling the (e.g. sub-)frame FD-LTP lag (3 or 4 bits) and / or sign (1 bit) parameters, e.g. on both the encoder side and the decoder side.
[0614] 5. References and further reading
[0615] patent
[0616] - M. Dietz, G. Fuchs, CR Helmrich, and G. Markovic, Low-Complexity Tonality-Adaptive Audio Signal Quantization, U.S. Patent PCT / EP2014 / 0516242014 (on quantization with tonality-based dead zones).
[0617] -S.Disch, M.Gayer, CRHelmrich, G.Markovic, and M.Luis Valero, NoiseFilling Concept, US Patent PCT / EP2014 / 0516302014 (on the filling and shaping of continuous zero quantization spectrum regions).
[0618] -E. Ravelli, CR Helmrich, G. Markovic, M. Neusinger, S. Disch, M. Jander, and M. Dietz, Apparatus and Method for Processing an Audio Signal Using a HarmonicPost-Filter, U.S. Patent PCT / EP2015 / 066998.
[0619] -E. Ravelli, M. Schnell, C. Benndorf, M. Lutzky, M. Dietz, and S. Korse, Apparatus and Method for Encoding and Decoding an Audio Signal Using Downsampling or Interpolation of Scale Parameters, U.S. Patent PCT / EP2018 / 0801372018 (regarding the IVAS SNS method, i.e., the calculation of the noise shaping envelope).
[0620] -A. Niedermeier, C. Ertel, R. Geiger, F. Ghido, and CR Helmrich, Apparatus and Method for Decoding or Encoding an Audio Signal Using Energy Information Values for a Reconstruction Band, U.S. Patent PCT / EP2014 / 0651102013 (on smart gap filling in EVS or MPEG-H Audio, band energy aspects).
[0621] -S.Disch, R.Geiger, C.Helmrich, F.Nagel, C.Neukam, K.Schmidt, andM.Fischer, Apparatus, Method and Computer Program for Decoding an Encoded AudioSignal, U.S. Patent PCT / EP2014 / 0651182013.
[0622] -S. Disch, F. Nagel, R. Geiger, BN Hoshkahna, K. Schmidt, S. Bayer, C. Neukam, B. Edler, and CR Helmrich, Apparatus and Method for Encoding or Decoding an Audio Signal with Intelligent Gap Filling in the Spectral Domain, U.S. Patent PCT / EP2014 / 0651232013 (another basic IGF application).
[0623] paper
[0624] -CRHelmrich, G.Markovic, and B.Edler, Improved Low-Delay MDCT-BasedCoding of Both Stationary and Transient Audio Signals, Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 6954-6958, 2014. Online: https: / / ieeexplore.ieee.org / document / 6854948 /
[0625] -CR Helmrich, A. Niedermeier, S. Disch, and F. Ghido, Spectral Envelope Reconstruction via IGF for Audio Transform Coding, Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 389-393, 2015. Online: https: / / ieeexplore.ieee.org / document / 7177997 /
[0626] -G.Fuchs, CRHelmrich, G.Markovic, M.Neusinger, E.Ravelli, and T.Moriya, Low Delay LPC and MDCT Based Audio Coding in the EVS Codec, Proc.IEEEInt.Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 5723-5727, 2015. Online: https: / / ieeexplore.ieee.org / document / 7179068 /
[0627] - CR Helmrich, A. Niedermeier, S. Bayer, and B. Edler, Low-Complexity Semi-Parametric Joint-Stereo Audio Transform Coding, Proc. EURASIP 23rd EUSIPCO, pp. 799-803, 2015. (Stereo Filling and IGF Stereo)
[0628] -K. Schmidt and C. Neukam, Low Complexity Tonality Control in the Intelligent Gap Filling Tool, Proc. IEEE ICASSP, pp. 644-648, 2016. Online: https: / / ieeexplore.ieee.org / document / 7471754 / (whitening).
[0629] Link
[0630] -Simple Linear Regression Method: https: / / en.m.wikipedia.org / wiki / Simple_linear_ regression
[0631] - Spectral flatness (measurement): https: / / en.m.wikipedia.org / wiki / Spectral_flatness
[0632] Appendix 1: Application of FD-LTP adaptive filtering to temporal noise shaping filtering
[0633] It should be noted that the aspects explained below may be used independently and may also be optionally used in combination with any of the features, functionalities, and details disclosed herein.
[0634] It is mentioned in Remark 1 in Section 4.3.2 that the proposed strength adaptive filtering operation defined by steps 4 and 5 in Section 4.3.2 can also be applied, for example, to temporal noise shaping (TNS) synthesis filtering. For example, the two steps can effectively attenuate the filter (i.e., its strength, as an example) in a sample-by-sample index (i) manner, for example, if and only if
[0635] The transmitted current spectral coefficient c(i) at index i has been quantized to zero, and
[0636] ● Located at index iP' sf The "previous" spectral coefficient c(iP' sf ) has not yet been quantized to zero,
[0637] where the attenuation is preferably 1 / 2 (see, for example, step 5), and represents, for example, the spectral distance from i (abbreviated as d in this paragraph) sf ) of the FD-LTP hysteresis P' sf >0. For example, this two-part condition can be generalized as follows, e.g. in order to make it applicable to a TNS-like filter, e.g., instead of lag and gain, characterized by a filter order and one or more filter weights or filter coefficients, e.g., where the number of these weights depends on the filter order (the number of filter coefficients may, e.g., be equal to the filter order). Set the distance d sf = equal to the filter order, the TNS or FD-LTP filter may be attenuated, for example, by multiplying each filter weight by 1 / 2 (eg, for each i), for example, if and only if
[0638] ● One or more signaled spectral coefficients c(id sf +1)...c(i) is quantized to zero, and
[0639] ● Located at index id sf The "previous" spectral coefficient c(id sf ) has not yet been quantized to zero.
[0640] In other words, when both of the above conditions are met, the spectral coefficients c(i) may be filtered with a TNS and / or FD-LTP filter (e.g., whose weights have been attenuated). Conversely, when at least one of the conditions is not met, the coefficients c(i) may be filtered, for example, with an unchanged TNS and / or FD-LTP filter. Fig.24 and Fig.25 , where the (sub)frame subscript sf has been omitted for clarity. It should be noted that the FD-LTP filter can be used, for example, by setting d sf =P' sf and will be in the index range 1...d sf All filter weights in -1 are designated equal to zero and represented by a TNS-like filter. Thus, the above generalized conditions for intensity adaptive filtering can be applied to both TNS and FD-LTP, for example.
[0641] It should also be noted that in the case of IIR filtering, for example as proposed in Section 4.3.2, the spectral coefficient vector (e.g., Fig.24 ) and the spectral coefficient vector input to the actual embedded filtering operation (e.g., Fig.25 The “spectrum to be filtered” in ) may be different, for example: the former may specify the spectrum before noise filling (e.g., for marking in step 2 in section 4.3.2), while the latter may specify the FD-LTP filtered spectrum after noise filling.
[0642] Appendix 2: Proposal of three [eg partially] non-dependent decoders according to embodiments of the invention
[0643] In the following, other embodiments of the present invention will be explained with respect to other proposals of decoders according to embodiments of the present invention. It should be noted that the embodiments according to the aspects explained below can be used independently and can also be optionally used in combination with any of the features, functionalities and details disclosed herein. In other words, the embodiments described here, for example according to aspects 1 to 4, can be optionally supplemented by any of the features, functionalities and details disclosed herein.
[0644] 1 Zoomed?
[0645] Aspect 1 (Adaptive Tilt Correction, Section 4.1) (Example 1)
[0646] 1. The audio transform decoder performs a replacement of zero-quantized spectral samples by noise samples, where the frame-by-frame or sub-frame spectral tilt correction values t are read from the bitstream. sf , from t sf The frequency dependence is derived from a sloped curve, such as a line function in the logarithmic domain, and the noise samples replacing the zero quantized samples are multiplied by 1 Tilt curve.
[0647] Aspect 2 (Adaptive Gap Filling Selection, Section 4.2) (Example 2)
[0648] 2. The audio transform decoder performs or is configured to perform replacing zero-quantized spectral samples by or with padded samples, wherein the frame-by-frame or sub-frame spectral (eg LTP) distance value p is read from the bitstream sf , if p sf If the indication is zero, the first spectrum replacement method is selected, such as noise filling or a certain gap filling; otherwise, another spectrum replacement method is selected, namely, noise filling+FD-LTP of aspect 3.
[0649] Aspect 3 (Noise Filling with FD-LTP, Section 4.3) (Example 3)
[0650] 3. The audio transform decoder is configured to perform or executes replacing zero-quantized spectrum samples by noise samples, wherein the noise samples replacing the zero-quantized samples Filtered (e.g., LTP, TNS) such that the filtering strength depends on the spectral distance d from i sf The quantized value c(id sf ).
[0651] In the case of FD-LTP, d sf =P' sf For generalization, (after noise filling) is used as a marker to distinguish it from c (before noise filling), because c (id sf ) may, for example, always be the value before noise filling.
[0652] Aspect 4 (Gap filling based on tonality, Part 4.4) (Example 4)
[0653] For example, the following embodiments may address adaptive (sub)frame-by-frame selection of three types of spectrum replacement and switching therebetween (where such functionality may optionally be used in any of the embodiments disclosed herein). The following embodiments may for example be inventive further developments or for example improvements of the embodiments explained in Part Aspect 2 (Adaptive Gap Filling Selection, Section 4.2), but may optionally be used together with any other embodiments disclosed herein or independently of the other embodiments.
[0654] 2.1. According to embodiment 2, the audio transform decoder is, for example, configured to perform or perform a replacement of zero-quantized samples, wherein frame-by-frame or sub-frame-by-sub-frame temporal (eg audio tonality) pitch information j is read from the bit stream sf , if p sfis equal to zero, then the first spectrum replacement method (e.g., noise filling) or the second spectrum replacement method (e.g., gap filling) is selected, otherwise, another spectrum replacement method (e.g., noise filling+FD-LTP, aspect 3) is selected, and the selection between the first spectrum replacement method and the second spectrum replacement method depends on the pitch information j sf .
[0655] j sf It is not explicitly mentioned in this document because it is known in some conventional solutions (e.g. in some of the conventional solutions mentioned in this document). In general, embodiments according to the invention relate to the selection and implementation of a method to be used for generating spectral values, with which the decoder replaces the "zero quantized spectral samples" before the frequency-to-time conversion, as an example, i.e., these generated spectral values are generated by a noise generator (e.g., noise filling = first spectral replacement method, optionally with subsequent FD-LTP filtering = other spectral replacement method), or they are generated by a "copy" conversion method (gap filling = second spectral replacement method).
[0656] As an extension of the IGF method used in EVS and MPEG-H, the details of the "second spectral replacement method" (i.e., for example, a harmonically accurate "tonality-based gap filling") are not essential for embodiments according to the present invention. Embodiments of the present invention optionally address the scaling of RF spectral values quantized (and, for example, replaced) to zero by 8kHz to 10kHz (e.g., by "RF energy (difference) values") in the case of the first or second spectral replacement method (e.g., because only the RF energy (difference) values are then transmitted). It should be noted that this concept (e.g., scaling of RF spectral values quantized to zero) can be used, for example, in combination with embodiments according to aspect 2, but can optionally be used in combination with any of the other embodiments of the present invention or even independently. It should be noted that the use of such RF energy (difference) values can also be effective when the legacy noise filling (first spectral replacement method) is selected; therefore, such scaling in the range of 8kHz to 10kHz can, for example, not be combined with the "copying" based filling method.
[0657] Furthermore, it should be noted that filtering according to embodiments of the present invention may, for example, include processing of one or more spectral values or sampling values of the same frame or the same subframe or the same frequency band or the same time interval and / or processing of one or more spectral values or sampling values of different frames or different subframes or different frequency bands or different intervals.
[0658] According to some embodiments, filtering may, for example, comprise linear filtering or nonlinear filtering, wherein the filtered value is obtained based on one or more input values (e.g. at least one sample value or spectral value). For example, filtering may provide filtered values based on multiple input values (e.g. sample values or spectral values). Filtering according to embodiments may, for example, comprise determination of interpolated (or extrapolated) spectral values or interpolated sample values. For example, filtering may be used in order to obtain spectral values or sample values with good robustness and / or determinism.
[0659] Furthermore, it should be noted that the prediction according to embodiments of the present invention may, for example, include processing of one or more spectral values or sampling values of the same frame or the same subframe or the same frequency band or the same time interval and / or processing of one or more spectral values or sampling values of different frames or different subframes or different frequency bands or different intervals.
[0660] According to some embodiments, the prediction may, for example, comprise determining one or more sample values or spectral values based on one or more "earlier" values (e.g., values associated with one or more times prior to the time at which the predicted value is to be obtained by the prediction, or values associated with one or more frequencies lower than the frequency of the predicted value to be obtained by the prediction). The prediction according to an embodiment may, for example, comprise an extrapolation of spectral values or sample values (e.g., in time or in the frequency direction). Thus, the prediction may, for example, comprise the processing of frequency values of a certain frequency band in order to obtain frequency values, such as spectral coefficients, in another (preferably higher) frequency band. This may also apply, for example, to sample values in the time domain.
[0661] Furthermore, it should be noted that, according to some embodiments, filtering and prediction may, for example, be used interchangeably, or in other words, may even be identical, for example in the context of a prediction filter. Thus, filtering may, for example, be performed in order to predict a value. In other words, prediction may, for example, be performed using filtering, but other prediction algorithms that do not use filtering may also be optionally used, for example. Likewise, for example, some filtering operations may perform predictions, while other filtering operations may, for example, use values (or samples) before (e.g., before in time) and after (e.g., after in time) the value to be obtained. Thus, for example, filtering and prediction may be considered similar or identical concepts in some cases, while, for example, there are filtering operations that do not perform predictions, and vice versa.
[0662] Furthermore, it should be noted that embodiments according to the present invention may be used, for example, in the context of EVS, Intelligent Gap Filling (IGF), IVAS, MDCT coding, MPEG-H 3D audio, noise filling. Embodiments may be used, for example, or may be part of the technical field of MDCT-based audio coding for 3GPP IVAS. Embodiments may be used, for example, in 3GPP IVAS, IIS proprietary low-rate speech and audio codecs.
[0663] In the following, embodiments according to the present invention are discussed using different words:
[0664] Embodiments according to the invention may, for example, relate to ways of perceptually improving the computation of spectral envelopes (e.g. as applied in modern audio transform codecs), and e.g. ways of improving the reconstruction of spectral and / or temporal fine structure of spectral regions quantized to zero in an encoder. In other words, embodiments may, for example, relate to spectral envelopes representing time-varying and / or frequency-varying masking thresholds used, for example, during spectral quantization in conventional audio codecs, wherein, for example, each spectrum may, for example, be divided by an associated masking threshold before quantization, and e.g. multiplied by an associated masking threshold after quantization, thereby optionally resulting in a spectral shaping of the quantization distortion according to the masking threshold. Furthermore, embodiments may, for example, relate to spectral replacement or "filling" of spectral gaps (e.g. zero quantized frequency coefficients after encoding), e.g. caused by coarse quantization (e.g. at a relatively low target bitrate).
[0665] Examples may include:
[0666] 1. Transmitting frame-by-frame and / or sub-frame-by-sub-frame spectral tilt corrections t in an audio transform codec, optionally applying spectral gap filling, e.g. without explicitly transmitting target energies in zero-quantized frequency bands.
[0667] 2. In the spectral domain, applying a long-term predictive filter, for example during a noise filling routine at the decoder side.
[0668] The above embodiments are merely illustrative of the principles of the present invention. It should be understood that modifications and variations to the configurations and details described herein will be apparent to those skilled in the art. Therefore, it is intended to be limited only by the scope of the appended patent claims, rather than by the specific details presented by the description and explanation of the embodiments provided herein.
Claims
1. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) for providing a decoded audio representation (104, 404, 802, 1002, 1304, 1404) based on an encoded audio representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602); wherein the audio decoder is configured to fill spectral holes of a decoded set of spectral values (142, 442, 812, 1012, 1312, 1422) using corresponding filling values (122, 124, 222, 424, 814, 922, 1014, 1314, 1432); wherein the audio decoder is configured to determine the filler value using prediction or filtering such that a given filler value associated with a given frequency is obtained dependent on another spectral value (914, 924, 1322, 1334, 1434, 1614) associated with a different frequency, Wherein the audio decoder is configured to adapt the filtering strength depending on the encoded or quantized spectral values (914, 924, 1322, 1334, 1434, 1614) associated with the different frequencies.
2. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to claim 1, wherein the filtering strength determines the influence of the other spectral value (914, 924, 1322, 1334, 1434, 1614) on the given filling value (122, 124, 222, 424, 814, 922, 1014, 1314, 1432).
3. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 2, wherein the audio decoder is configured to adapt the filtering strength depending on the spectral values associated with the different frequencies, the spectral values being determined by the encoded representation of the individual spectral values in the encoded audio information (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602).
4. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 3, wherein the audio decoder is configured to adapt the filtering strength depending on the spectral values associated with the different frequencies before applying noise filling.
5. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 4, wherein the audio decoder is configured to adapt the filtering strength depending on whether the spectral values associated with the different frequencies are quantized to zero.
6. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 4, wherein the audio decoder is configured to adapt the filtering strength depending on whether noise filling is applied to the spectral values associated with the different frequencies.
7. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 6, wherein the audio decoder is configured to selectively apply filtering in the frequency direction or prediction in the frequency direction for spectral values to which noise filling is applied.
8. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 7, wherein the audio decoder is configured to apply the prediction or the filtering so as to determine the given filling value (122, 124, 222, 424, 814, 922, 1014, 1314, 1432) based on a random or pseudo-random noise value (916, 1432).
9. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 8, wherein the audio decoder is configured to perform A weighted combination of the noise value associated with the given frequency and the noise value associated with another frequency or a weighted combination of a noise value associated with the given frequency and a fill value associated with another frequency, so as to obtain the given fill value; and The audio decoder is configured to adjust a weight (1452) given to the noise value associated with the further frequency or a weight given to the filling value associated with the further frequency depending on whether noise filling has been applied to spectral values associated with the further frequency.
10. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 9, wherein the audio decoder is configured to determine the spectral distance between the fill value (122, 124, 222, 424, 814, 922, 1014, 1314, 1432) associated with the given frequency and the other spectral value (914, 924, 1322, 1334, 1434, 1614) associated with the different frequency based on encoded information describing the spectral distance included in the encoded representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602) of the audio information.
11. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 10, wherein the audio decoder is configured to determine a weight (1452) to be applied to the noise value (916) associated with the given frequency based on gain information (1132, 1212) included in the encoded representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602) of the audio information.
12. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 11, wherein the audio decoder is configured to determine a weight (1452) to be applied to the noise value associated with the other frequency or to the filling value (1322, 1334, 1434, 1614) associated with the other frequency depending on gain information (1132, 1212) included in the encoded representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602) of the audio information.
13. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 12, wherein the audio decoder is configured to determine the weight (1452) applied to the noise value associated with the other frequency or to the filling value (1322, 1334, 1434, 1614) associated with the other frequency depending on symbol information included in the encoded representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602) of the audio information.
14. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 13, wherein the audio decoder is configured to determine the given padding value according to the following formulas: (122, 124, 222, 424, 814, 922, 1014, 1314, 1432) If the coefficient c(iP' sf ) is obtained by using noise filling, then according to and If the coefficient c(iP' sf ) is not obtained by using noise filling, then according to Where c(i) represents the spectral coefficient obtained using noise filling and having spectral index i; Where d is the attenuation coefficient, Where G' sf representation weights (1452), the weights (1452) being based on gain values (1132, 1212) included in the encoded audio representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602); and Where c(iP' sf ) represents a spectrum index iP' sf The spectral coefficients of Where P' sf The prediction parameters or filter parameters are based on prediction parameter information (1412) included in the encoded audio representation.
15. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 14, wherein the audio decoder is configured to sf =p sf +B obtains the prediction parameter or filtering parameter P' sf , where p sf is a lag index included in the encoded audio representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602), and wherein B is a constant; and / or The audio decoder is configured according to G' sf =(-1) Ssf *(3+2*g sf ) / 8 obtain the weight G' sf (1452), where S sf is a binary value included in the encoded representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602), and wherein g sf is a binary value comprised in said encoded representation; and / or The audio decoder is configured to sf ) / 8 to obtain the attenuation coefficient d, where g sf is a binary value included in the encoded representation.
16. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 15, wherein the audio decoder is configured to fill zero quantized spectral coefficients with signature noise, and Wherein the audio decoder is configured to selectively use a reduced filtering strength applied to unmarked spectral coefficients.
17. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 1 to 16, wherein the audio decoder is configured to perform the following processing for a plurality of subframes (sf):
1. Set P 'sf =p sf +B,G' sf =(-1) Ssf *(3+2*g sf ) / 8, and d=(7.5-g sf ) / 8; 2. Perform noise filling and mark the noise filling zero quantized spectrum coefficients 3. For position i>=P' sf Multiple noise filled zero quantized spectrum coefficients c at are performed:
4. If the position iP' is marked in step 2 sf The coefficient c at the position is d*c(i)+G' sf *c(iP' sf ) replaces c(i); otherwise 5. Use d*c(i)+1 / 2*G' sf *c(iP' sf ) replaces c(i).
18. An audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) for providing a decoded audio representation (104, 404, 802, 1002, 1304, 1404) based on an encoded audio representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602), wherein the audio decoder is configured to determine the processed spectral values using prediction or filtering such that a given processed spectral value associated with a given frequency is obtained in dependence on another spectral value (914, 924, 1322, 1334, 1434, 1614) associated with a different frequency, Wherein the audio decoder is configured to adapt the filtering strength depending on the encoded or quantized spectral values (914, 924, 1322, 1334, 1434, 1614) associated with the different frequencies.
19. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to claim 18, wherein the audio decoder is configured to adapt the filtering strength to reduce the contribution of non-zero quantized spectral coefficients included in the prediction or filtering.
20. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 18 to 19, Wherein the audio decoder is configured to selectively adapt the filter strength if a current spectral coefficient is zero and a previous spectral coefficient has not been encoded to zero or has not been quantized to zero.
21. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 18 to 20, Wherein the audio decoder is configured to selectively reduce the filter strength to a value between 0.25 and 0.75 in order to adapt the filter strength.
22. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 18 to 21, Wherein the audio decoder is configured to selectively reduce the filtering strength of filtering taking into account a plurality of previous spectral coefficients depending on values of a plurality of previous spectral coefficients if the current spectral coefficient is encoded or quantized to zero.
23. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to claim 22, The audio decoder is configured to selectively reduce the filtering strength if the current spectral coefficient is encoded or quantized or signaled to be zero and if all previous spectral coefficients considered in the filtering except one previous spectral coefficient considered in the filtering were encoded or quantized or signaled to be zero.
24. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to claim 22 or 23, The audio decoder is configured to depend on a spectrum index id sf A plurality of previous spectral coefficients from i to i-1 are filtered or predicted to obtain a filtered current spectral coefficient with spectral index i, The audio decoder is configured to have a spectrum index id sf In the case where one or more spectral coefficients from +1 to i have been quantized or encoded or signaled as zero and have a spectral index id sf The filtering strength is selectively reduced when the spectral coefficients of the filter have not been quantized or encoded or signaled as zero.
25. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to claim 24, Which has id between sf The filter coefficients associated with spectral coefficients of spectral indices between +1 and i-1 are equal to zero.
26. The audio decoder (100, 400, 800, 1000, 1300, 1400, 1600) according to any one of claims 18 to 25, wherein the audio decoder is configured to use encoded or quantized or signaled spectral coefficients for determining the filtering strength, and Wherein the audio decoder is configured to use pre-processed spectral coefficients as input for the filtering or prediction.
27. A method (2100) for providing a decoded audio representation (104, 404, 802, 1002, 1304, 1404) based on an encoded audio representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602), the method comprising: filling (2101) spectral holes of a decoded set of spectral values (142, 442, 812, 1012, 1312, 1422) using corresponding filling values (122, 124, 222, 424, 814, 922, 1014, 1314, 1432); determining (2102) a padding value using prediction or filtering such that a given padding value associated with a given frequency is obtained dependent on another spectral value (914, 924, 1322, 1334, 1434, 1614) associated with a different frequency, The filter strength is adapted (2103) depending on the encoded or quantized spectral values (914, 924, 1322, 1334, 1434, 1614) associated with the different frequencies.
28. A method (2200) for providing a decoded audio representation (104, 404, 802, 1002, 1304, 1404) based on an encoded audio representation (102, 402, 504, 604, 804, 1004, 1102, 1202, 1302, 1402, 1602), the method comprising: determining (2201) processed spectral values using prediction or filtering such that a given processed spectral value associated with a given frequency is obtained in dependence on another spectral value (914, 924, 1322, 1334, 1434, 1614) associated with a different frequency, The filter strength is adapted (2202) depending on the encoded or quantized spectral values (914, 924, 1322, 1334, 1434, 1614) associated with the different frequencies.
29. A computer program for performing the method according to any one of claims 27 to 28 when the computer program is run on a computer.
Citation Information
Cited By
Method and apparatus for spectrotemporally improved spectral gap filling in audio coding using a filtering
US20240420710A1
