Apparatus and method for encoding an audio signal using compensation values ​​- Patents.com

Abstract

To improve perceptual quality by new parametric compensation techniques when there is a spectral microstructure mismatch between first and second spectral bands.SOLUTION: A speech signal encoder includes a core encoder for core encoding a first speech data in a first spectral band and a parametric coder 120 for parametric encoding second speech data in a second spectral band different from the first spectral band. The parametric coder includes: a signal analyzer that analyzes the first speech data in the first spectral band to obtain a first analysis result and the second speech data in the second spectral band to obtain a second analysis result; a compensator that calculates a compensation value using the first and second analysis results; and a parameter calculator for calculating parameters from the second speech data in the second spectral band using the compensation values.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to speech encoding and decoding, and in particular to speech encoding / decoding using bandwidth extension or spectral enhancement techniques such as Spectral Band Replication (SBR) or Intelligent Gap Filling (IGF). [Background technology]

[0002] Storage or transmission of speech signals is often subject to strict bitrate limitations. In the past, when only very low bitrates were available, the coder significantly reduced the bandwidth of the transmitted speech. Modern speech codecs can nowadays encode wideband signals using Bandwidth Expansion (BWE) methods [1-2]. Such algorithms rely on a parametric representation of the high-frequency content (HF), which is generated from a waveform that encodes the low-frequency part (LF) of the decoded signal by transposing it into the HF spectral domain ("patching") and applying parameter-driven post-processing. However, if the spectral fine structure in the patch copied to some target region, for example, is significantly different from that of the original content, unpleasant artifacts will appear, reducing the perceptual quality of the decoded speech signal.

[0003] In BWE schemes, the reconstruction of the HF spectral region above a given so-called crossover frequency is often based on spectral patching. Typically, the HF region consists of several adjacent patches, each of which is fed from a band-pass (BP) region of the LF spectrum below the given crossover frequency. In state-of-the-art systems, the patching is efficiently done in a filter bank representation by copying a set of adjacent subband coefficients from the source domain to the target domain. In a next step, the spectral envelope is adjusted to approximate the envelope of the original HF signal measured at the encoder and transmitted in the bitstream as side information.

[0004] However, there are often inconsistencies in the spectral fine structure that can lead to the perception of artifacts. A commonly known inconsistency concerns tonality: if the original HF contains a tone with a rather dominant energy content, and the patch copied at the spectral location of the tone is noisy, this bandpass noise will be magnified and may be heard as an unpleasant noise burst.

[0005] Spectral Band Replication (SBR) is a well-known BWE employed in modern speech codecs [1]. In SBR, the tonality mismatch problem is addressed by inserting an artificial replacement sine wave. However, this requires additional side information to be transmitted to the decoder, which increases the bit demand of the BWE data. Furthermore, the inserted tones may become unstable over time if the tonal insertion is switched on and off for subsequent blocks.

[0006] Intelligent Gap Filling (IGF) refers to a semi-parametric coding technique in modern codecs such as MPEG-H 3D Audio, or 3GPP EVS codec. IGF can be adapted to fill spectral holes introduced by the quantization process of the encoder due to low bitrate limitations. Usually, when bit allocation is limited and transparent coding is not possible, spectral holes first appear in the high frequency (HF) region of the signal and gradually affect the entire upper spectral range where the bitrate is lowest. On the decoder side, such spectral holes are replaced via IGF with synthetic HF content generated in a semi-parametric way from the low frequency (LF) content and post-processing controlled by another parametric side information.

[0007] Although IGF is essentially based on filling the high-frequency spectrum by copying spectral parts (called tiles) from lower frequencies and adjusting the energy by applying gain factors, this can be problematic if, from the point of view of spectral fine structure, in the original signal, the frequency range used as source for the copy-up process differs from its destination.

[0008] One such case that can have a strong perceptual impact is a tonality discrepancy. This tonality mismatch can occur in two different ways: first, because a frequency range with strong tonality is copied into a spectral region that appears structurally noise-like, and second, conversely, because noise is replaced by tonal components in the original signal. The former case is more common in IGF since most speech signals usually become more noise-like towards higher frequencies, and is addressed by applying spectral whitening, and if necessary, a parameter is sent to the decoder that tells how much whitening is required. For the latter case, the tonality can be corrected using the full-band coding capabilities of the core coder to preserve the tonal lines in the HF band via waveform coding. This so-called "surviving line" can be selected based on the strength of the tonality. Waveform coding is very demanding in terms of bitrate and is most likely not available in low bitrate scenarios. Furthermore, switching between frames that code tonal components and those that do not causes unpleasant artifacts and needs to be prevented.

[0009] The intelligent gap filling technique is further disclosed and explained in EP 2830054. The IGF technique addresses the problem of separation of the bandwidth extension on the one hand and the core decoding on the other hand by performing the bandwidth extension in the same spectral region that the core decoder processes. Thus, a full-rate core encoder / decoder is provided, encoding and decoding the entire audio signal range. This does not require a downsampler on the encoder side and an upsampler on the decoder side. Instead, the entire processing is performed at the full sampling rate, or in the full bandwidth region. To obtain a high coding gain, the audio signal is analyzed to find a first set of first spectral parts, which need to be coded with high resolution, and which may in some embodiments include the tonal parts of the audio signal. On the other hand, the non-tonal or noisy components in the audio signal, which constitute the second set of second spectral parts, are parametrically coded with low spectral resolution. The encoded audio signal then requires that only a first set of first spectral parts is encoded in a waveform-preserving manner at a high spectral resolution, and a second set of second spectral parts is parametrically encoded at a low resolution using frequency "tiles" provided from the first set. At the decoder side, a core decoder, which is a full-band decoder, reconstructs the first set of first spectral parts in a waveform-preserving manner, i.e. without knowing that there is an additional frequency regeneration. However, the spectrum thus generated has a number of spectral gaps. Such gaps are then filled with the inventive intelligent gap-filling (IGF) technique, on the one hand using frequency regeneration applying parametric data, and on the other hand using the source spectral range, i.e. the first spectral parts reconstructed by the full-rate audio decoder.

[0010] The IGF technology is also included and disclosed in 3GPP TS26.445V13.2.0(2016-06), Third Generation Partnership Project, Technical Specification Group Services and System Aspect, Codec for Enhanced Voice Services(EVS), Detailed Algorithmic Description(release13). In particular, for the encoder side, reference is made to Section 5.3.3.2.11 "Intelligent Gap Filling" of this reference, and for the decoder side implementation, reference is further made to Section 6, particularly Section 6.2.2.3.8 "IGF Apply", other IGF-related sections such as Section 6.2.2.2.9 "IGF Bitstream Reader", or Section 6.2.2.3.11 "IGF Temporal Flattening".

[0011] EP 2301027 A1 discloses an apparatus and method for generating bandwidth extension output data. For voiced signals, lowering the calculated noise floor provides a higher perceptual quality compared to the original calculated noise floor. The resulting speech is less reverberant in this case. For sibilant speech signals, the artificial increase of the noise floor may cover the shortcomings of the patching method for sibilant sounds. Thus, the reference discloses decreasing the noise floor for voiced and other signals, and increasing the noise floor for sibilant and other signals. To distinguish between different signals, the embodiment uses energy distribution data (e.g., sibilance parameters) to measure whether most of the energy is located at multiple or one high frequencies, i.e. whether the spectral display of the speech signal shows an increase or decrease that slopes towards high frequencies. In another implementation, the first LPC coefficient (LPC stands for Linear Predictive Code) is further used to generate the sibilance parameters. [Prior art documents] [Patent documents]

[0012] [Patent Document 1] European Patent Application Publication No. 2830054 [Patent Document 2] European Patent Application Publication No. 2301027 [Non-patent literature]

[0013] [Non-Patent Document 1] 3GPP TS26.445V13.2.0(2016-06), Third Generation Partnership Project, Technical Specification Group Services and System Aspect, Codec for Enhanced Voice Services(EVS), Detailed Algorithmic Description(release13) Summary of the Invention [Problem to be solved by the invention]

[0014] It is an object of the present invention to provide an improved concept for audio coding or processing. [Means for solving the problem]

[0015] This object is achieved by an apparatus for encoding an audio signal according to claim 1, a method for encoding an audio signal according to claim 23, a system for processing an audio signal according to claim 24, a method for processing an audio signal according to claim 25 or a computer program according to claim 26.

[0016] The apparatus for encoding an audio signal comprises a core encoder for core-encoding first audio data in a first spectral band and a parametric coder for parametrically encoding second audio data in a second spectral band, different from the first spectral band. In particular, the parametric coder comprises an analyzer for analyzing the first audio data in the first spectral band to obtain a first analysis result and for analyzing the second audio data in the second spectral band to obtain a second analysis result. The compensator calculates a compensation value using the first analysis result and the second analysis result. Furthermore, a parameter calculator calculates parameters from the second audio data in the second spectral band using the compensation value determined by the compensator.

[0017] The invention is therefore based on the understanding that a first spectral band, usually a source band, is analysed to obtain a first analysis result in order to establish whether the reconstruction performed at the decoder side with some parameters addresses some features required for the audio signal. Similarly, a second spectral band, usually a target band and reconstructed at the decoder side with the first spectral band, i.e. the source band, is further analysed by the analyser to obtain a second analysis result. Thus, for the source band, a separate analysis result is calculated as for the target band.

[0018] Then, based on these two analysis results, the compensator calculates compensation values ​​in order to modify some parameters that would have been obtained without compensation for the correction values. In other words, unlike the usual procedure in which parameters of a second spectral band are calculated from the original audio signal and transmitted to the decoder so that the second spectral band is reconstructed with the calculated parameters, the invention instead produces compensated parameters calculated from the target band on the one hand and compensation values ​​that depend on both the first and the second analysis results on the other hand.

[0019] The calculation of the compensated parameters can be performed by first calculating the uncompensated parameters and then combining the uncompensated parameters with the compensation values ​​to obtain the compensated parameters, or the compensated parameters can be calculated in one go without obtaining the uncompensated parameters as an intermediate result. The compensated parameters can then be transmitted from the encoder to the decoder, which then applies a certain bandwidth extension technique, such as spectral band duplication, or intelligent gap filling, or any other procedure using the compensated parameter values. Thus, regardless of whether the parameters result in the desired spectral band extension result, the adherence to a certain parameter calculation algorithm can be flexibly addressed by performing a signal analysis in the source band and the target band in addition to the parameter calculation, and then performing the calculation of the compensation values ​​based on the results from the source band and the results from the target band, i.e., based on the first spectral band and the second spectral band, respectively.

[0020] Preferably, the analyzer and / or the compensator apply a kind of psychoacoustic model to determine the psychoacoustic inconsistency. Thus, in an embodiment, the calculation of the compensation value is based on the detection of the psychoacoustic inconsistency of some signal parameters, such as tonality, and the compensation approach is applied to the overall minimization of the perceptual discomfort through the modification of other signal parameters, such as the spectral band gain factors. Thus, a perceptually well-balanced result is obtained by trading off different kinds of artifacts.

[0021] In contrast to the prior art approach of "fixing tonality at all costs," embodiments instead teach improving artifacts by applying attenuation to problematic parts of the spectrum where tonality mismatch is detected, thereby trading off mismatch in spectral energy envelopes against tonality mismatch.

[0022] At the input of some signal parameters, a compensation approach that includes a model of perceptual unpleasantness can determine the approach that gives the best perceptual match, rather than simply which signal parameters are matched.

[0023] The approach consists of weighting the perceptual importance of potential artifacts and selecting a combination of parameters to minimize the overall artifacts.

[0024] The technique is primarily intended to be applied within BWE, based on transforms such as MDCT, nevertheless the teachings of the present invention are equally applicable, for example, within systems based on quadrature mirror filter banks (QMF) as well.

[0025] One possible scenario in which this technique can be applied is the detection and subsequent attenuation of noise bands in the context of Intelligent Gap Filling (IGF).

[0026] In an embodiment, possible tonal mismatches are handled by detecting their occurrence and attenuating the corresponding scaling factors to reduce their impact, which on the one hand may lead to deviations from the spectral energy envelope of the original sound, but on the other hand may lead to a reduction in the noisiness of the HF, which helps to improve the overall perceptual quality.

[0027] Thus, the present embodiment specifically improves perceptual quality, which is typically driven by models of perceptual unpleasantness, by a new parametric compensation technique, for example when there is a mismatch in the spectral fine structure between a source or first spectral band and a target or second spectral band.

[0028] Preferred embodiments are described below with reference to the accompanying drawings. [Brief description of the drawings]

[0029] [Figure 1]1 shows a block diagram of an apparatus for encoding an audio signal according to an embodiment; [Diagram 2] 1 shows a block diagram of an apparatus for encoding, with a compensation detector at the center. [Figure 3a] 1 shows a schematic diagram of an audio spectrum with a source range and an IGF or bandwidth extension range and the associated mapping between source and destination bands. [Figure 3b] The core encoder applies the IGF technique to show the spectrum of the audio signal, with the remaining lines in a second spectral band. [Figure 3c] 2 shows a diagram of simulated first audio data in a first spectral band used to calculate a first analysis result; [Figure 4] 1 shows a diagram of the compensator in more detail. [Diagram 5] 1 shows a diagram of the parameter calculator in more detail. [Figure 6] 1 shows a flow chart illustrating the functionality of a compensation detector in one embodiment. [Figure 7] 13 illustrates the functionality of the parameter calculator to calculate the uncompensated gain factors. [Figure 8a] 1 illustrates an implementation of an encoder having a core decoder that calculates a first analysis result from a first encoded and decoded spectral band. [Figure 8b] 1 illustrates a block diagram of an encoder in one embodiment, where a patch simulator is adapted to generate a first spectral bandwidth line shifted from a second spectral band to obtain a first analysis result. [Figure 9] 13 illustrates the effect of tonal mismatch on the implementation of intelligent gap filling. [Figure 10] 1 illustrates an implementation of a parametric encoder in one embodiment. [Figure 11a] 4 shows the results of a listening test obtained by encoding speech data using the compensated parameter values. [Figure 11b] 4 shows further results of a listening test obtained by encoding speech data using the compensated parameter values. [Figure 11c] 4 shows further results of a listening test obtained by encoding speech data using the compensated parameter values. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Fig. 1 shows an apparatus for encoding an audio signal 100 according to an embodiment of the present invention. The apparatus comprises a core encoder 110 and a parametric coder 120. Furthermore, the core encoder 110 and the parametric coder 120 are connected at their inputs to a spectral analyzer 130 and at their outputs to an output interface 140. The output interface 140 generates an encoded audio signal 150. The output interface 140 receives on the one hand an encoded core signal 160 and on an input line 170 a complete parameter representation including at least one parameter of a second spectral band and usually also the parameters of the second spectral band. Furthermore, the spectral analyzer 130 separates the audio signal 100 into a first spectral band 180 and a second spectral band 190. In particular, the parameter calculator comprises an analyzer 121, illustrated in Fig. 1 as a signal analyzer, which analyzes the first audio data in a first spectral band 180 to obtain a first analysis result 122 and the second audio data in a second spectral band 190 to obtain a second analysis result 123. The first analysis result 122 and the second analysis result 123 are both provided to a compensator 124 to calculate a compensation value 125. The compensator 124 is thus configured to calculate the compensation value using the first analysis result 122 and the second analysis result 123. Then, the compensation value 125 on the one hand and at least second audio data from the second spectral band 190 (the first spectral data from the first spectral band may be used as well) are both provided to a parameter calculator 126 to calculate the parameters 170 from the second audio data in the second spectral band using the compensation value 125.

[0031] The spectrum analyzer 130 in Fig. 1 can be, for example, a simple time-to-frequency transformer to obtain the individual spectral bands, or MDCT lines. Thus, in this implementation, the spectrum analyzer 130 performs a modified discrete cosine transform (MDCT) to obtain the spectral data. This spectral data is then further analyzed to separate data for the core encoder 110 on the one hand and data for the parametric coder 120 on the other hand. The data for the core encoder 110 includes at least the first spectral band. Furthermore, when the core encoder encodes one or more source bands, the core data may further include other source data.

[0032] Thus, the core encoder may receive the entire bandwidth below the crossover frequency as input data to be core coded in the case of the spectral band replication technique, and the parametric coder then receives all audio data above this crossover frequency.

[0033] However, in the case of an intelligent gap filling framework, the core encoder 110 may further receive spectral lines above the IGF start frequency, which are also analyzed by the spectrum analyzer 130, so that the spectrum analyzer 130 further determines data above the IGF start frequency, which is further encoded by the core encoder. For this purpose, the spectrum analyzer 130 may be implemented as a "tonal mask", as for example described in "IGF Tonal Mask" disclosed in 3GPP TS 26.445 V13.0.0(12) section 5.3.3.2.11.5. Thus, a tonal mask is calculated by the spectrum analyzer 130 to determine which spectral components should be transmitted by the core encoder. Thus, all significant spectral content is identified, while content well suited for parametric coding via IGF is quantized to zero by the tonal mask. The spectral analyzer 130 forwards to the parametric coder 120 spectral content that is well suited for parametric coding, which may for example be data that has been set to zero by a tonal masking process.

[0034] In the embodiment shown in Fig. 2, the parametric coder 120 is further configured to parametrically code the third audio data in a third spectral band to obtain further parameters 200 for this third spectral band. In this case, the analyzer 121 is configured to analyze the third audio data in the third spectral band 202 to obtain a third analysis result 204 in addition to the first analysis result 122 and the second analysis result 123.

[0035] Furthermore, the parametric coder 120 of Fig. 1 further comprises a compensation detector 210 for detecting whether the third spectral band is compensated using at least the third analysis result 204. This detection result is output by a control line 212 indicating the presence or absence of the compensation status of the third spectral band. If the compensation detector detects that the third spectral band is not compensated, as indicated by the control line 212, the parameter calculator 126 is configured to calculate the further parameter 200 of the third spectral band without a compensation value. However, if the compensation detector detects that the third spectral band is compensated, the parameter calculator is configured to calculate the further parameter 200 of the third spectral band from the third analysis result 200 with the additional compensation value calculated by the compensator 124.

[0036] In a preferred embodiment, when quantitative compensation is applied, the analyzer 121 is configured to calculate a first quantitative value 122 as a first analysis result and a second quantitative value 123 as a second analysis result. Then, the compensator 124 is configured to calculate a quantitative compensation value 125 from the first quantitative value and from the second quantitative value. Finally, the parameter calculator is configured to calculate a quantitative parameter using the quantitative compensation value.

[0037] However, the invention is also applicable when only qualitative analysis results are available. In this case, a qualitative compensation value is calculated, which in turn controls a parameter calculator to lower or raise a certain uncompensated parameter to a certain degree. Thus, both analysis results may increase or decrease a parameter to a certain degree, and the degree of increase or decrease is fixed and does not depend on any quantitative result. However, quantitative results are preferred to fixed increase / decrease increments, although the calculation of fixed increase / decrease increments is less computationally intensive.

[0038] Preferably, the signal analyzer 121 analyses a first feature of the audio data to obtain a first analysis result and further analyzes the same first feature of the second audio data in a second spectral band to obtain a second analysis result. Conversely, the parameter calculator is configured to calculate parameters from the second audio data in the second spectral band by evaluating a second feature, which second feature is different from the first feature.

[0039] By way of example, Fig. 2 shows a situation where the first feature is the spectral fine structure or energy distribution within a certain band, such as the first band, the second band or any other band. Conversely, the second feature applied by or determined by the parameter calculator is a spectral envelope measure, an energy measure or a power measure, or in general an amplitude-related measure, such as a gain factor, giving an absolute or relative measure of the power / energy within the band. However, other parameters measuring different features from the gain factor feature can be calculated by the parameter calculator as well. Furthermore, other features of the separate source bands, on the one hand, and destination bands, respectively the first and second spectral bands, can be applied and analyzed by the analyzer 121.

[0040] Furthermore, the analyzer 121 is configured to calculate the first analysis result 122 without using the second audio data in the second spectral band 190, and further calculate the second analysis result 123 without using the first audio data in the first spectral band 180, in this embodiment the first spectral band and the second spectral band are mutually exclusive, i.e. they do not overlap with each other.

[0041] Moreover, the spectrum analyzer 130 is further configured to window the incoming stream of audio samples to construct frames of the audio signal, or to obtain frames of audio samples, where the audio samples of adjacent frames overlap each other, e.g., in the case of 50% overlap, a later part of an earlier frame has audio samples derived from the same original audio samples contained in a first half of a later frame, and the audio samples within a frame are derived from the original audio samples by windowing.

[0042] In this case, the audio signal comprises a time series of frames, e.g. as further provided by block 130 of Fig. 1 further comprising the functionality of a frame builder, and the compensator 124 is configured to calculate a current compensation value for a current frame using previous compensated frame values ​​of previous frames, which is usually a kind of smoothing process.

[0043] As will be outlined below, compensation detector 210 shown in FIG. 2 may additionally or alternatively include power spectrum and transient inputs, indicated at 221 and 223, respectively, from other functions in FIG.

[0044] In particular, the compensation detector 210 is configured to indicate that the parameter calculator 126 should only use compensation when the power spectrum of the original audio signal 100 of Figure 1 is available. This, i.e., the availability of the power spectrum, is signaled by several data elements or flags.

[0045] Furthermore, the compensation detector 210 is configured to only enable the compensation process via the control line 212 when the transient information line 223 signals that no transient is present for the current frame. Thus, when the line 223 signals that a transient is present, the entire compensation process is disabled, regardless of the analysis result. This is of course applied to the third spectral band when a signal is sent informing compensation for the second spectral band. However, for some frames, this is also applied to the second spectral band if a situation such as a transient situation is detected for this frame. This situation therefore allows and will prevent parameter compensation from being performed at all for some time frames.

[0046] Figure 3a shows the amplitude A(f), or the squared amplitude A 2 (f) A plot of the spectrum is shown, in particular the XOVER or IGF onset frequency is shown.

[0047] Further, a set of overlapping source bands are shown, including a first spectral band 180, another source band 302, and yet another source band 303. Also, destination bands above the IGF or XOVER frequency are, for example, a second spectral band 190, another destination band 305, yet another destination band 307, and a third spectral band 202.

[0048] Typically, a mapping function within the IGF or bandwidth extension framework defines the mapping between the individual source bands 180, 302, 303 and the individual destination bands 305, 190, 307, 202. This mapping can be fixed, as in the case of 3GPP TS 26.445, or it can be adaptively determined by some IGF encoder algorithms. In either case, in Fig. 3a, the table below shows the mapping between destination bands and source bands for non-overlapping destination bands and overlapping source bands, and the spectrum is shown in the upper part of Fig. 3a, regardless of whether this mapping is fixed for several frames or is actually adaptively determined.

[0049] FIG. 4 shows an implementation of the compensator 124 in more detail. In this implementation, the compensator 124 receives an analysis result 123 for the second spectral band in addition to a first analysis result 122, which may be a spectral flatness measure, a crest factor, a spectral tilt value, or any other kind of parametric data for the first spectral band. This analysis result may again be a spectral flatness measure for the second spectral band, a crest factor or a tilt value for the second spectral band, i.e. a spectral tilt value limited to the second spectral band, the tilt value or the spectral tilt value for the first spectral band also being limited to the first spectral band. The compensator 124 also receives spectral information for the second spectral band, such as a stop line for the second spectral band. Thus, in a situation where the parametric calculator 126 of FIG. 2 is configured to parametrically encode the third audio data in the third spectral band 202, the third spectral band includes higher frequencies than the second spectral band. This is also illustrated in the example of Fig. 3a, where the third spectral band is at a higher frequency than the second spectral band, i.e. band 202 has a higher frequency than band 190. In this situation, compensator 124 is configured to use a weighting value for the calculation of the compensation value for the third spectral band, which third weighting value is different from the weighting value used for the calculation of the compensation value for the second spectral band. Compensator 124 therefore generally influences the calculation of compensation value 125 in such a way that, for the same other input values, the compensation value becomes smaller at higher frequencies.

[0050] The weighting value can be, for example, an exponent, such as the exponent α, that is applied to the calculation of the compensation value based on the results of the first and second analysis, as described below, or it can also be, for example, a multiplication value, or an added or subtracted value, that has a different influence on higher frequencies compared to its influence when calculating parameters for lower frequencies.

[0051] Also, as shown in Fig. 4, the compensator receives the tone-to-noise ratio of the second spectral band, and calculates a compensation value dependent on the tone-to-noise ratio of the second audio data in the second spectral band, so that a first compensation value for the first tone-to-noise ratio is obtained, or a second compensation value for the second tone-to-noise ratio is obtained, and when the first tone-to-noise ratio is greater than the second tone-to-noise ratio, the first compensation value is greater than the second compensation value.

[0052] As mentioned above, the compensator 124 is configured to approximately determine the compensation value by applying a psychoacoustic model, which is configured to evaluate the psychoacoustic mismatch between the first and second audio data using the first and second analysis results to obtain the compensation value. This psychoacoustic model for evaluating the psychoacoustic mismatch can be implemented as a feedforward calculation, as described below in the context of the subsequent SFM calculation, or alternatively can be a feedback calculation module that applies a certain analysis by synthesis procedure. Furthermore, the psychoacoustic model can be implemented as a neural network or a similar structure that is automatically fed by certain training data to determine in which cases compensation is necessary and in which cases compensation is not necessary.

[0053] Subsequently, the functionality of the compensation detector 210 shown in FIG. 2, or generally the detector included in the parameter calculator 120, is shown.

[0054] The compensation detector function is configured to detect a compensation situation when the difference between the first and second analysis results has a predetermined characteristic, for example as shown in 600 and 602 in FIG. 6. Block 600 is configured to calculate the difference between the first and second analysis results, and then block 602 determines whether the difference has a predetermined characteristic or a predetermined value. If it is determined that the predetermined characteristic is not present, then block 602 determines that no compensation is performed, as indicated by reference numeral 603. However, if it is determined that the predetermined characteristic is present, control proceeds via line 604. Furthermore, the detector is alternatively or additionally configured to determine whether the second analysis result has some predetermined value or some predetermined characteristic. If it is determined that the characteristic is not present, then line 605 sends a signal indicating that no compensation is performed. However, if it is determined that the predetermined value is present, control proceeds via line 606. In this embodiment, lines 604 and 606 are considered to be sufficient to determine whether there is compensation or not. However, in the embodiment shown in FIG. 6, a different determination is made based on the spectral tilt of the second audio data in the second spectral band 190 of FIG. 1, as will be described below.

[0055] In one embodiment, the analyzer is configured to calculate as a first analysis result a spectral flatness measure, a crest factor, or a quotient of the spectral flatness measure and the crest factor for the first spectral band, and to calculate as a second analysis result a spectral flatness measure, or a crest factor, or a quotient of the spectral flatness measure and the crest factor of the second audio data.

[0056] In such an embodiment, the parameter calculator 126 is further configured to calculate the spectral envelope information, or the gain factor, from the second audio data.

[0057] Further, in such an embodiment, the compensator 124 is configured to calculate a compensation value 125, such that a first compensation value is obtained for a first difference between the first analysis result and the second analysis result, and a second compensation value is calculated for a difference between the first analysis result and the second analysis result, and when the first compensation value is greater than the second compensation value, the first difference is greater than the second difference.

[0058] Next, the description of FIG. 6 further indicates that an optional additional determination is made as to whether a compensation situation is detected.

[0059] In block 608, a spectral tilt is calculated from the second audio data. If it is determined that the spectral tilt is below a threshold, as shown at 610, then a compensation condition is affirmed and confirmed, as shown at 612. However, if it is determined that the spectral tilt is not below the predefined threshold but is above it, then a signal is sent on line 614 indicating this condition. In block 616, it is determined whether a tonal component is near the boundary of the second spectral band 190. If it is determined that there is a tonal component near the boundary, as shown at 618, then a compensation condition is again affirmed and confirmed. However, if it is determined that there is no tonal component near the boundary, then the compensation is cancelled, i.e., switched off, as shown at line 620. The determination in block 616, i.e., whether the tonal component is near the boundary, is made by performing a shifted SFM calculation in any embodiment. When there is an abrupt decrease in slope, as determined in block 608, the frequency region over which the SFM is calculated is shifted downward by half the width of the corresponding scale factor band (SFB), or second spectral band. For strong slopes, the frequency region over which the SFM is calculated is shifted upward by half the width of the second spectral band. In this way, tonal components that would otherwise be attenuated due to their low SFM can be correctly detected, and no attenuation is applied for high SFM values.

[0060] 5 will now be described in more detail. In particular, the parameter calculator 126 may comprise a calculator 501 for calculating the uncompensated parameters from the audio data of the second spectral band, i.e. the destination band, and the parameter calculator 126 further comprises a combiner 503 for combining the uncompensated parameters 502 and the compensation value 125. This combination may be a multiplication, for example when the uncompensated parameters 502 are gain values ​​and the compensation value 105 is a quantitative compensation value. However, the combination performed by the combiner 503 may alternatively be a weighting process using the compensation value as an exponential or additive correction, and the compensation value is used as an additive or subtractive value.

[0061] Furthermore, it should be noted that the embodiment shown in Fig. 5 is the only embodiment in which the uncompensated parameters are calculated followed by a combination with the combined value. In an alternative embodiment, the compensation value can already be introduced into the calculation of the compensated parameters, so that no intermediate results with explicit uncompensated parameters are generated. Instead, only a single process is performed, and the result of this "single process" is the calculation of the compensated parameters using the compensation value and using a calculation algorithm, and when the compensation value 125 is not introduced into such calculation, the calculation result is the uncompensated parameter.

[0062] FIG. 7 illustrates the procedure applied by the calculator 501 to calculate the uncompensated parameters. The "Calculation of IGF Scale Factors" illustrated in FIG. 7 corresponds roughly to clause 5.3.3.2.11.4 of 3GPP TS 26.445 V13.3.3 (2015 / 12). Once the "complex" TCX power spectrum P (the spectrum in which the real and imaginary parts of the spectral lines are evaluated) is available, the calculator 501 of FIG. 5 for calculating the uncompensated parameters performs a calculation of the amplitude-related measures of the second spectral band from the power spectrum P, as indicated at 700. Furthermore, the calculator 501 performs a calculation of the amplitude-related measures of the first spectral band, as indicated at 702, from the complex spectrum P. The calculator 501 also performs a calculation of the amplitude-related measures from the real part of the first spectral band, i.e. the source band, as indicated at 704, resulting in the three amplitude-related measures, Ecplx,target , E cplx,source , E real,source is obtained and input to another gain coefficient calculation function 706, which finally gives E real,source E cplx,source Divide by E cplx,target A gain factor is obtained which is a function of the multiplication of

[0063] Alternatively, when the complex TCX power spectrum is not available, the amplitude-related measures are calculated from the actual second spectral band only, as shown in the lower part of FIG.

[0064] Further, it is noted that the TCX power spectrum P is calculated based on the following formula, for example as shown in subclause 5.3.3.2.11.1.2:

[0065] P(sb)=R 2 (sb)+I 2 (sb), sb=0,1,2,…,n-1 where n is the actual TCX window length, R is a vector containing the (cosine transformed) real-valued part of the current TCX spectrum, and I is a vector containing the (sine transformed) imaginary part of the current TCX spectrum. In particular, the term "TCX" refers to 3GPP terminology, but generally refers to the spectral values ​​of the first or second spectral band as provided by the spectrum analyzer 130 to the core encoder 110 or parametric coder 120 of FIG. 1.

[0066] FIG. 8a shows a preferred embodiment, in which the signal analyzer 121 further comprises a core decoder 800 which calculates the encoded and re-decoded first spectral band and, of course, the encoded / decoded audio data of the first spectral band.

[0067] The core decoder 800 then sends the coded / decoded first spectral band to an analysis result calculator 801 included in a signal analyzer 821 for calculating a first analysis result 122. Furthermore, the signal analyzer comprises a second analysis result calculator 802 included in the signal analyzer 121 of Fig. 1 for calculating a calculated second analysis result 123. Thus, the signal analyzer 121 is arranged such that the actual first analysis result 122 is calculated using the coded and re-decoded first spectral band, while the second analysis result is calculated from the original second spectral band. Thus, the situation on the decoder side is well simulated on the encoder side, since the input to the analysis result calculator 801 already has all the quantization errors included in the decoded first audio data of the first spectral band available to the decoder.

[0068] Figure 8b shows another preferred implementation of the signal analyzer, which instead of or in addition to the steps of Figure 8a, has a patch simulator 804. The patch simulator 804 specifically recognises the functionality of the IGF encoder, i.e. it recognises that there may be multiple lines, or at least one line, in the second destination band that is actually encoded by the core encoder.

[0069] Specifically, this situation is shown in FIG. 3b. Figure 3b, like the top diagram of Figure 3a, illustrates a first spectral band 180 and a second spectral band 190. However, in addition to what was discussed in Figure 3a, the second spectral band includes certain lines 351, 352 contained within the second spectral band that have been determined by the spectral analyzer 130 as lines to be further encoded by the core encoder 110 in addition to the first spectral band 180.

[0070] This particular encoding of some lines above the IGF start frequency 310 is done by the core encoder 110 using a maximum f maxThis reflects the situation where the crossover frequency is a full-band encoder with a Nyquist frequency of 354. This is in contrast to implementations related to SBR techniques, where the crossover frequency is also the maximum frequency and therefore the Nyquist frequency of the core encoder 110.

[0071] The test simulator 804 receives from the core decoder 800 either the first spectral band 180 or the encoded first spectral band, and also receives information from the spectrum analyzer 130 or the core encoder 110 that there are actual lines in the second spectral band, contained in the core encoder output signal, which the spectrum analyzer 130 signals via line 806 or the core encoder signals via line 808. Here, the patch simulator 804 simulates the first audio data in the first spectral band by using simple first audio data in the four spectral bands and by shifting the lines 351, 352 into the first spectral band and inserting the lines 351, 352 from the second spectral band into the first spectral band. Thus, the lines 351' and 352' represent the spectral lines obtained by shifting the lines 351, 352 of FIG. 3b from the second spectral band into the first spectral band. Preferably, the spectral lines 351, 352 are generated for the first spectral band such that the positions of these lines within the band boundary are identical in both bands, i.e. the difference frequency between the lines and the band boundary is identical for the second spectral band 190 and for the first spectral band 180.

[0072] Thus, the patch simulator outputs simulated data 808, shown in Figure 3c, which has the simple first spectral band data, plus lines shifted from the second spectral band to the first spectral band. Here, the analysis result calculator 801 calculates the first analysis result 102 using the particular data 808, and the analysis result calculator 802 calculates the second analysis result 123 from the original second audio data in the second spectral band, i.e. the original audio data including lines 351, 352, shown in Figure 3b.

[0073] This procedure using the patch simulator 804 has the advantage that it is not necessary to input, for example, tonality or other constant conditions, for the additional lines 351, 352. Instead, it is entirely up to the spectrum analyzer 130, or the core encoder 110, to decide whether some lines of the second spectral band are to be coded by the core encoder or not. However, the result of this process is calculated automatically by using these lines as additional inputs for the calculation of the first analysis result 122, as shown in Fig. 8b.

[0074] Subsequently, the impact of tonality mismatch within the framework of intelligent gap filling is illustrated.

[0075] To detect noise band artifacts, it is necessary to determine the tonality difference between the source scale factor band (SFB) and the target SFB. The Spectral Flatness Measure (SFM) can be used to calculate the tonality. If a tonality mismatch is found where the source band is noisier than the target band, a certain amount of attenuation needs to be applied. This situation is shown in Figure 9, where the processing of the present invention has not been applied.

[0076] It also makes sense to apply some smoothing to the attenuation coefficients to avoid abrupt on / off behavior of the tool. The steps required to apply the attenuation in the right place are detailed below. (Note that the attenuation is applied only if both the TCX power spectrum P is available and the frame is non-transient (flag isTransient is disabled).

[0077] Detection of tonal mismatch: parameters In a first step, we need to identify those SFBs whose tonality mismatch may cause noise band artifacts. To do this, we need to determine the tonality of each SFB in the IGF range and the corresponding bands used for copy-up. One suitable measure to calculate tonality is the Spectral Flatness Measure (SFM), which is based on the quotient of the geometric mean of the spectrum divided by its arithmetic mean and ranges from 0 to 1. Values ​​close to 0 indicate strong tonality, while values ​​close to 1 represent a very noisy spectrum. The formula is: TIFF0007626790000001.tif19104

[0078] where P is the TCX power spectrum, b is the start line, e is the stop line of the current SFB, and p is defined as: TIFF0007626790000002.tif1888

[0079] In addition to SFM, the crest factor is calculated to further indicate how the energy is distributed within the spectrum by dividing the maximum energy by the average energy of all frequency bins in the spectrum. Dividing the SFM by the crest factor is a measure of the tonality of the SFB in the current frame. The crest factor is calculated as follows: TIFF0007626790000003.tif21141

[0080] where P is the TCX power spectrum, b is the start line, e is the stop line of the current SFB, and E max is defined as follows: TIFF0007626790000004.tif1390

[0081] However, to achieve a smooth tonality estimation, it makes sense to additionally use the results of previous frames. The tonality estimation is then performed according to the following formula: TIFF0007626790000005.tif13110 where sfm denotes the result of the actual spectral flatness calculation, the variable SFM includes the quotient by the crest factor as well as the smoothing.

[0082] Here, the tonality difference between the source and destination is calculated. A positive difference indicates a condition in which a noisier spectrum than the target spectrum is used to perform the copy-up. Such an SFB is a good candidate for attenuation.

[0083] However, low SFM values ​​do not necessarily indicate strong tonality, but may also be due to a sudden decrease or gradient in the energy in the SFB. This is especially applicable for items that have a band limit somewhere in the middle of the SFB. This can lead to undesirable attenuation, giving the impression that the signal is slightly low-pass filtered.

[0084] To avoid attenuation in such cases, SFM diff The potentially affected SFBs are determined by calculating the spectral tilt of the energy in all bands where A is positive; a strong tilt in one direction may indicate a steep drop causing a low SFM value. The spectral tilt is calculated as a linear regression through all spectral bins in the SFB, with the slope of the regression line given by: TIFF0007626790000007.tif17160x is the bin number, P is the TCX power spectrum, b is the start line, and e is the stop line of the current SFB.

[0085] However, tonal components close to the boundaries of the SFB may also cause steep slopes, but must also be attenuated. To separate these two cases, a separate shifted SFM calculation must be performed on the band with the steep slope.

[0086] The threshold value of the gradient is defined by the following formula: TIFF0007626790000008.tif1243 To normalize, divide by SFB width.

[0087] Strong incline If TIFF0007626790000009.tif649 is present, the frequency domain in which the SFM is calculated is shifted downwards by half the width of the SFB, resulting in a strong gradient For TIFF0007626790000010.tif750 it is shifted upwards. In this way, tonal components that would have been attenuated due to their low SFM can be correctly detected and no attenuation is applied for high SFM values. The threshold here is defined as a value of 0.04 and attenuation is applied only if the shifted SFM is below the threshold.

[0088] A model of perceptual discomfort Attenuation is positive SFM diff should not be applied to the target SFB and only becomes effective if the target SFB is indeed very tonal. In a given SFB, when the source signal is superimposed on a noisy background, the perceived difference for the noisy bands becomes rather small, and it may even be an advantage that the sound is less noticeable due to the energy loss caused by attenuation.

[0089] To ensure that it is applied within reasonable limits, attenuation should only be used if the tonality of the target SFB is indeed very high. Thus, TIFF0007626790000011.tif733 and Attenuation should be applied only if both TIFF0007626790000012.tif638 are preserved.

[0090] Another issue to consider is that of the background of tonal components in the IGF spectrum. The perceptual degradation caused by noise band artifacts is likely to be most noticeable whenever there is little or no spurious noise background surrounding the original tonal components. In this case, when comparing the original sound with the IGF-generated HF spectrum, the introduced noise band will be perceived as something completely new and very noticeable and prominent. On the other hand, if a significant amount of background noise is already present, the other noise will blend into the background, resulting in a less unpleasant perceptual difference. Therefore, the amount of attenuation applied must also depend on the tonal-to-noise ratio of the affected SFB.

[0091] For this tonal-to-noise ratio calculation, the squared TCX power spectrum values ​​P of all bins i in the SFB are summed and divided by the width of the SFB (determined by the start line b and the stop line e) to obtain the average energy of the band. This average is effectively used to normalize the total energy in the band. Normalized energy P below TIFF0007626790000013.tif181241 norm,k All bins with noise is counted as Anything above the threshold 1+adap, as determined by TIFF0007626790000014.tif1436, is in the tonal section P tonal This threshold depends on the width of the SFB, so that narrower bands have lower thresholds and higher averages due to the larger influence of high energy bins of tonal content. Finally, a log ratio is calculated from the tonal and noise parts. TIFF0007626790000015.tif1487

[0092] The attenuation depends on both the SFM difference between the source and destination and the SFM of the target SFB, and a larger difference and a smaller target SFM both lead to stronger attenuation. It makes sense that the larger the tonality difference, the stronger the attenuation needs to be applied. Furthermore, the amount of attenuation needs to increase more quickly when the target SFM is low, i.e. the target SFB is more tonal. That is, stronger attenuation is applied to very tonal SFBs than to SFBs whose SFM falls within the attenuation range.

[0093] Also, attenuation must be applied more carefully for higher frequencies, since the loss of energy in the highest frequencies is likely to lead to the perceptual impression of band-limitedness, and the fine structure of the SFB becomes less important at higher frequencies, as the human auditory system becomes less sensitive to them.

[0094] Compensating for tonal mismatch: Calculating the attenuation coefficient To incorporate all these considerations into a single attenuation formula, the ratio of the target SFM to the source SFM is taken as the basis of the formula. In this way, both larger absolute difference values ​​of SFM and smaller target SFM values ​​result in stronger attenuation, which fits better than simply taking the difference. Adjustment parameters are applied to this ratio to add further dependence on frequency and tone-to-noise ratio. The attenuation formula can thus be expressed as follows: TIFF0007626790000016.tif1458

[0095] where d is the damping coefficient multiplied by the scaling factor, and α and β are damping adjustment parameters, calculated as follows: TIFF0007626790000017.tif1252

[0096] where e is the current SFB stop line, TIFF0007626790000018.tif8114Here, adap depends on the width of the SFB and is calculated by the following formula: TIFF0007626790000019.tif1130

[0097] The parameter α decreases with frequency to apply less attenuation to higher frequencies, and β is used to further reduce the strength of the attenuation when the tonal-to-noise ratio of the SFB being attenuated falls below a threshold, the more significantly below which the attenuation is reduced.

[0098] Because the damping is only effective within certain constraints, smoothing needs to be applied to prevent abrupt transitions from on to off. To achieve this, several smoothing mechanisms are enabled.

[0099] After the transition, either directly,core switching to TCX or damping of the previous frame that was not damped, all of its power is applied gradually to avoid extreme energy drops after high energy transients.,Furthermore, a forgetting factor in the form of an IIR filter is used to,also take into account the results of previous frames.

[0100] All of the smoothing techniques are contained in the following equation: TIFF0007626790000020.tif1497 here d prev is the damping coefficient for the previous frame. If damping was not enabled in the previous frame, d prev d curr is overridden by attenuation, but limited to a minimum of 0.1. The variable smoothing is another smoothing factor, which is set to 2 during a transient frame (flag isTransient is enabled) if attenuation was disabled in the previous frame, or to 1 after a core switch (flag isCelpToTCX is enabled). Each frame with attenuation the variable decreases towards 1, but never goes below 0.

[0101] In the final step, the damping factor d is multiplied by a scaling gain g. TIFF0007626790000021.tif741 FIG. 10 illustrates a preferred implementation of the present invention.

[0102] For example, the audio signal as output by the spectrum analyzer 130 can be used as an MDCT spectrum, or a complex spectrum, as shown in (c) on the left side of FIG.

[0103] The signal analyzer 121 is implemented by tonality detectors 801 and 802 of FIG. 10 to detect the tonality of the target content by block 802 and to detect the tonality of the (simulated) source content in code 801 .

[0104] Next, a damping factor calculation 124 is performed to obtain a compensation value, which is then processed by compensator 503 using the data obtained from 501, 700-706. 501 and 700-706 reflect the envelope estimation from the target content and the envelope estimation from the simulated source content, and the subsequent scaling factor calculations shown, for example, in FIG. 7 at 700-706.

[0105] Therefore, the uncompensated scaling vector is input as value 502 into block 503, similar to what was described with reference to Figure 5. Furthermore, although the noise model 1000 is shown in Figure 10 as a separate building block, the same could also be included directly within the attenuation coefficient calculator 124, as described with reference to Figure 4.

[0106] The parametric IGF encoder of Fig. 10 further comprising a whitening estimator is also configured to calculate whitening levels, for example as described in section 5.3.3.2.11.6.4 of "Coding of IGF whitening levels". In particular, the IGF whitening levels are calculated and transmitted using one or two bits per tile. This data is also introduced to the bitstream multiplexer 140 to finally obtain the complete IGF parametric data.

[0107] Also provided is a further block "sparsify spectrum", which corresponds to block 130 for determining the spectral lines encoded by core encoder 110 and is shown in Fig. 10 as another block 1020. This information is preferably used by compensator 503 to reflect the particular IGF situation.

[0108] Furthermore, the term "simulated" to the left of block 801 and the block "envelope estimation" in Figure 10 refer to the situation shown in Figure 8a, where the "simulated source content" is audio data that has been encoded and decoded again in a first spectral band.

[0109] Alternatively, the "simulated" source content is data obtained by patch simulator 804 from original first audio data in a first spectral band, as shown by line 180, or the decoded first spectral band obtained by core decoder 800, enhanced with a line shifted from a second spectral band to the first spectral band.

[0110] Subsequently, another embodiment of the invention is illustrated, which constitutes a modified version of the 3GPP TS 26.445 codec. The newly added characters specifying the process of the invention are shown below. Here, explicit reference is made to some sub-clauses already included in the 3GPP TS 26.445 standard.

[0111] 5.3.3.2.11.1.9 Spectral slope function SLOPE Let TIFF0007626790000022.tif517 be the TCX power spectrum calculated in accordance with subclause 5.3.3.2.11.1.2, where b is the start line and e is the stop line of the spectral tilt measurement range.

[0112] The SLOPE function applied to the IGF is defined by the following formula:

[0113] SLOPE: TIFF0007626790000023.tif643TIFF0007626790000024.tif23115Here n is the actual TCX window length and x is the bin number.

[0114] 5.3.3.2.11.1.10. Tone-to-noise ratio function TNR Let TIFF0007626790000025.tif517 be the TCX power spectrum calculated in accordance with subclause 5.3.3.2.11.1.2, where b is the start line and e is the stop line of the tone-to-noise ratio measurement range.

[0115] The TNR function applied to the IGF is defined as follows:

[0116] TNR: TIFF0007626790000026.tif536TIFF0007626790000027.tif19114where n is the actual TCX window length, TIFF0007626790000028.tif725 is defined by the following formula: TIFF0007626790000029.tif1185And adap is defined by the following formula: TIFF0007626790000030.tif1030 Attenuation: For the IGF damping coefficient calculation, 6 static arrays (prevTargetFIR, prevSrcFIR, prevTargetIIR, and prevSrcIIR for SFM calculations in the target and source ranges, and prevDamp and dampSmooth) all of size nB are needed to hold the filter state across frames, and a static flag wasTransient is needed to save the information of the input flag isTransient from the previous frame.

[0117] Reset Filter State The vectors prevTargetFIR, prevSrcFIR, prevTargetIIR, prevSrcIIR, as well as prevDamp and dampSmooth are all static arrays of size nB in the IGF module and are initialized as follows:

[0118] for k=0,1,…,nB-1 TIFF0007626790000031.tif4059This initialization shall be performed under the following conditions:

[0119] -Involves starting a codec -Bitrate switching -Involves switching between codec types With transition from CELP to TCX, such as isCelpToTCX=true If the current frame has transient properties, such as isTransient=true When TCX power spectrum P is not available Calculating the damping coefficient If the TCX power spectrum P is available and isTransient is false, it is calculated according to the following formula: TIFF0007626790000032.tif14129 and TIFF0007626790000033.tif17125 here Assume that TIFF0007626790000034.tif744 has already been mapped by function tF, see subclause 5.3.3.2.11.1.1. TIFF0007626790000035.tif522 is a mapping function that maps the IGF target range to the IGF source range, as described in subclause 5.3.3.2.11.1.8, and nB is the number of scale factor bands as shown in Table 94. SFM is the spectral flatness scaling function, as described in subclause 5.3.3.2.11.1.3, and CREST is the crest factor function, as described in subclause 5.3.3.2.11.1.4.

[0120] If isCelpToTCX is true or wasTransient is true, it is set as follows.

[0121] for k=0,1,…,nB-1 TIFF0007626790000036.tif2679Calculate as follows. TIFF0007626790000037.tif16160 and TIFF0007626790000038.tif9116With these vectors, we calculate as follows: TIFF0007626790000039.tif6118if for TIFF0007626790000040.tif543TIFF0007626790000041.tif636or The image is TIFF0007626790000042.tif640, and the following is set: TIFF0007626790000043.tif1348

[0122] Alternatively, the spectral slope is calculated with the SLOPE function as described in subclause 5.3.3.2.11.1.9. TIFF0007626790000044.tif6108if for TIFF0007626790000045.tif542TIFF0007626790000046.tif648or else if TIFF0007626790000047.tif692

[0123] where threshTilt is defined as follows: TIFF0007626790000048.tif1365Calculate the SFM of the shifted spectrum. TIFF0007626790000049.tif13108 shift is defined as follows: TIFF0007626790000050.tif1382if Set TIFF0007626790000051.tif675 or less. TIFF0007626790000052.tif634if for TIFF0007626790000053.tif542TIFF0007626790000054.tif638In band k, set the damping coefficient dampCurr for the current frame to zero. TIFF0007626790000055.tif644 or Calculate TIFF0007626790000056.tif629 as follows. TIFF0007626790000057.tif10107

[0124] where alpha is defined as: TIFF0007626790000058.tif1270beta is defined as follows: TIFF0007626790000059.tif8119where TNR is the tone-to-noise ratio function described in subclause 5.3.3.2.11.1.10 and adap is defined as follows: TIFF0007626790000060.tif1255if for TIFF0007626790000061.tif541TIFF0007626790000062.tif645Set as follows. TIFF0007626790000063.tif688 Calculate a vector of attenuation coefficients d of size nB. TIFF0007626790000064.tif11150Finally, if isTransient is false and the power spectrum P is available, update the filter.

[0125] for k=0,1…,nB-1 TIFF0007626790000065.tif2578 The names of the values / indexes / parameters in the above mentioned parts are the same as the corresponding parameters / indexes / values ​​mentioned throughout this specification. Some results of the listening test will now be described with reference to Figures 11a to 11c.

[0126] These comprehension tests were conducted to demonstrate the benefit of attenuation by comparing items coded with enabled attenuation to items coded without attenuation.

[0127] The first result, shown in Figure 11a, is an AB comparison test with mono-items at a bit rate of 13.2 kbps and a sample rate of 32 kHz. In Figure 11a, the results of the AB test with attenuation vs. without attenuation at 13.2 kbps are shown.

[0128] The second set of results, shown in Figure 11b, is from a MUSHRA test using mono items at 24.4 kbps and a sample rate of 32 kHz. Here, two versions without attenuation were compared with a new version with attenuation. The results are shown in Figure 11b (absolute rating) and Figure 11c (differential rating).

[0129] The encoded audio signal according to the present invention may be stored in a digital storage medium, or a non-transitory storage medium, or may be transmitted over a transmission medium, such as a wireless transmission medium, or a wired transmission medium such as the Internet.

[0130] Although some aspects have been described in terms of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, with blocks or devices corresponding to method steps or features of method steps. Similarly, aspects described in terms of method steps further represent a description of corresponding blocks or symbols, or features of corresponding apparatus.

[0131] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software, which can be carried out using digital storage media such as floppy disks, DVDs, CDs, ROMs, PROMs, EPROMs, EEPROMs or flash memories, which store electronically readable control signals and which cooperate (or can cooperate) with a programmable computer system to carry out the respective methods.

[0132] Some embodiments according to the invention comprise a data carrier including electronically readable control signals, which can cooperate with a programmable computer system to perform one of the methods described herein.

[0133] Typically, embodiments of the present invention can be implemented as a computer program product having a program code which is effective to perform one of the methods when the computer program product is run on a computer. The program code may for example be stored on a machine readable carrier.

[0134] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium.

[0135] In other words, an embodiment of the inventive method is, therefore, a computer program comprising a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0136] Another embodiment of the inventive method is therefore a data carrier (or a digital storage medium or a computer readable medium) comprising a computer program recorded thereon for performing one of the methods described herein.

[0137] A further embodiment of the inventive method is therefore a data sequence or a sequence of signals representing the computer program for performing one of the methods described herein, the data sequence or sequence of signals being adapted to be transferred via a data communication connection, for example via the Internet.

[0138] Another embodiment comprises a processing means, such as a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0139] Another embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0140] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed in any hardware apparatus.

[0141] The above-described embodiments are merely illustrative of the principles of the present invention. It will be understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims and not by the specific details presented as descriptions and illustrations of the embodiments herein.

Claims

1. 1. An apparatus for encoding an audio signal, comprising: a core encoder (110) for core-encoding first audio data in a first spectral band (180) of the audio signal to obtain core-encoded audio data (160) in the first spectral band (180); A parametric coder (120) for parametrically encoding second audio data in a second spectral band (190) of the audio signal, the second spectral band being different from the first spectral band (180) of the audio signal, the parametric coder (120) comprising: an analyzer (121) for analyzing the first audio data in the first spectral band (180) of the audio signal to obtain a first analysis result (122) and for analyzing the second audio data in the second spectral band (190) of the audio signal to obtain a second analysis result (123); a compensator (124) for calculating a compensation value (125) using the first analysis result (122) and the second analysis result (123); a parameter calculator (126) for calculating parameters (170) from the second audio data in the second spectral band (190) of the audio signal using the compensation values ​​(125); an output interface (140) for outputting an encoded audio signal (150) including the core encoded audio data (160) in the first spectral band (180) of the audio signal and the parameters (170) representative of the second audio data in the second spectral band (190) of the audio signal; Equipped with the analyzer (121) is configured to further calculate a tonal-to-noise ratio (400) of the second audio data in the second spectral band (190) of the audio signal, and the compensator (124) is configured to calculate the compensation value (125) depending on the tonal-to-noise ratio (400) of the second audio data, such that a first compensation value for a first tone-to-noise ratio is obtained, or a second compensation value for a second tone-to-noise ratio is obtained, the first compensation value being greater than the second compensation value and the first tone-to-noise ratio being greater than the second tone-to-noise ratio, or The apparatus, wherein the compensator (124) is configured to determine the compensation value (125) by applying a model of perceptual unpleasantness (1000), the model of perceptual unpleasantness (1000) being configured to evaluate a psychoacoustic mismatch between the first audio data and the second audio data using the first analysis result (122) and the second analysis result (123) to obtain the compensation value (125).

2. the analyzer (121) is configured to analyze a first feature of the first audio data to obtain the first analysis result (122) and to analyze the same first feature of the second audio data of the second spectral band (190) of the audio signal to obtain the second analysis result (123), the first feature being a spectral fine structure feature or an energy distribution feature within the first spectral band (180) of the audio signal, the parameter calculator (126) is configured to calculate the parameters (170) from the second audio data of the second spectral band (190) of the audio signal by evaluating a second feature, the second feature being an envelope measure of spectral values ​​within the second spectral band (190) of the audio signal, or an energy-related or power-related measure.

2. The apparatus of claim 1.

3. the parameter calculator (126) is configured to calculate uncompensated parameters (502) from the second audio data in the second spectral band (190) of the audio signal, and is configured to combine (503) the uncompensated parameters (502) and the compensation values ​​(125) to obtain the parameters (170).

2. The apparatus of claim 1.

4. the audio signal (100) comprises a time sequence of frames, the analyzer (121) is configured to analyze the first audio data in the first spectral band (180) of the audio signal for a frame to obtain the first analysis result (122) and to analyze the second audio data in the second spectral band (190) of the audio signal for the frame to obtain a second analysis result (123) for the frame, the compensator (124) is configured to calculate a compensation value (125) for the frame using the first analysis result (122) for the frame and the second analysis result (123) for the frame; the parameter calculator (126) is configured to calculate the parameters (170) from the second audio data of the frame in the second spectral band (190) using the compensation values ​​(125) for the frame.

2. The apparatus of claim 1.

5. the parametric coder (120) further comprises a compensation detector (210) for detecting, based on the first analysis result (122) and the second analysis result (123), whether the parameters (170) of the second spectral band of a frame are calculated using the compensation values ​​(125) in a compensated situation or without using the compensation values ​​(125) in a non-compensated situation, 2. The apparatus of claim 1.

6. the parametric coder (120) comprises a compensation detector (210); the compensation detector (210) is configured to detect a compensation situation when a difference between the first analysis result (122) and the second analysis result (123) has a predetermined characteristic or when the second analysis result (123) has a predetermined characteristic; the compensation detector (210) is configured to detect that a spectral band is not compensated when a power spectrum is not available to the device for encoding the audio signal or when it is detected that the current frame is a transient frame (223); or the compensator (124) is configured to calculate the compensation value (125) based on a quotient of the first analysis result (122) and the second analysis result (123).

2. The apparatus of claim 1.

7. the analyzer (121) is configured to calculate as the first analysis result (122) a spectral flatness measure, a crest factor, or a quotient of the spectral flatness measure and the crest factor for the first spectral band (180) of the audio signal, and is configured to calculate as the second analysis result (123) a spectral flatness measure or a crest factor, or a quotient of the spectral flatness measure and the crest factor for the second spectral band (190) of the audio signal, or the parameter calculator (126) is configured to calculate spectral envelope information or a gain factor from the second audio data; 2. The apparatus of claim 1.

8. the compensator (124) is configured to calculate the compensation value (125), such that for a first difference between the first analysis result (122) and the second analysis result (123), a first compensation value is obtained, and for a second difference between the first analysis result (122) and the second analysis result (123), a second compensation value is calculated, such that the first difference is greater than the second difference and the first compensation value is greater than the second compensation value.

2. The apparatus of claim 1.

9. the parametric coder (120) comprises a compensation detector (210); the analyzer (121) is configured to calculate a spectral tilt (608) from the second audio data; the analyzer (121) is configured to check (616) for the presence of tonal components near the boundaries of the second spectral band (190) of the audio signal; the compensation detector (210) is configured to determine (612) to calculate the parameters (170) using the compensation value (125) only when the spectral tilt is below a predefined threshold (610) or when the spectral tilt is above a predefined threshold (614) and the examination (616) determines (618) that a tonal component is present near the boundary.

9. Apparatus according to claim 7 or 8.

10. a decoder (800) for decoding the core encoded audio data (160) for the first spectral band (180) of the audio signal to obtain encoded and decoded first audio data; the analyzer (121) is configured to calculate (801) the first analysis result (122) using the encoded and decoded first audio data; and and calculating (802) said second analysis result (123) from said second audio data coming from said audio signal input to said device for encoding said audio signal.

2. The apparatus of claim 1.

11. a patch simulator (804) for simulating a patching result of the second spectral band (190) of the audio signal, the patching result comprising at least one spectral line (351, 352) from the second spectral band (190) of the audio signal contained in the core encoded audio data (160); the analyzer (121) is configured to calculate the first analysis result (122) using the first audio data and the at least one spectral line (351', 352') from the second spectral band (190) of the audio signal; and configured to calculate said second analysis result (123) from said second audio data coming from said audio signal (100) input to said device for encoding, 2. The apparatus of claim 1.

12. the core encoder (110) is configured to encode the first audio data in a series of real-valued spectra; the analyzer (121) is configured to calculate the first analysis result (122) and the second analysis result (123) from a series of power spectra; a power spectrum is calculated from the audio signal (100) input to the device for encoding, or is derived from a real-valued spectrum used by the core encoder (110); 2. The apparatus of claim 1.

13. the core encoder (110) is configured to core encode the audio signal in a core band extending at least up to an emphasis start frequency (310); the core band includes the first spectral band (180) of the audio signal and at least one further source band (302, 303) that overlaps with the first spectral band (180) of the audio signal; the audio signal has an emphasis range extending from the emphasis start frequency (310) to a maximum frequency (354), the second spectral band (190) of the audio signal and at least one other target band are included in the emphasis range, and the second spectral band (190) of the audio signal and the other target band do not overlap with each other; 2. The apparatus of claim 1.

14. the emphasis start frequency (310) is a crossover frequency and the core encoded audio data (160) is band-limited to the crossover frequency (310); or the emphasis start frequency (310) is an intelligent gap filling (IGF) start frequency, and the core encoded speech data (160) is band-limited to the maximum frequency (354) that is greater than the emphasis start frequency.

14. The apparatus of claim 13.

15. The parameter calculator (126) calculating a gain factor for the second spectral band (190) of the audio signal based on the second audio data for the second spectral band (190) of the audio signal; Calculating a damping coefficient as the compensation value (125); and configured to multiply the gain factor for the second spectral band by the attenuation factor to obtain a compensated gain factor as the parameter (170); the output interface (140) is configured to output the compensated gain factor as the parameter (170); 2. The apparatus of claim 1.

16. 1. A method for encoding an audio signal, comprising the steps of: - core encoding (110) first audio data for a first spectral band (180) of the audio signal to obtain core encoded audio data (160) for the first spectral band (180) of the audio signal; Parametrically encoding (120) second audio data in a second spectral band (190) of the audio signal, the second spectral band (180) being different from the first spectral band (180) of the audio signal, the parametric encoding (120) comprising: - analysing (121) the first audio data in the first spectral band (180) of the audio signal to obtain a first analysis result (122) and analysing (121) the second audio data in the second spectral band (190) of the audio signal to obtain a second analysis result (123); calculating (124) a compensation value (125) using the first analysis result (122) and the second analysis result (123); said parametric encoding (120) step including calculating (126) parameters (170) from said second audio data in said second spectral band (190) of said audio signal using said compensation values ​​(125); outputting an encoded audio signal (150) comprising the core encoded audio data (160) in the first spectral band (180) of the audio signal and the parameters (170) representative of the second audio data in the second spectral band (190) of the audio signal; Including, the analyzing (121) step further comprises calculating a tonal-to-noise ratio (400) of the second audio data in the second spectral band (190) of the audio signal, and the calculating (124) step comprises calculating the compensation value (125) as a function of the tonal-to-noise ratio (400) of the second audio data, such that a first compensation value for a first tone-to-noise ratio is obtained, or a second compensation value for a second tone-to-noise ratio is obtained, the first compensation value being greater than the second compensation value and the first tone-to-noise ratio being greater than the second tone-to-noise ratio, or 13. The method of claim 12, wherein the step of calculating (124) the compensation value (125) comprises determining the compensation value (125) by applying a model of perceptual unpleasantness (1000), the model of perceptual unpleasantness (1000) being configured to evaluate a psychoacoustic mismatch between the first audio data and the second audio data using the first analysis result (122) and the second analysis result (123) to obtain the compensation value (125).

17. 1. A system for processing an audio signal, comprising: A device for encoding an audio signal according to claim 1 to obtain an encoded audio signal (150), a decoder receiving the encoded audio signal (150) including core encoded first audio data (160) for a first spectral band (180) of the audio signal and parameters (170) representative of second audio data for a second spectral band (190) of the audio signal, the decoder is configured to perform a spectral enhancement process using the parameters (170) and the decoded first audio data for the first spectral band (180) of the audio signal to regenerate synthesized audio data for the second spectral band (190) of the audio signal.

18. 1. A method for processing an audio signal, comprising the steps of: - encoding said audio signal according to the method of claim 16 to obtain an encoded audio signal (150); receiving the encoded audio signal (150), the encoded audio signal comprising core encoded first audio data (160) for a first spectral band (180) of the audio signal and parameters (170) representative of second audio data for a second spectral band (190) of the audio signal; performing a spectral enhancement process using the parameters (170) and the decoded first audio data for the first spectral band (180) of the audio signal to regenerate synthesized audio data for the second spectral band (190) of the audio signal; A method comprising:

19. A computer program which, when executed on a computer or processor, performs the method according to claim 16 or 18.

Images