Apparatus and method for removing undesirable sound roughness

Abstract

According to one embodiment, an apparatus (100) for processing an audio input signal to obtain an audio output signal is provided. The apparatus (100) comprises a signal analyzer (110) configured to determine information regarding the roughness of one or more spectral bands of the audio input signal. Furthermore, the apparatus (100) comprises a signal processor (120) configured to process the audio input signal depending on the information regarding the roughness of the one or more spectral bands.

Description

[Technical Field]

[0001] The present invention relates to an apparatus and method for removing undesirable audio roughness. [Background technology]

[0002] In perceptual audio coding at very low bitrates, modulation artifacts are sometimes introduced into audio signals containing distinct tonal components. Such modulation artifacts are often perceived as sonic roughness. This can be due to quantization errors or audio bandwidth expansion that causes irregular harmonic structures at the edges of replicated bands. Roughness artifacts, particularly those resulting from quantization errors, are difficult to overcome without allocating a considerable number of bits to encoding the tonal components.

[0003] Low-bitrate audio encoding employs highly efficient representations of audio signals that require significantly less digital information compared to raw, uncompressed 16-bit sampled PCM audio signals. In modern transcoders like xHE-AAC and MPEG-H, efficiency is partially achieved by converting the raw input audio signal to a time-frequency domain representation using MDCT, where each audio frame can be monitored by a psychoacoustic model and represented with variable precision constrained by the available bit budget. By applying both control mechanisms in the encoding process, the result is an audio bitstream where quantization noise varies across time frames and frequency bands.

[0004] In an ideal scenario, the encoder shapes quantization noise so that it becomes inaudible due to auditory masking. However, at very low bitrates, quantization noise becomes audible at some point, especially if long-duration pure-tone components are present in the audio signal. This is because quantizing these pure-tone components can alter their amplitude across audio frames, causing audible amplitude modulation. If the audio frame rate of a typical transcoder is 43 Hz, these modulations are added to the signal at up to half this rate. This is lower than the modulation rate that causes roughness perception, but within the range that causes (slow) r-roughness. Furthermore, due to the short-window processing used to transpose time-domain audio frames to the frequency domain, perfectly stationary pure-tone components are represented within the range of adjacent frequency bins, some of which tend to be quantized to zero, especially at very low bitrates.

[0005] Lowering the bitrate below the range required for good audio quality for a pure conversion coder is possible by using Spectral Band Replication [1] used in xHE-AAC, Intelligent Gap Filling [2] used with SBR or MPEG-H, or additional semi-parametric techniques such as IGF. High-frequency components are reconstructed using shifted copies of the low-frequency spectrum and spectral envelope shaping. Good audio quality can be maintained using SBR or IGF, respectively.

[0006] However, since pure-tone frequency components are copied along with already existing time modulation, SBR and IGF can amplify roughness artifacts.

[0007] In addition, these techniques can introduce new roughness artifacts, particularly in the transition regions between replicated bands, and many audio frames may exhibit deviations from the regular harmonic grid present in the original signal. Recent studies have shown that adaptively determining the best replication mapping using psychoacoustic models can lead to improvements in audio quality.[5]

[0008] Post-filtering approaches to suppress noise in timbral signals partially remove signal coarseness. These approaches rely on measuring the fundamental frequency and removing noise through the application of a comb filter tuned to that fundamental frequency, or on predictive coding such as a Long-Term Predictor (LTP). All of these approaches work only for mono-pitch signals and cannot remove noise from polyphonic or dissonant content exhibiting multiple pitches. Furthermore, these methods cannot distinguish between noise present in the original signal and noise introduced as a result of the encoding-decoding process.

[0009] Therefore, any improved concept for removing sound harshness would be highly appreciated. [Prior art documents] [Non-patent literature]

[0010] [Non-Patent Document 1] [1] Dietz, M., Liljeryd, L., Kjorling, K., and Kunz, O., “Spectral Band Replication, a Novel Approach in Audio Coding,” in Audio Engineering Society Convention 112, 2002. [Non-Patent Document 2] [2] Disch, S., Niedermeier, A., Helmrich, CR, Neukam, C., Schmidt, K., Geiger, R., Lecomte, J., Ghido, F., Nagel, F., and Edler, B., “Intelligent Gap Filling in Perceptual Transform Coding of Audio,” in Audio Engineering Society Convention 14, 11 2016.

Table 3

Fashion 4

Wood 5

[0011] The object of the present invention is to provide an improved concept for removing sound roughness. The object of the present invention is solved by the apparatus described in claim 1, the audio encoder described in claim 27, the method according to claim 38, the method according to claim 39, and the computer program described in claim 40. [Means for solving the problem]

[0012] An apparatus for processing an audio input signal and obtaining an audio output signal is provided according to one embodiment. The apparatus comprises a signal analyzer configured to determine information regarding the sound roughness of one or more spectral bands of the audio input signal. Furthermore, the apparatus comprises a signal processor configured to process the audio input signal in accordance with the information regarding the sound roughness of one or more spectral bands.

[0013] Furthermore, an audio encoder is provided according to an embodiment for encoding an initial audio signal to obtain an encoded audio signal and auxiliary information. The audio encoder includes an encoding module for encoding the initial audio signal to obtain an encoded audio signal. Furthermore, the audio encoder includes a side information generator for generating and outputting auxiliary information according to the initial audio signal and further according to the encoded audio signal. The auxiliary information includes an indication indicating one or more of a plurality of spectral bands for which roughness information of sound must be determined on the decoder side.

[0014] Furthermore, a method is provided according to an embodiment for processing an audio input signal to obtain an audio output signal. This method includes - determining information regarding the roughness of sound in one or more spectral bands of the audio input signal, and - processing the audio input signal according to the information regarding the roughness of sound in one or more spectral bands.

[0015] Furthermore, a method is provided for encoding an initial audio signal to obtain an encoded audio signal and auxiliary information. This method includes - encoding the initial audio signal to obtain an encoded audio signal, and - generating and outputting auxiliary information according to the initial audio signal and further according to the encoded audio signal.

[0016] The auxiliary information includes an indication indicating one or more of a plurality of spectral bands for which roughness information of sound must be determined on the decoder side.

[0017] Furthermore, a computer program is provided, each of which is configured to implement one of the methods described above when executed on a computer or a signal processor.

[0018] In particular, the present invention is based on the discovery that quantization error-induced roughness artifacts are particularly difficult to reduce without expending a significant number of bits in encoding the tonal components. Embodiments provide a novel inventive concept for removing these roughness artifacts on the decoder side, controlled by a small amount of guidance information transmitted by the encoder.

[0019] Some embodiments are based on the discovery that, on a per-frame basis, it is very difficult to see amplitude modulation occurring over successive frames, but the human auditory system will still perceive them as roughness artifacts because it evaluates the audio signal over a time span longer than the typical frame lengths used in audio coding. In some embodiments, the decoded audio signal may be analyzed, for example, with a longer frame length, which will make the amplitude modulation artifacts present in the tonal components more prominent in the amplitude spectrum as sidebands appearing next to the primary tonal components, or even as side peaks.

[0020] In view of the appearance of such side peaks, in principle, it would be possible to detect and remove these side peaks from the spectrum. Initial experiments have shown that this can actually be done and, as a result, the roughness artifacts are significantly reduced.

[0021] However, arbitrarily removing such side peaks can introduce undesirable perceptual changes in the audio signal. For example, consider an original audio signal that contains a very rough signal portion itself. In this case, the roughness should not be removed. In fact, arbitrarily applying side peak removal has been found to cause a clearly audible "tubiness" artifact in sections of the audio signal having a spectrum similar to noise or densely packed.

[0022] To overcome the above problems, it appears that side peak removal needs to be performed selectively, that is, only in the portions of the audio signal where the encoding and decoding processes cause roughness artifacts. Since this decision relates to the perception of such artifacts, such a decision can be made by a psychoacoustic model that compares the original signal with the decoded signal to determine in which time-frequency domains roughness artifacts are caused.

[0023] To remove the aforementioned roughness artifact, a method is provided that uses an amplitude-modulation-sensitive psychoacoustic model. This model is based on the model of Dau et al. [3] but includes many modifications already described in [4], which will be detailed later. The decision that the psychoacoustic model makes about whether the roughness artifact should be removed may, for example, require access to the original signal and therefore must be made on the encoder side of the audio encoding / decoding chain. This means that auxiliary information must be sent from the encoder to the decoder. This increases the bitrate, but the increment has been found to be very small and can be easily taken out of the bit budget of the conversion coder.

[0024] The embodiment removes roughness artifacts on the decoder side, which is controlled by a small amount of guidance information transmitted from the encoder in the bitstream.

[0025] The embodiment provides a concept for eliminating sound roughness.

[0026] Some of these embodiments reduce or eliminate roughness artifacts on the decoder side based on the concept that modulation of pure-tone components generates spectral side peaks next to the first pure tone. These side peaks may be better observed, for example, when spectral analysis is based on a long time window. In some specific embodiments, the analysis window may be extended beyond, for example, the length of a typical encoding frame.

[0027] In principle, spectral side peaks can be removed from the spectrum, and in this way, roughness artifacts are also removed. This algorithm can, for example, select side peaks that need to be removed based on their spectral proximity to stronger first-order pure-tone components. When such roughness removal is applied indiscriminately to an audio signal, it also removes roughness that was present in the original audio signal.

[0028] In one embodiment, a psychoacoustic model analyzes the spectral time intervals at which roughness is introduced by a low-bitrate codec. The spectral time intervals at which roughness should be removed are then signaled in an auxiliary portion of the bitstream and transmitted to the decoder.

[0029] According to one embodiment, the postprocessor of the decoder supplied by the bitstream may include, for example, small guidance information for controlling de-roughness.

[0030] In another embodiment, guidance information may be inferred, for example, on the decoder side.

[0031] Next, embodiments of the present invention will be described in more detail with reference to the figures. [Brief explanation of the drawing]

[0032] [Figure 1] This figure illustrates an example of a device for processing an audio input signal and obtaining an audio output signal, according to one embodiment. [Figure 2] This figure illustrates an apparatus for generating an audio output signal, comprising an audio decoder and the processing apparatus shown in Figure 1. [Figure 3] This figure illustrates an audio encoder according to one embodiment, for encoding an initial audio signal and obtaining an encoded audio signal and auxiliary information. [Figure 4] This figure illustrates a system comprising the audio encoder shown in Figure 3 and the device shown in Figure 2 for generating an audio output signal from an encoded audio signal, according to one embodiment. [Figure 5] This figure illustrates an overview of the entire processing chain for roughness reduction according to one embodiment. [Figure 6] This figure illustrates an overview of the encoder processing for roughness reduction (RR) according to one embodiment. [Figure 7] This figure illustrates an overview of the decoder processing for roughness reduction according to one embodiment. [Figure 8] This figure illustrates a detailed diagram of a sparsification process according to one embodiment. [Figure 9] This figure illustrates an overview of the frame-by-frame processing of a roughness removal decoder algorithm according to one embodiment. [Figure 10] This figure illustrates, in blue, unsmoothed amplitude spectrum samples alongside smoothed amplitude spectra. [Figure 11] This diagram illustrates a psychoacoustic model consisting of a basilar membrane filterbank, a haircell model, an adaptive loop, and a modulation filterbank. [Figure 12] This figure illustrates the results of the first set of items, consisting of stereo signals from a listening test using the Web-MUSHRA tool. [Figure 13] This figure illustrates the results of the second set of items, consisting of monaural signals from a listening test using the Web-MUSHRA tool. [Modes for carrying out the invention]

[0033] Figure 1 illustrates an example of a device 100 for processing an audio input signal and obtaining an audio output signal, according to one embodiment.

[0034] The device 100 includes a signal analyzer 110 configured to determine information regarding the sound roughness of one or more spectral bands of an audio input signal.

[0035] Furthermore, the device 100 includes a signal processor 120 configured to process the audio input signal in accordance with information regarding the coarseness of sound in one or more spectral bands.

[0036] According to one embodiment, the coarseness of sound in one or more spectral bands of the audio input signal may depend, for example, on encoding errors introduced by encoding the original audio signal to obtain an encoded audio signal, and / or by decoding the encoded audio signal to obtain an audio input signal.

[0037] In one embodiment, the signal analyzer 110 is configured to determine a plurality of pure-tone components within one or more spectral bands. The signal analyzer 110 may be configured, for example, to select one or more pure-tone components from the plurality of pure-tone components based on the spectral proximity from one of the plurality of pure-tone components to the other of the plurality of pure-tone components. Furthermore, the signal processor 120 may be configured, for example, to remove and / or attenuate and / or modify one or more pure-tone components.

[0038] For example, a processor might modify the spectral neighborhood of a removed or attenuated peak to preserve bandwidth energy after peak manipulation, or shift the remaining principal peak to preserve the local spectral centroid. This requires applying complex coefficients to the spectral neighborhood.

[0039] According to one embodiment, the signal analyzer 110 may be configured to receive, for example, a bitstream containing steering information. Furthermore, the signal analyzer 110 may be configured to select one or more pure tonal components from a group of pure tonal components in accordance with the steering information.

[0040] In one embodiment, the steering information may be represented, for example, in a first time-frequency domain or a first frequency domain, and the steering information has a first spectral resolution. The signal analyzer 110 may be configured to determine, for example, a plurality of pure tone components in a second time-frequency domain having a second spectral resolution, the second spectral resolution being different from the first spectral resolution. In one embodiment, the second spectral resolution may be, for example, coarser than the first spectral resolution. In another embodiment, the second spectral resolution may be, for example, finer than the first spectral resolution.

[0041] According to one embodiment, the signal processor 120 may be configured to remove and / or attenuate and / or modify one or more pure tone components, for example, by employing time smoothing or by employing time attenuation.

[0042] In one embodiment, the signal processor 120 may be configured to process an audio input signal by, for example, removing or attenuating one or more side peaks from the amplitude spectrum of the audio input signal, each of which one or more side peaks may be a local peak in the amplitude spectrum, for example, located within a predefined frequency distance from another local peak in the amplitude spectrum and having a smaller amplitude than the other local peak.

[0043] According to one embodiment, the signal analyzer 110 may be configured to determine, for example, a plurality of local peaks in the initial amplitude spectrum of one or more spectral bands of an audio input signal in order to obtain information about sound roughness.

[0044] In one embodiment, a plurality of local peaks constitute a first group of a plurality of local peaks. The signal analyzer 110 may be configured, for example, to smooth the initial amplitude spectrum of one or more spectral bands in order to obtain a smoothed amplitude spectrum. Furthermore, the signal analyzer 110 may be configured, for example, to determine a second group of one or more local peaks in the smoothed amplitude spectrum. Furthermore, the signal analyzer 110 may be configured, for example, to determine a third group of one or more local peaks, which includes all the local peaks of the first group of plurality of local peaks that do not have corresponding peaks in the second group of local peaks, as information regarding sound roughness, such that the third group of one or more local peaks does not include any local peaks from the second group of one or more local peaks.

[0045] According to one embodiment, the signal analyzer 110 may be configured to determine, for example, for each peak of a plurality of peaks in a first group, whether the second group includes a peak associated with the peak, such that, for example, a peak of the second group located at the same frequency as the peak is associated with the peak, and a peak of the second group located within a predefined frequency distance from the peak is associated with the peak, and a peak of the second group located outside the predefined frequency distance from the peak cannot be associated with the peak.

[0046] In one embodiment, the signal processor 120 may be configured to process an audio input signal by, for example, removing or attenuating one or more local peaks of a third group in the initial amplitude spectrum of one or more spectral bands in order to obtain the amplitude spectrum of one or more spectral bands of the audio output signal.

[0047] According to one embodiment, in order to remove or attenuate each of the one or more side peaks or one or more local peaks of the third group, the signal processor 120 may be configured, for example, to attenuate the peak and the surrounding region of the peak.

[0048] In one embodiment, the signal processor 120 may be configured to determine, for example, the peripheral region of the peak such that the minimum value immediately preceding the peak and the minimum value immediately following the peak limit the peripheral region.

[0049] According to one embodiment, the frequency spectrum of an audio input signal includes a plurality of spectral bands. Furthermore, the signal analyzer 110 may be configured to receive or determine one or more spectral bands from the plurality of spectral bands, for example, from which information regarding sound roughness must be determined. Furthermore, the signal analyzer 110 may be configured to determine information regarding sound roughness for the one or more spectral bands of the audio input signal. Furthermore, the signal analyzer 110 may be configured not to determine information regarding sound roughness for any other spectral bands from the plurality of spectral bands of the audio input signal.

[0050] In one embodiment, the signal analyzer 110 may be configured to receive, for example, information on one or more spectral bands from the encoder side on which information regarding sound coarseness must be determined.

[0051] According to one embodiment, the signal analyzer 110 may be configured to receive information about one or more spectral bands, for example, as a binary mask or a compressed binary mask, on which information about the coarseness of the sound must be determined.

[0052] According to one embodiment, the device 100 may be configured to receive, for example, a selection filter. The signal analyzer 110 may be configured to determine, for example, one or more spectral bands out of a plurality of spectral bands, in which information regarding sound roughness must be determined, depending on the selection filter.

[0053] According to one embodiment, the signal analyzer 110 may be configured to determine, for example, one or more spectral bands out of a plurality of spectral bands from which information regarding sound coarseness must be determined.

[0054] In one embodiment, the signal analyzer 110 may be configured to determine one or more spectral bands among a plurality of spectral bands for which information regarding sound roughness must be determined, without the signal analyzer 110 receiving side information indicating the information regarding one or more spectral bands for which information regarding sound roughness must be determined.

[0055] According to one embodiment, the signal analyzer 110 may be configured to determine one or more spectral bands out of a plurality of spectral bands on which information regarding sound roughness must be determined, for example by employing the concept of artificial intelligence.

[0056] In one embodiment, the signal analyzer 110 may be configured to determine one or more spectral bands out of a plurality of spectral bands on which information about sound coarseness must be determined, for example by employing a neural network as the concept of artificial intelligence employed by the signal analyzer 110. The neural network may be, for example, a convolutional neural network.

[0057] According to one embodiment, the signal analyzer 110 may be configured not to use information about sound roughness for a spectral band among a plurality of spectral bands that include one or more transients (for example, in a filter that removes roughness peaks). For example, in this algorithm, the filter cannot be simply applied, for example, during the duration of a frame that includes transients.

[0058] Figure 2 illustrates an example of a device 200 for generating an audio output signal from an encoded audio signal, according to one embodiment.

[0059] The device 200 in Figure 2 includes an audio decoder 210 configured to decode an encoded audio signal and obtain a decoded audio signal.

[0060] Furthermore, the apparatus 200 in Figure 2 further includes the apparatus 100 for processing shown in Figure 1.

[0061] The audio decoder 210 is configured to supply the decoded audio signal to the device 100 as an audio input signal for processing.

[0062] The processing device 100 is configured to process the decoded audio signal and obtain an audio output signal.

[0063] According to one embodiment, the audio decoder 210 may be configured to decode an encoded audio signal using, for example, processing for each first time block having a first frame length.

[0064] The signal analyzer 110 of the processing apparatus 100 may be configured to determine information regarding sound roughness using, for example, processing for each second time block having a second frame length, the second frame length may be, for example, longer than the first frame length.

[0065] In one embodiment, the audio decoder 210 may be configured, for example, to decode an encoded audio signal to obtain a decoded audio signal which is a mid-side signal including a mid-channel and a side channel. The processing apparatus 100 may be configured, for example, to process the mid-side signal to obtain an audio output signal of the processing apparatus 100. The generation apparatus 200 may further include, for example, a conversion module that converts the audio output signal so that, after conversion, the audio output signal includes the left and right channels of a stereo signal.

[0066] Figure 3 illustrates an audio encoder 300 according to one embodiment, for encoding an initial audio signal and obtaining an encoded audio signal and auxiliary information.

[0067] The audio encoder 300 includes an encoding module 310 for encoding an initial audio signal and obtaining an encoded audio signal.

[0068] Furthermore, the audio encoder 300 includes a side information generator 320 for generating and outputting auxiliary information in accordance with the initial audio signal and, further, in accordance with the encoded audio signal.

[0069] The auxiliary information includes an indication of one or more spectral bands out of several spectral bands, from which the decoder must determine the sound coarseness.

[0070] According to one embodiment, the side information generator 320 may be configured to generate additional information in accordance with, for example, a perceptual analysis model or a psychoacoustic model.

[0071] In one embodiment, the side information generator 320 may be configured to estimate perceived changes in sound roughness in the encoded audio signal, for example, using a perceptual analysis model or a psychoacoustic model.

[0072] According to one embodiment, the side information generator 320 may be configured to generate a binary mask that, for example, indicates an increase in coarseness, and which shows one or more spectral bands out of a plurality of spectral bands, and which indicates an increase in coarseness, and for which information regarding sound coarseness must be determined on the decoder side.

[0073] In one embodiment, the side information generator 320 may be configured, for example, to generate a binary mask as a compressed binary mask.

[0074] According to one embodiment, the side information generator 320 may be configured to generate auxiliary information by, for example, employing time modulation processing.

[0075] In one embodiment, the side information generator 320 may be configured to generate auxiliary information, for example, by generating a selection filter.

[0076] According to one embodiment, the side information generator 320 may be configured to generate a selection filter by, for example, employing time smoothing.

[0077] In one embodiment, the side information generator 320 may be configured to generate auxiliary information indicating one or more spectral bands out of a plurality of spectral bands, for example by employing a neural network, which must determine information regarding sound coarseness on the decoder side. The neural network may be, for example, a convolutional neural network.

[0078] Figure 4 illustrates a system according to one embodiment.

[0079] The system includes an audio encoder 300 (Figure 3) to encode the initial audio signal and obtain the encoded audio signal and auxiliary information.

[0080] Furthermore, the system includes the device 200 shown in Figure 2 for generating an audio output signal from the encoded audio signal.

[0081] The device 200 for generating an audio output signal is configured to generate an audio output signal according to an encoded audio signal and according to auxiliary information.

[0082] Several embodiments of the present invention are described below.

[0083] Figure 5 illustrates an overview of the entire processing chain for roughness reduction (RR) according to one embodiment. The green blocks represent the roughness reduction of the present invention, and the blue blocks relate to processing blocks that are typically present in an audio codec.

[0084] Figure 6 illustrates an overview of roughness reduction (RR) encoder processing according to one embodiment. In the encoder, the roughness reduction encoder unit uses a perceptual analysis (PA) model to compare the original PCM signal with the encoded and re-encoded signals. To make this method work, the use of an advanced modulation-based psychoacoustic model is a good choice. The PA model estimates the perceived changes in the sonic roughness of the signal and derives a binary mask that shows the spectral bands that present increased roughness. This binary mask is compressed and added to the perceptual coder's bitstream as side information. Experiments have shown that this side information requires only about 0.4 kbps of additional bitrate for mono and stereo signals. A sketch of the signal flow is shown in Figure 6.

[0085] Figure 7 illustrates an overview of the decoder processing for roughness reduction (RR) according to one embodiment. In the decoder, the roughness reduction decoder unit extracts side information from the bitstream and supplies it to a processing block labeled "sparsification." This block removes unwanted pure-tone side peaks in the bandwidth indicated by the binary mask as having increased roughness. The signal flow is shown in Figure 7. For stereo signals, sparsification is performed using M / S representation to avoid perceived spatial variations.

[0086] Figure 8 illustrates a detailed diagram of the "sparsification" process according to one embodiment.

[0087] Embodiments of the present invention are described in more detail below.

[0088] First, the concept of guided auditory roughness removal for audio codecs according to the embodiment will be explained.

[0089] In particular, roughness reduction (RR) algorithms are described. In some of these embodiments, it may be necessary to extract auxiliary information on the encoder side in order to steer the roughness reduction performed after the audio signal has been decoded.

[0090] Returning to Figure 5, a schema is illustrated showing how a standard audio encoder and decoder are connected to an RR encoder that sends auxiliary information within the RR bitstream to the RR decoder. In particular, Figure 5 illustrates an overview of the application context of a deroughness codec, which is built around a conventional audio encoder-decoder pair (shown in blue).

[0091] To explain how it is used, the core of the algorithm is first described, where the spectral components are modified (on the RR decoder side) to remove roughness, and then the psychoacoustic model proceeds to how to select the portion of the signal (on the RR encoder side) into which the roughness artifact is introduced.

[0092] The following section provides a more detailed explanation of roughness removal.

[0093] Figure 9 illustrates an overview of the frame-by-frame processing of a roughness removal decoder algorithm according to one embodiment. Time-domain frames and auxiliary information are used as input. A time-domain output frame is generated from which spectral components causing roughness artifacts have been removed.

[0094] The roughness removal decoder operates frame by frame. An overview of the processing within each frame is shown in Figure 9. As can be seen, the time frame is converted to a spectral representation. In principle, the only processing performed on this spectrum is to apply an attenuation filter (H) to the spectrum and then convert it back to a time-domain frame. Filter H should be designed so that spectral peaks that cause roughness artifacts are attenuated.

[0095] To derive the attenuation filter, two separate filters are first derived, which can be seen at the two lower branches in Figure 9. First, based on the signal spectrum, the algorithm determines all peaks associated with roughness. Based on these specific peaks, an attenuation mask H with high spectral resolution is created. sThe following is derived. This attenuation mask will simply remove all peaks that cause roughness, including peaks that were present in the original encoded signal. For this reason, the auxiliary information obtained by the roughness removal encoder is picked up to determine the spectral bands in which the audio encoding algorithm introduces perceptible roughness artifacts. For these spectral bands, a second attenuation mask is derived that has a low gain for the bands with perceptible roughness artifacts (H a ). Perceptual models only lead to yes-no decisions, H a It was found to be beneficial to apply a low-pass filter to the output. Then both attenuation filters are combined into a single attenuation filter H. The output of that filter is H in the next frame. a It is used as a precedent state for the low-pass filter applied to it. It is the attenuation H of the previous frame. s This means that it will continue to be effective in the current frame.

[0096] Since r-roughness and coarseness are associated with amplitude modulation, audio components that sound rough are represented by a principal spectral peak where adjacent side peaks can be separated at a low frequency of 10 Hz. A sufficiently long analysis window must be used so that such side peaks can be observed. In the algorithm presented herein, an analysis window of 5644 samples at 44.1 kHz, or a sample length adapted according to the sampling frequency, was used.

[0097] The following describes the process of finding roughness peaks, with reference to Figure 9. Many methods can be considered to remove side peaks representing the introduced r-roughness artifact. Here, a method is provided to consider how the roughness artifact is introduced. First, all local peaks are selected within the spectra obtained from 5644 sample intervals,

[0098]

number

[0099] This is shown in Figure 3. In Figure 3, the spectrum is shown in blue, and the peaks are indicated by blue circles. (Note that many small peaks with low amplitude appear.) Secondly, the amplitude spectrum is smoothed in a 10-sample length Han window (shown in red), and the red circles indicate the peaks found. In this smoothed spectrum, it is thought that most of the side peaks resulting from the encoding process are removed, as seen in the leftmost peak of sample number 620, and the prominent side peaks in the unsmoothed spectrum (blue) are no longer present in the smoothed spectrum (red). In this smoothed spectrum, all local peaks are again selected,

[0100]

number

[0101] It is represented as follows.

[0102] In principle, the side peaks that are removed are

[0103]

number

[0104] Inspect and

[0105]

number

[0106] This can be determined by determining which elements are not found in the original spectrum.

[0107]

number

[0108] The strong peaks (which are elements in the spectrum) are smoothed in the spectrum (the peaks are

[0109]

number

[0110] It should be noted that the spectral configuration may not be exactly the same in the (represented by) . When the surrounding spectrum is tilted, this can cause a bias in the position of the dominant peak after smoothing. For such reasons, first,

[0111]

number

[0112] What components are present in?

[0113]

number

[0114] A mapping is derived showing whether the peaks still exist, albeit shifted in their spectral position. The remaining peaks are classified as side peaks that need to be removed.

[0115]

number

[0116] It is written as follows.

[0117] These side peaks

[0118]

number

[0119] As shown in s , to remove this, first, the surrounding spectral range is selected for each peak to be removed. This range is delimited by the first minimum found on either side of the peak in the unsmoothed spectrum. Within this range, a 20 dB attenuation is then inserted into the frequency domain filter H s that initially has unity gain. This procedure is repeated for each peak to be removed. As described above, this filter H

[0120] cannot be applied directly to the spectrum because it would also remove the peaks that already exist in the original signal and cause roughness. a For this reason, a second filter H s should be used as a selection filter that determines which regions within the side peak removal filter H H = 1-(1 - H s )(1 - H a )(1) are actually to be applied for filtering, based on auxiliary information from the encoder side. This selection is obtained via the equation

[0121] The effect of this combination is that both H s and H a should cause attenuation to result in attenuation in the new filter H. This new attenuation filter H can then be applied to the spectrum to remove the side peaks that cause roughness introduced by the encoding process, but it has been found that this causes some perceptible instability in the audio excerpt. This can be due to the uncertainty of the encoder-side decision process regarding which bands contain roughness artifacts. In addition, the decision on the encoder side is an all-or-nothing decision motivated by keeping the bit rate for transmitting the auxiliary information significantly restricted. To reduce the instability, the filter H aTime smoothing is applied. To do this, the filter H obtained in the previous frame is replaced with a newly calculated filter H having coefficients of 0.4 and 0.6, respectively. a It is combined with this.

[0122] Figure 10 illustrates unsmoothed amplitude spectrum samples in blue, along with smoothed amplitude spectra shown in red. The corresponding colored circles represent local peaks in the spectrum.

[0123] In Figure 10, the attenuation filter is applied to the original spectrum (blue), and as a result, the green curve is visible only in the spectral region where a significant attenuation occurred. Therefore, around sample 620, where there was a peak in the original spectrum (blue) but no peak in the smoothed spectrum (red), the peak in the blue spectrum is significantly attenuated, demonstrating that this method reduces potential audible modulation artifacts.

[0124] The following describes a psychoacoustic model for steering roughness removal.

[0125] As mentioned in the previous section, roughness-induced side peaks should only be removed when they arise as a result of the audio encoding process. This information can only be obtained on the encoder side, for example, by accessing the original signal. This section explains how psychoacoustic models capable of detecting roughness in audio signals are used for this purpose.

[0126] The psychoacoustic model used for this purpose has previously been used to steer encoding decisions in parametric audio encoders[5] and has since been shown to be very suitable for making predictions about perceived degradation resulting from various audio encoding methods[4]. This model is an extension of Dau et al.'s model[3] and assumes that for each auditory filter channel, a modulation filter bank provides an analysis of the audio signal with respect to time modulation.

[0127] This model is schematically shown in Figure 11. In particular, Figure 11 illustrates a psychoacoustic model consisting of a basilar membrane filter bank, a hair cell model, an adaptive loop, and a modulation filter bank, following Dau et al. [3].

[0128] First, the audio signal is processed by a number of parallel gammatone filters having bandpass characteristics that approximate frequency selection processing in the human cochlea, and the gammatone filter bank provides a complex numerical output from which the amplitude is extracted, thus effectively extracting the Hilbert envelope of the gammatone output, except that this is consistent with the original model by Dau et al. [3] and previous publications [4], [5]. This modification is included because of its interaction with the adaptive loop, which is the next stage of the model to be explained when describing the adaptive loop.

[0129] Adaptive loops are included in the Dau model to model the adaptive process in the auditory pathway (e.g., the auditory nerve). Each adaptive loop is modeled as a decay step, and the decay coefficient is a low-pass filtered version of the loop's output. As a result, the adaptive loops have a reduced gain that persists after the signal start and after the offset of the input signal. This property is used to model the forward masking effect observed in listening tests. A total of five adaptive loops are proposed in the Dau model, each with a different time constant. In a steady state, i.e., after a long time has elapsed since the start, it can be shown that the adaptive loops approximate the shape of a logarithmic transformation.

[0130] At the start of a signal, the adaptive loop does not yet have the reduced gain that is found toward a steady state, thus causing a significant overshoot that results in disproportionate sensitivity to any changes made to the start of a signal that do not conform to psychoacoustic observations. For this reason, the maximum gain of the adaptive loop was made to depend on the input level according to the logarithmic law.

[0131] For very low frequency signals (<100Hz), the time constant of the adaptive loop allows for some degree of attenuation between the two periods. This effectively reduces the average attenuation and thus increases the overall sensitivity to any changes in the input signal at low frequencies. For this reason, a Hilbert envelope is extracted before the adaptive loop. This Hilbert envelope replaces the haircell processing used in the original Dow model, which consisted of half-wave rectification followed by a low-pass filter.

[0132] After the adaptive loop for each auditory channel, the output is fed to a modulation filter bank, which is comparable to the filter bank proposed by Dau et al. and has an additional step of removing the DC component from the filter (see [4]). This is important because the DC component of the Hilbert envelope can be considerably higher than the modulation component. Due to the shallow filter geometry of the modulation filter, the modulation filter output can be dominated by the DC component (see [5]). This characteristic is not so important in Dau et al.'s original model, as that model only dealt with significant differences in stimuli, but in the current setting, it is interesting to know whether a strong baseline modulation already exists in the original audio signal. When this is the case, listening tests have shown that the added modulation is difficult to detect. The presence of a strong DC component in the output of the modulation filter makes it difficult to obtain the baseline modulation.

[0133] Finally, the output of the modulation filter bank results in an internal representation that is a function of time t, auditory filter number k, and modulation filter number m, and is dependent on the input signal x. The internal representation is processed to determine whether a significant additional modulation is introduced within the modulation frequency range associated with coarseness. For this purpose, the ratio between the increase in modulation intensity in the modulation filters centered from 5 Hz to 35 Hz and the baseline modulation intensity of the original audio signal in the same filters is calculated.

[0134] This method determines the relative increase in modulation intensity. When this exceeds a threshold of 0.6, the corresponding time and frequency interval is signaled to the encoder as an interval where side peaks need to be removed. In the standard configuration of the algorithm, the value is also averaged across two adjacent bands to reduce the bitrate for side information. However, in listening tests, an additional condition is added where this averaging across adjacent elements is omitted to investigate its impact on quality.

[0135] The following describes the characteristics of the roughness removal encoder and / or decoder.

[0136] As shown in Figure 5, the de-roughness algorithm is built around a typical encoder-decoder combination; that is, the algorithm can be applied independently of the codec, but may also be integrated with the codec. On the encoder side, the audio signal is first encoded, and the resulting bitstream is sent to the decoder side.

[0137] The deroughness encoder takes in the original input signal and bitstream and directly decodes the audio signal again. Using the psychoacoustic model outlined in the previous section, the decoder determines which time-frequency intervals are subject to the deroughness algorithm outlined in Sect. 2.1. This decision is based on a mono downmix of the input signal if the input signal is stereo, further limiting the relatively high bitrate required for this method.

[0138] The auxiliary information (RR bitstream) is sent to the roughness removal decoder, which uses the decoded signal available to remove the roughness-causing side peaks from the appropriate signal portions.

[0139] Removing side peaks within frames containing transients was found to cause significant pre-echo.

[0140] This is caused by narrowband spectral correction performed along with side-peak rejection. To avoid the introduction of pre-echoes, the decoder's transient detector signals frames where side-peak rejection should not be performed. Note that the filter calculation for side-peak rejection continues even in such transient frames; it is simply not applied to the signal.

[0141] In principle, for stereo signals, the de-roughness algorithm could be applied independently to both channels.

[0142] In some cases, it was considered beneficial to first convert the stereo signal to a mid-side representation and then apply this algorithm twice, independently to both the mid-channel and the side channel.

[0143] In the listening test, both options are evaluated. In the encoding process, the relatively slow frame rate and the fact that frames are divided into 2822 samples at a sampling frequency of 44.1 kHz (15.6 Hz) are beneficial. In addition, the standard setting provides supplementary information grouped together for 21 pairs of 42 bandwidths.

[0144] The auxiliary information, consisting of a single bit for each decision, is grouped into six auditory bands and stored as a single number in a Huffman encoder to utilize possible correlations between bands with frequencies close to each other. An average bitrate of 0.30 kbit / s is obtained for items used in a listening test when decisions are transmitted band by band pair, and an average bitrate of 0.65 bit / s is obtained when information for a single band is transmitted.

[0145] Informal listening experiments have been conducted. These experiments evaluate the quality gains that can be obtained by employing the concepts described above in the embodiments. In particular, the listening tests show a clear improvement in sound quality for items encoded in stereo at approximately 14kbps using waveform and parametric coders. In addition, items encoded in 32kbps mono with a pure waveform coder also show improvement when the proposed algorithm is applied. In both cases, the quality improvement is attributed to the removal of roughness artifacts.

[0146] To investigate whether the proposed method actually resulted in improved sound quality, a MUSHRA listing test was conducted. Two different sets of items were used for listening: the first set consisted of items encoded in stereo, and the second set consisted of items encoded in mono. Most of the stereo items were encoded using an experimental waveform encoder, which encoded the left and right ear signals independently, each at a bitrate of 32 kbit / s.

[0147] In addition, one item was encoded using an IGF-based method. The second set of items was also encoded entirely using an IGF-based method. A summary of these items is provided in Table 1.

[0148] [Table 1]

[0149] Within the algorithm, in addition to including mid-side coding (default), there is also an option to encode the signals from the left and right ears independently. For this reason, both options were included in the MUSHRA test in the first set of items. Furthermore, auxiliary information can be transmitted for each pair of auditory bands (default) or independently for each auditory band. These two options were included in the second set of items. All measurement conditions are listed in Table 2.

[0150] [Table 2]

[0151] The hidden reference is the original audio signal, the anchor is the 3.5kHz low-pass filtered version of the original signal, the unprocessed decoded signal represents the signal without roughness reduction, and RR refers to various conditions under which a roughness reduction algorithm is applied, either through mid-side processing, independent left / right processing, or the use of two or a single bandwidth for each bit of auxiliary information.

[0152] A total of N participants took part in the listening test. The listening test was conducted in a home office using the Web-MUSHRA tool and high-quality headphones.

[0153] The results are shown in Figures 12 and 13.

[0154] In particular, Figure 12 illustrates the results of the first set of items, consisting of stereo signals from a listening test using the Web-MUSHRA tool.

[0155] Figure 13 illustrates the results of the second set of items, consisting of monaural signals from a listening test using the Web-MUSHRA tool.

[0156] Further embodiments are described below.

[0157] According to one embodiment, an apparatus / method (for example, post-processing) is provided which identifies, for example, pure tonal components in a (decoded) audio signal based on their spectral proximity to adjacent components, and removes or attenuates them.

[0158] In one embodiment, an apparatus / method is provided for removing or attenuating (e.g., post-processing) pure-tone components in a decoded signal that is (partially) steering by information transmitted in a bitstream.

[0159] According to one embodiment, an apparatus / method is provided that uses coarse t / f resolution information from a bitstream and finer spectral resolution information derived on the decoder side (for example, for post-processing).

[0160] In one embodiment, for example, time-block processing using a longer frame length than that used in an audio decoder may be employed.

[0161] According to one embodiment, for example, time smoothing or time decay may be employed.

[0162] In one embodiment, for example, a transient steered switching window or a skipping block with a transient in post-processing may be employed.

[0163] According to one embodiment, for example, a stereo signal using mid-side synchronization or coding may be employed.

[0164] In one embodiment, for example, time modulation processing may be employed on the encoder side based on an auditory model to determine the information in the bitstream.

[0165] According to one embodiment, for example, an additional selection filter may be employed, which is driven by a bitstream that selects a region in which pure tonal components are removed or attenuated.

[0166] In one embodiment, for example, a selection filter having a smooth transition within the spectral region may be employed.

[0167] According to one embodiment, for example, the filter may assume, for example, temporal smoothing.

[0168] While some embodiments are described within the context of the apparatus, these embodiments also serve as descriptions of the corresponding methods, and it is clear that the blocks or apparatus correspond to method steps or features of method steps. Similarly, embodiments described within the context of method steps also serve as descriptions of the corresponding blocks, items, or features of the corresponding apparatus. Some or all of the method steps may be performed by (or using) hardware devices, such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such devices.

[0169] Depending on the requirements of several implementations, embodiments of the present invention may be implemented in hardware or software, or at least partially in hardware or at least partially in software. The implementations may be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, which stores electronically readable control signals and which interacts with (or can interact with) a programmable computer system on which each method is performed. Therefore, the digital storage medium may be computer-readable.

[0170] Some embodiments of the present invention include a data carrier containing electronically readable control signals that can work in conjunction with a programmable computer system such that one of the methods described herein is performed.

[0171] Generally, embodiments of the present invention can be implemented as a computer program product with program code, the program code operable to execute one of the methods when the computer program product runs on a computer. The program code may be stored, for example, on a machine-readable carrier.

[0172] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0173] Therefore, in other words, one embodiment of the method of the present invention is a computer program having program code for performing one of the methods described herein while the computer program is running on a computer.

[0174] Accordingly, a further embodiment of the method of the present invention is a data carrier (or digital storage medium or computer-readable medium) on which a computer program for performing one of the methods described herein is recorded. The data carrier, digital storage medium, and recording medium are typically tangible and / or non-temporary.

[0175] Therefore, a further embodiment of the method of the invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals may be configured to be transmitted, for example, over a data communication network, such as the Internet.

[0176] A further embodiment includes processing means, such as a computer or a programmable logical device, configured or adapted to perform one of the methods described herein.

[0177] A further embodiment includes a computer on which a computer program for performing one of the methods described herein is installed.

[0178] A further embodiment of the present invention includes an apparatus or system configured to transfer a computer program for performing one of the methods described herein to a receiver (for example, electronically or optically). The receiver may be, for example, a computer, a mobile device, a memory device, or the like. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0179] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field-programmable gate array may work in conjunction with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0180] The apparatus described herein may be implemented using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0181] The methods described herein may be performed using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0182] The embodiments described above are merely illustrative of the principles of the present invention. Modifications and changes to the arrangements and details described herein will be obvious to those skilled in the art. Therefore, it is intended that the invention is limited only by the scope of the claims set forth below, and not by the specific details presented using the descriptions and explanations of the embodiments herein. [Explanation of Symbols]

[0183] 100 devices 110 Signal Analyzer 120 signal processors 200 equipment 210 Audio Decoder 300 Audio Encoders 310 Encoding Module 320 Side Information Generator

Claims

1. A device (100) for processing an audio input signal and obtaining an audio output signal, A signal analyzer (110) configured to determine information regarding the sound coarseness of one or more spectral bands of the audio input signal, The system includes a signal processor (120) configured to process the audio input signal according to the information relating to the sound coarseness of one or more spectral bands, The signal analyzer (110) is configured to determine a plurality of pure tone components within one or more spectral bands, The signal analyzer (110) is configured to select one or more pure tone components from among the plurality of pure tone components according to the spectral proximity from one of the plurality of pure tone components to the other of the plurality of pure tone components. The signal processor (120) is configured to remove and / or attenuate and / or modify the one or more pure tone components. Equipment (100).

2. The apparatus (100) according to claim 1, wherein the coarseness of the sound in one or more spectral bands of the audio input signal depends on encoding errors introduced by encoding the original audio signal to obtain an encoded audio signal and / or introduced by decoding the encoded audio signal to obtain the audio input signal.

3. The apparatus (100) according to claim 1, wherein the signal processor (120) is configured to remove and / or attenuate and / or modify one or more pure tone components by employing time smoothing or time attenuation.

4. A device (100) for processing an audio input signal and obtaining an audio output signal, A signal analyzer (110) configured to determine information regarding the sound coarseness of one or more spectral bands of the audio input signal, The system includes a signal processor (120) configured to process the audio input signal according to the information relating to the sound coarseness of one or more spectral bands, The signal processor (120) is configured to process the audio input signal by removing or attenuating one or more side peaks from the amplitude spectrum of the audio input signal, wherein each of the one or more side peaks is a local peak in the amplitude spectrum that is located within a predefined frequency distance from further local peaks in the amplitude spectrum and has a smaller amplitude than the further local peaks, the device (100).

5. A device (100) for processing an audio input signal and obtaining an audio output signal, A signal analyzer (110) configured to determine information regarding the sound coarseness of one or more spectral bands of the audio input signal, The system includes a signal processor (120) configured to process the audio input signal according to the information relating to the sound coarseness of one or more spectral bands, The signal analyzer (110) is configured to determine a plurality of local peaks in the initial amplitude spectrum of one or more spectral bands of the audio input signal in order to obtain the information relating to the coarseness of the sound, and the apparatus (100).

6. The aforementioned multiple local peaks are a first group of multiple local peaks, The signal analyzer (110) is configured to smooth the initial amplitude spectrum of one or more spectral bands in order to obtain a smoothed amplitude spectrum. The signal analyzer (110) is configured to determine a second group of one or more local peaks in the smoothed amplitude spectrum, The apparatus (100) according to claim 5, wherein the signal analyzer (110) is configured to determine, as information relating to the coarseness of sound, one or more third groups of local peaks, which include all local peaks of the first group of plurality of local peaks that do not have corresponding peaks in the second group of local peaks, such that the one or more third groups of local peaks does not include any local peaks of the second group of one or more local peaks.

7. The apparatus (100) according to claim 6, wherein the signal analyzer (110) is configured to determine, for each of the plurality of local peaks of the first group, whether the second group includes a peak associated with the local peak, such that a peak of the second group located at the same frequency as the local peak is associated with the local peak, a peak of the second group located within a predefined frequency distance from the local peak is associated with the local peak, and a peak of the second group located outside the predefined frequency distance from the local peak is not associated with the local peak.

8. The apparatus (100) according to claim 6, wherein the signal processor (120) is configured to process the audio input signal by removing or attenuating one or more local peaks of the third group in the initial amplitude spectrum of the one or more spectral bands in order to obtain the amplitude spectrum of the one or more spectral bands of the audio output signal.

9. The apparatus (100) according to claim 8, wherein the signal processor (120) is configured to attenuate the local peak and the surrounding region of the local peak in order to remove or attenuate each of the one or more side peaks or the one or more local peaks of the third group.

10. The apparatus (100) according to claim 9, wherein the signal processor (120) is configured to determine the peripheral region of the local peak such that the minimum value immediately preceding the local peak and the minimum value immediately following the local peak indicate the boundary of the peripheral region.

11. An apparatus (100) for processing an audio input signal and obtaining an audio output signal, A signal analyzer (110) configured to determine information regarding the sound coarseness of one or more spectral bands of the audio input signal, The system includes a signal processor (120) configured to process the audio input signal according to the information relating to the sound coarseness of one or more spectral bands, The frequency spectrum of the aforementioned audio input signal includes multiple spectral bands, The signal analyzer (110) is configured to receive or determine one or more spectral bands out of the plurality of spectral bands on which the information relating to the coarseness of the sound must be determined. The signal analyzer (110) is configured to determine the information relating to the sound roughness for one or more spectral bands of the audio input signal. The signal analyzer (110) is configured not to determine information regarding the sound roughness for any other spectral band among the plurality of spectral bands of the audio input signal. The signal analyzer (110) is configured to receive from the encoder side the information relating to one or more spectral bands on which the information relating to the coarseness of the sound must be determined, and / or The signal analyzer (110) is configured to receive the information relating to one or more spectral bands, on which the information relating to the coarseness of the sound must be determined as a binary mask or compressed binary mask, and / or The device (100) is configured to receive a selection filter, The signal analyzer (110) is configured to determine one or more spectral bands out of a plurality of spectral bands, in which the information relating to the coarseness of the sound must be determined according to the selection filter, and the apparatus (100).

12. The apparatus (100) according to claim 11, wherein the signal analyzer 110 is configured not to use the information relating to the sound roughness for any of the plurality of spectral bands that include one or more transients.

13. A device (200) for generating an audio output signal from an encoded audio signal, An audio decoder (210) configured to decode the encoded audio signal and obtain a decoded audio signal, The apparatus (100) for processing according to claim 1 comprises The audio decoder (210) is configured to supply the decoded audio signal as the audio input signal to the apparatus (100) for processing according to claim 1. The apparatus (100) for processing according to claim 1 is an apparatus (200) configured to process the decoded audio signal and obtain the audio output signal.

14. The audio decoder (210) is configured to decode the encoded audio signal using processing for each first time block having a first frame length. The apparatus (200) according to claim 13, wherein the signal analyzer (110) of the apparatus (100) for processing is configured to determine the information relating to the coarseness of the sound using processing for each second time block having a second frame length, the second frame length being longer than the first frame length.

15. The audio decoder (210) is configured to decode the encoded audio signal to obtain the decoded audio signal, which is a mid-side signal including the mid-channel and side channels. The processing device (100) is configured to process the midside signal and obtain the audio output signal of the processing device (100), The apparatus (200) for generation further comprises a conversion module that converts the audio output signal such that, after conversion, the audio output signal includes the left and right channels of a stereo signal, according to claim 13.

16. It is a system, An audio encoder (300) for encoding an initial audio signal and obtaining an encoded audio signal and auxiliary information, Apparatus (200) according to claim 13 for generating an audio output signal from an encoded audio signal, Equipped with, The apparatus (200) according to claim 13 is configured to generate the audio output signal in accordance with the encoded audio signal and the auxiliary information, system.

17. The aforementioned audio encoder (300) An encoding module (310) for encoding the initial audio signal and obtaining the encoded audio signal, The system includes a side information generator (320) for generating and outputting the auxiliary information in accordance with the initial audio signal and, further, in accordance with the encoded audio signal. The system according to claim 16, wherein the auxiliary information includes an instruction indicating one or more spectral bands among a plurality of spectral bands for which information on sound coarseness must be determined on the decoder side.

18. The system according to claim 17, wherein the side information generator (320) is configured to generate additional information in accordance with a perceptual analysis model or a psychoacoustic model.

19. The system according to claim 18, wherein the side information generator (320) is configured to estimate perceived changes in sound roughness in the encoded audio signal using the perceptual analysis model or the psychoacoustic model.

20. The system according to claim 17, wherein the side information generator (320) is configured to generate a binary mask indicating one or more spectral bands out of a plurality of spectral bands, which, as auxiliary information, indicate an increase in coarseness, and the information relating to the coarseness of the sound must be determined on the decoder side.

21. The system according to claim 20, wherein the side information generator (320) is configured to generate the binary mask as a compressed binary mask.

22. The system according to claim 17, wherein the side information generator (320) is configured to generate the auxiliary information by employing time modulation processing.

23. The system according to claim 17, wherein the side information generator (320) is configured to generate the auxiliary information by generating a selection filter.

24. The system according to claim 23, wherein the side information generator (320) is configured to generate the selection filter by employing time smoothing.

25. The system according to claim 17, wherein the side information generator (320) is configured to generate the instructions for the auxiliary information indicating one or more spectral bands among the plurality of spectral bands, the information regarding the coarseness of the sound must be determined on the decoder side by employing a neural network.

26. The system according to claim 25, wherein the neural network is a convolutional neural network.

27. A method for processing an audio input signal and obtaining an audio output signal, wherein the method is performed by an apparatus or computer, The steps include determining information regarding the sound coarseness of one or more spectral bands of the audio input signal, The steps include processing the audio input signal according to the information relating to the sound coarseness of one or more spectral bands, The method includes the step of determining a plurality of pure tone components within one or more spectral bands, the method includes the step of selecting one or more pure tone components from among the plurality of pure tone components according to the spectral proximity from one of each of the plurality of pure tone components to the other of the plurality of pure tone components, the method includes the step of removing and / or attenuating and / or modifying the one or more pure tone components, or The method includes processing the audio input signal by removing or attenuating one or more side peaks from the amplitude spectrum of the audio input signal, wherein each of the one or more side peaks is located within a predefined frequency distance from further local peaks in the amplitude spectrum and has a smaller amplitude than the further local peaks, or The frequency spectrum of the audio input signal includes a plurality of spectral bands, and the method includes the step of receiving or determining one or more spectral bands from the plurality of spectral bands for which the information relating to the coarseness of the sound must be determined, and the method includes the step of determining the information relating to the coarseness of the sound for one or more spectral bands of the audio input signal, and not determining the information relating to the coarseness of the sound for any other spectral bands from the plurality of spectral bands of the audio input signal, and the method includes the step of receiving the information relating to one or more spectral bands from the encoder side for which the information relating to the coarseness of the sound must be determined, and / or the step of receiving the information relating to one or more spectral bands for which the information relating to the coarseness of the sound must be determined as a binary mask or compressed binary mask, and / or the step of receiving a selection filter and determining one or more spectral bands from the plurality of spectral bands for which the information relating to the coarseness of the sound must be determined according to the selection filter. method.

28. The aforementioned audio input signal is an encoded audio signal. The auxiliary information includes information regarding the coarseness of the sound, To obtain the encoded audio signal and the auxiliary information, the method is: The steps include encoding an initial audio signal to obtain the encoded audio signal, The step includes generating and outputting the auxiliary information in accordance with the initial audio signal and further in accordance with the encoded audio signal, The aforementioned auxiliary information includes an indication of one or more spectral bands among a plurality of spectral bands, from which the information on sound coarseness must be determined on the decoder side. The method according to claim 27.

29. A computer program for performing the method described in claim 27 when executed on a computer or signal processor.

Images