Method and apparatus for adaptive control of decorrelation filters - Patents.com

Adaptive control of correlation separation filters addresses the inconsistency in rendering non-coherent signal components by using performance metrics, improving the quality of spatial audio rendering across varying auditory scenes.

JP7591549B2Active Publication Date: 2024-11-28TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2022205672
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2016-12-06
Filing Date
2022-12-22
Publication Date
2024-11-28
Estimated Expiration
2037-11-23

AI Technical Summary

Technical Problem

Existing spatial audio processing technologies fail to adaptively control the characteristics of correlation separation filters, leading to inconsistent rendering of non-coherent signal components across varying auditory scenes, resulting in unnatural audio environments.

Method used

Adaptive control of correlation separation filters based on performance metrics, such as reverberation length and inter-channel coherence, to match the characteristics of the input signal, using methods like Discrete Fourier Transform and all-pass filters.

Benefits of technology

Improves the rendering of spatial audio by accurately controlling reverberation and coherence, enhancing the perceived quality of decoded audio signals in different environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007591549000042
    Figure 0007591549000042
  • Figure 0007591549000043
    Figure 0007591549000043
  • Figure 0007591549000044
    Figure 0007591549000044
Patent Text Reader

Abstract

A method, an apparatus, a decorrelator, a stereo audio codec, a parametric stereo decoder and a multi-channel audio codec are provided for adaptively controlling the characteristics of a decorrelator for representing non-coherent signal components used in a multi-channel audio decoder. [Solution] An audio signal processing method for adaptively adjusting a correlation separator includes obtaining a control parameter and calculating the average value and variance of the control parameter, calculating a ratio between the variance and the average value of the control parameter and calculating a correlation separation parameter based on the ratio, and supplying the calculated correlation separation parameter to the correlation separator.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] This application relates to spatial audio coding and rendering. [Background technology]

[0002] Spatial audio or 3D audio is a general term that describes many kinds of multi-channel audio signals. An audio scene is represented by a spatial audio format depending on the capture and rendering method. General spatial audio formats defined by the capture method (microphone) are represented as, for example, stereo, binaural, ambisonics, etc. A spatial audio rendering system (headphones or loudspeakers) can render a spatial audio scene using stereo (left and right channel 2.0) or more advanced multi-channel audio signals (2.1, 5.1, 7.1, etc.).

[0003] Recent techniques for the transmission and processing of such audio signals can provide end users with an improved audio experience with higher spatial quality, often resulting in better intelligibility as well as augmented reality. Spatial audio coding techniques such as MPEG Surround Audio or MPEG-H 3D Audio generate compact representations of spatial audio signals that are suitable for data-rate constrained applications, such as streaming over the Internet. However, when data-rate constraints are strong and therefore post-processing of decoded audio channels is also used to improve spatial audio reproduction, the transmission of spatial audio signals is limited. A commonly used technique may blindly upmix, for example, decoded mono or stereo signals to multi-channel audio (5.1 channels or more).

[0004] Spatial audio coding and processing techniques exploit the spatial characteristics of multi-channel audio signals to efficiently render spatial audio scenes. In particular, the time and level differences between channels of spatial audio capture are used to approximate the interaural cues that characterize the listener's perception of directional sound in space. It is very important that the inter-channel time differences are relevant to the perceptual aspects, since they can only approximate what the hearing system can detect (i.e., the interaural time and level differences at the ear entrance). Inter-channel time and level differences (ICTD and ICLD) are generally used to model the directional components of multi-channel audio signals, while inter-channel cross-correlation (ICC), which models the interaural cross-correlation (IACC), is used to characterize the width of the sound image. Inter-channel phase difference (ICPD) may also be used to model the stereo image, especially for low frequencies.

[0005] It should be noted that the binaural cues relevant to spatial hearing are called interaural level difference (ILD), interaural time difference (ITD), and interaural coherence or correlation (IC or IACC). When considering a general multi-channel signal, the corresponding cues related to the channels are inter-channel level difference (ICLD), inter-channel time difference (ICTD), and inter-channel coherence or correlation (ICC). Since spatial audio processing operates primarily on the captured audio channels, the terms ITD, ILD, and IC are often used, with the "C" omitted, when referring to the audio channels. Figure 1 provides an illustration of these parameters. In Figure 1, spatial audio reproduction is shown using a 5.1 surround system (separate 5 + 1 bass effect). To approximate ITD, ILD, and IACC, inter-channel parameters such as ICTD, ICLD, and ICC are extracted from the audio channels, which model the human perception of sound in space.

[0006] A typical setup employing parametric spatial audio analysis is shown in Fig. 2. Fig. 2 illustrates a basic block diagram of a parametric stereo coder. A stereo signal pair is input to the stereo encoder 201. A parameter extraction 202 assists the downmix process by the downmixer 204 to prepare a single channel representation of the two input channels to be encoded using the mono encoder 206. The extracted parameters are encoded by the parameter encoder 208. That is, the stereo channels are downmixed to a mono signal 207, which is encoded and transmitted to the decoder 203 together with the encoded parameters 205 describing the spatial image. Usually, some of the stereo parameters are represented in spectral subbands on a perceptual frequency scale, such as the Equivalent Rectangular Bandwidth (ERB) scale. The decoder performs the stereo synthesis based on the decoded mono signal and the transmitted parameters. That is, the decoder reconstructs the single channel using the mono decoder 210 and synthesizes the stereo channel using the parametric representation. The decoded mono signal and the received encoded parameters are input to a parameter synthesis unit 212, or a process that decodes the parameters and synthesizes the stereo channels using the decoded parameters, outputting a synthesized stereo signal pair.

[0007] Since the encoded parameters are used to render spatial audio for the human auditory system, it is important to take perception into account in extracting and encoding the inter-channel parameters in order to maximize the perceived quality.

[0008] The side channels may not be explicitly coded and may be approximated by the decorrelation property of the central channel. Decorrelation techniques are generally filtering methods used to generate an output signal that is incoherent with respect to the input signal in terms of fine structure. The spectral and temporal envelopes of the decorrelated signal are assumed to remain ideal. Decorrelation filters are generally all-pass filters with phase changes of the input signal. Summary of the Invention

[0009] The essence of the embodiment is adaptive control of the characteristics of a decorrelator for representing non-coherent signal components used in a multi-channel audio decoder. The adaptation is based on the transmitted performance indicators and how they change over time. Different aspects of the decorrelator can be adaptively controlled using the same basic method to match the characteristics of the input signal. One of the most important aspects of the decorrelator characteristics is the selection of the decorrelator filter length, which is explained in the detailed description. Other aspects of the decorrelator may be adaptively controlled in a similar way, such as the control of the strength of the decorrelated components or other aspects that may need to be adaptively controlled to match the characteristics of the input signal.

[0010] A method is provided for decorrelation filter length adaptation. The method includes receiving or obtaining a control parameter and calculating the mean and variance of the control parameter. A ratio of the variance of the control parameter to the mean is calculated, and an optimal or targeted decorrelation filter length is calculated based on the current ratio. The optimal or targeted decorrelation filter length is then applied or provided to a decorrelation filter.

[0011] According to a first aspect, an audio signal processing method for adaptively adjusting a decorrelator is presented. The method includes obtaining a control parameter and calculating an average value and a variance of the control parameter. A ratio of the variance of the control parameter to the average value is calculated, and a decorrelation parameter is calculated based on the ratio. The decorrelation parameter is then provided to the decorrelation device.

[0012] The control parameter may be a performance index, which may be obtained from an estimated reverberation length, a correlation index, an estimate of the spatial width or a prediction gain.

[0013] The control parameters are either received from an encoder, such as a parametric stereo encoder, or are obtained from information already available at the decoder or by combining available information with transmitted information (i.e. information received by the decoder).

[0014] The decorrelation filter length can be adapted in at least two subbands so that each frequency band can have an optimal decorrelation filter length, which means that for a particular frequency subband or coefficient, a filter shorter or longer than the targeted length can be used.

[0015] This method is performed by a parametric stereo decoder or a stereo audio codec.

[0016] According to a second aspect, there is provided an apparatus for adaptively adjusting a decorrelator, the apparatus comprising a processor and a storage device including instructions executable by the processor, the apparatus being operable to obtain a control parameter and calculate an average value and a change in the control parameter, the apparatus being operable to calculate a ratio of the change in the control parameter to the average value and to calculate a decorrelation parameter based on the ratio, the apparatus being further operable to provide the decorrelation parameter to the decorrelation separator.

[0017] According to a third aspect, there is provided a computer program comprising instructions which, when executed by a processor, cause an apparatus to perform the actions of the method of the first aspect.

[0018] According to a fourth aspect, there is provided a computer program product embodied on a non-transitory computer readable medium, the computer program product including computer code including computer executable instructions to cause a processor to perform the process of the first aspect.

[0019] According to a fifth aspect, there is provided an audio signal processing method for adaptively adjusting a decorrelator, the method comprising obtaining a control parameter and calculating a targeted decorrelation parameter based on a change in the control parameter.

[0020] According to a sixth aspect there is provided a multi-channel audio codec comprising means for performing the method of the fifth aspect.

[0021] For a more complete understanding of the exemplary embodiments of the present invention, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0022] [Figure 1] 1 is an illustration of spatial audio reproduction using a 5.1 surround system. [Diagram 2] FIG. 1 illustrates a basic block diagram of a parametric stereo coder. [Diagram 3] 1 is a graphical illustration of auditory object width as a function of IACC. [Figure 4] FIG. 2 is a diagram illustrating an example of an audio signal. [Diagram 5] FIG. 2 is a block diagram describing a method according to one embodiment. [Figure 6] FIG. 11 is a block diagram describing a method according to an alternative embodiment. [Figure 7] FIG. 1 illustrates an example of an apparatus. [Figure 8]FIG. 1 illustrates a device comprising a decorrelation filter length calculator. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Exemplary embodiments of the present invention and their possible advantages can be understood by referring to Figures 1-8 of the drawings.

[0024] Existing solutions for representing non-coherent signal components are based on time-invariant decorrelation filters, where the amount of non-coherent components in the decoded multi-channel audio is controlled by mixing decorrelated and non-decorrelated signal components.

[0025] Such time-invariant decorrelation filters have the problem that the decorrelation signal does not adapt to the characteristics of the input signal, which are affected by changes in the auditory scene. For example, the environment of a recording of a single sound source in a low reverberant environment should be represented by the decorrelation signal components from the same filter as that for a recording of a symphony orchestra performance in a large concert hall with a fairly long reverberation. Even if the amount of decorrelation components is controlled over time, the reverberation length and other characteristics of the decorrelation are not controlled. This causes the auditory scene of a recording with high reverberation to be perceived as too narrow, while the sound environment of a recording with low reverberation is too wide. The short reverberation length that is desirable for recordings with low reverberation often results in a metallic and unnatural environment for recordings with wider sound.

[0026] The proposed solution takes into account how the incoherent audio varies over time in the representation of the incoherent components in the decoded and rendered multi-channel audio signal, and uses this information to improve the control of the incoherent audio signal by adaptively controlling the decorrelation characteristics, e.g. the reverberation length.

[0027] The adaptation may be based on signal characteristics of the input signal at the encoder and controlled by transmitting one or several control parameters to the decoder, or the adaptation may be controlled from information already available at the decoder, or by combining available and transmitted information (i.e. information the decoder received from the encoder), without transmitting explicit control parameters.

[0028] The transmitted control parameters may be based on an estimated performance of a parametric description of the spatial characteristics, e.g. the stereo image in case of a two-channel input, i.e. the control parameters may be performance indices, which may be obtained from an estimated reverberation length, a correlation index, an estimate of the spatial width or a predicted gain.

[0029] This solution results in better control of reverberation in the decoded and rendered audio signal, which improves the perceived quality of various signal types, such as clean speech signals with low reverberation or wide music signals with wide audio scenes with high reverberation.

[0030] The essence of the embodiment is adaptive control of the length of the decorrelation filters for representing non-coherent signal components used in a multi-channel audio decoder. The adaptation is based on the transmitted performance indicators and how they change over time. In addition, the strength of the decorrelated components may be controlled based on the same control parameters as the decorrelation length.

[0031] The proposed solution may operate on frames or samples in the time domain on the frequency bands of a filter bank or in the transform domain, for example using the Discrete Fourier Transform (DFT), for processing on the frequency coefficients of the frequency bands. Operations performed in one domain may be performed in another domain as well, and the given embodiment is not limited to the illustrated domains.

[0032] In one embodiment, the proposed solution is utilized for a stereo audio codec that uses a parametric description of the coded downmix channels and spatial characteristics, for example as illustrated in FIG. 2. One or more parameters describing the inter-channel incoherent components, which can be extracted by parametric analysis, can be used to adaptively adjust the perceived amount of incoherent components in the synthesized stereo audio. As illustrated in FIG. 3, the IACC, i.e., inter-channel coherence, will affect the perceived width of an object or scene in spatial hearing. As the IACC decreases, the source width increases until the sound is perceived as two separate decorrelated audio sources. In order to be able to represent a wide environment in a stereo recording, the inter-channel incoherent components need to be synthesized in the decoder.

[0033] The downmix channels of the two input channels X and Y are TIFF0007591549000001.tif11170, where M are the downmix channels and S are the side channels. The downmix matrix U1 may be selected such that the energy of the M channels is maximized and the energy of the S channel is minimized. The downmix operation may include phase or time adjustments of the input signals. An example of a passive downmix is The side channel S may not be explicitly coded but may be modeled parametrically, for example by using a predictive filter, TIFF0007591549000003.tif6170 is the decoded center channel TIFF0007591549000004.tif6170 and used for spatial synthesis at the decoder. In this case, prediction parameters, e.g. prediction filter coefficients, may be coded and transmitted to the decoder.

[0034] Another way to model the side channel is to approximate it by decorrelation of the central channel. Decorrelation techniques are generally filtering methods used to generate an output signal that is incoherent with respect to the input signal in terms of fine structure. The spectral and temporal envelopes of the decorrelated signal are assumed to remain ideal. The decorrelation filter is generally an all-pass filter with a phase change of the input signal.

[0035] In this embodiment, the proposed solution is used to adaptively adjust the decorrelators used for spatial synthesis in a parametric stereo decoder.

[0036] Spatial rendering (upmix) of encoded mono channels TIFF0007591549000005.tif6170 is Obtained by TIFF0007591549000006.tif12170, U2 is the upmix matrix, and D is the fine structure in terms of The upmix matrix is ​​ideally decorrelated to the synthesized left channel TIFF0007591549000008.tif7170 and the right channel In TIFF0007591549000009.tif7170 TIFF0007591549000010.tif7170 and D. Note that the upmix may also include further signal components, such as an encoded residual signal.

[0037] An example of an upmix matrix used in parametric stereo with ILD and ICC transmission is: given by TIFF0007591549000011.tif13170, where TIFF0007591549000012.tif17170TIFF0007591549000013.tif14170.

[0038] A rotation angle α is used to determine the amount of correlation between the combined channels; Given by TIFF0007591549000014.tif11170.

[0039] The overall rotation angle β is The file is retrieved as TIFF0007591549000015.tif14170.

[0040] The ILD between two channels x[n] and y[n] is The image is given by TIFF0007591549000016.tif12170, where n=[1,...,N] is the sample index for a frame of N samples.

[0041] Inter-channel coherence can be estimated by inter-channel cross-correlation (ICC). A traditional estimate of ICC is the cross-correlation function (CCF), r xy The CCF is a measure of the similarity between two waveforms x[n] and y[n], and is generally expressed as r xy [n, τ] = E[x[n]y[n+τ]] (10) where τ is the time lag and E[·] is the expectation operator. For a signal frame of length N, the cross-correlation is generally given by Estimated as TIFF0007591549000017.tif10170.

[0042] The ICC is then taken as the maximum of the CCFs normalized by the signal energy as follows: TIFF0007591549000018.tif19170

[0043] Additional parameters may be used in describing the stereo image, which may for example reflect the phase difference or the time difference between the channels.

[0044] The decorrelation filter has its own impulse response h d (n) or the transfer function H in the DFT domain d (k), where n is the sample index and k is the frequency index. In the DFT domain, the decorrelated signal M d but The signal is obtained by filtering the time-domain signal by filtering the k-th frequency coefficient. The image is taken by TIFF0007591549000020.tif8170, where n is the sample index.

[0045] In one embodiment, a reverberator based on A consecutively connected all-pass filters is TIFF0007591549000021.tif11170, where ψ[a] specifies the feedback decay and d[a] specifies the feedback delay. This is just an example of a reverberator that can be used for decorrelation; alternative reverberators exist, e.g., a small sample delay can be used. The damping factor ψ[a] may be chosen to be greater than 1 in the interval [0,1], resulting in an unstable filter. By choosing damping factor ψ[a]=0, the filter will be d[a] samples delayed. The filter length will then be given by the maximum delay d[a] among the set of filters in the reverberator.

[0046] In multi-channel audio, in this example two-channel audio, there is of course a varying amount of coherence between channels depending on the signal characteristics. For a single loudspeaker recorded in a damped environment, there should be a small amount of reflection and reverberation and a high coherence between channels. As the reverberation increases, there should generally be a decrease in coherence. This means that for a clean speech signal with a low noise environment, the length of the decorrelation filter should probably be shorter than for a single loudspeaker in a reverberant environment. The length of the decorrelation filter is one of the important parameters that control the characteristics of the generated decorrelated signal. An embodiment of the present invention can also be used to adaptively control other parameters, such as parameters related to the level control of the decorrelated signal, in order to match the characteristics of the decorrelated signal to those of the input signal.

[0047] By utilizing a reverberator for rendering of non-coherent signal components, the amount of delay can be controlled to adapt to different spatial characteristics of the encoded audio. More generally, the length of the decorrelation filter's impulse response can be controlled. As mentioned before, controlling the filter length can be equivalent to controlling the delay of a reverberator without feedback.

[0048] In one embodiment, the delay d (which is equivalent to the filter length in this case) of a reverberator without feedback is a function f1(·) of the control parameter c1 as follows: d=f1(c1) (16)

[0049] The transmitted control parameters may be based on the estimated performance of a parametric description of the spatial characteristics, e.g. the stereo image in case of a two-channel input. The performance index r may be obtained, for example, from an estimated reverberation length, a correlation index, an estimate of the spatial width or a prediction gain. The length d of the decorrelation filter may then be controlled based on this performance index, i.e. c1 is the performance index r. An example of a suitable control function f1(·) is TIFF0007591549000022.tif14170 and γ1 is generally the maximum allowable delay D max The range [0,D max ], and θ1 is an upper bound on g(r). If g(r)>θ1, then a shorter delay, e.g. d=1, is chosen.

[0050] θ1 is a tuning parameter and may be set, for example, to θ1=7.0. There is a relationship between θ1 and the dynamics of g(r), which may be, for example, θ1=0.22 in another embodiment. The subfunction g(r) may be defined as the ratio between the change in r and the average of r over time. This ratio will be larger for sounds with a large change in the performance index compared to the average performance index, which is typically the case for sparse sounds with little background noise or reverberation. For denser sounds, such as music or speech with background noise, the ratio will be smaller, and thus it will act like a sound classifier, classifying the characteristics of the incoherent components of the original input signal. This ratio can be expressed as: TIFF0007591549000023.tif11170, θ max is an upper limit, set to, for example, 200, and θ min is a lower limit, for example set to 0. The limit may be related to, for example, a tuning parameter θ1, for example θ max =1.5θ1.

[0051] The estimate of the mean value of the transmitted performance metric is given for frame i by The file is obtained as TIFF0007591549000024.tif16170.

[0052] First frame r mean For [i-1], it may be initialized to 0. pos and α neg may be selected such that the increase and decrease in r are different. pos = 0.005 and α neg= 0.5 means that the average estimate will track more towards the minimum of the average performance index over time. In another embodiment, the positive and negative smoothing coefficients can be set to any value, e.g., α pos = α neg =0.1.

[0053] Similarly, a smoothed estimate of the performance index change is It is obtained as TIFF0007591549000025.tif16170, where r c [i]=|r[i]-r mean [i] | (21) It is.

[0054] Alternatively, the variance of r is It can be estimated as TIFF0007591549000026.tif20170.

[0055] Then, the ratio g(r) is TIFF0007591549000027.tif12170 average value mean may be associated with, i.e. TIFF0007591549000028.tif10170, or one may relate the variance to the mean square, i.e. The result is TIFF0007591549000029.tif11170.

[0056] Another estimate of the standard deviation is The image can be given by TIFF0007591549000030.tif20170, which is simpler.

[0057] Smoothing coefficient β pos and β neg is r c may be selected such that the increasing and decreasing changes in β are different. pos = 0.5 and β neg = 0.05 means that the average estimate largely tracks the maximum in the change in the performance index over time. In another embodiment, the positive and negative smoothing coefficients can be set to, for example, βpos =β neg =0.1.

[0058] In general, for any given example, the transition between the two smoothing factors can be made with respect to any threshold against which the update value of the current frame is compared. That is, in the given example of Equation (25), c [i]>θ thres It is.

[0059] In addition, the ratio g(r) that controls the delay over time is TIFF0007591549000031.tif9170 may be smoothed by a smoothing factor α s is a tuning coefficient, set for example to 0.01. This means that for frame i, g(r[i]) in equation (17) is would be replaced by TIFF0007591549000032.tif7170.

[0060] In another embodiment, the ratio g(r) is conditionally smoothed based on a performance index c1 as follows: TIFF0007591549000033.tif8170

[0061] An example of such a function is TIFF0007591549000034.tif15170, where the smoothing parameter is a function of the performance index. For example, The result is TIFF0007591549000035.tif15170.

[0062] Function f thres may be chosen differently depending on the performance metric used. thres may be, for example, an average, a percentile (e.g., a median), a minimum or maximum value c1 over a set of frames or samples, or over a set of frequency subbands or coefficients, i.e., f thres (c1) = max(c1[b]) (30) and b=b0,...,b N-1 are the indices of the N frequency subbands. The smoothing coefficient is the threshold θ high (e.g., set to 0.6) controls the amount of smoothing when is exceeded or not, and can be equal or different for positive and negative updates, e.g., κ pos_high = 0.03, kappa neg_high = 0.05, kappa pos_low = 0.1, kappa neg_low =0.001.

[0063] It may be noted that additional smoothing or limiting of the change in the length of the correlation separation filter obtained between samples or frames is possible to prevent artifacts. In addition, the set of filter lengths used for correlation separation may be limited to reduce the number of different colorations obtained when mixing signals. For example, there may be two different lengths, one relatively short and one relatively long.

[0064] In one embodiment, a set of two available filters of different lengths d1 and d2 is used. The targeted filter length d is, for example, TIFF0007591549000036.tif12170, and γ1 is, for example, γ1=d2-d1+δ (32) where d2 is assumed to be larger than d1. It is noted that while the target filter length is the control parameter, for different frequencies different filter lengths or reverberator delays may be utilized. This means that for a particular frequency subband or coefficient, a filter shorter or longer than the targeted length may be used.

[0065] In this case, the composite channel TIFF0007591549000037.tif7170 and The decorrelation filter strength s, which controls the amount of decorrelated signal D in TIFF0007591549000038.tif7170, may be controlled by the same control parameter, in this case with one control parameter being the performance index c1≡r.

[0066] In another embodiment, the adaptation of the decorrelation filter length is performed in several, ie at least two, subbands such that each frequency band may have an optimal decorrelation filter length.

[0067] In one embodiment, if the reverberator uses a set of filters with feedback as expressed in equation (15), the amount of feedback ψ[a] may also be adapted in a similar manner to the delay parameter d[a]. In such an embodiment, the length of the generated environment is a combination of both these parameters, and therefore both must be adapted to achieve the appropriate environment length.

[0068] In yet another embodiment, the decorrelation filter length or reverberator delay d and the decorrelation signal strength s are functions of two or more different control parameters, i.e. d=f2(c 21 ,c 22 ,...) (33) s = f3(c 31 ,c 32 ,...) (34) It is controlled as:

[0069] In yet another embodiment, the length of the decorrelation filter and the strength of the decorrelation signal are controlled by analyzing the decoded audio signal.

[0070] In addition, the reverberation length may be specially controlled for transients, ie, sudden increases in energy, or other signals with special characteristics.

[0071] As the filters change over time, something should be done for the changes across frames or samples. This could be for example an interpolation or a window function with overlapping frames. The interpolation can be done between the previous filter, each of controlled length, and the currently targeted filter length across several samples or frames. The interpolation can be achieved by increasing the gain of the current filter of the currently targeted length across samples or frames while successively decreasing the gain of the previous filter. In another embodiment, the targeted filter length controls the filter gain of each available filter, so as to mix available filters of different lengths when the targeted filter length is not available. In the case of two available filters, filter h1 of length d1 and filter h2 of length d2, the gains s1 and s2 of both filters are s1=f3(d1,d2,c1) (35) s2=f4(d1,d2,c1) (36) It can be obtained as:

[0072] The filter gains may depend on each other, for example to obtain equal energy of the filtered signal, i.e. s2=f(s1) if h1 is the reference filter, whose gain is controlled by c1. For example, the filter gain s1 is s1=(d2-d) / (d2-d1) (37) where d is the targeted filter length in the range [d1, d2], d2>d1. The second filter gain can then be obtained as, for example, The file is retrieved as TIFF0007591549000039.tif10170.

[0073] Then, if the filtering operation is performed in the time domain, the filtered signal m d [n] is The file is obtained as TIFF0007591549000040.tif8170.

[0074] If the strength s of the decorrelation signal is controlled by a control parameter c1, it may be beneficial to control the strength s as a function f4(·) of the control parameter of the previous frame and the length d of the decorrelation filter, i.e. s[i]=f4(d,c1[i],c1[i-1],...,c1[iN M ]) (40) It becomes.

[0075] An example of such a function is s[i]=min(β4c1[id],c1[id](1-α4)+α4c1[i]) (41) where α4 and β4 are tuning parameters, e.g., α4 = 0.8 or α4 = 0.6 and β4 = 1.0. α4 should generally be in the range [0,1], and β4 may be greater than 1.

[0076] In the case of mixing multiple filters, the filtered signal m d [n], The strength s in the upmix with TIFF0007591549000041.tif7170 can be obtained, for example, based on a weighted average, i.e., for two filters h1 and h2: s[i]=min(β4w[i],w[i](1-α4)+α4c1[i]) (42) where w[i]=s1c1[i-d1]+s2c1[i-d2] (43) It is.

[0077] FIG. 4 shows an example of a signal where the first half contains clean speech and the second half contains classical music. The average value of the performance index is relatively large in the second half containing music. The change in the performance index is also larger in the second half, but the ratio between the average value of the performance index and the change in the performance index is much smaller in the second half. A signal where the change in the performance index is much larger than the average value of the performance index is considered to be a signal with a continuously large amount of diffuse components, and therefore the length of the decorrelation filter in this example should be shorter in the first half than in the second half. Note that the signals in the graph are all smoothed and partially limited for a more controlled behavior. In this case, the targeted decorrelation filter length is expressed in a discrete number of frames, but in other embodiments it may vary continuously.

[0078] 5 and 6 illustrate an exemplary method for adjusting a decorrelator. The method includes obtaining a control parameter and calculating the mean and variance of the control parameter. A ratio of the variance of the control parameter to the mean is calculated, and a decorrelation parameter is calculated based on the ratio. The decorrelation parameter is then provided to the decorrelation device.

[0079] FIG. 5 describes the steps involved in adapting the length of the decorrelation filter. The method 500 starts at block 501 with receiving a performance indicator parameter, i.e. a control parameter. The performance indicator is calculated in the audio encoder and transmitted to the audio decoder. Alternatively, the control parameter is obtained from information already available in the decoder or by combining the available information and the transmitted information. First, the average value and the variance of the performance indicator are calculated, as shown in blocks 502 and 504. Then, at 506, the ratio of the variance of the performance indicator to the average value is calculated. Based on this ratio, at 508, the optimal length of the decorrelation filter is calculated. Finally, at 510, a new decorrelation filter length is applied to obtain a decorrelated signal, for example from the received mono signal.

[0080] FIG. 6 describes another embodiment of the decorrelation filter length adaptation. The method 600 starts at block 601 with receiving a performance indicator parameter, i.e. a control parameter. The performance indicator is calculated in the audio encoder and transmitted to the audio decoder. Alternatively, the control parameter is obtained from information already available in the decoder or by combining the available information and the transmitted information. First, the mean value and the variance of the performance indicator are calculated, as shown in blocks 602 and 604. Then, in 606, the ratio of the variance of the performance indicator to the mean value is calculated. Based on this ratio, in 608, the targeted decorrelation filter length is calculated. The final step is to provide the new targeted decorrelation filter length to the decorrelation filter in 610.

[0081] This method may be performed by a parametric stereo decoder or a stereo audio codec.

[0082] Figure 7 shows an example of an apparatus for performing the methods described in Figures 5 and 6. The apparatus 700 comprises a processor 710, e.g. a central processing unit (CPU), and a computer program product 720 in the form of a storage device for storing instructions, e.g. a computer program 730, which when retrieved from the storage device and executed by the processor 710, causes the apparatus 700 to perform processes related to embodiments for adaptively adjusting a decorrelator. The processor 710 is communicatively coupled to the storage device 720. The apparatus 700 may further comprise an input node for receiving input parameters or performance indicators, and an output node for outputting processed parameters, such as the length of the decorrelating filter. Both the input node and the output node are communicatively coupled to the processor 710.

[0083] The apparatus 700 may be comprised in an audio decoder, such as the parametric stereo decoder shown in the lower part of Fig. 2. The apparatus 700 may be comprised in a stereo audio codec.

[0084] Fig. 8 shows a device 800 comprising a decorrelation filter length calculator 802. The device 800 may be a decoder, for example a speech decoder or an audio decoder. The input signal 804 is an encoded mono signal with encoded parameters describing a spatial image. The input parameters may include control parameters such as a performance index. The output signal 806 is a synthesized stereo or multi-channel signal, i.e. a reconstructed audio signal. The device 800 may further comprise a receiver (not shown) for receiving the input signal from an audio encoder. The device 800 may further comprise a mono decoder and a parametric synthesis unit as shown in Fig. 2.

[0085] In one embodiment, the correlation separation length calculator 802 comprises an acquisition unit for receiving or acquiring a performance index parameter, i.e., a control parameter. The correlation separation length calculator 802 further comprises a first calculation unit for calculating the average value and the change of the performance index, a second calculation unit for calculating the ratio of the change of the performance index to the average value, and a third calculation unit for calculating a targeted correlation separation filter length. The correlation separation length calculator 802 may further comprise a supply unit for supplying the targeted correlation separation filter length to the correlation separation unit.

[0086] As an example, the software or computer program 730 may be realized as a computer program product, typically carried or stored on a computer readable medium, preferably a non-volatile computer readable storage medium. The computer readable medium may include one or more removable or non-removable storage devices, including but not limited to a read only memory (ROM), a random access memory (RAM), a compact disk (CD), a digital versatile disk (DVD), a Blu-ray disk, a universal serial bus (USB) storage device, a hard disk drive (HDD) storage device, a flash memory, a magnetic tape, or any other conventional storage device.

[0087] An embodiment of the present invention may be implemented in software, hardware, application logic, or a combination of software, hardware and application logic. The software, application logic and / or hardware may reside on a storage device, microprocessor or central processing unit. If desired, portions of the software, application logic and / or hardware may reside on a host device or storage device, host microprocessor or central processing unit. In an exemplary embodiment, the application logic, software or instruction set is stored on any of a variety of conventional computer readable media.

[0088] Abbreviation ILD / ICLD Inter-channel level difference IPD / ICPD Inter-channel phase difference ITD / ICTD Inter-channel time difference IACC Interaural Cross-Correlation ICC Inter-Channel Correlation DFT Discrete Fourier Transform CCF Cross-Correlation Function

Claims

1. An audio signal processing method (500, 600) for adaptively adjusting the decorrelation of signal components in the decoding of an audio signal, comprising: Obtaining control parameters (601); estimating (602) a mean value of the control parameter; Estimating (604) a change in the control parameter; Calculating (606) a ratio of the change in the controlled parameter to the average value; Calculating (608) a targeted decorrelation filter length based on the ratio; providing (610) the targeted decorrelation filter length to a decorrelation filter; The method includes:

2. The method of claim 1 , further comprising: calculating a strength of the decorrelation signal based on the calculated targeted decorrelation filter length.

3. The method of claim 2, wherein the intensity of the correlated separated signal is controlled as a function of two or more different control parameters.

4. The method according to claim 1 , wherein the control parameters are obtained from an estimated reverberation length, a correlation measure, a spatial width estimate or a prediction gain.

5. 5. A method according to claim 1, wherein the adaptation of the decorrelation filter length is performed in at least two subbands, each subband having an adapted decorrelation filter length.

6. A method according to any one of claims 2 to 5, in which the length of the decorrelation filter is controlled as a function of two or more different control parameters.

7. An apparatus (700, 802) for adaptively adjusting decorrelation of signal components in decoding of an audio signal, comprising: Obtain the control parameters, estimating an average value of the control parameter; Estimating a change in the control parameter; calculating a ratio of the change in the controlled parameter to the average value; Calculating a targeted decorrelation filter length based on said ratio; An apparatus adapted to provide the targeted decorrelation filter length to a decorrelation filter.

8. The apparatus of claim 7 , further configured to calculate a strength of the decorrelated signal based on the calculated targeted decorrelation filter length.

9. The apparatus of claim 8, further configured to control the intensity of the correlated separated signal as a function of two or more different control parameters.

10. 10. The apparatus according to claim 7, wherein the control parameters are obtained from an estimated reverberation length, a correlation measure, a spatial width estimate or a prediction gain.

11. 11. The apparatus of claim 7, further configured to perform decorrelation filter length adaptation in at least two subbands, each subband having an adapted decorrelation filter length.

12. 12. Apparatus according to any one of claims 8 to 11, further arranged to control a length of the decorrelation filter as a function of two or more different control parameters.

13. A stereo or multi-channel audio codec comprising an apparatus according to claim 7.

14. A parametric stereo decoder comprising an apparatus according to claim 7.

Citation Information

Patent Citations

  • A Novel Processing and Adaptive Time Signaling Method Based on a Complex Exponential Modulation Filterbank

    JP2006524832A

  • Signal processing apparatus

    JP2007065497A

  • Sound field control apparatus

    JP2012124616A

  • A method for controlling the inter-channel coherence of an upmixed audio signal

    JP2016510434A

  • Method and apparatus for adaptive control of decorrelation filters

    JP2020502562A