Encoder for encoding multi-channel audio signal

Abstract

There is described an audio encoder (600) for encoding a multi-channel audio signal (602) into an encoded signal (632), the multi-channel audio signal (602) having a plurality of channels comprising a first channel and a second channel, the audio encoder comprising: a signal shaping unit (610), a signal shaping unit (610) configured to shape each of the plurality of channels using a plurality of scaling parameters to obtain a shaped channel (612), the signal shaping unit (610) configured to derive the plurality of scaling parameters for each of the plurality of channels; a stereo processing unit (620) configured to receive the shaped channel (612) and to provide a joint shaped audio signal (622) from the shaped channel (612); an encoded signal writer (630) configured to form an encoded signal (632) using at least the joint shaped audio signal (622); and a feature determiner (640) configured to determine, from the plurality of channels, a feature (642) having a feature state selected between at least one first feature state and one second feature state, the first feature state being different from the second feature state. The signal shaping unit (610) is configured to be controlled by the feature determiner (640) and: in a first feature state, for each channel of the plurality of channels, derive a plurality of scaling parameters using channel-specific parameters for the channel; and deriving, in the second characteristic state, a plurality of scaling parameters for each of the plurality of channels using the joint parameter derived from the first channel and the second channel.

Description

Technical Field

[0001] This invention relates to an audio encoder, particularly to a stereo decision-making process for spectrum shaping and conversion of multi-channel signals to center and side channels.

[0002] In some examples, the invention relates to an encoder for encoding multichannel audio signals, thereby deciding whether to use the same spectral tilt for different channels. The invention also relates to adaptive signal synchronization of spectral tilt used in the whitening of stereo signals. The invention further relates to audio signal processing, and, for example, to MDCT-based stereo processing (e.g., of Immersive Speech and Audio Services (IVAS) codecs). Background Technology

[0003] In MDCT stereo processing, as described in reference [1] (e.g., Figure 1), system 100 includes a transform unit 102', a preprocessing unit 105, a stereo processing unit 120, a stereo bandwidth extension stage 125, and an entropy encoder 140 for encoding the multichannel audio signal 102 into a bitstream 142. The frequency domain noise shaping (FDNS) spectrum is normalized using a single ILD parameter, followed by a band-wise middle / side (M / S) pair (vs) left / right (L / R) decision (at 120), and energy-based bitrate allocation between channels processed by band-wise M / S is achieved. The processing steps are shown in Figure 1 and described below:

[0004] - Apply the estimates 115 of encoding tools, such as Temporal Noise Shaping (TNS) 105 or Long-Term Prediction (LTP) Gain 115', to the original left and right channels (L, R).

[0005] - Whitening / normalization of the signal using FDNS 110 was also performed on the left and right channels respectively;

[0006] - A band-wise M / S stereo transform is performed on the broadband ILD-normalized and whitened signal at position 120. The M / S to L / R decision at position 120 is based on arithmetic coding bit consumption estimation;

[0007] The bit rate allocation at -120 is based on the energy of the stereo-processed signal.

[0008] FDNS level 110, for example, can be implemented using Linear Predictive Coding Analysis (LPC), as described in reference [2], or using Spectral Noise Shaping (SNS) techniques, as described in reference [3]. SNS is a low-complexity alternative to LCP-based noise shaping, which calculates the scaling factor required to fully whiten the signal in the spectral domain. The scaling factor is obtained by interpolation of a small number of SNS parameters, which are directly derived from the power spectrum of the signal. In the calculation of the parameters, a pre-emphasis is applied to the signal using a spectral tilt value. This tilt value depends on the sampling frequency of the signal, which is the same in both channels of a stereo signal.

[0009] The spectral tilt used in SNS-based whitening can also be adaptively changed according to signal characteristics.

[0010] In reference [4], a monophonic signal encoder using SNS to adaptively tilt the signal by controlling the harmonicity of the signal is described. For harmonic signals (e.g., speech), a higher tilt is used to emphasize lower frequencies, while for non-harmonic signals, the tilt is reduced. Thus, for harmonic signals, lower frequencies are quantized with more detail, while for non-harmonic signals with a flatter spectrum (e.g., transients), the quantization step size is more evenly distributed across the entire spectrum, so that non-harmonic signals with a flatter spectrum can be encoded more perceptually.

[0011] Using adaptive tilt in SNS allows for the adaptation of noise-shaping pre-emphasis based on current signal characteristics, enabling perceptually effective quantization of the spectra of both harmonic and non-harmonic signals. Adding this technique to stereo encoders (e.g., MDCT-stereo) can, in principle, be easily achieved by simply deriving harmonicity metrics for each of the two channels and applying them in the corresponding FDNS level. This aims to generate harmonicity metrics that optimally fit each channel, regardless of subsequent stereo processing. Typically, the derived harmonicity metrics differ between channels (except in the general case where both channels contain the same signal), so the FDNS levels for the two channels often apply different pre-emphasis to the corresponding channel signals, resulting in different spectral envelopes used in signal whitening. Significant differences in the spectral envelopes used can be problematic for subsequent stereo processing, as different whitenings can lead to reduced energy compression in the M / S transform. This situation is not a problem if the stereo channels are generally uncorrelated, as the stereo channels are expected to be encoded independently (non-M / S transform). However, this situation can also occur with more correlated signals for various reasons (such as background noise or defects in the harmonicity measurement estimation process). For highly correlated signals, M / S transformations are expected across most or all of the stereo band, and using excessively different spectral tilts is undesirable.

[0012] A simple solution to this problem is to always use L / R (independent) encoding in these cases. However, this is often suboptimal for panned correlated signals and leads to different types of artifacts, such as stereo demasking, and typically higher quantization noise levels, which often significantly degrade perceptual quality. Another option is to always use the same spectral tilt, but this limits the encoder's ability to adapt its noise-shaping operations as well as possible to the signal characteristics. This is particularly unoptimal for very different signals (e.g., hard panned signals) that may have vastly different harmonic values ​​in the two channels.

[0013] Figure 2 illustrates a simplified stereo encoder 200 according to the prior art, which converts a multi-channel signal 102 from spatial channels to joint channels 222 based on stereo decisions made at stereo processing block 220. Here is shown an LTP parameter calculation block 215 for long-term prediction of signal 102 (e.g., in TD); a TD-FD converter 223 (shown here as converting a TD signal using MDCT); and an FDNS stage 210 for using parameters g received from LTP parameter calculation block 215. l and g r The signal output from the TD-FD converter 223 is shaped to whiten the signal. Stereo processing at 220 is applied to the whitened domain. This stereo processing block 220 is functionally equivalent to the MDCT-stereo system 100 shown in Figure 1, but with the addition of adaptive signal tilt. This tilt is only used for FDNS operations that have been completed before stereo processing. For simplicity, some preprocessing tools, quantization, and bitstream writing steps are omitted in Figure 2. Stereo processing block 220 includes the same stereo processing as in reference [1]—global ILD compensation, band-by-band M / S decision at 220, and energy-based bit rate allocation.

[0014] The operation of LTP parameter calculation block 215 in Figure 2 is similar to that of LTP unit 115 in Figure 1, and serves the same purpose as the LTP filter used in EVS[2]. It does not change the signal, but instead calculates the gain (g) for the TCX-LTP filter. l ,g r The gain is quantized and transmitted in the bitstream (not shown in the figure). The parameter g in the figure... l and g rThese represent the unquantized versions of the gains calculated for the left and right channels, respectively. MDCT block 223 uses MDCT to transform the signal from the time domain to the frequency domain. Then, at block 210, frequency domain noise shaping (FDNS) using SNS[3] is applied to obtain a whitened version of the channel signal. FDNS block 210 includes the calculation of the SNS parameters and the actual whitening of the signal. In the SNS parameter calculation, a spectral tilt is applied, which is calculated from a constant value adjusted for different signal bandwidths. This value is then multiplied by the unquantized LTP filter gain of the corresponding channel to achieve adaptive signal tilt. Summary of the Invention

[0015] According to one aspect, an audio encoder is provided for encoding a multi-channel audio signal into an encoded signal, the multi-channel audio signal having multiple channels including a first channel and a second channel, the audio encoder comprising:

[0016] The signal shaping unit is configured to shape each of the multiple channels using multiple scaling parameters to obtain a shaped channel. The signal shaping unit is configured to derive multiple scaling parameters for each of the multiple channels.

[0017] The stereo processing unit is configured to receive the shaped channels and provide a combined shaped audio signal from the shaped channels;

[0018] An encoded signal writer is configured to form an encoded signal using at least the combined shaped audio signal; and

[0019] The feature determiner is configured to determine from a plurality of channels a feature having a feature state selected between at least one first feature state and one second feature state, the first feature state being different from the second feature state.

[0020] The signal shaping unit is configured to be controlled by the feature determiner and:

[0021] In the first characteristic state, for each of the multiple channels, multiple scaling parameters are derived using channel-specific parameters for that channel; and

[0022] In the second characteristic state, for each of the multiple channels, multiple scaling parameters are derived using the joint parameters obtained from the first and second channels.

[0023] According to one aspect, an audio encoder is provided for encoding a multi-channel audio signal into an encoded signal, the multi-channel audio signal having multiple channels including a first channel and a second channel, the audio encoder comprising:

[0024] The signal shaping unit is configured to shape each of the multiple channels using multiple scaling parameters to obtain a shaped channel. The signal shaping unit is configured to derive multiple scaling parameters for each of the multiple channels.

[0025] The stereo processing unit is configured to receive the shaped channels and provide a combined shaped audio signal from the shaped channels;

[0026] An encoded signal writer is configured to form an encoded signal using at least the combined shaped audio signal; and

[0027] The feature determiner is configured to determine from a plurality of channels a feature having at least one of a first feature state and a second feature state, the first feature state being different from the second feature state.

[0028] The signal shaping unit is configured to be controlled by the feature determiner and:

[0029] In the first characteristic state, for each of the multiple channels, multiple scaling parameters are derived using channel-specific parameters for that channel; and

[0030] In the second characteristic state, for each of the multiple channels, multiple scaling parameters are derived using the joint parameters obtained from the first and second channels.

[0031] According to one aspect, the signal shaping unit is configured to use a harmonicity metric for a specific channel or a metric derived from a harmonicity metric as a channel-specific parameter, and / or,

[0032] The joint parameters are derived from the harmonicity of the vocal tract.

[0033] According to one aspect, the signal shaping unit is configured to use the long-term prediction (LTP) parameter of the channel or a metric derived from the LTP parameter as a channel-specific parameter, and / or,

[0034] The joint parameters are derived from the long-term prediction (LTP) parameters of the vocal tract.

[0035] According to one aspect, the signal shaping unit is configured to use quantized channel-specific parameters or a metric derived from quantized channel-specific parameters as channel-specific parameters, and / or,

[0036] The joint parameters are derived from the quantized channel-specific parameters.

[0037] According to one aspect, the signal shaping unit is configured to use normalized channel-specific parameters or a metric derived from normalized channel-specific parameters as channel-specific parameters, and / or,

[0038] The joint parameters are derived from the normalized tract-specific parameters.

[0039] According to one aspect, the signal shaping unit is configured to use a spectral flatness metric calculated for the corresponding channel or a metric derived from a spectral flatness metric calculated for the corresponding channel as a channel-specific parameter, and / or,

[0040] The joint parameters are derived from the spectral flatness measure calculated for each channel.

[0041] According to one aspect, in the first characteristic state, the signal shaping unit is configured to apply channel-specific parameters to control the pre-emphasis tilt of the channel-specific energy applied to each frequency band, thereby deriving the pre-emphasized channel-specific energy for each frequency band, wherein multiple scaling parameters are derived from the pre-emphasized channel-specific energy for each frequency band, and / or,

[0042] In the second characteristic state, the signal shaping unit is configured to apply joint parameters to all channels to control the pre-emphasis tilt of the channel-specific energy applied to each frequency band, thereby deriving the pre-emphasized channel-specific energy of each frequency band, from which multiple scaling parameters are derived.

[0043] According to one aspect, the audio encoder is configured to calculate the pre-emphasis tilt of the first and second channels for each frequency band in the following manner:

[0044] First, calculate the common terms shared by the two vocal channels;

[0045] after,

[0046] In the first characteristic state, for each channel, the common terms are scaled using channel-specific parameters; and

[0047] In the second characteristic state, the common term is scaled using joint parameters for both channels.

[0048] According to one aspect, the audio encoder is configured such that a relatively higher channel-specific parameter results in a higher pre-emphasis tilt being applied to the channel-specific energy of each frequency band compared to a relatively lower channel-specific parameter, and / or,

[0049] A relatively higher joint parameter than a relatively lower joint parameter results in a higher pre-emphasis tilt being applied to the channel-specific energy of each frequency band.

[0050] According to any of the above-described audio encoders, wherein, in the first characteristic state, the channel-specific energy of each frequency band confirms:

[0051] ,

[0052] in, It applies to exponents where d > 1, and h > 0 is fixed. It is a channel-specific parameter or derived from a channel-specific parameter. It is predefined, and b is the index of the frequency band among the nb frequency bands.

[0053] On the one hand, specific parameters of a channel are the same for all or multiple frequency bands of the same channel, and / or,

[0054] The joint parameters are the same for all or multiple frequency bands of the same channel.

[0055] According to one aspect, the audio encoder is configured to use a joint parameter that is the average value between channel-specific parameters of a channel, or a joint parameter that is defined as the average value between channel-specific parameters of a channel, or a joint parameter that is defined as the intermediate value between at least channel-specific parameters of a channel.

[0056] According to one aspect, the audio encoder is configured to use a combination of parameters as information about the integral values ​​or integral values ​​between specific parameters of a channel, or a combination of parameters as values ​​of channel-specific parameters indicating a channel, or a combination of parameters as values ​​derived from specific parameters of a channel, or a combination of parameters defined as such based on information about the integral values ​​or integral values ​​between specific parameters of a channel, or a combination of parameters defined as such based on values ​​of channel-specific parameters indicating a channel, or a combination of parameters defined as such based on values ​​derived from specific parameters of a channel.

[0057] According to one aspect, the audio encoder is configured to weight channel-specific parameters by applying a first weight to channel-specific parameters of the first channel and a second weight to channel-specific parameters of the second channel, to use joint parameters, wherein the first weight and the second weight are proportional to the energy of the first channel and the energy of the second channel, respectively.

[0058] Based on one aspect, in the second characteristic state, the channel-specific energy of each frequency band and each channel confirms:

[0059] ,

[0060] in, It is the exponent applied when d>0 (e.g., d>1), and h>0 is fixed. is the joint parameter, and b is the index of the frequency band in nb frequency bands or derived from it.

[0061] According to one aspect, the audio encoder is configured to use features as coherence between multiple channels, or to use features as determined from the coherence between multiple channels, wherein a relatively high coherence value causes the feature to be in a second feature state, while a relatively low coherence value causes the feature to be in a first feature state.

[0062] According to one aspect, the audio encoder is configured to use features as correlations between multiple channels, or to use features as determined from the correlations between multiple channels, wherein a relatively high correlation value causes the feature to be in a second feature state, while a relatively low correlation value causes the feature to be in a first feature state.

[0063] According to one aspect, the audio encoder is configured to use features as the covariance between multiple channels, or to use features as determined from the covariance between multiple channels, wherein a relatively high covariance value results in the feature being in a second feature state, while a relatively low covariance value results in the feature being in a first feature state.

[0064] According to one aspect, the audio encoder is configured to use features as similarity between multiple channels, or to use features determined from the similarity between multiple channels, wherein a relatively high similarity value results in the feature being in a second feature state, while a relatively low similarity value results in the feature being in a first feature state.

[0065] According to one aspect, the stereo processing unit is configured to make a band-wise decision between the following two:

[0066] Multiple shaped channels are converted into a center channel and side channels, thus forming a combined channel between the center channel and side channels; and

[0067] The combined channel is defined as multiple post-shaping channels.

[0068] According to one aspect, the stereo processing unit is configured to make decisions, at least in part, based on minimizing bit rate requirements, between converting shaped audio signals from multiple shaped channels to center and side channels and defining joint channels as multiple channels.

[0069] According to one aspect, the stereo processing unit is configured to make decisions, at least in part, based on the energy distribution between the joint channels, between converting shaped audio signals from multiple shaped channels to the center and side channels and defining the joint channels as multiple channels.

[0070] According to one aspect, the stereo processing unit is configured to make decisions, at least in part, based on a measure of the cross-correlation between the shaped channels, between converting shaped audio signals from multiple shaped channels to the middle and side channels and defining a joint channel as multiple channels.

[0071] According to one aspect, the stereo processing unit is configured to make decisions, at least in part, based on a measure of coherence or similarity between the shaped channels, regarding the conversion of shaped audio signals from multiple shaped channels to the middle and side channels and the definition of a joint channel as multiple channels.

[0072] According to one aspect, the audio encoder is configured to use a feature as the number of frequency bands (in at least one previous frame, the stereo processing unit has decided to convert shaped audio signals from multiple channels to the center and side channels for these frequency bands), or to use a feature as determined from the number of frequency bands (in at least one previous frame, the stereo processing unit has decided to convert shaped audio signals from multiple channels to the center and side channels for these frequency bands), such that if the number of frequency bands for which the decision to convert shaped audio signals from multiple channels to the center and side channels exceeds a predetermined threshold, the feature is in a second feature state; otherwise, the feature is in a first feature state.

[0073] According to one aspect, the audio encoder is configured to use, with respect to the entirety of multiple channels, a feature as the number of frequency bands (for which the stereo processing unit has decided in at least one previous frame to convert the shaped audio signals from the multiple channels to the center and side channels), or with respect to the entirety of multiple channels, a feature as determined from the number of frequency bands (for which the stereo processing unit has decided in at least one previous frame to convert the shaped audio signals from the multiple channels to the center and side channels).

[0074] According to one aspect, the audio encoder is configured to use a feature as the number of frequency bands (in at least one previous frame, the stereo processing unit has decided to convert the shaped audio signal from the multiple channels to the center and side channels) with respect to a limited number of channels selected from multiple channels, or to use a feature as determined from the number of frequency bands (in at least one previous frame, the stereo processing unit has decided to convert the shaped audio signal from the multiple channels to the center and side channels) with respect to a limited number of channels selected from multiple channels.

[0075] According to one aspect, the audio encoder is configured to use a predetermined threshold of more than 50% of the total number of frequency bands or a limited number of multiple channels.

[0076] According to one aspect, the audio encoder is configured to use a predetermined threshold between 70% and 90% of the total number of frequency bands or a limited number of multiple channels.

[0077] According to one aspect, the audio encoder is configured to use the immediately preceding previous frame as at least one previous frame.

[0078] According to one aspect, the audio encoder is configured to transform the audio channel from the time domain to the frequency domain, wherein the signal shaping unit is configured to shape the audio channel in the frequency domain.

[0079] According to one aspect, the audio encoder is configured to determine features from the temporal version of the audio channel.

[0080] According to one aspect, the encoded signal writer is configured to insert information about features and / or channel-specific parameters and / or combined parameters into the encoded signal.

[0081] According to one aspect, the audio encoder also includes a long-term prediction (LTP) unit to obtain LTP gain, which is further configured to use the LTP gain as a signal-specific parameter and / or a joint parameter, or to use the LTP gain to obtain a signal-specific parameter and / or a joint parameter.

[0082] According to one aspect, the audio encoder also includes a long-term prediction (LTP) unit to obtain an LTP gain including pitch search, and is further configured to use the normalized autocorrelation value of the pitch value found by pitch search as a signal-specific parameter or joint parameter, or to use the normalized autocorrelation value of the pitch value found by pitch search to obtain a signal-specific parameter or joint parameter.

[0083] According to one aspect, the signal shaping unit is configured to spectrally tilt the audio signal based on shaping parameters obtained by applying pre-emphasis tilt to the energy of the frequency band based on channel-specific parameters for each channel, wherein the channel-specific parameters are channel-specific in a first characteristic state and equal in a second characteristic state for multiple channels.

[0084] On one hand, features indicate the similarity between multiple vocal channels.

[0085] According to one aspect, the audio encoder is configured to apply channel-specific parameters as either 1 or another constant value B > 0 when the channel is fully harmonic, and to apply channel-specific parameters as either 0 when the channel is fully non-harmonic.

[0086] The parameter is configured to be applied as a joint parameter that is the average and / or integral and / or intermediate value between two channel-specific parameters, each of which is 1 if the channel is fully harmonic or another constant value B>0 if the channel is fully non-harmonic.

[0087] According to one aspect, the signal shaping unit is configured to apply a higher pre-emphasis tilt when the harmonicity is high, and a lower pre-emphasis tilt when the harmonicity is low, in a first characteristic state, and...

[0088] In the second characteristic state, a higher pre-emphasis tilt is applied when the average value or integral value of the harmonics is high, and a lower pre-emphasis tilt is applied when the average value or integral value of the harmonics is low.

[0089] According to one aspect, a method is provided for encoding a multichannel audio signal into an encoded signal, the multichannel audio signal having multiple channels including a first channel and a second channel, the method comprising:

[0090] The shaping of each of the multiple channels is achieved by using multiple scaling parameters to obtain the shaped channels. The shaping involves deriving multiple scaling parameters for each of the multiple channels.

[0091] Stereo processing is performed, which includes providing a combined shaped audio signal from the shaped channels;

[0092] At least the combined shaped audio signal is used to form the encoded signal; and

[0093] Identify features from multiple sound channels that have at least one of a first characteristic state and a second characteristic state, wherein the first characteristic state is different from the second characteristic state.

[0094] Among them, the shaping is controlled by features:

[0095] In the first characteristic state, for each of the multiple channels, multiple scaling parameters are derived using channel-specific parameters for that channel; and

[0096] In the second characteristic state, for each of the multiple channels, multiple scaling parameters are derived using the joint parameters obtained from the first and second channels.

[0097] According to one aspect, a non-transient storage unit is provided, on which instructions are stored, which, when executed by a processor, cause the processor to perform the methods described above. Attached Figure Description

[0098] Figures 1 and 2 show encoders according to the prior art.

[0099] Figures 3 to 6 An encoder designed according to this solution is shown. Detailed Implementation

[0100] Figure 6 An example of an audio encoder 600 according to this technology is shown. Other examples of these audio encoders will be described in detail below.

[0101] The audio encoder 600 encodes the multi-channel audio signal 602 into an encoded signal 632. In general examples, either the multi-channel audio signal or the encoded signal 632 can be in any domain (e.g., time domain, frequency domain, etc.) and have any dimension. Generally, the encoded signal 632 can be understood as a compressed version of the multi-channel audio system signal 602. In some cases, at least one or both of the multi-channel audio signal 602 and the decoder signal 632 are stereo.

[0102] Signal shaping unit 610 can shape each channel of the multi-channel audio signal 602. Signal shaping unit 610 can utilize, for example, multiple scaling parameters (the number of scaling parameters can be a fixed number, such as 1, or multiple numbers, such as n channels). Scaling parameters can be, for example, shaping parameters (e.g., signal-to-noise shaping parameters, etc.). Scaling parameters can be, for example, whitening parameters. Scaling parameters can be, for example, FDNS parameters or SNS parameters. Therefore, the audio signal 602 can be constrained by signal shaping, and its shaped version 612 can present a whitened spectrum relative to the original version 602. It should be noted (although...) Figure 6 and Figure 3-5 (Not explicitly shown), scaling parameters are typically also encoded into the encoded signal so that the decoder can reconstruct the audio signal that is a reproduction of signal 602.

[0103] Generally, the channels of signal 602 (or its shaped version 612) may be in the frequency domain. In the frequency domain, the signal may have, for example, a first channel (which may be the left (L) channel) and a second channel (which may be the right (R) channel). Typically, for the sake of brevity, the same reference number as the signal may be used to indicate the channel when referring to it collectively (e.g., instead of "channels l and r" or "channels L and R", "channel 602" or other reference numbers indicating processed versions of the signal may be used).

[0104] Audio encoder 600 may include stereo processing unit 620. Stereo processing unit 620 may receive shaped channels 612 of audio signal 602. Stereo processing unit 620 may provide a combined shaped audio signal 622 (e.g., as an output) from shaped channels 612. The combined shaped audio signal 622 may include, for example, shaped channels 612, which may be the same L / R shaped channels of version 612. Alternatively, following a decision by stereo processing unit 620, stereo processing unit 620 may provide channels converted in the middle-side domain as a combined channel 622, i.e., including the middle channel (M) and side channels (S). Therefore, stereo processing unit 620 may decide whether to convert shaped channels 612. Stereo processing unit 620 may make stereo decisions based on minimizing bit rate requirements. In other alternatives, stereo decisions may be based on the energy distribution between combined channels 622. In another alternative, stereo decisions can be based on a measure of the cross-correlation between the shaped channels 612. Stereo decisions (and subsequently whether to switch from L / R to M / S) can be made band-by-band, i.e., for each band, there can be a decision about whether to switch from L / R to M / S.

[0105] The audio encoder 600 may include an encoded signal writer (e.g., a bitstream writer) 630. The encoded signal writer 630 may form an encoded signal 632 using at least the combined shaped audio signal 622. In some cases, parameters such as scaling parameters are present (therefore, the encoded audio signal 622 may include transmission channels and parameters, such as scaling parameters as side information). The encoded signal 622 may be a bitstream (or may be a portion thereof).

[0106] The encoded signal writer (e.g., a bitstream writer) 630 may include, for example, a quantizer for quantizing the shaped signal 622 (or a processed version thereof) before actually writing the encoded signal (bitstream) 622. The encoded signal writer (e.g., a bitstream writer) 630 may include, for example, at least one of a quantizer, an IGF (Intelligent Gap Fill) unit, and an entropy encoder. In some representations of this example (e.g.) Figure 4 and Figure 5 For the sake of brevity, at least one of the quantizer, IGF unit and entropy encoder is represented by a single block 450 and is shown outside the encoded signal writer (e.g. bitstream writer) 630.

[0107] Audio encoder 600 may include feature determiner 640 (embodied in some examples by a "tilted sync stage"). Feature determiner 640 may determine feature 642 from multiple channels (e.g., in versions thereof of signal 602 or 612). Feature 642 may have at least one of a first feature state and a second feature state, the first feature state being different from the second feature state. Thus, the feature state of a feature may be selected by feature determiner 640 between at least the first feature state and the second feature state. In examples, selection may be made between only two feature states. In other examples, more than two feature states may exist. Feature states may not intersect with each other. The second feature state may be associated with, for example, a relatively high coherence between channels of a multi-channel audio signal, compared to the first feature state. Additionally or alternatively, the second feature state may be associated with a relatively high correlation between channels of a multi-channel audio signal, compared to the first feature state. Additionally or alternatively, the second feature state may be associated with a relatively high covariance between channels of a multi-channel audio signal, compared to the first feature state. Alternatively, the second characteristic state may be associated with a relatively high similarity between the channels of the multi-channel audio signal, compared to the first characteristic state. Generally, the second characteristic state indicates that the channels tend to be similar (coherent, correlated, covariant, etc.), while the second characteristic state indicates that the channels tend to be different (incoherent, uncorrelated, non-covariant, etc.).

[0108] Feature determiner 640 selects signal feature 642 by comparing at least one coherence value (or correlation value, or covariance value, or similarity value) with at least one corresponding threshold (which may be a coherence threshold, correlation threshold, covariance threshold, or threshold, respectively). Accordingly, feature determiner 640 may select a second feature state when the coherence value (or correlation value, or covariance value, or generally, similarity value) is higher than the threshold (thus indicating higher similarity), and select a first feature state when at least one coherence value (or correlation value, or covariance value, or similarity value) is lower than the corresponding threshold. The threshold can be understood as distinguishing between low coherence, low covariance, low correlation, low similarity, etc., when the coherence value, covariance value, correlation value, or similarity value is lower than the threshold (meaning the selection of the first feature state), and high coherence, high covariance, high correlation, high similarity, etc., when the coherence value, covariance value, correlation value, or similarity value is higher than the threshold (meaning the selection of the second feature state). In some cases, the feature determiner 640 may make decisions based at least in part on the time-domain version of the signal 602. In some cases, the feature determiner 640 may make decisions based at least in part on the results of stereo processing (e.g., for a previous frame).

[0109] The following will show (for example in) Figures 3 to 5In the middle), the decision made by the feature determiner 640 can take the form of providing specific parameters (e.g., joint parameters and / or channel-specific parameters) to the signal shaping unit 610.

[0110] It should be noted that stereo decision-making (at stereo processing unit 620) can be performed band-by-band (e.g., stereo conversion can be selected for a first band, while skipping the conversion step can be selected for another band in the same frame), while the determination of signal feature 642 (at feature determiner 640) can be performed for multiple bands (e.g., all bands in the same frame or multiple consecutive frames). Therefore, signal feature 642 can be globally effective, for example, effective for all (or at least multiple) bands in the same frame. Thus, in the example, a signal feature is determined once for each frame (and subsequently classified between the first feature state and the second feature state). Therefore, this feature is generally globally effective for all bands in a frame.

[0111] The signal shaping unit 610 can be configured to be controlled by the feature determiner 640 (specifically, by the current information of feature 642), and to derive:

[0112] • In the first characteristic state (e.g., low correlation, low coherence, low covariance, and / or low similarity between channels, with a smaller number of frequency bands converted to M / S channels), for each channel, the signal shaping unit 610 uses channel-specific parameters (e.g., for the left channel, the scaling parameter is obtained from a specific metric of the left channel; for the right channel, the scaling parameter is a specific metric of the right channel or is derived from a specific metric of the right channel, e.g., "individually").

[0113] • In the second characteristic state (e.g., high correlation, high coherence, high covariance, and / or high similarity between channels, with a large number of frequency bands converted to M / S channels), for all channels, the signal shaping unit 610 uses joint parameters that are for the first and second channels or derived from the first and second channels (e.g., “synchronization”).

[0114] As will be specifically shown below, for example, in the case of a second characteristic state (e.g., high correlation, high coherence, high covariance, and / or high similarity between channels, with a large number of frequency bands converted to M / S channels), scaling parameters can be obtained by applying the same spectral tilt to different channels. In the case of a first characteristic state (e.g., low correlation, low coherence, low covariance, and / or low similarity between channels, with a smaller number of frequency bands converted to M / S channels), scaling parameters can be obtained by applying different spectral tilts (i.e., applying a first spectral tilt to the first channel and a second spectral tilt to the second channel, the first spectral tilt being derived from channel-specific parameters of the first channel, and the second spectral tilt being derived from channel-specific parameters of the second channel).

[0115] Alternatively or alternatively, the signal shaping unit 610 may:

[0116] a. For each channel, use the long-term prediction (LTP) parameters of the same channel (e.g., LTP gain and / or cross-correlation, e.g., normalized cross-correlation) as channel-specific parameters in the case of a first characteristic state (e.g., measured or expected low correlation, etc.); and / or,

[0117] b. Use common (joint) parameters for all channels as joint parameters in the case of a second characteristic state (e.g., high correlation as measured or expected), where the common (joint) parameters are long-term prediction (LTP) parameters of the two channels (e.g., LTP gain and / or cross-correlation, e.g., normalized cross-correlation), or derived from the long-term prediction (LTP) parameters of the two channels (e.g., by the average of the LTP parameters, or more generally by a linear combination of the LTP parameters, or by the intermediate value of the LTP parameters).

[0118] (The LTP parameters used for the first and / or second characteristic states may be quantized in some examples and in others.)

[0119] (The LTP parameters used for the first feature state and / or the second feature state can be normalized, while in other examples they can be unnormalized).

[0120] Alternatively or alternatively, the signal shaping unit 610 may:

[0121] a. For each channel, use quantized channel-specific parameters (e.g., parameters that are the same as those written into the coded signal 632) (channel-specific parameters can be, for example, quantized LTP parameters, and / or quantized whitening parameters, and / or quantized FDNS parameters for a specific channel) as channel-specific parameters in the case of a first characteristic state (e.g., measured or expected low correlation, etc.); and / or,

[0122] b. Use common (joint) parameters for all channels as joint parameters in the case of a second characteristic state (e.g., measured or expected high correlation, etc.). The common (joint) parameters are quantized channel-specific parameters of the channels (e.g., those quantized channel-specific parameters written into the coded signal 632) or derived from the quantized channel-specific parameters of the channels.

[0123] (The joint parameter can be, for example, the average value of the quantized channel-specific parameters, or more generally, a linear combination of the quantized channel-specific parameters, or an intermediate value between the channel-specific parameters.) (The quantized channel-specific parameters can be, for example, the quantized LTP parameter, or the quantized whitening parameter, or the quantized FDNS parameter.)

[0124] (The quantized channel-specific parameters can be, for example, quantized LTP parameters that are averaged among different channels or more generally linearly combined among different channels, or quantized whitening parameters or quantized FDNS parameters.)

[0125] (The quantized parameters used for the first feature state and / or the second feature state may be normalized, or, in other examples, may be non-normalized).

[0126] Alternatively or alternatively, the signal shaping unit 610 may:

[0127] a. For each channel, use a spectral flatness metric or a value derived from (or indicative of) a spectral flatness metric as a channel-specific parameter in a first characteristic state (e.g., measured or expected low correlation, etc.); and / or,

[0128] b. Use the common (joint) parameter obtained from the spectral flatness measure calculated for the two channels as the joint parameter in the case of the second characteristic state (e.g., the measured or expected high correlation, etc.). The joint parameter may be, for example, the average value of the spectral flatness measure or the information derived from the spectral flatness measure, or more generally, a linear combination or intermediate value between the spectral flatness measure or the information derived from the spectral flatness measure.

[0129] More generally, the signal shaping unit 610 can:

[0130] a. For each channel, use a harmonicity metric or a value derived from a harmonicity metric as a channel-specific parameter under a first characteristic state (e.g., measured or expected low correlation, etc.); and / or,

[0131] b. Use the common (joint) parameter derived from the harmonicity measure calculated for the two channels as the joint parameter in the case of the second characteristic state (e.g., the measured or expected high correlation, etc.). The joint parameter may be, for example, the average value of the harmonicity measure or the information derived from the harmonicity measure, or more generally, a linear combination or intermediate value between the values ​​of the harmonicity measure or the information derived from the harmonicity measure.

[0132] (Examples of harmonicity measures are LTP parameters, such as LTP gain and / or cross-correlation, for example, normalized cross-correlation, which can be quantized or unquantized.)

[0133] As will be shown below, in some cases, the determiner's decision regarding the state of feature 642 can be based on coherence, correlation, covariance, similarity, etc., between channels (e.g., in the time-domain version of signal 602). However, in some cases, the decision regarding the state of the feature can be based on the immediately preceding frame, for example, by counting the number of frequency bands for which M / S conversion has been performed at stereo processing unit 620, thereby providing an expected indication of coherence, correlation, covariance, similarity, etc., for the current frame.

[0134] Even if not in Figure 6 As shown, the parameters considered for use in controlling the signal shaping unit 610 (channel-specific parameters and / or joint parameters) (which may be, for example, harmonicity measures, such as LTP parameters, such as LTP gain, and / or cross-correlation, such as normalized cross-correlation) may be those parameters encoded in the encoded signal 632 (e.g., after quantization). Even if not in Figure 6 It is shown in the figure, but will be shown in the following figures.

[0135] In the examples above and below, for completeness, it is often shown that the channel-specific parameters in the first characteristic state and the joint parameters in the second characteristic state are derived from homogeneous metrics (e.g., the gain of the LTP filter used for both the joint parameters and the channel-specific parameters). This is generally considered advantageous, but it is also possible that the channel-specific parameters in the first characteristic state are taken from a channel-specific metric, while the joint parameters in the second characteristic state are derived from a homogeneous metric.

[0136] Furthermore, signal 602 may undergo multiple processing steps upstream of signal shaping unit 610. Therefore, a version of signal 602 input to signal shaping unit 610 may be in the frequency domain, while the original version of signal 602 may be in the time domain. Thus, the audio encoder 600 shown in some subsequent figures may also include a time-domain to frequency-domain converter. Additionally, in some examples, feature determiner 640 may make decisions between a first feature state and a second feature state based on the time-domain version of signal 602. Alternatively or additionally, channel-specific parameters and / or joint parameters may be obtained from the time-domain version of signal 602. Moreover, in some examples, signal feature 642 may be applied to the frequency-domain version of signal 602.

[0137] In any case, signal 602 (and its processing versions 912, 622, etc.) can be, for example, a type divided into frames (e.g., consecutive frames) according to a specific sequence. For example, the duration of a frame can be, for example, 20 ms (but it could also be other lengths). In the time domain, each frame has multiple time-domain values, while in the frequency domain, each frame has multiple bins. Depending on some techniques (e.g., improved discrete cosine transform (MDCT), improved discrete sine transform (MDST), etc.), consecutive frames in a sequence can partially overlap each other.

[0138] An example of using feature 642 to control noise shaping at 610 could be controlling spectral tilt (for pre-emphasis). The purpose of pre-emphasis is to increase the amplitude of the shaped spectrum (612) in low frequencies, thereby reducing quantization noise in low frequencies. By controlling spectral tilt (in the first and / or second characteristic states) using a harmonicity metric (or another similar parameter), the increase in the amplitude of the shaped spectrum (612) in low frequencies can be controlled based on the harmonicity of the channel of signal 602. Therefore, in general, if the channel of signal 602 is highly harmonic (e.g., primarily speech), the amplitude of the shaped spectrum (602) increases at low frequencies (typically speech), while its amplitude may decrease with respect to high frequencies (primarily noise). If the channel is weakly harmonic (e.g., primarily noise), the lower frequency portion of the spectrum increases less (or not at all) compared to the case of a highly harmonic channel, and the higher frequency portion of the spectrum decreases less (or not at all) compared to the case of a highly harmonic channel. Using the current technique, the following may occur:

[0139] - In the case of the first characteristic state (e.g., low similarity, low correlation, low covariance, low coherence, etc. measured or expected between channels), the spectral tilt in the channels is different, and for each channel, the spectral tilt increases or decreases based on channel-specific parameters (e.g., harmonicity), so for each channel, lower harmonicity means lower tilt (while higher harmonicity means higher tilt).

[0140] - In the case of a second characteristic state (e.g., high similarity, high correlation, high covariance, or high coherence between channels as measured or expected), the same spectral tilt is applied to both channels, and for all channels, the spectral tilt increases or decreases synchronously based on a joint parameter (e.g., the average or another intermediate value between the harmonics of the channels), so for all channels, a lower joint parameter means a lower tilt (while a higher joint parameter means a higher tilt).

[0141] An example of using spectral tilt for pre-emphasis is given by Provided, among which, This is an exponent, for example, applied to d>1 (e.g., d=10), while h>0 (e.g., h≥1, or h=10) is fixed. (For example, () is a channel-specific parameter or is derived from a channel-specific parameter. It is predefined and generally depends on the sampling frequency (e.g., for higher sampling frequencies, (Possibly high), b is the index of the frequency band among nb frequency bands.

[0142] A more common way to remember it is This is the same as before, but the base of the exponentiation is 10, as always. Then, pre-emphasis is applied to the spectral energy. Thus, the energy information of pre-emphasis is obtained.

[0143] ,

[0144] Or more commonly,

[0145] .

[0146] It should be noted that the spectral energy is usually different between the two channels. Therefore, the notation... Specifically

[0147] For the first (e.g., left) channel; and

[0148] For the second (e.g., right) channel.

[0149] Even though the energy of each frequency band is different, they can be tilted equally in the second characteristic state.

[0150] Note that the frequency bands are usually indexed by index b, which can be between a lower limit index (e.g., 0) indicating low frequency (e.g., DC in the case of 0) and a maximum index (e.g., equal to nb, which could be 63 in the case of the signal being subdivided into 64 frequency bands) and increase to the maximum index.

[0151] Furthermore, due to this solution, the spectrum tilt also depends on . It can be:

[0152]

[0153]

[0154] It is worth noting that, This is an example of a joint (common) parameter for two channels, and is used as a parameter. (First / Left Channel Specific) and The average value (but in some examples, it can be a linear combination or an intermediate value) of the second / right channel is obtained. Therefore, again, in the case of the second characteristic state (e.g., high correlation, etc.), the same spectral tilt is applied to both channels, achieving "tilt synchronization." In the case of the first characteristic state (e.g., low correlation, etc.), different spectral tilt values ​​are applied to the channels. Thus, a channel-specific tilt based on channel-specific parameters is obtained for each channel. It is worth noting that... (First / Left Channel Specific) and (Second / Right Channel Specific) can be, for example, harmonics and / or parameters derived from harmonics. In the example, the higher the harmonics, the better. (First / Left Channel Specific) and (Second / Right Channel Specific) The higher the value, the higher the spectral tilt, and the more detailed the lower frequencies of the shaped spectrum are quantized relative to the higher frequencies. In the case of the second characteristic state, intermediate values ​​(e.g., average values) can be used.

[0155] Therefore, the issue discussed above mainly addresses the situation where most channels are expected to be converted to M / S channels, and the signal primarily uses the same spectral tilt value.

[0156] It is important to note that spectral tilt control is typically about reducing spectral tilt by a fixed value, because weights between 0 and 1 can be applied in the first and second characteristic states (e.g., 0 for completely noisy or transient channels, and 1 for completely harmonic channels). This can be achieved, for example, by weighting the tilt using normalized values ​​(such as normalized harmonicity). Therefore, and Each of the values ​​can be between 0 and 1.

[0157] The following section provides an example of how to use spectral tilt (and more generally, feature determiner 640 and feature 642) to obtain an unconstrained representation of the shaped channels of the shaped signal 612. Note that the described steps are applied in the same manner to all signals, except where the concept of inter-channel differences exists (most notably in step 3). The signal 602 to be shaped has... The indicated spectrum is considered to be in the frequency domain, such as the MDCT domain (however, other frequency domains can also be used), and the shaped signal 622 is determined by the spectral values. and scaling factor This indicates that both will be encoded. Assuming there are... Each frequency band (or possibly other number of frequency bands) is indicated by an index b that increases with frequency. Each frequency interval is indicated by k and is within the frequency band. The first interval to frequency band The last interval The changes between them.

[0158] Step 1: Energy of each frequency band

[0159] Energy per frequency band Yes, for example, it can be calculated in the following way (other techniques can also be used):

[0160]

[0161] in These are MDCT coefficients. It refers to the number of frequency bands, and It is a frequency band index. Frequency bands are non-uniform and follow a perceptually correlated Barker scale (smaller in low frequencies and larger in high frequencies). It can be instantiated for the first channel and the second channel respectively. and ,and They can be instantiated as and .

[0162] Step 2: Smoothing

[0163] Optionally, the energy per frequency band Smoothing can be achieved using the following methods (other techniques can also be used):

[0164]

[0165] Note: This step is mainly used to smooth the vector. Potential instabilities may arise. Without smoothing, these instabilities are amplified when transformed to the logarithmic domain (see step 5), particularly at valleys where the energy is close to 0. In this case, They can be instantiated as and .

[0166] Step 3: Pre-weighting

[0167] For the first (e.g., left) channel, use the following formula for the smoothed energy per frequency band. Perform pre-weighting:

[0168]

[0169] And for the second (e.g., right) channel:

[0170]

[0171] in:

[0172] a.

[0173] b.

[0174] (Other examples of defining union parameters can also be used).

[0175] It can depend on the sampling frequency. Yes, for example, 21 at 16kHz and 26 at 32kHz (or more generally, for higher sampling frequencies, Higher, for lower frequencies, (Lower).

[0176] Step 4: Noise

[0177] For example, the following formula can be used for each channel to... Add optional noise floor, for example -40dB:

[0178]

[0179] The background noise can be calculated using the following formula:

[0180]

[0181] This can improve the quality of signals with very high spectral dynamics (such as glockenspiels) by limiting the amplitude amplification of the shaped spectrum at the valleys. The indirect effect is to reduce quantization noise at the peaks, at the cost of increasing quantization noise at the valleys (which is imperceptible in any case). They can be instantiated as and .

[0182] Step 5: Logarithm

[0183] Alternatively, the following formula can be used to perform the transformation to the logarithmic field:

[0184]

[0185] (A logarithmic base other than 2 can be used, and / or a divisor other than 2 can be used). Can be instantiated as and .

[0186] Step 6: Downsampling

[0187] Alternatively, the vector can be factored by 4 (or any other factor). Downsampling. For example, you can use...

[0188]

[0189] in,

[0190]

[0191] This step can be understood as processing the vector before extraction. A low-pass filter (w(k)) is applied. This low-pass filter has a similar effect to the spread function used in psychoacoustic models: it reduces quantization noise at the peak, at the cost of increasing quantization noise around the peak, which is perceptually masked in any case. They can be instantiated as and .

[0192] Step 7: Mean Removal and Scaling

[0193] The final scaling factor is obtained after mean removal and scaling by a factor of 0.85, as shown in the following formula:

[0194]

[0195] Because the codec has additional global gain, the mean can be removed without losing any information. Mean removal also allows for more efficient vector quantization.

[0196] A scaling factor of 0.85 slightly compresses the amplitude of the noise shaping curve. It has a similar perceptual effect to the spreading function mentioned in step 6: reducing quantization noise at peaks and increasing quantization noise at valleys.

[0197] scf(n) can be instantiated as scf respectively. l (n) and scf r (n).

[0198] Step 8: Quantification

[0199] The scaling factor can be quantized using vector quantization, producing indices (which are then packed into the bitstream and sent to the decoder) and the quantized scaling factor. .

[0200] Step 9: Interpolation

[0201] Quantized scaling factor Interpolation can be performed using the following formula:

[0202]

[0203] Then transform back to the linear domain using the following formula:

[0204]

[0205] Interpolation is used to obtain a smooth noise shaping curve, thereby avoiding any large amplitude jumps between adjacent frequency bands. Similarly, They can be instantiated as and .

[0206] Step 10: Spectrum Shaping

[0207] SNS scaling factor The MDCT frequency lines were applied separately to each frequency band to generate the shaped spectrum. :

[0208]

[0209] They can be instantiated as and Therefore, a scaling factor is used to shape the signal 602 and obtain the shaped channel 612. The calculation can be controlled by the spectral tilt value. It's important to note that a relatively high spectral tilt results in more detail quantization at lower frequencies of the shaped spectrum, while a relatively low spectral tilt results in more uniform quantization across the entire spectrum. The pre-emphasis applied by the signal shaping unit 610 can increase the amplitude of the shaped spectrum 622 at low frequencies, thereby reducing quantization noise at low frequencies. Using channel-specific parameters and / or joint parameters (e.g., harmonicity measures) to control the spectral tilt allows the intensity of this effect to be adapted to the channel-specific parameters and / or joint parameters (e.g., harmonicity measures) of the audio signal 602. Therefore, for highly harmonic signals, the effect is to increase the amplitude of the shaped spectrum 622 at low frequencies, thereby reducing quantization noise; while for non-harmonic signals, a weaker spectral tilt is applied to the shaped energy (the lower frequency portion of the spectrum is not over-amplified or not amplified at all compared to the higher frequencies), thus allowing for more uniform quantization across the entire spectrum.

[0210] It should be noted that different techniques can be selected based on feature 642 to define the spectral tilt. In some cases, any of steps 1, 2, and 4-10 can be omitted.

[0211] More generally, it is not necessary to define spectral tilt as in step 3. In some examples, in the first characteristic state, the value of spectral tilt may be independent of harmonicity (or more generally, independent of channel-specific parameters), while in the second characteristic state, the value of spectral tilt may depend on harmonicity (or more generally, on joint parameters). In other cases, in the first characteristic state, the value of spectral tilt may depend on harmonicity (or more generally, on channel-specific parameters), while in the second characteristic state, the value of spectral tilt may be independent of harmonicity (or more generally, independent of joint parameters). In still other examples, the tilt value may be independent of harmonicity in both the first and second characteristics. In many examples, the tilt in the second characteristic state is higher than the tilt in the first characteristic state.

[0212] Figures 3 to 5 It shows Figure 6 A specific example.

[0213] Figure 3 An example of encoder 300 is shown, which can be Figure 6A specific instantiation of encoder 600. Here it is shown that, for each channel, audio signal 302 (in this case, the time-domain version of signal 602) undergoes signal shaping at stage 310 (which could be an example of signal shaping unit 610). Here, channels l and r both undergo LTP at LTP stage 315, and are then transformed to the frequency domain at stage 323 (in this case, this domain is shown as the MDCT domain), denoted by L and R, thus obtaining frequency-domain version 304 of signal 302 (signal 602 can be instantiated as either or both of versions 302 and 304). Signal noise shaping at stage 310 (instantiation) Figure 6 (610) is based on the LTP gain obtained at LTP level 315. However, it is worth noting that if channels L and R are highly correlated (e.g., highly covariant, or highly similar), then the feature determiner 340 (which can be) Figure 6 (Instantiation of block 640) selects the second characteristic state. Accordingly, if the channels are similar to each other (or expected to be similar), the audio signal can be shaped using the same spectral tilt for both channels. Otherwise, different spectral tilts are used for different channels, such as for channel L in signal shaping at stage 310 (610). For use of channel R Stereo processing unit 320 (which can be...) Figure 6 The stereo processing unit 620 (instantiated) can perform band-by-band stereo decision-making and, while making the decision, can perform a conversion to the joint channel of signal 622 (otherwise, maintain spatial channels L and R). In this case, feature determiner 340 (640) can receive or measure a metric 624 about how many frequency bands in the previous frame have been converted to the middle-side domain. Feature determiner (tilt sync stage) 340 (640) can determine the feature based on the number of frequency bands converted to the middle-side domain for the immediate preceding frame (or multiple previous frames). Therefore, arrow 624 providing information num_MSbands is thus provided to feature determiner 340. Symbol 624' is used to indicate frame delay. Therefore, feature determiner 340 decides whether to cause the same spectral tilt to different channels at signal shaping block 310 based on whether num_MSbands exceeds a threshold. For example, if more than 80% (or another threshold, such as between 70% and 90%, or more than 50%) of the frequency band in an immediately preceding frame has been shifted to the middle / side domain (e.g., num_MSbands > 80%), then the signal shaper 310 will use the same spectral tilt for both channels. Otherwise (e.g., num_MSbands < 80%), different spectral tilts will be used for different channels. In examples where frames are subdivided into subframes (e.g., in the case of block switching), an average can be calculated between subframes, or only the last subframe can be considered.

[0214] In an alternative, information 624 can be avoided, and in this case, feature determiner 640 can make decisions based on, for example, measurements of similarity between different channels (e.g., covariance, correlation, coherence, similarity, etc.) taken from the time-domain version of signal 302.

[0215] Figure 4 Example 400 is shown, which can be an instantiation of the encoder 600 described above. Here, the input audio signal 402 can be converted into a frequency domain representation (channels L and R) 403 at stage 423. The audio signal representations 402 and 403 can be considered to correspond to versions 302 and 304 respectively, and are instantiated. Figure 6 The audio signal 602. The time domain also provides LTP parameter level 415 (which can be... Figure 3 The LTP parameter calculation is based on the instantiation of 315. Therefore, the LTP parameter... and It can be quantized at parameter quantizer stage 370 (indicated by 371) and then inserted into the bitstream (including encoded signals 432 and 632) by encoded signal encoder 430 (which may be an embodiment of encoded signal writer 630). Here, feature determiner (skew synchronization stage) 440 (which may be an embodiment of feature determiner 640 and 340) can be used to determine whether the feature is in a first feature state or a second feature state. Figure 3 Similarly, information about the number of frequency bands (for which previous frames (e.g., the immediately preceding previous frame) have been converted to the intermediate-side domain) is provided as 424 (and via a delay of 424'). Even in this case, a signal shaping unit 410 (which may emulate signal shaping unit 610) is also provided for providing parameters obtained from LTP stage 415. and This provides the shaped channels L' and R' (shaped signal 412). For the remainder, stereo processing 420 (which can be embodied in stereo processing 620) operates in the same manner, providing the joint channel 422 (622) in signal 422 (622). Here, IGF and quantization and entropy coding stages 450 are provided such that the resulting signal 452 is provided to the coded signal writer 430 (630).

[0216] Although Figure 4 The mid-channel specific parameters and / or joint parameters are provided in an unquantized version, but may also be provided in a quantized version (e.g., version 371 provided to the bitstream writer 430). 'and .

[0217] Figure 5 Another example 500 is shown, which could also be Figure 6Example 600 and / or Figure 4 Example 400 or Figure 3 Example 300 is an embodiment. Here, except where it is necessary to identify some differences, it uses the same... Figure 4 The same reference number is used for the example. Specifically, in this case, the feature determiner (tilted synchronization level) 540 (instantiated) Figure 6 Feature determiner 640 and / or Figure 4 The feature determiner 430 does not decide whether to impose the same spectral tilt on the signal shaping unit 410 based on the number of frequency bands involved in the conversion to the center-side channels. Instead, the feature determiner 540 bases its decision on a measurement 524 (also indicated by c), which can be, for example, the inter-channel correlation c between channels l and r (in this case, e.g., in the time domain) (e.g., obtained from the correlation calculation unit 525). For example, it can be specified as follows:

[0218]

[0219]

[0220] However, as mentioned above, a linear combination can be used instead of the average, or g can be used. l and g r The interchannel correlation c can be normalized, for example, to the range [0, 1.0]. α can be a threshold for the correlation measure, beyond which M / S encoding is expected to be predominantly chosen in subsequent stereo processing. α can be, for example, 0.8 (or, for example, a value between 0.7 and 0.9). The interchannel correlation c can be, for example, obtained from the time-domain version 102 of the audio signal.

[0221] In some examples, the joint parameters can be calculated using the following formula: Where a + b = 1, a > 0, b > 0. a and b can be determined, for example, based on the energy of the channels (e.g., the higher the energy of the first channel relative to the energy of the second channel, the larger a is; the higher the energy of the second channel relative to the energy of the first channel, the larger b is), such that the spectral tilt of channels with higher energy has a greater weight in the joint parameter values. Therefore, a and b are proportional to the energy of their respective channels. Thus, the coefficients a and b divide the channel-specific parameters according to the energy of each channel.

[0222] This invention concludes that different spectral tilts can be avoided in highly correlated channels. This may sound counterintuitive, as adapting the spectral tilt to the harmonicity of each channel generally helps to adapt the quantization step size to the signal characteristics across the spectral range. This generally holds true for encoding only one (mono) signal. However, the inventors have considered joint encoding when encoding more than one (stereo) signal in a joint manner. Highly correlated signals can be (primarily) efficiently encoded into M / S representations, achieving good perceptual quality for such signals. However, encoding a signal into an M / S representation inserts correlated quantization noise into the final decoded signal. Using different pre-emphasized FDNS parameters for the input channels—that is, using parameters calculated with different spectral tilts—results in different spectral shaping of the two decoded channels and the inserted quantization noise (which is the same in both decoded channels for M / S encoded bands) during signal decoding. This can lead to spatial demasking of the quantization noise, which significantly degrades perceptual quality and is therefore undesirable. In contrast, the less-than-ideal FDNS parameters used for pre-emphasis in the corresponding channels, achieved by co-coding the channels into such signals, are offset by the quality improvement. Therefore, even when the tilt synchronization decision is applied to all bands, the benefit of synchronizing the tilt in co-coded bands outweighs the potential suboptimal tilt used in non-co-coded bands—especially if the (identical) tilt is used only when most bands are expected to be co-coded. For related channels, synchronizing (making them identical) the spectral tilt not only enables efficient co-coding of the signals but also efficient co-coding of the SNS parameters themselves (using, for example, reference [5]), thereby reducing the bit requirements for transmitting the SNS parameters. One might assume that for highly correlated signals, the calculated noise shapes should be similar enough that synchronization is unnecessary. However, the inventors have recognized that effects such as signal fluctuations caused by background noise or general deficiencies in signal analysis algorithms can still lead to differences in the applied spectral tilt.

[0223] Compared to a possible alternative that generates two sets of shaped channels (one with independent tilt and the other with joint tilt) and uses channels with independent tilt for the L / R coded band and channels with joint tilt for the M / S coded band in subsequent stereo processing, another advantage lies in its reduced complexity. This would require two whitening operations in the encoder. Another disadvantage of this alternative is that it requires transmitting two sets of FDNS parameters to the decoder to decode the L / R coded signal portion and the M / S coded signal portion.

[0224] discuss

[0225] This invention allows, among other things, adaptive synchronization (e.g., using the same) of spectral tilt between different channels to achieve a balance between using the most accurate parameters possible for individual channel coding tools and achieving good channel compression in stereo processing.

[0226] The above examples are discussed below (for example, Figures 3 to 5 Some examples of implementations (e.g., examples). To simplify reading, although the above assumptions are somewhat general, some less general assumptions are made below.

[0227] Figure 3 This illustrates an example of current technology, which could be... Figure 6 Example of an implementation. Unquantized LTP filter gain g l and g r These are not directly applied to the calculation of SNS tilt, but they can be synchronized, i.e., set to the same value for both channels. The decision on whether to use the filter gain of each channel to calculate the SNS parameter or to use the same value is based on the number of frequency bands (denoted by n) encoded as M / S in the previous frame. As an example, if the number of M / S frequency bands in the previous frame is higher than an experimentally adjusted threshold, the spectral tilt of both channels is multiplied by the average of the gains of the two LTP filter filters, instead of using g separately for the left channel. l and using g on the right channel r Otherwise, as shown in Figure 2, the spectral tilt of each channel is multiplied by the corresponding channel's LTP filter gain.

[0228] This approach avoids stereo demasking artifacts that can occur when there is high correlation between the two channels (thus using M / S coding in most or all frequency bands) and when different harmonic metrics between the channels lead to different spectral tilts. Harmonic metrics can differ due to signal fluctuations, background noise, and limitations in the estimation algorithm, and this may not be entirely avoidable. An obvious solution to this would be to force L / R coding or always use the same spectral tilt for both channels. In a stereo decision-making sense, forcing L / R coding would be suboptimal because the problematic signal portion would still be correlated even if the harmonic metrics—and therefore the spectral tilt and scaling factor used to whiten the signal—differences, whereas M / S coding achieves better perceptual quality in this sense. In a perceptual noise-shaping sense, always using the same spectral tilt in both channels could also be suboptimal because adapting the spectral tilt to the harmonics of the signal generally reduces perceptible quantization noise. Therefore, this adaptive synchronization mechanism generally helps to use the optimal spectral tilt for each channel, while when there is high correlation between channels, only the potentially suboptimal spectral tilt needs to be sacrificed to avoid stereo demasking artifacts. Using the number of M / S coded bandwidths from previous frames as the decision criterion results in lower computational costs and leverages existing encoder architectures. Therefore, this invention can be easily added to stereo encoders without significantly increasing computational complexity or causing structural changes to the entire system.

[0229] Figure 4 The integration of the present invention into an MDCT-stereo framework is illustrated. The left and right input channels of input signal 402 are represented by l and r in the time domain, respectively, and are processed in blocks (frames). Input signal 402 is transformed (to obtain signal 403) to the frequency domain, for example, using MDCT at stage 423, and is preprocessed, for example, using TNS (also indicated at stage 423). Different time-to-frequency transformation or preprocessing methods (with or without TNS) can be applied.

[0230] LTP parameter calculation block 315 may be the same as block 115 in Figure 1 and / or functionally equivalent to what is described in reference [2], except (in some examples) the output gain value g. l and g r Unquantized and normalized to the range [0, 1.0]. Quantization of the gain is represented by the downstream Q-block and can be the same as that applied in reference [2]. The unquantized gain is processed by the tilt synchronization stage (feature determiner) 440 (640) to generate the spectral tilt value of the current frame to be used in FDNS, g l 'and g r It can be described as follows:

[0231]

[0232]

[0233] , where n MS It is the number of M / S coded frequency bands of the previous frame as determined in the stereo processing block (as described in reference [1]), n bands β is the total number of frequency bands used in stereo processing, and β is a threshold below which M / S encoding is expected to be primarily selected in subsequent stereo processing. β can be, for example, 0.2. If the current frame is further divided into subframes (e.g., by using block switching), n is calculated separately for each subframe and the average value is used. Alternatively, instead of using the average value n over all subframes, the value of n obtained only in the last subframe can be used. Since no previous frames are available at the start of the signal, n is set to 1 in the first frame.

[0234] The transformed and preprocessed signal is fed back to FDNS -1 The whitening signals L' and R' are generated in the block respectively. Here, FDNS is implemented using SNS[3] with adaptive spectral tilt[4]. The spectral tilt is changed by multiplying a constant tilt value with the corresponding output of the tilt synchronization level. Therefore, step 3 of [Reference 3, page 15] is modified as follows:

[0235] Regarding the left channel:

[0236]

[0237] For the right channel:

[0238]

[0239] g tilt The fixed value of g depends on the sampling rate, and, for example, at 16kHz tilt The value is 21, and g is at 32kHz. tilt It is 26.

[0240] Then, the whitened signals L' and R' are stereo-processed as described in reference [1] to generate two joint channels. Afterward, bandwidth-extended (BWE) encoding (e.g., using IGF), quantization, and entropy encoding (e.g., using a range encoder) are applied to the joint channels. Finally, all quantized parameters are written into a bitstream for transmission or storage.

[0241] Alternatively, the maximum value of the normalized autocorrelation of the pitch found in the LTP parameter calculation (as described in reference [2]) can be used as a substitute for the unquantized LTP gain value. In this case, g l and g rThey are set to the maximum autocorrelation value for their respective channels. Other processing remains unchanged.

[0242] Figure 5 One possible variation is shown. The block with the same name... Figure 4 The same applies, except for the following changes. The condition for setting the spectral tilt to the same value in the tilt synchronization level is not like... Figure 4 Instead of using the M / S coding bands as in the previous example, a metric c of inter-channel correlation is calculated for the stereo channels. This can be calculated in the time domain, for example, as the cross-correlation coefficient between two channels; or in the frequency domain, using, for example, the cross-coherence metric. The output of the tilt sync stage is then:

[0243]

[0244]

[0245] c is normalized to the range [0, 1.0], and α is a threshold for the correlation metric; when c is above this threshold, M / S encoding is expected to be primarily chosen in subsequent stereo processing. α can be, for example, 0.8.

[0246] Important aspects

[0247] The following summarizes the key aspects.

[0248] This technique includes applying band-by-band M / S decision-making in the whitening spectral domain (the whitening process is controlled by adaptive signal parameters), configured to adaptively decide whether to apply independent channel parameters or calculate and use common parameter values ​​during whitening.

[0249] This technique includes applying band-by-band M / S decision in the whitening spectral domain. The whitening process is controlled by signal adaptive parameters, which are harmonic parameters that are larger for harmonic signals and smaller for non-harmonic signals.

[0250] Optionally,

[0251] - The parameter is the maximum normalized autocorrelation value of the pitch value determined in the LPT gain calculation.

[0252] - The parameter is the LPT gain.

[0253] This technique includes applying band-by-band M / S decision-making in the whitening spectral domain (the whitening process is controlled by adaptive signal parameters), configured to adaptively decide whether to apply independent channel parameters or calculate and use common parameter values ​​during whitening, where the decision is based on the number of M / S coded bands of the previous frame.

[0254] This technique includes applying band-by-band M / S decision-making in the spectral domain of whitening (the whitening process is controlled by adaptive signal parameters), configured to adaptively decide whether to apply independent channel parameters or calculate and use common parameter values ​​during whitening, wherein the decision is based on an inter-channel correlation metric.

[0255] This technique includes applying band-by-band M / S decision-making in the whitening spectral domain (the whitening process is controlled by adaptive signal parameters), configured to adaptively decide whether to apply independent channel parameters or calculate and use common parameter values ​​during whitening, wherein the decision is based on inter-channel coherence metrics.

[0256] It should be noted that scaling the spectrum (pre-emphasis) tilt slightly increases the computational cost, for example, because it requires passing through... and / or Scaling. However, when calculating as well as The index can be calculated using the following steps. and :

[0257] 1) First, calculation items (Common for both channels and all frequency bands);

[0258] 2) Second, this applies to all frequency bands (or at least to multiple frequency bands being evaluated), but for each channel, it is done separately through... and Scaling common items ;as well as

[0259] 3) Third, for each frequency band and each channel, the items obtained by scaling through the frequency band index b will be... Thus, the above index is obtained.

[0260] More generally, the pre-emphasis tilt of the first and second channels is calculated for each frequency band as follows:

[0261] First, calculate the common terms shared by the two channels (e.g., based on the sampling frequency).

[0262] Then,

[0263] In the first characteristic state, for each channel, the common terms are scaled using channel-specific parameters; and

[0264] In the second characteristic state, the common term is scaled using joint parameters for both channels.

[0265] The term b associated with each frequency band can be multiplied by a channel-specific parameter (or, respectively, a joint parameter) before or after scaling.

[0266] Finally, the term nb can also be used with b, i.e., as n / nb.

[0267] Therefore, the computational cost increased by adaptive spectrum tilt is relatively small, thus maintaining the overall low complexity of the spectrum noise shaping technique.

[0268] Other implementations

[0269] Depending on the specific implementation requirements, the example can be implemented in hardware. The above implementation can be carried out using a digital storage medium on which electronically readable control signals are stored, such as a floppy disk, digital versatile optical disc (DVD), Blu-ray disc, optical disc (CD), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory, where the electronically readable control signals cooperate (or are capable of cooperating) with a programmable computer system to perform the corresponding methods. Therefore, the digital storage medium can be computer-readable.

[0270] Typically, the example can be implemented as a computer program product having program instructions that, when run on a computer, are operable to perform one of the methods described above. The program instructions may, for example, be stored on a machine-readable medium.

[0271] Other examples include a computer program for performing one of the methods described herein, stored on a machine-readable medium. In other words, therefore, an example of this method is a computer program that includes program instructions that, when run on a computer, perform one of the methods described herein.

[0272] Therefore, another example of this method is a data carrier medium (or digital storage medium, or computer-readable medium) on which a computer program for performing one of the methods described herein is recorded. The data carrier medium, digital storage medium, or recording medium is a tangible and / or non-transient signal, rather than an intangible and transient signal.

[0273] Another example includes a processing unit, such as a computer or programmable logic device, for performing one of the methods described herein.

[0274] Another example includes a computer on which a computer program is installed for performing one of the methods described herein.

[0275] Another example includes a device or system that transmits (e.g., electronic or optical transmission) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The device or system may include, for example, a file server for transmitting the computer program to the receiver.

[0276] In some examples, a programmable logic device (e.g., a field-programmable array) may be used to perform some or all of the functions of the methods described herein. In some examples, the field-programmable array may cooperate with a microprocessor to perform one of the methods described herein. Typically, the methods can be executed by any suitable hardware device.

[0277] The examples above are used to illustrate the principles discussed herein. It should be understood that modifications and variations to the arrangements and details described herein will be readily apparent. Therefore, the intent is limited only to the scope of the pending patent claims, and not to the specific details presented herein through the description and explanation of the examples.

[0278] References

[0279] [1] G. Markovic, E. Ravelli, M. Schnell, S. Döhla, W. Jägers, M.Dietz, C. Helmrich, E. Fotopoulou, M. Multrus, S. Bayer, G. Fuchs and J.Herre, „APPARATUS AND METHOD FOR MDCT M / S STEREO WITH GLOBAL ILD WITHIMPROVED MID / SIDE DECISION". WO-Patent WO2017EP51177, 20. 01. 2017.

[0280] [2] 3GPP TS 26.445, Codec for Enhanced Voice Services (EVS); Detailedalgorithmic description.

[0281] [3] E. Ravelli, M. Schnell, C. Benndorf, M. Lutzky und M. Dietz, „Apparatus and method for encoding and decoding an audio signal usingdownsampling or interpolation of scale parameters“. WO-Patent WO 2019091904A1 , 5. 11. 2018.

[0282] [4] Markovic, Goran. Transform-based Coding Methods for Speech andother Audio Signals. Diss. Friedrich-Alexander-Universität Erlangen-Nürnberg(FAU), 2022.

[0283] [5] E. Fotopoulou, F. Reutelhuber, G. Markovic, J. Kiene, S. Döhla, „Audio Decoder, Au-dio Encoder, and Related Methods Using Joint Coding ofScale Parameters for Channels of a Multi-Channel Audio Signal”, EuropeanPatent EP20184555.9.

Claims

1. An audio encoder (600) for encoding a multi-channel audio signal (602) into an encoded signal (632), the multi-channel audio signal (602) having multiple channels including a first channel and a second channel, the audio encoder comprising: The signal shaping unit (610) is configured to shape each of the multiple channels using multiple scaling parameters to obtain a shaped channel (612), and the signal shaping unit (610) is configured to derive multiple scaling parameters for each of the multiple channels. The stereo processing unit (620) is configured to receive the shaped channel (612) and provide a combined shaped audio signal (622) from the shaped channel (612). The encoded signal writer (630) is configured to form an encoded signal (632) using at least the combined shaped audio signal (622); and Feature determiner (640) is configured to determine from a plurality of channels a feature (642) having a feature state selected between at least one first feature state and a second feature state, the first feature state being different from the second feature state. The signal shaping unit (610) is configured to be controlled by the feature determiner (640) and: In the first characteristic state, for each of the multiple channels, multiple scaling parameters are derived using channel-specific parameters for the channel. as well as In the second characteristic state, for each of the multiple channels, multiple scaling parameters are derived using the joint parameters obtained from the first and second channels.

2. The audio encoder according to claim 1, wherein, The signal shaping unit (610) is configured to use a harmonicity metric for a specific channel or a metric derived from a harmonicity metric as a channel-specific parameter, and / or, The joint parameters are derived from the harmonicity of the vocal tract.

3. The audio encoder according to any one of the preceding claims, wherein, The signal shaping unit (610) is configured to use the long-term prediction (LTP) parameters of the channel or a metric derived from the LTP parameters as channel-specific parameters, and / or, The joint parameters are derived from the long-term prediction LTP parameters of the vocal tract.

4. The audio encoder according to any one of the preceding claims, wherein, The signal shaping unit (610) is configured to use quantized channel-specific parameters or a metric derived from quantized channel-specific parameters as channel-specific parameters, and / or, The joint parameters are derived from the quantized channel-specific parameters.

5. The audio encoder according to any one of the preceding claims, wherein, The signal shaping unit (610) is configured to use normalized channel-specific parameters or a metric derived from normalized channel-specific parameters as channel-specific parameters, and / or, The joint parameters are derived from the normalized tract-specific parameters.

6. The audio encoder according to any one of the preceding claims, wherein, The signal shaping unit (610) is configured to use a spectral flatness metric calculated for the corresponding channel or a metric derived from a spectral flatness metric calculated for the corresponding channel as a channel-specific parameter, and / or, The joint parameters are derived from the spectral flatness measure calculated for each channel.

7. The audio encoder according to any one of the preceding claims, wherein, In the first characteristic state, the signal shaping unit (610) is configured to apply channel-specific parameters to each channel to control the pre-emphasis tilt of the channel-specific energy applied to each frequency band, thereby deriving the pre-emphasized channel-specific energy for each frequency band, from which multiple scaling parameters are derived, and / or, In the second characteristic state, the signal shaping unit (610) is configured to apply joint parameters to all channels to control the pre-emphasis tilt of the channel-specific energy applied to each frequency band, thereby deriving the pre-emphasized channel-specific energy of each frequency band, from which multiple scaling parameters are derived.

8. The audio encoder of claim 7, configured to calculate the pre-emphasis tilt for the first and second channels for each frequency band in the following manner: First, calculate the common terms shared by the two vocal channels; Then, In the first characteristic state, for each channel, the common terms are scaled using channel-specific parameters; and In the second characteristic state, the common term is scaled using joint parameters for both channels.

9. The audio encoder according to any one of the preceding claims is configured such that a relatively higher channel-specific parameter results in a higher pre-emphasis tilt being applied to the channel-specific energy per frequency band compared to a relatively lower channel-specific parameter, and / or, A relatively higher joint parameter than a relatively lower joint parameter results in a higher pre-emphasis tilt being applied to the channel-specific energy per frequency band.

10. The audio encoder according to any one of the preceding claims, wherein, In the first characteristic state, the channel-specific energy for each frequency band is confirmed as follows: , in, It applies to exponents where d > 1, and h > 0 is fixed. It is a channel-specific parameter or derived from a channel-specific parameter. It is predefined, and b is the index of the frequency band among the nb frequency bands.

11. The audio encoder according to any one of the preceding claims, wherein, Channel-specific parameters are the same for all or multiple frequency bands of the same channel, and / or, The joint parameters are the same for all or multiple frequency bands of the same channel.

12. The audio encoder according to any one of the preceding claims is configured to use a joint parameter as an average value between channel-specific parameters of a channel, or a joint parameter as defined by an average value between channel-specific parameters of a channel, or a joint parameter as defined by an intermediate value between at least channel-specific parameters of a channel.

13. The audio encoder according to any one of the preceding claims is configured to use a combination of parameters as information about the integral values ​​or integral values ​​between specific parameters of a channel, or a combination of parameters as values ​​of channel-specific parameters indicating a channel, or a combination of parameters as values ​​derived from specific parameters of a channel, or a combination of parameters defined as based on information about the integral values ​​or integral values ​​between specific parameters of a channel, or a combination of parameters defined as based on values ​​of channel-specific parameters indicating a channel, or a combination of parameters defined as based on values ​​derived from specific parameters of a channel.

14. The audio encoder according to any one of the preceding claims is configured to weight channel-specific parameters by applying a first weight to channel-specific parameters of a first channel and a second weight to channel-specific parameters of a second channel, to use joint parameters, wherein the first weight and the second weight are proportional to the energy of the first channel and the energy of the second channel, respectively.

15. The audio encoder according to any one of the preceding claims, wherein, In the second characteristic state, the channel-specific energy used for each frequency band and each channel is confirmed as follows: , in, It applies to exponents where d > 0, while h > 0 is fixed. is a joint parameter or derived from a joint parameter, and b is an index indicating a frequency band among nb frequency bands.

16. The audio encoder according to any one of the preceding claims, configured to use features as coherence between a plurality of channels, or to use features determined based on coherence between a plurality of channels, wherein, A relatively high coherence value results in a feature being in the second feature state, while a relatively low coherence value results in a feature being in the first feature state.

17. The audio encoder according to any one of the preceding claims, configured to use features as correlations between a plurality of channels, or to use features determined based on the correlations between a plurality of channels, wherein, A relatively high correlation value results in a feature being in the second feature state, while a relatively low correlation value results in a feature being in the first feature state.

18. The audio encoder according to any one of the preceding claims, configured to use features as covariance between a plurality of channels, or to use features determined based on the covariance between a plurality of channels, wherein, A relatively high covariance value results in a feature being in the second feature state, while a relatively low covariance value results in a feature being in the first feature state.

19. The audio encoder according to any one of the preceding claims, configured to use features as similarity between a plurality of channels, or to use features determined based on the similarity between a plurality of channels, wherein, A relatively high similarity value results in a feature being in the second feature state, while a relatively low similarity value results in a feature being in the first feature state.

20. The audio encoder according to any one of the preceding claims, wherein, The stereo processing unit is configured to make a band-wise decision between the following two: Multiple reshaped channels are converted to the center and side channels, thus forming a combined channel between the center and side channels; and The combined channel is defined as multiple post-shaping channels.

21. The audio encoder according to claim 20, wherein, The stereo processing unit is configured to make decisions, at least in part, based on minimizing bit rate requirements, between converting shaped audio signals from multiple shaped channels to center and side channels and defining joint channels as multiple channels.

22. The audio encoder according to claim 20 or 21, wherein, The stereo processing unit is configured to make decisions, at least in part, based on the energy distribution between the joint channels, when converting shaped audio signals from multiple shaped channels to the center and side channels and defining the joint channels as multiple channels.

23. The audio encoder according to any one of claims 20 to 22, wherein, The stereo processing unit is configured to make decisions, at least in part, based on a measure of the cross-correlation between the shaped channels, when converting shaped audio signals from multiple shaped channels to the center and side channels and defining a joint channel as multiple channels.

24. The audio encoder according to any one of claims 20 to 23, wherein, The stereo processing unit is configured to make decisions, at least in part, based on a measure of coherence or similarity between the shaped channels, when converting shaped audio signals from multiple shaped channels to the center and side channels and defining a joint channel as multiple channels.

25. The audio encoder according to any one of claims 20 to 24, configured to use a feature as the number of frequency bands for which a stereo processing unit has decided to convert shaped audio signals from multiple channels to the middle and side channels in at least one previous frame, or to use a feature determined according to the number of frequency bands for which a stereo processing unit has decided to convert shaped audio signals from multiple channels to the middle and side channels in at least one previous frame, such that if the number of frequency bands for which a stereo processing unit has decided to convert shaped audio signals from multiple channels to the middle and side channels exceeds a predetermined threshold, the feature is in a second feature state; otherwise, the feature is in a first feature state.

26. The audio encoder of claim 25, configured to use, with respect to the entire plurality of channels, a feature as the number of frequency bands for which the stereo processing unit has decided to convert shaped audio signals from the plurality of channels to the middle and side channels in at least one previous frame, or with respect to the entire plurality of channels, a feature determined according to the number of frequency bands for which the stereo processing unit has decided to convert shaped audio signals from the plurality of channels to the middle and side channels in at least one previous frame.

27. The audio encoder of claim 25, configured to use, with respect to a limited number of channels selected from a plurality of channels, a feature representing the number of frequency bands in at least one previous frame for which the stereo processing unit has decided to convert shaped audio signals from the plurality of channels to the middle and side channels, or, with respect to a limited number of channels selected from a plurality of channels, a feature determined based on the number of frequency bands in at least one previous frame for which the stereo processing unit has decided to convert shaped audio signals from the plurality of channels to the middle and side channels.

28. The audio encoder of claim 25, 26 or 27, configured to use a predetermined threshold of more than 50% of the total number of frequency bands or a limited number of multiple channels.

29. The audio encoder of claim 27 is configured to use a predetermined threshold between 70% and 90% of the total number of frequency bands or a limited number of multiple channels.

30. The audio encoder according to any one of claims 25 to 29, configured to use an immediately preceding previous frame as at least one previous frame.

31. The audio encoder according to any one of the preceding claims, configured to transform a channel from the time domain to the frequency domain, wherein, The signal shaping unit (610) is configured to shape the channel in the frequency domain.

32. The audio encoder according to claim 31, wherein, The feature determiner is configured to determine features from the temporal version of the audio channel.

33. The audio encoder according to any one of the preceding claims, wherein, The encoded signal writer (630) is configured to insert information about features and / or channel-specific parameters and / or combined parameters into the encoded signal.

34. The audio encoder according to any one of the preceding claims further includes a long-term prediction (LTP) unit to obtain LTP gain, and is further configured to use the LTP gain as a signal-specific parameter and / or a joint parameter, or to use the LTP gain to obtain a signal-specific parameter and / or a joint parameter.

35. The audio encoder according to any one of the preceding claims further includes a long-term prediction (LTP) unit to obtain an LTP gain including pitch search, and is further configured to use a normalized autocorrelation value of the pitch value found by pitch search as a signal-specific parameter or a joint parameter, or to use a normalized autocorrelation value of the pitch value found by pitch search to obtain a signal-specific parameter or a joint parameter.

36. The audio encoder according to any one of the preceding claims, wherein, The signal shaping unit (610) is configured to spectrally tilt the audio signal based on shaping parameters obtained by applying pre-emphasis tilt to the energy of the frequency band based on channel-specific parameters for each channel, wherein the channel-specific parameters are channel-specific in a first characteristic state and equal in a second characteristic state for multiple channels.

37. The audio encoder according to any one of the preceding claims, wherein, Features indicate the similarity between multiple vocal channels.

38. The audio encoder according to any one of the preceding claims, configured to apply a channel-specific parameter as a parameter of 1 or another constant value B > 0 when the channel is fully harmonic, and to apply a channel-specific parameter as a parameter of 0 when the channel is fully non-harmonic, and, The parameter is configured to be applied as a joint parameter that is the average and / or integral and / or intermediate value between two channel-specific parameters, each of which is 1 if the channel is fully harmonic or another constant value B>0 if the channel is fully non-harmonic.

39. The audio encoder according to any one of the preceding claims, wherein, The signal shaping unit (610) is configured to apply a higher pre-emphasis tilt when the harmonicity is high, and a lower pre-emphasis tilt when the harmonicity is low, in a first characteristic state. In the second characteristic state, a higher pre-emphasis tilt is applied when the average value or integral value of the harmonics is high, and a lower pre-emphasis tilt is applied when the average value or integral value of the harmonics is low.

40. A method for encoding a multichannel audio signal (632) into an encoded signal, the multichannel audio signal (632) having a plurality of channels including a first channel and a second channel, the method comprising: The shaping of each of the multiple channels is achieved by using multiple scaling parameters to obtain the shaped channel. The shaping includes deriving multiple scaling parameters (612) for each of the multiple channels. Stereo processing is performed, which includes providing a combined shaped audio signal from the shaped channels (622). At least the combined shaped audio signal (622) is used to form the encoded signal (632); and Determine a feature (642) from multiple channels that has at least one of a first characteristic state and a second characteristic state, wherein the first characteristic state is different from the second characteristic state. Among them, the shaping is controlled by feature (642): In the first characteristic state, for each of the multiple channels, multiple scaling parameters are derived using channel-specific parameters for the channel. as well as In the second characteristic state, for each of the multiple channels, multiple scaling parameters are derived using the joint parameters obtained from the first and second channels.

41. A non-transient storage unit having instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to claim 40.

Images