Directional Loudness Map Based Audio Processing

Abstract

To provide improved, efficient and highly accurate audio analysis, audio coding and audio decoding. A format converter (500) converts the format of audio content (520) in a first format representing an audio scene into audio content (530) in a second format, the format converter comprising: a format conversion (510) for providing a representation of the audio content in the second format based on the representation of the audio content in the first format; and an overall directional loudness map (142) of the audio scene (e.g., a plurality of different estimated directional ranges L(m,Ψ 0、j and a complexity adjustment 540 for adjusting the complexity of the format conversion depending on the contribution of the input audio signal in the first format to the first format.

Description

[Technical field]

[0001] SUMMARY OF THE PRESENT EMBODIMENTS The present invention relates to directional loudness map based audio processing. [Background technology]

[0002] The advent of perceptual audio coders has generated great interest in developing algorithms that can predict the audio quality of coded signals without resorting to extensive subjective listening tests to save time and resources. Algorithms that perform so-called objective assessments of quality for mono coded signals, such as PEAQ [3] or POLQA [4], are widespread. However, their performance for signals coded with spatial audio techniques is still considered insufficient [5]. Furthermore, since many of the features extracted for analysis assume waveform-preserving conditions, non-waveform-preserving techniques, such as bandwidth extension (BWE), are also known to make these algorithms overestimate the quality loss [6]. Spatial audio and BWE techniques are mainly used in low bitrate audio coding (around 32 kbps per channel).

[0003] It is assumed that spatial audio content of three or more channels can be rendered into a binaural representation of the signals entering the left and right ears by using a set of head-related transfer functions (HRTFs) and / or binaural room impulse responses (BRIRs) [5, 7]. Most of the proposed extensions for binaural objective assessment of quality are based on well-known binaural auditory cues related to human perception of sound image localization and perceived auditory source width, such as interaural level difference (ILD), interaural time difference (ITD), and interaural cross correlation (IACC) between the signals entering the left and right ears [1, 5, 8, 9]. In the context of objective quality assessment, features are extracted based on these spatial cues from the reference and test signals, and the distance measure between the two is used as a distortion index. Consideration of these spatial cues and their associated perceived distortions has enabled considerable progress in the context of spatial audio coding algorithm design [7]. However, in the use case of predicting the quality of overall spatial audio coding, the interplay of these cue distortions and mono / timbral distortions (especially in the case of non-waveform preserving) renders a complex scenario with various outcomes when using features to predict a single quality score given by subjective quality tests such as MUSHRA

[11]

[10] . Other alternative models have also been proposed [2], where the output of the binaural model is further processed by a clustering algorithm to identify the number of involved sound sources in the momentary auditory image, and thus is also an abstraction of the classical auditory cue distortion model. Nevertheless, the model in [2] focuses mainly on moving sources in space, and its performance is also limited by the accuracy and tracking ability of the associated clustering algorithm. The number of additional features to make this model usable is also important.

[0004] An objective audio quality measurement system should also use the smallest possible, mutually independent and most relevant extracted signal features to avoid the risk of overfitting, given the limited amount of ground truth data for mapping feature distortions to quality scores provided by listening tests [3].

[0005] One of the most prominent distortion characteristics reported in listening tests of spatially encoded audio signals at low bit rates is described as a collapse of the stereo image towards the center position and channel crosstalk

[12] . Summary of the Invention [Problem to be solved by the invention]

[0006] It would therefore be desirable to have concepts that provide improved, efficient and highly accurate audio analysis, audio coding and audio decoding. This is achieved by the subject matter of the independent claims of the present application. Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application. [Means for solving the problem]

[0007] An embodiment according to the invention relates to an audio analyzer, for example an audio signal analyzer. The audio analyzer is configured to obtain a spectral domain representation of two or more input audio signals. Thus, the audio analyzer is configured to determine or receive, for example, the spectral domain representation. According to an embodiment, the audio analyzer is configured to obtain the spectral domain representation by decomposing the two or more input audio signals into time-frequency tiles. Furthermore, the audio analyzer is configured to obtain directional information related to the spectral bands of the spectral domain representation. The directional information represents, for example, the directions (or positions) of different audio components contained in the two or more input audio signals. According to an embodiment, the directional information can be understood, for example, as a panning index describing a sound source position in a sound field generated by the two or more input audio signals in binaural processing. Furthermore, the audio analyzer is configured to obtain, as an analysis result, loudness information related to different directions, and the contribution to the loudness information is determined depending on the directional information. In other words, the audio analyzer is configured to obtain, as an analysis result, loudness information related, for example, to different panning directions or panning indexes or to a plurality of different evaluated directional ranges. According to an embodiment, the different directions, e.g. panning directions, panning indexes and / or direction ranges, can be obtained from the direction information. The loudness information comprises, e.g., a directional loudness map or level information or energy information. The contributions to the loudness information are, e.g., contributions of spectral bands of the spectral domain representation to the loudness information. According to an embodiment, the contributions to the loudness information are contributions to values ​​of the loudness information related to the different directions.

[0008] This embodiment is based on the idea that it is advantageous to determine loudness information depending on directional information obtained from two or more input audio signals. This makes it possible to obtain information about the loudness of different sound sources in a stereo audio mix realized by two or more audio signals. Thus, in an audio analyzer, the perception of two or more audio signals can be analyzed very efficiently by obtaining loudness information related to different directions as an analysis result. According to an embodiment, the loudness information can include or represent a directional loudness map that gives information about the loudness of a combination of two or more signals in different directions, averaged over all ERB bands, or about the loudness of at least one common time signal of the two or more input audio signals (ERB = equivalent rectangular bandwidth).

[0009] According to one embodiment, the audio analyzer is configured to obtain a plurality of weighted spectral domain (e.g., time-frequency domain) representations (e.g., "directional signals") based on the spectral domain (e.g., time-frequency domain) representations of two or more input audio signals. Values ​​of the one or more spectral domain representations are weighted (e.g., represented by weighting factors) according to different directions (e.g., panning direct) of audio components (e.g., of spectral bins or spectral bands) (e.g., tuning from an instrument or singer) in the two or more input audio signals to obtain a plurality of weighted spectral domain representations (e.g., "directional signals"). The audio analyzer is configured to obtain, as an analysis result, loudness information (e.g., loudness values ​​of a plurality of different directions; e.g., "directional loudness map") related to different directions (e.g., panning directions) based on the weighted spectral domain representations (e.g., "directional signals").

[0010] This means, for example, that the audio analyzer analyzes in which of the different directions of the audio components the values ​​of one or more spectral domain representations are influencing the loudness information. Each spectral bin is, for example, associated with a certain direction, and the loudness information associated with a certain direction can be determined by the audio analyzer based on the multiple spectral bins associated with this direction. The weighting can be performed for each bin or each spectral band of the one or more spectral domain representations. According to an embodiment, the values ​​of the frequency bins or frequency groups are windowed by weighting to one of the different directions. For example, they are weighted to the direction to which they are associated and / or to the neighboring directions. The direction is, for example, associated with the direction in which the frequency bin or frequency group influences the loudness information. Values ​​that deviate from that direction are, for example, less important. Thus, the multiple weighted spectral domain representations can provide an indication of the spectral bins or spectral bands that influence the loudness information in the different directions. According to an embodiment, the multiple weighted spectral domain representations can at least partially represent the contribution to the loudness information.

[0011] According to one embodiment, the audio analyzer is configured to decompose (e.g., transform) two or more input audio signals into a short-time Fourier transform (STFT) domain (e.g., using a Hann window) to obtain two or more transformed audio signals, which may represent a spectral domain (e.g., time-frequency domain) representation of the two or more input audio signals.

[0012] According to one embodiment, the audio analyzer is configured to group the spectral bins of the two or more transformed audio signals into spectral bands of the two or more transformed audio signals (e.g., such that the bandwidth of the groups or spectral bands increases with increasing frequency) (e.g., based on the frequency selectivity of the human cochlea). Furthermore, the audio analyzer is configured to weight the spectral bands (e.g., the spectral bins within the spectral bands) using different weights based on the outer ear model and the middle ear model to obtain one or more spectral domain representations of the two or more input audio signals. By specifically grouping the spectral bins into spectral bands and weighting the spectral bands, the two or more input audio signals are prepared, and the loudness perception of the two or more input audio signals by a user listening to said signals can be estimated or determined by the audio analyzer very accurately and efficiently in terms of determining loudness information. Due to this feature, the transformed audio signals adapt the spectral domain representations of the two or more input audio signals, respectively, to the human ear, improving the information content of the loudness information obtained by the audio analyzer.

[0013] According to an embodiment, the two or more input audio signals are associated with different directions or different speaker positions (e.g., L (left), R (right)). The different directions or different speaker positions can represent different channels of a stereo and / or multi-channel audio scene. The two or more input audio signals can be distinguished from each other by indexes, which can be represented, for example, by letters of the alphabet (e.g., L (left), R (right), M (center)) or, for example, by positive integers indicating the numbers of the channels of the two or more input audio signals. Thus, the indexes can indicate the different directions or speaker positions with which the two or more input audio signals are associated (e.g., they indicate the positions where the input signals occur in the listening space). According to an embodiment, the different directions (hereinafter, for example, the first different directions) of the two or more input audio signals are not related to the different directions (hereinafter, for example, the second different directions) with which the volume information obtained by the audio analyzer is associated. Thus, the first different direction directions can represent channels of a signal of two or more input audio signals, and the second different direction directions can represent directions of audio components of a signal of two or more input audio signals. The second different direction can be located between the first directions. Additionally or alternatively, the second different direction can be located outside and / or in the first direction.

[0014] According to an embodiment, the audio analyzer is configured to determine a direction-dependent weighting (e.g. based on the panning direction) for each spectral bin (e.g. and for each time step / frame) and for a number of predefined directions (desired panning directions). The predefined directions represent, for example, equidistant directions that can be associated with a predefined panning direction / index. Alternatively, the predefined directions are determined using direction information related to, for example, a spectral band of the spectral domain representation acquired by the audio analyzer. According to an embodiment, the direction information can include a predefined direction. The direction-dependent weighting is, for example, applied by the audio analyzer to one or more spectral domain representations of the two or more input audio signals. In the direction-dependent weighting, the value of a spectral bin is, for example, associated with one or more directions of the predefined directions. This direction-dependent weighting is, for example, based on the idea that each spectral bin of the spectral domain representations of the two or more input audio signals contributes to loudness information in one or more different directions of the predefined directions. Since each spectral bin, for example, mainly contributes to one direction and only slightly to adjacent directions, it is advantageous to weight the value of the spectral bin differently for different directions.

[0015] According to an embodiment, the audio analyzer is configured to determine the direction-dependent weighting using a Gaussian function such that the direction-dependent weighting decreases as the deviation between the respective extracted direction value (e.g. associated with the time-frequency bin under consideration) and the respective predefined direction value increases. Each extracted direction value may represent a direction of an audio component in the two or more input audio signals. The interval of each extracted direction value may be between a direction completely to the left and a direction completely to the right, the left and right directions being with respect to a user perceiving the two or more input audio signals (e.g. facing the speaker). According to an embodiment, the audio analyzer may determine each extracted direction value as a predefined direction value or equidistant direction values ​​as predefined direction values. Thus, for example, one or more spectral bins corresponding to an extracted direction are weighted in predefined directions adjacent to this extracted direction according to a Gaussian function that are less important than the predefined direction corresponding to the extracted direction value. The greater the distance of the predefined direction to the extracted direction, the less weighting of the spectral bin or spectral band decreases, e.g. a spectral bin has little or no effect on the loudness perception at a position far away from the corresponding extracted direction.

[0016] According to one embodiment, the audio analyzer is configured to determine a panning index value as an extracted directional value, the panning index value uniquely indicating a direction of a time-frequency component (i.e., a spectral bin) of a sound source within, for example, a stereo mix generated by two or more input audio signals.

[0017] According to one embodiment, the audio analyzer is configured to determine the extracted directional value as a function of the spectral domain values ​​of the input audio signals (e.g. values ​​of the spectral domain representation of the input audio signals). The extracted directional value is determined, for example, based on an evaluation of amplitude panning of signal components (e.g. time-frequency bins) between the input audio signals or based on a relationship between the amplitudes of corresponding spectral domain values ​​of the input audio signals. According to one embodiment, the extracted directional value defines a similarity between the spectral domain values ​​of the input audio signals.

[0018] According to one embodiment, the audio analyzer comprises: TIFF0007631273000001.tif531, in a given direction (e.g. index TIFF0007631273000002.tif34), a time (or time frame) specified by time index m, a time specified by time index m, and a direction-dependent weighting associated with a spectral bin specified by spectral bin index k. TIFF0007631273000003.tif312, where: TIFF0007631273000004.tif32 is a predefined value (which controls, for example, the width of the Gaussian window), TIFF0007631273000005.tif38 specifies the extracted directional value associated with the time (or time frame) specified by the time index m, and the spectral bin specified by the spectral bin index k, TIFF0007631273000006.tif34 is a directional value that specifies (or is associated with) a given direction (e.g., has a directional index j). The direction-dependent weighting is used to equalize the extracted directional values ​​(e.g., panning indexes). TIFF0007631273000007.tif34 (e.g., making a given direction equal) the spectral values ​​or spectral bins or spectral bands are passed through without changing the direction-dependent weighting, The idea is that the spectral values ​​or spectral bins or spectral bands of the extracted directional values ​​(e.g. panning index) that deviate from TIFF0007631273000008.tif34 are weighted. The spectral values ​​or spectral bins or spectral bands of the extracted directional values ​​that are close to TIFF0007631273000009.tif34 are weighted and passed on, and the remaining values ​​are rejected (e.g., not processed further).

[0019] According to one embodiment, the audio analyzer is configured to apply direction-dependent weighting to one or more spectral domain representations of two or more input audio signals to obtain a weighted spectral domain representation (e.g., a "directional signal"). The weighted spectral domain representation thus includes, for example, spectral bins (i.e., time-frequency components) of one or more spectral domain representations of two or more input audio signals corresponding to one or more predefined directions within a tolerance (e.g., also spectral bins associated with different predefined directions adjacent to the selected predefined direction). According to one embodiment, for each predefined direction, the weighted spectral domain representation can be realized by a direction-dependent weighting (e.g., the weighted spectral domain representation can include direction-dependent weighted spectral values, spectral bins or spectral bands associated with the predefined direction and / or associated with directions in the neighborhood of the predefined direction over time). Alternatively, for each spectral domain representation (e.g., of two or more input audio signals), one weighted spectral domain representation is obtained, e.g., representing the corresponding spectral domain representation weighted for all predefined directions.

[0020] According to one embodiment, the audio analyzer is configured to obtain the weighted spectral domain representations such that signal components related to a first predefined direction (e.g., a first panning direction) are emphasized in the first weighted spectral domain representation more than signal components having other relevant directions (different from the first predefined direction, e.g., attenuated according to a Gaussian function) and signal components related to a second predefined direction (e.g., a second panning direction) (different from the first predefined direction) are emphasized in the second weighted spectral domain representation more than signal components having other relevant directions (different from the second predefined direction, e.g., attenuated according to a Gaussian function). Thus, for example, a weighted spectral domain representation for each of the two or more input audio signals for each predefined direction can be determined.

[0021] According to one embodiment, the audio analyzer comprises an input audio signal or a combination of input audio signals designated by index i, a spectral band designated by index b, A weighted spectral domain representation relating the orientation specified by TIFF0007631273000010.tif34, the time (or time frame) specified by the time index m, and the spectral bin specified by the spectral bin index k It is configured to get TIFF0007631273000011.tif313, TIFF0007631273000012.tif348, specifying a spectral domain representation relating to an input audio signal or combination of input audio signals specified by index i (e.g., i=L or i=R or i=DM; (L=left, R=right, and DM=downmix)), a spectral band specified by index b, a time (or time frame) specified by time index m, and a spectral bin specified by spectral bin index k; TIFF0007631273000013.tif312 is the index tif34, a time (or time frame) specified by time index m, and a spectral bin specified by spectral bin index k. Thus, the weighted spectral domain representation may be determined, for example, by weighting a spectral domain representation associated with the input audio signal or combination of input audio signals by the direction-dependent weighting.

[0022] According to one embodiment, the audio analyzer is configured to determine an average over a plurality of band volume values ​​(e.g. associated with different frequency bands but associated with the same direction, e.g. the predetermined direction and / or directions in the vicinity of the predetermined direction) to obtain a composite volume value (e.g. associated with a given direction or panning direction, i.e. a predefined direction). The composite volume value may represent the volume information obtained by the audio analyzer as an analysis result. Alternatively, the volume information obtained by the audio analyzer as an analysis result may include the composite volume value. Thus, the volume information may include composite volume values ​​associated with different predefined directions, from which a directional volume map may be obtained.

[0023] According to one embodiment, the audio analyzer is configured to obtain band loudness values ​​for a plurality of spectral bands (e.g., ERB bands) based on a weighted combined spectral domain representation representing a plurality of input audio signals (e.g., a combination of two or more input audio signals) (e.g., a weighted combined spectral representation can combine weighted spectral domain representations associated with the input audio signals). Furthermore, the audio analyzer is configured to obtain a plurality of combined loudness values ​​(covering a plurality of spectral bands; e.g., in the form of a single scalar value) as an analysis result based on the band loudness values ​​obtained for a plurality of different directions (or panning directions). Thus, for example, the audio analyzer is configured to average all band loudness values ​​associated with the same direction to obtain a combined loudness value associated with that direction (e.g., resulting in a plurality of combined loudness values). The audio analyzer is configured to obtain, for example, a combined loudness value for each predefined direction.

[0024] According to one embodiment, the audio analyzer is configured to calculate an average of the squared spectral values ​​of the weighted combined spectral domain representation over the spectral values ​​of the frequency band (or over the spectral bins of the frequency band) and to apply a power operation with an exponent between 0 and ½ (and preferably equal to or less than ⅓ or ¼) to the average of the squared spectral values ​​in order to determine a band loudness value (associated with a respective frequency band).

[0025] According to one embodiment, the audio analyzer comprises: Spectral band specified by index b, index according to TIFF0007631273000015.tif543 Band volume value relative to the time (or time frame) specified by time index m in the direction specified in TIFF0007631273000016.tif34 It is configured to obtain TIFF0007631273000017.tif310. Coefficient K bspecifies the number of spectral bins in the frequency band with frequency band index b. The variable k is a running variable that specifies the number of spectral bins in the frequency band with frequency band index b, where b specifies the spectral band. TIFF0007631273000018.tif315 is the spectral band specified by index b, index TIFF0007631273000019.tif34 specifies a weighted combined spectral domain representation associated with the orientation specified by, the time (or time frame) specified by time index m, and the spectral bin specified by spectral bin index k.

[0026] According to one embodiment, the audio analyzer comprises: Index according to TIFF0007631273000020.tif429 Multiple composite volume values ​​L(m, The coefficient B specifies the total number of spectral bands b, TIFF0007631273000023.tif310 is the spectral band specified by index b, index Specifies the band volume value relative to the direction specified in TIFF0007631273000024.tif34 and the time (or time frame) specified by time index m.

[0027] According to one embodiment, the audio analyzer is configured to assign loudness contributions to histogram bins associated with different directions (e.g., as described above, a second different direction; e.g., a predefined direction) depending on the directional information to obtain an analysis result. The loudness contributions are represented, for example, by a plurality of composite loudness values ​​or a plurality of band loudness values. Thus, for example, the analysis result comprises a directional loudness map defined by the histogram bins. Each histogram bin is, for example, associated with one of the predefined directions.

[0028] According to one embodiment, the audio analyzer is configured to obtain loudness information related to the spectral bins based on the spectral domain representation (e.g., to obtain a composite loudness per T / F tile). The audio analyzer is configured to add a loudness contribution to one or more histogram bins based on the loudness information related to a given spectral bin. The loudness contribution associated with a given spectral bin is added to different histogram bins with, for example, different weights (e.g., depending on the direction corresponding to the histogram bin). The selection (i.e., addition) of the loudness contribution to one or more histogram bins is based on the determination of the directional information (i.e., the extracted directional value) of the given spectral bin. According to one embodiment, each histogram bin can represent a time-directional tile. Thus, a histogram bin is associated, for example, with the loudness of the combined two or more input audio signals in a particular time frame and direction. To determine the directional information of a given spectral bin, for example, the level information of the corresponding spectral bins of the spectral domain representation of the two or more input audio signals is analyzed.

[0029] According to one embodiment, the audio analyzer is configured to add loudness contributions to a number of histogram bins based on loudness information associated with a given spectral bin, such that a maximum contribution (e.g., a major contribution) is added to a histogram bin associated with a direction (i.e., of the extracted direction value) corresponding to the directional information associated with the given spectral bin, and a reduced contribution (e.g., relatively smaller than the maximum or major contribution) is added to one or more histogram bins associated with further directions (e.g., in the vicinity of the direction corresponding to the directional information associated with the given spectral bin). As mentioned above, each histogram bin can represent a time-directional tile. According to one embodiment, the multiple histogram bins can define a directional loudness map, which for example defines loudness in different directions over time for a combination of two or more input audio signals.

[0030] According to one embodiment, the audio analyzer is configured to obtain directional information based on the audio content of the two or more input audio signals. The directional information may for example include directions of components or sound sources within the audio content of the two or more input audio signals. In other words, the directional information may include panning directions or panning indexes of sound sources within a stereo mix of the two or more input audio signals.

[0031] According to one embodiment, the audio analyzer is configured to obtain the directional information based on an analysis of the amplitude panning of the audio content. Additionally or alternatively, the audio analyzer is configured to obtain the directional information based on an analysis of the phase relationship and / or the time delay and / or the correlation between the audio content of the two or more input audio signals. Additionally or alternatively, the audio analyzer is configured to obtain the directional information based on the identification of the expanded (e.g., decorrelated and / or panned) sound source. The analysis of the amplitude panning of the audio content can include an analysis of the level correlation between corresponding spectral bins of the spectral domain representation of the two or more input audio signals (e.g., corresponding spectral bins having the same level can be associated with the center direction of two speakers each transmitting one of the two input audio signals). Similarly, an analysis of the phase relationship and / or the time delay and / or the correlation between the audio contents can be performed. Thus, for example, the phase relationship and / or the time delay and / or the correlation between the audio contents are analyzed for corresponding spectral bins of the spectral domain representation of the two or more input audio signals. Additionally or alternatively, apart from inter-channel level / time difference comparison, there is a further (e.g. third) method for directional information estimation, which consists in matching the spectral information of the incident sound with pre-measured "template spectral responses / filters" of head-related transfer functions (HRFs) in different directions.

[0032] For example, in a particular time / frequency tile, the spectral envelope of the input signal at 35 degrees from the left and right channels may closely match the shape of linear filters for the left and right ears measured at an angle of 35 degrees. An optimization algorithm or pattern matching procedure then assigns the sound arrival direction to 35°. Further information can be found in https: / / iem.kug.ac.at / fileadmin / media / iem / projects / 2011 / baumgartner_robert.pdf (see, for example, Chapter 2). This method has the advantage that it makes it possible to estimate the arrival direction of ascending sound sources (sagittal plane) in addition to horizontal sound sources. This method is based, for example, on the comparison of spectral levels.

[0033] According to one embodiment, the audio analyzer is configured to spread the loudness information in multiple directions (e.g., beyond the direction indicated by the directional information) according to a spreading rule (e.g., a Gaussian spreading rule, or a limited discrete spreading rule). This means that, for example, loudness information corresponding to a particular spectral bin associated with a particular directional information may also contribute to neighboring directions (of the particular direction of the spectral bin) according to a spreading rule. According to one embodiment, the spreading rule may include or correspond to a direction-dependent weighting, where the direction-dependent weighting defines, for example, different weighting contributions of the loudness information of a particular spectral bin to multiple directions.

[0034] An embodiment according to the present invention relates to an audio similarity evaluator configured to obtain first loudness information (e.g., directional loudness map; e.g., one or more composite loudness values) related to different (e.g., panning) directions based on a first set of two or more input audio signals. The audio similarity evaluator is configured to compare the first loudness information with second (e.g., corresponding) loudness information (e.g., reference loudness information, reference directional loudness map, and / or reference composite loudness value) related to different (e.g., panning) directions and a set of two or more reference audio signals to obtain similarity information (e.g., "Model Output Variable" (MOV); e.g., a single scalar value) describing a similarity between the first set of two or more input audio signals and the set of two or more reference audio signals (or, e.g., representing a quality of the first set of two or more input audio signals when compared to the set of two or more reference audio signals).

[0035] This embodiment is based on the idea that comparing directional loudness information (e.g., first loudness information) of two or more input audio signals with directional loudness information (e.g., second loudness information) of two or more reference audio signals is efficient and improves the accuracy of the indication of audio quality (e.g., similarity information). The use of loudness information associated with different directions is particularly advantageous for stereo or multi-channel mixes, since the different directions can be associated with, for example, directions (i.e., panning directions, panning indexes) of sound sources (i.e., audio components) in the mix. Thus, the degradation of the quality of the processed combination of two or more input audio signals can be effectively measured. Another advantage is that the loudness information of the stereo or multi-channel image is determined, for example, in the short-time Fourier transform (STFT) domain, so that non-waveform-preserving audio processing, such as bandwidth extension (BWE), only minimally affects or does not affect the similarity information. Furthermore, the similarity information based on loudness information can be easily complemented with mono / temporal similarity information to improve the perceptual prediction of two or more input audio signals. Thus, for example, only one additional similarity information is used for the mono quality descriptor, which allows reducing the number of independent relevant signal features used by the objective audio quality measurement system with respect to known systems that use only mono quality descriptors. Using fewer features for the same performance reduces the risk of overfitting and indicates their higher perceptual relevance.

[0036] According to one embodiment, the audio similarity evaluator is configured to obtain the first loudness information (e.g., a directional loudness map) such that the first loudness information (e.g., a vector including composite loudness values ​​of a plurality of predefined directions) is associated with a first set of two or more input audio signals and includes a plurality of composite loudness values ​​associated with respective predefined directions, the composite loudness values ​​of the first loudness information describing the loudness of signal components of the first set of two or more input audio signals associated with respective predefined directions (e.g., each combined volume value is associated with a different direction). Thus, for example, each composite loudness value can be represented by a vector defining, for example, the change in loudness over time for a particular direction. This means, for example, that one composite loudness value can include one or more volume values ​​associated with successive time frames. The predefined directions can be represented by panning directions / panning indexes of the signal components of the first set of two or more input audio signals. Thus, for example, the predefined directions can be predefined by an amplitude laser panning technique used for positioning of directional signals in a stereo or multi-channel mix represented by the first set of two or more input audio signals.

[0037] According to one embodiment, the audio similarity evaluator is configured to obtain the first loudness information (e.g., a directional loudness map) such that the first loudness information is associated with a combination of multiple weighted spectral domain representations of (e.g., of each audio signal) of a first set of two or more input audio signals, each associated with a respective predefined direction (e.g., each composite loudness value and / or weighted spectral domain representation is associated with a different predefined direction). This means, for example, that for each input audio signal, at least one weighted spectral domain representation is calculated and then all weighted spectral domain representations associated with the same predefined direction are combined. Thus, the first loudness information represents, for example, loudness values ​​associated with multiple spectral bins associated with the same predefined direction. At least some of the multiple spectral bins are, for example, weighted differently from other bins of the multiple spectral bins.

[0038] According to one embodiment, the audio similarity evaluator is configured to determine a difference between the second loudness information and the first loudness information to obtain a residual loudness information. According to one embodiment, the residual loudness information can represent similarity information or the similarity information can be determined based on the residual loudness information. The residual loudness information can be understood, for example, as a measure of the distance between the second loudness information and the first loudness information. The residual loudness information can therefore be understood as a directional loudness distance (for example, DirLoudDist). This feature allows a very efficient determination of the quality of two or more input audio signals relative to the first loudness information.

[0039] According to one embodiment, the audio similarity evaluator is configured to determine a value (e.g., a single scalar value) that quantifies the difference across multiple directions (and optionally also over time, e.g., over multiple frames). The audio similarity evaluator is configured to determine, for example, the average of the magnitude of the residual loudness information across all directions (e.g., panning directions) and over time as a value that quantifies the difference. This determines, for example, a single number called a Model Output Variable (MOV), which defines the similarity of a first set of two or more input audio signals to a set of two or more reference audio signals.

[0040] According to one embodiment, the audio similarity evaluator is configured to obtain the first loudness information and / or the second loudness information (e.g. as a directional loudness map) using an audio analyzer according to one of the embodiments described herein.

[0041] According to one embodiment, the audio similarity evaluator is configured to use metadata representing position information of speakers associated with the input audio signals to obtain directional components (e.g., directional information) used to obtain loudness information associated with different directions (e.g., one or more directional loudness maps). The different directions are not necessarily associated with directional components. According to one embodiment, the directional components are associated with two or more input audio signals. Thus, the directional components can represent, for example, speaker identifiers or channel identifiers dedicated to different directions or positions of the speakers. Conversely, the different directions to which the loudness information is associated can represent directions or positions of audio components of an audio scene realized by the two or more input audio signals. Alternatively, the different directions can represent equally spaced directions or positions within a position interval (e.g., [-1;1], where -1 represents a signal panned completely to the left and +1 represents a signal panned completely to the right) in which the audio scene realized by the two or more input audio signals can unfold. According to one embodiment, the different directions can be associated with predefined directions as described herein. The directional components are, for example, mapped to boundary points of the position interval.

[0042] An embodiment according to the invention relates to an audio encoder for encoding an input audio content comprising one or more input audio signals (preferably a plurality of input audio signals). The audio encoder is configured to provide one or more encoded (e.g. quantized and then losslessly encoded) audio signals (e.g. encoded spectral domain representations) based on the one or more input audio signals (e.g. left and right signals) or one or more signals derived therefrom (e.g. intermediate or downmix signals and side or difference signals). Furthermore, the audio encoder is configured to adapt encoding parameters (e.g. quantization parameters, for example, to provide one or more encoded audio signals) depending on one or more directional loudness maps representing loudness information related to a plurality of different directions (e.g. panning directions) of the one or more signals to be encoded (e.g. depending on the contribution of the individual directional loudness maps of the one or more signals to be quantized to an overall directional loudness map associated with the plurality of input audio signals (e.g. each signal of the one or more input audio signals).

[0043] Audio content comprising one input audio signal may be associated with a mono audio scene, audio content comprising two input audio signals may be associated with a stereo audio scene and audio content comprising more than two input audio signals may be associated with a multi-channel audio scene. According to an embodiment, the audio encoder provides as output signals a separate encoded audio signal for each input audio signal or one combined output signal comprising two or more encoded audio signals of the two or more input audio signals.

[0044] The directional loudness map (i.e., DirLoudMap) on which the adaptation of the coding parameters depends may vary for different audio contents. Thus, in the case of a mono audio scene, the directional loudness map includes only one directional loudness value that deviates from 0 (based on only one input audio signal) and includes all other directional loudness values ​​that are, for example, equal to 0. In the case of a stereo audio scene, the directional loudness map represents, for example, loudness information related to both input audio signals, and the different directions relate, for example, to the positions or directions of the audio components of the two input audio signals. In the case of three or more input audio signals, the adaptation of the coding parameters depends, for example, on three or more directional loudness maps, each directional loudness map corresponding to loudness information related to two of the three input audio signals (for example, a first DirLoudMap may correspond to the first and second input audio signals, a second DirLoudMap may correspond to the first and third input audio signals, and a third DirLoudMap may correspond to the second and third input audio signals). As explained for a stereo audio scene, different directions of the directional loudness map are associated with positions or directions of audio components of multiple input audio signals, e.g. in the case of a multi-channel audio scene.

[0045] This embodiment of the audio encoder is based on the idea that relying on the adaptation of the coding parameters to one or more directional loudness maps is efficient and improves the coding accuracy. The coding parameters are adapted, for example, depending on the difference between the directional loudness map associated with one or more input audio signals and the directional loudness map associated with one or more reference audio signals. According to an embodiment, the overall directional loudness map of the combination of all input audio signals and the combination of all reference audio signals is compared, or the directional loudness map of the individual or pair of signals is compared with the overall directional loudness map of all input audio signals (e.g., two or more differences can be determined). The difference between the DirLoudMaps can represent a measure of the coding quality. Thus, the coding parameters are adapted, for example, such that the difference is minimized to ensure a high quality coding of the audio content, or the coding parameters are adapted such that only signals of the audio content that correspond to a difference below a certain threshold are coded, to reduce the coding complexity. Alternatively, the coding parameters are adapted, for example, depending on the ratio (e.g., contribution) of the individual signal DirLoudMaps or signal pair DirLoudMaps to the overall DirLoudMap (e.g., the DirLoudMap associated to the combination of all input audio signals). This ratio may indicate the similarity between individual signals or signal pairs of the audio content, or between individual signals and combinations or signal pairs of all signals of the audio content, and result in higher quality encoding and / or reduced encoding complexity.

[0046] According to an embodiment, the audio encoder is configured to adapt a bit distribution between one or more signals and / or parameters to be encoded (e.g. between a residual signal and a downmix signal, or between a left channel signal and a right channel signal, or between two or more signals provided by a joint coding of a plurality of signals, or between a parameter and a signal provided by a joint coding of a plurality of signals) to the overall directional loudness map depending on the contribution of the individual directional loudness map of the one or more signals and / or parameters to be encoded (or between two or more signals and / or parameters to be encoded, for example). The adaptation of the bit distribution is for example understood as an adaptation of the coding parameters by the audio encoder. The bit distribution can also be understood as a bit rate distribution. The bit distribution is for example adapted by controlling the quantization precision of one or more input audio signals of the audio encoder. According to an embodiment, a high contribution may indicate a high relevance of a corresponding input audio signal or input audio signal pair for a high quality perception of an audio scene generated by the audio content. Thus, for example, the audio encoder may be configured to provide more bits for signals with high contribution and few or no bits for signals with low contribution. This allows for an efficient and high quality coding to be achieved.

[0047] According to one embodiment, the audio encoder is configured to disable the encoding of a given one of the signals to be encoded (e.g., the residual signal) when the contribution of the individual directional loudness map (e.g., the residual signal) of a given one of the signals to be encoded to the overall directional loudness map falls below a (e.g., predetermined) threshold. For example, the encoding is disabled if the average ratio or the ratio of the directions of the maximum relative contribution falls below a threshold. Alternatively or additionally, the contribution of the directional loudness map of a signal pair (e.g., the individual directional loudness maps of a signal pair (e.g., as a signal pair, a combination of two signals can be understood. For example, as a signal pair, a combination of signals related to different channels and / or residual signals and / or downmix signals can be understood.)) to the overall directional loudness map can be used by the encoder to disable the encoding of a given one of the signals (e.g., for three signals to be encoded, the three directional loudness maps of the signal pair can be analyzed with respect to the overall directional loudness map as described above. Thus, the encoder can be configured to determine the signal pair with the highest contribution to the overall directional loudness map and to encode only these two signals and to disable the encoding of the remaining signals.). Disabling the coding of a signal is understood, for example, as an adaptation of coding parameters. Thus, signals that are less relevant for the listener's perception of the audio content do not need to be coded, resulting in a very efficient coding. According to an embodiment, the threshold can be set to 5%, 10%, 15%, 20% or 50% or less of the loudness information of the global directional loudness map.

[0048] According to one embodiment, the audio encoder is configured to adapt the quantization precision of one or more signals to be encoded (e.g. between the residual signal and the downmix signal) depending on the contribution of the individual directional loudness map of the (respective) one or more signals to be encoded to the overall directional loudness map. Alternatively or additionally, similar to the nulling described above, the contribution of the directional loudness map of a signal pair to the overall directional loudness map can be used by the encoder to adapt the quantization precision of one or more signals to be encoded. The adaptation of the quantization precision can be understood as an example for adapting encoding parameters by an audio encoder.

[0049] According to one embodiment, the audio encoder is configured to quantize a spectral domain representation of one or more input audio signals (e.g. a left signal and a right signal: e.g. the one or more input audio signals correspond e.g. to a plurality of different channels. Thus, the audio encoder receives a multi-channel input) or one or more signals derived therefrom (e.g. an intermediate signal or a downmix signal and a side signal or a difference signal) using one or more quantization parameters (e.g. a scale factor or parameter describing which quantization precision or quantization step should be applied to which spectral bin or frequency band of the one or more signals to be quantized) to obtain one or more quantized spectral domain representations. The audio encoder is configured to adjust the one or more quantization parameters (e.g. to adapt the bit distribution among the one or more signals to be encoded) to adapt the one or more quantization parameters (e.g. in accordance with the contribution of the individual directional loudness maps of the one or more signals to be quantized to an overall directional loudness map associated with the plurality of input audio signals (e.g. each signal of the one or more input audio signals)) in response to one or more directional loudness maps representing loudness information related to a plurality of different directions (e.g. panning directions) of the one or more signals to be quantized. Furthermore, the audio encoder is configured to encode the one or more quantized spectral domain representations to obtain one or more encoded audio signals.

[0050] According to one embodiment, the audio encoder is configured to adjust one or more quantization parameters depending on the contribution of individual directional loudness maps of one or more signals to be quantized to an overall directional loudness map.

[0051] According to one embodiment, the audio encoder is configured to determine an overall directional loudness map based on the input audio signals, such that the overall directional loudness map represents loudness information related to different directions (e.g. audio components; e.g. panning directions) of an audio scene represented by the input audio signals (or to be represented, e.g. after decoder-side rendering) (possibly in combination with knowledge or side information regarding loudspeaker positions and / or knowledge or side information describing audio object positions). The overall directional loudness map represents, for example, the (e.g. combined) loudness information related to all input audio signals.

[0052] According to one embodiment, the one or more signals to be quantized are associated (e.g., in a fixed, signal-independent manner) with different directions (e.g., first different directions), or with different speakers (e.g., at different pre-defined speaker positions), or with different audio objects (e.g., audio objects that are rendered at different positions according to, for example, object rendering information, such as a panning index).

[0053] According to one embodiment, the signal to be quantized comprises components of a joint multi-signal coding of two or more input audio signals, for example a mid signal and a side signal of a mid-side stereo coding.

[0054] According to one embodiment, the audio encoder is configured to estimate a contribution of a residual signal of the joint multi-signal coding to a global directional loudness map and to adjust one or more quantization parameters accordingly, the estimated contribution being represented, for example, by a contribution of the directional loudness map of the residual signal to the global directional loudness map.

[0055] According to an embodiment, the audio encoder is configured to adapt the bit distribution between one or more signals and / or parameters to be encoded separately for different spectral bins or separately for different frequency bands. Additionally or alternatively, the audio encoder is configured to adapt the quantization precision of one or more signals to be encoded separately for different spectral bins or separately for different frequency bands. By adapting the quantization precision, the audio encoder is configured to, for example, also adapt the bit distribution. Thus, the audio encoder is configured to, for example, adapt the bit distribution between one or more input audio signals of an audio content to be encoded by the audio encoder. Additionally or alternatively, the bit distribution between the parameters to be encoded is adapted. The adaptation of the bit distribution can be performed by the audio encoder separately for different spectral bins or separately for different frequency bands. According to an embodiment, it is also possible that the bit distribution between the signal and the parameter is adapted. In other words, each signal of the one or more signals to be encoded by the audio encoder can include an individual bit distribution for different spectral bins and / or different frequency bands (e.g. of the corresponding signal), and this individual bit distribution for each of the one or more signals to be encoded can be adapted by the audio encoder.

[0056] According to an embodiment, the audio encoder is configured to adapt the bit distribution between one or more signals and / or parameters to be encoded (e.g., separately for each spectral bin or for each frequency band) depending on an evaluation of spatial masking between two or more signals to be encoded. Furthermore, the audio encoder is configured to evaluate spatial masking based on a directional loudness map associated with the two or more signals to be encoded. This is based on the idea that, for example, the directional loudness map is decomposed spatially and / or temporally. Thus, for example, few bits or no bits are spent on the masked signal, and more bits (e.g., more than the masked signal) are spent on the encoding of related signals or signal components (e.g., signals or signal components that are not masked by other signals or signal components). According to an embodiment, the spatial masking depends, for example, on the levels associated with the spectral bins and / or frequency bands of the two or more signals to be encoded, on the spatial distance between the spectral bins and / or frequency bands, and / or on the temporal distance between the spectral bins and / or frequency bands. Directional loudness maps can directly provide loudness information for individual spectral bins and / or frequency bands of individual signals or combinations of signals (eg, signal pairs), resulting in efficient analysis of spatial masking by the encoder.

[0057] According to one embodiment, the audio encoder is configured to evaluate the masking effect of a loudness contribution related to a first direction of a first signal to be encoded against a loudness contribution related to a second direction, different from the first direction, of a second signal to be encoded (e.g. the masking effect decreases as the difference in angle increases). The masking effect defines for example the relevance of spatial masking. This means for example that for a loudness contribution related to a masking effect lower than a threshold, more bits are spent than a signal related to a masking effect higher than the threshold (e.g. a spatially masked signal). According to one embodiment, the threshold can be defined as a masking of 20%, 50%, 60%, 70% or 75% of the total masking. This means for example that the masking effect of neighboring spectral bins or frequency bands is evaluated depending on the loudness information of the directional loudness map.

[0058] According to one embodiment, the audio encoder comprises an audio analyzer according to one of the embodiments described in this specification, and the volume information associated with different directions (e.g., a “directional volume map”) forms a directional volume map.

[0059] According to one embodiment, the audio encoder is configured to adapt the noise (e.g., quantization noise) introduced by the encoder depending on one or more directional loudness maps. Thus, for example, one or more directional loudness maps of one or more signals to be encoded can be compared by the encoder with one or more directional loudness maps of one or more reference signals. Based on this comparison, the audio encoder is configured, for example, to evaluate a difference indicative of the introduced noise. The noise can be adapted by adapting the quantization performed by the audio encoder.

[0060] According to one embodiment, the audio encoder is configured to use the deviation between the directional loudness map associated with a given uncoded input audio signal (or a given uncoded input audio signal pair) and the directional loudness map achievable by the encoded version of the given input audio signal (or a given input audio signal pair) as a criterion (e.g., a target criterion) for adapting the provision of the given encoded audio signal (or a given encoded audio signal pair). The following examples are only described for one given uncoded input audio signal, but it is clear that they are also applicable to a given uncoded input audio signal pair. The directional loudness map associated with a given uncoded input audio signal can be associated with or can represent a reference directional loudness map. Thus, the deviation between the reference directional loudness map and the directional loudness map of the encoded version of the given input audio signal can be indicative of noise introduced by the encoder. To reduce the noise, the audio encoder can be configured to adapt the encoding parameters to reduce the deviation in order to provide a high quality encoded audio signal. This is for example achieved by a feedback loop that controls on a deviation basis. Thus, the encoding parameters are adapted until the deviation is below a predefined threshold. According to one embodiment, the threshold can be defined as a deviation of 5%, 10%, 15%, 20% or 25%. Alternatively, the adaptation by the encoder is performed using a neural network (e.g., implementing a feed-forward loop). Using a neural network, the directional loudness map of the encoded version of a given input audio signal can be estimated without being determined directly by an audio encoder or an audio analyzer. This allows for very fast and accurate audio coding.

[0061] According to one embodiment, the audio encoder is configured to activate and deactivate a joint coding tool (e.g., to jointly code two or more of the input audio signal or signals derived from the input audio signal) (e.g., to decide on / off the M / S (Middle / Side signal)) depending on one or more directional loudness maps representing loudness information related to multiple different directions of the signal or signals to be encoded. To activate or deactivate the joint coding tool, the audio encoder can be configured to determine the contribution of the directional loudness map of each signal or each candidate signal pair to the overall directional loudness map of the entire scene. According to one embodiment, a contribution higher than a threshold (e.g., a contribution of at least 10% or at least 20% or at least 30% or at least 50%) indicates whether joint coding of the input audio signals is reasonable. For example, the threshold may be relatively low for this use case (e.g., lower than other use cases) to exclude mainly irrelevant pairs. Based on the directional loudness map, the audio encoder can check whether joint coding of the signals results in more efficient and / or view-bit high resolution coding.

[0062] According to an embodiment, the audio encoder is configured to determine one or more parameters of a joint coding tool (e.g., for jointly coding two or more of the input audio signal or signals derived from the input audio signal) depending on one or more directional loudness maps representing loudness information related to multiple different directions of the one or more signals to be encoded (e.g., for setting parameters of an "intensity stereo" joint coding tool, for example, for controlling smoothing of frequency-dependent prediction coefficients). The one or more directional loudness information maps include, for example, information on loudness in a given direction and time frame. Thus, for example, the audio encoder is configured to determine one or more parameters of a current time frame based on loudness information of a previous time frame. Based on the directional loudness maps, a masking effect can be very efficiently analyzed and indicated by one or more parameters, whereby frequency-dependent prediction coefficients can be determined based on the one or more parameters such that predicted sample values ​​are close to original sample values ​​(related to the signal to be encoded). Thus, the encoder is capable of determining frequency-dependent prediction coefficients representing an approximation of a masking threshold rather than the signal to be encoded. Furthermore, the directional loudness map is based, for example, on a psychoacoustic model, whereby the determination of the frequency-dependent prediction coefficients based on the one or more parameters can be further improved, resulting in a highly accurate prediction. Alternatively, the parameters of the joint coding tool define, for example, which signals or signal pairs should be coded together by the audio encoder. The audio encoder is configured to make the determination of the one or more parameters based, for example, on the contribution of each directional loudness map associated with the signal to be coded or the signal pair of the signal to be coded to the overall directional loudness map. Thus, for example, the one or more parameters indicate individual signals and / or signal pairs having a maximum contribution or a contribution equal to or greater than a threshold (see, for example, the definition of threshold above).Based on the one or more parameters, the audio encoder is configured to, for example, jointly code the signals indicated by the one or more parameters. Alternatively, for example, signal pairs having high proximity / similarity in their respective directional loudness maps can be indicated by one or more parameters of a joint coding tool. The selected signal pairs are, for example, jointly represented by a downmix. Thus, the downmix signal or residual signal of the signals to be jointly coded is very small, so that the bits required for coding are minimized or reduced.

[0063] According to one embodiment, the audio encoder is configured to determine or estimate an influence of a variation of one or more control parameters controlling the provision of one or more encoded signals to a directional loudness map of the one or more encoded signals, and adjust the one or more control parameters in response to the determination or estimation of the influence. The influence of the control parameters on the directional loudness map of the one or more encoded signals may include a measure of induced noise due to the encoding of the audio encoder (e.g., a control parameter relating to the quantization positions may be adjusted), a measure of audio distortion, and / or a measure of listener perception degradation. According to one embodiment, the control parameters may be represented by or may include the encoding parameters.

[0064] According to one embodiment, the audio encoder is configured to use metadata representing position information of the speakers associated with the input audio signal to obtain directional components (e.g., directional information) used to obtain one or more directional loudness maps (this concept can also be used in other audio encoders). The directional components are represented, for example, by first different directions as described herein associated with different channels or speakers associated with the input audio signal. According to one embodiment, based on the directional components, the obtained one or more directional loudness maps can be associated with the input audio signal and / or with a signal pair of input audio signals having the same directional component. Thus, for example, the directional loudness map can have an index L, and the input audio signal can have an index L, where L indicates a signal for the left channel or left speaker. Alternatively, the directional components can be represented by a vector such as (1,3) indicating a combination of the input audio signals of the first and third channels. Thus, a directional loudness map with index (1,3) can be associated with this signal pair. According to one embodiment, each channel can be associated with a different speaker.

[0065] One embodiment according to the invention relates to an audio encoder for encoding input audio content comprising one or more input audio signals (preferably a plurality of input audio signals). The audio encoder is configured to provide one or more encoded (e.g., quantized and then losslessly encoded) audio signals (e.g., encoded spectral domain representations) of the two or more signals to be encoded together (e.g., using an intermediate signal or a downmix signal and a side signal or a difference signal) based on two or more input audio signals (e.g., a left signal and a right signal) or based on two or more signals derived therefrom. Further, the audio encoder is configured to select a signal to be encoded together from among the multiple candidate signals or from among the multiple pairs of candidate signals (e.g., from the two or more input audio signals or from two or more signals derived from the two or more input audio signals) in response to a directional loudness map representing loudness information related to a plurality of different directions (e.g., panning directions) of the candidate signal or pair of candidate signals (e.g., in response to a contribution of an individual directional loudness map of the candidate signals to an overall directional loudness map (e.g. associated with all of the input audio signals) associated with the multiple input audio signals (e.g., each signal of the one or more input audio signals) or in response to a contribution of the directional loudness map of the pair of candidate signals to the overall directional loudness map).

[0066] According to an embodiment, the audio encoder can be configured to activate and deactivate the joint coding. Thus, for example, if the audio content includes only one input audio signal, the joint coding is deactivated and is activated only if the audio content includes two or more input audio signals. Thus, it is possible to use the audio encoder to encode mono audio content, stereo audio content, and / or audio content including three or more input audio signals (i.e., multi-channel audio content). According to an embodiment, the audio encoder provides for each input audio signal a separate encoded audio signal as an output signal (e.g., suitable for audio content including only one single input audio signal) or provides one combined output signal (e.g., a jointly encoded signal) including two or more encoded audio signals of the two or more input audio signals.

[0067] This embodiment of the audio encoder is based on the idea that joint coding based on directional loudness maps is efficient and improves the accuracy of coding. The use of directional loudness maps is advantageous since it can indicate the perception of the audio content by the listener and thus improve the audio quality of the coded audio content, especially in the context of joint coding. For example, by analyzing the directional loudness maps, it is possible to optimize the selection of signal pairs to be coded together. The analysis of the directional loudness maps gives information about signals or signal pairs that are, for example, negligible (e.g., signals that have little effect on the listener's perception), resulting in a small amount of bits required for the coded audio content (e.g., including two or more coded signals) by the audio encoder. This means, for example, that signals that have a low contribution of their respective directional loudness maps to the overall directional loudness map can be ignored. Alternatively, the analysis can indicate signals that have a high degree of similarity (e.g., signals with similar directional loudness maps), thereby optimizing the residual signal, for example, by joint coding.

[0068] According to one embodiment, the audio encoder is configured to select a signal to be jointly encoded from a plurality of candidate signals or from a plurality of pairs of candidate signals (e.g. associated with a plurality of input audio signals (e.g. each of one or more input audio signals)) depending on the contribution of the individual directional loudness maps of the candidate signals to the overall directional loudness map or depending on the contribution of the directional loudness map of the pair of candidate signals to the overall directional loudness map. The overall directional loudness map represents, for example, loudness information related to different directions (e.g. audio components) of the audio scene represented by the input audio signals (or to be represented, for example after decoder-side rendering) (possibly in combination with knowledge or side information regarding the positions of speakers and / or knowledge or side information describing the positions of audio objects).

[0069] According to one embodiment, the audio encoder is configured to determine the contribution of the pair of candidate signals to the overall directional loudness map. Furthermore, the audio encoder is configured to determine the contribution of the pair of candidate signals to the overall directional loudness map, and the audio encoder is configured to select one or more pairs of candidate signals with the largest contribution to the overall directional loudness map for joint encoding, or the audio encoder is configured to select one or more pairs of candidate signals with a contribution to the overall directional loudness map greater than a predefined threshold for joint encoding (e.g., at least 60%, 70%, 80% or 90% contribution). With respect to the largest contribution, it is possible that only one pair of candidate signals has the largest contribution, but it is also possible that two or more pairs of candidate signals have the same contribution, which represents the largest contribution, or two or more pairs of candidate signals have similar contributions within a small variance of the largest contribution. Thus, the audio encoder is configured, for example, to select two or more signals or signal pairs for joint encoding. The features described in this embodiment make it possible to find relevant signal pairs for improved joint encoding and to discard signals or signal pairs that do not significantly affect the perception of the encoded audio content by the listener.

[0070] According to one embodiment, the audio encoder is configured to determine individual directional loudness maps of two or more candidate signals (e.g., directional loudness maps associated with a signal pair). Furthermore, the audio encoder is configured to compare the individual directional loudness maps of the two or more candidate signals and select two or more of the candidate signals for joint encoding depending on the result of the comparison (e.g., such that the candidate signal (e.g., a signal pair, a signal triplet, a signal quadruplet, etc.) whose individual loudness maps include a similarity higher than a maximum similarity or a similarity threshold is selected for joint encoding). Thus, for example, few or no bits are spent on the residual signal (e.g., a side channel relative to a middle channel) that maintains a high quality of the encoded audio content.

[0071] According to one embodiment, the audio encoder is configured to determine a global directional loudness map using a downmix of the input audio signals and / or using a binauralization of the input audio signals, the downmix or binauralization assuming, for example, a direction (e.g., an association with a channel or speaker for each input audio signal). The global directional loudness map can be associated with loudness information corresponding to an audio scene created by all the input audio signals.

[0072] An embodiment according to the invention relates to an audio encoder for encoding an input audio content comprising one or more input audio signals (preferably a plurality of input audio signals). The audio encoder is configured to provide one or more encoded (e.g. quantized and then losslessly encoded) audio signals (e.g. encoded spectral domain representations) based on two or more input audio signals (e.g. left and right signals) or based on two or more signals derived therefrom. Furthermore, the audio encoder is configured to determine an overall directional loudness map (e.g. a target directional loudness map of a scene) based on the input audio signals and / or to determine one or more individual directional loudness maps associated with an individual input audio signal (or associated with two or more input audio signals such as a signal pair). Furthermore, the audio encoder is configured to encode the overall directional loudness map and / or the one or more individual directional loudness maps as side information.

[0073] Thus, for example, if the audio content includes only one input audio signal, the audio encoder is configured to encode only this signal together with the corresponding individual directional loudness map. If the audio content includes two or more input audio signals, the audio encoder is configured to encode, for example, all or at least some (e.g., one individual signal and one signal pair of three input audio signals) signals individually together with their respective directional loudness maps (e.g., individual directional loudness maps of the individual encoded signals, and / or directional loudness maps corresponding to signal pairs or other combinations of the three or more signals, and / or an overall directional loudness map associated with all input audio signals). According to an embodiment, the audio encoder is configured to encode, for example, all or at least some signals resulting in one encoded audio signal together with an overall directional loudness map as output (e.g., one combined output signal (e.g., jointly encoded signal) including two or more encoded audio signals of the two or more input audio signals). Thus, it is possible to encode audio content including mono audio content, stereo audio content, and / or three or more input audio signals (i.e., multi-channel audio content) using the audio encoder.

[0074] This embodiment of the audio encoder is based on the idea that it is advantageous to determine and encode one or more directional loudness maps, since they indicate the perception of the audio content by the listener and thus can improve the audio quality of the encoded audio content. According to one embodiment, the one or more directional loudness maps can be used by the encoder to improve the encoding, for example by adapting the encoding parameters based on the one or more directional loudness maps. The encoding of one or more directional loudness maps is therefore particularly advantageous, since it can represent information about the effect of the encoding. With the one or more directional loudness maps as side information in the encoded audio content provided by the audio encoder, a very accurate decoding can be achieved, since information about the encoding is provided by the audio encoder (for example in the data stream).

[0075] According to one embodiment, the audio encoder is configured to determine an overall directional loudness map based on the input audio signal, such that the overall directional loudness map represents loudness information related to different directions (e.g. audio components) of the audio scene represented by the input audio signal (or to be represented, e.g. after decoder-side rendering) (possibly in combination with knowledge or side information regarding loudspeaker positions and / or knowledge or side information describing positions of audio objects). The different directions of the audio scene may represent, for example, second different directions as described herein.

[0076] According to an embodiment, the audio encoder is configured to encode the global directional loudness map in the form of a set of (e.g., scalar) values ​​associated with different directions (preferably in multiple frequency bins or frequency bands). If the global directional loudness map is encoded in the form of a set of values, the value associated with a particular direction may include loudness information for multiple frequency bins or frequency bands. Alternatively, the audio encoder is configured to encode the global directional loudness map using a center position value (e.g., describing an angle or panning index at which a maximum value of the global directional loudness map occurs for a given frequency bin or frequency band) and gradient information (e.g., one or more scalar values ​​describing a gradient of the values ​​of the global directional loudness map in an angle or panning index direction). The encoding of the global directional loudness map using the center position value and the gradient information may be performed for different given frequency bins or frequency bands. Thus, for example, the global directional loudness map may include information of center position values ​​and gradient information for two or more frequency bins or frequency bands. Alternatively, the audio encoder is configured to encode the global directional loudness map in the form of a polynomial representation, or the audio encoder is configured to encode the global directional loudness map in the form of a spline representation. The coding of the global directional loudness map in the form of a polynomial or spline representation is a cost-effective coding. Although these features are described with respect to the global directional loudness map, this coding can also be performed for individual directional loudness maps (e.g., individual signals, signal pairs, and / or groups of three or more signals). These features therefore make the directional loudness map very efficient to code and provide the information on which the coding is based.

[0077] According to an embodiment, the audio encoder is configured to encode (e.g., transmit or include in the encoded audio representation) one (e.g., only one) downmix signal obtained based on the multiple input audio signals and the global directional loudness map. Alternatively, the audio encoder is configured to encode (e.g., transmit or include in the encoded audio representation) multiple signals (e.g., input audio signals or signals derived from the input audio signals) and to encode (e.g., transmit or include in the encoded audio representation) individual directional loudness maps of the multiple signals to be encoded (e.g., directional loudness maps of individual signals and / or pairs of signals and / or groups of three or more signals). Alternatively, the audio encoder is configured to encode (e.g., transmit or include in the encoded audio representation) the global directional loudness map, the multiple signals, e.g., input audio signals or signals derived therefrom, and the contributions to be encoded to the global directional loudness map, e.g., parameters describing (e.g., relative) contributions of the signals. According to an embodiment, the parameters describing the contributions can be represented by scalar values. Thus, by an audio decoder receiving an encoded audio representation (e.g., audio content or a data stream including an encoded signal, an overall directional loudness map, and parameters), it is possible to reconstruct the individual directional loudness maps of the signals based on the overall directional loudness map and the parameters describing the contributions of the signals.

[0078] An embodiment according to the invention relates to an audio decoder for decoding encoded audio content. The audio decoder is configured to receive an encoded representation of one or more audio signals and provide a decoded representation of the one or more audio signals (e.g., using AAC-like decoding or using decoding of entropy-coded spectral values). Furthermore, the audio decoder is configured to receive encoded directional loudness map information and decode the encoded directional loudness map information to obtain one or more (e.g., decoded) directional loudness maps. Furthermore, the audio decoder is configured to use the decoded representation of the one or more audio signals and to reconstruct an audio scene using the one or more directional loudness maps. The audio content may include an encoded representation of one or more audio signals and the encoded directional loudness map information. The encoded directional loudness map information may include directional loudness maps of individual signals, signal pairs, and / or groups of three or more signals.

[0079] This embodiment of the audio decoder is based on the idea that it is advantageous to determine and decode one or more directional loudness maps, since they indicate the perception of the audio content by the listener and thus can improve the audio quality of the decoded audio content. The audio decoder is for example configured to determine a high-quality prediction signal based on one or more directional loudness maps, thereby improving the residual decoding (or joint decoding). According to an embodiment, the directional loudness map defines loudness information of different directions in the audio scene over time. The loudness information of a particular direction at a particular point in time or in a particular time frame can for example include loudness information of different audio signals or one audio signal in different frequency bins or frequency bands. Thus, for example, the provision of a decoded representation of one or more audio signals by the audio decoder can be improved, for example, by adapting the decoding of the encoded representation of one or more audio signals based on the decoded directional loudness map. Thus, the reconstructed audio scene is optimized, since the decoded representation of one or more audio signals can achieve a minimum deviation with respect to the original audio signal based on the analysis of one or more directional loudness maps, resulting in a high-quality audio scene. According to one embodiment, an audio decoder may be configured to use one or more directional loudness maps for adaptation of decoding parameters to provide an efficient and accurate decoded representation of one or more audio signals.

[0080] According to one embodiment, the audio decoder is configured to obtain the output signal such that one or more directional loudness maps associated with the output signal approximate or are equal to one or more target directional loudness maps. The one or more target directional loudness maps are based on or equal to one or more decoded directional loudness maps. The audio decoder is configured, for example, to use a suitable scaling or combination of one or more decoded audio signals to obtain the output signal. The target directional loudness map is, for example, understood as a reference directional loudness map. According to one embodiment, the target directional loudness map may represent loudness information of one or more audio signals prior to encoding and decoding of the audio signals. Alternatively, the target directional loudness map may represent loudness information related to the encoded representation of one or more audio signals (for example, one or more decoded directional loudness maps). The audio decoder receives, for example, encoding parameters used for encoding to provide the encoded audio content. The audio decoder is configured, for example, to determine decoding parameters based on the encoding parameters, for example, to scale one or more decoded directional loudness maps to determine one or more target directional loudness maps. The audio decoder may also comprise an audio analyzer configured to determine a target directional loudness map based on the decoded directional loudness map and the one or more decoded audio signals, for example the decoded directional loudness map is scaled based on the one or more decoded audio signals. Since the one or more target directional loudness maps may be associated with an optimal or optimized audio scene realized by the audio signals, it is advantageous to minimize the deviation between the one or more directional loudness maps associated with the output signal and the one or more target directional loudness maps.According to one embodiment, this deviation can be minimized by the audio decoder by adapting the decoding parameters or by adapting the parameters related to the reconstruction of the audio scene. Thus, with this feature, the quality of the output signal is controlled, for example, by a feedback loop that analyzes one or more directional loudness maps associated with the output signal. The audio decoder is, for example, configured to determine one or more directional loudness maps of the output signal (for example, the audio decoder comprises an audio analyzer as described herein for determining the directional loudness maps). Thus, the audio decoder provides an output signal associated with a directional loudness map that is close to or equal to the target directional loudness map.

[0081] According to one embodiment, the audio decoder is configured to receive one (e.g. only one) encoded downmix signal (e.g. obtained based on multiple input audio signals) and an overall directional loudness map, or multiple encoded audio signals (e.g. input audio signals of an encoder or signals derived therefrom) and individual directional loudness maps of the multiple encoded signals, or an overall directional loudness map, multiple encoded audio signals (e.g. input audio signals received by an audio encoder or signals derived therefrom) and parameters describing the (e.g. relative) contributions of the encoded audio signals to the overall directional loudness map. The audio decoder is configured to provide an output signal based thereon.

[0082] An embodiment according to the invention relates to a format converter for converting a format of an audio content representing an audio scene (e.g. a spatial audio scene) from a first format to a second format. The first format may for example comprise a first number of channels or input audio signals and side information or spatial side information adapted to the first number of channels or input audio signals, and the second format may for example comprise a second number of channels or output audio signals which may differ from the first number of channels or input audio signals and side information or spatial side information adapted to the second number of channels or output audio signals. Furthermore, the format converter is configured to provide a representation of the audio content in the second format based on the representation of the audio content in the first format. Furthermore, the format converter is configured to adjust the complexity of the format conversion (e.g., by skipping one or more of the input audio signals of the first format that contribute to a directional loudness map below a threshold in the format conversion process) depending on the contribution of the input audio signals of the first format (e.g., one or more audio signals, one or more downmix signals, one or more residual signals, etc.) to the overall directional loudness map of the audio scene (the overall directional loudness map may, for example, be described by the side information of the first format received by the format converter). Thus, for example, for the format conversion complexity adjustment, the contribution of the individual directional loudness maps associated with the individual input audio signals to the overall directional loudness map of the audio scene is analyzed. Alternatively, this adjustment can be performed by the format converter depending on the contribution of the directional loudness maps corresponding to combinations of input audio signals (e.g., signal pairs, intermediate signals, side signals, downmix signals, residual signals, difference signals, and / or groups of three or more signals) to the overall directional loudness map of the audio scene.

[0083] An embodiment of the format converter is based on the idea that it is advantageous to convert the format of the audio content based on one or more directional loudness maps, since this can indicate the perception of the audio content by the listener, thus realizing a high quality of the audio content in the second format and reducing the complexity of the format conversion according to the directional loudness map. By contribution, it is possible to obtain information of signals related to a high quality audio perception of the format converted audio content. Thus, for example, the audio content in the second format contains fewer signals (e.g. only signals related according to the directional loudness map) than the audio content in the first format and has approximately the same audio quality.

[0084] According to an embodiment, the format converter is configured to receive the directional loudness map information and obtain an overall directional loudness map (e.g., of the decoded audio scene; e.g., of the audio content in the first format) and / or one or more directional loudness maps based thereon. The directional loudness map information (i.e., one or more directional loudness maps associated with individual signals of the audio content or with a signal pair or a combination of three or more signals of the audio content) can represent the audio content in the first format, can be part of the audio content in the first format, or can be determined by the format converter based on the audio content in the first format (e.g., by an audio analyzer as described herein; e.g., the format converter comprises an audio analyzer). According to an embodiment, the format converter is configured to also determine directional loudness map information of the audio content in the second format. Thus, for example, the directional loudness maps before and after the format conversion can be compared to reduce the perceived quality degradation due to the format conversion. This is achieved, for example, by minimizing the deviation of the directional loudness maps before and after the format conversion.

[0085] According to one embodiment, the format converter is configured to derive an overall directional volume map (e.g., a decoded audio scene) from one or more (e.g., decoded) directional volume maps (e.g., associated with a signal in a first format).

[0086] According to one embodiment, the format converter is configured to calculate or estimate the contribution of a given input audio signal (e.g., a signal in a first format) to the overall directional loudness map of the audio scene. The format converter is configured to decide whether to consider the given input audio signal in the format conversion depending on the calculation or estimation of the contribution (e.g., by comparing the calculated or estimated contribution with a predefined absolute or relative threshold). For example, if the contribution is equal to or greater than an absolute or relative threshold, the corresponding signal can be considered relevant and thus the format converter can be configured to decide to consider this signal. This can be understood as a complexity adjustment by the format converter, since not all signals in the first format are necessarily converted to the second format. The predefined threshold can represent a contribution of at least 2% or at least 5% or at least 10% or at least 20% or at least 30%. This means, for example, excluding inaudible and / or irrelevant channels (or nearly inaudible and / or irrelevant channels), i.e. the threshold should be lower (e.g. when compared to other use cases), e.g. 5%, 10%, 20%, 30%.

[0087] An embodiment according to the invention relates to an audio decoder for decoding encoded audio content. The audio decoder is configured to receive encoded representations of one or more audio signals and provide decoded representations of the one or more audio signals (e.g. using AAC-like decoding or using decoding of entropy-coded spectral values). Furthermore, the audio decoder is configured to reconstruct an audio scene using the decoded representations of the one or more audio signals and to adjust the decoding complexity depending on the contribution of the encoded signals (e.g. one or more audio signals, one or more downmix signals, one or more residual signals, etc.) to the overall directional loudness map of the decoded audio scene.

[0088] This embodiment of the audio decoder is based on the idea that it is advantageous to adjust the decoding complexity based on one or more directional loudness maps, since they are indicative of the perception of the audio content by the listener, and thus can simultaneously realize a reduction in the decoding complexity and an improvement in the decoder audio quality of the audio content. Thus, for example, the audio decoder is configured to determine, based on the contributions, which encoded signals of the audio content should be decoded and used for the reconstruction of the audio scene by the audio decoder. This means, for example, that the encoded representation of one or more audio signals contains fewer audio signals (e.g. only audio signals that are relevant according to the directional loudness map) than the decoded representation of one or more audio signals with approximately the same audio quality.

[0089] According to one embodiment, the audio decoder is configured to receive the encoded directional loudness map information and to decode the encoded directional loudness map information in order to obtain an overall directional loudness map (e.g. of a decoded audio scene or, e.g., as a target directional loudness map of a decoded audio scene) and / or one or more (decoded) directional loudness maps. According to one embodiment, the format converter is configured to determine or receive directional loudness map information of the encoded audio content (e.g., received) and the decoded audio content (e.g., determined). Thus, for example, the directional loudness maps before and after decoding can be compared in order to reduce a perceived quality degradation due to the decoding and / or previous encoding (e.g., performed by an audio encoder as described herein). This is achieved, for example, by minimizing the deviation of the directional loudness map before and after the format conversion.

[0090] According to one embodiment, the audio decoder is configured to derive an overall directional loudness map (e.g., of a decoded audio scene, or, e.g., as a target directional loudness map for a decoded audio scene) from one or more (e.g., decoded) directional loudness maps.

[0091] According to one embodiment, the audio decoder is configured to calculate or estimate the contribution of a given encoded signal to the overall directional loudness map of the decoded audio scene. Alternatively, the audio decoder is configured to calculate the contribution of a given encoded signal to the overall directional loudness map of the encoded audio scene. The audio decoder is configured to decide whether to decode a given encoded signal depending on the calculation or estimation of the contribution (e.g. by comparing the calculated or estimated contribution with a predefined absolute or relative threshold). The predefined threshold may represent a contribution of at least 60%, 70%, 80% or 90%. To maintain good quality, the threshold should be lower, and may still reach this range in cases where the computing power is very limited (e.g. mobile devices), such as for example 10%, 20%, 40%, 60%. In other words, in some preferred embodiments, the predefined threshold should represent a contribution of at least 5%, or at least 10%, or at least 20%, or at least 40%, or at least 60%.

[0092] An embodiment according to the invention relates to a renderer (e.g., a binaural renderer or a soundbar renderer or a speaker renderer) for rendering audio content. According to an embodiment, the renderer is for distributing audio content represented using a first number of input audio channels and side information describing desired spatial characteristics such as the arrangement of audio objects or the relationship between the audio channels into a representation comprising a given number of channels independent of the first number of input audio channels (e.g., larger than the first number of input audio channels or smaller than the first number of input audio channels). The renderer is configured to reconstruct an audio scene based on one or more input audio signals (or, for example, based on two or more input audio signals). Furthermore, the renderer is configured to adjust the rendering complexity (e.g., by skipping, in the rendering process, one or more of the input audio signals that contribute to the directional loudness map below a threshold) depending on the contribution of the input audio signals (e.g., one or more audio signals, one or more downmix signals, one or more residual signals, etc.) to the overall directional loudness map of the rendered audio scene. The overall directional volume map can for example be described by side information received by the renderer.

[0093] According to one embodiment, the renderer is configured to obtain (e.g., receive or determine itself) directional volume map information and, based thereon, obtain an overall directional volume map (e.g., of a decoded audio scene) and / or one or more directional volume maps.

[0094] According to one embodiment, the renderer is configured to derive an overall directional volume map (e.g., a decoded audio scene) from one or more (e.g., two or more) (e.g., decoded or self-derived) directional volume maps.

[0095] According to one embodiment, the renderer is configured to calculate or estimate a contribution of a given input audio signal to an overall directional loudness map of the audio scene, and further configured to decide whether to take the given input audio signal into account in the rendering depending on the calculation or estimation of the contribution (e.g., by comparing the calculated or estimated contribution with a predefined absolute or relative threshold).

[0096] An embodiment according to the present invention relates to a method for analyzing an audio signal. The method includes obtaining a plurality of weighted spectral domain (e.g., time-frequency domain) representations (e.g., "directional signals") based on one or more spectral domain (e.g., time-frequency domain) representations of two or more input audio signals. Values ​​of the one or more spectral domain representations are weighted according to different directions (e.g., panning directions) (e.g., represented by weighting factors) of audio components (e.g., of spectral bins or spectral bands) (e.g., tuning from an instrument or singer) in the two or more input audio signals to obtain a plurality of weighted spectral domain representations (e.g., "directional signals"). Furthermore, the method includes obtaining loudness information (e.g., one or more "directional loudness maps") related to different directions (e.g., panning directions) as an analysis result based on the plurality of weighted spectral domain representations (e.g., "directional signals").

[0097] An embodiment according to the present invention relates to a method for evaluating the similarity of audio signals. The method includes obtaining first loudness information (e.g., directional loudness map; e.g., composite loudness value) related to different (e.g., panning) directions based on a first set of two or more input audio signals. Furthermore, the method includes comparing the first loudness information with second (e.g., corresponding) loudness information (e.g., reference loudness information; e.g., reference directional loudness map; e.g., reference composite loudness value) related to different panning directions and a set of two or more reference audio signals to obtain similarity information (e.g., "Model Output Variable" (MOV)) describing the similarity between the first set of two or more input audio signals and the set of two or more reference audio signals (or, for example, representing the quality of the first set of two or more input audio signals when compared to the set of two or more reference audio signals).

[0098] An embodiment according to the invention relates to a method for encoding an input audio content comprising one or more input audio signals (preferably a plurality of input audio signals). The method comprises providing one or more encoded (e.g. quantized and then losslessly encoded) audio signals (e.g. encoded spectral domain representations) based on the one or more input audio signals (e.g. left and right signals) or one or more signals derived therefrom (e.g. intermediate or downmix signals and side or difference signals). Furthermore, the method comprises adapting the provision of the one or more encoded audio signals in response to one or more directional loudness maps representing loudness information associated with a plurality of different directions (e.g. panning directions) of the one or more signals to be encoded. The adaptation of the provision of the one or more encoded audio signals is performed in response to, for example, the contribution of individual directional loudness maps (e.g. associated to individual signals, signal pairs or groups of three or more signals) of the one or more signals to be quantized to an overall directional loudness map associated with, for example, a plurality of input audio signals (e.g. each signal of the one or more input audio signals).

[0099] One embodiment according to the invention relates to a method for encoding input audio content comprising one or more input audio signals (preferably a plurality of input audio signals). The method includes providing one or more encoded (e.g., quantized and then losslessly encoded) audio signals (e.g., encoded spectral domain representations) of two or more signals to be jointly encoded (e.g., using an intermediate signal or a downmix signal and a side signal or a difference signal) based on two or more input audio signals (e.g., a left signal and a right signal) or based on two or more signals derived therefrom. Furthermore, the method includes selecting signals to be jointly encoded from a plurality of candidate signals or a plurality of pairs of candidate signals (e.g., from the two or more input audio signals or from two or more signals derived therefrom) in response to a directional loudness map representing loudness information related to a plurality of different directions (e.g., panning directions) of the candidate signals or pair of candidate signals. According to one embodiment, the signals to be jointly encoded are selected in response to a contribution of individual directional loudness maps of the candidate signals to an overall directional loudness map or in response to a contribution of a directional loudness map of a pair of candidate signals to an overall directional loudness map, e.g., associated with the plurality of input audio signals (e.g., each signal of the one or more input audio signals).

[0100] An embodiment according to the invention relates to a method for encoding an input audio content comprising one or more input audio signals (preferably a plurality of input audio signals). The method comprises providing one or more encoded (e.g. quantized and then losslessly encoded) audio signals (e.g. encoded spectral domain representations) based on two or more input audio signals (e.g. left and right signals) or based on two or more signals derived therefrom. Furthermore, the method comprises determining an overall directional loudness map (e.g. a target directional loudness map of a scene) based on the input audio signals and / or determining one or more individual directional loudness maps associated with the individual input audio signals (and / or determining one or more directional loudness maps associated with pairs of input audio signals). Furthermore, the method comprises encoding the overall directional loudness map and / or the one or more individual directional loudness maps as side information.

[0101] An embodiment according to the invention relates to a method for decoding encoded audio content. The method includes receiving an encoded representation of one or more audio signals, providing a decoded representation of the one or more audio signals (e.g. using AAC-like decoding or using decoding of entropy-coded spectral values). Furthermore, the method includes upon receiving encoded directional loudness map information, decoding the encoded directional loudness map information and obtaining one or more (e.g. decoded) directional loudness maps. Furthermore, the method includes reconstructing an audio scene using the one or more directional loudness maps using the decoded representation of the one or more audio signals.

[0102] An embodiment according to the invention relates to a method for converting a format of an audio content representing an audio scene (e.g. a spatial audio scene) from a first format to a second format, the first format may for example include a first number of channels or input audio signals and side information or spatial side information adapted to the first number of channels or input audio signals, and the second format may for example include a second number of channels or output audio signals that may differ from the first number of channels or input audio signals and side information or spatial side information adapted to the second number of channels or output audio signals. The method comprises providing a representation of the audio content in the second format based on a representation of the audio content in the first format, and comprises adjusting the complexity of the format conversion (e.g. by skipping in the format conversion process one or more of the input audio signals in the first format that contribute to the directional loudness map below a threshold) depending on the contribution of the input audio signals (e.g. one or more audio signals, one or more downmix signals, one or more residual signals, etc.) in the first format to an overall directional loudness map of the audio scene. The overall directional loudness map may for example be described by side information of the audio content in the first format received by the format converter.

[0103] One embodiment according to the invention relates to a method comprising receiving an encoded representation of one or more audio signals, providing a decoded representation of the one or more audio signals (e.g. using AAC-like decoding or using decoding of entropy-coded spectral values). The method comprises reconstructing an audio scene using the decoded representation of the one or more audio signals. Furthermore, the method comprises adjusting a decoding complexity depending on the contribution of the encoded signals (e.g. one or more audio signals, one or more downmix signals, one or more residual signals, etc.) to an overall directional loudness map of the decoded audio scene.

[0104] An embodiment according to the invention relates to a method for rendering audio content. According to an embodiment, the invention relates to a method for upmixing audio content represented using a first number of input audio channels and side information describing desired spatial characteristics such as the arrangement of audio objects or the relationship between the audio channels to a representation including a number of channels larger than the first number of input audio channels. The method comprises reconstructing an audio scene based on one or more input audio signals (or based on two or more input audio signals). Furthermore, the method comprises adjusting the rendering complexity (e.g. by skipping in the rendering process one or more of the input audio signals that contribute to the directional loudness map below a threshold) depending on the contribution of the input audio signals (e.g. one or more audio signals, one or more downmix signals, one or more residual signals, etc.) to an overall directional loudness map of the rendered audio scene. The overall directional loudness map can for example be described by the side information received by the renderer.

[0105] An embodiment according to the invention relates to a computer program having a program code for performing the method described herein, when the computer program runs on a computer.

[0106] One embodiment according to the invention relates to an encoded audio representation (eg an audio stream or a data stream) comprising an encoded representation of one or more audio signals and encoded directional volume map information.

[0107] The above-described method is based on the same considerations as the above-described audio analyzer, audio similarity evaluator, audio encoder, audio decoder, format converter and / or renderer. The method can be completed with all the features and functionality also described with respect to the audio analyzer, audio similarity evaluator, audio encoder, audio decoder, format converter and / or renderer.

[0108] The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which: [Brief description of the drawings]

[0109] [Figure 1] 1 shows a block diagram of an audio analyzer according to one embodiment of the present invention; [Diagram 2] 1 shows a detailed block diagram of an audio analyzer according to one embodiment of the present invention; [Figure 3a] 1 shows a block diagram of an audio analyzer using a first panning index approach according to an embodiment of the present invention; [Figure 3b] FIG. 2 shows a block diagram of an audio analyzer using a second panning index approach according to an embodiment of the present invention. [Figure 4a] 1 shows a block diagram of an audio analyzer using a first histogram technique according to an embodiment of the present invention; [Figure 4b] 2 shows a block diagram of an audio analyzer using a second histogram technique according to an embodiment of the present invention; [Diagram 5]2 shows a schematic diagram of a spectral domain representation analyzed by an audio analyzer according to an embodiment of the present invention and the results of a directional analysis, a loudness calculation per frequency bin, and a loudness calculation per direction by the audio analyzer; [Figure 6] FIG. 2 shows a diagram illustrating schematic histograms of two signals for directional analysis by an audio analyzer according to an embodiment of the present invention. [Figure 7a] FIG. 13 shows a diagram illustrating a matrix with one scaling factor different from 0 for each time / frequency tile associated with a direction for scaling performed by an audio analyzer according to an embodiment of the present invention. [Figure 7b] FIG. 1 shows a diagram illustrating a matrix with multiple scaling coefficients different from 0 for each time / frequency tile associated with a direction for scaling performed by an audio analyzer according to an embodiment of the present invention. [Figure 8] 2 shows a block diagram of an audio similarity evaluator according to one embodiment of the present invention; [Figure 9] FIG. 2 shows a block diagram of an audio similarity evaluator for analyzing a stereo signal according to one embodiment of the present invention. [Figure 10a] 4 shows a color plot of a reference directional loudness map usable by an audio similarity evaluator according to one embodiment of the present invention. [Figure 10b] 4 shows a color plot of a directional loudness map analyzed by an audio similarity evaluator according to one embodiment of the present invention. [Figure 10c] 4 illustrates a color plot of a difference directional loudness map determined by an audio similarity evaluator according to one embodiment of the present invention. [Figure 11] 2 shows a block diagram of an audio encoder according to an embodiment of the present invention; [Figure 12] 2 shows a block diagram of an audio encoder configured to adapt a quantization parameter according to an embodiment of the present invention; [Figure 13]1 shows a block diagram of an audio encoder configured to select a signal to be encoded according to an embodiment of the present invention; [Figure 14] 4 shows a schematic diagram illustrating the determination of the contribution of individual directional loudness maps of a candidate signal to the overall directional loudness map performed by an audio encoder according to an embodiment of the present invention; [Figure 15] FIG. 2 shows a block diagram of an audio encoder configured to encode directional loudness information as side information according to an embodiment of the present invention; [Figure 16] 2 shows a block diagram of an audio decoder according to an embodiment of the present invention; [Figure 17] 2 shows a block diagram of an audio decoder configured to adapt decoding parameters according to an embodiment of the present invention; [Figure 18] FIG. 2 shows a block diagram of a format converter according to one embodiment of the present invention. [Figure 19] 2 shows a block diagram of an audio decoder configured to adjust the decoding complexity according to one embodiment of the present invention; [Figure 20] FIG. 2 shows a block diagram of a renderer according to one embodiment of the present invention. [Figure 21] 1 shows a block diagram of a method for analyzing an audio signal according to an embodiment of the present invention; [Figure 22] FIG. 2 shows a block diagram of a method for assessing the similarity of audio signals according to an embodiment of the present invention. [Figure 23] 1 shows a block diagram of a method for encoding input audio content comprising one or more input audio signals according to an embodiment of the present invention; [Figure 24] 1 shows a block diagram of a method for jointly encoding audio signals according to an embodiment of the present invention; [Diagram 25] FIG. 2 shows a block diagram of a method for encoding one or more directional volume maps as side information according to an embodiment of the present invention. [Figure 26]FIG. 2 shows a block diagram of a method for decoding encoded audio content according to an embodiment of the present invention. [Figure 27] 1 shows a block diagram of a method for converting the format of an audio content representing an audio scene from a first format to a second format according to an embodiment of the present invention. [Figure 28] FIG. 2 shows a block diagram of a method for decoding encoded audio content and adjusting the decoding complexity according to an embodiment of the present invention. [Figure 29] 1 shows a block diagram of a method for rendering audio content according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0110] Equal or equivalent elements are elements having equal or equivalent functions, which are indicated in the following description by equal or equivalent reference numbers even if they occur in different figures.

[0111] In the following description, a number of details are set forth to provide a fuller overall description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention. Furthermore, the features of different embodiments described below may be combined with each other unless otherwise specified.

[0112] FIG. 1 shows a first input audio signal, e.g., X L,b Spectral domain representation of (m,k) 110 1 and a second input audio signal, e.g., X R,b Spectral domain representation of (m,k) 110 21 shows a block diagram of an audio analyzer 100 configured to obtain a spectral domain representation 110 as an input 110 to be analyzed. 1 , 110 2 This can be achieved, for example, by receiving the first input audio signal and the second input audio signal in the form of a spectral domain representation 110 by an external device or apparatus. 1 , 110 2 and then provided to the audio analyzer 100. Alternatively, the spectral domain representation 110 1 , 110 2 can be determined by the audio analyzer 100 as described with respect to FIG. 2. According to one embodiment, the spectral domain representation 110 is TIFF0007631273000025.tif311, for example, i={L;R;DM} or i It can be represented by TIFF0007631273000026.tif22[1;I].

[0113] According to one embodiment, the spectral domain representation 110 1 , 110 2 is fed to the direction information determination 120 to obtain the spectral domain representation 110 1 , 110 2 directional information 122 associated with a spectral band (e.g., spectral bin k in time frame m), e.g. TIFF0007631273000027.tif22(m,k). The direction information 122 may for example represent directions of different audio components contained in the two or more input audio signals. Thus, the direction information 122 may relate to the direction from which a listener hears the components contained in the two input audio signals. According to one embodiment, the direction information may represent a panning index. Thus, for example, the direction information 122 may comprise a first direction indicating a singer in a listening room and further directions corresponding to different instruments of a band in the audio scene. The direction information 122 may for example be calculated by the audio analyzer 100 for all frequency bins or frequency groups (for example for all spectral bins k or spectral bands b) from the spectral domain representation 110. 1 , 110 2 An example of the directional information determination 120 is described with respect to Figures 5 to 7b.

[0114] According to one embodiment, the audio analyzer 100 is configured to obtain directional information 122 based on an analysis of amplitude panning of the audio content, and / or based on an analysis of phase relationships and / or time delays and / or correlations between the audio content of two or more input audio signals, and / or based on an expanded (e.g., decorrelated and / or panned) sound source identification. The audio content may include the input audio signals and / or a spectral domain representation 110 of the input audio signals.

[0115] Directional Information 122 and Spectral Domain Representation 110 1 , 110 2 Based on this, the audio analyzer 100 determines the contributions 132 (e.g., TIFF0007631273000028.tif314 and According to one embodiment, the spectral domain representation 110 of the first input audio signal is determined. 1First contribution related to 132 1 is determined by the contribution determination 130 in response to the directional information 122 and a spectral domain representation 110 of the second input audio signal 2 Second contribution related to 132 2 is determined by the contribution determination 130 depending on the direction information 122. According to one embodiment, the direction information 122 is determined based on different directions (e.g., the extracted direction values The contribution 132 may be, for example, in a predetermined direction according to the direction information 122. According to one embodiment, the contribution 132 includes the volume information of the TIFF0007631273000031.tif34. TIFF0007631273000032.tif22(m,k) (corresponding to the direction information 122) is a specified direction Level information and / or its direction of the spectral band equal to TIFF0007631273000033.tif34 TIFF0007631273000034.tif22(m,k) is the specified direction TIFF0007631273000035.tif34 defines the scaled level information of adjacent spectral bands.

[0116] According to one embodiment, the extracted direction value TIFF0007631273000036.tif38 is determined according to the spectral domain value (e.g., in the notation of

[13] for the input audio signal) TIFF0007631273000037.tif39 as TIFF0007631273000038.tif313, and TIFF0007631273000039.tif39 as TIFF0007631273000040.tif313).

[0117] Different Directions TIFF0007631273000041.tif34 (e.g., a given direction) and associated volume information 142 (e.g., a plurality of different evaluated direction ranges). For TIFF0007631273000042.tif34, L(m, TIFF0007631273000043.tif34)(For a given direction of J To obtain a spectral domain representation 110 of the first input audio signal, the audio analyzer 100 calculates a spectral domain representation 110 of the first input audio signal, 1 The corresponding contribution of 132 1 (for example TIFF0007631273000045.tif314) and a spectral domain representation of a second input audio signal 110 2 The corresponding contribution of 132 2 (for example TIFF0007631273000046.tif314) to receive a composite signal as volume information 142 for, for example, two or more channels (e.g., a first channel associated with a first input audio signal and represented by index L, and a second channel associated with a second input audio signal and represented by index R). Thus, volume over time and different directions can be obtained. The volume information 142 that defines the volume for each of TIFF0007631273000047.tif34 is obtained. This is performed by the volume information determination unit 140, for example.

[0118] Fig. 2 illustrates an audio analyzer 100 that may include features and / or functionality described with respect to the audio analyzer 100 of Fig. 1. According to one embodiment, the audio analyzer 100 receives a first input audio signal x L 112 1 and the second input audio signal x R 112 2where index L is associated with left and index R is associated with right. The indexes may be associated with speakers (e.g., speaker positioning). According to one embodiment, the indexes may be represented by a number indicating a channel associated with the input audio signal.

[0119] According to one embodiment, the first input audio signal 112 1 and / or the second input audio signal 112 2 may represent time-domain signals that may be transformed by a time-domain to spectral-domain transform 114 to receive a spectral-domain representation 110 of each input audio signal. In other words, the time-domain to spectral-domain transform 114 transforms two or more input audio signals 112 1 , 112 2 (For example, x L , x R , x i ) into the short-time Fourier transform (STFT) domain to produce two or more transformed audio signals 115 1 , 115 2 (For example, X' L , X' R , X' i ) can be obtained. The first input audio signal 112 1 and / or the second input audio signal 112 2 If represents the spectral domain representation 110, the time domain to spectral domain transformation 114 can be skipped.

[0120] Optionally, the input audio signal 112 or the transformed audio signal 115 is processed by an ear model processing 116 to generate a respective input audio signal 112 1 and 112 2The input audio signal 112 is processed to obtain a spectral domain representation 110 of the input audio signal 112. The spectral bins of the signal to be processed, e.g. 112 or 115, are grouped into spectral bands, e.g. based on a model for the perception of spectral bands by the human ear, and the spectral bands can then be weighted based on an outer ear and / or middle ear model. Thus, an optimized spectral domain representation 110 of the input audio signal 112 can be determined using ear model processing 116.

[0121] According to one embodiment, the first input audio signal 112 1 Spectral domain representation of 110 1 , for example, X L,b (m,k) is the first input audio signal 112 1 , and a different spectral band (e.g., indicated by index b). For each spectral band b, a spectral domain representation 110 1 For example, represents the level information of all spectral bins k for time frame m and respective spectral band b.

[0122] According to one embodiment, the second input audio signal 112 2 Spectral domain representation of 110 2 , for example, X R,b (m,k) is the second input audio signal 112 2 (e.g., indicated by index R) and a different spectral band (e.g., indicated by index b). For each spectral band b, a spectral domain representation 110 2 For example, represents the level information of all spectral bins k for time frame m and respective spectral band b.

[0123] A spectral domain representation 110 of a first input audio signal 112 1 and a spectral domain representation 110 of the second input audio signal. 2Based on this, a directional information determination 120 can be performed by the audio analyzer 100. The directional analysis 124 can determine, for example, The panning direction information 125 may be determined, for example, based on the signal components (e.g., the first input audio signal 112 panned in a particular direction). 1 and the second input audio signal 112 2 2 , the panning direction information 125 may include panning indices corresponding to signal components panned entirely to the left or to the right or somewhere in between. According to one embodiment, the input audio signals 112 are associated with different directions, e.g. indicated by a left index L and a right index R. The panning indexes define, e.g., a direction between two or more input audio signals 112 or in the direction of the input audio signals 112. Thus, for example, in the case of a two-channel signal as shown in Fig. 2 , the panning direction information 125 may include panning indices corresponding to signal components panned entirely to the left or to the right or somewhere in between.

[0124] According to one embodiment, based on the panning direction information 125, the audio analyzer 100 performs a scaling factor determination 126 to determine a direction-dependent weighting 127, e.g., j About TIFF0007631273000049.tif22[1;i] The direction-dependent weighting 127 is configured to determine the direction of the panning direction extracted from the panning direction information 125. The direction-dependent weighting 127 defines a scaling coefficient according to a plurality of predetermined directions. According to one embodiment, the direction-dependent weighting 127 defines a function for each predefined direction. The function may be, for example, a function for each direction extracted from the panning direction information 125. The scaling factor depends on the direction extracted from the panning direction information 125, for example. TIFF0007631273000054.tif22(m,k) and the given direction TIFF0007631273000055.tif34. The scaling factor, i.e. the direction-dependent weighting 127, can be determined for each spectral bin and / or for each time step / time frame.

[0125] According to one embodiment, the direction-dependent weighting 127 uses a Gaussian function, so that the direction-dependent weighting is TIFF0007631273000056.tif22(m,k) and their respective given orientation values TIFF0007631273000057.tif34 decreases as the deviation between them increases.

[0126] According to one embodiment, the audio analyzer 100 includes: TIFF0007631273000058.tif531, in a given direction (e.g. index TIFF0007631273000059.tif34), a time (or time frame) specified by time index m, a time specified by time index m, and a direction-dependent weighting associated with a spectral bin specified by spectral bin index k. TIFF0007631273000060.tif312, where: TIFF0007631273000061.tif32 is a predefined value (which controls, for example, the width of the Gaussian window), TIFF0007631273000062.tif38 specifies the extracted orientation value associated with the time (or time frame) specified by the time index m, and the spectral bin specified by the spectral bin index k, TIFF0007631273000063.tif34 is a (e.g., predetermined) (or associated) directional value that specifies a predetermined direction (e.g., has a directional index j).

[0127] According to one embodiment, the audio analyzer 100 is configured to determine directional information including panning direction information 125 and / or direction-dependent weighting 127 by using a directional information determination 120. This directional information is obtained, for example, based on the audio content of two or more input audio signals 112.

[0128] According to one embodiment, the audio analyzer 100 comprises a scaler 134 and / or a combiner 136 for the contribution determination 130. By means of the scaler 134, the direction-dependent weighting 127 is calculated by a weighted spectral domain representation 135 (e.g., different TIFF0007631273000064.tif33(j About TIFF0007631273000065.tif22[1;J] or j={L;R;DM}) TIFF0007631273000066.tif329) is applied to one or more spectral domain representations 110 of two or more input audio signals 112. In other words, the spectral domain representation 110 of the first input audio signal 1 and a spectral domain representation 110 of the second input audio signal. 2 is the given direction Thus, for example, a weighted spectral domain representation 135 of the first input audio signal is 1 for example TIFF0007631273000068.tif314 is a specified orientation TIFF0007631273000069.tif34 or the signal components of the first input audio signal 112 related to the adjacent predetermined direction. 1Thus, one or more spectral domain representations 110 (e.g., TIFF0007631273000070.tif310) values ​​represent different directions of the audio components (e.g. panning directions). TIFF0007631273000071.tif34) are weighted according to (e.g., weighting factor (Represented by TIFF0007631273000072.tif38).

[0129] According to one embodiment, the scaling factor determination 126 determines, for each given direction, the extracted direction value TIFF0007631273000073.tif22(m,k) is the specified direction The signal components that deviate from the TIFF0007631273000074.tif34 are weighted and are used to extract the directional values. TIFF0007631273000075.tif22(m,k) is the specified direction TIFF0007631273000076.tif34. In other words, the first predetermined direction In the direction-dependent weighting 127 for TIFF0007631273000077.tif34, the first predetermined direction The signal components associated with TIFF0007631273000078.tif34 are in a first predetermined direction First weighted spectral domain representation corresponding to TIFF0007631273000079.tif34 In TIFF0007631273000080.tif314, signal components related to other directions are emphasized more than those related to other directions.

[0130] According to one embodiment, the audio analyzer 100 performs a step of extracting an input audio signal designated by index i (e.g., 110 for i=1). 1 , 110 for i=2 2) or a combination of input audio signals (e.g., two input audio signals 110 for i=1, 2) 1 and 110 2 ), the spectral band specified by index b, the index a weighted spectral domain representation 135 associated with a (e.g., predetermined) direction specified by TIFF0007631273000081.tif34, a time (or time frame) specified by a time index m, and a spectral bin specified by a spectral bin index k It is configured to get TIFF0007631273000082.tif313, It follows TIFF0007631273000083.tif338, TIFF0007631273000084.tif34 specifies a spectral domain representation 110 relating to an input audio signal 112 or a combination of input audio signals 112 specified by index i (e.g., i=L or i=R or i=DM or I is represented by a number and indicates a channel), a spectral band specified by index b, a time (or time frame) specified by time index m, and a spectral bin specified by spectral bin index k; TIFF0007631273000085.tif310 is the index TIFF0007631273000086.tif312 specifies a direction-dependent weighting 127 (weighting function) associated with a direction specified by the time index m, a time (or time frame) specified by the time index m, and a spectral bin specified by the spectral bin index k. Additional or alternative functions of scaler 134 are described with respect to Figures 6-7b.

[0131] According to one embodiment, the weighted spectral domain representation 135 of the first input audio signal 1 and a weighted spectral domain representation 135 of the second input audio signal. 2is the weighted joint spectral domain representation 137 TIFF0007631273000087.tif315 are combined by combiner 136 to obtain TIFF0007631273000087.tif315. All channels corresponding to TIFF0007631273000088.tif34 (first input audio signal 112 1 and the second input audio signal 112 2 The weighted spectral domain representations 135 of the combiner 136 (in the case of FIG. 2) are combined into one signal. This can be done, for example, by combining the weighted spectral domain representations 135 of the predefined all directions (j TIFF0007631273000089.tif22[1;i]) According to one embodiment, the weighted combined spectral domain representation 137 is associated with different frequency bands b.

[0132] Based on the weighted combined spectral domain representation 137, a loudness information determination 140 is performed to obtain loudness information 142 as an analysis result. According to one embodiment, the loudness information determination 140 comprises a band loudness determination 144 and an all-band loudness determination 146. According to one embodiment, the band loudness determination 144 is configured to determine a band loudness value 145 for each spectral band b based on the weighted combined spectral domain representation 137. In other words, the band loudness determination 144 is configured to determine a band loudness value 145 for each spectral band b based on the weighted combined spectral domain representation 137. TIFF0007631273000091.tif34 to determine the loudness in each spectral band. Thus, the obtained band loudness value 145 no longer depends on a single spectral bin k.

[0133] According to one embodiment, the audio analyzer measures band volume values ​​145 (associated with each frequency band (b)) (e.g., TIFF0007631273000092.tif310) to determine a weighted combined spectral domain representation 137 (e.g., TIFF0007631273000093.tif315) (or across the spectral bins of the frequency band) and applying a power operation with an exponent between 0 and ½ (and preferably less than ⅓ or ¼) to the average of the squared spectral values.

[0134] According to an embodiment, the audio analyzer performs a spectral band designated by index b, index The band volume value 145 associated with the time (or time frame) specified by the time index m according to the direction specified in TIFF0007631273000094.tif34. It is configured to get TIFF0007631273000095.tif310, According to TIFF0007631273000096.tif544, b where k is a running variable that specifies the number of spectral bins in a frequency band with frequency band index b, and b specifies the spectral band; TIFF0007631273000097.tif315 is the spectral band specified by index b, index TIFF0007631273000098.tif34 shows a weighted combined spectral domain representation 137 associated with a direction specified by the time index m, a time (or time window) specified by the time index m, and a spectral bin specified by the spectral bin index k.

[0135] In an all-band loudness information determination 146, the band loudness values ​​145 are averaged across all spectral bands to provide a loudness information 142 that is dependent on, for example, a given direction and at least one time frame m. According to one embodiment, the loudness information 142 may represent the general loudness caused by the input audio signal 112 in different directions in the listening room. According to one embodiment, the loudness information 142 may represent the general loudness caused by the input audio signal 112 in different given or predefined directions m. It can be associated with a synthetic volume value related to TIFF0007631273000099.tif34.

[0136] An audio analyzer according to one of claims 1 to 17, According to TIFF0007631273000100.tif430, index Multiple combined loudness values ​​L(m, TIFF0007631273000102.tif34), where B is the total number of spectral bands b, TIFF0007631273000103.tif310 is the spectral band specified by index b, index TIFF0007631273000104.tif34 indicates a band volume value 145 associated with the direction specified by TIFF0007631273000104.tif34 and the time (or time frame) specified by time index m.

[0137] 1 and 2, the audio analyzer 100 is configured to analyze the spectral domain representations 110 of two input audio signals, but the audio analyzer 100 may also be configured to analyze three or more spectral domain representations 110.

[0138] Figures 3a to 4b show different implementations of an audio analyzer 100. The audio analyzer shown in Figures 1 to 4b is not limited to the features and functionality shown for one implementation, but may also include features and functionality of other implementations of the audio analyzer shown in the different Figures 1 to 4b.

[0139] 3a and 3b show two different approaches by the audio analyzer 100 for determining the loudness information 142 based on determining the panning index.

[0140] The audio analyzer 100 shown in Fig. 3a is similar or equivalent to the audio analyzer 100 shown in Fig. 2. Two or more input signals 112 are transformed into time / frequency signals 110 by time / frequency decomposition 113. According to one embodiment, the time / frequency decomposition 113 may include a time-domain to spectral domain transformation and / or ear model processing.

[0141] Based on the time / frequency signal, a directional information determination 120 is performed. The directional information determination 120 includes, for example, a directional analysis 124 and a window function determination 126. In the contribution determination unit 130, a directional signal 132 is obtained by splitting the time / frequency signal 110 into directional signals, for example by applying a direction-dependent window function 127 to the time / frequency signal 110. Based on the directional signal 132, a loudness calculation 140 is performed to obtain loudness information 142 as an analysis result. The loudness information 142 may include a directional loudness map.

[0142] The audio analyzer 100 of Fig. 3b differs from the audio analyzer 100 of Fig. 3a in the loudness calculation 140. According to Fig. 3b, the loudness calculation 140 is performed before the directional signal of the time / frequency signal 110 is calculated. Thus, for example, according to Fig. 3b, the band loudness values ​​141 are calculated directly based on the time / frequency signal 110. By applying a direction-dependent window function 127 to the band loudness values ​​141, directional loudness information 142 can be obtained as an analysis result.

[0143] 4a and 4b show an audio analyzer 100 configured to determine loudness information 142 using a histogram technique according to one embodiment. According to one embodiment, the audio analyzer 100 is configured to determine a time / frequency signal 110 based on two or more input signals 112 using a time / frequency decomposition 113.

[0144] According to one embodiment, a loudness calculation 140 is performed based on the time / frequency signal 110 to obtain a composite loudness value 145 for each time / frequency tile. The composite loudness value 145 is not associated with any directional information. The composite loudness value is, for example, associated with the loudness resulting from the superposition of the input signal 112 on the time / frequency tile.

[0145] Furthermore, the audio analyzer 100 is configured to perform a directional analysis 124 of the time / frequency signal 110 in order to obtain directional information 122. According to Fig. 4a, the directional information 122 comprises one or more directional vectors with ratio values ​​indicative of time / frequency tiles having the same level ratio between two or more input signals 112. This directional analysis 124 is performed, for example, as described with respect to Fig. 5 or Fig. 6.

[0146] The audio analyzer 100 of FIG. 4b optionally includes a directional value 122 after the directional analysis 124. 1 4a is different from the audio analyzer 100 shown in FIG. 4a in that directional smearing 126 is performed. Also, the directional smearing 126 allows time / frequency tiles associated with directions adjacent to a given direction to be associated with the given direction, and the obtained directional information 122 2 may further include a scaling factor for these time / frequency tiles to minimize the effect in a given direction.

[0147] In Figures 4a and 4b, the audio analyzer 100 is configured to accumulate 146 synthetic loudness values ​​145 into directional histogram bins based on directional information 122 associated with the time / frequency tiles.

[0148] Further details regarding the audio analyzer 100 of Figures 3a and 3b are provided below in the chapters "General steps for computing a directional loudness map" and "Embodiments of different forms for computing a loudness map using a generalized criterion function".

[0149] FIG. 5 illustrates a spectral domain representation 110 of a first input audio signal to be analyzed by the audio analyzer described herein. 1 and a spectral domain representation 110 of the second input audio signal. 2 A directional analysis 124 of the spectral domain representation 110 results in directional information 122. According to one embodiment, the directional information 122 is obtained by subtracting the spectral domain representation 110 from the first input audio signal. 1 and a spectral domain representation 110 of the second input audio signal. 2 , and thus, for example, frequency tiles, e.g., time / frequency tiles, of the spectral domain representation 110 having the same level ratio are associated with the same direction 125.

[0150] According to one embodiment, the loudness calculation 140 results, for example, for each time / frequency tile in a composite loudness value 145. The composite loudness value 145 is, for example, associated with a combination of a first input audio signal and a second input audio signal (e.g., a combination of two or more input audio signals).

[0151] Based on the directional information 122 and the composite loudness values ​​145, the composite loudness values ​​145 can be accumulated 146 into direction and time dependent histogram bins. Thus, for example, all composite loudness values ​​145 associated with a particular direction are summed. According to the directional information 122, the direction is associated with a time / frequency tile. The accumulation 146 results in a directional loudness histogram, which can represent the loudness information 142 as an analysis result of the audio analyzer described herein.

[0152] It may also be possible to associate time / frequency tiles corresponding to the same and / or adjacent directions in different or adjacent time frames (e.g., previous or subsequent time frames) with directions in the current time step or time frame. This means, for example, that the direction information 122 includes direction information for each time-dependent frequency tile (or frequency bin). Thus, for example, the direction information 122 is obtained for multiple or all time frames. Further details regarding the histogram approach shown in FIG. 5 are given in the section "Embodiment 2 of a different form of computing the loudness map using a generalized criterion function".

[0153] FIG. 6 illustrates the contribution determination 130 based on panning direction information performed by the audio analyzer described herein. FIG. 6a illustrates a spectral domain representation of a first input audio signal, and FIG. 6b illustrates a spectral domain representation of a second input audio signal. According to FIG. 6a1 to FIG. 6a3.1 and FIG. 6b1 to FIG. 6b3.1, the spectral bins or bands corresponding to the same panning direction are selected to calculate the loudness information of this panning direction. FIG. 6a3.2 and FIG. 6b3.2 illustrate an alternative process in which not only the frequency bins or bands corresponding to the panning direction are considered, but also other frequency bins or frequency groups that are weighted or scaled to have less influence. Further details regarding FIG. 6 are provided in the chapter "Recovering a directional signal using a window / selection function derived from a panning index".

[0154] According to one embodiment, the direction information 122 may include scaling factors related to the direction 121 and the time / frequency tiles 123, as shown in Fig. 7a and / or Fig. 7b. According to one embodiment, in Fig. 7a and Fig. 7b, the time / frequency tiles 123 are shown for only one time step or time frame. Fig. 7a shows scaling factors where only the time / frequency tiles 123 contributing to a particular (e.g., predefined) direction 121 are taken into account, as explained for example with respect to Figs. 6a1-6a3.1 and 6b1-6b3.1. Alternatively, in Fig. 7b, neighboring directions are also taken into account, but scaled to reduce the influence of the respective time / frequency tile 123 on the neighboring directions. According to Fig. 7b, the time / frequency tiles 123 are scaled such that their influence is reduced as the deviation from the relevant direction increases. Instead, in Fig. 6a3.2 and Fig. 6b3.2, all time / frequency tiles corresponding to different panning directions are scaled equally. Different scaling or weighting is possible. Depending on the scaling, the accuracy of the analysis results of the audio analyzer can be improved.

[0155] 8 illustrates an embodiment of an audio similarity evaluator 200. The audio similarity evaluator 200 uses the first loudness information 142 1 (For example, L 1 (m, TIFF0007631273000105.tif34)) and the second volume information 142 2 (For example, L 2 (m, TIFF0007631273000106.tif34) is acquired. 1 is a first set of two or more input audio signals 112a (e.g., x for i ∈ [1;n] L , x R , or x i ) depending on different directions (e.g., a given panning direction) TIFF0007631273000107.tif34) and the second volume information 142 2is a set of reference audio signals 112b (e.g., x 2,R , x 2,L , x 2,i The first set of input audio signals 112a and the set of reference audio signals 112b may include n audio signals, where n represents an integer equal to or greater than 1. Each audio signal in the first set of input audio signals 112a and the set of reference audio signals 112b may be associated with a different speaker located at a different position in the listening space. The first volume information 142 1 and second volume information 142 2 can represent a volume distribution in a listening space (e.g., at or between speaker positions). According to one embodiment, the first volume information 142 1 and second volume information 142 2 contains loudness values ​​for discrete positions or directions in the listening space. Different directions can be associated with panning directions of audio signals dedicated to one of the sets of audio signals 112a or 112b depending on which set corresponds to the loudness information to be calculated.

[0156] First volume information 142 1 and second volume information 142 2 can be determined by a loudness information determination 100, which can be performed by an audio similarity evaluator 200. According to one embodiment, the loudness information determination 100 can be performed by an audio analyzer. Thus, for example, the audio similarity evaluator 200 can comprise an audio analyzer or receive the first loudness information 142 from an external audio analyzer. 1 and / or second volume information 142 2 According to an embodiment, the audio analyzer may have features and / or functions as described with respect to the audio analyzer of Figs. 1 to 4b. Alternatively, the first volume information 142 may be received. 1Only the second volume information 142 is determined by the volume information determination 100. 2 is received or obtained by the audio similarity evaluator 200 from a databank with reference loudness information. According to one embodiment, the databank may contain reference loudness information maps for different speaker settings and / or speaker configurations and / or different sets of reference audio signals 112b.

[0157] According to one embodiment, the set of reference audio signals 112b may represent an ideal set of audio signals for optimized audio perception by a listener in a listening space.

[0158] According to one embodiment, the first volume information 142 1 (For example, L 1 (m, TIFF0007631273000108.tif34) to L 1 (m, TIFF0007631273000109.tif34) vector containing the second volume information 142 2 (For example, L 2 (m, TIFF0007631273000110.tif34) to L 2 (m, The vector including the metadata (TIFF0007631273000111.tif34) may include a plurality of composite volume values ​​associated with each input audio signal (e.g., input audio signals corresponding to the first set of input audio signals 112a or the set of reference audio signals 112b (and associated with a respective predefined direction)). Each predefined direction may represent a panning index. Since each input audio signal is associated with, for example, a speaker, each predefined direction may be understood as an equally spaced position between the respective speakers (e.g., between adjacent speakers and / or other speaker pairs). In other words, the audio similarity evaluator 200 uses the metadata representing the position information of the speakers associated with the input audio signals to evaluate the volume information 142 having different directions (e.g., the second direction described herein). 1 and / or 142 2 The first volume information 142 is configured to obtain a directional component (e.g., a first direction as described herein) used to obtain the first volume information 142. 1 and / or second volume information 142 2 The first volume information 142 describes the volume of the respective sets of signal components of the input audio signals 112a and 112b associated with each given direction. 1 and / or second volume information 142 2 is associated with a combination of the weighted spectral domain representations associated with each given direction.

[0159] The audio similarity evaluator 200 calculates the first loudness information 142 to obtain similarity information 210 describing the similarity between the first set of two or more input audio signals 112a and the set of two or more reference audio signals 112b. 1 The second volume information 142 2This can be performed by a loudness information comparison unit 220. The similarity information 210 can indicate the quality of the first set of input audio signals 112a. To further improve the prediction of the perception of the first set of input audio signals 112a based on the similarity information 210, the first loudness information 142 can be compared with the first loudness information 142. 1 and / or second volume information 142 2 According to one embodiment, the first loudness information 142 may take into account only a subset of the frequency bands. 1 and / or second volume information 142 2 is determined only for the frequency band having frequencies above 1.5 kHz. Therefore, the volume information 142 to be compared 1 and 142 2 can be optimized based on the sensitivity of the human auditory system. Therefore, the loudness information comparison unit 220 compares the loudness information 142, which includes only the loudness values ​​of the relevant frequency bands. 1 and 142 2 The relevant frequency bands may be associated with frequency bands that correspond to a sensitivity (e.g., of the human ear) that is greater than a predetermined threshold to a predetermined level difference. To obtain the similarity information 210, for example, the second volume information 142 2 and the first volume information 142 1 The difference is calculated.

[0160] This difference may represent the residual loudness information and may already define the similarity information 210. Alternatively, the residual loudness information is further processed to obtain the similarity information 210. According to one embodiment, the audio similarity evaluator 200 is configured to determine a value that quantifies the difference across multiple directions. This value may be a single scalar value that represents the similarity information 210. To receive the scalar value, the loudness information comparison unit 220 calculates the difference for a partial or complete duration of the first set of input audio signals 112a and / or the set of reference audio signals 112b, and then compares the obtained residual loudness information with the first set of input audio signals 112a and / or the set of reference audio signals 112b in all panning directions (e.g., the first loudness information 142). 1and / or second volume information 142 2 can be configured to average over the different directions (to which the vectors are associated) to produce a single numbered Model Output Variable (MOV).

[0161] Fig. 9 shows an embodiment of an audio similarity evaluator 200 for calculating similarity information 210 based on a reference stereo input signal 112b and an analyzed stereo signal 112a (e.g., in this case, the signal under test (SUT)). According to an embodiment, the audio similarity evaluator 200 may include features and / or functionality as described with respect to the audio similarity evaluator of Fig. 8. The two stereo signals 112a and 112b may be processed by a peripheral ear model 116 to obtain spectral domain representations 110a and 110b of the stereo input audio signals 112a and 112b.

[0162] According to one embodiment, in a next step, the audio components of the stereo signals 112a and 112b can be analyzed for their directional information. Different panning directions 125 can be predefined and direction-dependent weightings 127 can be applied to the stereo signals 112a and 112b. 1 From 127 71. Based on the direction-dependent weighting 127 and the spectral domain representation 110a and / or 110b of the respective stereo input signal 112a and / or 112b, a panning index directional decomposition 130 can be performed to obtain the contributions 132a and / or 132b. According to an embodiment, the contributions 132a and / or 132b are then processed by a loudness calculation 144 to obtain, for example, loudnesses 145a and / or 145b per frequency band and panning direction. According to an embodiment, a frequency averaging per ERB 146 (ERB=equivalent rectangular bandwidth) is performed on the loudness signals 145b and / or 145a to obtain the directional loudness maps 142a and / or 142b for the loudness information comparison 220. The loudness information comparison 220 is configured to calculate a distance measure based on, for example, the two directional loudness maps 142a and 142b. The distance measure may represent a directional loudness map that includes the difference between the two directional loudness maps 142a and 142b. According to an embodiment, a single numbered model output variable MOV may be obtained as the similarity information 210 by averaging the distance measures over all panning directions and times.

[0163] FIG. 10c shows a distance measure as described in FIG. 9 or similarity information as described in FIG. 8, represented by a directional loudness map 210 showing the loudness difference between the directional loudness map 142b shown in FIG. 10a and the directional loudness map 142a shown in FIG. 10b. The directional loudness maps shown in FIG. 10a-c represent, for example, loudness values ​​and panning directions over time. The directional loudness map shown in FIG. 10a can represent loudness values ​​corresponding to a reference input signal. This directional loudness map can be calculated by an audio analyzer as described in FIG. 9 or in FIG. 1-4b, or can be retrieved from a database. The directional loudness map shown in FIG. 10b can represent, for example, loudness information corresponding to a stereo signal under test and determined by an audio analyzer as described in FIG. 1-4b and in FIG. 8 or FIG. 9.

[0164] FIG. 11 illustrates a block diagram of one or more input audio signals (e.g., x i 1 illustrates an audio encoder 300 for encoding 310 an input audio content 112 including a plurality of input audio signals, such as a stereo signal or a multi-channel signal. The input audio content 112 preferably includes a plurality of input audio signals, such as a stereo signal or a multi-channel signal. The audio encoder 300 is configured to provide one or more encoded audio signals 320 based on the one or more input audio signals 112 or based on one or more signals 110 derived from the one or more input audio signals 112 by optional processing 330. Thus, either the one or more input audio signals 112 or the one or more signals 110 derived therefrom are encoded 310 by the audio encoder 300. The processing 330 may include mid / side processing, downmix / difference processing, time-domain to spectral-domain conversion, and / or ear model processing. The encoding 310 includes, for example, quantization followed by lossless encoding.

[0165] The audio encoder 300 may include one or more directional loudness maps 142 (e.g., a plurality of different loudness maps) that represent loudness information associated with a plurality of different directions (e.g., a given direction or directions of one or more signals 112 to be encoded). L about TIFF0007631273000112.tif33 i (m, TIFF0007631273000113.tif34)). According to one embodiment, the coding parameters include a quantization parameter and / or other coding parameters such as bit distribution and / or parameters for disabling / enabling the coding 310.

[0166] According to an embodiment, the audio encoder 300 is configured to perform the loudness information determination 100 to obtain the directional loudness map 142 based on the input audio signal 112 or based on the processed input audio signal 110. Thus, for example, the audio encoder 300 may comprise an audio analyzer 100 as described with respect to figures 1 to 4b. Alternatively, the audio encoder 300 may receive the directional loudness map 142 from an external audio analyzer which performs the loudness information determination 100. According to an embodiment, the audio encoder 300 may obtain a plurality of directional loudness maps 142 associated with the input audio signal 112 and / or the processed input audio signal 110.

[0167] According to an embodiment, the audio encoder 300 may receive only one input audio signal 112. In this case, the directional loudness map 142 may, for example, include loudness values ​​for only one direction. According to an embodiment, the directional loudness map 142 may include loudness values ​​equal to 0 for directions different from the direction associated with the input audio signal 112. In case of only one input audio signal 112, the audio encoder 300 may decide based on the directional loudness map 142 whether an adaptation of the encoding parameters 340 should be performed. Thus, for example, the adaptation of the encoding parameters 340 may include a setting of the encoding parameters to standard encoding parameters for mono signals.

[0168] If the audio encoder 300 receives a stereo signal or a multi-channel signal as the input audio signal 112, the directional loudness map 142 may include loudness values ​​in different directions (e.g., different from 0). In the case of a stereo input audio signal, the audio encoder 300 obtains, for example, one directional loudness map 142 associated with the two input audio signals 112. In the case of a multi-channel input audio signal 112, the audio encoder 300 obtains, for example, one or more directional loudness maps 142 based on the input audio signal 112. In the case of a multi-channel signal 112 encoded by the audio encoder 300, for example, an overall directional loudness map 142 based on all channel signals and / or directional loudness maps, and / or one or more directional loudness maps 142 based on signal pairs of the multi-channel input audio signal 112 may be obtained by the loudness information determination 100. Thus, for example, the audio encoder 300 may be configured to perform an adaptation 340 of the encoding parameters depending on the contribution of individual directional loudness maps 142, such as, for example, signal pairs, intermediate signals, side signals, downmix signals, difference signals, and / or groups of three or more signals, to an overall directional loudness map 142 associated with a plurality of input audio signals, for example associated with the multi-channel input audio signal 112 or all signals of the processed multi-channel input audio signal 110.

[0169] The loudness information determination 100 described with respect to FIG. 11 is exemplary and may be performed identically or similarly by any of the following audio encoders or decoders:

[0170] FIG. 12 illustrates an embodiment of an audio encoder 300 that may include features and / or functionality described with respect to the audio encoder of FIG. 11. According to an embodiment, the encoding 310 may include quantization by a quantizer 312 and encoding by a coding unit 314, e.g., entropy coding. Thus, for example, the adaptation of the coding parameters 340 may include adaptation of a quantization parameter 342 and adaptation of coding parameters 344. The audio encoder 300 is configured to encode 310 an input audio content 112, e.g., comprising two or more input audio signals, to provide an encoded audio content 320, e.g., comprising the encoded two or more input audio signals. This encoding 310 may include, for example, a directional loudness map 142 or multiple directional loudness maps 142 (e.g., L ... i (m, TIFF0007631273000114.tif34)).

[0171] According to an embodiment, the input audio content 112 may be previously directly encoded 310 or optionally processed 330. As already mentioned above, the audio encoder 300 may be configured to determine a spectral domain representation 110 of one or more input audio signals of the input audio content 112 by a process 330. Alternatively, the process 330 may comprise a further processing step to derive one or more signals of the input audio content 112, which may undergo a transformation from the time domain to the spectral domain to receive the spectral domain representation 110. According to an embodiment, the signals derived by the process 330 may for example comprise an intermediate signal or a downmix signal and a side signal or a difference signal.

[0172] According to one embodiment, the input audio content 112 or the signal of the spectral domain representation 110 may be subjected to quantization by a quantizer 312, which for example uses one or more quantization parameters to obtain one or more quantized spectral domain representations 313. The one or more quantized spectral domain representations 313 may be encoded by an encoding unit 314 to obtain one or more encoded audio signals of the encoded audio content 320.

[0173] To optimize the encoding 310 by the audio encoder 300, the audio encoder 300 can be configured to adapt 342 quantization parameters. The quantization parameters include, for example, scale factors or parameters describing which quantization precision or quantization step should be applied to which spectral bins of the frequency band of the signal or signals to be quantized. According to an embodiment, the quantization parameters describe, for example, the allocation of bits to different signals and / or different frequency bands to be quantized. The adaptation 342 of the quantization parameters can be understood as an adaptation of the quantization precision and / or the noise introduced by the encoder 300 and / or as an adaptation of the bit distribution between the one or more signals 112 / 110 and / or parameters to be encoded by the audio encoder 300. In other words, the audio encoder 300 is configured to adjust one or more quantization parameters to adapt the bit distribution, to adapt the quantization precision and / or to adapt the noise. Furthermore, the quantization parameters and / or the coding parameters can be encoded (310) by the audio encoder.

[0174] According to an embodiment, the adaptation of coding parameters 340, such as the adaptation of quantization parameters 342 and the adaptation of coding parameters 344, can be performed depending on one or more directional loudness maps 142, which represent loudness information related to a number of different directions, panning directions, of the one or more signals 112 / 110 to be quantized. To be more precise, the adaptation 340 can be performed depending on the contribution of the individual directional loudness maps 142 of the one or more signals to be coded to the overall directional loudness map 142. This can be performed as described with respect to Fig. 11. Thus, for example, the adaptation of the bit distribution, the adaptation of the quantization precision and / or the adaptation of the noise can be performed depending on the contribution of the individual directional loudness maps of the one or more signals 112 / 110 to be coded to the overall directional loudness map. This is done, for example, by adjusting one or more quantization parameters by the adaptation 342.

[0175] According to an embodiment, the audio encoder 300 is configured to determine an overall directional loudness map based on the input audio signal 112, or on the spectral domain representation 110, whereby the overall directional loudness map represents loudness information related to different directions, for example of audio components, of the audio scene represented by the input audio content 112. Alternatively, the overall directional loudness map can represent loudness information related to different directions of the audio scene represented, for example, after decoder-side rendering. According to an embodiment, the different directions can be obtained by the loudness information determination 100, possibly in combination with knowledge or side information on the positions of the speakers and / or knowledge or side information describing the positions of the audio objects. This knowledge or side information can be obtained based on one or more signals 112 / 110 to be quantized, since these signals 112 / 110 are associated, for example, in a fixed, signal-independent manner, in different directions, or with different speakers, or with different audio objects. The signals are associated, for example, with specific channels that can be interpreted as directions of different directions (for example, the first direction described herein). According to one embodiment, audio objects of one or more signals are panned or rendered in different directions, which can be obtained by the loudness information determination 100 as object rendering information. This knowledge or side information can be obtained by the loudness information determination 100 for a group of two or more input audio signals of the input audio content 112 or the spectral domain representation 110.

[0176] According to one embodiment, the signal 112 / 110 to be quantized may comprise components of a joint multi-signal coding of two or more input audio signals 112, for example an intermediate signal and a side signal of a mid-side stereo coding. The audio encoder 300 is therefore configured to estimate said contribution of the directional loudness map 142 of one or more residual signals of the joint multi-signal coding to the overall directional loudness map 142 and to adjust one or more coding parameters 340 accordingly.

[0177] According to an embodiment, the audio encoder 300 is configured to adapt the bit distribution between the one or more signals 112 / 110 and / or parameters to be encoded and / or to adapt the quantization precision of the one or more signals 112 / 110 to be encoded and / or to adapt the noise introduced by the encoder 300 separately for different spectral bins or separately for different frequency bands. This means, for example, that an adaptation 342 of the quantization parameters is performed such that the encoding 310 is improved for individual spectral bins or for individual different frequency bands.

[0178] According to an embodiment, the audio encoder 300 is configured to adapt the bit distribution between one or more signals 112 / 110 to be encoded and / or parameters depending on an evaluation of spatial masking between the two or more signals to be encoded. The audio encoder is configured to evaluate spatial masking based on, for example, a directional loudness map 142 related to the two or more signals 112 / 110 to be encoded. Additionally or alternatively, the audio encoder is configured to evaluate the spatial masking or masking effect of a loudness contribution related to a first direction of a first signal to be encoded relative to a loudness contribution related to a second direction, different from the first direction, of a second signal to be encoded. According to an embodiment, the loudness contribution related to the first direction may represent, for example, loudness information of an audio object or audio component of the input audio content signal, and the loudness contribution related to the second direction may represent, for example, loudness information related to another audio object or audio component of the input audio content signal. Depending on the loudness information of the loudness contribution related to the first direction and the loudness contribution related to the second direction and depending on the distance between the first direction and the second direction, the masking effect or spatial masking can be evaluated. According to one embodiment, the masking effect decreases as the difference in angle between the first and second directions increases.Similarly, temporal masking can be evaluated.

[0179] According to an embodiment, the adaptation 342 of the quantization parameters can be performed by the audio encoder 300 in order to adapt the noise introduced by the encoder 300 based on the directional loudness map achievable by the encoded version 320 of the input audio content 112. The audio encoder 300 is thus configured to use, for example, a deviation between the directional loudness map 142 associated with a given unencoded input audio signal 112 / 110 (or a plurality of input audio signals) and the directional loudness map achievable by the encoded version 320 of the given input audio signal 112 / 110 (or a plurality of input audio signals) as a criterion for adapting the provision of the given encoded audio signal or audio signals of the encoded audio content 320. This deviation can represent the quality of the encoding 310 of the encoder 300. The encoder 300 can thus be configured to adapt 340 the encoding parameters in such a way that the deviation is below a certain threshold. Thus, a feedback loop 322 is implemented to improve the encoding 310 by the audio encoder 300 based on the directional loudness map 142 of the encoded audio content 320 and the directional loudness map 142 of the uncoded input audio content 112 or the uncoded spectral domain representation 110. According to one embodiment, in the feedback loop 322, the encoded audio content 320 is decoded and the loudness information decision 100 is performed based on the decoded audio signal. Alternatively, it is also possible that the directional loudness map 142 of the encoded audio content 320 is achieved by feedforward implemented by a neuronal network (e.g. prediction).

[0180] According to one embodiment, the audio encoder is configured to adjust one or more quantization parameters by adaptation 342 in order to adapt the presentation of one or more encoded audio signals of the encoded audio content 320.

[0181] According to an embodiment, the adaptation of coding parameters 340 can be performed to disable or enable the encoding 310 and / or to activate and deactivate, for example, joint coding tools used by the encoding unit 314. This is performed, for example, by the adaptation of coding parameters 344. According to an embodiment, the adaptation of coding parameters 344 can rely on the same considerations as the adaptation of quantization parameters 342. Thus, according to an embodiment, the audio encoder 300 is configured to disable the encoding 310 of a given one of the signals to be encoded, for example the residual signal, when the contribution of the individual directional loudness map 142 of the given one of the signals to be encoded (or the contribution of the directional loudness map 142 of, for example, a pair of signals to be encoded or a group of three or more signals to be encoded) to the overall directional loudness map falls below a threshold. Thus, the audio encoder 300 is configured to effectively code 310 only the relevant information.

[0182] According to an embodiment, the joint coding tool of the coding unit 314 is configured to jointly code two or more of the input audio signal 112 or the signal 110 derived therefrom, for example to make an M / S (middle / side signal) on / off decision. The coding parameter adaptation 344 can be performed such that the joint coding tool is activated or deactivated depending on one or more directional loudness maps 142, which represent loudness information related to multiple different directions of the one or more signals 112 / 110 to be coded. Alternatively or additionally, the audio encoder 300 can be configured to determine one or more parameters of the joint coding tool as coding parameters depending on one or more directional loudness maps 142. Thus, the coding parameter adaptation 344 can, for example, control the smoothing of frequency-dependent prediction coefficients to set parameters of, for example, an "intensity stereo" joint coding tool.

[0183] According to an embodiment, the quantization parameters and / or coding parameters may be understood as control parameters, which may control the provision of one or more encoded audio signals 320. Thus, the audio encoder 300 is configured to determine or estimate an influence of a variation of one or more control parameters on the directional loudness map 142 of the one or more encoded signals 320 and to adjust the one or more control parameters depending on the determination or estimation of the influence. This may be achieved by a feedback loop 322 and / or a feedforward, as described above.

[0184] FIG. 13 illustrates one or more input audio signals 112 1 , 112 2 13 shows an audio encoder 300 for encoding 310 an input audio content 112 comprising two or more input audio signals 112, as shown in FIG. 1 , 112 2 According to an embodiment, the input audio content 112 may include a time domain signal or a spectral domain signal. Optionally, the signal of the input audio content 112 is processed 330 by the audio encoder 300 to generate a first candidate signal 110. 1 and / or the second candidate signal 110 2 The process 330 may include, for example, a transformation from the time domain to the spectral domain if the input audio signal 112 is a time domain signal.

[0185] The audio encoder 300 is configured to select 350 signals to be jointly encoded 310 from among a plurality of candidate signals 110 or from among a plurality of pairs of candidate signals 110 according to a directional loudness map 142. The directional loudness map 142 represents loudness information associated with a plurality of different directions of the candidate signal 110 or pair of candidate signals 110 and / or a given direction, e.g., a panning direction.

[0186] According to one embodiment, the directional loudness map 142 may be calculated by the loudness information determination 100 as described herein. Thus, the loudness information determination 100 may be implemented as described with respect to the audio encoder 300 illustrated in Figure 11 or Figure 12. The directional loudness map 142 is based on a candidate signal 110, which represents the input audio signal of the input audio content 112 if no processing 330 is applied by the audio encoder 300.

[0187] If the input audio content 112 includes only one input audio signal, this signal is selected by signal selection 350 to be encoded by the audio encoder 300, e.g., using entropy coding to provide the one encoded audio signal as the encoded audio content 320. In this case, for example, the audio encoder is configured to disable joint coding 310 and switch to encoding of only the one signal.

[0188] The input audio content 112 is 1 and X 2 Two input audio signals 112 that can be described as 1 and 112 2 , both signals 112 are used to provide one or more encoded signals in the encoded audio content 320. 1 and 112 2 is selected (350) by the audio encoder 300 for joint encoding 310. Thus, the encoded audio content 320 optionally includes the intermediate signal and the side signal, or the downmix signal and the difference signal, or only one of these four signals.

[0189] If the input audio content 112 includes more than two input audio signals, the signal selection 350 is based on the directional loudness map 142 of the candidate signals 110. According to an embodiment, the audio encoder 300 is configured to use the signal selection 350 to select a signal pair from the multiple candidate signals 110, so as to achieve efficient audio coding and high quality audio output according to the directional loudness map 142. Alternatively or additionally, the signal selection 350 may also select three or more signals of the candidate signals 110 to be jointly encoded 310. Alternatively or additionally, the audio encoder 300 may use the signal selection 350 to select multiple signal pairs or signal groups for the joint encoding 310. The selection 350 of the signal 352 to be encoded may depend on the contribution of the individual directional loudness maps 142 of the combination of two or more signals to the overall directional loudness map. According to an embodiment, an overall directional loudness map is associated with each signal of the multiple selected input audio signals or input audio content 112. How this signal selection 350 can be performed by the audio encoder 300 is exemplarily described in FIG. 14 for an input audio content 112 comprising three input audio signals.

[0190] Thus, the audio encoder 300 encodes two or more input audio signals 112 using joint encoding 310 of two or more signals 352 to be jointly encoded. 1 , 112 2 Two or more signals 110 based on or derived therefrom 1 , 110 2 based on one or more encoded, e.g. quantized and then losslessly encoded audio signals, e.g. encoded spectral domain representations.

[0191] According to an embodiment, the audio encoder 300 is configured, for example, to determine individual directional loudness maps 142 of two or more candidate signals and to compare the individual directional loudness maps 142 of the two or more candidate signals. Furthermore, the audio encoder is configured to select two or more of the candidate signals for joint coding depending on the result of the comparison, such that, for example, the candidate signal whose individual loudness maps include a maximum similarity or a similarity higher than a similarity threshold is selected for joint coding. This optimized selection can achieve a very efficient coding, since a high similarity of the signals to be coded together can result in coding using only a few bits. This means, for example, that the downmix signal or the residual signal of the selected candidate pair can be efficiently coded together.

[0192] Figure 14 illustrates one embodiment of a signal selection 350 that may be performed by any audio encoder 300 described herein, such as the audio encoder 300 of Figure 13. The audio encoder may use the signal selection 350 as shown in Figure 14, or may apply the described signal selection 350 to more than three input audio signals, to select the directional loudness maps 142a of the candidate signals according to the contribution of the individual directional loudness maps of the candidate signals to the overall directional loudness map 142b, or the directional loudness maps 142a of the pairs of candidate signals as shown in Figure 14. 1 From 142a 3 may be configured to select a jointly encoded signal from multiple candidate signals, or from multiple pairs of candidate signals, depending on their contribution to the overall directional loudness map 142b.

[0193] According to FIG. 14, for each possible signal pair, for example, a directional volume map 142a 1 From 142a 3 is received by the signal selection 350, and an overall directional loudness map 142b relating to all three signals of the input audio content is received by the signal selection unit 350. 1 ~142a 3The directional loudness map of the input audio signal 112 and the global directional loudness map 142b can be received from an audio analyzer or can be determined by an audio encoder and provided for signal selection 350. According to one embodiment, the global directional loudness map 142b can represent the overall audio scene represented by, for example, the input audio content, for example, prior to processing by the audio encoder ... 1 From 112 3 represents loudness information related to different directions of, e.g., audio components of, an audio scene that is or is to be represented, e.g., after decoder-side rendering. The global directional loudness map 142b is represented, e.g., as DirLoudMap(1,2,3). According to one embodiment, the global directional loudness map 142b is 1 From 112 3 or by using a downmix of the input audio signal 112 1 From 112 3 The binauralization of the input signal is determined by the audio encoder.

[0194] FIG. 14 shows a first input audio signal 112 1 , the second input audio signal 112 2 , or the third input audio signal 112 3 1 shows a signal selection 350 for each of the three channels CH1 to CH3 associated with the first directional volume map 142a. 1 For example, DirLoudMap(1,2) is the first input audio signal 112 1 and the second input audio signal 112 2 Based on the second directional volume map 142a 2 For example, DirLoudMap(2,3) corresponds to the second input audio signal 112 2 and the third input audio signal 112 3 Based on the above, a third directional volume map 142a 3For example, DirLoudMap(1,3) is the first input audio signal 112 1 and the third input audio signal 112 3 Based on.

[0195] According to one embodiment, each directional loudness map 142 represents loudness information related to a different direction. The different directions are indicated in FIG. 14 by lines between L and R, where L is associated with panning the audio component to the left and R is associated with panning the audio component to the right. Thus, the different directions include the left and right sides and the directions or angles between the left and right sides. Although the directional loudness map 142 shown in FIG. 14 is represented as a diagram, alternatively, the directional loudness map 142 can be represented by a directional loudness histogram as shown in FIG. 5 or a matrix as shown in FIG. 10a to 10c. It is clear that only the information related to the directional loudness map 142 is relevant for the signal selection 350 and the graphical representation is only for the purpose of improving understanding.

[0196] According to one embodiment, signal selection 350 is performed such that the contribution of the candidate signal pair to the overall directional loudness map 142b is determined. 1 ~142a 3 The relationship between can be described by the following equation:

[0197] DirLoudMap(1,2,3)=a*DirLoudMap(1,2,3)+b*DirLoudMap(2,3)+c*DirLoudMap(1,3). The contributions determined by an audio encoder using signal selection can be represented by the coefficients a, b, and c.

[0198] According to one embodiment, the audio encoder selects the candidate signal 112 having the largest contribution to the overall directional loudness map 142b for joint coding. 1 From 112 3This means, for example, that a pair of candidate signals is selected by signal selection 350 that is associated with the highest coefficient among coefficients a, b and c.

[0199] Alternatively, the audio encoder may select candidate signals 112 having a contribution to the global directional loudness map 142b that is greater than a predefined threshold for joint coding. 1 From 112 3 This means, for example, that a predetermined threshold is selected, each coefficient a, b, c is compared with the predetermined threshold, and each signal pair associated with a coefficient larger than the predetermined threshold is selected.

[0200] According to one embodiment, the contribution can be in the range of 0% to 100%, which means for example a range of 0 to 1 for the coefficients a, b and c. A contribution of 100% is for example associated to a directional loudness map 142a that is exactly equal to the overall directional loudness map 142b. According to one embodiment, the predefined threshold depends on how many input audio signals are included in the input audio content. According to one embodiment, the predefined threshold can be defined as a contribution of at least 35%, or at least 50%, or at least 60%, or at least 75%.

[0201] According to one embodiment, the predefined threshold depends on the number of signals that must be selected by the signal selection 350 for joint coding. For example, if at least two signal pairs must be selected, two signal pairs can be selected, which are associated with the directional loudness map 142a that has the highest contribution to the overall directional loudness map 142b. This means, for example, that the signal pairs with the highest contribution and the next highest contribution are selected 350.

[0202] Since the comparison of the directional loudness maps can indicate the quality of the perception of the encoded audio signal by the listener, it is advantageous to base the selection of the signal to be encoded by the audio encoder on the directional loudness map 142. According to one embodiment, the signal selection 350 is performed by the audio encoder such that a signal pair or signal pairs are selected whose directional loudness map 142a is most similar to the overall directional loudness map 142b. This can result in a similar perception of the selected candidate pair or pairs compared to the perception of all input audio signals. This can improve the quality of the encoded audio content.

[0203] Fig. 15 illustrates an embodiment of an audio encoder 300 for encoding 310 an input audio content 112 comprising one or more input audio signals. Preferably, two or more input audio signals are encoded 310 by the audio encoder 300. The audio encoder 300 is configured to provide one or more encoded audio signals 320 based on two or more input audio signals 112 or based on two or more signals 110 derived therefrom. The signals 110 may be derived from the input audio signals 112 by an optional process 330. According to an embodiment, the optional process 330 may include features and / or functionality as described with respect to the other audio encoders 300 described herein. By encoding 310, the signals to be encoded are losslessly encoded, for example after being quantized.

[0204] The audio encoder 300 is configured to determine 100 an overall directional loudness map based on the input audio signal 112 and / or to determine 100 one or more individual directional loudness maps 142 associated with the individual input audio signals 112. The overall directional loudness map is L(m,φ 0,j ) and each directional volume map can be expressed as L i (m,φ 0,j) according to one embodiment, the global directional loudness map may represent a target directional loudness map for the scene. In other words, the global directional loudness map may be associated with a desired directional loudness map for a combination of encoded audio signals. Additionally or alternatively, the directional loudness map L of a signal pair or group of three or more signals may be represented as i (m,φ 0,j ) may be determined 100 by the audio encoder 300.

[0205] The audio encoder 300 is configured to encode 310 the overall directional loudness map 142 and / or one or more individual directional loudness maps 142 and / or one or more directional loudness maps of signal pairs or groups of three or more input audio signals 112 as side information. The encoded audio content 320 thus comprises the encoded audio signals and the encoded directional loudness map. According to an embodiment, the encoding 310 can rely on one or more directional loudness maps 142, thereby advantageously encoding these directional loudness maps 142 as well to enable high-quality decoding of the encoded audio content 320. When the directional loudness maps 142 are used as encoded side information, the encoded audio content 320 provides the originally intended quality characteristics (e.g., achievable by the encoding 310 and / or the audio decoder).

[0206] According to one embodiment, the audio encoder 300 calculates a global directional loudness map L(m,φ) based on the input audio signal 112, such that the global directional loudness map represents loudness information associated with different directions, e.g. audio components, of the audio scene represented by the input audio signal 112. 0,j ) or a global directional volume map L(m,φ 0,j) represents loudness information relating to different orientations of, e.g., audio components of, an audio scene to be represented after, e.g., decoder-side rendering with the input audio signal. The loudness information determination 100 can be performed by the audio encoder 300, optionally in combination with knowledge or side information regarding the positions of the speakers and / or knowledge or side information describing the positions of audio objects within the input audio signal 112. According to one embodiment, the loudness information determination 100 may be implemented as described in other audio encoders 300 described herein.

[0207] The audio encoder 300 may, for example, generate a global directional volume map L(m,φ 0,j ) in the form of a set of values, e.g. scalar values, associated with different directions. According to one embodiment, the values ​​are further associated with a number of frequency bins of a frequency band. One or more respective values ​​in the discrete directions of the overall directional loudness map can be coded. This means, for example, that each value of a color matrix as shown in Fig. 10a-c, or values ​​of different histogram bins as shown in Fig. 5, or values ​​of a directional loudness map curve as shown in Fig. 14 for discrete directions are coded.

[0208] Alternatively, the audio encoder 300 may, for example, use the center position values ​​and gradient information to generate a global directional loudness map L(m,φ 0,j ). The central location value describes, for example, an angle or direction in which a maximum value of the global directional loudness map of a given frequency band or frequency bin, or multiple frequency bins or frequency bands, is located. The gradient information represents, for example, one or more scalar values ​​describing a gradient of the values ​​of the global directional loudness map in an angular direction. The scalar values ​​of the gradient information are, for example, values ​​of the global directional loudness map in directions adjacent to the central location value. The central location value can represent a scalar value of the loudness information and / or a scalar value of a direction corresponding to the loudness value.

[0209] Alternatively, the audio encoder may generate a global directional loudness map L(m,φ), e.g., in the form of a polynomial representation or in the form of a spline representation. 0,j ) is configured to encode

[0210] According to one embodiment, the global directional volume map L(m,φ 0,j The above-mentioned encoding possibilities 310 of the individual directional volume maps L i (m,φ 0,j ) and / or directional loudness maps associated with signal pairs or groups of three or more signals.

[0211] According to one embodiment, the audio encoder 300 receives a number of input audio signals 112 and a global directional loudness map L(m,φ 0,j Optionally, the contribution of a directional loudness map associated with the downmix signal to the overall directional loudness map is also coded, e.g. as side information.

[0212] Alternatively, the audio encoder 300 may, for example, encode 310 a plurality of signals, for example an input audio signal 112 or a signal 110 derived therefrom, and generate individual loudness maps L of the encoded 310 plurality of signals 112 / 110. i (m,φ 0,j ) (e.g., individual signals, signal pairs, or groups of three or more signals). The encoded multiple signals and the encoded individual directional volume maps are, for example, transmitted to or included in the encoded audio representation 320.

[0213] According to an alternative embodiment, the audio encoder 300 calculates a global directional loudness map L(m,φ 0,j), a number of signals, e.g. an input audio signal 112 or a signal 110 derived therefrom, and a contribution to be encoded into an overall directional loudness map, and is configured to encode (310) parameters describing the relative contributions of the signals. According to one embodiment, the parameters can be represented by parameters a, b and c as described in Fig. 14. Thus, for example, the audio encoder 300 is configured to encode 310 all information on which the encoding 310 is based, e.g. to provide information for a high quality decoding of the provided encoded audio content 320.

[0214] According to one embodiment, the audio encoder may include or combine individual features and / or functionality as described with respect to one or more of the audio encoders 300 described in Figures 11-15.

[0215] 16 shows an embodiment of an audio decoder 400 for decoding 410 encoded audio content 420. The encoded audio content 420 may include encoded representations 422 of one or more audio signals and encoded directional volume map information 424.

[0216] The audio decoder 400 is configured to receive an encoded representation 422 of one or more audio signals and provide a decoded representation 412 of the one or more audio signals. Additionally, the audio decoder 400 is configured to receive encoded directional volume map information 424 and decode 410 the encoded directional volume map information 424 to obtain one or more decoded directional volume maps 414. The decoded directional volume maps 414 may include features and / or functionality as described with respect to the directional volume map 142 above.

[0217] According to one embodiment, the decoding 410 may be performed by the audio decoder 400 using AAC-like decoding, or using decoding of entropy coded spectral values, or using decoding of entropy coded loudness values.

[0218] The audio decoder 400 is configured to reconstruct (430) an audio scene using the decoded representation 412 of the one or more audio signals and using one or more directional volume maps 414. Based on the reconstruction 430, a decoded audio content 432, such as a multi-channel representation, can be determined by the audio decoder 400.

[0219] According to one embodiment, the directional loudness map 414 may represent a target directional loudness map achievable by the decoded audio content 432. The directional loudness map 414 may thus be used to optimize the reconstruction of the audio scene 430 to provide a high quality listener's perception of the decoded audio content 432. This is based on the idea that the directional loudness map 414 may be indicative of the listener's desired perception.

[0220] FIG. 17 illustrates the encoder 400 of FIG. 16 with an optional feature of adaptation of decoding parameters 440. According to an embodiment, the decoded audio content may include an output signal 432, for example representing a time domain signal or a spectral domain signal. The audio decoder 400 is configured to obtain the output signal 432, for example such that one or more directional loudness maps associated with the output signal 432 approximate or are equal to one or more target directional loudness maps. The one or more target directional loudness maps are based on or equal to one or more decoded directional loudness maps 414. Optionally, the audio decoder 400 is configured to determine the one or more target directional loudness maps using an appropriate scaling or a combination of one or more decoded directional loudness maps 414.

[0221] According to one embodiment, the one or more directional loudness maps associated with the output signal 432 may be determined by the audio decoder 400. The audio decoder 400 may, for example, comprise an audio analyzer for determining the one or more directional loudness maps associated with the output signal 432 or may be configured to receive the one or more directional loudness maps associated with the output signal 432 from an external audio analyzer 100.

[0222] According to one embodiment, the audio decoder 400 is configured to compare one or more directional loudness maps associated with the output signal 432 with the decoded directional loudness map 414 or to compare one or more directional loudness maps associated with the output signal 432 with a directional loudness map derived from the decoded directional loudness map 414 and adapt 440 the decoding parameters or the reconstruction 430 based on this comparison. According to one embodiment, the audio decoder 400 is configured to adapt 440 the decoding parameters or adapt the reconstruction 430 such that a deviation between the one or more directional loudness maps associated with the output signal 432 and the one or more target directional loudness maps is below a predefined threshold. This may represent a feedback loop, whereby the decoding 410 and / or the reconstruction 430 are adapted such that the one or more directional loudness maps associated with the output signal 432 approximate the one or more target directional loudness maps by at least 75%, or at least 80%, or at least 85%, or at least 90%, or at least 95%.

[0223] According to an embodiment, the audio decoder 400 is configured to receive an encoded downmix signal as encoded representations 422 of one or more audio signals and to receive an overall directional loudness map as encoded directional loudness map information 424. The encoded downmix signal is for example obtained based on a plurality of input audio signals. Alternatively, the audio decoder 400 is configured to receive a plurality of encoded audio signals as encoded representations 422 of one or more audio signals and to receive individual directional loudness maps of the plurality of encoded signals as encoded directional loudness map information 424. The encoded audio signals represent for example input audio signals encoded by an encoder or signals derived from input audio signals encoded by an encoder. Alternatively, the audio decoder 400 is configured to receive an overall directional loudness map as encoded directional loudness map information 424 and a plurality of encoded audio signals as encoded representations 422 of one or more audio signals and further to receive parameters describing the contribution of the encoded audio signals to the overall directional loudness map. Thus, the encoded audio content 420 may further include parameters, which the audio decoder 400 may be configured to use to improve the adaptation of the decoding parameters 440 and / or to improve the reconstruction of the audio scene 430. The audio decoder 400 is configured to provide an output signal 432 based on one of the aforementioned encoded audio contents 420 .

[0224] 18 shows an embodiment of a format converter 500 for converting 510 a format of audio content 520 representing an audio scene. The format converter 500, for example, inputs audio content 520 in a first format and converts 510 the audio content 520 into audio content 530 in a second format. In other words, the format converter 500 is configured to provide a representation 530 of the audio content in the second format based on the representation 520 of the audio content in the first format. According to an embodiment, the audio content 520 and / or the audio content 530 may represent a spatial audio scene.

[0225] The first format may, for example, include a first number of channels or input audio signals and side information or spatial side information adapted to the first number of channels or input audio signals. The second format may, for example, include a second number of channels or output audio signals that may differ from the first number of channels or input audio signals and side information or spatial side information adapted to the second number of channels or output audio signals. The audio content 520 in the first format may, for example, include one or more audio signals, one or more downmix signals, one or more residual signals, one or more intermediate signals, one or more side signals and / or one or more distinct signals.

[0226] The format converter 500 is configured to adjust 540 the complexity of the format conversion 510 depending on the contribution of the input audio signal in the first format to the overall directional loudness map 142 of the audio scene. The audio content 520 comprises, for example, an input audio signal in the first format. The contribution can directly represent the contribution of the input audio signal in the first format to the overall directional loudness map 142 of the audio scene, or can represent the contribution of individual directional loudness maps of the input audio signal in the first format to the overall directional loudness map 142, or can represent the contribution of paired directional loudness maps of the input audio signal in the first format to the overall directional loudness map 142. According to an embodiment, the contribution can be calculated by the format converter 500 as described in Fig. 13 or Fig. 14. According to an embodiment, the overall directional loudness map 142 may, for example, be described by side information of the first format received by the format converter 500. Alternatively, the format converter 500 is configured to determine the overall directional loudness map 142 based on the input audio signal of the audio content 520. Optionally, the format converter 500 includes an audio analyzer as described with respect to Figures 1 to 4b to calculate the overall directional volume map 142, or the format converter 500 is configured to receive the overall directional volume map 142 from an external audio analyzer as described with respect to Figures 1 to 4b.

[0227] The audio content 520 in the first format may include directional loudness map information of the input audio signals in the first format. Based on the directional loudness map information, the format converter 500 is configured to obtain, for example, the overall directional loudness map 142 and / or one or more directional loudness maps. The one or more directional loudness maps may represent a directional loudness map of each input audio signal in the first format and / or a directional loudness map of a group or pair of signals in the first format. The format converter 500 is configured to derive, for example, the overall directional loudness map 142 from the one or more directional loudness maps or directional loudness map information.

[0228] The complexity adjustment 540 is performed such that it is controlled whether skipping of one or more of the input audio signals of the first format that contribute to the directional loudness map below a threshold is possible. In other words, the format converter 500 is configured to, for example, calculate or estimate the contribution of a given input audio signal to the overall directional loudness map 142 of the audio scene and to decide whether to consider the given input audio signal in the format conversion 510 depending on the calculation or estimation of the contribution. The calculated or estimated contribution is, for example, compared by the format converter 500 with a predefined absolute or relative threshold.

[0229] The contribution of the first format input audio signals to the overall directional loudness map 142 may indicate the relevance of the respective input audio signal to the perceived quality of the audio content 530 in the second format, so that, for example, only the more relevant first format audio signals are format converted 510, resulting in a high quality audio content 530 in the second format.

[0230] Fig. 19 shows an audio decoder 400 for decoding 410 encoded audio content 420. The audio decoder 400 is configured to receive an encoded representation 420 of one or more audio signals and provide a decoded representation 412 of the one or more audio signals. The decoding 410 uses, for example, AAC-like decoding or decoding of entropy-coded spectral values. The audio decoder 400 is configured to reconstruct 430 an audio scene using the decoded representation 412 of the one or more audio signals. The audio decoder 400 is configured to adjust 440 a decoding complexity depending on the contribution of the encoded signal to the overall directional loudness map 142 of the decoded audio scene 434. The decoding complexity adjustment 440 may be performed by the audio decoder 400 in a similar manner as the complexity adjustment 540 of the format converter 500 of FIG.

[0231] According to one embodiment, the audio decoder 400 is configured to receive encoded directional loudness map information, e.g. extracted from the encoded audio content 420. The encoded directional loudness map information can be decoded 410 by the audio decoder 400 to determine decoded directional loudness information 414. Based on the decoded directional loudness information 414, an overall directional loudness map of one or more audio signals of the encoded audio content 420 and / or one or more individual directional loudness maps of one or more audio signals of the encoded audio content 420 can be obtained. The overall directional loudness map of one or more audio signals of the encoded audio content 420 is, e.g., derived from one or more individual directional loudness maps.

[0232] The overall directional loudness map 142 of the decoded audio scene 434 may be calculated by a directional loudness map determination 100 which may optionally be performed by the audio decoder 400. According to an embodiment, the audio decoder 400 comprises an audio analyzer as described with respect to Figure 1 or Figure 4b in order to perform the directional loudness map determination 100, or the audio decoder 400 may transmit the decoded audio scene 434 to an external audio analyzer and receive the overall directional loudness map 142 of the decoded audio scene 434 from the external audio analyzer.

[0233] According to one embodiment, the audio decoder 400 is configured to calculate or estimate a contribution of a given encoded signal to the global directional loudness map 142 of a decoded audio scene and to decide (410) whether to decode the given encoded signal depending on the calculated or estimated contribution. Thus, for example, the global directional loudness map of one or more audio signals of the encoded audio content 420 may be compared to the global directional loudness map of the decoded audio scene 434. The determination of the contribution may be performed as described above (e.g., as described with respect to Figure 13 or Figure 14) or similarly.

[0234] Alternatively, the audio decoder 400 is configured to calculate or estimate a contribution of a given encoded signal to a decoded overall directional volume map 414 of an encoded audio scene, and decide (410) whether to decode the given encoded signal depending on the calculated or estimated contribution.

[0235] The complexity adjustment 440 is performed, for example, to control whether one or more of the coded representations of one or more input audio signals that contribute to a directional loudness map below a threshold can be skipped. Additionally or alternatively, the decoding complexity adjustment 440 may be configured to adapt the decoding parameters based on the contribution.

[0236] Additionally or alternatively, the decoding complexity adjustment 440 may be configured to compare the decoded directional loudness map 414 with an overall directional loudness map of the decoded audio scene 434 (e.g., the overall directional loudness map of the decoded audio scene 434 is a target directional loudness map) to adapt the decoding parameters.

[0237] Fig. 20 shows an embodiment of a renderer 600. The renderer 600 is for example a binaural renderer, a soundbar renderer or a loudspeaker renderer. In the renderer 600, an audio content 620 is rendered to obtain a rendered audio content 630. The audio content 620 may comprise one or more input audio signals 622. The renderer 600 uses the one or more input audio signals 622 for example to reconstruct 640 an audio scene. Preferably, the reconstruction 640 performed by the renderer 600 is based on two or more input audio signals 622. According to an embodiment, the input audio signals 622 may comprise one or more audio signals, one or more downmix signals, one or more residual signals, other audio signals and / or additional information.

[0238] According to one embodiment, for the reconstruction 640 of the audio scene, the renderer 600 is configured to analyze one or more input audio signals 622 in order to optimize the rendering to obtain a desired audio scene. Thus, for example, the renderer 600 is configured to modify the spatial arrangement of the audio objects of the audio content 620. This means, for example, that the renderer 600 can reconstruct 640 a new audio scene. The new audio scene includes, for example, rearranged audio objects compared to the original audio scene of the audio content 620. This means, for example, that the guitarist and / or singer and / or other audio objects are positioned in the new audio scene in a different spatial position than in the original audio scene.

[0239] Additionally or alternatively, multiple audio channels or relationships between audio channels may be rendered by the audio renderer 600. Thus, for example, the renderer 600 may render the audio content 620, which includes a multi-channel signal, into, for example, a two-channel signal. This may be desirable, for example, if only two speakers are available for the presentation of the audio content 620.

[0240] According to one embodiment, the rendering is performed by the renderer 600 in such a way that the new audio scene shows only small deviations with respect to the original audio scene.

[0241] The renderer 600 is configured to adjust 650 the rendering complexity depending on the contribution of the input audio signal 622 to the global directional loudness map 142 of the rendered audio scene 642. According to an embodiment, the rendered audio scene 642 may represent a new audio scene as described above. According to an embodiment, the audio content 620 may include the global directional loudness map 142 as side information. This global directional loudness map 142 received by the renderer 600 as side information may indicate a desired audio scene of the rendered audio content 630. Alternatively, the directional loudness map determination 100 may determine the global directional loudness map 142 based on the rendered audio scene received from the reconstruction unit 640. According to an embodiment, the renderer 600 may include the directional loudness map determination 100 or may receive the global directional loudness map 142 of an external directional loudness map determination 100. According to an embodiment, the directional loudness map determination 100 may be performed by an audio analyzer as described above.

[0242] According to one embodiment, the rendering complexity adjustment 650 is performed, for example, by skipping one or more of the input audio signals 622. The skipped input audio signals 622 are, for example, signals that contribute to the directional loudness map 142 below a threshold. Thus, only the relevant input audio signals are rendered by the audio renderer 600.

[0243] According to one embodiment, the renderer 600 is configured to calculate or estimate a contribution of a given input audio signal 622 to the overall directional loudness map 142 of an audio scene, for example of a rendered audio scene 642. Furthermore, the renderer 600 is configured to decide whether to take the given input audio signal into account in the rendering depending on the calculation or estimation of the contribution. Thus, for example, the calculated or estimated contribution is compared to a predefined absolute or relative threshold.

[0244] Fig. 21 shows a method 1000 for analysing an audio signal. The method comprises the steps of: L ,x R ,x i ) one or more spectral domain (e.g., time-frequency domain) representations of the TIFF0007631273000115.tif311, e.g. i={L;R}; or TIFF0007631273000116.tif312) based on multiple weighted spectral domain (e.g., time-frequency domain) representations (different TIFF0007631273000117.tif33(j About TIFF0007631273000118.tif22[1;J] TIFF0007631273000119.tif329, “directional signal”) 1100. TIFF0007631273000120.tif310) is a representation of multiple weighted spectral domains (different For TIFF0007631273000121.tif33 TIFF0007631273000122.tif329(j TIFF0007631273000123.tif22[1;J]; “directional signal”), by detecting different directions (e.g., panning directions) of audio components (e.g., of spectral bins or spectral bands) (e.g., tuning from an instrument or a singer) in two or more input audio signals. TIFF0007631273000124.tif33) (e.g. weighting factor The weighting 1200 is performed according to a plurality of weighted spectral domain representations (represented by TIFF0007631273000125.tif38). TIFF0007631273000126.tif33(j For TIFF0007631273000127.tif22[1;J]) TIFF0007631273000128.tif329; "directional signal") based on different directions (e.g. panning direction TIFF0007631273000129.tif33) related volume information (e.g., multiple different For TIFF0007631273000130.tif33, L(m, TIFF0007631273000131.tif34); for example, a “directional volume map” is obtained 1300 as an analysis result.

[0245] FIG. 22 shows a method 2000 for assessing the similarity of audio signals. The method comprises: R ,x L ,x i ) based on different (e.g., panning) directions (e.g., The first volume information (L 1 (m, TIFF0007631273000133.tif34); directional volume map; composite volume value) is acquired 2100, and the first volume information (L 1 (m, TIFF0007631273000134.tif34)) in different panning directions (e.g. TIFF0007631273000135.tif34) 2 (m, TIFF0007631273000136.tif34); reference loudness information; reference directional loudness map; reference composite loudness value) and two or more reference audio signals (x 2,R ,x 2,L ,x 2,i ) to compare 2200 a first set of two or more input audio signals with two or more reference audio signals (x R ,x L ,x i) and two or more reference audio signals (x 2,R ,x 2,L ,x 2,i ) (or representing the quality of the first set of two or more input audio signals when compared to the first set of two or more reference audio signals).

[0246] 23 shows a method 3000 for encoding an input audio content including one or more input audio signals (preferably a plurality of input audio signals). The method includes providing 3100 one or more encoded (e.g., quantized and then losslessly encoded) audio signals (e.g., encoded spectral domain representations) based on the one or more input audio signals (e.g., left and right signals) or one or more signals derived therefrom (e.g., intermediate or downmix signals and side or difference signals). Furthermore, the method 3000 includes adapting 3200 the provision of one or more encoded audio signals depending on one or more directional loudness maps representing loudness information associated with a plurality of different directions (e.g., panning directions) of the one or more signals to be encoded (e.g., depending on the contribution of the individual directional loudness maps of the one or more signals to be quantized to an overall directional loudness map associated with the plurality of input audio signals (e.g., each signal of the one or more input audio signals)).

[0247] FIG. 24 shows a method 4000 for encoding input audio content, which includes one or more input audio signals (preferably a plurality of input audio signals). The method includes providing 4100 one or more encoded (e.g., quantized and then losslessly encoded) audio signals (e.g., encoded spectral domain representations) of two or more signals to be jointly encoded (e.g., using an intermediate signal or downmix signal and a side signal or difference signal) based on two or more input audio signals (e.g., a left signal and a right signal) or based on two or more signals derived therefrom. Further, the method 4000 includes selecting 4200 a signal to be jointly encoded from among the multiple candidate signals or from among the multiple pair of candidate signals (e.g., from the two or more input audio signals or from two or more signals derived from the two or more input audio signals) in response to a directional loudness map representing loudness information related to a plurality of different directions (e.g., panning directions) of the candidate signal or pair of candidate signals (e.g., in response to a contribution of an individual directional loudness map of the candidate signals to an overall directional loudness map associated with the plurality of input audio signals (e.g., each signal of the one or more input audio signals) or in response to a contribution of a directional loudness map of a pair of candidate signals to an overall directional loudness map).

[0248] 25 shows a method 5000 for encoding input audio content including one or more input audio signals (preferably a plurality of input audio signals). The method includes providing 5100 one or more encoded (e.g., quantized and then losslessly encoded) audio signals (e.g., encoded spectral domain representations) based on two or more input audio signals (e.g., a left signal and a right signal) or based on two or more signals derived therefrom. Furthermore, the method 5000 includes determining 5200 an overall directional loudness map (e.g., a target directional loudness map of a scene) based on the input audio signals and / or determining one or more individual directional loudness maps associated with the individual input audio signals, and encoding 5300 the overall directional loudness map and / or the one or more individual directional loudness maps as side information.

[0249] 26 shows a method 6000 for decoding encoded audio content, including receiving 6100 an encoded representation of one or more audio signals and providing 6200 a decoded representation of the one or more audio signals (e.g., using AAC-like decoding or using decoding of entropy-coded spectral values). The method 6000 includes receiving 6300 encoded directional loudness map information, decoding 6400 the encoded directional loudness map information, and obtaining 6500 one or more (decoded) directional loudness maps. Furthermore, the method 6000 includes reconstructing 6600 an audio scene using the one or more directional loudness maps using the decoded representation of the one or more audio signals.

[0250] FIG. 27 illustrates a method 7000 for converting 7100 the format of audio content representing an audio scene (e.g., a spatial audio scene) from a first format to a second format (the first format may, for example, include a first number of channels or input audio signals and side information or spatial side information adapted to the first number of channels or input audio signals, and the second format may, for example, include a second number of channels or output audio signals that may differ from the first number of channels or input audio signals and side information or spatial side information adapted to the second number of channels or output audio signals). The method 7000 includes providing a representation of audio content in a second format based on a representation of audio content in a first format, and includes adjusting 7200 a complexity of the format conversion depending on the contribution of input audio signals (e.g., one or more audio signals, one or more downmix signals, one or more residual signals, etc.) of the first format to an overall directional loudness map of the audio scene (the overall directional loudness map may be described, for example, by side information of the first format received by the format converter).

[0251] FIG. 28 shows a method 8000 for decoding encoded audio content, comprising receiving 8100 an encoded representation of one or more audio signals and providing 8200 a decoded representation of the one or more audio signals (e.g., using AAC-like decoding or using decoding of entropy-coded spectral values). The method 8000 comprises reconstructing 8300 an audio scene using the decoded representation of the one or more audio signals. Furthermore, the method 8000 comprises adjusting 8400 a decoding complexity depending on the contribution of the encoded signals (e.g., one or more audio signals, one or more downmix signals, one or more residual signals, etc.) to the overall directional loudness map of the decoded audio scene.

[0252] FIG. 29 shows a method 9000 for rendering audio content (e.g., to upmix audio content represented using a first number of input audio channels and side information describing desired spatial characteristics such as placement of audio objects or relationships between the audio channels to a representation including a larger number of channels than the first number of input audio channels), which includes reconstructing 9100 an audio scene based on one or more input audio signals (or based on two or more input audio signals). The method 9000 includes adjusting 9200 a rendering complexity depending on the contribution of the input audio signals (e.g., one or more audio signals, one or more downmix signals, one or more residual signals, etc.) to an overall directional loudness map of the rendered audio scene (e.g., by skipping in the rendering process one or more of the input audio signals that contribute to the directional loudness map below a threshold). (The overall directional loudness map may, for example, be described by side information in a first format received by the renderer). remarks

[0253] In the following, various embodiments and aspects of the present invention are described in the chapters "Objective assessment of spatial audio quality using directional loudness maps", "Using directional loudness for audio coding and objective quality measurements", "Directional loudness for audio coding", "General steps for computing a directional loudness map (DirLoudMap)", "Example: Restoring a directional signal using a window / selection function derived from a panning index", and "Embodiments of different forms of computing a loudness map using a generalized criterion function". Further embodiments are defined by the appended claims.

[0254] It is to be noted that any embodiment defined by the claims can be supplemented by any of the details (features and functions) described in the preceding paragraphs.

[0255] Also, the embodiments described in the above paragraphs can be used individually or can be supplemented by any of the features of the other paragraphs or by any features included in the claims.

[0256] It should also be noted that the individual aspects described herein may be used individually or in combination, and thus details may be added to each of the individual aspects without adding the details to another one of the aspects.

[0257] It should also be noted that this disclosure explicitly or implicitly describes functionality usable in an audio encoder (apparatus for providing an encoded representation of an input audio signal) and an audio decoder (apparatus for providing a decoded representation of an audio signal based on the encoded representation). Thus, any of the features described herein may be used in the context of an audio encoder as well as in the context of an audio decoder.

[0258] Moreover, features and functions disclosed herein in relation to a method may also be used in an apparatus (configured to perform such functions). Moreover, any features and functions disclosed herein in relation to an apparatus may also be used in a corresponding method. In other words, the methods disclosed herein may be complemented by any of the features and functions described in relation to the apparatus.

[0259] Additionally, any of the features and functionality described herein can be implemented in hardware or software, or using a combination of hardware and software, as described in the "Implementation Alternatives" section. Implementation Alternatives

[0260] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, in which a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0261] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM or flash memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to execute the respective method. Thus, the digital storage medium may be computer readable.

[0262] Some embodiments according to the invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to cause one of the methods described herein to be performed.

[0263] Generally, embodiments of the present invention can be implemented as a computer program product having program code that operates to perform one of the methods when the computer program product is run on a computer. The program code can, for example, be stored on a machine readable carrier. Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0264] In other words, therefore, an embodiment of the inventive methods is a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0265] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recorded medium is typically tangible and / or non-transitory.

[0266] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, The data stream or the sequence of signals can for example be arranged to be transferred via a data communication connection, for example via the Internet.

[0267] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein. A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0268] Further embodiments according to the invention comprise an apparatus or system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may comprise, for example, a file server for transferring the computer program to the receiver.

[0269] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0270] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer. The apparatus described herein, or any components of the apparatus described herein, may be implemented at least in part in hardware and / or software.

[0271] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0272] Any of the methods described herein, or components of the apparatus described herein, may be implemented at least in part by hardware and / or software.

[0273] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the claims that follow, and not by the specific details presented as descriptions and explanations of the embodiments herein. Objective assessment of spatial audio quality using directional loudness maps summary

[0274] This work introduces features extracted from stereo / binaural audio signals that serve as a measure of perceived quality degradation in a processed spatial auditory scene, for example. The features can be based on a simplified model that assumes a stereo mix produced by directional signals positioned using amplitude level panning techniques. For example, we calculate the associated loudness in the stereo image for each directional signal in the short-time Fourier transform (STFT) domain to compare a reference signal with a degraded version, and derive distortion measures aimed at describing the perceived degradation scores reported in listening tests.

[0275] The measure has been tested on an extensive listening test database with stereo signals processed by state-of-the-art perceptual audio codecs using non-waveform-preserving techniques such as bandwidth extension and joint stereo coding, which are known to present challenges to existing quality predictors [1], [2]. Results suggest that the derived distortion measure can be incorporated as an extension to existing automatic perceptual quality assessment algorithms to improve prediction of spatially coded audio signals. Index Terms- Spatial Audio, Objective Quality Assessment, PEAQ, Panning Index. 1. Introduction

[0276] For example, we propose a simple feature

[13] that aims to describe the degradation of the perceived auditory stereo image based on loudness changes in regions that share a common panning index, i.e., time and frequency regions of the binaural signal that share the same intensity level ratio between the left and right channels and thus correspond to a given perceived direction in the horizontal plane of the auditory image.

[0277] The use of directional loudness measurements in the context of auditory scene analysis for the audio rendering of complex virtual environments has also been proposed in

[14] , but the current work focuses on the objective assessment of the quality of the overall spatial audio coding.

[0278] The distortion of the perceived stereo image can be reflected as a change on a directional loudness map of a given granularity that corresponds to the amount of the panning index value evaluated as a parameter. 2. Method

[0279] According to one embodiment, a reference signal (REF) and a signal under test (SUT) are processed in parallel to extract features which, when compared, aim to describe the degradation of the perceived hearing quality caused by the operations performed to generate the SUT.

[0280] Both binaural signals can be first processed by a peripheral ear model block. Each input signal is processed by a block of, for example, size TIFF0007631273000137.tif212 Sample Hann window and TIFF0007631273000138.tifDecomposed into STFT domain using 35 overlaps, TIFF0007631273000139.tif214 gives a time resolution of 21 ms. The frequency bins of the converted signal are then, for example, TIFF0007631273000140.tif29 Frequency bin subsets or bands are grouped to take into account the frequency selectivity of the human cochlea according to the ERB scale

[15] . Each band can then be weighted by a value derived from a combined linear transfer function that models the outer and middle ear as described in [3].

[0281] The peripheral model is then TIFF0007631273000141.tif22 and frequency bins The signal is TIFF0007631273000142.tif22 TIFF0007631273000143.tif310, different widths represented in frequency bins TIFF0007631273000144.tif33 with each channel TIFF0007631273000145.tif310 and each frequency group Outputs TIFF0007631273000146.tif317. 2.1. Calculation of Directional Loudness (Performed, for example, by an Audio Analyzer and / or Audio Similarity Evaluator Described Herein)

[0282] According to one embodiment, the directional volume calculation is, for example, TIFF0007631273000147.tif33 is jε[1;J] TIFF0007631273000148.tif34 can be interpreted as: TIFF0007631273000148.tif34 The following concept is based on the method presented in

[13] , in which a similarity measure between the left and right channels of a binaural signal in the STFT domain can be used to extract the time and frequency domains occupied by each sound source in a stereo recording based on their specified panning coefficients during the mixing process.

[0283] Peripheral Models Given the output of TIFF0007631273000149.tif310, the input is given a window function By multiplying TIFF0007631273000150.tif34, a given panning direction Time-frequency (T / F) tile from the input signal corresponding to TIFF0007631273000151.tif33 TIFF0007631273000152.tif36 can be recovered. TIFF0007631273000153.tif336(1)

[0284] The restored signal is within the tolerance of the panning direction. With the T / F components of the input corresponding to TIFF0007631273000154.tif33. The window function can be defined as a Gaussian window centered on the desired panning direction. TIFF0007631273000155.tif529(2)

[0285] During the ceremony, TIFF0007631273000156.tif38 corresponds to a signal panned fully left or right, respectively The panning index calculated in

[13] using the defined support of TIFF0007631273000157.tif37. TIFF0007631273000158.tif36 is a function of the left and right channel values. TIFF0007631273000159.tif22, The frequency bins may include frequency bins with values ​​at or near TIFF0007631273000160.tif33. All other components may be attenuated according to a Gaussian function. The values ​​in TIFF0007631273000161.tif32 represent the width of the window and therefore the mentioned neighborhood per panning direction. For example, the value of TIFF0007631273000162.tif310 is TIFF0007631273000163.tif was selected for a signal-to-interference ratio (SIR) of 25 dB

[13] . Optionally, Panning directions at equal intervals in TIFF0007631273000164.tif37 The set TIFF0007631273000165.tif23 is The panning direction dependent loudness calculation

[16] for each reconstructed signal at each ERB band is, for example, TIFF0007631273000167.tif541(3)

[0286] During the ceremony, TIFF0007631273000168.tif34 is a channel TIFF0007631273000169.tif310 is the sum signal. The volume is then averaged over all ERB bands, for example, over the time frame Panning region across TIFF0007631273000170.tif312 Provides a directional volume map defined across TIFF0007631273000171.tif22. TIFF0007631273000172.tif428(4)

[0287] For further refinement, according to the dual theory

[17] , Considering only the subset of ERB bands corresponding to the frequency region above 23 kHz, we can calculate Equation 4, which corresponds to the sensitivity of the human auditory system to level differences in this region. According to one embodiment, From TIFF0007631273000174.tif 25kHz TIFF0007631273000175.tifBand corresponding to frequencies up to 35 TIFF0007631273000176.tif314 is used.

[0288] As a step, the duration directional loudness maps of the reference signal and the SUT are e.g. subtracted, and then the absolute value of the residual is averaged over all panning directions and times, following the terminology of [3], to generate a single number called the Model Output Variable (MOV). This number, which effectively represents the distortion between the reference directional loudness map and the SUT, is expected to be a predictor of the associated subjective quality degradation reported in listening tests.

[0289] Figure 9 shows a block diagram of the proposed MOV (Model Output Value) calculation. Figures 10a-c show an example of the application of the concept of directional loudness maps to pairs of reference (REF) and degraded (SUT) signals and the absolute value of their difference (DIFF). Figures 10a-c show an example of a 5-second solo violin recording panned to the left. Clearer areas on the map represent, for example, louder content. The degraded signal (SUT) exhibits a temporary collapse of the panning direction of auditory events from left to center between times 2-2.5 s and again at 3-3.5 s. 3. Experiment Description

[0290] To test and validate the usefulness of the proposed MOV, we conducted regression experiments similar to those in

[18] , where the MOV was calculated for reference and SUT pairs in the database and compared with their respective subjective quality scores from listening tests. The predictive performance of this MOV-based system was compared with the subjective data ( TIFF0007631273000177.tif22), Absolute Error Score ( TIFF0007631273000178.tif25), number of outliers ( It is evaluated by correlation with TIFF0007631273000179.tif22).

[0291] The database used for the experiments corresponds to a part of the Unified Speech Audio Coding (USAC) validation test

[19] set 2, which contains stereo signals coded at bit rates ranging from 16 to 24 kbps using joint stereo

[12] and bandwidth extension tools, together with quality scores on the MUSHRA scale. Speech items were excluded, since the proposed MOV is not expected to describe the main causes of distortion in speech signals. A total of 88 items (e.g., average length 8 seconds) remained in the database for the experiments.

[0292] To account for possible monaural / cerebral distortions in the database, the outputs of an implementation of the standard PEAQ (Advanced Edition) called Objective Difference Grade (ODG) called Mean Opinion Score (MOS) and POLQA were considered as additional MOVs complementing the directional loudness distortions (DirLoudDist; e.g., DLD) described in the previous section. All MOVs could be normalized and adapted to give a score of 0 to indicate the best quality and a score of 1 to indicate the worst possible quality. Listening test scores were scaled accordingly.

[0293] One random portion of the available content of the database (60%, 53 items) was reserved for training a regression model using multivariate fitted regression splines (MARS) [8], which maps MOV to the subjective scores of items. The remainder (35 items) was used to test the performance of the trained regression model. To remove the effect of the training procedure from the overall MOV performance analysis, the train / test cycle was performed 500 times with randomized train / test items, e.g. TIFF0007631273000180.tif22, TIFF0007631273000181.tif25, and The mean value of TIFF0007631273000182.tif22 was considered as the performance measure. 4. Results and Discussion [Table 1]

[0294] Table 1: Average performance values ​​for 500 training / validation (e.g., testing) cycles of regression models using different sets of MOVs. CHOI represents the three binaural MOVs calculated in

[20] , EITDD corresponds to the high-frequency envelope ITD distortion MOV calculated in [1]. SEO corresponds to the four binaural MOVs from [1] including EITDD. DirLoudDist is the proposed MOV. The numbers in brackets represent the total number of MOVs used. (Optional)

[0295] Table 1 shows the average performance values ​​(correlation, absolute error score, number of outliers) of the experiments described in Section 3. In addition to the proposed MOV, the methods proposed in

[20] and [1] for the objective assessment of spatially encoded audio signals were also tested for comparison. Both compared implementations make use of the classical interaural cue distortions mentioned in the introduction, namely IACC Distortion (IACCD), ILD Distortion (ILDD), and ITDD.

[0296] As mentioned above, the baseline performance is given by ODG and MOS, both of which are measured separately. TIFF0007631273000184.tif29, but the combination performance as shown in Table 1 TIFF0007631273000185.tif29 is shown, which confirms that the features are complementary in assessing mono distortion.

[0297] Considering the work of Choi et al.

[20] , the addition of the three binaural distortions (CHOI in Table 1) to the two mono quality indices (comprising up to five joint MOVs) does not provide any further gain to the system in terms of predictive performance for the dataset used.

[0298] In [1], some further optional model refinements were made to the mentioned features, with regard to lateral localization and cue distortion detectability, and also incorporating novel MOVs that take into account, for example, high-frequency envelope interaural time difference distortion (EITDD)

[21] . These set of four binaural MOVs (denoted as SEO in Table 1) + two monaural descriptors (six MOVs in total) significantly improves the system performance on the current dataset.

[0299] Looking at the improvement contributions from EITDD, it has been suggested that the frequency-time-energy envelope used in joint stereo techniques

[12] represents a salient aspect of the perception of overall quality.

[0300] However, the presented MOVs based on directional loudness map distortion (DirLoudDist) correlate even better with the perceived quality degradation than EITDD and even reach similar performance figures as the combination of all binaural MOVs in [1] while using one additional MOV for two mono quality descriptors instead of four. Using fewer features for the same performance reduces the risk of overfitting, indicating their higher perceptual relevance. The maximum average correlation for the subjective scores in the database of TIFF0007631273000186.tif25 indicates that there is still room for improvement.

[0301] According to an embodiment, the proposed features are based on the model described herein and assume a simplified description of stereo signals, in which auditory objects are laterally localized only by ITDs, which is usually the case for studio-produced audio content

[13] . In the case of ITD distortions, which are usually present when encoding multi-microphone recordings or more natural sounds, the model needs to be extended or complemented by appropriate ITD distortion measures. 5. Conclusions and Future Research

[0302] According to one embodiment, a distortion metric is introduced that describes the change in the representation of an auditory scene based on the loudness of events corresponding to a given panning direction. The significant improvement in performance with respect to mono-only quality prediction demonstrates the effectiveness of the proposed method. This approach also suggests a possible alternative or complement in measuring the quality of low-bitrate spatial audio coding, where established distortion measures based on classical binaural cues do not perform satisfactorily, possibly due to the non-waveform-preserving nature of the associated audio processing.

[0303] Performance measurements indicate that there are still areas for improvement towards a more complete model that also includes auditory distortions based on effects other than channel level differences. Future work also includes investigating how the model can describe temporal instabilities / modulations in the stereo image as reported in

[12] , as opposed to static distortions. Using directional loudness for audio coding and objective quality measurement. For further explanation, please refer to the chapter "Objective Assessment of Spatial Audio Quality Using Directional Loudness Maps". Description (e.g., description of Figure 9)

[0304] For example, features extracted from stereo / binaural audio signals in spatial (stereo) auditory scenes are presented. The features are based on a simplified model of the stereo mix that extracts, for example, the panning direction of events in the stereo image. The associated loudness in the stereo image per panning direction in the Short-Time Fourier Transform (STFT) domain can be calculated. The features are optionally calculated for the reference signal and the encoded signal and then compared to derive a distortion measure aimed at describing the perceived degradation score reported in the listening test. The results show improved robustness in the face of low-bitrate, non-waveform-preserving parametric technical tools such as joint stereo and bandwidth extension when compared to existing methods. It can be integrated into standardized objective quality assessment measurement systems such as PEAQ or POLQA (PEAQ = Objective Measure of Perceived Audio Quality; POLQA = Perceptual Objective Listening Quality Analysis). term: Signal: A spatial signal representing, for example, an object, a downmix, a residual, etc.

[0305] Directional Loudness Map (DirLoudMap): derived from each signal, e.g. representing the loudness in the T / F (time / frequency) domains associated with each panning direction of the auditory scene. This can be derived from 3 or more signals by using binaural rendering (HRTF (head-related transfer function) / BRIR (binaural room impulse response)). Use (embodiment): 1. Automatic assessment of quality (embodiment 1): Explained in the chapter "Objective Assessment of Spatial Audio Quality Using Directional Loudness Maps"

[0306] 2. Directional loudness-based bit distribution in an audio encoder based on the ratio (contribution) of individual signal DirLoudMaps to the overall DirLoudMap (embodiment 2). Optional variant 1 (independent stereo pair): audio signals as speakers or objects.

[0307] Optional Variant 2 (Downmix / Residual Pair): Contribution of the downmix signal DirLoudMap and the residual DirLoudMap to the overall DirLoudMap. "Amount of contribution" in the auditory scene with respect to bit distribution criteria.

[0308] 1. An audio encoder that performs joint coding of two or more channels, resulting in, for example, one or more downmix and residual signals each, and the contribution of each residual signal to the overall directional loudness map is determined, for example, from a fixed decoding rule (e.g., MS-Stereo) or by estimating the inverse joint coding process from joint coding parameters (e.g., rotation in MCT). Based on the contribution of the residual signal to the overall DirLoudMap, the bitrate distribution between the downmix and the residual signal is adapted, for example, by controlling the quantization accuracy of the signals, or by directly discarding residual signals whose contribution is below a threshold. Possible criteria for "contribution" are, for example, the average ratio or the ratio of directional maximum relative contributions. Problem: Combining and contribution estimation of individual DirLoudMaps to the resulting / total loudness map. 3. (Embodiment 3) On the decoder side, directional volume can help the decoder make informed decisions regarding:

[0309] Complexity Scaling / Format Converter: Each audio signal is included or excluded from the decoding process based on its contribution to the overall DirLoudMap (either transmitted as a separate parameter or inferred from other parameters), thus allowing varying rendering complexity for different application / format conversions. This allows for reduced complexity decoding when only limited resources are available (i.e. multi-channel signals rendered on mobile devices).

[0310] Since the resulting DirLoudMap may depend on the target playback settings, this ensures that the most important / salient signals of each individual scenario are played back, and therefore this has an advantage over spatially uninformed approaches such as simple signal / object priority levels. 4. Joint coding decision (embodiment 4) (e.g., explanation of FIG. 14) Determine the contribution of the directional loudness map of each signal or each candidate signal pair to the contribution of the DirLoudMap of the whole scene. 1. Optional Variation 1) Selecting the signal pair with the highest contribution to the overall loudness map

[0311] 2. Optional Variation 2) Select signal pairs whose signals have high proximity / similarity in their respective DirLoudMaps => can be represented together by downmixing

[0312] There may be cascaded joint coding of signals, so that e.g. the DirLoudMap of a downmix signal does not necessarily correspond to a point source from one direction (e.g. one loudspeaker) and thus the contribution to the DirLoudMap is estimated from e.g. the joint coding parameters. A scene-wide DirLoudMap can be computed by some kind of downmix or binauralization that takes into account the signal direction. 5. Parametric Audio Codec Based on Directional Volume (Embodiment 5) For example, a directional volume map of the scene is transmitted as side information in a parametric format, e.g. 1. "PCM Style" = Quantization values ​​across directions 2. Center position + left and right linear tilt 3. Polynomial or Spline Representation For example, send one signal / fewer signals / efficient transmission, 1. Optional Variation 1) Send a parameterized target DirLoudMap for the Scene+1 downmix channel 2. Optional Variation 2) Transmitting Multiple Signals, Each with an Associated DirLoudMap

[0313] 3. Optional Variation 3) Transmitting an Overall Target DirLoudMap and Multiple Signals with Parameterized Relative Contributions to the Overall DirLoudMap For example, synthesizing a complete audio scene from the transmitted signal based on a directional loudness map of the scene. Directional volume for audio coding Introduction and Definitions DirLoudMap = Directional Loudness Map An embodiment for computing DirLoudMap: a) perform t / f decomposition (+grouping into limit bands (CB)) (e.g. by filter bank, STFT, ...) b) Perform directional analysis function for each t / f tile c) Optionally feed / accumulate the results of b) into a DirLoudMap histogram (if required by the application): d) Summarizing the output through the CB to provide a wideband DirLoudMap DirLoudMap / Directional Analysis Function Level Implementation:

[0314] Level 1 (optional): Map the contributing directions according to the spatial playback position of the signal (channel / object) - (without knowledge of the utilized signal content). Use a directional analysis function that only considers the playback direction of the channel / object + / - diffusion window L1 playback direction of the channel / object + / - diffusion window (this can be wideband, i.e. the same for all frequencies).

[0315] Level 2 (optional): In addition to the spatial playback position of the signals (channels / objects), map the contribution directions according to a dynamic function (directional analysis function) of the content of the channel / object signals at different refinement levels. Identifiable

[0316] Optionally, L2a) panned phantom source (->panning index) [level], or optionally L2b) level + time delayed panned phantom source [level and time], or optionally L2c) extended (uncorrelated) panned phantom source (even more advanced) Applications for perceptual audio coding Embodiment A) Masking of each channel / object - without joint coding tool -> target:

[0317] Control of coder quantization noise (so that the original and encoded / decoded DirLoudMap deviate by less than a certain threshold, i.e., a target criterion for the DirLoudMap domain) Embodiment B) Masking of each channel / object - Joint coding tools (e.g., M / S+prediction, MCT)

[0318] -> Target: Control the coder quantization noise in the tool processed signal (e.g., M or rotated "sum" signal) to meet a target criterion in the DirLoudMap domain Example of B) 1) For example, calculate the overall DirLoudMap from all signals 2) Apply the joint coding tool

[0319] 3) Determine the contribution of the tool processed signals (e.g., "sum" and "residue") to DirLoudMap, taking into account the decoding function (e.g., rotation / predictive panning) 4) Control quantization with a) Consider the effect of quantization noise on DirLoudMap b) Consider the effect of quantizing the signal part from 0 to DirLoudMap Embodiment C) Controlling the application (e.g., MS on / off) and / or parameters (e.g., prediction coefficients) of a joint coding tool Target: Control the encoder / decoder parameters of the joint coding tool to meet the target criteria of the DirLoudMap domain Example C Control M / S on / off decision based on DirLoudMap Controls the smoothing of frequency-dependent prediction coefficients based on the effect of parameter changes on DirLoudMap (On cheaper differential encoding of parameters) (=Control trade-off between side information and prediction accuracy) Embodiment D) *Determine parameters (on / off, ILD, ...) of parametric joint coding tools (e.g. intensity stereo) -> Target: Control the parameters of the Parametric Joint Coding Tool to meet the target criteria of the DirLoudMap domain

[0320] Embodiment E) A parametric encoder-decoder system that transmits DirLoudMap as side information (rather than traditional spatial cues, e.g., ILD, ITD / IPD, ICC, ...)

[0321] -> The encoder determines the parameters based on the analysis of the DirLoudMap and the downmix signal(s) and (bitstream) parameters, e.g. the overall DirLoudMap + each signal's contribution to the DirLoudMap -> The decoder synthesizes the sent DirLoudMap in an appropriate way. Embodiment F) Reducing the Complexity of Decoders / Renderers / Format Converters

[0322] Determine each signal's "importance" by determining its contribution to the overall DirLoudMap (possibly based on transmitted side information). In applications with limited computational power, skip decoding / rendering signals that contribute to the DirLoudMap below a threshold. General steps for computing a directional loudness map (DirLoudMap) This is valid, for example, for any embodiment: (e.g., the illustrations of Figs. 3a and / or 4a) a) Perform t / f decomposition of some input audio signal. Optional: Grouping of spectral components into processing bands related to the frequency resolution of the human auditory system (HAS). -Optional: weighting according to HAS sensitivity in different frequency ranges (e.g. outer / middle ear transfer function) -> Result: t / f tiles (e.g. spectral domain representation, spectral bands, spectral bins, ...) For some (e.g. each) frequency band (loop):

[0323] b) For example, calculate a directional analysis function for t / f tiles of some audio input channels -> result: direction d (e.g. direction TIFF0007631273000187.tif38 or panning direction TIFF0007631273000188.tif34). c) For example, calculate the volume over t / f tiles of several audio input channels ->Result: Volume L

[0324] -The calculation of loudness can be simply energy, or more sophisticated energy (or Zwicker model: alpha = 0.25-0.27). da) For example, input / accumulate l contribution to DirLoudMap under direction d - Optional: spread of l distributions between adjacent directions (panning index: windowing) Conclusion Optionally (if required by the application): Compute a wideband DirLoudMap

[0325] db) DirLoudMap is summarized across several (avoid:all) frequency bands to provide a wideband DirLoudMap, showing sound "activity" as a function of direction / space. Example: Restoring directional signals using window / selection functions derived from panning indices (e.g., illustration of Figure 6)

[0326] The left (see FIG. 6a, red) and right (see FIG. 6b, blue) channel signals are shown, for example, in FIG. 6a and FIG. 6b. The bars can be DFT bins (Discrete Fourier Transform) of the whole spectrum, critical bands (frequency bin groups), or DFT bins within critical bands, etc. The criterion function is It is arbitrarily defined, such as TIFF0007631273000189.tif318. The criterion is for example "panning direction according to level", e.g. the level of each or some FFT bins.

[0327] a) From the basis function, a windowing / weighting function can be extracted that selects the appropriate frequency bins / spectral groups / components and restores the directional signal. Thus, the input spectrum (e.g., L and R) is filtered using different window functions TIFF0007631273000190.tif22 (each panning direction TIFF0007631273000191.tif33) is multiplied by a window function. b) From the criterion function, TIFF0007631273000192.tif22 has different orientations associated with different values ​​of L (i.e. the level ratio between L and R). To recover the signal using method a)

[0328] Example 1) Center of panning direction TIFF0007631273000193.tif37, (Relationship Keep only the bars with TIFF0007631273000194.tif312, which is the directional signal (see Figure 6a1 and Figure 6b1).

[0329] Example 2) Panning direction slightly to the left TIFF0007631273000195.tif310(Related (Keep only the bars with TIFF0007631273000196.tif315). This is the directional signal (see Figure 6a2 and Figure 6b2).

[0330] Example 3) Panning direction slightly to the right TIFF0007631273000197.tif310(Related (Only keep the bars with TIFF0007631273000198.tif315). This is the directional signal (see Figure 6a3.1 and Figure 6b3.1).

[0331] The criterion functions are the level of each DFT bin, the energy per DFT bin group (critical band) TIFF0007631273000199.tif513, or Critical Band It can be arbitrarily defined as the volume per TIFF0007631273000200.tif515. Different applications may have different criteria. Weighting (optional) Note: Not to be confused with outer / middle ear (peripheral model) transfer function weighting, which for example weights critical bands.

[0332] Weighting: Depending on the situation, Instead of getting the exact value for TIFF0007631273000201.tif33, use a tolerance, Use a weight that gives less importance to values ​​that deviate from TIFF0007631273000202.tif33, i.e. "Take all the bars that follow the 4 / 3 relationship, pass them with a weight of 1, and take the values ​​that are close to it and weight them less than 1 → for this you can use a Gaussian function. In the above example, the directional signal has more bins that are not weighted with 1 but have lower values."

[0333] Motivation: Weighting allows for a "smoother" transition between different directional signals, with some "leakage" between the different directional signals so that the separation is not as abrupt. For example 3), it looks like the one shown in Figure 6a3.2 and Figure 6b3.2. Various embodiments of computing the volume map using a generalized criterion function Option 1: Panning index approach (see Figure 3a and Figure 3b):

[0334] (All) Different For TIFF0007631273000203.tif33, we can construct a "value" map of this function in time. A so-called "directional volume map" can be constructed by either:

[0335] Example 1) Criterion function for "panning direction according to the level of each FFT bin" Using TIFF0007631273000204.tif510, the directional signals are composed of, for example, individual DFT bins. Then, for example, we calculate the energy of each critical band (DFT bin group) of each directional signal, and then raise these energies per critical band to an exponent, for example 0.25. → Similar to the chapter "Objective assessment of spatial audio quality using directional loudness maps" Example 2) Instead of windowing the amplitude spectrum, we can window the volume spectrum. The directional signal is already in the volume domain.

[0336] Example 3) Criterion function for "panning direction according to the volume of each critical band" Use TIFF0007631273000205.tif59 directly. Then, the directional signal is It consists of chunks of the entire band of interest according to the values ​​given by TIFF0007631273000206.tif33. for example, For TIFF0007631273000207.tif310, the directional signal can be: ·Y=1*critical_band_1+0.2*critical_band_2+0.001*critical_band_3

[0337] Different combinations of other panning directions / directional signals may be applied. Note that if weighting is used, different panning directions may contain the same important bands, but most likely with different weight values. If no weighting is applied, the directional signals are mutually exclusive. Option 2: Histogram approach (see Figure 4b):

[0338] This is a more general description of overall directional loudness. It does not necessarily utilize a panning index (i.e., one does not need to recover a "directional signal" by windowing the spectrum to calculate loudness). The overall loudness of a frequency spectrum is "distributed" according to the "analyzed directions" of the corresponding frequency regions. The directional analysis can be level difference based, time difference based, or other forms. For each time frame (see Figure 5):

[0339] histogram The resolution of TIFF0007631273000208.tif34 is, for example, It is given by the amount of values ​​given to the set of TIFF0007631273000209.tif33. This can be done, for example, in a time frame When evaluating TIFF0007631273000210.tif22 The amount of bins available to group occurrences of TIFF0007631273000211.tif33 into. The value can be used, for example, to Accumulated and smoothed over time using TIFF0007631273000212.tif22. TIFF0007631273000213.tif336 where n is the time frame index.

Claims

1. A format converter (500) for converting (510) a format of audio content (520) representing an audio scene from a first format to a second format, comprising: the format converter (500) is configured to provide a representation (530) of the audio content in the second format based on a representation of the audio content in the first format; The format converter (500) converts two or more input audio signals (112, 112) in the first format into an overall directional loudness map of the audio scene. １ , 112 ２ , 112 ３ , 112a, 112b) (a) adjusting (540) the complexity of the format conversion depending on the contribution of the The format converter (500) is configured to calculate or estimate a contribution of a given input audio signal to the overall directional loudness map of the audio scene; the format converter (500) is configured to decide whether to take the given input audio signal into account in the format conversion depending on the calculation or estimation of the contribution, thereby adapting the complexity, where taking the given input audio signal into account means not skipping the given input audio signal in the format conversion, and not taking the given input audio signal into account means skipping the given input audio signal in the format conversion; the global directional loudness map indicating loudness information associated with different directions of the audio scene; the contribution of the input audio signals in the first format to the overall directional loudness map of the audio scene is a contribution of an individual directional loudness map of the input audio signals in the first format to the overall directional loudness map or a contribution of a directional loudness map of a signal combination of the input audio signals in the first format to the overall directional loudness map, A format converter (500) wherein the individual directional loudness maps of the input audio signals indicate loudness information associated with different directions of the respective input audio signals, and the directional loudness map of the signal combinations of the input audio signals indicates loudness information associated with different directions of the respective signal combinations of the input audio signals.

2. The format converter (500) of claim 1 , wherein the global directional loudness map is determined based on the input audio signal.

3. The format converter (500) of claim 1 , wherein the loudness information indicated by the global directional loudness map is associated with a combination of the input audio signals.

4. The format converter (500) according to claim 1 , wherein the format converter (500) is configured to receive directional volume map information and to obtain the overall directional volume map and / or one or more directional volume maps based thereon.

5. The format converter (500) of claim 4, wherein the format converter (500) is configured to derive the overall directional loudness map from the one or more directional loudness maps.

6. An audio decoder (400) for decoding (410) encoded audio content (420), comprising: The audio decoder (400) is configured to receive an encoded representation (420) of one or more audio signals and to provide a decoded representation (432) of the one or more audio signals; the audio decoder (400) is configured to reconstruct (430) an audio scene using the decoded representation (432) of the one or more audio signals; the audio decoder (400) is configured to adjust (440) a decoding complexity depending on the contributions of two or more encoded signals to an overall directional loudness map of a decoded audio scene, the audio decoder (400) is configured to calculate or estimate a contribution of a given encoded signal to the overall directional loudness map of the decoded audio scene, the audio decoder (400) is configured to decide whether to decode the given encoded signal depending on the calculation or estimation of the contribution, thereby adapting the decoding complexity; the global directional loudness map indicates loudness information associated with different directions of the decoded audio scene; the contributions of the encoded signals to the overall directional loudness map of the decoded audio scene are contributions of individual directional loudness maps of the encoded signals to the overall directional loudness map or contributions of directional loudness maps of signal combinations of the encoded signals to the overall directional loudness map, An audio decoder (400), wherein the individual directional loudness maps of the encoded signals indicate loudness information associated with different directions of the respective encoded signals, and the directional loudness maps of signal combinations of the encoded signals indicate loudness information associated with different directions of each combination of signals of the encoded signals.

7. 7. The audio decoder (400) of claim 6, configured to receive encoded directional volume map information (424) and to decode the encoded directional volume map information (424) to obtain the overall directional volume map and / or one or more directional volume maps.

8. 8. The audio decoder (400) of claim 7, wherein the audio decoder (400) is configured to derive the overall directional loudness map from the one or more directional loudness maps.

9. A renderer (600) for rendering audio content, comprising: The renderer (600) receives two or more input audio signals (112, 112 １ , 112 ２ , 112 ３ , 112a, 112b), The renderer (600) converts the two or more input audio signals (112, 112A, 112B) into an overall directional volume map (142) of a rendered audio scene (642). １ , 112 ２ , 112 ３ , 112a, 112b) (i.e., 112b), The renderer (600) is configured to calculate or estimate a contribution of a given input audio signal to the overall directional loudness map of the audio scene, the renderer (600) is configured to decide whether to take into account the given input audio signal in the rendering depending on the calculation or estimation of the contribution, thereby adapting a complexity of the rendering, where taking into account the given input audio signal means not skipping the given input audio signal in the rendering, and where not taking into account the given input audio signal means skipping the given input audio signal in the rendering; the global directional loudness map indicating loudness information associated with different directions of the audio scene; the contributions of the input audio signals to the overall directional loudness map of the rendered audio scene are contributions of individual directional loudness maps of the input audio signals to the overall directional loudness map or contributions of directional loudness maps of signal combinations of the input audio signals to the overall directional loudness map, A renderer (600), wherein the individual directional volume maps of the input audio signals indicate volume information associated with different directions of the respective input audio signals, and the directional volume map of a signal combination of the input audio signals indicates volume information associated with different directions of the respective signal combination of the input audio signals.

10. The renderer (600) of claim 9, configured to obtain directional volume map information (142) and, based thereon, obtain the overall directional volume map and / or one or more directional volume maps.

11. The renderer (600) of claim 10, wherein the renderer (600) is configured to derive the overall directional volume map from the one or more directional volume maps.

12. A method (7000) for converting (7100) a format of an audio content representing an audio scene from a first format to a second format, comprising: The method includes providing a representation of the audio content in the second format based on a representation of the audio content in the first format; The method comprises adjusting (7200) a complexity of the format conversion depending on a contribution of two or more input audio signals in the first format to an overall directional loudness map of the audio scene, The method comprises calculating or estimating a contribution of a given input audio signal to the global directional loudness map of the audio scene; the method comprises deciding whether to take the given input audio signal into account in the format conversion depending on the calculation or estimation of the contribution and thereby adapting the complexity, where taking the given input audio signal into account means not skipping the given input audio signal in the format conversion and not taking the given input audio signal into account means skipping the given input audio signal in the format conversion; the global directional loudness map indicating loudness information associated with different directions of the audio scene; the contribution of the input audio signals in the first format to the overall directional loudness map of the audio scene is a contribution of an individual directional loudness map of the input audio signals in the first format to the overall directional loudness map or a contribution of a directional loudness map of a signal combination of the input audio signals in the first format to the overall directional loudness map, A method (7000) wherein the individual directional volume maps of the input audio signals indicate volume information associated with different directions of the respective input audio signals, and the directional volume map of a signal combination of the input audio signals indicates volume information associated with different directions of the respective signal combination of the input audio signals.

13. A method (8000) for decoding encoded audio content, comprising: The method includes receiving (8100) an encoded representation of one or more audio signals and providing (8200) a decoded representation of the one or more audio signals, The method includes reconstructing (8300) an audio scene using the decoded representation of the one or more audio signals; The method comprises adjusting (8400) a decoding complexity depending on the contributions of two or more encoded signals to an overall directional loudness map of a decoded audio scene, The method comprises calculating or estimating a contribution of a given encoded signal to the overall directional loudness map of the audio scene, The method comprises deciding whether to decode the given encoded signal in the format conversion depending on the calculation or estimation of the contribution, thereby adapting the decoding complexity; the global directional loudness map indicates loudness information associated with different directions of the decoded audio scene; the contributions of the encoded signals to the overall directional loudness map of the decoded audio scene are contributions of individual directional loudness maps of the encoded signals to the overall directional loudness map or contributions of directional loudness maps of signal combinations of the encoded signals to the overall directional loudness map, A method (8000) wherein the individual directional loudness maps of the encoded signals indicate loudness information associated with different directions of each encoded signal, and the directional loudness maps of signal combinations of the encoded signals indicate loudness information associated with different directions of each combination of signals of the encoded signals.

14. A method (9000) for rendering audio content, comprising: The method comprises a step (9100) of reconstructing an audio scene based on two or more input audio signals, The method comprises a step (9200) of adjusting a rendering complexity depending on the contribution of the two or more input audio signals to an overall directional loudness map of a rendered audio scene, The method comprises calculating or estimating a contribution of a given input audio signal to the global directional loudness map of the audio scene; the method comprises deciding whether to take the given input audio signal into account in the rendering depending on the calculation or estimation of the contribution, thereby adapting a complexity of the rendering, where taking the given input audio signal into account means not skipping the given input audio signal in the rendering, and where not taking the given input audio signal into account means skipping the given input audio signal in the rendering, the global directional loudness map indicating loudness information associated with different directions of the audio scene; the contributions of the input audio signals to the overall directional loudness map of the rendered audio scene are contributions of individual directional loudness maps of the input audio signals to the overall directional loudness map or contributions of directional loudness maps of signal combinations of the input audio signals to the overall directional loudness map, A method (9000), wherein the individual directional loudness maps of the input audio signals indicate loudness information associated with different directions of the respective input audio signals, and the directional loudness map of a signal combination of the input audio signals indicates loudness information associated with different directions of the respective signal combination of the input audio signals.

15. A computer program having a program code for performing the method according to claims 12 to 14, when the computer program runs on a computer.

Images