Loudness control for user interaction in audio encoding system

The audio processor addresses inconsistent loudness in immersive audio systems by modifying audio signals based on user input and metadata, ensuring consistent loudness normalization during user interactions.

JP2025000845AActive Publication Date: 2025-01-07FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2024170184
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2015-06-17
Filing Date
2024-09-30
Publication Date
2025-01-07
Estimated Expiration
2036-06-09

AI Technical Summary

Technical Problem

Existing audio encoding systems lack effective mechanisms for loudness compensation that adapt to user interactions, particularly in immersive audio formats like MPEG-H 3D audio, leading to inconsistent loudness levels during user-driven adjustments.

Method used

An audio processor that includes an audio signal modification unit and a loudness compensation unit, which modifies audio signals based on user input and metadata to balance loudness levels by determining a loudness compensation gain based on reference and modified gains, considering user preferences and audio signal metadata.

Benefits of technology

Ensures consistent loudness normalization across user interactions, maintaining desired loudness levels even when users adjust audio elements or groups, thereby enhancing the user experience in immersive audio environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2025000845000001_ABST
    Figure 2025000845000001_ABST
Patent Text Reader

Abstract

To provide an audio processor and an audio encoder for improving feasibility of loudness compensation.SOLUTION: An audio processor (1) for processing an audio signal (100) includes: an audio signal correction part (2) for correcting an audio signal (100) in accordance with user input (200); a loudness control part (6) for determining a loudness compensation gain (C) based on reference loudness (Lref) or a reference gain (gi), and corrected loudness (Lmod) or corrected gain (hi); and a loudness operation part (5) for operating loudness of the signal using the loudness compensation gain (C). The corrected loudness (Lmod) or the corrected gain (hi) depends on user input.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an audio processor and an audio encoder. The present invention also relates to a corresponding method. [Background technology]

[0002] Modern audio coding systems do not only provide the means to efficiently transmit audio content in a loudspeaker channel-based representation for simple playback at the decoder side, but also contain additional advanced features that allow the user to interact with the content and thus influence how the audio is played and rendered at the decoder. This enables new forms of user experience compared to legacy audio coding systems.

[0003] An example for an advanced audio coding system is the MPEG-H 3D Audio standard (Non-Patent Document 1), which allows the transmission of immersive audio content in three different formats: channel-based, object-based, and scene-based using higher order ambisonics (HOA). It has been designed to offer new possibilities such as personalized user interaction and audio adaptation to different usage scenarios.

[0004] The three different categories of content formats can be described as follows: -Channel-based: Traditionally, spatial audio content (starting with simple two-channel stereo) has been delivered as a set of precisely defined channel signals directed to be reproduced by loudspeakers at fixed target positions relative to the listener. - Object-based: an audio object is a signal that should be played back as coming from a specific target location, specified by associated side information provided as metadata with the audio. In contrast to channel signals, the actual placement of audio objects can change over time and does not need to be predefined during the sound production process, but is determined by rendering to a target loudspeaker setup at the time of playback. The actual placement of audio objects may include user interactivity regarding the location or level of an object or group of objects. - Higher Order Ambisonics (HOA) is an alternative approach to capture the 3D sound field by transmitting several "coefficient signals" that have no direct relation to the channels or objects. The actual object signals for reproduction are generated at the decoder, taking into account a given loudspeaker configuration.

[0005] A method for loudness compensation in an object-based audio coding system including user interaction is disclosed in US Pat. No. 6,399,436. A decoder receives an audio input signal including audio object signals and generates an audio output signal. A signal processor determines loudness compensation values ​​for the audio output signal based on loudness information associated with the audio input signal and rendering information. The rendering information indicates whether one or more audio object signals should be amplified or attenuated and may be adjusted according to user preferences. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] European Patent Application Publication No. 2879131 [Non-patent literature]

[0007] [Non-Patent Document 1] J. Herre at al., “MPEG-H Audio - The New Standard for Universal Spatial / 3D Audio Coding”, 137th AES Convention, 2014, Los Angeles Summary of the Invention [Problem to be solved by the invention]

[0008] An object of the present invention is to improve the feasibility of loudness compensation. [Means for solving the problem]

[0009] The object is achieved by an audio processor for processing an audio signal, comprising the following elements: an audio signal modification unit configured to modify the audio signal in response to a user input, a loudness control unit configured to determine a loudness compensation gain based on a reference loudness or reference gain on the one hand and based on a modified loudness or modified gain on the other hand, where the modified loudness or modified gain is dependent on the user input, the loudness control unit being configured to determine the loudness compensation gain based on metadata of the audio signal indicating which groups should or should not be used for determining the loudness compensation gain, where the groups comprise one or more audio elements, and a loudness manipulation unit configured to manipulate the loudness of a signal using the loudness compensation gain.

[0010] An audio processor - or decoder or device for processing an audio signal - receives an audio signal and, in one embodiment, generates an output signal, which includes audio objects, audio elements etc. of the audio signal to be played, for example by loudspeakers or earphones, or to be stored on a medium etc.

[0011] The audio processor reacts to the user input via an audio signal modification unit configured to modify the audio signal in response to the user input. The user input, in one embodiment, indicates amplification or attenuation of a group and / or switching off a group or switching on a group. The group comprises one or more audio elements, e.g. audio objects, channels, objects or HOA components. Depending on the embodiment, the user input also refers to data related to a playback configuration to be used for the playback of the signal. A further user input also refers to a selection of a preset. A preset refers to a set of at least one group and - depending on the embodiment - specifies a group loudness value and / or a gain value measured specifically for each group. The user input is used by the audio modification unit to appropriately modify the audio signal. In an embodiment, the metadata comprises data attributable to a number of presets.

[0012] The preset, in one embodiment, refers to a set of one group, and in another embodiment, defines multiple groups that are not attributable to the preset.

[0013] The audio processor further comprises a loudness control unit configured to determine a loudness compensation gain, herein called C, which makes it possible to balance the effect of a user input in order to provide a signal with an overall loudness desired or set by the user. The loudness compensation gain is determined on the one hand based on a reference loudness or reference gain and on the other hand based on a modified loudness or modified gain. Thus, the loudness compensation gain is determined on the basis of the reference loudness or reference gain and the modified loudness or modified gain. The modified loudness or modified gain is dependent on the user input.

[0014] The loudness control unit is additionally configured to determine the loudness compensation gain based on metadata of the audio signal, the metadata associated with the audio signal conveying information about the audio signal and the individual groups and, in one embodiment, being included in the audio signal itself.

[0015] The metadata data (in the embodiments of the audio processor described herein) indicates whether a certain group (particularly contained in the audio signal) should be used (e.g., considered) or not (e.g., ignored) for determining the loudness compensation gain. Thus, information about the corresponding group is either considered or ignored for determining the loudness compensation gain. In at least one embodiment, whether a group or groups are considered or ignored is further dependent on user input.

[0016] In one embodiment, taking into account or ignoring a group includes partially taking into account or ignoring in the sense that the group and its individual values ​​are only used for determining part of the loudness compensation gain, e.g. only used for calculating the reference loudness or the modified loudness.

[0017] The loudness compensation gain is used by a loudness manipulation unit included in the audio processor, which manipulates the loudness of the signal using the loudness compensation gain, the loudness compensation gain being influenced by user input but also as a result of metadata data associated with or belonging to the audio signal.

[0018] The signal manipulated by the loudness manipulation unit is according to one embodiment an output signal provided by an audio processor and based on the audio signal, the loudness manipulation unit in this embodiment providing an output signal and manipulating the loudness of the output signal using a loudness compensation gain.

[0019] In a different embodiment, the loudness manipulation manipulates the loudness of a signal that is provided to the loudness manipulation and that is preferably already modified by user input, in which part of the audio processor provides or generates a signal that is fed to the loudness manipulation and processed accordingly, i.e. modified with respect to its loudness, by the loudness manipulation.

[0020] In a further embodiment, the signal whose loudness is manipulated by the loudness manipulation unit is an audio signal. In this case, the loudness manipulation unit modifies metadata of the audio signal by the modification. This embodiment relates to a further embodiment, in which an audio processor provides a modified audio signal. This modified audio signal has been modified according to user input and according to the loudness modification. This modified audio signal later also becomes a bitstream.

[0021] According to an embodiment of the audio processor, the loudness control unit is configured to determine the loudness compensation gain based on at least one flag included in the data of the metadata, which flag indicates whether or how a group should be considered for determining the loudness compensation gain. In this embodiment, the metadata includes a flag having either a value of "true" or "false" indicating, respectively, whether the associated group should be considered for calculating the loudness compensation gain. Considering a group also refers in an embodiment to the question for which step of the calculation the group should be used. This also refers, for example, to the calculation of the reference loudness and the modified loudness. The reference loudness and the modified loudness are the overall loudness calculated before and after the user input, respectively. In a different embodiment, the flag indicates that the corresponding group is only present for a short period of time and therefore can be ignored for determining the loudness compensation gain.

[0022] According to an embodiment of the audio processor, the loudness control unit is configured to use only some groups for determining the loudness compensation gains if these groups belong to anchors contained in the metadata of the audio signal, which anchors in an embodiment refer to audio elements belonging to, for example, voice, dialogue or special sound effects.

[0023] The handling of groups belonging to an anchor is further explained in the following embodiments.

[0024] In one embodiment, the loudness control unit is configured to use only groups belonging to an anchor for determining the loudness-compensation gain if the modified gain of at least one group belonging to said anchor is greater than the corresponding reference gain, so that if the gain value of at least one of these "anchor groups" is increased in response to a user input, i.e. if the user amplifies at least one of these groups, then only those groups of anchors are used for the calculation of the loudness-compensation gain.

[0025] In an alternative or additional embodiment, the loudness control unit is configured to use the groups belonging to the anchor and the groups not belonging to the anchor for determining the loudness compensation gain when the modified gain of at least one group belonging to the anchor is smaller than the corresponding reference gain. Thus, in this embodiment, not only the groups belonging to the anchor but also the groups not belonging to the anchor are used for the calculation when the gain value of at least one anchor group is decreased due to a user input.

[0026] In one embodiment, the previous two embodiments are combined: thus, modifying the gain of at least one group belonging to an anchor determines whether only the anchor group or the anchor group and non-anchor groups are used to determine the loudness compensation gain.

[0027] The object of the invention is achieved by an audio processor for processing an audio signal comprising the following elements: an audio signal modification unit configured to modify the audio signal in response to a user input, a loudness control unit configured to determine a loudness compensation gain based on a reference loudness or reference gain on the one hand and based on a modified loudness or modified gain on the other hand, said modified loudness or modified gain being dependent on said user input, said loudness control unit configured to determine said loudness compensation gain based on metadata of the audio signal indicating at least one preset, said preset referring to a set of at least one group comprising one or more audio elements, and a loudness manipulation unit configured to manipulate the loudness of a signal using said loudness compensation gain.

[0028] For a general description of the audio processor, please see the description above.

[0029] The loudness control of the audio processor refers to data of metadata associated with or belonging to the audio signal. The data indicates a preset, where a preset refers to a set of at least one group containing one or more audio elements. In this embodiment, the case is considered where a combination of groups is associated with a particular loudness and / or gain value for a particular preset. Hence, the metadata includes data about the groups depending on the different presets or at least a default preset. Thus, the loudness control uses data associated with the preset selected by the user or the preset that is the default preset.

[0030] The audio processor is in one embodiment configured according to at least one of the above-described embodiments, and thus the above-described embodiments are at least partially realized using the above-described audio processor.

[0031] According to an embodiment of the audio processor, the loudness control unit is configured to determine the loudness compensation gain based on a group loudness and / or gain value of at least one group of a set indicated by a preset, the preset referring to a specific set of groups of audio elements contained in the audio signal, for which the metadata contains specific data, i.e. group loudness and / or gain value, to be used for determining the loudness compensation gain when the corresponding preset is selected or set as a default preset.

[0032] In a further embodiment, the loudness control unit is configured to determine a reference loudness for the set indicated by the preset using the individual group loudnesses and the individual gain values. The loudness control unit is also configured to determine a modified loudness for the set indicated by the preset using the individual group loudnesses and the individual modified gain values, the modified gain values ​​being modified by user input. In this embodiment, the reference loudness and the modified loudness are determined based on values ​​for the groups associated with and belonging to the preset. The determination also carries an indication of whether and how the groups should be used - for example for determining the reference loudness or the modified loudness.

[0033] In a further embodiment, the loudness control section is configured to determine the loudness compensation gain based on data included in metadata of the audio signal indicating a selected preset, the preset being selected by user input, in this embodiment said preset being selected by a user via user input.

[0034] According to one embodiment of the audio processor, the loudness control unit is configured to determine the loudness compensation gains based on data included in the metadata of the audio signal indicating a default preset, said default preset being set prior to or separately from the user input. This embodiment handles the case where the user does not select a preset. To this end, the default preset is used, for example, prior to any user input, thereby ensuring that, even in the absence of e.g. user interaction, a certain set of data, here covering the default preset, is used to determine the loudness compensation gains.

[0035] The object of the invention is achieved by an audio processor for processing an audio signal comprising the following elements: an audio signal modification unit configured to modify an audio signal in response to a user input, a loudness control unit configured to determine a loudness compensation gain based on a reference loudness or reference gain on the one hand and based on a modified loudness or modified gain on the other hand, said modified loudness or modified gain being dependent on the user input, the loudness control unit being configured to determine the loudness compensation gain based on metadata of the audio signal indicating whether a group is switched off or switched on, said group comprising one or more audio elements, and a loudness manipulation unit configured to manipulate the loudness of a signal using the loudness compensation gain.

[0036] For a general description of the audio processor of this embodiment, please see the description above.

[0037] Here, the loudness control unit is configured to determine the loudness compensation gain based on metadata of the audio signal indicating whether a group is switched off or switched on. In one example, the audio signal may contain, as audio objects, different soundtracks belonging to different language versions of one movie. The presets may also indicate different language versions. Thus, for each different preset, one soundtrack of one language will be switched on while the other version will be switched off. This embodiment also shows that a user may switch between different language versions as desired and provided language version, and thus switch off the soundtrack associated with the default preset. However, switching on a group does not always mean switching off another group and vice versa.

[0038] The audio processor in one embodiment is configured according to at least one of the preceding embodiments.

[0039] Thus, the above-described embodiments may be at least partially realized using the audio processor described above, which may also be realized using the audio processor described below, in at least one embodiment contemplated by the embodiments described below.

[0040] According to one embodiment, the loudness control section determines a loudness compensation gain based on a user input depending on whether a group is switched off or switched on by the user input, where user interaction influences the loudness control section gain determination.

[0041] According to an embodiment of the audio processor, the loudness control is configured to exclude a group for determining the modified loudness if said group is switched off in response to a user input, in this embodiment, when a group is switched off by the user, said group is not used for determining the modified loudness resulting from the loudness value representative of the user preference.

[0042] In a further embodiment, the loudness control unit is configured to exclude a group for determining the reference loudness if the group is switched off in the metadata and to include the group for determining the modified loudness if the group is switched on by user input. In this embodiment, a group is switched off in the metadata and is not used for determining the reference loudness. If the user switches on the group, the group is included for the evaluation of the modified loudness.

[0043] According to an embodiment of the audio processor, the loudness control unit is configured to include a group for determining the reference loudness when said group is switched on in the metadata and to exclude said group for determining the modified loudness when said group is switched off by user input. In this embodiment, the reverse case of the previous embodiment is considered.

[0044] The object of the invention is also achieved by an audio processor for processing an audio signal comprising the following elements: an audio signal modification unit configured to modify the audio signal in response to a user input, a loudness control unit configured to determine a loudness compensation gain based on a reference loudness or reference gain on the one hand and based on a modified loudness or modified gain on the other hand, said modified loudness or said modified gain being dependent on the user input, said loudness control unit configured to determine said loudness compensation gain based on metadata of the audio signal missing at least one group loudness among a group of metadata contained in the audio signal, and a loudness manipulation unit configured to manipulate the loudness of a signal using said loudness compensation gain.

[0045] For a general description of the audio processor in this embodiment, please see the description above.

[0046] In this audio processor (or decoder), the loudness control section handles the situation where for a certain group present in the audio signal, there is no corresponding group loudness, which may be missing for a particular preset or playback configuration, etc., or the metadata may be completely empty of any group loudness for this group.

[0047] In one embodiment, the audio processor is configured according to at least one of the above-described embodiments. Thus, the above-described embodiment is at least partially realized using the audio processor described above. The above-described audio processor is likewise realized using the audio processor described below, in at least one embodiment taking into account the embodiments described below.

[0048] According to one embodiment of the audio processor, the loudness control unit is configured to calculate a missing group loudness using a preset loudness, a reference gain for a group having a missing group loudness, and a group loudness and a reference gain for a group having a group loudness, where the preset loudness is the overall loudness of the preset groups.

[0049] In a further embodiment, the loudness control unit is configured to determine a loudness compensation gain using only the at least one reference gain and the at least one modified gain for blind loudness compensation when the metadata of the audio signal is missing at least one group loudness. In this embodiment, the case where at least one group loudness is missing and the case where all group loudnesses are missing are treated in the same way.

[0050] According to one embodiment of the audio processor, the loudness control unit is configured to determine a loudness compensation gain using only at least one reference gain and at least one modified gain for blind loudness compensation when metadata of the audio signal is invalid for group loudness.

[0051] The object of the invention is also achieved by an audio processor for processing an audio signal comprising the following elements: an audio signal modification unit configured to modify the audio signal in response to a user input, a loudness control unit configured to determine a loudness compensation gain based on a reference loudness or reference gain on the one hand and based on a modified loudness or modified gain on the other hand, said modified loudness or modified gain being dependent on the user input, said loudness compensation gain being configured to determine said loudness compensation gain based on metadata of the audio signal referring to a playback configuration for the reproduction of the signal, and a loudness manipulation unit configured to manipulate the loudness of a signal using said loudness compensation gain.

[0052] For a general description of the audio processor in this embodiment, please see the description above.

[0053] The audio processor determines the loudness compensation gain based on data indicative of a particular playback configuration. Metadata associated with the audio signal, and in one embodiment included in the audio signal, thus includes data specific to at least one playback configuration. In one embodiment, for each playback configuration, the metadata includes data corresponding to the individual playback-or replay-configuration.

[0054] In one embodiment, the audio processor is configured according to at least one of the previous embodiments. Thus, in one embodiment, the audio processor is combined with at least one of the previous embodiments.

[0055] According to one embodiment of the audio processor, the loudness control unit is configured to determine the loudness compensation gains based on metadata data indicative of the playback configuration and including associated group loudness and / or reference gain values, such that different playback configurations are associated with different gain values ​​and / or group loudness for the individual groups.

[0056] In one embodiment, the metadata includes data about different presets and different playback configurations.

[0057] In a further embodiment, the audio processor comprises a configuration conversion unit for converting data contained in the metadata and indicative of a playback configuration into data indicative of a current playback configuration, and the loudness control unit is configured to determine the loudness compensation gains using the data provided by the configuration conversion unit. In this embodiment, the audio processor handles situations where a current playback configuration for the reproduction of the signal differs from the playback configuration provided by the metadata. Thus, the metadata data is converted to match the current playback configuration and the converted data is used for determining the loudness compensation gains.

[0058] In one embodiment, the audio processor includes a format converter for converting the signal into a predefined playback configuration, hi a further embodiment, the loudness control unit is configured to select a loudness value specific for the particular playback configuration used by the format converter.

[0059] The following embodiments may be realized by any of the above embodiments.

[0060] In one embodiment, the audio signal includes a bitstream having metadata, and the metadata includes a reference gain for at least one group.

[0061] According to an embodiment of the audio processor, the metadata of the audio signal comprises a group loudness for at least one group. In a further embodiment, the metadata comprises a group loudness for multiple groups belonging to the audio signal.

[0062] In a further embodiment, the loudness control unit is configured to determine a reference loudness for at least one group using a group loudness and a gain value for -at least one- group, and the loudness control unit is configured to determine a modified loudness using said group loudness and a modified gain value, said modified gain value being modified by user input.

[0063] In one embodiment, the loudness control unit uses the individual group loudnesses, referred to as Li, and gain values, referred to as gi, of the plurality of groups to calculate a reference loudness, referred to as L, for the plurality of groups. ref Further, the loudness control unit is configured to determine a modified loudness-L for the plurality of groups using the individual group loudnesses-Li- and the modified gain values-hi. mod In one embodiment, the two groups are the same, in another embodiment they are different. The groups also depend on the individual data of the metadata.

[0064] In a further embodiment, the loudness control unit is configured to perform a limiting operation on the loudness compensation gain such that the loudness compensation gain is below an upper threshold and / or such that the loudness compensation gain is above a lower threshold.

[0065] According to one embodiment of the audio processor, the loudness manipulation unit is configured to apply a corrected gain to the signal determined by a loudness compensation gain and a normalization gain, the normalization gain being determined by a target loudness level set by a user input and a metadata loudness level included in the metadata of the audio signal. In one embodiment, the normalization gain is determined using a ratio between the loudness levels of the individual groups of the audio signal and a loudness level set by the user to be experienced by the user for playback of the audio signal.

[0066] The above described embodiment of the audio processor allows loudness compensation according to user input. The loudness compensation is improved by taking into account data describing groups of audio signals and their associations or usage for loudness compensation.

[0067] The above embodiments refer to an audio processor or an audio decoder. In the following, an encoder is described, providing an audio signal associated with or including metadata to be used by the audio processor.

[0068] The object is achieved by an audio encoder for generating an audio signal including metadata, the audio encoder comprising a loudness determiner for determining a loudness value for at least one group including one or more audio elements, and a metadata writer for introducing said determined loudness value into said metadata as a group loudness.

[0069] According to an embodiment of the audio encoder, the loudness determiner is configured to determine different loudness values ​​and / or different gain values ​​for different playback configurations and the metadata writer is configured to introduce the determined different loudness values ​​and / or different gain values ​​into the metadata associated with the respective playback configurations, in this embodiment the metadata contains different data for groups related to the different playback configurations, thus improving the playback of each group of the audio signal.

[0070] In one embodiment, the loudness determiner is configured to determine different loudness values ​​and / or different gain values ​​for different presets that indicate a set of at least one group containing one or more audio elements. Furthermore, the metadata writer is configured to introduce the determined different loudness values ​​and / or different gain values ​​into the metadata associated with the respective presets. In this embodiment, the presets indicate a particular set of groups that are associated with a particular group loudness and / or reference gain value.

[0071] In a further embodiment, the audio encoder further comprises a controller configured to determine which groups should be used or ignored for determining the loudness compensation gain, and the metadata writer configured to write an indication into the metadata indicating which groups should be used or ignored for determining the loudness compensation gain, which indication is in one embodiment a flag, and in some embodiments the indication indicates a preset, a playback configuration, an anchor and / or a duration, thus indicating the relevance of the groups.

[0072] In at least one embodiment, the metadata includes different data (eg, group loudness or reference gain) having different values ​​for at least one group of audio signals.

[0073] According to one embodiment of the audio encoder, the audio encoder further comprises an estimator configured to calculate a group loudness value for a group, the group loudness value for which is not determined by the loudness determiner. The metadata writer is configured to introduce the calculated group loudness value into the metadata such that all groups of the audio signal have an associated group loudness. In this embodiment, the audio encoder compensates for the lost group loudness by calculating the group loudness based on valid data.

[0074] The object of the invention is also achieved by a method for processing an audio signal.

[0075] The method includes at least the following steps. modifying the audio signal in response to user input determining a loudness compensation gain based on a reference loudness (as the overall loudness of the relevant individual groups before being modified by the user) or a reference gain on the one hand, and on a modified loudness (as the combined loudness of the relevant groups after being modified by the user, paired with the reference loudness) or a modified gain on the other hand, said modified loudness or said modified gain being dependent on said user input. The determination of the loudness compensation gain - referred to as C - is performed using at least one or a combination of the following embodiments: the loudness compensation gain is determined based on metadata data associated with - or contained in - the audio signal. In different embodiments, each group contains one or more audio elements, the data being: The data indicates whether certain groups contained in the audio signal should be taken into account or ignored for determining this loudness compensation gain. The data refers to a preset, which in turn refers to a set of at least one group. -The data shows whether a group is switched off or switched on. In the data, at least one group loudness is missing for a group contained in the audio signal. The data refers to a playback configuration for regeneration of the signal. Manipulating the loudness of the output signal relative to the audio signal using a loudness compensation gain.

[0076] The object of the invention is also achieved by a method for generating an audio signal including metadata, comprising the steps of determining a loudness value for a group including one or more audio elements and introducing the determined loudness value for the group into the metadata as a group loudness.

[0077] The objects of the invention are also achieved by a computer program for carrying out the above-mentioned method when it runs on a computer or processor.

[0078] An apparatus embodiment (whether an audio processor or an audio encoder) may also be implemented by the steps of the method and corresponding embodiments of the method, so the description of the apparatus embodiment is also applicable to the method.

[0079] The present invention will now be described with reference to the accompanying drawings, in which: FIG. [Brief description of the drawings]

[0080] [Figure 1] FIG. 2 is a schematic diagram of an audio decoder; [Diagram 2] 1 is a schematic diagram of an audio processor according to the present invention; [Diagram 3] 1 is a schematic diagram of an inventive audio encoder; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0081] FIG. 1 shows a schematic diagram of an MPEG-H 3D audio decoder as an example of an audio processor, showing all the main building blocks of the system. As a first step, the received audio stream 500 (containing the transmitted audio signal, which may be a channel, object or HOA component with associated metadata) is decoded by a decoder 501, providing the audio content 502 and associated metadata 503. The channel signals are mapped to the target playback loudspeaker setup using a format converter 504, which acts as a channel renderer and format converter. · The object signals are rendered by the Object Renderer 505 using the associated object metadata into the target playback loudspeaker setup. Higher order Ambisonics content is rendered by the HOA Renderer 506 into the target playback loudspeaker setup using the associated HOA metadata. The loudspeaker signals corresponding to the different elements (channels, objects, HOA) in the form of an audio signal 507 as output of the format converter 504, the object renderer 505 and the HOA renderer 506 are then mixed together in a mixing stage. This is performed by a mixer 508, providing a mixed audio signal 509. The output 509 of the mixer 508 is then processed by a loudness control stage, where the audio signal is normalized to a desired target loudness level. The loudness control 510 performs loudness compensation together with the normalization. For this purpose, the loudness control 510 receives a user input 511, which as a result of a user interaction also refers to information about the loudness configuration to be used for playback, which is also provided to the format converter 504, the object renderer 505 and the HOA renderer 506. The loudness control 510 is provided with metadata 503, which in particular refers to the rendering and / or loudness information extracted by the decoder 501 from the received audio stream 500. The resulting signal 512 is provided to the loudspeakers of the loudspeaker configuration available for playback in the illustrated embodiment.

[0082] Possible user interactivity can be split, for example, into two different categories. -Selection of presets for transmitted audio programs Manipulating default rendering for groups of audio elements

[0083] The meaning of presets and groups in the context of MPEG-H 3D Audio and the present invention is explained below.

[0084] The individual channels, objects and HOA scenes available for a transmitted audio program are called audio elements. A group refers to a specific collection of individual audio elements. The specific grouping information of the audio elements is contained in the MPEG-H 3D Audio metadata, which is transmitted together with the audio content in the audio stream. The elements of a group cannot be modified interactively on their own. Only the whole group can be manipulated, i.e. all contained elements are manipulated together. An example is given by a group consisting of channels corresponding to a stereo or 5.1 loudspeaker configuration. In special cases, a group can consist of only a single element, e.g. a dialogue object of a program. The user can then change the level of this dialogue object within the audio scene, for example.

[0085] A preset defines a combination of groups in an audio scene. Presets can be used to efficiently signal different representations of the same audio program within the same audio stream. This preset definition also contains default or initial rendering information for each group, which is used if the user does not apply any actions. The most important example of this rendering information is the gain applied to a group when rendering the entire audio scene. The configuration information defining a preset is determined at the encoder and is part of the metadata, e.g. MPEG-H 3D audio metadata.

[0086] It should be noted that a primary or default audio scene can be thought of as a special form of preset that includes all audio elements without necessarily specifying grouping information, however default or initial rendering information (e.g. gains) for individual audio elements is typically also provided in the metadata for the primary audio scene.

[0087] One of the most important features for the next generation of audio distribution is advanced loudness control, i.e. proper signalization of loudness information and loudness normalization. Loudness control is particularly important in broadcast applications, where it represents an essential feature to satisfy applicable broadcasting regulations and recommendations.

[0088] The loudness control scheme included in MPEG-H 3D audio is based on metadata representing the measured loudness of the audio program. The metadata is transmitted together with the actual audio content in an audio stream as an example of an audio signal to be processed by an audio processor. In a decoder according to an embodiment, a loudness normalization gain is calculated based on the transmitted loudness information and a target loudness level. The loudness normalization gain of an embodiment is applied to the audio signal after the mixer 508, as shown as an example in FIG. 1.

[0089] To realize the unique feature of having multiple presets of the same audio program in the same audio stream, additional loudness metadata corresponding to the measured loudness of each preset is included in the audio stream. Processing steps such as format conversion (downmixing) or dynamic range processing can potentially change the loudness of the audio. Therefore, in one embodiment, additional loudness information is included in these cases to ensure accurate loudness normalization.

[0090] In other embodiments, loudness information is transmitted for individual groups or even for single audio elements. Group loudness information is provided in one embodiment for different loudspeaker configurations. For example, if a group consists of multiple channel signals, different group loudspeaker information may be included assuming playback to a stereo or 5.1 loudspeaker configuration. The group loudness information will be used for loudness control in the interactive scenarios proposed in this invention.

[0091] The loudness information above refers to a large variety of configurations for a program (e.g. different presets, or different loudspeaker playback layouts). Since these configurations are static, one embodiment envisages measuring their loudness at the encoder (or before the encoding process) and introducing corresponding metadata fields, for example, in the MPEG-H 3DA stream.

[0092] However, as mentioned above, an important feature of modern audio coding systems such as MPEG-H 3DA is the support of user interactivity at the decoder. The user can, for example, adjust the volume of specific groups or switch them on or off. An important use case is dialog enhancement, where the user can manipulate the level of a dialogue object or a group associated with that dialogue. In another embodiment, the user increases the level of an immersive acoustic bed represented by a HOA-based group. In another embodiment, the user may wish to switch to a particular group, representing, for example, a video description for a track for the hearing impaired or a voice-over track.

[0093] Changing the levels of a group means that the overall loudness of the rendered audio scene is changed compared to the unmodified case. Thus, a consistent playback loudness is no longer guaranteed after gain bidirectionality. As the user may change the levels of different objects more frequently, the loudness level of the audio output may change over time, even for the same program.

[0094] It is highly desirable to provide loudness control not only for the static representation of an audio program, but also to take into account user interactions that modify the loudness of an audio scene. The present invention allows for improved loudness control at the decoder to allow consistent loudness normalization even in the case of user interactions on the levels of groups of audio elements.

[0095] When the user changes the level of an audio element or group within the rendered audio scene, the program or preset loudness is preserved. In one embodiment, a loudness compensation gain is determined based on a reference loudness corresponding to the original audio scene and a modified loudness that takes into account the user's gain interaction. The loudness compensation gain is then applied to the rendered audio signal together with a standard loudness normalization gain to achieve the desired decoder target loudness.

[0096] 2 shows a schematic representation of an embodiment of an audio processor 1 - also called a decoder or simply a device for processing audio signals - which receives an audio signal 100 and provides an output signal 101. The output signal 101 in this embodiment is an audio signal suitable to be fed to an amplifier (not shown) connected to a loudspeaker in a playback situation or directly to a loudspeaker or headphones. The audio signal 100 comprises a bitstream with the audio signals of the individual audio objects and metadata providing information about the audio elements and how to handle them.

[0097] An audio signal 100 is provided to an audio signal modifier 2 which receives a user input 200. The user input 200 refers - in this embodiment - to the selection of at least a certain preset. A preset refers to a unique combination of a group of audio elements and an associated reference gain gi and / or group loudness Li of the corresponding group of audio elements. If the user does not select a preset, in this embodiment a default preset with default values ​​will be used.

[0098] Furthermore, the user sets the gain values ​​of the individual groups via the user input 200. The modified gain value hi means that the corresponding group will be amplified or attenuated corresponding to the reference gain value gi contained in the metadata. For example, the user may like to hear an amplified background choir and not (usually) want to hear the leading voice. Therefore, the user will increase the gain value of the background choir and decrease the gain value of the leading voice or switch this voice off.

[0099] The user also has the possibility to switch off or on a group. If the user does not want to hear a group, the group can be switched off. Conversely, if the metadata contains a flag which means that a group is switched off for a particular preset, the user can switch on the group. This may be the case, for example, if the audio signal contains different language versions of the dictated text and the presets point to different languages. Switching on or off a group therefore indicates whether the group is used in the playback or not.

[0100] In short, the signal modification unit 2 modifies the audio signal 100 according to the user input 200 by amplifying or attenuating groups of audio elements belonging to the audio signal 100, and according to a selected preset or a default preset covered by individual data of the metadata.

[0101] The signal modification unit 2 is followed by a configuration transformation unit 3 which transforms the data into the current playback configuration with which the audio signal 100 will be reproduced. Which playback configuration is given and therefore the current situation is also covered by user input 200, e.g. in the form of a selection from a list. For example, the metadata may refer to a surround sound situation, whereas the current playback state may allow stereo playback. This transformation in one embodiment refers to gain values ​​together with loudness values.

[0102] The composition transformation unit 3 provides the transformed data to a loudness control unit 6 which also receives user input 200. Based on these data the loudness control unit 6 calculates a loudness compensation gain C which is provided to the loudness manipulation unit 5.

[0103] The loudness manipulation unit 5 uses the loudness compensation gain C and the signal received from the mixer 4 to set the overall loudness of the output signal 101. The mixer 4 in this embodiment receives via the composition transformation unit 3 the audio signal 100 after modification by the audio signal modification unit 2 and transformation by the composition transformation unit 3 and combines the different groups of audio elements (compare Fig. 1).

[0104] For purposes of explanation, in the illustrated example, we consider the case where a unique audio scene is defined by a unique combination of presets, i.e. groups. Each group has an associated initial / default gain for a given preset. We further assume that the loudness of each group in its preset is available. The preset may be selected by the user or may be set as the default preset. The following notations are used: Li is the loudness of the i-th group of presets gi is the initial / default gain of the ith group (e.g. given in dB scale) hi is the corrected interactive (two-way) gain of the ith group (e.g. given in dB scale) M ref indicates the set of indices representing the groups included for the calculation of the reference loudness of a certain preset (or default audio scene). M mod indicates the set of indexes representing the groups included for the calculation of the modified loudness of a certain preset (or modified audio scene).

[0105] In case a group consists of a set of channel signals corresponding to a unique loudspeaker configuration, e.g., an HOA audio scene, multiple group loudness values ​​may be included in the metadata. These different loudness values ​​are associated with different loudspeaker configurations used for playback. For example, if a group represents a channel bed with a 5.1 or 22.2 loudspeaker configuration, a different loudness may be measured to play the group for the original 5.1 or 22.2 loudspeaker configuration compared to the case where the channel bed should be mapped to a stereo playback system using a format converter. In this case, in one embodiment, the group loudness associated with the stereo playback is selected if available in the transmitted metadata. Otherwise, the group loudness associated with the original loudspeaker configuration is used. A similar scheme is proposed to select the appropriate group loudness if a group represents an HOA-based audio scene. In this case, the group loudness associated with the current playback loudspeaker configuration should be used (if available in the metadata) instead of the group loudness associated with the reference loudspeaker layout.

[0106] In some embodiments, loudness information is not provided separately for each group, but rather the same loudness value is cited by a collection of groups.

[0107] In general, it is reasonable to assume that the audio signals of different groups are uncorrelated. In that case, the preset reference loudness can be calculated as follows:

number

[0108] Similarly, the loudness of the modified audio scene is calculated as follows:

number

[0109] If a group is switched off in the preset default settings, the group will have a reference loudness L ref Similarly, if the user switches off a group, it is excluded when calculating the corrected loudness L mod If a group is switched off in the default preset but is switched on by the user in the modified scene, the corresponding group loudness Li is calculated based on the reference loudness L ref is excluded from the calculation of the corrected loudness L mod Removing a switched-off group reduces its gain (gi or hi) by -∞ It should be noted that this is interpreted as being equivalent to setting M ref =M mod Therefore, the loudness of both L ref and L mod is calculated citing the same set of groups.

[0110] The loudness compensation gain C is the preset reference loudness L ref Preset corrected loudness L mod The metric is obtained by relating

number

[0111] The loudness compensation gain C is, in one embodiment, limited within a certain range of permitted gain to avoid undesirable behavior for extreme cases.

number

[0112] The loudness normalization gain G used for loudness normalization according to the prior art (see, for example, US Pat. No. 5,399,313) N is corrected according to the following formula:

number

[0113] The above explanation was based on a certain preset of an audio program. There may not always be presets available for a program, and only a single global default scene may be defined. In this case, it is treated the same as for the presets mentioned above, where the index M ref and M mod Each set refers to a group of the default scene and its modified version.

[0114] It may be appropriate to intentionally exclude a group from the loudness compensation process. For example, a group may be active only for a very short period in the program and completely silent for the rest of the time. Such a group may still have significant measurable loudness due to gating during the loudness measurement period, for example according to ITU-R BS.1770-3 - by the ITU Radiocommunication Sector (ITU-R) as one of the three sectors of the International Telecommunication Union (ITU). In that case, even though this group is only active for a very short period, this group loudness will affect the loudness compensation gain throughout the entire program duration. On the other hand, such sparse group signals have only a negligible contribution to the loudness measurement of the entire program / preset mix.

[0115] For example, if the user chooses to boost such a sparse group / object, the loudness compensation will attenuate all remaining object elements for the entire program duration. Such behavior is undesirable and the loudness compensation process should ignore such special sparse groups. Therefore, the metadata contains a corresponding flag for this group to be ignored for the loudness compensation calculation.

[0116] To provide the above mentioned functionality, information is added to the metadata contained in the audio stream or audio signal indicating whether a group should be excluded from loudness compensation, i.e. from the calculation of the reference loudness and modified loudness of a preset or global audio scene. This information is, in one embodiment, a simple flag for each group indicating whether the group is included in the loudness compensation process or not.

[0117] Different broadcasting regulations regarding loudness control use different approaches to define program loudness: EBU-R128 requires measurement of the loudness of the full program mix, while ATSC A / 85 recommends measurement of the loudness only of the anchor elements of a program, which are typically represented by dialogue.

[0118] Such different approaches to measuring loudness for a program are also considered for loudness compensation. Anchor-based loudness compensation follows immediately from full-mix loudness compensation as described above.

[0119] For the anchor-based reference and modified loudness of a preset (or program default mix), only groups that contribute to the program anchor are included. The information of which groups are part of the program anchor is included in one embodiment in the metadata of the audio stream / audio signal. The reference loudness is given by:

number

[0120] Similarly, a set A of group indexes (pointing to groups that are part of the anchor element of the modified audio scene or preset) mod The modified loudness for anchor-based loudness compensation using

number

[0121] From the above, the compensation gain is obtained as follows:

number

[0122] The remaining steps for performing loudness compensation are unchanged compared to the full-program mix case (as described above).

[0123] In some cases, a mix of both loudness compensation approaches - anchor-based and full-program mix-based - is beneficial for the user experience of loudness compensation.

[0124] In one embodiment, the anchor-based approach is used when one or all of the anchor groups are amplified by the user, i.e., when hi > gi. On the other hand, if the anchor group is attenuated, i.e., when hi < gi, then the loudness compensation for the full mix loudness is used. Information about the anchor group is included in the metadata.

[0125] The loudness compensation approach as described above requires information about the loudness of each group within a preset or global audio scene. In some scenarios, loudness information may be available for only some of the groups and may be missing for other groups. Thus, in one embodiment, the missing group loudness information is calculated from the loudness of the preset (or default audio scene) and the available group loudness values.

[0126] L p Assume that L represents the measured loudness of the preset being considered for the audio program, i.e., the measured joint loudness of the audio objects belonging to an individual preset. Further, assume that B represents the set of indices for the groups for which loudness information is available. The residual loudness L res of the preset is calculated from the preset loudness, the available group loudness information, and the default / initial gain of these groups.

Equation

[0127] An alternative representation of the residual loudness may be obtained by considering unavailable group loudness values ​​and corresponding default / initial gains.

number

[0128] In practice, it is reasonable to assume that the loudness of each group where loudness information is missing is equal.

number

[0129] In this case, the residual loudness can be expressed as:

number

[0130] From this, an estimate of the missing group loudness value can be readily obtained from:

number

[0131] Then, the reference loudness and modified loudness required for loudness compensation are calculated as already described above, and any missing group loudness Li is added to the corresponding estimate L A is replaced by.

[0132] The estimation of the missing group loudness information is performed either at the encoder side or at the decoder side of the audio coding system.

[0133] If said estimation is performed on the encoder side, information about group loudness in the metadata transmitted in the audio stream can be measured or alternatively a corresponding estimation as described above can be included, in which case the loudness compensation stage at the decoder side has all the necessary loudness information and can perform its processing according to the case where all group loudness has been pre-measured by the encoder.

[0134] If the estimation is performed in the decoder, the missing group loudness values ​​in the metadata of the audio stream are estimated as described above, and loudness compensation is then performed based on the estimated group loudness values.

[0135] There are special use cases where no information about the loudness of any group is provided in the metadata of the audio stream. In this case, loudness compensation has to work only based on the available relevant rendering information, i.e. the default or initial gain gi of a group and its modified version hi after user interaction. This operation is called blind loudness compensation, since the loudness information for the groups is not known on the decoder side. In other embodiments, blind loudness compensation is also performed if only one group loudness is missing in the metadata.

[0136] For the compensation, the assumption is used that the loudness values ​​of all groups in a preset are the same. In one embodiment of blind loudness compensation, M ref and M mod For all groups included in each A This gives the rule for calculating the loudness compensation gain according to the following formula:

number

[0137] It should be noted that the gain factors for blind loudness compensation only require information about the group gains, but no loudness related information.

[0138] In a further embodiment, blind loudness compensation is performed if at least one group loudness is missing, thus resulting in blind loudness compensation even if only one group loudness is missing.

[0139] This section summarizes what has been discussed above.

[0140] In one embodiment, a general set of indexes is specified that indicates which groups should be included for the calculation of the reference loudness of a preset or default audio scene. This set is derived from information in the metadata of the audio stream and indicates whether a group should be included to perform loudness compensation for a default audio scene or preset. This information is typically introduced in the metadata of the audio stream at the encoder.

[0141] In the encoder, the loudness compensation process is controlled by appropriately defining these bitstream elements. For example, when a group should be excluded, the corresponding bitstream element is set to "false". In one embodiment, anchor-based loudness compensation is achieved by including only groups that are part of the anchor elements of the default audio scene or default preset, and setting the corresponding bitstream element to "true". Other ways of providing this information can be used in different configurations.

[0142] As already explained in one embodiment, if a group is switched off in the default audio scene or preset, the group is switched off with the reference loudness L ref The resulting set of indices is K ref It is shown as:

[0143] Similarly, any groups switched off in the modified scene will have a different modified loudness L mod If a group is switched off in the default scene and switched on by the user in a modified scene, the corresponding group loudness is calculated by subtracting ref is excluded from the calculation of the corrected loudness L mod is included in the calculation of the corrected loudness L mod The set of group indices of K mod As shown in the figure.

[0144] Next, the loudness compensation gain is M ref K ref Replace with M mod K mod The calculation is the same as above, except that

[0145] If any of the group loudness information required to calculate either the reference or modified loudness is missing in the decoder, blind loudness compensation is used as a fallback mode. ref and K mod The same approach as above regarding selecting the ) applies in fallback mode.

[0146] 3 shows an embodiment of an audio encoder 20 which generates a digital audio signal 100 based on different audio sources. The audio signal 100 includes metadata which may be used, for example, by the audio processors mentioned above.

[0147] The audio encoder 20 includes a loudness determiner 21 for determining loudness values ​​for at least one group having one or more audio elements 50. In the illustrated example, there are three audio sources X1, X2 and X3, all of which are included in one group. The loudness values ​​of two of them, namely X2 and X3, are determined as L2 and L3 and fed to a metadata writer 22. The metadata writer 22 introduces the determined loudness values ​​for the two groups X2 and X3 as corresponding group reference loudness information L2 and L3 into the metadata of the audio signal 100.

[0148] Gain values ​​as reference gains g1, g2, g3 for groups X1, X2 and X3 are also written by the metadata writer 22 into the metadata of the audio signal 100. According to a further embodiment, the group loudness and reference gain values ​​are determined for a particular preset and / or different playback configurations. Also, the individual global loudness L p The loudness for different presets as is also measured.

[0149] The loudness of the first audio element 50, marked as X1, is not measured by the loudness determination unit 21 but is calculated or estimated by the estimation unit 24 (see above) and supplied as a corresponding reference loudness L1 to the metadata writing unit 22 and written into the metadata.

[0150] The controller 23 in the illustrated embodiment is connected to the loudness determiner 21 and to the metadata writer 22. The controller 23 decides which groups should be taken into account or ignored for determining the loudness compensation gain C. An indication is written into the metadata by the metadata writer 22 as data regarding the usage of the groups. Corresponding data, for example in the form of a flag, indicates which groups should be used or which groups should be ignored for determining the loudness compensation gain C by the audio program or the decoder.

[0151] The resulting audio signal 100 contains the real signals received from the audio objects 50 and metadata characterizing these real signals and their intended treatment by the audio decoder 1. The metadata data refers to groups of audio objects, while it is also possible that one group covers only one audio object / element.

[0152] The metadata includes at least some of the following data: Measured loudness values ​​for individual groups L i · Reference gain value g for individual groups i a reference gain value g, which represents the loudness or prominence of each group relative to the union of other related groups; i Reference loudness L as the resulting loudness of the combined group for a given preset and / or a given playback configuration ref An indicator of whether or how a group or its corresponding values ​​are used for the determination of the loudness compensation gain C (e.g. whether the group belongs to an anchor or whether the duration of the group is so short that it can be ignored) or

[0153] For each group, the metadata preferably contains different sets of data for different presets and / or different playback configurations, so that different recording and different playback situations may result in different data sets for the associated group.

[0154] The invention is described below through various embodiments that implement loudness compensation for user interaction using an audio coding system. At the encoder side, the loudness of each group contained in the default audio scene and / or preset is determined. The loudness information is introduced into the audio stream or into metadata included as part of the audio signal. Multiple loudness values ​​are included for at least one group, different values ​​being associated with different loudspeaker playback configurations (e.g. stereo, 5.1 or other). On the encoder side, additional metadata is generated corresponding to information whether a group should be included to perform loudness compensation, i.e. whether the group should be considered for the calculation of the reference loudness and the modified loudness, respectively. For example, anchor-based loudness compensation is realized by configuring the metadata to include only groups that are part of the anchor elements of the default audio scene or a given preset. A decoder receives an audio stream representing an audio signal and associated metadata. The decoder decodes the audio stream to generate a decoded audio signal corresponding to a channel and / or object and / or higher order Ambisonics format. Based on the metadata, the decoder selects all group indices that should be included for loudness compensation of a given audio scene or preset. The decoder uses the reference loudness L of the audio scene or preset. ref is the default gain g for each selected group. i and the corresponding loudness information. If multiple loudness values ​​are transmitted for a group, the loudness value associated with the given playback loudspeaker configuration is selected. Similarly, corrected loudness L mod is the loudness information of the selected group and the corrected gain h after user interaction. i It is calculated from. The loudness compensation gain C for the default audio scene or preset is set to the reference loudness L refand corrected loudness L mod It is calculated based on: The loudness compensation gain C is applied to the audio signal before playback to provide the output signal.

[0155] In some embodiments, it is not possible to measure the necessary loudness information for all groups at the encoder, so the encoder calculates estimates of the missing group loudness values. The encoder may apply different methods to estimate the missing (unmeasured) group loudness information, in which case loudness compensation at the decoder is performed in the same way as if loudness information was measured for all groups.

[0156] In a further embodiment, the audio stream has loudness information only for a limited number of groups, in which case the missing group loudness information is estimated at the decoder, where loudness compensation is then performed in the same way as if all the necessary loudness information was included in the metadata of the audio stream.

[0157] Other embodiments include blind loudness compensation as a fallback mode in case the decoder lacks any required group loudness information to perform accurate loudness compensation. As mentioned above, a set of indexes K for selecting groups to be included in the reference and modified loudness calculations is used. ref and K mod The same mechanism for determining K is used in the fallback mode. In other words, the set of group indices K ref and K mod The selection is still based on corresponding information generated on the encoder side, which is provided together with the metadata of the audio stream.

[0158] Several embodiments of the present invention are described below, which can be combined with the above-mentioned embodiments.

[0159] A first embodiment refers to an audio processor for processing an audio signal, the audio processor comprising an audio signal modification unit configured to modify the audio signal in response to a user input, a loudness control unit configured to determine a loudness compensation gain based on a reference loudness or a reference gain and based on a modified loudness or a modified gain, the modified loudness or the modified gain being dependent on the user input, and a loudness manipulation unit configured to manipulate the loudness of a signal using the loudness compensation gain.

[0160] A second embodiment, which is dependent on the first embodiment, refers to an apparatus, in which an audio signal comprises a bitstream with metadata, the metadata comprising a group loudness for a group and a gain value for a group.

[0161] A third embodiment, which is dependent on the first or second embodiment, refers to an apparatus, in which a loudness control unit is configured to calculate a reference loudness for a group or a set of groups using the group loudness or the group loudnesses and a reference gain value for the group or the reference gain value for the set of groups, and to calculate a modified loudness for a group or a set of groups using the group loudness or the group loudnesses and a modified gain value for the group or the modified gain value for the set of groups, wherein the modified gain value or the modified gain values ​​are modified by user input.

[0162] A fourth embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which the loudness control unit is configured to exclude a group for determining the reference loudness if said group is switched off in the metadata of the audio signal, or to exclude said group for determining the modified loudness if said group is switched off in response to user input, or to exclude a group from the calculation of the reference loudness if said group is switched off in the metadata and switched on by user input, and vice versa.

[0163] A fifth embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which a loudness control unit is configured to calculate a loudness compensation gain by relating a reference loudness to a loudness of a preset, the preset including one or more groups, a group including one or more objects.

[0164] A sixth embodiment, which depends on any of the preceding embodiments, refers to an apparatus, in which a loudness control unit is configured to perform a limiting operation on the loudness compensation gain such that the loudness compensation gain is below an upper threshold or such that the loudness compensation gain is above a lower threshold.

[0165] A seventh embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which a loudness manipulation unit is configured to apply a gain to the signal determined by a loudness compensation gain and an original normalization gain determined by a target level set by an audio processor and a metadata level indicated in the metadata of the audio signal.

[0166] An eighth embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which an audio signal includes compensation metadata information indicating which groups should or should not be used for determining a loudness compensation gain, and a loudness control unit is configured to use only the groups indicated by the compensation metadata information to be used for determining the loudness compensation gain, or to not use the groups indicated by the compensation metadata information not to be used for determining the loudness compensation gain.

[0167] A ninth embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which an audio signal is indicated to have an anchor element, and the loudness control unit is configured to use only information about one audio object or group of audio objects of the anchor element to determine a loudness compensation gain.

[0168] A tenth embodiment, which depends on one of the first to eighth embodiments, refers to an apparatus, in which an audio signal is indicated to have an anchor element, and a loudness control unit is configured to use only information about one audio object or a group of audio objects of the anchor element to determine a loudness compensation gain if one or more audio objects of the anchor element are amplified by user input, and to use information from one or more audio objects of the anchor element and information of one or more audio objects not included in the anchor element if one or more audio objects of the anchor element are attenuated by user input.

[0169] An eleventh embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which a loudness control unit is configured to calculate a missing group loudness in an audio signal using a loudness preset that includes at least two groups and non-missing gain and loudness information for that preset.

[0170] A twelfth embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which a loudness control unit is configured to perform blind loudness compensation using one or more gain values ​​for one or more groups and one or more modified gain values ​​for one or more groups.

[0171] A thirteenth embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which a loudness control unit checks whether an audio signal includes reference loudness information, and if the audio signal does not include reference loudness information, performs blind loudness compensation using one or more reference gain values ​​for one or more groups and one or more modified gain values ​​for the one or more groups, or checks whether modified loudness information cannot be calculated, and if modified loudness information cannot be calculated, performs blind loudness compensation, which includes using one or more reference gain values ​​for one or more groups and one or more modified gain values ​​for the one or more groups.

[0172] A fourteenth embodiment, which depends on one of the preceding embodiments, refers to an apparatus, in which an audio signal has different reference loudness information values ​​for different playback configurations, the apparatus further comprising a format conversion unit for converting the signal into a predetermined playback configuration, and the loudness control unit is configured to select a specific loudness value for the specific playback configuration used by the format conversion unit.

[0173] The fifteenth embodiment refers to an audio encoder that generates an audio signal including metadata, the audio encoder comprising a loudness determination unit that determines loudness for a group having one or more audio objects, and a metadata writing unit that introduces the loudness for the group as reference loudness information into the metadata.

[0174] A sixteenth embodiment, which is dependent on the fifteenth embodiment, refers to an audio encoder, in which a loudness determination unit is configured to determine different loudness values ​​for different playback configurations, and a metadata writing unit is configured to introduce the different loudness values ​​into metadata for the different playback configurations.

[0175] A 17th embodiment, which is dependent on the 15th or 16th embodiment, refers to an audio encoder, where the audio encoder further comprises a controller for determining which groups should or should not be used for loudness compensation, and a metadata writing unit configured to write an indication into the metadata indicating which groups should or should not be used for loudness compensation.

[0176] An 18th embodiment, which depends on one of the 15th to 17th embodiments, refers to an audio encoder, in which a loudness determination unit is configured to calculate a group loudness value for a group, which group loudness value is missing in the metadata, and a metadata writing unit is configured to introduce the missing loudness value into the metadata, such that all groups of the audio signal have associated reference loudness information.

[0177] A 19th embodiment refers to a method for processing an audio signal, comprising the steps of modifying the audio signal in response to a user input, determining a loudness compensation gain based on a reference loudness or a reference gain and based on a modified loudness or a modified gain, where the modified loudness or the modified gain is dependent on the user input, and manipulating the loudness of a signal using the loudness compensation gain.

[0178] A twentieth embodiment refers to a method for generating an audio signal including metadata, comprising the steps of determining loudness for a group having one or more audio objects and introducing the loudness for the group as reference loudness information into the metadata.

[0179] A twenty-first embodiment refers to a computer program for carrying out the method according to the nineteenth or twentieth embodiment when running on a computer or processor.

[0180] Although some aspects have been described above in the context of an apparatus, these aspects also represent a description of the corresponding method, and it is clear that a block or an apparatus corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a description of a method step also represent a corresponding block, item, or feature of the corresponding apparatus. Some or all of the method steps can be performed by (using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0181] The transmitted or encoded signals of the present invention may be stored on a digital storage medium or transmitted over a transmission medium, such as a wireless transmission medium or a wired transmission medium, such as the Internet.

[0182] Depending on certain implementation requirements, embodiments of the present invention can be configured in hardware or software. This can be implemented using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, flash memory, etc., having electronically readable control signals stored therein that cooperate (or are capable of cooperating) with a programmable computer system such that the methods of the present invention are performed. Thus, the digital storage medium can be computer readable.

[0183] Some embodiments according to the invention include a data carrier having electronically readable control signals which are operable with a computer system programmable to carry out one of the methods described above.

[0184] Generally, embodiments of the present invention may be configured as a computer program product having program code operable to perform one of the methods of the present invention when the computer program product runs on a computer, the program code may for example be stored on a machine readable carrier.

[0185] Other embodiments of the invention comprise the computer program stored on a machine readable carrier for performing one of the methods described above.

[0186] In other words, an embodiment of the inventive method is therefore a computer program having a program code for performing one of the above described methods, when the computer program runs on a computer.

[0187] Another embodiment of the invention is a data carrier (or digital storage medium, or computer readable medium) comprising a computer program recorded thereon for performing one of the methods described above. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transitory.

[0188] Another embodiment of the invention is also a data stream or a sequence of signals representing the computer program for performing one of the above mentioned methods, which data stream or sequence of signals may be adapted to be transmitted via a data communication connection, for example the Internet.

[0189] Another embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the above described methods.

[0190] Another embodiment comprises a computer having installed thereon the computer program for performing one of the methods described above.

[0191] Further embodiments of the invention include an apparatus or system configured to transmit (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, comprise a file server that transmits the computer program to the receiver.

[0192] In some embodiments, a programmable logic device (such as a field programmable gate array) may be used to perform some or all of the functions of the methods described above. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described above. In general, such methods may be preferably performed by any hardware apparatus.

[0193] The above-described embodiments are merely illustrative of the principles of the present invention. It will be understood that modifications and variations of the above-described arrangements and details will be apparent to those skilled in the art. It is therefore the intention to be limited only by the subject matter of the following appended claims, and not by the specific details expressed in the manner of the description and illustration of the embodiments. [remarks] [Claim 1] An audio processor (1) for processing an audio signal (100), comprising: an audio signal modifying unit (2) configured to modify the audio signal (100) in response to user input; On the other hand, the reference loudness (L ref ) or based on the reference gain (gi) and on the other hand on the corrected loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), mod a loudness control unit (6) configured to determine the loudness compensation gain (C) based on metadata of the audio signal (100) indicating which groups should or should not be used to determine the loudness compensation gain (C), the groups including one or more audio elements; a loudness manipulation unit (5) configured to manipulate the loudness of a signal using the loudness compensation gain (C); An audio processor comprising: [Claim 2] An audio processor (1) according to claim 1, the loudness control unit (6) is configured to determine the loudness compensation gain (C) based on at least one flag included in the metadata data; An audio processor, the flag indicating whether or how a group should be considered for determining the loudness compensation gain (C). [Claim 3] An audio processor (1) according to claim 1 or 2, The loudness control unit (6) is configured to use only a group for determining the loudness compensation gain (C) if the group belongs to an anchor included in the metadata of the audio signal (100). [Claim 4] An audio processor (1) according to claim 3, the loudness control unit (6) is configured to use only the group belonging to the anchor for determining the loudness compensation gain (C) if the modified gain (hi) of at least one group belonging to the anchor is greater than a corresponding reference gain (gi); and / or the loudness control unit (6) is configured to use the group belonging to the anchor and the group not belonging to the anchor to determine the loudness compensation gain (C) when the modified gain (hi) of at least one group belonging to the anchor is smaller than the corresponding reference gain (gi) and the modified gain (hi) depends on the user input. Audio processor. [Claim 5] An audio processor (1) for processing an audio signal (100), comprising: an audio signal modifying unit (2) configured to modify the audio signal (100) in response to user input; On the other hand, the reference loudness (L ref ) or based on the reference gain (gi) and on the other hand on the corrected loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), moda loudness control unit (6) configured to determine the loudness compensation gain (C) based on metadata of the audio signal (100) that refers to at least one preset, the preset referring to a set of at least one group containing one or more audio elements; a loudness manipulation unit (5) configured to manipulate the loudness of a signal using the loudness compensation gain (C); An audio processor comprising: [Claim 6] An audio processor (1) according to claim 5, 5. The audio processor (1) constructed according to any one of claims 1 to 4. [Claim 7] An audio processor (1) according to any one of claims 1 to 6, The loudness control unit (6) is configured to determine the loudness compensation gain (C) based on a group loudness (Li) and / or a gain value (gi) of the at least one group of the set referenced by the preset. [Claim 8] An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) calculates the reference loudness (L) for the set mentioned by the preset using the individual group loudness (Li) and the individual gain value (gi). ref ) configured to determine The loudness control unit (6) calculates the modified loudness (L) for the set mentioned by the preset using the individual group loudness (Li) and the individual modified gain value (hi). mod ), and the modified gain value (hi) is modified by the user input; Audio processor. [Claim 9] An audio processor (1) according to any one of claims 5 to 8, the loudness control unit (6) is configured to determine the loudness compensation gain (C) based on data of the metadata referring to a selected preset, The preset is selected by the user input. [Claim 10] An audio processor (1) according to any one of claims 5 to 9, the loudness control unit (6) is configured to determine the loudness compensation gain (C) based on data of the metadata referring to a default preset, An audio processor, wherein the default preset is set prior to or independent of the user input. [Claim 11] An audio processor (1) for processing an audio signal (100), comprising: an audio signal modifying unit (2) configured to modify the audio signal (100) in response to user input; On the other hand, the reference loudness (L ref ) or based on the reference gain (gi) and on the other hand on the corrected loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), mod a loudness control unit (6) configured to determine the loudness compensation gain (C) based on metadata of the audio signal (100) indicating whether a group is switched off or switched on, the group comprising one or more audio elements; a loudness manipulation unit (5) configured to manipulate the loudness of a signal using the loudness compensation gain (C); An audio processor comprising: [Claim 12] An audio processor (1) according to claim 11, The audio processor (1) is constructed according to any one of claims 1 to 10. [Claim 13] An audio processor (1) according to claim 11 or 12, The loudness control unit (6) adjusts the modified loudness (L mod ) to determine a group of adjacent audio signals. [Claim 14] An audio processor (1) according to any one of claims 11 to 13, The loudness control unit (6) reduces the reference loudness (L ref ) and if a group is switched on by the user input, the modified loudness (L mod ) configured to include said group to determine and / or The loudness control unit (6) controls the reference loudness (L ref ), and when a group is switched off by the user input, mod ) by excluding the group. Audio processor. [Claim 15] An audio processor (1) for processing an audio signal (100), comprising: an audio signal modifying unit (2) configured to modify the audio signal (100) in response to user input; On the other hand, the reference loudness (Lref ) or based on the reference gain (gi) and on the other hand on the corrected loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), mod a loudness control unit (6) configured to determine the loudness compensation gain (C) based on metadata of the audio signal (100) lacking at least one group loudness among a group of metadata contained in the audio signal, the modified gain (hi) being dependent on the user input; a loudness manipulation unit (5) configured to manipulate the loudness of the signal (101) using the loudness compensation gain (C); An audio processor comprising: [Claim 16] An audio processor (1) according to claim 15, 15. The audio processor (1) constructed according to any one of claims 1 to 14. [Claim 17] An audio processor (1) according to claim 15 or 16, The loudness control unit (6) calculates the missing group loudness (Lp) using a preset loudness (Lp), a reference gain (gi) for a group having a missing group loudness, and a group loudness (Li) and a reference gain (gi) for a group having a group loudness (Li). A ). [Claim 18] An audio processor (1) according to any one of claims 15 to 17, The loudness control unit (6) is configured to determine the loudness compensation gain (C) using only at least one reference gain (gi) and at least one modified gain (hi) for blind loudness compensation when metadata of the audio signal (100) is missing at least one group loudness. [Claim 19] An audio processor (1) according to any one of claims 15 to 18, The loudness control unit (6) is configured to determine the loudness compensation gain (C) using only at least one reference gain (gi) and at least one modified gain (hi) for blind loudness compensation when metadata of the audio signal (100) is invalid for group loudness. [Claim 20] An audio processor (1) for processing an audio signal (100), comprising: an audio signal modifying unit (2) configured to modify the audio signal (100) in response to user input; On the other hand, the reference loudness (L ref ) or based on the reference gain (gi) and on the other hand on the corrected loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), mod a loudness control unit (6) configured to determine the loudness compensation gain (C) based on metadata of the audio signal (100) referring to a playback configuration for the reproduction of the audio signal (100), the modified gain (hi) being dependent on the user input; a loudness manipulation unit (5) configured to manipulate the loudness of the signal (101) using the loudness compensation gain (C); An audio processor comprising: [Claim 21] An audio processor (1) according to claim 20, 20. The audio processor (1) constructed according to any one of claims 1 to 19. [Claim 22] An audio processor (1) according to claim 20 or 21, The loudness control unit (6) is configured to determine the loudness compensation gain (C) based on data of the metadata, which refers to a playback configuration and includes an associated group loudness (Li) and / or reference gain value (gi). [Claim 23] An audio processor (1) according to any one of the preceding claims, the audio signal (100) comprises a bitstream having the metadata; and An audio processor, wherein the metadata includes the reference gain (gi) for at least one group. [Claim 24] An audio processor (1) according to any one of the preceding claims, 13. An audio processor, comprising: an audio signal processor, the audio signal processor comprising: a first audio signal processor; [Claim 25] An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) calculates the reference loudness (L) for at least one group by using the group loudness (Li) and the gain value (gi) of the at least one group. ref ) configured to determine The loudness control unit (6) calculates the modified loudness (L mod ) configured to determine The modified gain value (hi) is modified by the user input. Audio processor. [Claim 26] An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) calculates the reference loudness (L) for each of the groups by using the group loudness (Li) and the gain value (gi) of each of the groups. ref ) configured to determine The loudness control unit (6) calculates the modified loudness (L) for each of the groups using the group loudness (Li) and the modified gain value (hi) of each of the groups. mod ), Audio processor. [Claim 27] An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) controls the loudness compensation gain (C) to exceed an upper threshold (C max ) and / or the loudness compensation gain (C) is lower than a lower threshold (C min ) . [Claim 28] An audio processor (1) according to any one of the preceding claims, The loudness operation unit (5) controls the loudness compensation gain (C) and the normalization gain (G N ) and the corrected gain (G corrected ) to the signal, and the normalized gain (G N ) is determined by a target loudness level set by user input and a metadata loudness level included in the metadata of the audio signal (100). [Claim 29] An audio encoder (20) for generating an audio signal (100) including metadata, comprising: a loudness determiner (21) for determining a loudness value for at least one group having one or more audio elements (50); a metadata writing unit (22) for introducing the determined loudness value into the metadata as a group loudness (Li); 2. An audio encoder comprising: [Claim 30] 30. An audio encoder (20) according to claim 29, comprising: the loudness determiner (21) is configured to determine different loudness values ​​and / or different gain values ​​for different playback configurations; and The metadata writer (22) is configured to introduce the determined different loudness values ​​and / or different gain values ​​into the metadata in association with respective playback configurations. [Claim 31] An audio encoder (20) according to claim 29 or 30, the loudness determiner (21) is configured to determine different loudness values ​​and / or different gain values ​​for different presets referring to a set of at least one group comprising one or more audio elements; and The metadata writing unit (22) is configured to introduce the determined different loudness values ​​and / or different gain values ​​into the metadata in association with respective presets. [Claim 32] An audio encoder (20) according to any one of claims 29 to 31, comprising: Further comprising a controller (23); the controller (23) is configured to determine which groups should be used or ignored for determining the loudness compensation gain (C); and The metadata writing unit (22) is configured to write an indication into the metadata indicating which groups should be used or ignored for determining the loudness compensation gain (C). [Claim 33] An audio encoder (20) according to any one of claims 29 to 32, comprising: an estimator (24); The estimator (24) is configured to calculate a group loudness value for a group; the group loudness value for the group is not determined by the loudness determination unit (21), The metadata writer (22) is configured to introduce calculated group loudness values ​​into the metadata such that every group of the audio signal (100) has an associated group loudness. [Claim 34] A method for processing an audio signal (100), comprising the steps of: modifying the audio signal (100) in response to user input; On the other hand, the reference loudness (L ref ) or based on the reference gain (gi) and on the other hand on the corrected loudness (L mod ) or a modified gain (hi), determining a loudness compensation gain (C), mod ) or the modified gain (hi) is dependent on the user input; the loudness compensation gain (C) is determined based on metadata of the audio signal (100) indicating whether a group contained in the audio signal (100) should or should not be used to determine the loudness compensation gain (C), the group comprising one or more audio elements; and / or the loudness compensation gain (C) is determined based on metadata of the audio signal (100) that refers to a preset, the preset referring to a set of at least one group containing one or more audio elements; and / or the loudness compensation gain (C) is determined based on metadata of the audio signal (100) indicating whether a group is switched off or switched on, the group comprising one or more audio elements; and / or The loudness compensation gain (C) is a gain that is calculated based on at least one group loudness (L) of metadata of one group included in the audio signal (100). A ) is determined based on metadata of the audio signal (100) in a missing state, and / or the loudness compensation gain (C) is determined based on metadata of the audio signal (100) referring to a playback configuration for reproduction of the audio signal (100); manipulating the loudness of a signal using said loudness compensation gain (C); A method for providing the above. [Claim 35] A method for generating an audio signal (100) including metadata, the method comprising: determining a loudness value for a group having one or more audio elements; introducing the loudness value determined for the group into the metadata as group loudness (Li); The method includes: [Claim 36] 36. A computer program for carrying out the method according to claim 34 or 35 when running on a computer or processor.

Claims

1. An audio processor (1) for processing an audio signal (100), comprising: the audio signal (100) includes metadata; The audio processor (1) an audio signal modification unit (2) configured to modify said audio signal (100) in response to a user input (200) to obtain a modified signal (101); On the one hand, the reference loudness (L ref ) or the reference gain (gi), and on the other hand the modified loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), mod ) or the modified gain (hi) is dependent on the user input (200), a loudness control unit (6) configured to determine the loudness compensation gain (C) based on metadata of the audio signal (100) indicating which groups should or should not be used for determining the loudness compensation gain (C), the groups including one or more audio elements; a loudness manipulation unit (5) configured to manipulate the loudness of the modified signal (101) using the loudness compensation gain (C); An audio processor comprising:

2. 2. An audio processor (1) according to claim 1, the loudness control unit (6) is configured to determine the loudness compensation gain (C) based on at least one flag included in the metadata data; An audio processor, wherein the flag indicates whether or how a group should be considered for determining the loudness compensation gain (C).

3. An audio processor (1) according to claim 1 or 2, The loudness control unit (6) is configured to use only a group for determining the loudness compensation gain (C) if the group belongs to an anchor included in the metadata of the audio signal (100).

4. An audio processor (1) for processing an audio signal (100), comprising: the audio signal (100) includes metadata; The audio processor (1) an audio signal modification unit (2) configured to modify said audio signal (100) in response to a user input (200) to obtain a modified signal (101); On the one hand, the reference loudness (L ref ) or the reference gain (gi), and on the other hand the modified loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), mod ) or the modified gain (hi) is dependent on the user input (200), a loudness control unit (6) configured to determine the loudness compensation gain (C) based on the metadata of the audio signal (100) referring to at least one preset, the preset referring to a set of at least one group containing one or more audio elements; a loudness manipulation unit (5) configured to manipulate the loudness of the modified signal (101) using the loudness compensation gain (C); An audio processor comprising:

5. An audio processor (1) according to claim 4, The audio processor (1) is constructed according to any one of claims 1 to 3.

6. An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) is configured to determine the loudness compensation gain (C) based on a group loudness (Li) and / or a reference gain (gi) of the at least one group of the set referenced by the preset.

7. An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) calculates the reference loudness (L) for the set mentioned by the preset using the individual group loudness (Li) and the individual reference gain (gi). ref ) , The loudness control unit (6) uses the individual group loudness (Li) and the individual modified gain (hi) to calculate the modified loudness (L mod ) , and The modified gain (hi) is modified by the user input. Audio processor.

8. An audio processor (1) according to any one of claims 4 to 7, the loudness control unit (6) is configured to determine the loudness compensation gain (C) based on data of the metadata referring to a selected preset, The preset is selected by the user input (200).

9. An audio processor (1) according to any one of claims 4 to 8, the loudness control unit (6) is configured to determine the loudness compensation gain (C) based on data of the metadata referring to a default preset, An audio processor, wherein the default preset is set prior to or independent of the user input (200).

10. An audio processor (1) for processing an audio signal (100), comprising: the audio signal (100) includes metadata; The audio processor (1) an audio signal modification unit (2) configured to modify said audio signal (100) in response to a user input to obtain a modified signal (101); On the one hand, the reference loudness (L ref ) or the reference gain (gi), and on the other hand the modified loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), mod ) or the modified gain (hi) is dependent on the user input (200), a loudness control unit (6) configured to determine the loudness compensation gain (C) based on the metadata of the audio signal (100) indicating whether a group is switched off or switched on, the group comprising one or more audio elements; a loudness manipulation unit (5) configured to manipulate the loudness of the modified signal (101) using the loudness compensation gain (C); An audio processor comprising:

11. An audio processor (1) according to claim 10, The audio processor (1) is constructed according to any one of claims 1 to 9.

12. An audio processor (1) according to claim 10 or 11, The loudness control unit (6) adjusts the modified loudness (L mod ) excluding said group.

13. An audio processor (1) according to any one of claims 10 to 12, The loudness control unit (6) reduces the reference loudness (L ref ) and if a group is switched on by the user input (200), the modified loudness (L mod ) , and / or The loudness control unit (6) adjusts the reference loudness (L ref ), and if a group is switched off by the user input (200), mod ) by excluding the group. Audio processor.

14. An audio processor (1) for processing an audio signal (100), comprising: the audio signal (100) includes metadata; The audio processor (1) an audio signal modification unit (2) configured to modify said audio signal (100) in response to a user input to obtain a modified signal (101); On the one hand, the reference loudness (L ref ) or the reference gain (gi), and on the other hand the modified loudness (L mod ) or a modified gain (hi), the loudness control unit (6) configured to determine a loudness compensation gain (C), mod ) or the modified gain (hi) is dependent on the user input (200), a loudness control unit (6) configured to determine the loudness compensation gain (C) based on metadata of the audio signal (100) in a state where at least one group loudness is missing among the metadata of one group included in the audio signal; a loudness manipulation unit (5) configured to manipulate the loudness of the modified signal (101) using the loudness compensation gain (C); An audio processor comprising:

15. An audio processor (1) according to claim 14, The audio processor (1) is constructed according to any one of claims 1 to 13.

16. An audio processor (1) according to claim 14 or 15, The loudness control unit (6) is configured to determine the loudness compensation gain (C) using only at least one reference gain (gi) and at least one modified gain (hi) for blind loudness compensation when metadata of the audio signal (100) is missing at least one group loudness.

17. An audio processor (1) according to any one of claims 14 to 16, The loudness control unit (6) is configured to determine the loudness compensation gain (C) using only at least one reference gain (gi) and at least one modified gain (hi) for blind loudness compensation when metadata of the audio signal (100) is invalid for group loudness.

18. An audio processor (1) according to any one of the preceding claims, the audio signal (100) comprises a bitstream having the metadata; and An audio processor, wherein the metadata includes the reference gain (gi) for at least one group.

19. An audio processor (1) according to any one of the preceding claims, 13. An audio processor, comprising: an audio signal processor, the audio signal processor comprising: a first audio signal processor;

20. An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) uses the group loudness (Li) and the reference gain (gi) of at least one group to calculate the reference loudness (L ref ) , The loudness control unit (6) calculates the modified loudness (L mod ) , The modified gain (hi) is modified by the user input (200). Audio processor.

21. An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) calculates the reference loudness (L) for each of the plurality of groups by using the group loudness (Li) of each of the plurality of groups and the reference gain (gi). ref ) , The loudness control unit (6) calculates the modified loudness (L) for each of the groups using the group loudness (Li) and the modified gain (hi) for each of the groups. mod ) configured to determine Audio processor.

22. An audio processor (1) according to any one of the preceding claims, The loudness control unit (6) controls the loudness compensation gain (C) to be equal to or lower than an upper threshold (C max ) and / or the loudness compensation gain (C) is lower than a lower threshold (C min ) .

23. An audio processor (1) according to any one of the preceding claims, The loudness operation unit (5) is a corrected gain (G corrected ) to the modified signal, corrected ) is the loudness compensation gain (C) and the loudness normalization gain (G N ) and the loudness normalized gain (G N ) is determined by the target loudness level set by the user input (200) and a metadata loudness level included in the metadata of the audio signal (100).

24. A method for processing an audio signal (100), comprising the steps of: the audio signal (100) includes metadata; The method comprises: modifying said audio signal (100) in response to user input (200) to obtain a modified signal (101); On the other hand, the reference loudness (L ref ) or the modified loudness (L mod ) or a modified gain (hi), determining a loudness compensation gain (C), mod ) or the modified gain (hi) is dependent on the user input; the loudness compensation gain (C) is determined based on the metadata of the audio signal (100) indicating whether a group contained in the audio signal (100) should or should not be used to determine the loudness compensation gain (C), the group comprising one or more audio elements; and / or the loudness compensation gain (C) is determined based on the metadata of the audio signal (100) referring to a preset, the preset referring to at least one set of groups containing one or more audio elements; and / or the loudness compensation gain (C) is determined based on the metadata of the audio signal (100) indicating whether a group is switched off or switched on, the group comprising one or more audio elements; and / or The loudness compensation gain (C) is a gain that compensates for at least one group loudness (L) in one group of the metadata included in the audio signal (100). A ) based on the metadata of the audio signal (100) in a missing state; and manipulating the loudness of the modified signal (101) using the loudness compensation gain (C); A method comprising:

25. 24. A computer program for carrying out the method according to claim 23 when running on a computer or processor.

Citation Information

Patent Citations

  • Decoder, encoder and method for informed loudness estimation employing by-pass audio object signals in object-based audio coding systems

    WO2015078964A1

  • Decoder, encoder and method for informed loudness estimation in object-based audio coding systems

    EP2879131A1