Method and apparatus for audio data processing

Abstract

Decoder devices, computer programs and methods for processing audio data for playback are described. They include receiving a bitstream, the bitstream including encoded audio data, metadata including DRC set(s), and, for each DRC set, an indication of whether the DRC set is configured to provide a loudness leveling effect. The metadata further includes personalized experience information. The method further includes identifying a DRC set configured to provide a dynamic range compensation effect; decoding the encoded audio data to obtain decoded audio data; selecting one of the identified DRC sets configured to provide a loudness leveling effect; extracting one or more DRC gains from the bitstream corresponding to the selected DRC set; applying the one or more DRC gains corresponding to the selected DRC set to the decoded audio data to obtain dynamic loudness compensated audio data; and outputting the dynamic loudness compensated audio data for playback.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to the following priority applications: U.S. Provisional Application No. 63 / 328,035, filed April 06, 2022, and European Application No. 22172243.2, filed May 09, 2022.

[0002] technology The present disclosure generally relates to methods for metadata-based dynamic processing of audio data for playback, and in particular to methods for determining and applying one or more processing parameters to audio data for loudness leveling and / or dynamic range compression in combination with personalization settings (dialogue enhancement, home or away commentary, etc.). The present disclosure further relates to methods for encoding audio data and metadata for loudness leveling and / or dynamic range compression into a bitstream. The present disclosure further relates to respective decoders and encoders, and respective systems and computer program products. The present disclosure further relates to methods for processing audio data for playback, decoders for processing audio data for playback, and respective computer program products.

[0003] Although certain embodiments are described herein with particular reference to that disclosure, it will be understood that the disclosure is not limited to such fields of use but is applicable in a broader context. [Background technology]

[0004] Any discussion of background technology throughout this disclosure should in no way be taken as an admission that such technology is widely known or forms part of the common general knowledge in the art.

[0005] When reproducing audio content, loudness is the individual experience of sound pressure. In cinema or television content, the loudness of the dialogue in a program has been found to be the most decisive parameter determining the perception of program loudness by the listener.

[0006] An analysis of the entire program must be performed to determine the average loudness of the program, whether the complete program or just the dialogue. Average loudness is typically required for loudness compliance (e.g. CALM Act in the United States) and is also used to align Dynamic Range Control (DRC) parameters. The dynamic range of a program is the difference between its quietest and loudest sounds. The dynamic range of a program depends on its content, for example an action movie may have a different, wider dynamic range than a documentary, reflecting the creator's intention. However, the ability of devices to reproduce audio content in its original dynamic range varies greatly. In addition to loudness management, dynamic range control is a further important factor in providing an optimal listening experience.

[0007] To perform loudness management and dynamic range control, the entire audio program or audio program segments must be analyzed and the resulting loudness and DRC parameters may be delivered together with the audio data or encoded audio data to be applied at a decoder or playback device.

[0008] When analysis of the entire audio program or audio program segments prior to encoding is not available, e.g., in real-time (dynamic) encoding, loudness processing or leveling is used to ensure loudness compliance and, if applicable, potential dynamic range constraints depending on the playback requirements. This approach delivers processed audio that is "optimized" for a single playback environment.

[0009] What is needed, therefore, is a metadata-based process that delivers the "original", unprocessed audio along with accompanying metadata, and allows playback devices to use the metadata to dynamically modify the audio depending on device constraints, user requirements, and user settings (e.g., audio personalization settings).

[0010] Additionally, industry audio standards include descriptions and syntax to enable loudness control and / or loudness management. For example, the Moving Picture Experts Group (MPEG), a coalition of working groups jointly established by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), sets standards for media coding, including audio coding. MPEG is organized under ISO / IEC SC 29, and the audio group is currently identified as Working Group (WG) 6. This WG-6 helped establish the MPEG-H 3D audio standard, which includes compatibility for loudness control and / or loudness management (DRC) techniques, but the existing standard needs to be further revised to handle the metadata mentioned above. Summary of the Invention [Means for solving the problem]

[0011] According to a first aspect of the present disclosure, there is provided a method of metadata-based dynamic processing of audio data for playback. The method may include receiving, by a decoder, a bitstream including audio data and metadata for loudness leveling. The method may further include decoding, by the decoder, the audio data and the metadata to obtain decoded audio data and the metadata. The method may further include determining, by the decoder, from the metadata, one or more processing parameters for loudness leveling based on playback conditions. The method may further include applying the determined one or more processing parameters to the decoded audio data to obtain processed audio data. The method may also include outputting the processed audio data for playback.

[0012] In some embodiments, the metadata may indicate processing parameters for loudness leveling for multiple playback conditions.

[0013] In some embodiments, determining the one or more processing parameters may further include determining one or more processing parameters for dynamic range compression (DRC) based on playback conditions.

[0014] In some embodiments, the playback conditions may include one or more of the decoder device type, playback device characteristics, loudspeaker characteristics, loudspeaker setup, background noise characteristics, ambient noise characteristics, a personalized experience selected on the device, and characteristics of the acoustic environment.

[0015] The personalized experience may be based on the version of the audio, such as language, or it may be based on the user experience, such as enhancing dialogue, and may also include the ability to select different experiences or perspectives, for example, selecting home team commentary or away team commentary, or selecting the home crowd or away crowd as the backdrop.

[0016] The personalized experience may depend on previous listening experiences and / or the capabilities of the listening device, or the personalized experience may be selected by the device (including through the inclusion of external data via the cloud) based on previous listening preferences.

[0017] Personalized experiences may be encoded in real-time, for example sports with home and away commentary, where loudness leveling is used to ensure that the audio follows loudness compliance (e.g. CALM Act in the US). For metadata-based solutions, loudness leveling metadata, which may also include DRC metadata, is generated for each of the different personalized experiences, device capabilities.

[0018] In some embodiments, determining the one or more processing parameters may further include selecting, by the decoder, at least one of a set of DRC sequences DRCSet, a set of equalizer parameters EQSet and a downmix corresponding to playback conditions.

[0019] In some embodiments, determining the one or more processing parameters may further include identifying metadata identifiers indicative of the at least one selected DRCSet, EQSet and downmix to determine the one or more processing parameters from metadata.

[0020] In some embodiments, the metadata may include one or more processing parameters related to the average loudness value and optionally one or more processing parameters related to dynamic range compression characteristics.

[0021] In some embodiments, the bitstream may further include additional metadata for static loudness adjustments to be applied to the decoded audio data.

[0022] In some embodiments, the bitstream may be an MPEG-D DRC bitstream and the presence of metadata may be signaled based on MPEG-D DRC bitstream syntax.

[0023] In some embodiments, the uniDrcConfigExtension() element may be used to carry the metadata as a payload.

[0024] In some embodiments, the metadata may include one or more metadata payloads, each of which may include multiple sets of parameters and identifiers, each set may include at least one of a DRCSet identifier drcSetId, an EQSet identifier eqSetId, and a downmix identifier downmixId in combination with one or more processing parameters related to the identifiers in that set.

[0025] In some embodiments, determining the one or more processing parameters may involve selecting a set from among the multiple sets in the payload based on the at least one DRCSet, EQSet and a downmix selected by the decoder, and the one or more processing parameters determined by the decoder may be the one or more processing parameters related to an identifier in a selected set.

[0026] According to a second aspect of the present disclosure, there is provided a decoder for metadata-based dynamic processing of audio data for playback, the decoder may have one or more processors and non-transitory memory configured to perform a method including receiving, by the decoder, a bitstream including audio data and metadata for loudness leveling, decoding, by the decoder, the audio data and the metadata to obtain decoded audio data and the metadata, determining, by the decoder, from the metadata one or more processing parameters for loudness leveling based on a playback condition, applying the determined one or more processing parameters to the decoded audio data to obtain processed audio data, and outputting the processed audio data for playback.

[0027] According to a third aspect of the present disclosure, there is provided a method of encoding audio data and metadata for loudness leveling into a bitstream. The method may include inputting original audio data to a loudness leveler for loudness processing and obtaining loudness processed audio data as output from the loudness leveler. The method may further include generating metadata for loudness leveling based on the loudness processed audio data and the original audio data. The method may also include encoding the original audio data and the metadata into a bitstream.

[0028] In some embodiments, the method may further include generating additional metadata for static loudness adjustment for use by the decoder.

[0029] In some embodiments, generating the metadata may include comparing the loudness processed audio data with the original audio data, and the metadata may be generated based on a result of the comparison.

[0030] In some embodiments, generating the metadata may further include measuring loudness over one or more predefined time periods, and the metadata may be generated further based on the measured loudness.

[0031] In some embodiments, the measuring may include measuring an overall loudness of the audio data.

[0032] In some embodiments, the measuring may include measuring a loudness of dialogue in the audio data.

[0033] In some embodiments, the bitstream may be an MPEG-D DRC bitstream and the presence of metadata may be signaled based on MPEG-D DRC bitstream syntax.

[0034] In some embodiments, the uniDrcConfigExtension() element may be used to carry the metadata as a payload.

[0035] In some embodiments, the metadata may include one or more metadata payloads, each of which may include multiple sets of parameters and identifiers, each set may include at least one of a DRCSet identifier drcSetId, an EQSet identifier eqSetId, and a downmix identifier downmixId in combination with one or more processing parameters related to the identifiers in the set, wherein the one or more processing parameters may be parameters for loudness leveling by a decoder.

[0036] In some embodiments, the at least one of drcSetId, eqSetId and downmixId may relate to at least one of a set of DRC sequences DRCSet, a set of equalizer parameters EQSet and a downmix to be selected by the decoder.

[0037] According to a fourth aspect of the present disclosure, there is provided an encoder for encoding original audio data and metadata for loudness leveling in a bitstream, The encoder may comprise one or more processors and non-transitory memory configured to perform a method including: inputting original audio data to a loudness leveler for loudness processing to obtain loudness processed audio data as output from the loudness leveler; generating metadata for loudness leveling based on the loudness processed audio data and the original audio data; and encoding the original audio data and the metadata into a bitstream.

[0038] According to a fifth aspect of the present disclosure, there is provided a system including an encoder for encoding original audio data and metadata for loudness leveling in a bitstream and a decoder for metadata-based dynamic processing of the audio data for playback.

[0039] According to a sixth aspect of the present disclosure, there is provided a computer program product including a computer readable storage medium having instructions adapted, when executed by an apparatus having processing capability, to cause the apparatus to perform a method for metadata based dynamic processing of audio data for playback or a method for encoding audio data and metadata for loudness leveling into a bitstream.

[0040] According to a seventh aspect of the present disclosure there is provided a computer readable storage medium having stored thereon a computer program product as described herein.

[0041] According to an eighth aspect of the present disclosure, there is provided a method of processing audio data for playback. The method may include receiving, by a decoder, a bitstream including encoded audio data and metadata. The metadata includes one or more dynamic range control (DRC) sets, and for each DRC set, an indication of whether the DRC set is configured to provide a loudness leveling effect. The method may further include parsing, by the decoder, the metadata to identify a DRC set configured to provide a loudness leveling effect. The method may further include decoding, by the decoder, the encoded audio data to obtain decoded audio data. The method may further include selecting, by the decoder, one of the identified DRC sets configured to provide a loudness leveling effect. The method may further include applying, by the decoder, the one or more DRC gains corresponding to the selected DRC set to the decoded audio data to obtain dynamic loudness compensated audio data. The method may also include outputting the dynamic loudness compensated audio data for playback.

[0042] In some embodiments, the metadata may include a plurality of DRC sets configured to provide loudness leveling, each of the plurality of DRC sets may also be associated with one or more playback conditions, and the selecting may be performed in response to an indication of a playback condition provided to the decoder.

[0043] In some embodiments, in addition to providing loudness leveling effects, the one or more DRC sets may be configured to provide dynamic range control.

[0044] In some embodiments, the playback conditions may include one or more of: a decoder device type, playback device characteristics, loudspeaker characteristics, loudspeaker setup, background noise characteristics, ambient noise characteristics, and acoustic environment characteristics.

[0045] In some embodiments, an indication of whether a DRC set is configured to provide a loudness leveling effect may be provided in a parameter indicating one or more effects provided by the DRC set.

[0046] In some embodiments, the parameter indicating the one or more effects provided by the DRC set may be a drcSetEffect bitfield in an MPEG-D DRC bitstream, with individual bits of the drcSetEffect bitfield corresponding to different effects, and one of the bits of the drcSetEffect bitfield corresponding to a loudness leveling effect.

[0047] In some embodiments, the indication of whether a DRC set is configured to provide a loudness leveling effect may be whether a DRC set is specified in a loudness leveling bitstream payload.

[0048] In some embodiments, the loudness leveling bitstream payload may be included in an extension field of a previously defined bitstream syntax.

[0049] In some embodiments, the extension field may be the uniDrcConfigExtension field of an MPEG-D DRC bitstream, and the loudness leveling bitstream payload may only be included for certain values ​​of the uniDrcConfigExtType parameter.

[0050] In some embodiments, multiple loudness leveling payloads specifying multiple DRC sets configured to provide loudness leveling effects may be included in an extension field of the previously defined bitstream syntax.

[0051] In some embodiments, the indication of whether a DRC set is configured to provide loudness leveling effects may be a field of a pre-existing configuration element of a previously defined bitstream syntax.

[0052] In some embodiments, the field may be a levelingPresent parameter and the pre-existing component may be a downmixInstructions element, a drcInstructionsBasic element, or a drcInstructionsUniDRC element of an MPEG-D DRC bitstream.

[0053] In some embodiments, the field may be a pre-existing field reserved for future use.

[0054] In some embodiments, the indication of whether a DRC set is configured to provide loudness leveling effects may be a field in an updated version of a pre-existing component of a previously defined bitstream syntax.

[0055] In some embodiments, the field may be a levelingPresent parameter and the updated version of a pre-existing component may be a downmixInstructionsV2 element or a drcInstructionsUniDrcV2 element.

[0056] In some embodiments, an indication that a loudness leveling effect is desired may be provided to the decoder via the interface, and a DRC set may be selected in response to the indication provided to the decoder via the interface.

[0057] In some embodiments, an indication of an additional desired effect may be provided to the decoder through an interface, and the metadata may include multiple DRC sets configured to provide a loudness leveling effect, with the selection depending on the additional desired effect.

[0058] In some embodiments, an indication that a loudness leveling effect is desired may be provided via a drcEffectTypeRequest parameter in the dynamicRangeControllerInterface payload.

[0059] In some embodiments, the metadata may include one or more static loudness values ​​configured to provide a static loudness adjustment to the decoded audio data.

[0060] In some embodiments, a static loudness adjustment may be applied to the decoded audio data or the dynamic loudness compensated audio data in response to one or more of the static loudness values.

[0061] In some embodiments, a first one of the DRC sets may be configured to provide dynamic range control, and the first DRC set may include an indication that a selected DRC set configured to provide a loudness leveling effect may be applied in combination with the first DRC set.

[0062] In some embodiments, the selected DRC set may include an indication of whether one or more DRC gains corresponding to the selected DRC set may be applied only in combination with a DRC gain corresponding to the first DRC set.

[0063] In some embodiments, a DRC gain corresponding to the first DRC set may be extracted from the bitstream and applied to the decoded audio data.

[0064] According to a ninth aspect of the present disclosure, there is provided a decoder for processing audio data for playback, the decoder may comprise one or more processors and non-transitory memory configured to perform the method for processing audio data for playback as described above.

[0065] According to a tenth aspect of the present disclosure there is provided a computer program product having a computer readable storage medium having instructions adapted to cause a device to perform a method of processing audio data for playback as described above.

[0066] According to an eleventh aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program product as described herein. [Brief description of the drawings]

[0067] Exemplary embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0068] [Figure 1] An example of a decoder for metadata-based dynamic processing of audio data for playback is shown.

[0069] [Diagram 2] An example of a method for metadata-based dynamic processing of audio data for playback is presented.

[0070] [Figure 3A] We present an example of an encoder that encodes the original audio data and metadata for loudness leveling in the bitstream.

[0071] [Figure 3B] 1 shows an example of an encoder for encoding original audio data and metadata for loudness leveling and personalization in a bitstream.

[0072] [Figure 4A] 1 shows an example of how audio data and metadata for loudness leveling can be encoded into a bitstream.

[0073] [Figure 4B] An example of how audio data and metadata for personalized loudness leveling can be encoded into a bitstream is shown.

[0074] [Diagram 5] 1 illustrates an example of a device comprising one or more processors and non-transitory memory configured to perform the methods described herein.

[0075] [Figure 6] Demonstrates an example of how to process audio data for playback. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0076] Overview The average loudness of a program or dialogue is the main parameter or value used for loudness compliance of broadcast or streaming programs. The average loudness is typically set to -24 or -23 LKFS. In audio codecs that support loudness metadata, this single loudness value, which represents the loudness of the entire program, is carried in the bitstream. Using this value in the decoding process allows gain adjustments that result in predictable playback levels, so that programs are played back at a known and consistent level. It is therefore important that this loudness value is set properly and accurately. Since the average loudness relies on measuring the entire program before encoding, this is not possible in real-time situations such as dynamic encoding with unknown loudness and dynamic range variations.

[0077] When it is not possible to measure the loudness of the entire file before encoding, dynamic loudness levelers are often used to correct or contour the audio data before encoding to meet the required loudness. This type of loudness management is often considered a poor way to meet compliance, as it often changes the dynamic range correlation in the audio content, and therefore may change the creative intent. This is especially true when it is desired to deliver one audio asset for all playback devices. This is one of the advantages of metadata-driven codecs and delivery systems.

[0078] In some approaches, the audio content is mixed with the required target loudness and the corresponding loudness metadata is set to that value. A loudness leveler can still be used in these situations, as it is used to help steer the audio content towards the target loudness, but it is not "active" and is only used when the audio content starts to deviate from the required target loudness.

[0079] In view of the above, the method and apparatus described herein aims to make even real-time processing situations, also called dynamic processing situations, metadata-driven. The metadata allows loudness leveling and dynamic range compression in real-time situations. The described method and apparatus advantageously enable: Use of real-time loudness adjustment and DRC in MPEG-D DRC and MPEG-H 3D audio syntax; · Real-time loudness adjustment and DRC usage in combination with downmixId; · Real-time loudness adjustment and use of DRC in combination with drcSetId; · Real-time loudness adjustment and DRC usage in combination with eqSetId; Use of real-time loudness adjustments and DRC in combination with personalization settings identified using mae_groupID and mae_groupPresetID.

[0080] That is, depending on the decoder settings (e.g. DRCSet, EQSet, downmix and personalization settings), the decoder can search a given payload for an appropriate set of parameters and identifiers based on said syntax by matching said settings to identifiers. The parameters included in the set whose identifier best matches the settings can then be selected as processing parameters for loudness leveling to be applied to the original audio data received for correction.

[0081] Additionally, multiple sets of parameters for dynamic processing (multiple instances of dynLoudCompValue) can be sent.

[0082] In addition to correcting the overall loudness, metadata-driven loudness leveling can also be used to "center" the DRC gain calculation and application. This centering may be a result of correcting the loudness of the content via loudness leveling and the way DRC is typically calculated and applied. In this sense, metadata for loudness leveling can be said to be used to align the DRC parameters.

[0083] Metadata-based dynamic processing of audio data A decoder 100 for metadata-based dynamic processing of audio data for playback is described with reference to the example of Fig. 1. The decoder 100 may have one or more processors and non-transitory memory configured to perform a method including the process as illustrated in the example of Fig. 2 by steps S101 to S105.

[0084] The decoder 100 may receive a bitstream containing audio data and metadata, and may be able to output, depending on requirements, unprocessed (original) audio data, processed audio data after applying dynamic processing parameters determined from the metadata and / or the metadata itself. The metadata includes personalization metadata (including interactivity metadata) as described herein. The personalization metadata may include information for identifying a personalized experience.

[0085] The personalized experience may be based on the version of the audio, such as language, or it may be based on the user experience, such as enhancing dialogue, and may also include the ability to select different experiences or perspectives, for example, selecting home team commentary or away team commentary, or selecting the home crowd or away crowd as the backdrop.

[0086] The personalized experience may depend on previous listening experiences and / or the capabilities of the listening device, or the personalized experience may be selected by the device (including through the inclusion of external data via the cloud) based on previous listening preferences.

[0087] Personalized experiences may be encoded in real-time, for example sports with home and away commentary, where loudness leveling is used to ensure that the audio follows loudness compliance (e.g. CALM Act in the US). For metadata-based solutions, loudness leveling metadata, which may also include DRC metadata, is generated for each of the different personalized experiences, device capabilities.

[0088] Referring to the example of Fig. 2, in step S101, the decoder 100 may receive a bitstream including audio data and metadata for loudness leveling and optionally dynamic range compression (DRC). The audio data may be encoded audio data, and the audio data may further be unprocessed, i.e., the audio data may be original audio data.

[0089] The format of the bitstream is not limited, but in one embodiment the bitstream may be an MPEG-D DRC bitstream. The presence of metadata for dynamic processing of audio data may then be signaled based on MPEG-D DRC bitstream syntax. In one embodiment, a uniDrcConfigExtension() element may be used to carry the metadata as payload, as further detailed below. Alternatively, the bitstream may be an MPEG-H 3D Audio compatible bitstream, and the metadata may be signaled using MPEG-H 3D Audio bitstream syntax, for example, using MPEG-H Audio Stream (MHAS) packets.

[0090] In step S102, the audio data and metadata may then be decoded by a decoder to obtain decoded audio data and said metadata. In an embodiment, the metadata may include one or more processing parameters related to an average loudness value and, optionally, one or more processing parameters related to dynamic range compression characteristics. The metadata allows for the application of dynamic or real-time corrections. For example, when encoding and decoding for live real-time playback, application of "real-time" or dynamic loudness metadata is desired to ensure that the live playback audio is properly loudness managed. The metadata further includes personalization metadata as referred to herein, which may be generalized metadata to allow processing by a variety of devices and device capabilities.

[0091] In step S103, the decoder determines from the metadata one or more processing parameters for loudness leveling based on the playback conditions. In an embodiment, the playback conditions may include one or more of the decoder's device type, playback device characteristics, loudspeaker characteristics, loudspeaker setup, background noise characteristics, ambient noise characteristics, and acoustic environment characteristics. Additionally, the playback conditions may include information regarding a personalized experience as described herein. Consideration of the playback conditions allows the decoder to make a targeted selection of processing parameters for loudness leveling with respect to device and environmental constraints.

[0092] In an embodiment, the process of determining one or more processing parameters in step S103 may further include selecting, by the decoder, at least one of a set of DRC sequences DRCSet, a set of equalizer parameters EQSet, and a downmix corresponding to the playback conditions, such that said at least one of DRCSet, EQSet, and downmix correlates with or is indicative of individual device and environmental constraints resulting from the playback conditions.

[0093] In an embodiment, the process of determining in step S103 may further comprise identifying a metadata identifier indicative of said at least one selected DRCSet, EQSet and DownmixSet in order to determine one or more processing parameters from the metadata. The metadata identifier thus allows to associate the metadata with the corresponding selected DRCSet, EQSet and / or downmix and thus with the respective playback conditions.

[0094] In an embodiment, the metadata may include one or more metadata payloads (e.g., dynLoudComp() payloads as shown in Table 5 below). Each metadata payload may include multiple sets of parameters (e.g., parameter dynLoudCompValue) and identifiers, each set may include at least one of a DRCSet identifier drcSetId, an EQSet identifier eqSetId, a downmix identifier downmixId, and a personalization identifier mae_groupID / mae_groupPresetID in combination with one or more processing parameters related to an identifier in the set. That is, each payload may include an array of items, each item including a processing parameter and an identifier (e.g., drcSetId, eqSetId, downmixId, mae_groupID / mae_groupPresetID). In a further embodiment, the determination in step S103 may thus involve selecting a set from among the multiple sets in the payload based on the at least one DRCSet, EQSet, and downmix selected by the decoder, and the one or more processing parameters determined in step S103 may be the one or more processing parameters related to an identifier in the selected set. That is, depending on the settings present in the decoder (e.g. DRCSet, EQSet, downmix, personalization settings), the decoder can search a given payload for an appropriate set of parameters and identifiers by matching said settings to identifiers. The parameters included in the set whose identifier best matches the settings can be selected as processing parameters for loudness leveling.

[0095] In step S104, the determined processing parameter or parameters are then applied by the decoder to the decoded audio data to obtain processed audio data, the processed audio data, e.g. live real-time audio data, being thus appropriately loudness managed.

[0096] In step S105, the processed audio data may be output for playback.

[0097] In some embodiments, the bitstream may further include additional metadata for static loudness adjustment applied to the decoded audio data, where static loudness adjustment refers to processing performed for general loudness normalization, as opposed to dynamic processing for real-time situations.

[0098] Carrying metadata for dynamic processing separately from the additional metadata for general loudness normalization allows not applying "real-time" corrections.

[0099] For example, when encoding and decoding for live real-time playback, the application of dynamic processing is desired to ensure that the live playback audio is properly loudness managed, but for non-real-time playback, or transcoding where dynamic compensation is not desired or required, dynamic processing parameters determined from the metadata do not need to be applied.

[0100] By further retaining (dynamic / real-time) metadata for dynamic processing separately from the additional metadata, the original unprocessed content can be preserved if desired. The original audio is encoded along with the metadata. This allows the playback device to selectively apply dynamic processing, further enabling playback of the original audio content on high-end devices capable of playing the original audio.

[0101] As mentioned above, keeping the dynamic loudness metadata separate from long-term loudness measurements / information such as contentLoudness (in ISO / IEC23003-4) has several advantages: when combined, the available metadata is a composite value, so the content loudness (or what it should be after the dynamic loudness metadata has been applied) does not indicate the actual loudness of the content. In addition to removing this ambiguity of what the content loudness (or program or anchor loudness) is, there are several cases where this is particularly beneficial:

[0102] Keeping the metadata for dynamic processing separate allows a decoder or playback device to turn off the application of dynamic processing and instead apply an implemented real-time loudness leveler to avoid cascade leveling. This situation may arise, for example, if the device's own real-time leveling solution is better than the one used with the audio codec, or, for example, if the device's own real-time leveling solution cannot be disabled and is therefore always active, leading to a compromised playback experience in further processing.

[0103] Keeping the metadata for dynamic processing separate also allows for transcoding to codecs that don't support loudness leveling, and may wish to apply their own loudness processing before re-encoding.

[0104] A broadcast from live with a single encoding for the live feed is a further example. Dynamic processing metadata may be used or stored for archive or on-demand services. Thus, for archive or on-demand services, a more accurate or compliant loudness measurement based on the entire program may be performed and the appropriate metadata may be reset.

[0105] This is also beneficial for use cases where a fixed target loudness is used throughout the workflow, for example in an R128 compliant situation where -23LKFS (or LUFS) is recommended. In this scenario, the addition of dynamic processing metadata is a "safety" measure, the content is close to the required target, and the addition of dynamic processing metadata is assumed to be a secondary check. Thus, it is desirable to have the ability to turn it off.

[0106] Encoding of audio data and metadata for loudness leveling With reference to the examples of Figures 3A, 3B and 4, an encoder for encoding original audio data and metadata for loudness leveling and optionally dynamic range compression (DRC) in a bitstream is described, which may have one or more processors and non-transitory memory configured to carry out a method including processes such as those shown in the steps in the examples of Figures 4A and 4B.

[0107] Referring to the examples of Figures 3A and 4A, in step S201, original audio data may be input to a loudness leveller 201 for loudness processing, and loudness-processed audio data may be obtained as an output from the loudness leveller 201.

[0108] In step S202, metadata for loudness leveling may be generated based on the loudness processed audio data and the original audio data. Appropriate smoothing and time frames may be used to reduce artifacts.

[0109] In an embodiment, step S202 may include a comparison by the analyzer 202 of the loudness processed audio data with the original audio data, and metadata may be generated based on the results of said comparison. The metadata thus generated may emulate the effect of a leveller at the decoder site. The metadata may include: -Gain (wideband and / or multiband) processing parameters that when applied to the original audio produce loudness-compliant audio for playback; - Processing parameters that describe the dynamics of the audio, for example: Peaks - Sample and true peaks ○Short-term loudness value ○Short-term loudness value changes

[0110] In an embodiment, step S202 may further include measuring, by the analyzer 202, the loudness over one or more predefined time periods, and the metadata may be generated further based on the measured loudness. In an embodiment, the measuring may include measuring an overall loudness of the audio data. Alternatively or additionally, in an embodiment, the measuring may include measuring a loudness of dialogue in the audio data.

[0111] In step S203, the original audio data and the metadata may then be encoded into a bitstream. The format of the bitstream is not limited, but in one embodiment the bitstream may be an MPEG-D DRC bitstream and the presence of metadata may be signaled based on MPEG-D DRC bitstream syntax. In this case, in one embodiment, a uniDrcConfigExtension() element may be used to carry the metadata as payload, as described further below.

[0112] In an embodiment, the metadata may include one or more metadata payloads, each of which may include multiple sets of parameters and identifiers, each set including at least one of a DRCSet identifier drcSetId, an EQSet identifier eqSetId, a downmix identifier downmixId, and a personalization identifier, mae_groupID / mae_groupPresetID, in combination with one or more processing parameters related to the identifiers in the set, which may be parameters for loudness leveling by the decoder. In this case, in an embodiment, the at least one of drcSetId, eqSetId, and downmixId may relate to at least one of a set of DRC sequences DRCSet, a set of equalizer parameters EQSet, and a downmix to be selected by the decoder.

[0113] In one embodiment, the method may further include generating additional metadata for static loudness adjustment for use by the decoder. Keeping the metadata for loudness leveling and the additional metadata separate in the bitstream and further encoding the original audio data into the bitstream has several advantages, as detailed above.

[0114] 3B and 4B, in step S200B, original audio data may be input to a personalization block 200B, which may process the original audio (and / or additional information or metadata) to determine and / or generate personalized audio and personalization metadata.

[0115] In step S201B, the personalized audio data from 200B is input to a loudness leveller 201B for loudness processing, and personalized loudness processed audio data is obtained as output from the loudness leveller 201B, which may be similar or the same as the loudness leveller 201 described in the context of Figures 3A and 3B.

[0116] In step S202B, metadata for loudness leveling may be generated based on the personalized loudness processed audio data output from S201B / analyser 201B and the personalized audio data output from S200B / personalization block 200B. In an embodiment, step S202 may include a comparison of the loudness processed audio data with the original audio data by the analyzer 202B, and metadata may be generated based on the results of said comparison. Step 202B of FIG. 3B may be performed according to the description of step S202 of FIG. 3A.

[0117] In step S203B, the personalized audio data from S200B / 200B and the metadata from S200B and / or S202B may be encoded into a bitstream. Step 203B outputs the encoded audio data and metadata, including the personalized metadata as described herein. The format of the bitstream is not limited, but in one embodiment, the bitstream may be an MPEG-D DRC bitstream and the presence of the metadata may be signaled based on MPEG-D DRC bitstream syntax. Alternatively, the bitstream may be an MPEG-H 3D Audio compatible bitstream and the metadata may be signaled based on MPEG-H 3D Audio bitstream syntax. In one embodiment, the metadata may include one or more metadata payloads as described in the context of step S203 of FIG. 3A. The methods described herein (e.g., in the context of FIG. 3A and FIG. 3B) may be implemented on a decoder or encoder, respectively, and the decoder and encoder may comprise one or more processors and non-transitory memories configured to perform the methods. An example of a device having such processing capabilities is shown in the example of FIG. 5, which shows said device 300 including two processors 301 and non-transitory memory 302 .

[0118] It is noted that the method described herein with respect to Figures 3A and 3B may further be implemented on a system as described herein, including an encoder for encoding original audio data and metadata for loudness leveling and optionally dynamic range compression (DRC) in a bitstream, and a decoder for metadata-based dynamic processing of the audio data for playback.

[0119] The methods may further be implemented as a computer program product comprising a computer readable storage medium having instructions adapted, when executed by a device having processing capability, to cause the device to perform the method. The computer program product may be stored in the computer readable storage medium.

[0120] Modified bitstream syntax for MPEG-D DRC The following describes how the MPEG-D DRC bitstream syntax as described in ISO / IEC 23003-4 may be modified in accordance with the embodiments described herein.

[0121] The MPEG-D DRC syntax may be extended, for example, by the loudnessInfoSetExtension() element shown in Table 1 below, to carry dynamic processing metadata as frame-based dynLoudComp updates.

[0122] For example, another switch case UNIDRCLOUDEXT_DYNLOUDCOMP may be added to the loudnessInfoSetExtension() element as shown in Table 1. The switch case UNIDRCLOUDEXT_DYNLOUDCOMP may be used to identify a new element dynLoudComp() as shown in Table 5. The loudnessInfoSetExtension() element may be an extension of the loudnessInfoSet() element as shown in Table 2. Furthermore, the loudnessInfoSet() element may be part of the uniDRC() element as shown in Table 3. [Syntax, No. of bits, Mnemonic] [Table 1] [Table 2] [Table 3] [Table 4] [Table 5] · drcSetId allows dynLoudComp (metadata related) to be applied per DRC set. · eqSetId allows dynLoudComp to be applied in combination with different settings for the equalization tool. downmixId allows dynLoudComp to be applied per downmixId. · mae_groupID allows dynLoudComp to be applied depending on the personalization settings set on the device. mae_groupPresetID allows dynLoudComp to be applied depending on the personalization preset settings set on the device.

[0123] In some cases, in addition to the above parameters, it may be beneficial for the dynLoudComp() element to also include a methodDefinition parameter (e.g. specified by 4 bits) specifying the loudness measurement method used to derive the dynamic program loudness metadata (e.g. anchor loudness, program loudness, short-term loudness, instantaneous loudness, etc.) and / or a measurementSystem parameter (e.g. specified by 4 bits) specifying the loudness measurement system used to measure the dynamic program loudness metadata (e.g. EBU R.128, ITU-R BS-1770 with or without preprocessing, ITU-R BS-1771, etc.). Such parameters may for example be included in the dynLoudComp() element between the downmixId parameter and the dynLoudCompValue parameter. Alternative Syntax 1 [Syntax, No. of bits, Mnemonic] [Table 6] [Table 7] [Table 8]

[0124] In some cases, it may be beneficial to modify the syntax shown above in Table 8 so that the levelingPresent and (if levelingPresent==1) dynLoudCompValue parameters follow the reliability parameter within the measurementCount loop of the loudnessInfoV2() payload, rather than outside the measurementCount loop. Additionally, it may also be beneficial to set dynLoudCompValue equal to 0 when levelingPresent is 0.

[0125] Alternative Syntax 2 Alternatively, the dynLoudComp() element can be placed within a uniDrcGainExtension() element. [Syntax, No. of bits, Mnemonic] [Table 9] [Table 10] [Table 11] Semantics dynLoudCompValue This field contains the value of dynLoudCompDb. The value is encoded according to Table 12. The default value is 0dB. [Table 12]

[0126] In one example, the dynLoudCompValue may be transported in the bitstream using a syntax for transporting the DRC gain relative to the DRC set. Updated MPEG-D DRC loudness normalization process [Table 13]

[0127] Pseudocode for dynLoudComp selection and processing Additional parameters such as mae_groupID or mae_PresetID that identify different combinations of audio elements for a user personalized setting may be used for the selection process. / * Selection process * / / * The following settings are derived from the user / decoder settings drcSetId = 1; eqSetID = 2; downmixId = 3; mae_groupID = 4; mae_groupPresetID = 5; * / findMatchingDynLoudComp (drcSetId, eqSetID, downmixId) { dynLoudComp = UNDEFINED; / * Check if matching loudnessInfo set is present * / if (targetLoudnessPresent(drcSetId, eqSetID, downmixId, mae_groupID, mae_groupPresetID)) continue; else { levelingPresent = false; exit(0); } / * if all values ​​are defined * / if (drcSetId != UNDEFINED && eqSetID != UNDEFINED && downmixId != UNDEFINED) { for (num=0; num <num_of_dynLoudCompValues; num++) { if (dynLoudCompArray[num].drcSetId == drcSetId && dynLoudCompArray[num].eqSetID == eqSetID && dynLoudCompArray[num].downmixId == downmixId) { / * Correct entry found, assign to dynLoudComp * / dynLoudComp = dynLoudCompArray[num].dynLoudCompValue; } } } else if (drcSetId == UNDEFINED) { for (num=0; num <num_of_dynLoudCompValues; num++) { if (dynLoudCompArray[num].eqSetID == eqSetID && dynLoudCompArray[num].downmixId == downmixId) { / * Correct entry found, assign to dynLoudComp * / dynLoudComp = dynLoudCompArray[num].dynLoudCompValue; } } } else if (eqSetID == UNDEFINED) { for (num=0; num<num_of_dynLoudCompValues; num++) { if (dynLoudCompArray[num].drcSetId == drcSetId && dynLoudCompArray[num].downmixId == downmixId) { / * Correct entry found, assign to dynLoudComp * / dynLoudComp = dynLoudCompArray[num].dynLoudCompValue; } } } else if (downmixId == UNDEFINED) { for (num=0; num<num_of_dynLoudCompValues; num++) { if (dynLoudCompArray[num].drcSetId == drcSetId && dynLoudCompArray[num].eqSetID == eqSetID) { / * Correct entry found, assign to dynLoudComp * / dynLoudComp = dynLoudCompArray[num].dynLoudCompValue; } } } else if (drcSetId == UNDEFINED && downmixId == UNDEFINED) { for (num=0; num <num_of_dynLoudCompValues; num++) { if (dynLoudCompArray[num].eqSetID == eqSetID) { / * Correct entry found, assign to dynLoudComp * / dynLoudComp = dynLoudCompArray[num].dynLoudCompValue; } } } else if (drcSetId == UNDEFINED && eqSetID == UNDEFINED) { for (num=0; num <num_of_dynLoudCompValues; num++) { if (dynLoudCompArray[num].downmixId == downmixId) { / * Correct entry found, assign to dynLoudComp * / dynLoudComp = dynLoudCompArray[num].dynLoudCompValue; } } } else if (eqSetID == UNDEFINED && downmixId == UNDEFINED) { for (num=0; num <num_of_dynLoudCompValues; num++) { if (dynLoudCompArray[num].eqSetID == eqSetID) { / * Correct item found, assigned to dynLoudComp * / dynLoudComp = dynLoudCompArray[num].dynLoudCompValue; } } } if (dynLoudComp == UNDEFINED){ levelingPresent = false; exit(1); } else { levelingPresent = true; exit(0); } }

[0128] In some cases, in addition to the selection process shown in the pseudocode above (e.g., taking into account drcSetId, eqSetID, downmixId, mae_groupID and mae_PresetID to select the dynLoudCompValue parameter), it may be beneficial for the selection process to also take into account the methodDefinition parameter and / or the measurementSystem parameter to select the dynLoudCompValue parameter. Alternative updated MPEG-D DRC loudness normalization process [Table 14]

[0129] If the alternative loudness normalization process of Table 14 above is used, the loudness normalization process pseudo-code above may be replaced by the alternative loudness normalization process pseudo-code below: Note that a default value for dynLoudCompDb, e.g., 0dB, may be assumed to ensure that the value of dynLoudCompDb is defined even if loudness leveling metadata is not present in the bitstream.

[0130] Alternative Syntax 3 In some cases it may be beneficial to combine the syntax described above in Tables 1 to 5 with Alternative Syntax 1 described above in Tables 6 to 8, as shown in the tables below. This allows for greater flexibility for the transmission of loudness leveling values. [Table 15] [Table 16] [Table 17] [Table 18]

[0131] In some cases, it may be beneficial to allow control, e.g., by an end user, of whether loudness leveling is performed even when loudness leveling information is present in the received bitstream. Such control may be provided by updating the MPEG-D DRC interface syntax to include an additional interface extension (e.g., UNIDRCINTERFACEEXT_LEVELING) that contains a loudness leveling control interface payload (e.g., levelingControlInterface()), as shown in the tables below. [Table 19] [Table 20] [Table 21] Interface Extension Semantics loudnessLevelingOn This flag signals whether loudness leveling should be turned on or off. If loudness leveling is turned on, this flag is equal to 1. If loudness leveling is turned off, this flag is equal to 0. The default value is 1.

[0132] Additional methods for enabling loudness leveling In addition to the above, additional ways to enable loudness leveling are possible. For example, instead of including a parameter indicating a specific loudness leveling value to be applied to the signal, it is possible to define a specific set of DRC gains that already include loudness leveling, so that loudness leveling is applied when those specific DRC gains are applied by the decoder. A set of DRC gains suitable for applying such loudness leveling may be identified through a specific bitstream element, as described further below. The advantage of such an approach is that it is not necessary to transmit explicit loudness leveling gains in addition to other loudness information.

[0133] Additionally, the user can specify through the interface that DRC gains, including loudness leveling, should be selected and applied, as described further below.

[0134] Finally, additional bitstream elements may be provided to allow tighter creative control over whether and how loudness leveling is performed. For example, a flag may indicate whether loudness leveling can be turned off by the user. In such a case, if the content creator allows loudness leveling to be disabled, loudness leveling will either be applied or not applied, as specified by the user.

[0135] Specifying a DRC set with loudness leveling via the DRC Set Effect field In the MPEG-D DRC standard, it is possible to indicate one or more DRC effects provided by a particular DRC set. For example, a DRC set may be suitable for "late night" viewing, viewing in a "noisy environment", viewing at "low playback levels", etc. A DRC set effect parameter may indicate which particular effects are provided by the DRC set.

[0136] Furthermore, the MPEG-D DRC standard allows a user to specify one or more desired DRC effects, as well as one or more optional fallback DRC effects. Such information from the user may be used to select the most appropriate DRC set from the available DRC sets. For example, if a DRC set exists that matches the desired DRC effect, that set may be selected. If no such set exists, but a set exists that matches the fallback DRC effect, that set may be selected.

[0137] The list of defined DRC effects may be expanded or updated to include a DRC effect that provides loudness leveling. Additionally, the list of DRC effects that may be specified by a user through the interface may also be expanded or updated to include a DRC set effect that provides loudness leveling. Thus, loudness leveling may be provided by indicating a desire for a DRC set that provides loudness leveling through the interface, including a DRC set that is indicated as providing a loudness leveling effect, in which case the decoder selects and applies the DRC set that corresponds to the loudness leveling DRC effect.

[0138] As mentioned above, a previous version of a table specifying DRC set effects that may be signaled in the bitstream (e.g., Table A.45 of ISO / IEC 23003-4:2020) may be updated to include an entry for a "loudness leveling" effect as shown below. Specifically, a row with a particular bit position (e.g., 13) corresponding to the loudness leveling effect may be added as shown in the table below. Alternatively, an existing row with a particular bit position (e.g., 12) corresponding to the "Ducking / Leveling self" effect may be updated as shown in the table below. [Table 22]

[0139] In the above example, a bit in a bit field associated with a particular DRC set (e.g., the drcSetEffect bit field of a drcInstructionsBasic() or drcInstructionsUniDrc() or drcInstructionsUniDrcV1() payload) may be set to indicate that the DRC set provides a loudness leveling effect (e.g., bit position 13 or bit position 12).

[0140] Similarly, a previous version of a table (e.g., Table 12 in ISO / IEC 23003-4:2020) specifying DRC set effects that may be specified to a decoder through an interface may be updated to include an entry for loudness leveling effect as shown below. Specifically, a row with a specific index value (e.g., 9) corresponding to the loudness leveling effect may be added as shown in the table below. [Table 23]

[0141] In the above example, the interface parameter (e.g., the drcEffectTypeRequest parameter of the dynamicRangeControllerInterface() payload) may be set to a particular value (e.g., 9) corresponding to a loudness leveling effect to instruct the decoder that a DRC set providing a loudness leveling effect set should be selected and applied if available. However, since loudness leveling may be intended to be used in combination with loudness normalization, in some cases, a decoder may require that in order to select and apply a DRC set providing a loudness leveling effect, the decoder must also perform loudness normalization. Such a requirement may be achieved by allowing the decoder to select a DRC set providing a loudness leveling effect only when loudness normalization is also enabled in the decoder. Alternatively, or in addition, such a requirement may be met by modifying the interface (e.g., by setting both the loudness normalization on flag and the target loudness value) to require that loudness normalization must also be enabled when requesting a DRC set providing a loudness leveling effect.

[0142] Specifying DRC sets with loudness leveling using the new extended payload Alternatively, a new payload (e.g., dynLoudInstructions()) may be defined that contains instructions for loudness leveling. One or more of such payloads may be included, for example, in an extension field (e.g., UNIDRCCONFEXT_V2) of an existing bitstream. Including such payloads in the extension field may maintain compatibility with existing decoders that were not designed to parse the new payload because such decoders would simply ignore the additional payload. Each such payload would be assigned a unique identifier (e.g., drcSetId) that corresponds to the DRC set that provides loudness leveling, and a flag (e.g., levelingPresent flag) may be set to 1 for each such DRC set signaled through the new payload to help easily identify such sets.

[0143] Additionally, the user interface described above for selecting a DRC set that provides a loudness leveling effect may also be used for DRC sets signaled through this new type of payload (e.g., dynLoudInstructions() payload). Specifically, a user may indicate to the decoder through this interface (e.g., using the drcEffectTypeRequest field) that a DRC set that provides loudness leveling is desired. The decoder then selects the DRC set signaled through the new payload, if present (e.g., a DRC set with a DRC set ID identified as corresponding to loudness leveling, e.g., with the levelingPresent flag set to 1). However, as described above, loudness leveling may be intended to be used in combination with loudness normalization, so in some cases, a decoder may require that in order to select and apply a DRC set that provides a loudness leveling effect, the decoder must also perform loudness normalization. Such a requirement may be achieved by allowing the decoder to select a DRC set that provides a loudness leveling effect only if loudness normalization is also enabled in the decoder. Alternatively or additionally, such a requirement could be met by modifying the interface to require that when requesting a DRC set that provides loudness leveling effects, loudness normalization must also be enabled (e.g., by setting both the loudness normalization on flag and a target loudness value).

[0144] An advantage of using this type of signaling (instead of signaling a DRC set that provides loudness leveling through an existing parameter such as, for example, the drcSetEffect parameter) is that such a new payload can include additional parameters indicating whether and how the user can enable and / or disable loudness leveling, as described above. For example, the new payload could also include, for each DRC set that applies loudness leveling, a parameter (e.g., a dynamicLoudCompSwOffAllowed parameter) indicating whether it is allowed to turn off loudness leveling through the user interface.

[0145] Example syntax for new payloads for loudness leveling (e.g., dynLoudInstructions() payload) is shown in Table 24. [Table 24]

[0146] As described above, a unique DRC set ID may be associated with the payload, as well as a flag (e.g., levelingPresent) that specifically identifies the DRC set as a DRC set that provides loudness leveling. Additionally, the payload may be associated with one or more downmix identification parameters (e.g., downmixId parameter) that indicate that the DRC set is intended to be used with one or more particular downmixes / downmix configurations of an audio program. Such a DRC set identifier (e.g., drcSetId parameter) as well as the flag (e.g., levelingPresent flag) permits selection of the DRC set associated with the payload in response to a user indication provided through an interface that a loudness leveling DRC effect is desired by the user. Additionally, the inclusion of one or more downmix identifiers (e.g., downmixID parameter) permits selection of a DRC set that provides loudness leveling that is intended to be used with one or more particular downmixes of an audio program. Additionally, the payload may be associated with one or more personalization identification parameters (e.g., mae_groupID, mae_groupPresetID) that indicate that the DRC set is intended for use with one or more particular personalization configurations of the audio program. Finally, as discussed above, the content creator may have additional control over whether loudness leveling may be applied by the decoder by including a parameter (e.g., dynamicLoudComSwOffAllowed) that indicates whether loudness leveling is allowed to be turned off.

[0147] Finally, as mentioned above, such additional payloads (e.g., dynLoudInstructions() payloads) may be included in an extension payload (e.g., uniDrcConfigExtension() payload) of the configuration payload. A specific extension type (e.g., uniDrcConfigExtType of UNIDRCCONFEXT_V2 with a value of 0x2) may be used to indicate the presence of such additional payloads in the extension field of the configuration payload.

[0148] An example of the syntax for the extension field of a configuration payload (eg, uniDrcConfigExtension() payload) is shown in Table 25. [Table 25] TIFF2025511384000027.tif236166 TIFF2025511384000028.tif154166

[0149] The data associated with a particular extension type in the extension field of the configuration payload may include a parameter indicating whether loudness leveling instructions are present (e.g., a dynLoudPresent parameter), and if present, a parameter indicating the number of sets of loudness leveling instructions in the extension payload (e.g., a dynLoudInstructionsCount parameter), and each set of loudness leveling instructions (e.g., each dynLoudInstructions() payload), where the syntax of the loudness leveling instructions may be as shown in Table 25.

[0150] Signaling of DRC sets for loudness leveling through existing components Further alternatively, it may be possible to indicate that a particular DRC set provides loudness leveling by adding a new field to an existing component. For example, a new parameter (e.g., a levelingPresent parameter) indicating whether a DRC set provides loudness leveling or not may be included in an existing component (e.g., in a downmixInstructions() element, a drcInstructionsBasic() element, or a drcInstructionsUniDrc() element). Ideally, to avoid compatibility issues with legacy decoders, the new element may be included in a field that is already ignored by legacy decoders (e.g., a reserved field). Alternatively, the new parameter may be included in an updated version of an existing component (e.g., in a downmixInstructionsV2() element or a drcInstructionsUniDrcV2() element), which will be ignored by legacy decoders.

[0151] In one example, signaling a DRC set for loudness leveling through an existing component may be using the extension of the uniDrcConfigExtension() element shown in Table 26 to carry instructions to control and apply the loudness leveling metadata received as dynamic processing metadata as frame-based updates based on the DRC set.

[0152] The switch case UNIDRCCONFEXT_LEVELING may be added to the uniDrcConfigExtension() element as shown in Table 26. The switch case UNIDRCCONFEXT_LEVELING may be used to identify new elements including levelingPresent and duckingOnlyDrcSetPresent. The uniDrcConfigExtension() element may be an extension of the uniDrcConfig() element described in ISO / IEC 23003-4. The new parameters including levelingPresent and duckingOnlyDrcSetPresent may be used to extend the drcInstructionsBasic(), drcInstructionsUniDrc() and drcInstructionsUniDrcV1() elements defined in ISO / IEC 23003-4.

[0153] The new elements, including levelingPresent and duckingOnlyDrcSetPresent, can be used as stand-alone parameters or as included parameters in the drcInstructionsBasic(), drcInstructionsUniDrc() and drcInstructionsUniDrcV1() elements for DRC set selection as described in ISO / IEC 23003-4, 6.3.7.

[0154] Any leveling gains that need to be applied as part of the loudness leveling process (such as the parameter dynLoudCompValue in the other syntax examples) may be included in the bitstream using the respective uniDrcGain() elements. [Table 26] TIFF2025511384000030.tif237163

[0155] The semantics of the new elements, including levelingPresent and duckingOnlyDrcSetPresent, may be defined as shown in Table 27. [Table 27]

[0156] Finally, similar to the selection process described above, a DRC set thus signaled as providing loudness leveling can be selected through an interface to the decoder by indicating that a DRC set providing a loudness leveling effect is desired. When such an indication is provided to the decoder through the interface, the decoder identifies and selects the DRC set that is indicated as providing loudness leveling (e.g., has a levelingPresent parameter equal to 1). If multiple DRC sets are indicated as providing loudness leveling, the decoder may use other rules (e.g., predefined rules as defined in ISO / IEC 23003-4:2020) to select the most appropriate DRC set among the multiple DRC sets that provide loudness leveling.

[0157] However, since loudness leveling may be intended to be used in combination with loudness normalization, in some cases a decoder may require that in order to select and apply a DRC set that provides a loudness leveling effect, the decoder must also perform loudness normalization. Such a requirement may be achieved by allowing the decoder to select a DRC set that provides a loudness leveling effect only when loudness normalization is also enabled in the decoder. Alternatively, or in addition, such a requirement may be met by modifying the interface (e.g., by setting both the loudness normalization on flag and the target loudness value) to require that loudness normalization must also be enabled when requesting a DRC set that provides a loudness leveling effect.

[0158] FIG. 6 illustrates an exemplary method of encoding audio data for playback as described above. Referring to the example of FIG. 6, in step S301, the decoder 100 may receive a bitstream including encoded audio data and metadata, where the metadata includes one or more dynamic range control (DRC) sets and, for each DRC set, an indication of whether the DRC set is configured to provide a loudness leveling effect matching a selected personalized experience, e.g., as described above. The metadata may include personalization information and / or instructions as described herein.

[0159] The format of the bitstream is not limited, but in one embodiment the bitstream may be an MPEG-D DRC bitstream. Alternatively, the bitstream may be an MPEG-H 3D Audio compliant bitstream. The presence of metadata for providing loudness leveling effects may then be signaled based on, for example, the MPEG-D DRC bitstream syntax as described above.

[0160] In step S302, the decoder may then parse the metadata to identify a DRC set configured to provide a loudness leveling effect that matches the selected personalization experience, for example as described above. Further, in step S302, the decoder may parse the metadata to identify personalization metadata that allows for providing a loudness leveling effect that matches the selected personalization experience.

[0161] In step S303, the decoder may then decode the audio data, for example, as described above, to obtain decoded audio data.

[0162] In step S304, the decoder may select one of the identified DRC sets configured to provide a loudness leveling effect that matches the selected personalization experience, e.g., as described above, The selection may be based on the personalization metadata identified in step S302.

[0163] In step S305, the decoder may extract from the bitstream one or more DRC gains corresponding to the selected DRC set, eg, as described above.

[0164] In step S306, the decoder may apply one or more DRC gains corresponding to the selected DRC set to the decoded audio data, e.g., as described above, to obtain loudness-leveled audio data.

[0165] Finally, in step S307, the loudness leveled audio data may be output for playback, for example as described above.

[0166] Efficient encoding / transmission of loudness leveling data As mentioned above, loudness leveling in combination with dynamic range control may be achieved by transmitting a DRC set that includes gains that are a combination of dynamic range control gains and loudness leveling gains. However, doing so may require redundant transmission of dynamic range control gain data. For example, a first DRC set may include gains for applying dynamic range control to a signal, and it may be important to allow that DRC gain set to be applied independently of other gains (e.g., loudness leveling gains). Thus, if a DRC set that includes gains for applying a combination of dynamic range control and loudness leveling is desired, a second DRC set needs to be specified that represents the combination of the first set of dynamic range control gains and the desired loudness leveling gains. Unfortunately, doing so requires that both the first and second DRC sets include information about the first set of dynamic range control gains, which is inefficient.

[0167] A more efficient way of transmitting data to achieve the same goal would be to provide only dynamic range control gains in a first DRC set and only loudness leveling gains in a second DRC set, along with some additional metadata indicating the relationship between the two DRC sets. For example, the first DRC set may include a parameter indicating that there is another DRC set that depends on the first DRC set. The decoder would then understand that it is possible to apply the first set of gains independently, for example if only dynamic range control is desired. Furthermore, the decoder would understand that it is also possible to combine the gains of the first and second DRC sets to obtain a combination of dynamic range control and loudness leveling. Finally, if the gains of the second set are never intended to be applied alone, metadata can be included with the second DRC set to indicate whether the gains of that DRC set may only be used in conjunction with the first set, or whether they may be used independently of the gains of the first set.

[0168] For example, consider two DRC sets, a first DRC set identified as DRC set 1 (e.g., having a drcSetId equal to 1) that provides gains for dynamic range control, and a second DRC set identified as DRC set 2 (e.g., having a drcSetId equal to 2) that provides gains for loudness leveling. Because DRC set 2 is intended to be used in combination with DRC set 1, DRC set 1 may include a parameter indicating that DRC set 2 depends on DRC set 1 (e.g., the dependsOnDrcSet parameter of DRC set 1 may be equal to 2). Furthermore, if loudness leveling is only intended to be used in combination with dynamic range control, DRC set 2 may include a parameter indicating that the gains of DRC set 2 are not to be used independently (e.g., the noIndependentUse flag of DRC set 2 may be set to a value of 1). Conversely, if DRC Set 2 is not intended to be used solely in combination with dynamic range control, DRC Set 2 may include a parameter indicating that the gains of DRC Set 2 may be used independently (e.g., the noIndependentUse flag of DRC Set 2 may be set to a value of 0).

[0169] The efficient encoding / transmission of loudness leveling data described above may be used in conjunction with any of the additional methods for enabling loudness leveling described above.

[0170] For example, if the presence of a DRC set that provides a loudness leveling effect is first signaled through an existing effect flag (e.g., a bit in a drcSetEffect bitfield), DRC set 1 may have a drcSetEffect parameter with the bit corresponding to the loudness leveling effect set to 0, and DRC set 2 may have the bit corresponding to the loudness leveling effect set to 1, with all other bits in the bitfield being the same for both DRC sets. Through such signaling, when a user requests a loudness leveling effect through an interface, the decoder extracts and applies the gains of DRC set 1 and DRC set 2 to apply a combination of dynamic range control and loudness leveling.

[0171] Table 28 shows exemplary values ​​of parameters that achieve the above signaling of efficient transmission of loudness leveling data. [Table 28]

[0172] In the second case where the presence of a DRC set providing loudness leveling effect is signaled using a new payload, DRC set 2 is signaled through the new payload (e.g., the drcSetId parameter of the dynLoudInstructions payload is set to 2). DRC set 1 is signaled as described above (e.g., the drcSetId parameter is set to 1 and the dependsonDrcSet parameter is set to 2 to indicate that DRC set 2 depends on DRC set 1). Finally, it would not be possible to indicate through the above-mentioned new payload (e.g., dynLoudInstructions) that DRC set 2 should not be used independently (e.g., that DRC set 2 should always be used together with DRC set 1). Thus, an additional parameter may be added to the new payload (e.g., the noIndependentUse parameter may be added to the dynLoudInstructions payload) such that when the new parameter is set to 1, DRC set 2 is used only together with DRC set 1, and when the new parameter is set to 0, DRC set 2 may be used independently of DRC set 1. As in the first case, when the user requests the loudness leveling effect through the interface, the decoder extracts and applies the gains from both DRC Set 1 and DRC Set 2 to apply a combination of dynamic range control and loudness leveling.

[0173] Finally, in the third case, existing components are used that already contain all of the necessary parameters to efficiently enable loudness leveling as described above. Specifically, DRC Set 1 contains an indication that DRC Set 2 depends on DRC Set 1 as described above, and DRC Set 2 contains an indication whether it may be used independently or only in conjunction with DRC Set 2. Again, as in the first two cases, when the user requests a loudness leveling effect through the interface, the decoder extracts and applies the gains of DRC Set 1 and DRC Set 2 to apply a combination of dynamic range control and loudness leveling.

[0174] As mentioned above, the advantage of using a dependent DRC set to enable loudness leveling is that it allows for more efficient transmission of the data needed for loudness leveling, since it avoids the need to transmit redundant dynamic range control gains in a DRC set that includes loudness leveling gains.

[0175] Differences for DRC implementation in MPEG-H 3D Audio from MPEG-D DRC As noted above, references herein to MPEG-D and MPEG-D DRC refer to the standard in ISO / IEC 23003-4, also identified as Information Technology -- MPEG Audio Technology -- Part 4: Dynamic Range Control. This standard is further incorporated into the MPEG-H 3D Audio standard defined as ISO / IEC 23008-3 (MPEG-H Part 3). The MPEG-H 3D Audio standard is an audio coding standard developed by MPEG to support the encoding of audio as audio channels, audio objects, or Higher Order Ambisonics (HOA).

[0176] The MPEG-H 3D Audio standard refers to the MPEG-D standard and includes some modifications to the syntax and technology of the MPEG-D standard in order to provide appropriate technical compatibility. In particular, the main differences regarding MPEG-D DRC in MPEG-H Audio related to the present invention include the following. · In MPEG-D DRC (and thus also in MPEG-D USAC defined as ISO / IEC 23003-3), DRC commands are defined in the dynamic loudness control profile using the drcInstructionsUniDrcV1() syntax element. In the low complexity and baseline profiles of MPEG-H 3D Audio, DRC commands are defined using the drcInstructionsUniDrc() syntax element. · MPEG-H 3D Audio provides the possibility to personalize the user experience. Thus, there are additional steps for the DRC set selection process as defined in 6.4.4.2 and 6.4.4.3 of ISO / IEC 23008-3 (MPEG-H 3D Audio). · The MPEG-D DRC decoder interface in MPEG-H 3D Audio is realized via a bitstream interface based on MPEG-H Audio Stream (MHAS) packets using the packet type of PACTYP_LOUDNESS_DRC.

[0177] Inventive Syntax Updates for MPEG-H 3D Audio Since MPEG-H 3D Audio uses the drcInstructionsUniDrc() syntax element to define the commands used for the DRC set selection process, the uniDrcConfigExtension() syntax shown in Table 26 needs to be updated as shown in the elements of the present invention shown in Table 29, particularly in the for loop for (i = 0; i < drcInstructionsUniDrcCount; i++). Such syntax may be part of the playback conditions considered in figures such as Figure 2 and Figure 6.

[0178] This additional syntax allows for mapping the DRC instruction parameters required for loudness leveling (e.g., levelingPresent and duckingOnlyDrcSetPresent) to the DRC instruction parameters required for selecting the appropriate DRC set that matches the playback conditions (including the selected personalization experience, e.g., mae_groupID and mae_groupPresetID). Furthermore, this mapping allows the decoder to select the appropriate DRC set based on the playback conditions (including the selected personalization experience) and the loudness leveling process that has been turned on or off (e.g., using levelingControlInterface()). [Table 29] TIFF2025511384000034.tif232167 TIFF2025511384000035.tif124167

[0179] Alternatively, the syntax between "case_UNIDRCCONFEXT_DYNLOUD:" and "break;" may be referred to as a separate syntax element, for example, levelingInstructions() as shown in Table 30. [Table 30]

[0180] MPEG-H 3D Audio Differences in the DRC Set Selection Process In the features and syntax defined by MPEG-H 3D Audio, there may be two or more DRC sets with ducking / leveling effects, with different maeGroupID or maePresetID values.

[0181] One inventive aspect includes taking into account newly added parameters related to loudness leveling in any syntax element (e.g., levelingPresent, duckingOnlyDrcSetPresent in the uniDrcConfigExtension() element) for the DRC set selection process as described in section 6.4.4 of MPEG-H 3D Audio to allow selection of the correct DRC set depending on the current playback conditions (including the selected personalized experience, e.g., mae_groupID and mae_groupPresetID). Selection of the correct DRC set may further depend on the state of loudness leveling (e.g., switched on or off), which may be controlled using the MPEG-D DRC Decoder Interface. The MPEG-D DRC decoder interface for MPEG-H 3D Audio may be realized through an MHAS-based bitstream interface using a packet type of PACTYP_LOUDNESS_DRC, which enables the transport of the mpegh3daLoudnessDrcInterface() element, which contains the uniDrcInterfaceExtension() element and the uniDrcInterface() element. The uniDrcInterfaceExtension() element may be updated as shown in Tables 19 through 21.

[0182] interpretation Unless otherwise indicated, and as will be apparent from the discussion that follows, discussions throughout this disclosure using the terms "processing," "computing," "calculation," "determining," "analyzing," and the like will be understood to refer to the actions and / or processes of a computer or computing system, or similar electronic computing device, that manipulates and / or transforms data represented as physical quantities, e.g., electronic quantities, into other data also represented as physical quantities.

[0183] Similarly, the term "processor" may refer to any device or portion of a device for processing electronic data, e.g., from a register and / or memory, to render the electronic data into other electronic data, which may be stored, e.g., in a register and / or memory. A "computer" or a "calculating machine" or a "computing platform" may include one or more processors.

[0184] The methods described herein, in an exemplary embodiment, are executable by one or more processors that accept computer-readable (also referred to as machine-readable) code that includes a set of instructions that, when executed by one or more of said processors, perform at least one of the methods described herein. Any processor capable of executing a set of instructions (sequential or otherwise) that specify actions to be performed is included. Thus, one example is a typical processing system that includes one or more processors. Each processor may include one or more of a CPU, a graphics processing unit, and a programmable DSP unit. The processing system may further include a memory subsystem including a main RAM and / or static RAM and / or ROM. A bus subsystem for communicating between components may be included. The processing system may further be a distributed processing system having processors coupled by a network. If the processing system requires a display, such a display may be included, for example, a liquid crystal display (LCD) or a cathode ray tube (CRT) display. If manual data entry is required, the processing system also includes one or more input devices, such as an alphanumeric input unit, such as a keyboard, a pointing control device, such as a mouse, etc. The processing system may also include a storage system, such as a disk drive unit. The processing system in some configurations may include an audio output device and a network interface device. Thus, the memory subsystem includes a computer-readable carrier medium carrying computer-readable code (e.g., software) that includes a set of instructions that, when executed by one or more processors, cause the processor to perform one or more of the methods described herein. It should be noted that when a method includes several elements, e.g., several steps, no ordering of such elements is implied unless specifically stated.The software may reside in the hard disk or, during its execution by the computer system, may reside, completely or at least partially, in the RAM and / or in the processor. Thus, the memory and the processor also constitute a computer-readable carrier medium carrying the computer-readable code. Furthermore, the computer-readable carrier medium may form or be included in a computer program product.

[0185] In alternative exemplary embodiments, the one or more processors may operate as standalone devices or, in a networked deployment, may be connected, e.g., networked, to other processors, and the one or more processors may operate in the capacity of a server or user machine in a server-user network environment, or as a peer machine in a peer-to-peer or distributed network environment. The one or more processors may form a personal computer (PC), tablet PC, personal digital assistant (PDA), cellular telephone, web appliance, network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.

[0186] It should be noted that the term "machine" is also intended to include any collection of machines that individually or jointly execute a set (or sets) of instructions for performing any one or more of the methodologies discussed herein.

[0187] Thus, one exemplary embodiment of each method described herein is in the form of a computer readable carrier medium carrying a set of instructions, e.g., a computer program for execution on one or more processors, e.g., one or more processors that are part of a web server configuration. Thus, as will be appreciated by those skilled in the art, exemplary embodiments of the present disclosure may be embodied as a method, an apparatus, such as a special purpose device, an apparatus, such as a data processing system, or a computer readable carrier medium, e.g., a computer program product. The computer readable carrier medium carries computer readable code, which when executed on one or more processors, causes the processor(s) to perform the method. Thus, aspects of the present disclosure may take the form of a method, an entirely hardware exemplary embodiment, an entirely software exemplary embodiment, or an exemplary embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a carrier medium (e.g., a computer program product on a computer readable storage medium) carrying computer readable program code embodied in the medium.

[0188] The software may also be transmitted and received over a network via a network interface device. Although the carrier medium is a single medium in an exemplary embodiment, the term "carrier medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of instructions. The term "carrier medium" should also be taken to include any medium that can store, encode, or carry a set of instructions for execution by one or more of said processors, causing said one or more processors to perform any one or more of the methods disclosed herein. Carrier media can take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical disks, magnetic disks, and magneto-optical disks. Volatile media include dynamic memory, such as main memory. Transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up a bus subsystem. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications. For example, the term "carrier medium" should be taken to include, but is not limited to, a computer product embodied in solid-state memory, optical and magnetic media; a medium carrying a propagated signal detectable by at least one processor, or one or more processors, and representing a set of instructions that when executed implements a method; and a transmission medium in a network carrying a propagated signal detectable by at least one of the one or more processors, and representing a set of instructions.

[0189] It will be appreciated that the steps of the methods discussed are, in an exemplary embodiment, performed by a suitable processor or processors of a processing (e.g., computer) system executing instructions (computer readable code) stored in a storage device. It will also be appreciated that the present disclosure is not limited to any particular implementation or programming technique, but may be implemented using any suitable technique for implementing the functions described herein. The present disclosure is not limited to any particular programming language or operating system.

[0190] Throughout this disclosure, a reference to "one embodiment," "some embodiments," or "an exemplary embodiment" means that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the disclosure. Thus, the appearances of the phrases "in one embodiment," "some embodiments," or "in an exemplary embodiment" in various places throughout this disclosure do not necessarily all refer to the same exemplary embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more exemplary embodiments, as would be apparent to one of ordinary skill in the art from this disclosure.

[0191] As used herein, unless otherwise noted, the use of the ordinal adjectives "first," "second," "third," etc. to describe a common object indicates merely that different instances of a similar object are being referred to, and is not intended to imply that the objects so described must be in a given order, temporally, spatially, in ranking, or in any other way.

[0192] In the claims and the description herein, any of the terms including, including, or having are open terms that include at least the recited elements / features but do not exclude others. Thus, the term including / having when used in the claims should not be interpreted as being limited to the recited means, elements, or steps. For example, a device having A and B should not be limited to a device consisting of elements A and B only. As used herein, any of the terms including, including, or containing are also open terms that include at least the recited elements / features but do not exclude others. Thus, including is synonymous with having and means having.

[0193] In the above description of exemplary embodiments of the present disclosure, it should be understood that various features of the present disclosure may be grouped together in a single exemplary embodiment, figure, or description thereof for the purpose of facilitating the flow of the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, this method of disclosure should not be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed exemplary embodiment. Thus, the claims following this specification are hereby expressly incorporated into this specification, with each claim standing on its own as a separate exemplary embodiment of the present disclosure.

[0194] Furthermore, while some exemplary embodiments described herein may include some features included in other exemplary embodiments but not others, combinations of features of different exemplary embodiments are intended to be within the scope of the present disclosure and make different exemplary embodiments, as will be understood by those skilled in the art. For example, in the following claims, any of the exemplary embodiments recited in the claims may be used in any combination.

[0195] In the description provided herein, numerous specific details are set forth. However, it will be understood that the exemplary embodiments of the present disclosure may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this document.

[0196] Thus, while what is believed to be the best mode of the disclosure has been described, those skilled in the art will recognize that other and further modifications may be made without departing from the spirit of the disclosure, and will recognize that it is intended to claim all such changes and modifications as falling within the scope of the present disclosure. For example, any formulas set forth above are merely representative of procedures that may be used. Functions may be added or removed from the block diagrams, and operations may be interchanged between functional blocks. Steps may be added or removed to methods described within the scope of the present disclosure.

[0197] Below, two sets of enumerated example embodiments (EEEA and EEEB) describe some structure, features, and functionality of some aspects of the example embodiments disclosed herein. [EEEA1] 1. A method for metadata-based dynamic processing of audio data for playback, the method comprising: (a) receiving, by a decoder, a bitstream containing audio data and metadata for loudness leveling; (b) decoding, by the decoder, the audio data and the metadata to obtain decoded audio data and the metadata; (c) determining, by the decoder, from the metadata, one or more processing parameters for loudness leveling based on playback conditions; (d) applying the determined one or more processing parameters to the decoded audio data to obtain processed audio data; (e) outputting the processed audio data for playback. method. [EEEA2] The method of claim EEEA1, wherein the metadata indicates processing parameters for loudness leveling for a plurality of playback conditions. [EEEA3] The method of any one of claims 1 to 2, wherein determining the one or more processing parameters further comprises determining one or more processing parameters for dynamic range compression (DRC) based on the playback conditions. [EEEA4] The method of any one of EEEA1 to 3, wherein the playback conditions include one or more of the device type of the decoder, playback device characteristics, loudspeaker characteristics, loudspeaker setup, background noise characteristics, ambient noise characteristics, and acoustic environment characteristics. [EEEA5] The method of any one of EEEA1 to 4, wherein process (c) further comprises selecting, by the decoder, at least one of a set of DRC sequences DRCSet, a set of equalizer parameters EQSet, and a downmix corresponding to the playback conditions. [EEEA6] The method of claim EEEA5, wherein process (c) further comprises identifying a metadata identifier indicating the at least one selected DRCSet, EQSet and downmix to determine the one or more processing parameters from the metadata. [EEEA7] The method of any one of EEE1 to 6, wherein the metadata includes one or more processing parameters related to average loudness values ​​and optionally one or more processing parameters related to dynamic range compression characteristics. [EEEA8] 8. The method of any one of claims EEEA1 to 7, wherein the bitstream further comprises additional metadata for static loudness adjustment to be applied to the decoded audio data. [EEEA9] 9. The method of any one of EEE1 to 8, wherein the bitstream is an MPEG-D DRC bitstream and the presence of metadata is signaled based on MPEG-D DRC bitstream syntax. [EEEA10] The method according to EEE9, wherein the uniDrcConfigExtension() element is used to carry the metadata as a payload. [EEEA11] A method according to any one of EEE1 to 10, wherein the metadata comprises one or more metadata payloads, each metadata payload comprising a plurality of sets of parameters and identifiers, each set comprising at least one of a DRCSet identifier drcSetId, an EQSet identifier eqSetId and a downmix identifier downmixId in combination with one or more processing parameters related to the identifiers in that set. [EEEA12] The method described in EEEA11 when citing EEEA5, wherein process (c) involves selecting a set from the plurality of sets in the payload based on the at least one DRCSet, EQSet and downmix selected by the decoder, and the one or more processing parameters determined in process (c) are the one or more processing parameters related to the identifier in the selected set. [EEEA13] 1. A decoder for metadata-based dynamic processing of audio data for playback, the decoder comprising: (a) receiving, by a decoder, a bitstream containing audio data and metadata for loudness leveling; (b) decoding, by the decoder, the audio data and the metadata to obtain decoded audio data and the metadata; (c) determining, by the decoder, from the metadata, one or more processing parameters for loudness leveling based on playback conditions; (d) applying the determined one or more processing parameters to the decoded audio data to obtain processed audio data; (e) outputting the processed audio data for playback. 16. A decoder having one or more processors and a non-transitory memory configured to perform a method including: [EEEA14] 1. A method for encoding audio data and metadata for loudness leveling into a bitstream, the method comprising: (a) inputting original audio data to a loudness leveller for loudness processing, and obtaining loudness processed audio data as output from the loudness leveller; (b) generating metadata for loudness leveling based on the loudness processed audio data and the original audio data; (c) encoding the original audio data and the metadata into the bitstream; method. [EEEA15] The method of claim EEEA14, further comprising generating additional metadata for static loudness adjustment for use by a decoder. [EEEA16] 16. The method according to claim EEEA14 or 15, wherein process (b) comprises comparing the loudness processed audio data with original audio data, and the metadata is generated based on a result of the comparison. [EEEA17] 6. The method of claim 16, wherein process (b) further comprises measuring loudness over one or more predefined time periods, and wherein the metadata is generated further based on the measured loudness. [EEEA18] The method of claim 17, wherein said measuring includes measuring an overall loudness of the audio data. [EEEA19] The method of claim 17, wherein said measuring includes measuring loudness of dialogue in said audio data. [EEEA20] 20. The method of any one of claims EEEA14 to 19, wherein the bitstream is an MPEG-D DRC bitstream and the presence of the metadata is signaled based on MPEG-D DRC bitstream syntax. [EEEA21] The method of claim 20, wherein a uniDrcConfigExtension() element is used to carry the metadata as a payload. [EEEA22] 22. The method of any one of claims EEEA14 to 21, wherein the metadata comprises one or more metadata payloads, each metadata payload comprising a plurality of sets of parameters and identifiers, each set comprising at least one of a DRCSet identifier drcSetId, an EQSet identifier eqSetId and a downmix identifier downmixId in combination with one or more processing parameters related to the identifiers in that set, the one or more processing parameters being parameters for loudness levelling by a decoder. [EEEA23] The method of claim EEEA22, wherein the at least one of the drcSetId, the eqSetId and the downmixId relates to at least one of a set of DRC sequences DRCSet, a set of equalizer parameters EQSet and a downmix to be selected by the decoder. [EEEA24] 1. An encoder for encoding original audio data and metadata for loudness leveling in a bitstream, the encoder comprising: (a) inputting original audio data to a loudness leveller for loudness processing, and obtaining loudness processed audio data as output from the loudness leveller; (b) generating said metadata for loudness leveling based on said loudness processed audio data and original audio data; (c) encoding the original audio data and said metadata into said bitstream; 16. An encoder comprising: one or more processors and a non-transitory memory configured to perform a method including: [EEEA25] A system comprising an encoder for encoding in a bitstream original audio data and metadata for loudness leveling and / or dynamic range compression DRC as described in EEEA24 and a decoder for metadata-based dynamic processing of audio data for playback as described in EEEA13. [EEEA26] A computer program product comprising a computer readable storage medium having instructions adapted, when executed by a device having processing capability, to cause the device to perform a method according to any one of EEEA1 to 12 or EEEA14 to 23. [EEEA27] A computer readable storage medium storing a computer program product according to EEEA26. [EEEA28] A method according to any one of claims EEEA1 to 12, further comprising: receiving, by the decoder, via an interface, an indication of whether or not to perform the metadata based dynamic processing of audio data for playback; and when the decoder receives an indication not to perform the metadata based dynamic processing of audio data for playback, bypassing at least the step of applying the determined one or more processing parameters to the decoded audio data. [EEEA29] The method of claim EEEA28, wherein the decoder at least bypasses the step of applying the determined one or more processing parameters to the decoded audio data until the decoder receives, via the interface, the indication of whether or not to perform the metadata-based dynamic processing of audio data for playback. [EEEA30] 30. The method of any one of claims EEEA1 to 12, EEEA28 or 29, wherein the metadata indicates processing parameters for loudness leveling for a plurality of playback conditions, and the metadata further includes parameters specifying a loudness measurement method used to derive one of the plurality of processing parameters. [EEEA31] 21. The method of claim 1, wherein the metadata indicates processing parameters for loudness leveling for a plurality of playback conditions, and the metadata further includes a parameter specifying a loudness measurement system used to measure one of the plurality of processing parameters. [EEEB1] 1. A method for metadata-based dynamic processing of audio data for playback, the method comprising: receiving, by a decoder, a bitstream comprising audio data and metadata for dynamic loudness adjustment; decoding, by the decoder, the audio data and the metadata to obtain decoded audio data and the metadata; selecting, by the decoder, at least one set of decoder settings corresponding to playback conditions, and determining from the metadata one or more processing parameters for dynamic loudness adjustment based on the at least one selected set; applying the determined one or more processing parameters to the decoded audio data to obtain processed audio data; outputting the processed audio data for playback. A method comprising: [EEEB2] The method of claim EEEB1, wherein the metadata indicates processing parameters for dynamic loudness adjustment for a plurality of playback conditions. [EEEB3] The method according to claim EEEB1 or 2, wherein determining the one or more processing parameters further comprises determining one or more processing parameters for dynamic range compression (DRC) based on the playback state. [EEEB4] The method according to any one of EEEB1 to 3, wherein the playback conditions include one or more of the device type of the decoder, playback device characteristics, loudspeaker characteristics, loudspeaker setup, background noise characteristics, ambient noise characteristics, and acoustic environment characteristics. [EEEB5] The method according to any one of claims EEEB1 to 4, wherein the at least one set of decoder settings comprises a set of DRC sequences DRCSet, a set of equalizer parameters EQSet and a downmix. [EEEB6] The method according to EEEB5, wherein determining the one or more processing parameters further comprises identifying metadata identifiers indicative of the at least one selected DRCSet, EQSet, and downmix to determine the one or more processing parameters from the metadata. [EEEB7] The method of any one of EEEB1 to 6, wherein the metadata includes one or more processing parameters related to an average loudness value and optionally one or more processing parameters related to dynamic range compression characteristics. [EEEB8] 8. The method of any one of claims EEEB1 to 7, wherein the bitstream further comprises additional metadata for static loudness adjustment applied to the decoded audio data. [EEEB9] 9. The method of any one of claims EEEB1 to 8, wherein the bitstream is an MPEG-D DRC bitstream and the presence of metadata is signaled based on MPEG-D DRC bitstream syntax. [EEEB10] The method of claim EEEB9, wherein a loudnessInfoSetExtension() element is used to carry the metadata as a payload. [EEEB11] 11. The method of any one of claims EEEB1 to 10, wherein the metadata includes one or more metadata payloads, each metadata payload including a plurality of sets of parameters and identifiers, each set including at least one of a DRCSet identifier drcSetId, an EQSet identifier eqSetId, and a downmix identifier downmixId in combination with one or more processing parameters associated with the identifiers in that set. [EEEB12] The method according to EEEB11 when citing EEEB5, wherein determining the one or more processing parameters involves selecting a set from among the plurality of sets in the payload based on the at least one DRCSet, EQSet, and a downmix selected by the decoder, and the one or more processing parameters determined by the decoder are the one or more processing parameters related to the identifier in the selected set. [EEEB13] 1. A decoder for metadata based dynamic processing of audio data for playback, the decoder comprising: receiving, by the decoder, a bitstream comprising audio data and metadata for dynamic loudness adjustment; decoding, by the decoder, the audio data and the metadata to obtain decoded audio data and the metadata; selecting, by the decoder, at least one set of decoder settings corresponding to playback conditions, and determining from the metadata one or more processing parameters for dynamic loudness adjustment based on the at least one selected set; applying the determined one or more processing parameters to the decoded audio data to obtain processed audio data; outputting the processed audio data for playback. having one or more processors and non-transitory memory configured to execute a method comprising: decoder.

Claims

1. 1. A method of processing audio data for playback, the method comprising: receiving, by a decoder, a bitstream comprising encoded audio data and metadata, the metadata comprising one or more dynamic range control (DRC) sets, and for each DRC set, an indication of whether the DRC set is configured to provide a loudness leveling effect, the metadata further comprising personalization metadata; parsing, by the decoder, the metadata to identify a DRC set configured to provide the loudness leveling effect, the DRC set being identified based at least in part on the personalization metadata; decoding, by the decoder, the encoded audio data to obtain decoded audio data; selecting, by the decoder, one of the identified DRC sets configured to provide the loudness leveling effect; extracting, by the decoder, one or more DRC gains from the bitstream that correspond to the selected DRC set; applying, by the decoder, the one or more DRC gains corresponding to the selected DRC set to the decoded audio data to obtain dynamic loudness compensated audio data; and outputting the dynamic loudness compensated audio data for playback. method.

2. The method of claim 1 , wherein the bitstream is a bitstream compatible with the MPEG-H 3D Audio standard.

3. The method of claim 2 , wherein the personalization metadata includes mae_groupID and maegroupPresetID syntax and semantics as described in the MPEG-H 3D Audio standard.

4. 4. The method of claim 1, wherein the metadata comprises a plurality of DRC sets configured to provide the loudness leveling effect, each of the plurality of DRC sets also being associated with one or more playback conditions, and the selection is performed in response to an indication of a playback condition provided to the decoder.

5. 5. The method of claim 1, wherein in addition to providing the loudness leveling effect, the one or more DRC sets are also configured to provide dynamic range control.

6. 5. The method of claim 1, wherein the playback conditions include one or more of the following: device type of the decoder, playback device characteristics, loudspeaker characteristics, loudspeaker setup, background noise characteristics, ambient noise characteristics, capabilities of the device type, a personalized experience selected on the device, and characteristics of the acoustic environment.

7. The method of claim 6 , wherein the personalized experience selected by a user or device is based on user preferences, e.g., language, user experience, previous listening choices, and / or device capabilities.

8. 8. The method of claim 1, wherein the indication of whether the DRC set is configured to provide the loudness leveling effect is provided in a parameter indicative of one or more effects provided by the DRC set.

9. 9. The method of claim 8, wherein the parameter indicating one or more effects provided by the DRC set is a drcSetEffect bitfield of an MPEG-D DRC bitstream, each bit of the drcSetEffect bitfield corresponding to a different effect, and one of the bits of the drcSetEffect bitfield corresponds to the loudness leveling effect.

10. 10. The method of claim 1, wherein the indication of whether the DRC set is configured to provide the loudness leveling effect is whether the DRC set is specified in a loudness leveling bitstream payload.

11. The method of claim 10, wherein the loudness leveling bitstream payload is included in an extension field of a previously defined bitstream syntax.

12. 12. The method of claim 11, wherein the extension field is a uniDrcConfigExtension field of an MPEG-H 3D audio bitstream and the loudness leveling bitstream payload is included only for certain values ​​of a uniDrcConfigExtType parameter.

13. 13. The method of claim 11 or 12, wherein a plurality of loudness leveling payloads specifying a plurality of DRC sets configured to provide the loudness leveling effect are included in the extension field of the previously defined bitstream syntax.

14. 14. The method of claim 1, wherein the indication of whether the DRC set is configured to provide the loudness leveling effect is a field of a pre-existing component of a previously defined bitstream syntax.

15. 15. The method of claim 1, wherein the indication of whether the DRC set is configured to provide the loudness leveling effect is a field in an updated version of a pre-existing component of a previously defined bitstream syntax.

16. 16. The method of claim 1, wherein an indication that a loudness leveling effect is desired is provided to the decoder through an interface, and the DRC set is selected in response to the indication provided to the decoder through the interface.

17. The method of claim 16 , wherein the interface receives the indication from an MHAS compatible syntax.

18. 20. The method of claim 17, wherein an indication of an additional desired effect is provided to the decoder through the interface, and the metadata includes a plurality of DRC sets configured to provide the loudness leveling effect, and the selection is dependent on the additional desired effect.

19. 19. The method of any one of claims 16 to 18, wherein the indication that a loudness leveling effect is desired is provided through a loudnessLevelingOn parameter in a levelingControlInterface payload.

20. 20. The method of claim 1, wherein the metadata comprises one or more static loudness values ​​configured to provide a static loudness adjustment to the decoded audio data.

21. 21. The method of claim 20, comprising applying a static loudness adjustment to the decoded audio data or the dynamic loudness compensated audio data in response to one or more of the static loudness values.

22. 2. The method of claim 1, wherein a first one of the DRC sets is configured to provide dynamic range control, and the first DRC set includes an indication that the selected DRC set configured to provide the loudness leveling effect is configured to be applied in combination with the first DRC set.

23. 23. The method of claim 22, wherein the selected DRC set includes an indication of whether the one or more DRC gains corresponding to the selected DRC set may only be applied in combination with a DRC gain corresponding to the first DRC set.

24. 24. The method of claim 22 or 23, further comprising: extracting the DRC gains corresponding to the first DRC set from the bitstream; and applying the DRC gains corresponding to the first DRC set to the decoded audio data.

25. 25. A decoder for processing audio data for reproduction, comprising one or more processors and a non-transitory memory configured to perform a method according to any one of claims 1 to 24.

26. A computer program product comprising a computer readable storage medium having instructions adapted to cause a device to perform the method of any one of claims 1 to 24.

27. 27. A computer readable storage medium storing the computer program product of claim 26.

Images