Loudness adjustment for downmixing audio content

Abstract

The present disclosure relates to loudness adjustment for downmixing audio content. Audio content encoded for a reference speaker configuration is downmixed to downmixed audio content encoded for a particular speaker configuration. One or more gain adjustments are performed on individual portions of the downmixed audio content encoded for the particular speaker configuration. Loudness measurements are then performed on the individual portions of the downmixed audio content. Audio content is produced that includes the audio content encoded for the reference speaker configuration and downmix loudness metadata. The downmix loudness metadata is created based at least in part on the loudness measurements on the individual portions of the downmixed audio content.

Description

[0001] This application is a divisional application of patent application No. 202310944485.9, filed on September 9, 2014, entitled "Loudness Adjustment for Downmixed Audio Content". Patent application No. 202310944485.9 is a divisional application of patent application No. 201911020119.4, filed on September 9, 2014, entitled "Loudness Adjustment for Downmixed Audio Content". Patent application No. 201911020119.4 is a divisional application of patent application No. 201480050050.9, filed on September 9, 2014, entitled "Loudness Adjustment for Downmixed Audio Content".

[0002] (Cross-reference to related applications)

[0003] This application claims priority to U.S. Provisional Patent Application No. 61 / 877230, filed September 12, 2013; U.S. Provisional Patent Application No. 61 / 891324, filed October 15, 2013; U.S. Provisional Patent Application No. 61 / 938043, filed February 10, 2014; and U.S. Provisional Patent Application No. 61 / 892313, filed October 17, 2013, the entire contents of which are incorporated herein by reference. Technical Field

[0004] The present invention relates generally to processing audio signals, and more particularly to techniques that can be used to apply dynamic range control and other types of audio processing actions to audio signals in any of a variety of playback environments. Background Technology

[0005] The increasing prevalence of media consumer devices presents new opportunities and challenges for creators and distributors of media content played back on these devices, as well as for device designers and manufacturers. Many consumer devices are capable of playing back a wide range of media content types and formats, including those often associated with high-quality, wide-bandwidth, and wide-dynamic-range audio content used in HDTV, Blu-ray, or DVD. Media processing devices can be used to play back this type of audio content arbitrarily on their own internal acoustic transducers or on external transducers such as headphones; however, they generally cannot reproduce the content with consistent loudness and intelligibility across various media formats and content types.

[0006] The methods described in this section are traceable methods, but not necessarily previously conceived or traced methods. Therefore, unless otherwise indicated, none of the methods described in this section should be assumed to be prior art simply by virtue of their inclusion in this section. Similarly, unless otherwise indicated, questions concerning the identification of one or more methods should not be presumed to have been identified in any prior art based on this section. Attached Figure Description

[0007] The invention is illustrated in the accompanying drawings by way of example and not limitation, and in these drawings, similar reference numerals refer to similar elements, wherein, Figure 1A and Figure 1B An exemplary audio decoder and an exemplary audio encoder are shown respectively; Figure 2A and Figure 2B An exemplary dynamic range compression curve is shown; Figure 3 An exemplary processing logic for determining / calculating the combined DRC and limiting gain is shown; Figure 4 An example differential encoding of the gain is shown; Figure 5 An exemplary codec system including an audio encoder and an audio decoder is shown; Figure 6A 6D illustrates an exemplary processing flow; and Figure 7 An exemplary hardware platform is shown that can implement the computer or computing device described herein. Detailed Implementation

[0008] This description relates to exemplary embodiments of applying dynamic range control processing and other types of audio processing actions to audio signals in various playback environments. In the following description, numerous detailed descriptions are set forth for purposes of explanation and to enable a thorough understanding of the invention. However, it will be apparent that the invention can be practiced without these specific details. In other instances, well-known structures and apparatuses are not described in exhaustive detail to avoid unnecessarily obscuring, concealing, or obscuring the invention.

[0009] An exemplary embodiment is described here based on the following outline: 1. General Overview 2. Dynamic range control 3. Audio decoder 4. Audio encoder 5. Dynamic range compression curve 6. DRC gain, gain limiting, and gain smoothing 7. Input smoothing and gain smoothing 8. DRC on multiple frequency bands 9. Volume adjustment in the loudness range 10. Lower Mix Loudness Adjustment 11. Additional actions related to gain 12. Specific and wideband (or broadband) loudness levels 13. Individual gain for each individual subset of the channel 14. Auditory Scene Analysis 15. Loudness level transition 16. Reset 17. Gain provided by the encoder 18. Exemplary systems and processes 19. Implementation Mechanism - Hardware Overview 20. Equivalents, extensions, substitutions, and miscellaneous items 1. General Overview This summary provides a basic description of some aspects of embodiments of the present invention. It should be noted that this summary is not an extensive or exhaustive summary of all aspects of the embodiments. Furthermore, it should be understood that this summary should not be construed as identifying any particularly important aspects or elements of the embodiments, nor as specifically defining any scope of the embodiments, nor as generally defining the invention. This summary only presents some concepts related to the exemplary embodiments in a general or simplified form and should be understood as a conceptual preamble to the more detailed description of the exemplary embodiments given below. Note that although individual embodiments are discussed herein, any combination of the embodiments and / or portions of the embodiments discussed herein can be combined to form other embodiments.

[0010] In some methods, the encoder assumes that the audio content is encoded for a specific environment for the purpose of dynamic range control, and determines audio processing parameters such as gain for dynamic range control for that specific environment. The gain determined by the encoder according to these methods is generally smoothed over time intervals by some time constant (e.g., in a function with exponential decay). Additionally, gain limits, to ensure that the loudness level does not exceed the trimming level of the assumed environment, may be added to the gain determined by the encoder according to these methods. Therefore, the gain encoded into an audio signal by the encoder from audio information according to these methods is the result of many different effects and is irreversible. A decoder receiving the gain according to these methods may not be able to distinguish which part of the gain is used for dynamic range control, which part for gain smoothing, which part for gain limiting, and so on.

[0011] According to the techniques described herein, the audio encoder does not assume that it only needs to support the specific playback environment at the audio decoder. In embodiments, the audio encoder transmits an encoded audio signal with audio content from which the correct loudness level (e.g., without trimming) can be determined. The audio encoder also transmits one or more dynamic range compression profiles to the audio decoder. Any of the one or more dynamic range compression profiles can be standard-based, proprietary, custom, content provider-specific, etc. Reference loudness level, attack time, release time, etc., can be transmitted by the audio encoder as part of or in combination with one or more dynamic range compression profiles. Any of the reference loudness level, attack time, release time, etc., can be standard-based, proprietary, custom, content provider-specific, etc.

[0012] In some embodiments, the audio encoder implements Auditory Scene Analysis (ASA) technology and uses ASA technology to detect auditory events in the audio content, and transmits one or more ASA parameters describing the detected auditory events to the audio decoder.

[0013] In some embodiments, the audio encoder may also be configured to detect a reset event in the audio content and transmit an indication of the reset event to a downstream device, such as an audio decoder, in a manner synchronized with the time of the audio content.

[0014] In some embodiments, an audio encoder may be configured to compute one or more sets of gains (e.g., DRC gains, etc.) for individual portions of audio content (e.g., audio data blocks, audio data frames, etc.), and to encode the multiple sets of gains into an encoded audio signal using the individual portions of the audio content. In some embodiments, the multiple sets of gains generated by the audio encoder correspond to one or more different gain profiles. In some embodiments, Huffman coding, differential coding, etc., may be used to encode the multiple sets of gains into components, branches, etc., of audio data frames, or to read the multiple sets of gains from them. These components, branches, etc., may be referred to as subframes in the audio data frame. Different sets of gains may correspond to different sets of subframes. Each set of gains or each set of subframes may contain two or more time components (e.g., subframes, etc.). In some embodiments, the bitstream formatter in the audio encoder described herein may use one or more for loops to write a set or more sets of gains together as differential data codes into a set or more sets of subframes in an audio data frame; correspondingly, the bitstream parser in the audio decoder described herein may read any one of the set or more sets of gains encoded as differential data codes from a set or more sets of subframes in an audio data frame.

[0015] In some embodiments, the audio encoder determines the dialogue loudness level in the audio content to be encoded into an encoded audio signal, and transmits the dialogue loudness level to the audio encoder using the audio content.

[0016] In some embodiments, audio content is encoded in the coded audio signal for a reference speaker configuration (surround sound configuration, 5.1 speaker configuration, etc.) that includes more audio channels or speakers than those that include a large number of audio decoders (e.g., mobile phones, tablets, etc.) (e.g., two-channel headphone configurations, etc.). Even with the same gain adjustment in both speaker configurations, the loudness level measured for each individual portion of the audio content in the reference speaker configuration may differ from the loudness level measured in a specific speaker configuration such as a two-channel configuration.

[0017] In some embodiments, the audio encoder described herein is configured to provide downmixing-related metadata (e.g., containing one or more downmixing loudness parameters, etc.) to a downstream audio decoder. For the purpose of producing a relatively accurate target loudness level in the downmixed sound output, the downstream audio decoder can use the downmixing-related metadata from the audio encoder (150) to efficiently and consistently perform additional downmixing-related gain adjustment actions (real-time, near real-time, etc.). The downstream audio decoder can use the additional downmixing-related gain adjustment actions to prevent inconsistencies in the measured loudness levels between a reference speaker configuration and a specific speaker configuration of the decoder.

[0018] When assuming a hypothetical playback environment, situation, etc., at a hypothetical audio decoder, the techniques described herein do not require the audio decoder to be locked (e.g., irreversible, etc.) in audio processing performed by an upstream device such as an audio encoder. For example, in order to distinguish different loudness levels existing in audio content, minimize the loss of audio perceived quality at or near boundary loudness levels (e.g., minimum or maximum loudness levels, etc.), maintain spatial balance between channels or subsets of channels, etc., the decoder described herein can be configured to customize audio processing actions based on specific playback situations.

[0019] An audio decoder that receives an encoded audio signal with a dynamic range compression curve, a reference loudness level, an onset time, a release time, etc., can determine the specific playback environment to be used at the decoder and select a specific compression curve with a corresponding reference loudness level that corresponds to the specific playback environment.

[0020] The decoder can calculate / determine the loudness level in individual portions (e.g., audio data blocks, audio data frames, etc.) of the audio content extracted from the encoded audio signal, or obtain the loudness level in individual portions of the audio content if the audio encoder calculates and provides the loudness level in the encoded audio signal. Based on one or more of the loudness levels in individual portions of the audio content, the loudness level in the preceding portions of the audio content, the loudness level in the subsequent portions of the audio content where available, a specific compression curve, a specific profile related to a specific playback environment or situation, the decoder determines audio processing parameters, such as gain (or DRC gain) for dynamic range control, attack time, release time, etc. Audio processing parameters may also include adjustments for aligning the dialogue loudness level with a specific reference loudness level (which may be user-adjustable) for a specific playback environment.

[0021] The decoder applies audio processing actions, including (e.g., multi-channel, multi-band, etc.) dynamic range control, dialogue level adjustment, etc., based on audio processing parameters. Audio processing actions performed by the decoder may also include, but are not limited to, gain smoothing based on the attack and release times provided as part of or in conjunction with a selected dynamic range compression curve, gain limiting to prevent pruning, etc. Different audio processing actions can be performed with different (e.g., adjustable, threshold-dependent, controllable, etc.) time constants. For example, gain limiting to prevent pruning can be applied to individual audio data blocks, individual audio data frames, etc., with relatively short time constants (e.g., instantaneous, approximately 5.3 milliseconds, etc.).

[0022] In some embodiments, the decoder may be configured to extract ASA parameters (e.g., the temporal location of auditory event boundaries, the time dependence value of event confidence measurements, etc.) from metadata in the encoded audio signal and control the speed of gain smoothing in auditory events based on the extracted ASA parameters (e.g., using a short time constant for onset at the auditory event boundary, using a long time constant to slow down gain smoothing within the auditory event, etc.).

[0023] In some embodiments, the decoder also maintains a histogram of instantaneous loudness levels for a given time interval or window, and, for example, uses the histogram to control the rate of gain change for loudness level transitions between programs, between programs and businesses, etc., by modifying the time constant.

[0024] In some embodiments, the decoder supports more than one speaker configuration (e.g., portable mode with speakers, portable mode with headphones, stereo mode, multi-channel mode, etc.). The decoder can be configured, for example, to maintain the same loudness level between two different speaker configurations (e.g., between stereo mode and multi-channel mode, etc.) when playing back the same audio content. The audio decoder can use one or more downmixers to downmix the multi-channel audio content received from the encoded audio signal to a reference speaker configuration for a specific speaker configuration at the audio decoder.

[0025] In some embodiments, automatic gain control (AGC) may be disabled in the audio decoder described herein.

[0026] In some embodiments, the mechanisms described herein form part of a media processing system, including but not limited to: audiovisual devices, flat-panel TVs, handheld devices, game consoles, televisions, home theater systems, tablets, mobile devices, laptop computers, notebook computers, cellular radios, e-book readers, point-of-sale terminals, desktop computers, computer workstations, computer kiosks, various other types of terminals and media processing units, etc.

[0027] Various modifications of the preferred embodiments and general principles and features described herein will be readily understood by those skilled in the art. Therefore, the disclosure is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein.

[0028] 2. Dynamic range control

[0029] Without customized dynamic range control, since the specific playback environment of the playback device may differ from the target playback environment where the encoded audio content has been encoded at the encoding device, the input audio information (e.g., PCM sampling, time-frequency sampling in a QMF matrix, etc.) is often reproduced at the playback device at a loudness level that is unsuitable for the specific playback environment of the playback device (e.g., including physical and / or mechanical playback limitations of the device).

[0030] The techniques described herein can be used to support dynamic range control of various customized audio content in a variety of playback environments, while maintaining the perceived quality of the audio content.

[0031] Dynamic range control (DRC) refers to a time-dependent audio processing action that modifies (e.g., compresses, cuts, expands, elevates, etc.) the input dynamic range of loudness levels in audio content to an output dynamic range different from the input dynamic range. For example, in a DRC scheme, soft sounds can be mapped (e.g., elevates, etc.) to a higher loudness level, and loud sounds can be mapped (e.g., cuts, etc.) to a lower loudness value. As a result, in the loudness domain, the output range of loudness levels becomes smaller than the input range of loudness levels in this example. However, in some embodiments, dynamic range control may be reversible, allowing the original range to be restored. For example, an expansion action can be performed to restore the original range, provided that the mapped loudness level in the output dynamic range mapped from the original loudness level is at or below the trimmed level, each unique original loudness level is mapped to a unique output loudness level, and so on.

[0032] The DRC technology described herein can be used to provide a better listening experience in certain playback environments or situations. For example, soft sounds in a noisy environment may be masked by noise, making them inaudible. Conversely, in some cases, loud sounds may not be desirable to avoid disturbing neighbors. Many devices with speakers that generally have small shape factors cannot reproduce sound at high output levels. In some cases, low signal levels may be reproduced below the human hearing threshold. DRC technology performs a mapping from input loudness level to output loudness level based on the DRC gain viewed using a dynamic range compression curve (e.g., scaling factors for scaling audio amplitude, boost ratio, cut-off ratio, etc.).

[0033] A dynamic range compression curve is a function (e.g., a lookup table, curve, multi-segment line, etc.) that maps individual input loudness levels (e.g., sounds other than dialogue) determined from individual audio data frames to individual gains or gains used for dynamic range control. Each of the individual gains indicates the amount of gain applied to the corresponding individual input loudness level. The output loudness level after applying the individual gains represents the target loudness level of the audio content in each individual audio data frame within a specific playback environment.

[0034] In addition to specifying the mapping between gain and loudness levels, the dynamic range compression curve may include or have specific release and attack times when the gain is applied. Attack refers to the increase in signal energy (or loudness) between consecutive time samples, while release refers to the decrease in energy (or loudness) between consecutive time samples. Attack time (e.g., 10 ms, 20 ms, etc.) refers to the time constant used in smoothing the DRC gain when the corresponding signal is in attack mode. Release time (e.g., 80 ms, 100 ms, etc.) refers to the time constant used in smoothing the DRC gain when the corresponding signal is in release mode. In some embodiments, additionally, optionally, or alternatively, a time constant is used for smoothing the signal energy (or loudness) before determining the DRC gain.

[0035] Different dynamic range compression profiles can correspond to different playback environments. For example, the dynamic range compression profile for a playback environment of a flat-panel TV may differ from the dynamic range compression profile for a playback environment of a portable device. In some embodiments, the playback device may have two or more playback environments. For example, the first dynamic range compression profile of a first playback environment of a portable device with a speaker may differ from the second dynamic range compression profile of a second playback environment of the same portable device with headphones.

[0036] 3. Audio decoder

[0037] Figure 1A An exemplary audio decoder 100 is shown, including a data extractor 104, a dynamic range controller 106, an audio renderer 108, and the like.

[0038] In some embodiments, the data extractor (104) is configured to receive an encoded input signal 102. The encoded input signal described herein may be a bitstream containing encoded (e.g., compressed, etc.) input audio data frames and metadata. The data extractor (104) is configured to extract / decode the input audio data frames and metadata from the encoded input signal (102). Each of the input audio data frames contains multiple encoded audio data blocks, each representing multiple audio samples. Each frame represents a (e.g., constant) time interval containing a certain number of audio samples. The frame size may vary with the sampling rate and the encoded data rate. The audio samples may be quantized audio data elements (e.g., input PCM samples, input time-frequency samples in a QMF matrix, etc.) representing spectral content in one, two, or more (audio) bands or frequency ranges. The quantized audio data elements in the input audio data frames may represent pressure waves in the digital (quantization) domain. The quantized audio data elements may cover loudness levels at or below their maximum possible value (e.g., trimming level, maximum loudness level, etc.).

[0039] Metadata can be used by various receptor decoders to process input audio data frames. Metadata may include various action parameters related to one or more actions performed by the decoder (100), one or more dynamic range compression curves, normalization parameters related to the dialogue loudness level represented in the input audio data frame, etc. Dialogue loudness level may refer to the (psychoacoustic, sensory, etc.) level of the entire program (e.g., a movie, TV program, radio broadcast, etc.), a part of the program, the dialogue loudness in the dialogue of the program, program loudness, average dialogue loudness, etc.

[0040] The actions and functions of some or all of the decoder (104) or modules (e.g., data extractor 104, dynamic range controller 106, etc.) can be adaptively adjusted in response to metadata extracted from the encoded input signal (102). For example, metadata—including but not limited to dynamic range compression curves, dialogue loudness levels, etc.—can be used by the decoder (100) to generate output audio data elements in the digital domain (e.g., output PCM samples, output time-frequency samples in a QMF matrix, etc.). The output data elements can then be used to drive audio channels or speakers to achieve a specified loudness or reference reproduction level during playback in a particular playback environment.

[0041] In some embodiments, the dynamic range controller (106) is configured to receive some or all of the audio data elements in the input audio data frame and metadata, and at least in part perform audio processing actions (e.g., dynamic range control actions, gain smoothing actions, gain limiting actions, etc.) on the audio data elements in the input audio data frame based on the metadata extracted from the encoded audio signal (102).

[0042] In some embodiments, the dynamic range controller (106) may include a selector 110, a loudness calculator 112, a DRC gain unit 114, etc. The selector (110) may be configured to determine a speaker configuration (e.g., planar mode, portable device with speakers, portable device with headphones, 5.1 speaker configuration, 7.1 speaker configuration, etc.) in relation to a particular playback environment at the decoder (100), select a specific dynamic range compression curve from a dynamic range compression curve extracted from the encoded input signal (102), etc.

[0043] The loudness calculator (112) can be configured to calculate one or more types of loudness levels represented by audio data elements in the input audio data frame. Examples of loudness level types include, but are not limited to: individual loudness levels in individual channels in individual time intervals, wideband (or broadband) loudness levels over a wide (or broad) frequency range in individual channels, loudness levels determined from or smoothed over audio data blocks or frames, loudness levels determined from or smoothed over more than one audio data block or frame, loudness levels smoothed over one or more time intervals, etc. Zero, one, or more of these loudness levels can be modified for the purpose of dynamic range control via the decoder (100).

[0044] To determine loudness levels, a loudness calculator (112) can determine one or more time-dependent physical acoustic properties, such as spatial pressure levels at specific audio frequencies, represented by audio data elements in an input audio data frame. The loudness calculator (112) can use one or more time-varying physical acoustic properties to derive one or more types of loudness levels based on one or more psychoacoustic functions modeled human loudness perception. The psychoacoustic functions can be nonlinear functions constructed based on models of the human auditory system that transform / map a specific spatial pressure level at a specific audio frequency to a specific loudness at that specific audio frequency.

[0045] Loudness levels at multiple (audio) frequencies or frequency bands (e.g., wideband, broadband, etc.) can be derived by integrating specific loudness levels at multiple (audio) frequencies or frequency bands. Loudness levels at one or more time intervals (e.g., longer than the time intervals represented by audio data elements in audio data blocks or frames) can be obtained by using one or more smoothing filters implemented as part of audio processing actions in the decoder (100).

[0046] In an exemplary embodiment, a specific loudness level for a different frequency band can be calculated for each audio data block sampled (e.g., 256, etc.). A pre-filter can be used to apply frequency weighting (e.g., similar to IEC B weighting) to a specific loudness level when integrating the specific loudness level into a wideband (or broadband) loudness level. Wide loudness levels can be added across two or more channels (e.g., front left, front right, center, left surround, right surround, etc.) to provide a total loudness level across two or more channels.

[0047] In some embodiments, the total loudness level may refer to the wideband (broadband) loudness level in a single channel (e.g., center, etc.) of a speaker configuration. In some embodiments, the total loudness level may refer to the wideband (or broadband) loudness level across multiple channels. The multiple channels may be all channels in the speaker configuration. Additionally, optionally, or alternatively, the multiple channels may include a subset of channels in the speaker configuration (e.g., a subset of channels containing left front, right front, and low-frequency effects (LFE), a subset of channels containing left surround and right surround, etc.).

[0048] Loudness levels (e.g., wideband, broadband, general, specific, etc.) can be used as input to find the corresponding (e.g., static, pre-smoothed, pre-limited, etc.) DRC gain from the selected dynamic range compression curve. The loudness level used as input to find the DRC gain can first be adjusted or normalized with respect to the dialogue loudness level from the metadata extracted from the encoded audio signal (102).

[0049] In some embodiments, the DRC gain unit (114) may be equipped with a DRC algorithm to generate gains (e.g., for dynamic range control, for gain limiting, for gain smoothing, etc.), apply gains to one or more loudness levels of one or more types of loudness levels represented by audio data elements in the input audio data frame to achieve a target loudness level for a particular playback environment, etc. The application of gains described herein (e.g., DRC gains, etc.) may, but does not need to, occur in the loudness domain. In some embodiments, gains may be generated based on direct smoothing of the input signal and the applied loudness calculation (which may be a sone or simply an SPL value for dialogue loudness level compensation, e.g., without conversion). In some embodiments, the techniques described herein may apply a gain to the signal in the loudness domain and then convert the signal from the loudness domain back to the (linear) SPL domain and calculate the corresponding gain to be applied to the signal by evaluating the signal before and after the gain is applied to the signal in the loudness domain. The ratio (or the difference represented in logarithmic dB representation) is then used to determine the corresponding gain of the signal.

[0050] In some embodiments, the DRC algorithm is operated on multiple DRC parameters. The DRC parameters contain dialogue loudness levels calculated by an upstream encoder (e.g., 150, etc.) and embedded in the encoded audio signal (102), and can be obtained from metadata in the encoded audio signal (102) by the decoder (100). The dialogue loudness level from the upstream encoder indicates the average dialogue loudness level (e.g., per program, energy relative to a full-scale 1kHz sine wave, energy relative to a reference rectangular wave, etc.). In some embodiments, the dialogue loudness level extracted from the encoded audio signal (102) can be used to reduce inter-program loudness level differences. In embodiments, the reference dialogue loudness level can be set to the same value between different programs in the same particular playback environment at the decoder (100). Based on the dialogue loudness level from metadata, the DRC gain unit (114) can apply dialogue loudness-related gain to each audio data block in the program, such that the averaged output dialogue loudness level across multiple audio data blocks of the program rises / falls to the program's (e.g., pre-configured, system default, user-configurable, profile-dependent, etc.) baseline dialogue loudness level.

[0051] In some embodiments, DRC gains can be used to address intra-procedural loudness level differences by increasing or cutting the input loudness level in soft and / or loud sounds according to a selected dynamic range compression curve. One or more of these DRC gains can be calculated / determined via a DRC algorithm based on a selected dynamic range compression curve and loudness levels (e.g., wideband, broadband, overall, specific, etc.) determined from one or more corresponding audio data blocks, audio data frames, etc.

[0052] The loudness level used to determine the DRC gain (stationary, pre-smoothed, pre-gain limited, etc.) by looking up a selected dynamic range compression curve can be calculated at short intervals (e.g., about 5.3 milliseconds, etc.). The integration time of the human auditory system can be much longer (e.g., about 200 milliseconds, etc.). The DRC gain obtained from the selected dynamic range compression curve can be smoothed by a time constant to account for the long integration time of the human auditory system. To implement a fast rate of change (increase or decrease) in the loudness level, a short time constant can be used to result in a change in the loudness level over a short time interval corresponding to the short time constant. Conversely, to implement a slow rate of change (increase or decrease) in the loudness level, a long time constant can be used to result in a change in the loudness level over a long time interval corresponding to the long time constant.

[0053] The human auditory system can respond to increasing and decreasing loudness levels with different integration times. In some embodiments, different time constants can be used to smooth the static DRC gain found from a selected dynamic range compression curve, depending on whether the loudness level will increase or decrease. For example, corresponding to the characteristics of the human auditory system, onset (increasing loudness level) can be smoothed with a relatively short time constant (e.g., onset time, etc.), while release (decreasing loudness level) can be smoothed with a relatively long time constant (e.g., release time, etc.).

[0054] The DRC gain of a portion of the audio content (e.g., one or more audio data blocks, audio data frames, etc.) can be calculated using a loudness level determined from a portion of the audio content. The loudness level used to find the selected dynamic range compression curve can first be adjusted with respect to (e.g., relative to, etc.) the dialogue loudness level in the metadata extracted from the encoded audio signal (102) (e.g., in a program where the audio content is a part).

[0055] A specific playback environment can be specified or a baseline dialogue loudness level can be established at the decoder (100) (e.g., -31dB in "Line" mode). FS -20dB in "RF" mode FS (etc.). Alternatively, or optionally, in some embodiments, the user can control the setting or change of the reference dialogue loudness level at the decoder (100).

[0056] The DRC gain unit (114) can be configured to determine a dialogue loudness-related gain for the audio content, such that the input dialogue loudness level changes to the output dialogue loudness level relative to a reference dialogue loudness level.

[0057] In some embodiments, the DRC gain unit (114) may be configured to operate the peak level in a specific playback environment at the decoder (100) and adjust the DRC gain to prevent clipping. In some embodiments, according to the first method, if the audio content extracted from the encoded audio signal (102) contains audio data elements of a reference multichannel configuration having more channels than the specific speaker configuration at the decoder (100), then downmixing from the reference multichannel configuration to the specific speaker configuration may be performed before determination and the peak level may be operated for clipping prevention purposes. Alternatively, or as an alternative, in some embodiments, according to the second method, if the audio content extracted from the encoded audio signal (102) contains audio data elements of a reference multichannel configuration having more channels than the specific speaker configuration at the decoder (100), then downmixing (e.g., ITU stereo downmixing, matrix surround compatible downmixing, etc.) may be used to obtain the peak level of the specific speaker configuration at the decoder (100). The peak level may be adjusted to reflect the reference dialogue loudness level as the output dialogue loudness level, which changes from the input dialogue loudness level. The maximum permissible gain that does not result in trimming (e.g., for audio data blocks, for audio data frames, etc.) can be determined at least in part based on the inversion of the peak level (e.g., multiplying by -1, etc.). Therefore, an audio decoder according to the technique described herein can be configured to precisely determine the peak level and apply trimming prevention specifically to the playback configuration on the decoder side; neither the audio decoder nor the audio encoder needs to make any assumptions about any worst-case scenarios at the decoder. In particular, the decoder in the first method described above can precisely determine the peak level and apply trimming prevention after downmixing without using the downmixing, downmixing channel gain, etc., that would be used in the second method described above.

[0058] In some embodiments, the combined adjustment of dialogue loudness level and DRC gain prevents clipping at the peak level, even in worst-case mixing (e.g., generating the maximum peak level after downmixing, generating the maximum downmixed channel gain, etc.). However, in some other embodiments, the combined adjustment of dialogue loudness level and DRC gain may still be insufficient to prevent clipping at the peak level. In these embodiments, the DRC gain may be replaced by the highest gain that effectively prevents clipping at the peak level (e.g., capped).

[0059] In some embodiments, the DRC gain unit (114) is configured to obtain a time constant (e.g., onset time, release time, etc.) from metadata extracted from the encoded audio signal (102). The DRC gain, time constant, maximum allowable gain, etc., can be used by the DRC gain unit (114) to perform DRC, gain smoothing, gain limiting, etc.

[0060] For example, the application of DRC gain can be smoothed by a filter controlled by a time constant. Gain limiting can be implemented by a min() function that takes the smaller of the gain to be applied and the maximum permissible gain, so that (e.g., pre-limiting, DRC, etc.) the gain can be immediately replaced by the maximum permissible gain over a relatively short time interval, thereby preventing pruning.

[0061] In some embodiments, the audio renderer (108) is configured to generate channel-specific audio data (116) for a specific speaker configuration (e.g., multi-channel, etc.) after applying a gain determined based on DRC, gain limiting, gain smoothing, etc., to input audio data extracted from the encoded audio signal (102). The channel-specific audio data (118) can be used to drive speakers, headphones, etc., represented in the speaker configuration.

[0062] Additionally and / or, optionally, in some embodiments, the decoder (100) may be configured to perform one or more other actions related to preprocessing, postprocessing, rendering, etc., which are related to the input audio data.

[0063] The technology described herein can be used with various surround sound configurations (e.g., 2.0, 3.0, 4.0, 4.1, 4.1, 5.1, 6.1, 7.1, 7.2, 10.2, 10...). Various speaker configurations (such as 60-speaker configuration, 60+ speaker configuration, etc.) and various presentation environments (e.g., cinemas, parking lots, opera houses, concert halls, bars, homes, auditoriums) can be used together.

[0064] 4. Audio encoder

[0065] Figure 1B An exemplary encoder 150 is shown. The encoder (150) may include an audio content interface 152, a dialogue loudness analyzer 154, a DRC benchmark library 156, an audio signal encoder 158, etc. The encoder 150 may be part of a broadcasting system, an Internet-based content server, an over-the-air network operator system, a film production system, etc.

[0066] In some embodiments, the audio content interface (152) is configured to receive audio content 160, audio content control input 162, etc., and to generate encoded audio signals (e.g., 102) based at least in part on some or all of the audio content (160), audio content control input (162), etc. For example, the audio content interface (152) may be used to receive audio content (160) and audio content control input (162) from a content creator, content provider, etc.

[0067] Audio content (160) may constitute some or all of the total media data, which consists only of audio and video. Audio content (160) may include multiple parts of a program, a program, several programs, one or more commercials, etc.

[0068] In some embodiments, the dialogue loudness analyzer (154) is configured to determine / establish one or more dialogue loudness levels for one or more portions of the audio content (152), such as one or more programs, one or more commercials, etc. In some embodiments, the audio content is represented by one or more sets of audio tracks. In some embodiments, the dialogue audio content of the audio content is located on a separate audio track. In some embodiments, at least a portion of the dialogue audio content of the audio content is located on an audio track containing non-dialogue audio content.

[0069] The audio content control input (162) may include some or all of the following: user control input, control input provided by a system / device outside the encoder (510), control input from the content creator, control input from the content provider, etc. For example, a user such as a mixing engineer may provide / specify one or more dynamic range compression curve identifiers; the identifiers may be used to retrieve one or more dynamic range compression curves that best fit the audio content (160) from a database such as a DRC benchmark library (156).

[0070] In some embodiments, the DRC reference library (156) is configured to store DRC reference parameter sets, etc. The DRC reference parameter sets may contain definition data for one or more dynamic range compression curves, etc. In some embodiments, the encoder (150) may (e.g., simultaneously, etc.) encode more than one dynamic range compression curve into the encoded audio signal (102). Zero, one, or more dynamic range compression curves may be standard-based, proprietary, custom, decoder-modifiable, etc. In an exemplary embodiment, Figure 2A and Figure 2B Both dynamic range compression curves can be (e.g., simultaneously, etc.) embedded into the encoded audio signal (102).

[0071] In some embodiments, the audio signal encoder (158) may be configured to receive audio content from an audio content interface (152), receive dialogue loudness levels from a dialogue loudness analyzer (154), retrieve one or more DRC reference parameter sets from a DRC reference library (156), format the audio content into audio data blocks / frames, format the dialogue loudness levels, DRC reference parameter sets, etc. into metadata (e.g., metadata containers, metadata columns, metadata structures, etc.), encode the audio data blocks / frames and metadata into an encoded audio signal (102), and so on.

[0072] The audio content described herein, to be encoded into an audio signal, can be received in one or more of various source audio formats via one or more methods, such as wirelessly, via a wired connection, via a file, via the Internet, etc.

[0073] The encoded audio signal described herein can be a portion of the total media data bitstream (e.g., used for audio broadcasting, audio programs, audiovisual programs, audiovisual broadcasting, etc.). The media data bitstream can be accessed from servers, computers, media storage devices, media databases, media files, etc. The media data bitstream can be broadcast, transmitted, or received via one or more wireless or wired network links. The media data bitstream can also be transmitted via one or more intermediaries such as network connections, USB connections, wide area networks, local area networks, wireless connections, optical connections, buses, crossbar connections, serial connections, etc.

[0074] Any of the components shown (e.g., Figure 1A , Figure 1B (etc.) can be implemented in hardware, software or a combination of hardware and software as one or more processes and / or one or more IC circuits (e.g., ASIC, FPGA, etc.).

[0075] 5. Dynamic range compression curve

[0076] Figure 2A and Figure 2B An exemplary dynamic range compression curve is shown that can be used by the DRC gain unit (104) in the decoder (100) to derive the DRC gain from the input loudness level. As shown, in order to provide total gain suitable for a particular playback environment, the dynamic range compression curve can be centered on a reference loudness level in the program. Exemplary definition data of the dynamic range compression curve (e.g., in the metadata of the encoded audio signal 102, etc.) is shown in the table below (e.g., in the metadata of the encoded audio signal 102, etc.), where each of the multiple profiles (e.g., movie standard, movie light, music standard, music light, speech, etc.) represents a particular playback environment (e.g., at the decoder 100, etc.).

[0077] Table 1

[0078] Some embodiments can receive dB SPL or dB FS loudness level and dB SPL The gain in dB is described by one or more compression curves, where the gain is related to dB. SPLDRC gain calculations are performed on different loudness representatives (e.g., Sone) with nonlinear relationships between loudness levels. The compression curves used in the DRC gain calculations can then be transformed to be described with respect to the different loudness representatives (e.g., Sone).

[0079] 6. DRC gain, gain limiting, and gain smoothing

[0080] Figure 3 An exemplary processing logic for determining / calculating the combined DRC and limiting gain is shown. This processing logic can be implemented by a decoder (100), encoder (150), etc. For illustrative purposes only, the DRC gain unit (e.g., 114) in the decoder (e.g., 100, etc.) can be used to implement this processing logic.

[0081] The DRC gain of a portion of the audio content (e.g., one or more audio data blocks, audio data frames, etc.) can be calculated using a loudness level determined from a portion of the audio content. The loudness level can first be adjusted with respect to (e.g., relative to, etc.) the dialogue loudness level in metadata extracted from the encoded audio signal (102) (e.g., in a program where the audio content is part). Figure 3 In the example shown, the difference between the loudness level of a portion of the audio content and the loudness level of the dialogue (“dialnorm”) can be used as input to find the DRC gain from the selected dynamic range compression curve.

[0082] To prevent clipping of output audio data elements in a particular playback environment, the DRC gain unit (114) can be configured to operate the peak level in a particular playback situation (e.g., a specific combination of the encoded audio signal 102 and the playback environment at the decoder 100), which can be one of a variety of possible playback situations (e.g., multi-channel situation, undermixing situation, etc.).

[0083] In some embodiments, the individual peak levels of individual portions of audio content at a specific time resolution (e.g., audio data blocks, several audio data blocks, audio data frames, etc.) may be provided as part of metadata extracted from the encoded audio signal (102).

[0084] In some embodiments, the DRC gain unit (114) may be configured to determine the peak level in these situations and adjust the DRC gain as necessary. During the calculation of the DRC gain, the peak level of the audio content may be determined using parallel processing via the DRC gain unit (114). For example, audio content may be encoded for a reference multichannel configuration having more channels than the specific speaker configuration used by the decoder (100). The audio content of the additional channels of the reference multichannel configuration may be converted into downmixed audio data (e.g., ITU stereo downmixing, matrix surround compatible downmixing, etc.) to derive fewer channels for the specific speaker configuration at the decoder (100). In some embodiments, according to the first method, downmixing from the reference multichannel configuration to the specific speaker configuration may be performed before determining and manipulating the peak level for pruning prevention purposes. Additionally, optionally, or alternatively, in some embodiments, according to the second method, the downmixed channel gain associated with the downmixed audio content may be used as part of the input for adjusting, deriving, calculating, etc., the peak level of the specific speaker configuration. In an exemplary embodiment, the downmixing channel gain may be based at least in part on one or more downmixing derived downmixing actions for implementing downmixing actions from a baseline multichannel configuration to a specific speaker configuration in the playback environment at the decoder (100).

[0085] In some media applications, a specific playback environment can be specified or assumed for the reference dialogue loudness level at the decoder (100) (e.g., -31dB in "Line" mode). FS -20dB in "RF" mode FS (etc.). In some embodiments, the user can control the setting or change of the baseline dialogue loudness level at the decoder (100).

[0086] A dialogue loudness-related gain can be applied to the audio content to adjust the (e.g., output) dialogue loudness level to a baseline dialogue loudness level. Therefore, the peak level should be adjusted to reflect this adjustment. In this example, the (input) dialogue loudness level could be -23 dB. FS With -31 dB FS In the "Line" mode, the adjustment to the (input) dialogue loudness level is -8 dB to generate the output dialogue loudness level at the baseline dialogue loudness level. In "Line" mode, the adjustment to the peak level is also -8 dB, the same as the adjustment to the dialogue loudness level. At -20 dB... FS In the "RF" mode, the input dialogue loudness level is adjusted by -3dB to generate the output dialogue loudness level at the baseline dialogue loudness level. In "RF" mode, the peak level is also adjusted by 3dB, which is the same as the adjustment to the dialogue loudness level.

[0087] The sum of the differences between the peak level and the reference dialogue loudness level (denoted as "dialref") and the dialogue loudness level ("dialnorm") from the metadata of the encoded audio signal (102) can be used as input to calculate the maximum (e.g., allowed, etc.) gain of the DRC gain. Since the adjusted peak level is in dB... FS Expression (relative to 0dB) FS The maximum allowable gain that does not result in trimming (e.g., for the current audio data block, for the current audio data frame, etc.) is simply the inversion of the peak level of the adjustment (e.g., multiplied by -1, etc.).

[0088] In some embodiments, even if the dynamic range compression curve for deriving the DRC gain is designed to cut off the loudness to some extent, the peak level may still exceed the trimming level (denoted as 0 dB). FS In some embodiments, even in worst-case mixing (e.g., generating maximum undermixed channel gain, etc.), the combined adjustment of the dialogue loudness level and DRC gain may prevent clipping at the peak level. However, in some other embodiments, the combined adjustment of the dialogue loudness level and DRC gain may still be insufficient to prevent clipping at the peak level. In these embodiments, the DRC gain may be replaced by the highest gain that effectively prevents clipping at the peak level (e.g., capping, etc.).

[0089] In some embodiments, the DRC gain unit (114) is configured to obtain time constants (e.g., attack time, release time, etc.) from metadata extracted from the encoded audio signal (102). These time constants may or may not change with one or more of the dialogue loudness level or the current loudness level of the audio content. The DRC gain found from the dynamic range compression curve, time constants, and maximum gain can be used to perform gain smoothing and limiting actions.

[0090] In some embodiments, the potentially gain-limited DRC gain may not exceed the maximum peak loudness level in a given playback environment. The stationary DRC gain derived from the loudness level can be smoothed using a filter controlled by a time constant. The limiting action can be implemented by one or more min() functions, such that the (pre-limited) DRC gain can be immediately replaced by the maximum allowed gain over short time intervals, thereby preventing trimming. The DRC algorithm can be configured to smoothly release from the trimmed gain to the lower gain as the peak level of the incoming audio content moves from above the trimmed level to below the trimmed level.

[0091] One or more different implementations (e.g., real-time, two-run, etc.) can be used to perform this. Figure 3The determination / calculation / application of the DRC gain is illustrated. For illustrative purposes only, adjustments to the dialogue loudness level, (e.g., static, etc.) DRC gain, time-dependent gain variations due to smoothing, gain trimming due to limitations, etc., have been described as combined gains from the DRC algorithm described above. However, in various embodiments, other methods may be used to apply gain to the audio content for controlling the dialogue loudness level (e.g., between different programs, etc.), for dynamic range control (e.g., for different parts of the same program, etc.), for preventing trimming, for gain smoothing, etc. For example, some or all of the adjustments to the dialogue loudness level, (e.g., static, etc.) DRC gain, time-dependent gain variations due to smoothing, and gain trimming due to limitations may be applied partially / single, applied serially, applied in parallel, applied partially serially and partially in parallel, etc.

[0092] 7. Input smoothing and gain smoothing

[0093] In addition to DRC gain smoothing, other smoothing processes according to the techniques described herein can be implemented in various embodiments. In one example, input smoothing can be used to smooth the input audio data extracted from the encoded audio signal (102), for example, with a simple single-pole smoothing filter, to obtain a spectrum of a specific loudness level with better temporal characteristics (e.g., more stable in time, less volatile in time, etc.) than the spectrum of a specific loudness level without input smoothing.

[0094] In some embodiments, the different smoothing processes described herein may use different time constants (e.g., 1 second, 4 seconds, etc.). In some embodiments, two or more smoothing processes may use the same time constant. In some embodiments, the time constant used in the smoothing processes described herein may be frequency-dependent. In some embodiments, the time constant used in the smoothing processes described herein may be frequency-independent.

[0095] One or more smoothing processes may be coupled to a reset process that supports automatic or manual reset of one or more smoothing processes. In some embodiments, when a reset occurs in a reset process, the smoothing process can accelerate the smoothing action by switching or transitioning to a smaller time constant. In some embodiments, when a reset occurs in a reset process, the memory of the smoothing process can be reset to a certain value. This value may be the last input sample for the smoothing process.

[0096] 8. DRC on multiple frequency bands

[0097] In some embodiments, a specific loudness level in a specific frequency band can be used to derive the corresponding DRC gain in that specific frequency band. However, even when the wideband (or broadband) loudness level remains constant across all frequency bands, this can lead to variations in timbre because the specific loudness level will also change significantly in different bands, resulting in different DRC gains.

[0098] In some embodiments, instead of applying a DRC gain that varies with each individual frequency band, a DRC gain that does not change with frequency band over time is applied. The same time-varying DRC gain is applied across all frequency bands. The time-averaged DRC gain of the time-varying DRC gain can be set to be the same as the stationary DRC gain derived from a selected dynamic range compression curve based on the total loudness level over a wideband, broadband, and / or wideband (or broadband) range or multiple frequency bands. As a result, variations in timbre caused by applying different DRC gains in different frequency bands can be prevented, as is the case in other methods.

[0099] In some embodiments, the DRC gain in each individual frequency band is controlled by a wideband (or broadband) DRC gain determined based on a wideband (or broadband) loudness level. The DRC gain in each individual frequency band can operate around a wideband (or broadband) DRC gain found in a dynamic range compression curve based on the wideband (or broadband) loudness level, such that the time-averaged DRC gain in each individual frequency band over certain time intervals (e.g., longer than 5.3 ms, 20 ms, 50 ms, 80 ms, 100 ms, etc.) is the same as the wideband (or broadband) loudness level indicated in the dynamic range compression curve. In some embodiments, loudness level fluctuations over short time intervals relative to a certain time interval of deviation from the time-averaged DRC gain are permitted between channels and / or frequency bands. The method ensures the application of the correct multi-channel and / or multi-band time-averaged DRC gain indicated in the dynamic range compression curve and prevents the DRC gain in short time intervals from deviating too much from this time-averaged DRC gain indicated in the dynamic range compression curve.

[0100] 9. Volume adjustment in the loudness range

[0101] Applying linear processing for volume adjustment to an audio excitation signal using other methods that do not implement the techniques described herein can cause low audible signal levels to become inaudible (e.g., below the frequency-dependent hearing threshold of the human auditory system).

[0102] According to the techniques described herein, this can be achieved in the loudness domain (e.g., represented by Sone, etc.) rather than in the physical domain (e.g., represented by dB). SPLVolume adjustment of audio content is performed or implemented in a manner that represents a volume level. In some embodiments, for the purpose of maintaining the perceived quality and / or the integrity of the loudness level relationship between all bands at all volume levels, the loudness level in all bands is scaled with the same factor in the loudness domain. Volume adjustment based on setting and adjusting the gain in the loudness domain described herein can be converted back to or implemented by nonlinear processing in the physical domain (or in the digital domain representing the physical domain) that applies different scaling factors to the audio excitation signals in different frequency bands. The nonlinear processing in the physical domain converted from volume adjustment in the loudness domain according to the techniques described herein attenuates or enhances the loudness level of the audio content with DRC gains that prevent most or all of the low audible levels in the audio content from becoming inaudible. In some embodiments, the loudness level difference between loud and soft sounds within a program is reduced by these DRC gains but does not disappear perceptibly, so that the low audible signal level remains above the hearing threshold of the human hearing system. In some embodiments, in order to maintain similarity in spectral perception and perceived timbre across a wide range of volume levels, frequencies or bands with excitation signal levels close to the hearing threshold at low volume levels are attenuated very little and are therefore perceptibly audible.

[0103] The techniques described herein enable conversions (e.g., reciprocating, etc.) between signal levels, gains, etc. in the physical domain (e.g., or in the digital domain representing the physical domain) and loudness levels, gains, etc. in the loudness domain. These conversions can be based on positive and negative versions of one or more nonlinear functions (e.g., mappings, curves, piecewise linear segments, lookup tables, etc.) constructed based on models of the human auditory system.

[0104] 10. Lower Mix Loudness Adjustment

[0105] In some embodiments, audio content (152) is encoded in an encoded audio signal (102) for a reference speaker configuration that includes multiple audio channels or speakers (e.g., surround sound configuration, 5.1 speaker configuration, etc.).

[0106] The receptor decoder, operating with a specific speaker configuration (e.g., a two-channel headphone configuration, etc.) having fewer audio channels or speakers, is expected to downmix the audio content (152) received from the encoded audio signal (102) from multiple audio channels in the reference speaker configuration (e.g., by one or more downmixing methods, etc.) to fewer audio channels in the decoder's specific speaker configuration, perform gain adjustment of the downmixed audio content, produce a downmixed output sound output, etc.

[0107] For the same individual portions of audio content (152), the loudness level of each individual portion of audio content (152) measured in a reference speaker configuration may differ from the loudness level measured in a specific speaker configuration such as a two-channel configuration. For example, if a portion of the audio content (152) before downmixing has a specific channel-dependent sound distribution concentrated in the left front and right front channels of the reference speaker configuration, then the loudness level of the same portion of the audio content (152) after downmixing to the two-channel configuration may be higher or louder than the loudness level of the same portion of the audio content (152) in the reference speaker configuration before downmixing. On the other hand, if a portion of the audio content (152) before downmixing has a specific channel-dependent sound distribution concentrated in channels other than the left front and right front channels of the reference speaker configuration, then the loudness level of the same portion of the audio content (152) after downmixing to the two-channel configuration may be lower or quieter than the loudness level of the same portion of the audio content (152) in the reference speaker configuration before downmixing.

[0108] In some embodiments, the audio encoder described herein (e.g., 150, etc.) is configured to provide downmixing-related metadata (e.g., containing one or more downmixing loudness parameters, etc.) to a downstream audio decoder. The downmixing-related metadata from the audio encoder (150) can be used by the downstream audio decoder to efficiently and consistently perform (e.g., real-time, near real-time, etc.) downmixing-related gain adjustment actions, allowing the downstream audio decoder to produce a relatively accurate actual target loudness level in the downmixed sound output, preventing inconsistencies in the measured loudness level between a reference speaker configuration and a specific speaker configuration of the decoder, etc.

[0109] In some embodiments, the audio encoder (150) determines one or more downmixing parameters based at least in part on audio content (152) encoded for a reference speaker configuration and a specific speaker configuration (e.g., a two-channel configuration, etc.) different from the reference speaker configuration. In some embodiments, the downmixing loudness parameters comprise one or more distinct groups of downmixing loudness parameters for different types of downmixing actions. The downmixing loudness parameters may comprise a single group of downmixing loudness parameters for use by a downstream audio decoder to perform a specific type of downmixing such as LtRt downmixing, LoRo downmixing, etc. The downmixing loudness parameters may comprise two or more groups of downmixing loudness parameters for use by a downstream audio decoder to perform any one of one or more specific types of downmixing such as LtRt downmixing, LoRo downmixing, etc. The downmixing loudness data generated by the audio encoder (150) may carry one or more specific tags to indicate the presence of one or more groups of downmixing loudness parameters for one or more different types of downmixing actions. The downmixing loudness data may also include preference tags to indicate which type of downmixing action is preferred for the audio content to be downmixed. The lower mixing loudness parameter can be transmitted to the downstream decoder as part of the metadata transmitted in the encoded audio signal containing audio content (152) encoded with reference speaker configuration.

[0110] Examples of downmix loudness parameters described herein may include, but are not limited to, any one of one or more downmix loudness metadata indicators, one or more downmix loudness data columns, etc. In an exemplary embodiment, a downmix loudness parameter may include an indicator for indicating the presence of downmix loudness offset data (e.g., a 1-bit data column denoted as "dmixloudoffste"), a data column for indicating downmix loudness offset (e.g., a 5-bit data column denoted as "5-bit dmixloudoffst"), etc. In some embodiments, one or more instances of these indicators and data columns may be generated by an audio encoder (150) for one or more different types of downmixing actions.

[0111] In some embodiments, the “dmixloudoffste” field may be set to one (1) only when the encoded audio signal (102) carries audio data (e.g., audio samples, etc.) for more than two channels; if the “dmixloudoffste” field is set to one (1), then the “dmixloudoffst” field may be carried. In the example where the encoded audio signal (102) is an AC-3 or E-AC-3 bitstream, the “dmixloudoffste” field may be set to one (1) when the audio encoding mode (e.g., “acmod”, etc.) for the AC-3 or E-AC-3 bitstream is set to a value greater than 2; such a value of the audio encoding mode indicates that the reference speaker configuration is a multi-channel speaker configuration containing more than two audio channels or speakers, and is neither a center speaker configuration only (e.g., a value of 1 for “acmod”, etc.) nor a left front and right front speaker configuration only (e.g., a value of 2 for “acmod”, etc.).

[0112] The “dmixloudoffst” field can be used to indicate, before performing a measurement of the loudness to be measured, the difference between the expected loudness of the downmixed sound output from (e.g., assumed, expected, etc.) an audio decoder (e.g., AC-3 decoder, E-AC-3 decoder, etc.) and the measured loudness of such downmixed sound output, due to gain adjustment caused by dialogue normalization, dynamic range compression, and fixed attenuation to prevent downmixing overload. In some embodiments, the measured loudness includes one or more different groups of downmixed sound output with one or more different groups of gain adjustment. In some embodiments, the audio encoder (150) generates one or more downmixes based on one or more types of downmixing actions (e.g., LtRt downmixing action, LoRo downmixing action, etc.). For example, the audio encoder (150) may apply one or more different sets of downmixing coefficients / equations (e.g., LtRt downmixing coefficients / equations, LoRo downmixing coefficients / equations, etc.) to audio content encoded against a reference speaker configuration (e.g., multi-channel, etc.) to produce one or more downmixes. In some embodiments, the audio encoder (150) may apply one or more different sets of gain adjustments to one or more downmixes to produce one or more different types of downmixed sound outputs for loudness measurements. Examples of multiple sets of gain adjustments include, but are not limited to, any one of the following: a set of gain adjustments with zero gain; a set of gain adjustments including gain adjustments related to dynamic range compression; a set of gain adjustments including gain adjustments related to dialogue normalization; a set of gain adjustments not including gain adjustments related to dynamic range compression; a set of gain adjustments not including gain adjustments related to dialogue normalization; a set of gain adjustments including gain adjustments related to both dynamic range compression and dialogue normalization; etc. After generating one or more different types of downmixed sound outputs for loudness measurement based on one or more different combinations of gain adjustment and downmixing from one or more different groups, the audio encoder (150) can generate the measured loudness by performing one or more different groups of downmixed loudness measurements in any one, some, or all of the one or more different types of downmixed sound outputs. Loudness can be measured by the audio encoder (150) in any of a variety of loudness measurement standards (e.g., LKFS, LUFS, etc.), methods, tools, etc. For illustrative purposes only, the measured loudness may be represented by an LKFS value.

[0113] In some embodiments, the audio encoder (150) assumes that the audio decoder (e.g., 100, etc.) of the encoded audio signal (102) having a dialogue loudness level (e.g., "dialnorm", etc.) described herein intends to apply a certain amount of attenuation (e.g., the difference between the reference loudness level and "dialnorm", etc.) during decoding to align / adjust the output dialogue loudness level of the downmixed sound output to the reference loudness level. For example, if the dialogue loudness level "dialnorm" (e.g., determined from audio content (152) encoded for a reference speaker configuration such as a 5.1 speaker configuration, etc.) has -24 dB. FS The value and if the reference loudness level of a particular speaker configuration of the decoder (e.g., a two-channel configuration for downmixing audio content (152), etc.) is -31 LKFS, then the audio decoder (100) is expected to apply a 7 dB attenuation to align / adjust the output dialogue loudness level to the reference loudness level. In some embodiments, the reference loudness level of a particular speaker configuration of the decoder (e.g., -31 LKFS, etc.) represents (e.g., the desired loudness level for 2-channel downmixed sound output, etc.)

[0114] In some embodiments, the “dmixloudoffst” column may be used by the audio encoder (150) to indicate any loudness deviation between the desired loudness level of the (1) 2-channel downmixed audio output and the (2) measured loudness level of the (2) 2-channel downmixed audio output after some or all of the following have been applied: gain adjustment due to dialogue normalization, dynamic range compression, fixed attenuation to prevent downmixing overload, etc. The “dmixloudoffst” column may contain one or more instances of one or more different types of downmixing after one or more different groups of gain adjustment, etc. The loudness deviation indicated by the “dmixloudoffst” column may, but is not limited to, include the difference in loudness level caused by downmixing audio content from a reference speaker configuration to a specific speaker configuration such as a two-channel configuration. In order to produce a reference loudness level in the downmixed audio output, the loudness deviation corresponds to (e.g., represents the opposite of) the loudness offset that should be applied by the decoder with the specific speaker configuration of the audio content (152) to be downmixed.

[0115] In the exemplary implementation, with -7.5LKFS For a loudness offset range of +7.5LKFS, with a step size of 0.5LKFS, the "dmixloudoffst" column (e.g., its instances, etc.) can be set to 0. Values ​​within the range of 30 (e.g., integers, etc.). Alternatively, or as an alternative, the value of 31 in the "dmixloudoffst" field can be specified as a reserved value and, if present, can be interpreted as the undermix loudness offset of 0LKFS.

[0116] In some embodiments, a positive LKFS value in the "dmixloudoffst" column (e.g., values ​​of 16, 17, ..., 30 for the "dmixloudoffst" column) indicates the measured loudness level of the downmixed sound output so that the indicated LKFS value is louder than the desired loudness level of the downmixed sound output. A negative LKFS value in the "dmixloudoffst" column (e.g., values ​​of 0, 1, ..., 15 for the "dmixloudoffst" column) indicates the measured loudness level of the downmixed sound output so that the indicated LKFS value is quieter or softer than the desired downmixed loudness.

[0117] To compensate for the difference in loudness level between individual parts of the audio content (152) in the encoded audio signal (102) caused by mixing the audio content (152) from a reference speaker configuration to a specific speaker configuration, some or all of the mixing loudness parameters may (e.g., additionally, optionally, as an alternative, etc.) be available to an audio decoder (e.g., 100, etc.) with a speaker configuration such as a specific speaker configuration to control one or more audio processing operations, algorithms, etc., acting on the audio content (152) in the encoded audio signal (102).

[0118] In some embodiments, the audio decoder described herein (e.g., 100, etc.) is configured to decode (e.g., multi-channel, etc.) audio content from an encoded audio signal (102), extract dialogue loudness levels (e.g., “dialnorm”, etc.) from loudness metadata transmitted with the audio content, and so on. The audio decoder (100) can operate via a specific speaker configuration (e.g., a two-channel configuration, etc.) having fewer audio channels than a reference speaker configuration corresponding to the audio content.

[0119] In some embodiments, the audio decoder (100) uses one or more downmixing equations to downmix multichannel audio content received from a reference speaker configuration encoded audio signal (102) for a specific speaker configuration at the audio decoder, performing one or more audio processing operations, algorithms, etc. on the downmixed audio content to produce a downmixed sound output, etc. The audio decoder (100) may be able to perform one or more different types of downmixing operations. The audio decoder (100) may be configured to determine and perform a specific type of downmixing operation (e.g., LtRt downmixing, LoRo downmixing, etc.) based on one or more factors. These factors may include, but are not limited to, user input specifying a preference for a specific user-selected type of downmixing operation, user input specifying a preference for a system-selected type of downmixing operation, the capabilities of a specific speaker configuration and / or the audio decoder (100), the availability of downmixing loudness metadata for a specific type of downmixing operation, encoder-generated preference tags for a certain type of downmixing operation, etc. In some embodiments, the audio decoder (100) may implement one or more priority rules, solicit other user input, etc., to determine a particular type of undermixing operation when these factors conflict with each other.

[0120] One or more audio processing operations, algorithms, etc., include, but are not limited to: applying a certain amount of attenuation (e.g., the difference between a reference loudness level and "dialnorm," etc.) to align / adjust the output dialogue loudness level of the downmixed audio output to the reference loudness level, at least in part, based on the dialogue loudness level (e.g., "dialnorm," etc.) and the reference loudness level (e.g., -31LKFS, etc.). In some embodiments, the audio decoder (100) further performs some or all of the following: gain adjustment due to dialogue normalization, dynamic range compression, fixed attenuation to prevent downmixing overload, etc. In some embodiments, these gain adjustments may correspond to those performed by the audio encoder (150) when determining the measured loudness level described above—for example, they may be the same or substantially the same. One or more of these gain adjustments may be specific to the type of downmixing operation (e.g., LtRt downmixing, LoRo downmixing, etc.) performed by the audio decoder (100).

[0121] Additionally, optionally, or as an alternative, in some embodiments, the audio decoder (100) is configured to extract undermix loudness metadata (e.g., “dmixloudoffste” column, “dmixloudoffst” column, etc.) from the encoded audio signal (102) as part of the metadata transmitted over the audio content. In some embodiments, the undermix loudness parameters in the extracted undermix loudness metadata include one or more different groups of undermix loudness parameters indicated by one or more markers carried in the undermix loudness metadata as having different types of undermixing operations. In response to determining that one or more groups of undermix loudness parameters exist, the audio decoder (100) may determine / select a group of undermix loudness parameters from one or more different groups of undermix loudness parameters that corresponds to a specific type of undermixing operation (e.g., LtRt undermixing, LoRo undermixing, etc.) performed by the audio decoder (100). The audio decoder (100) determines (e.g., based on whether the “dmixloudoffste” column has a value of 1 or 0, etc.) whether undermix loudness offset data exists in a particular group of undermix loudness parameters. The response determines (e.g., based on the "dmixloudoffst" column having a value of 1 or 0, etc.) that submix loudness offset data exists in the submix loudness parameters of a particular group, and the audio decoder (100) performs a loudness adjustment operation based on the submix loudness offset in the submix loudness metadata extracted from the encoded audio signal (102) with audio content (e.g., the "dmixloudoffst" column in the submix loudness parameters of the same group). After applying gain adjustments, etc., to one or more different groups, the submix loudness metadata may contain a "dmixloudoffst" column with one or more instances of one or more different types of submixes. Based on the actual undermixing operation and actual groups of gain adjustments performed by the audio decoder (100) (e.g., no gain adjustment, gain adjustment that does not include those related to DRC, gain adjustment that includes those related to DRC, gain adjustment that does not include those related to dialogue normalization, gain adjustment that includes those related to dialogue normalization, gain adjustment that includes those related to both dialogue normalization and DRC, etc.), the audio decoder (100) can determine / select a specific instance from one or more instances of the “dmixloudoffst” column in the undermix loudness metadata.

[0122] In response to determining that the “dmixloudoffst” column indicates a positive LKFS value (e.g., values ​​16, 17, ..., 30 for the “dmixloudoffst” column), which means that the loudness level of the downmixed sound output (measured by an upstream audio encoder such as 150) after applying some or all of the gain adjustment due to dialogue normalization, dynamic range compression, fixed attenuation to prevent downmixing overload, etc., is louder than the desired loudness level of the downmixed sound output by the magnitude of the indicated LKFS value, the audio decoder (100) performs further gain adjustment with a negative gain value having the magnitude of the indicated LKFS value, which reduces or adjusts the loudness level of the downmixed sound output to the desired loudness (e.g., a reference loudness level, etc.).

[0123] In response to determining that the loudness level of the downmixed audio output (measured by an upstream audio encoder such as 150) after applying some or all of the gain adjustment due to dialogue normalization, dynamic range compression, fixed attenuation to prevent downmixing overload, etc., is quieter or quieter than the LKFS value indicated by the desired loudness level of the downmixed audio output, the audio decoder (100) performs further gain adjustment with a negative gain value having the magnitude of the indicated LKFS value, which increases or adjusts the loudness level of the downmixed audio output to the desired loudness (e.g., reference loudness level, etc.).

[0124] The negative LKFS value in the “dmixloudoffst” column (e.g., values ​​of 0, 1, ..., 15 for the “dmixloudoffst” column) indicates the magnitude of the LKFS value indicating that the measured loudness level of the downmixed sound output is quieter or softer than the expected loudness level. In some embodiments, if the negative LKFS value is indicated / signaled to the receptor decoder in the encoded audio signal (102), then the receptor decoder (e.g., 150, etc.) can take action to ensure that any positive gain applied to the 2-channel downmixed sound output to compensate for the negative LKFS value does not introduce a trimming of the loudness level in the 2-channel downmixed sound output.

[0125] Further gain adjustments based on the loudness offset indicated in the undermix loudness metadata may or may not be limited to those specific to the type of undermixing operation performed by the audio decoder (100).

[0126] 11. Additional actions related to gain

[0127] According to the techniques described herein, other processing such as dynamic equalization, noise compensation, etc., can be performed not in the physical domain (or the digital domain representing the physical domain), but in the loudness (e.g., perception) domain.

[0128] In some embodiments, some or all of the gains from various processes such as DRC, equalization noise compensation, trimming prevention, gain smoothing, etc., may be combined in the same gain in the loudness domain and / or may be applied in parallel. In some other embodiments, some or all of the gains from various processes such as DRC, equalization noise compensation, trimming prevention, gain smoothing, etc., may be in separate gains in the loudness domain and / or may be applied at least partially in series. In some other embodiments, some or all of the gains from various processes such as DRC, equalization noise compensation, trimming prevention, gain smoothing, etc., may be applied sequentially.

[0129] 12. Specific and wideband (or broadband) loudness levels

[0130] One or more audio processing elements, units, components, such as transport filters, auditory filter banks, synthesis filter banks, short-time Fourier transforms, etc., can be used by an encoder or decoder to perform the audio processing actions described herein.

[0131] In some embodiments, one or more transport filters modeling the outer and middle ear filters of the human auditory system can be used to filter incoming audio signals (e.g., encoded audio signal 102, audio content from a content provider, etc.). In some embodiments, an auditory filter bank can be used to model the frequency selectivity and frequency spread of the human auditory system. The excitation signal levels from some or all of these filters can be determined / calculated and smoothed towards shorter frequency-dependent time constants at higher frequencies to model the integral of energy in the human auditory system. Subsequently, a profile of the frequency-dependent loudness level can be obtained using a nonlinear function (e.g., a relationship, curve, etc.) between the excitation signal and a specific loudness level. A wideband (or broadband) loudness level can be obtained by integrating the specific loudness over the frequency band.

[0132] Direct addition / integration of a specific loudness level (e.g., with equal weighting across all bands) works well for wideband signals. However, this approach may underestimate the loudness level (e.g., perceived loudness) of narrowband signals. In some embodiments, specific loudness levels at different frequencies or in different bands are assigned different weights.

[0133] In some embodiments, the auditory filter bank and / or transport filter described above may be replaced by one or more Short-Time Fourier Transform (STFT) filters. The response to the transport filter and auditory filter bank may be applied in the Fast Fourier Transform (FFT) domain. In some embodiments, for example, when one or more (e.g., forward, etc.) transport filters are used in or before a transformation from the physical domain (or in the digital domain representing the physical domain) to the loudness domain, one or more inverse transport filters may be used. In some embodiments, for example, when STFTs are used as an alternative to the auditory filter bank and / or transport filter, the inverse transport filter is not used. In some embodiments, the auditory filter bank is omitted; instead, one or more Quadrature Mirror Filters (QMFs) are used. In these embodiments, the basement membrane extension effect in the model of the human auditory system may be omitted without significantly affecting the performance of the audio processing actions described herein.

[0134] According to the techniques described herein, different numbers of frequency bands (e.g., 20 frequency bands, 40 sensing bands, etc.) can be used in various embodiments. Additionally, optionally, or alternatively, different bandwidths can also be used in various embodiments.

[0135] 13. Individual gain for each individual subset of the channel

[0136] In some embodiments, when a particular loudspeaker configuration is a multi-channel configuration, the total loudness level can be obtained by first summing the excitation signals of all channels before the conversion from the physical domain (or in the digital domain representing the physical domain) to the loudness domain. However, applying the same gain to all channels in a particular loudspeaker configuration cannot maintain spatial balance between the different channels of the particular loudspeaker configuration (e.g., with regard to the relative loudness levels between different channels).

[0137] In some embodiments, in order to maintain spatial balance so that the relative perceived loudness levels between different channels can be optimally or correctly maintained, a loudness level and a corresponding gain based on each loudness level can be determined or calculated for each channel. In some embodiments, the corresponding gains based on each loudness level are not equal to the same total gain; for example, each of some or all of the corresponding gains may be equal to the total gain plus a small correction (e.g., channel-specific).

[0138] In some embodiments, to maintain spatial balance, loudness levels and corresponding gains based on loudness levels may be determined or calculated for each subset of channels. In some embodiments, the corresponding gains based on loudness levels are not equal to the same total gain; for example, each of some or all of the corresponding gains may be equal to the total gain plus (e.g., channel-specific) a small correction. In some embodiments, a subset of channels may comprise two or more channels that form an appropriate subset of all channels in a particular speaker configuration (e.g., a subset of channels containing left front, right front, and low-frequency effects (LFE); a subset of channels containing left surround and right surround, etc.). The audio content of the subset of channels may constitute a submix of the total mix carried in the encoded audio signal (102). The same gain may be applied to the channels within the submix.

[0139] In some embodiments, in order to generate an actual loudness (e.g., actual perceived loudness) from a particular loudspeaker configuration, one or more calibration parameters can be used to correlate the signal level in the digital domain with the corresponding physical level in the physical domain represented by the digital domain (e.g., in dB). SPL (e.g., spatial pressure levels). One or more calibration parameters can be assigned values ​​specific to the physical sound equipment in a particular speaker configuration.

[0140] 14. Auditory Scene Analysis

[0141] In some embodiments, the encoder described herein may implement computer-based auditory scene analysis (ASA) to detect auditory event boundaries in audio content (e.g., encoded as encoded audio signal 102, etc.), generate one or more ASA parameters, and format one or more ASA parameters into a portion of the encoded audio signal (e.g., 102, etc.) to be transmitted to a downstream device (e.g., decoder 100, etc.). ASA parameters may include, but are not limited to, parameters indicating the location of auditory event boundaries, auditory event confidence measurements (explained further later), etc.

[0142] In some embodiments, the location of an auditory event boundary (e.g., temporal) may be indicated in metadata encoded within the encoded audio signal (102). Alternatively, or as an alternative, the location of an auditory event boundary (e.g., temporal) may be indicated in audio data blocks and / or frames that detect the location of the auditory event boundary (e.g., with markers, data bars, etc.).

[0143] Here, an auditory event boundary refers to the point where a preceding auditory event ends and / or a subsequent auditory event begins. Each auditory event occurs between two consecutive auditory event boundaries.

[0144] In some embodiments, the encoder (150) is configured to detect the boundary of an auditory event by the difference in a specific loudness spectrum between two (e.g., temporally) consecutive audio data frames. Each of the specific loudness spectra may contain a spectrum of unsmoothed loudness calculated from the corresponding audio data frames of the consecutive audio data frames.

[0145] In some embodiments, a specific loudness spectrum N [ b , t It can be normalized to obtain the normalized specific loudness spectrum shown in the following formula. N NORM [ b , t ]: here, b Indicates frequency band, t The index representing time or audio data frames, max b { N [ b , t ]} is the maximum specific loudness level across all frequency bands.

[0146] As shown in the following formula, normalized loudness spectra can be subtracted and used to derive the absolute difference of addition. D [ t ].

[0147] The absolute value of the addition is mapped to a value with 0. Measurement of auditory event confidence within a value range of 1 A [ t ]as follows: here, D min and D max These are minimum and maximum thresholds (e.g., user-configurable, system-configurable, related to audio content). D [ t (Past value distribution settings, etc.)

[0148] In some embodiments, the encoder (150) is configured to detect D [ t (For example, in a specific t (and so on) higher than D min Auditory event boundaries at time (e.g., specific) t wait).

[0149] In some embodiments, the decoder described herein (e.g., 100, etc.) extracts ASA parameters from the encoded audio signal (e.g., 102, etc.) and uses the ASA parameters to prevent unintentional boosting of soft sounds and / or unintentional cutting of loud sounds that could cause sensory distortion of auditory events.

[0150] The decoder (100) can be configured to reduce or prevent unintentional distortion of an auditory event by ensuring that the gain is closer to constant within the auditory event and by confining many gain changes to the vicinity of the auditory event boundary. For example, the decoder (100) can be configured to use a relatively small time constant (e.g., comparable to or shorter than the minimum duration of the auditory event) for gain changes in response to the onset (e.g., an increase in loudness level, etc.) at the boundary of the auditory event. Thus, the gain change in the onset can be achieved relatively quickly by the decoder (100). On the other hand, the decoder (100) can be configured to use a relatively long time constant relative to the duration of the auditory event for gain changes in response to the release (e.g., a decrease in loudness level, etc.) in the auditory event. Thus, the gain change in the release can be achieved relatively slowly by the decoder (100), making the sound that should be constant or gradually decaying inaudible or perceptibly disturbed. The rapid response in the onset of an auditory event at the boundary and the slow response in the release of the auditory event allow for a rapid perception of the arrival of the auditory event and maintain the sensory quality and / or integrity of the auditory event—including loud and soft sounds linked by specific loudness level relationships and / or specific time relationships—such as piano strings.

[0151] In some embodiments, the decoder (100) uses auditory events and auditory event boundaries indicated by ASA parameters to control gain variations of one, two, some, or all of the channels in a particular speaker configuration at the decoder (100).

[0152] 15. Loudness level transition

[0153] For example, loudness level transitions can occur between two programs, or between a program and a loud commercial. In some embodiments, the decoder (100) is configured to maintain a histogram of instantaneous loudness levels based on past audio content (e.g., received from encoded audio signal 102, the past 4 seconds, etc.). Two regions with a higher probability of increasing loudness level can be recorded in the histogram over the time interval from before to after the loudness level transition. One region is centered around the previous loudness level, while the other region is centered around the new loudness level.

[0154] The decoder (100) dynamically determines the smoothed loudness level as the audio content being processed and determines the corresponding cabinet of the histogram based on the smoothed loudness level (e.g., a cabinet containing the instantaneous loudness level with the same value as the smoothed loudness level). The decoder (100) is further configured to compare the probability at the corresponding cabinet with a threshold (e.g., 6%, 7%, 7.5%, etc.), where the total area of ​​the histogram curve (e.g., the sum of all cabinets) represents a probability of 100%. The decoder can be configured to detect the occurrence of a loudness level transition by determining that the probability at the corresponding cabinet is below the threshold. In response, the decoder (100) is configured to select a relatively small time constant to adapt to the new loudness level relatively quickly. Thus, the duration of the loud (or soft) start within the loudness level transition can be reduced.

[0155] In some embodiments, the decoder (100) uses a quiet / noise gate to prevent low transient loudness levels from entering the histogram and becoming high-probability boxes in the histogram. Alternatively, or as an alternative, the decoder (100) may be configured to use ASA parameters to detect auditory events contained in the histogram. In some embodiments, the decoder (100) may determine a time-averaged auditory event confidence measurement from the ASA parameters. The time-dependent value. In some embodiments, the decoder (100) determines (e.g., instantaneous, etc.) the auditory event confidence measurement from ASA parameters. A [ t The time-dependent value, and from ASA parameters and other auditory event-based confidence measurements (e.g., instantaneous, etc.). A [ t Time-dependent value calculation, time-averaged auditory event confidence measurement The decoder (100) can be configured to measure the certainty of a time-averaged auditory event simultaneously with the loudness level. Loudness levels below the histogram inclusion threshold (e.g., 0.1, 0.12, etc.) are excluded from the histogram.

[0156] In some embodiments, the loudness levels (e.g., instantaneous, etc.) allowed to be included in the histogram (e.g., corresponding) (Above the histogram threshold, etc.), loudness levels are assigned to a concurrent (contemporaneous) time-averaged auditory event confidence measurement. The time dependence values ​​are the same and proportional to the weights. As a result, loudness levels near the auditory event boundary have a greater impact on the histogram than other loudness levels that are not close to the auditory event boundary (e.g., (Has relatively large values, etc.).

[0157] 16. Reset

[0158] In some embodiments, the encoder described herein (e.g., 150, etc.) is configured to detect a reset event and include an indication of the reset event in the encoded audio signal (e.g., 102, etc.) generated by the encoder (150). In a first example, the encoder (150) detects the reset event in response to determining that a relatively quiet period of consecutive duration (e.g., 250 milliseconds, configurable by the system and / or user, etc.) has occurred. In a second example, the encoder (150) detects the reset event in response to determining that a large transient drop in excitation level occurs across all frequency bands. In a third example, the encoder is provided with inputs (e.g., metadata, user input, system controls, etc.) indicating a transition in content requiring a reset (e.g., program start / end, scene change, etc.).

[0159] In some embodiments, the decoder described herein (e.g., 100, etc.) implements a reset mechanism that can be used to instantaneously accelerate gain smoothing. The reset mechanism is useful and can be invoked when a switch occurs between channels or audiovisual inputs.

[0160] In some embodiments, the decoder (100) may be configured to determine whether a reset event has occurred by determining whether a relatively quiet period of consecutive (e.g., 250 milliseconds, which may be configurable by the system and / or user, etc.) occurs, whether a large instantaneous drop in excitation level across all frequency bands occurs, etc.

[0161] In some embodiments, the decoder (100) may be configured to determine the occurrence of a reset event in response to receiving an indication (e.g., an indication of a reset event, etc.) provided in the encoded audio signal (102) by an upstream encoder (e.g., 150, etc.).

[0162] A reset mechanism can be enabled to issue a reset when the decoder (100) determines that a reset event has occurred. In some embodiments, the reset mechanism is configured to utilize a slightly more aggressive cut behavior of the DRC compression curve to prevent (e.g., a loud program / channel / audiovisual source, etc.) hard start. Additionally, optionally, or alternatively, the decoder (100) can be configured to implement protection measures to gently recover if the decoder (100) detects that a reset has been erroneously triggered.

[0163] 17. Gain provided by the encoder

[0164] In some embodiments, an audio encoder may be configured to compute one or more sets of gains (DRC gains, etc.) for individual portions (e.g., audio data blocks, audio data frames, etc.) of audio content encoded into an encoded audio signal. The multiple sets of gains generated by the audio encoder may include one or more of the following: a first set of gains containing a single wideband (or broadband) gain for all channels (left front, right front, low-frequency effects or LFE, center, left surround, right surround, etc.); a second set of gains containing a single wideband (or broadband) gain for each individual subset of channels; a third set of gains containing a single wideband (or broadband) gain for each individual subset of channels and each of a first number (e.g., two, etc.) of single frequency bands (e.g., two frequency bands in each channel, etc.); a fourth set of gains containing a single wideband (or broadband) gain for each individual subset of channels and each of a second number (e.g., four, etc.) of single frequency bands (e.g., four frequency bands in each channel, etc.); and so on. The subset of channels described herein may be one or more of the following: a subset containing left front, right front, and LFE channels; a subset containing center channels; a subset containing left surround and right surround channels; etc.

[0165] In some embodiments, the audio encoder is configured to transmit one or more portions of audio content (e.g., audio data blocks, audio data frames, etc.) in a time-synchronized manner and to calculate one or more groups of gains for the one or more portions of the audio content. The audio decoder receiving the one or more portions of the audio content can select and apply one set of gains from the one or more groups with little or no delay. In some embodiments, the audio encoder can be implemented in... Figure 4 This illustrates a subframe technique that carries one or more sets of gains within one or more subframes (e.g., through differential coding, etc.). In one example, subframes may be encoded within an audio data block or audio data frame from which the gain is calculated. In another example, subframes may be encoded within an audio data block or audio data frame preceding the audio data block or audio data frame from which the gain is calculated. In yet another non-limiting example, subframes may be encoded within an audio data block or audio data frame over a period of time from the audio data block or audio data frame from which the gain is calculated. In some embodiments, Huffman coding and differential coding may be used to occupy and / or compress subframes carrying multiple sets of gains.

[0166] 18. Exemplary systems and processes

[0167] Figure 5An exemplary codec system is illustrated in a non-limiting exemplary embodiment. A content creator, which may be a processing unit in an audio encoder such as 150, is configured to provide audio content (“Audio”) to an encoder unit (“NGC Encoder”). The encoder unit formats the audio content into audio data blocks and / or frames, and encodes the audio data blocks and / or frames into encoded audio signals. The content creator is also configured to establish / generate one or more dialogue loudness levels (“dialnorm”) for a program, commercial, etc., in the audio content and one or more dynamic range compression curve IDs (“Compression curve IDs”). The content creator may determine the dialogue loudness level from one or more dialogue audio tracks in the audio content. The dynamic range compression curve IDs may be selected at least in part based on user input, system configuration parameters, etc. The content creator may be a person (artist, audio engineer, etc.) using tools to generate audio content and dialnorm.

[0168] Based on dynamic range compression curve identifiers, the encoder (150) generates one or more DRC parameter sets that include, but are not limited to, corresponding reference dialogue loudness levels (“Reference levels”) for multiple playback environments supported by one or more dynamic range compression curves. These DRC parameter sets can be in-band encoded, out-of-band encoded, etc., within the metadata of the encoded audio signal. As part of generating an encoded audio signal that can be transmitted to an audio decoder such as 100, actions such as compression and formatted multiplexing (“MUX”) can be performed. The audio signal can be encoded using a syntax encoding that supports the delivery of audio data elements, DRC parameter sets, reference loudness levels, dynamic range compression curves, functions, lookup tables, Huffman codes used in compression, subframes, etc. In some embodiments, the syntax allows upstream devices (e.g., encoders, decoders, modulators, etc.) to transfer gain to downstream devices (e.g., decoders, modulators, etc.). In some embodiments, the syntax for encoding data into encoded audio signals and / or decoding data therefrom is configured to support backward compatibility, such that means that depend on gain calculated by upstream means may optionally continue to do so.

[0169] In some embodiments, the encoder (150) calculates one, two, or more sets of gains (e.g., DRC gain, gain smoothing, loudness level, etc., based on an appropriate reference) for the audio content. In the metadata encoding the audio content into a coded audio signal, the multiple sets of gains may have one or more dynamic range compression curves. A first set of gains may correspond to the wideband (or broadband) gain of all channels in the speaker configuration or profile (e.g., default, etc.). A second set of gains may correspond to the wideband (or broadband) gain of each of all channels in the speaker configuration or profile. A third set of gains may correspond to the wideband (or broadband) gain of each of two frequency bands of each of all channels in the speaker configuration or profile. A fourth set of gains may correspond to the wideband (or broadband) gain of each of four frequency bands of each of all channels in the speaker configuration or profile. In some embodiments, the multiple sets of gains calculated for the speaker configuration may be conveyed using the dynamic range compression curve of the speaker configuration (e.g., parameterized, etc.) in the metadata. In some embodiments, the multiple sets of gains calculated for the speaker configuration may replace the dynamic range compression curve of the speaker configuration (e.g., parameterized, etc.) in the metadata. Additional speaker configurations or profiles can be found in the technical support described here.

[0170] The decoder (100) is configured to extract audio data blocks and / or frames and metadata from the encoded audio signal, for example, through actions such as decompression, deformatting, demultiplexing (“DEMUX”). The extracted audio data blocks and / or frames can be decoded into audio data elements or samples by a decoder unit (“NGC Decoder”). The decoder (100) is further configured to determine a profile of a specific playback environment at the decoder (100) where the audio content is to be presented, and to select a dynamic range compression curve from the metadata extracted from the encoded audio signal. The digital audio processing unit (“DAP”) is configured to apply DRC and other actions to the audio data elements or samples for the purpose of generating an audio signal that drives the audio channel in the specific playback environment. The decoder (100) can calculate and apply DRC gain based on the loudness level determined from the audio data blocks or frames and the selected dynamic range compression curve. The decoder (100) can also adjust the output dialogue loudness level based on a reference dialogue loudness level associated with the selected dynamic range compression curve and the dialogue loudness level in the metadata extracted from the encoded audio signal. The decoder (100) can then apply a gain limiter specific to the playback scenario and the specific playback environment. Thus, the decoder (100) can present / play audio content tailored to the playback scenario.

[0171] Figure 6A Figure 6D illustrates an exemplary processing flow. In some embodiments, this processing flow may be executed by one or more computing devices or units in a media processing system.

[0172] Figure 6A An exemplary processing flow that can be implemented using the audio decoder described herein is shown. Figure 6A In block 602, the first device (e.g., Figure 1A The audio decoder 100 (etc.) receives an audio signal containing audio content and definition data of one or more dynamic range compression curves.

[0173] In block 604, the first device determines the specific playback environment.

[0174] In block 606, the first device establishes a specific dynamic range compression curve for a specific playback environment based on definition data of one or more dynamic range compression curves extracted from the audio signal.

[0175] In block 608, the first device performs one or more dynamic range control (DRC) actions on one or more portions of the audio content extracted from the audio signal. The one or more DRC actions are based at least in part on one or more DRC gains obtained from a specific dynamic range compression curve.

[0176] In an embodiment, the definition data of one or more dynamic range compression curves includes one or more of the onset time, release time, or a reference loudness level associated with at least one of the one or more dynamic range compression curves.

[0177] In an embodiment, the first device is further configured to perform the following processes: calculating one or more loudness levels of one or more portions of the audio content; determining one or more DRC gains based on a specific dynamic range compression curve and one or more loudness levels of one or more portions of the audio content; and so on.

[0178] In an embodiment, at least one of the loudness levels calculated for one or more portions of the audio content is one or more of the following: a specific loudness level associated with one or more frequency bands, a wideband loudness level spanning a wideband range, a wideband loudness level spanning a wideband range, a wideband loudness level spanning multiple frequency bands, a wideband loudness level spanning multiple frequency bands, etc.

[0179] In an embodiment, at least one of the loudness levels calculated for one or more portions of the audio content is an instantaneous loudness level or one or more loudness levels smoothed over one or more time intervals.

[0180] In an embodiment, one or more actions include one or more actions related to one or more of the following: adjusting dialogue loudness level, gain smoothing, gain limiting, dynamic equalization, noise compensation, etc.

[0181] In an embodiment, the first device is further configured to perform the following processes: extracting one or more dialogue loudness levels from the encoded audio signal; adjusting one or more dialogue loudness levels to one or more reference dialogue loudness levels, etc.

[0182] In an embodiment, the first device is further configured to perform the following processes: extracting one or more auditory scene analysis (ASA) parameters from the encoded audio signal; and changing one or more time constants used when smoothing the gain applied to the audio signal, the gain being related to one or more DRC gains, gain smoothing, or gain limiting, etc.

[0183] In an embodiment, the first device is further configured to perform the following processes: determining that a reset event occurs in one or more portions of the audio content based on an indication of a reset event, the indication of which is extracted from the encoded audio signal; and, in response to determining that a reset event occurs in one or more portions of the audio content, taking one or more actions on one or more gain smoothing actions performed when the determination of the reset event occurs in one or more portions of the audio content.

[0184] In an embodiment, the first device is further configured to perform the following processes: maintaining a histogram of instantaneous loudness levels, the histogram being occupied by instantaneous loudness levels calculated from time intervals in the audio content; determining whether a particular loudness level is above a threshold in a high-probability region of the histogram, the particular loudness level being calculated from a portion of the audio content; and in response to determining that the particular loudness level is above the threshold in a high-probability region of the histogram, performing the following processes: determining that a loudness transition has occurred; shortening the time constant used in gain smoothing to accelerate the loudness transition.

[0185] Figure 6B An exemplary processing flow can be shown that can be implemented using the audio encoder described herein. Figure 6B In block 652, the second device (e.g., Figure 1B The audio encoder 150, etc., receives the audio content in the source audio format.

[0186] In block 654, the second device retrieves definition data for one or more dynamic range compression curves.

[0187] In block 656, the second device generates an audio signal containing audio content and definition data of one or more dynamic range compression curves.

[0188] In an embodiment, the second device is further configured to perform the following processes: determining one or more identifiers of one or more dynamic range compression curves; and retrieving definition data of one or more dynamic range compression curves from a benchmark database based on one or more identifiers, etc.

[0189] In an embodiment, the second device is further configured to perform the following processes: calculating one or more dialogue loudness levels for one or more portions of the audio content; encoding one or more dialogue loudness levels into an encoded audio signal using one or more portions of the audio content, etc.

[0190] In an embodiment, the second device is further configured to perform the following processes: performing auditory scene analysis (ASA) on one or more portions of the audio content; generating one or more ASA parameters based on the results of the ASA on one or more portions of the audio content; encoding one or more ASA parameters into an encoded audio signal using one or more portions of the audio content, etc.

[0191] In an embodiment, the second device is further configured to perform the following processes: determining that one or more reset events occur in one or more portions of the audio content; and encoding one or more indications of one or more reset events into an encoded audio signal using one or more portions of the audio content, etc.

[0192] In an embodiment, the second device is further configured to encode one or more portions of the audio content into one or more audio data frames or audio data blocks.

[0193] In one embodiment, a first DRC gain of one or more DRC gains is applied to channels in a first appropriate subset of all channels in a particular speaker configuration corresponding to a particular playback environment; while a second different DRC gain of one or more DRC gains is applied to channels in a second appropriate subset of all channels in a particular speaker configuration corresponding to a particular playback environment.

[0194] In one embodiment, a first DRC gain of one or more DRC gains is applied to a first frequency band, while a second different DRC gain of one or more DRC gains is applied to a second different frequency band.

[0195] In one embodiment, one or more portions of the audio content comprise one or more audio data frames or audio data blocks. In another embodiment, the encoded audio signal is part of the audiovisual signal.

[0196] In the embodiments, one or more DRC gains are defined in the loudness domain.

[0197] Figure 6C An exemplary processing flow can be shown that can be implemented using the audio encoder described herein. Figure 6C In block 662, the third device (e.g., Figure 1BThe audio encoder 150, etc., generates audio content encoded for the reference speaker configuration.

[0198] In block 664, the second device downmixes the audio content encoded for the reference speaker configuration into the downmixed audio content for the specific speaker configuration.

[0199] In block 666, the second device performs one or more gain adjustments on individual portions of the downmixed audio content encoded for a specific speaker configuration.

[0200] In block 668, the second device performs loudness measurements on each individual portion of the mixed audio content.

[0201] In block 670, the second device generates an audio signal containing audio content encoded with a reference speaker configuration and submixed loudness metadata created at least in part based on loudness measurements on individual portions of the submixed audio content.

[0202] In one embodiment, loudness measurements of each individual portion of the downmixed audio content are performed after one or more gain adjustments are applied to each individual portion of the downmixed audio content. In some embodiments, the loudness measurement is based on a loudness-K-weighted-Full-Scale (LKFS) standard. In some other embodiments, the loudness measurement is based on a loudness standard other than the LKFS standard.

[0203] In an embodiment, audio content encoded for a reference speaker configuration is downmixed to downmixed audio content encoded for a specific speaker configuration based on one or more types of downmixing operations; loudness measurements on individual portions of the downmixed audio content include loudness measurements on individual portions of the downmixed audio content associated with each of the one or more types of downmixing operations.

[0204] In an embodiment, the third device is further configured to prevent the encoding of undermixed audio content for a particular speaker configuration in the audio signal.

[0205] Figure 6D An exemplary processing flow that can be implemented using the audio decoder described herein is shown. Figure 6D In block 682, a fourth device that is configured to operate via a specific speaker (e.g., Figure 1A The audio decoder 100 (etc.) receives an audio signal containing audio content encoded with the reference speaker configuration and submixed loudness metadata.

[0206] In block 684, the first device downmixes audio content encoded for a reference speaker configuration into downmixed audio content encoded for a specific speaker configuration.

[0207] In block 686, the first device performs one or more gain adjustments on individual portions of the downmixed audio content encoded for a specific speaker configuration. The one or more gain adjustments are not based on the downmixed loudness metadata before the downmixed loudness metadata is generated by the upstream audio encoder; and correspond to one or more gain adjustments performed by the upstream audio encoder.

[0208] In block 688, the first device performs one or more additional gain adjustments on individual portions of the downmixed audio content encoded for a specific speaker configuration, the one or more additional gain adjustments being based on the downmixed loudness metadata.

[0209] In an embodiment, the first device is further configured to perform the following processes: determining a specific type of downmixing operation based on one or more selection factors; applying the specific type of downmixing operation when downmixing audio content encoded for a reference speaker configuration to downmixed audio content encoded for a specific speaker configuration; determining a specific set of downmixing loudness parameters corresponding to the specific type of downmixing operation from a set or more sets of downmixing loudness parameters in the downmixing loudness metadata; and performing one or more additional gain adjustments on each individual portion of the downmixed audio content encoded for the specific speaker configuration, based at least in part on the specific set of downmixing loudness parameters.

[0210] In one embodiment, one or more gain adjustments do not produce the desired loudness for at least one individual part of one or more individual parts of the downmixed audio content in the downmixed sound output, wherein one or more additional gain adjustments are performed to produce the desired loudness for at least one individual part of one or more individual parts of the downmixed audio content in the downmixed sound output.

[0211] In one embodiment, the reference speaker configuration is a surround speaker configuration, and the particular speaker configuration is a two-channel configuration.

[0212] In one embodiment, audio content encoded for a reference speaker configuration is downmixed to downmixed audio content encoded for a specific speaker configuration based on one or more downmixing equations.

[0213] In an embodiment, the submixing loudness metadata includes one or more sets of submixing loudness parameters, each of the two or more sets of submixing loudness parameters corresponding to a single type of submixing operation in one or more types of submixing operations corresponding to the one or more sets of submixing loudness parameters.

[0214] In an embodiment, one or more types of undermixing operations include at least one of LtRt undermixing operation or LoRo undermixing operation.

[0215] In an embodiment, one or more gain adjustments include at least one gain adjustment associated with one or more of dialogue normalization, dynamic range compression, or a fixed attenuation to prevent undermixing overload.

[0216] In one embodiment, one or more gain adjustments use different gain adjustment parameter values ​​for at least two different parts of each individual portion of the audio content.

[0217] In this embodiment, the lower-mix loudness metadata represents a portion of the total audio metadata encoded in the audio signal. In this embodiment, the lower-mix loudness metadata includes a data column indicating the lower-mix loudness offset. In this embodiment, the encoded audio signal is part of an audiovisual signal.

[0218] In an embodiment, an apparatus is provided that includes a processor and is configured to perform any of the methods described herein.

[0219] In embodiments, a non-transitory computer-readable storage medium is provided containing software instructions that, when executed by one or more processors, cause any of the methods described herein to be performed. Note that while individual embodiments are discussed herein, any combination of the embodiments and / or portions thereof discussed herein can be combined to form other embodiments.

[0220] 19. Implementation Mechanism - Hardware Overview

[0221] According to one embodiment, the techniques described herein are implemented via one or more dedicated computing devices. The dedicated computing devices may be hardwired to execute the techniques, or may contain digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs) that are permanently programmed to execute the techniques, or may contain one or more general-purpose hardware processors programmed to execute the techniques according to program instructions in firmware, memory, other memory, or a combination thereof. Such dedicated computing devices may also combine custom hardwired logic, ASICs, or FPGAs with custom programming to implement the techniques. The dedicated computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices, or any other devices incorporating hardwired and / or program logic to implement the techniques.

[0222] For example, Figure 7 This is a block diagram illustrating a computer system 700 that can implement an embodiment. The computer system 700 includes a bus 702 or other communication mechanism for transmitting information and a hardware processor 704 coupled to the bus 702 for processing information. The hardware processor 704 may be, for example, a general-purpose microprocessor.

[0223] The computer system 700 also includes main memory 706, such as random access memory (RAM) or other dynamic storage devices, coupled to a bus 702 for storing information and instructions to be executed by the processor 704. Main memory 706 can also be used to store time variables or other intermediate information while executing instructions to be executed by the processor 704. When these instructions are stored in a non-transitory storage medium accessible to the processor 704, they transform the computer system 700 into a dedicated machine customized to perform the actions specified in the instructions.

[0224] The computer system 700 also includes a read-only memory (ROM) 708 or other static storage device coupled to the bus 702 for storing static information and instructions of the processor 704. A storage device 710 for storing information and instructions, such as a magnetic disk or optical disk, is provided and coupled to the bus 702.

[0225] Computer system 700 can be coupled via bus 702 to a display 712, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 714, containing numeric, alphanumeric, and other keys, for transmitting information and command selections to processor 704, is coupled to bus 702. Another type of user input device is a cursor control 716, such as a mouse, trackball, or cursor arrow keys, for transmitting directional information and command selections to processor 704 and for controlling cursor movement on display 712. This input device generally has two degrees of freedom along two axes, namely a first axis (e.g., x) and a second axis (e.g., y), which allow the device to specify its position in a plane.

[0226] The computer system 700 can implement the techniques described herein by using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic combined with the computer system to cause the computer system 700 or to program it as a special-purpose machine. According to one embodiment, the techniques described herein are executed by the computer system 700 in response to a processor 704 executing one or more sequences of one or more instructions contained in main memory 706. Such instructions may be read into main memory 706 from another storage medium, such as storage device 710. Execution of the sequence of instructions contained in main memory 706 causes the processor 704 to perform the processing steps described herein. In alternative embodiments, hardwired circuitry may be used as an alternative to, or in combination with, software instructions.

[0227] As used herein, the term "storage medium" refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a particular manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical discs or magnetic disks, such as storage device 710. Volatile media include dynamic memory, such as main memory 706. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs, NVRAMs, any other memory chips or enclosures.

[0228] Storage media are distinct from transmission media, but can be used in combination with them. Transmission media participate in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including wires containing bus 702. Transmission media can also take the form of sound waves or light waves, such as those generated in radio waves and infrared-to-red data communication.

[0229] Various forms of media can participate in carrying one or more sequences of instructions for execution to processor 704. For example, instructions may first be carried on a disk or solid-state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and transmit them over a telephone line using a modem. A modem local to computer system 700 can receive data over the telephone line and use an infrared-red transmitter to convert the data into an infrared-red signal. An infrared-red detector can receive the data carried in the infrared-red signal, and appropriate circuitry can place the data on bus 702. Bus 702 carries the data to main memory 706, from which processor 704 retrieves and executes instructions. Instructions received via main memory 706 may optionally be stored on storage device 710 before or after execution by processor 704.

[0230] Computer system 700 may include a communication interface 718 coupled to bus 702. Communication interface 718 provides bidirectional data communication coupling to network link 720 connected to local area network 722. For example, communication interface 718 may be an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem providing data communication connectivity to a corresponding type of telephone line. As another example, communication interface 718 may be a local area network (LAN) card providing data communication connectivity to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 718 transmits and receives electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0231] Network link 720 typically provides data communication to other data devices via one or more networks. For example, network link 720 may provide connectivity to host computer 724 via local area network 722 or to data devices operating through Internet service provider (ISP) 726. ISP 726, in turn, provides data communication services through a worldwide packet data communication network now commonly referred to as the "Internet" 728. Both local area network 722 and Internet 728 use electrical, electromagnetic, or optical signals that carry digital data streams. The signals carrying digital data through various networks and on network link 620, as well as through communication interface 718, are exemplary forms of transmission media for computer system 700.

[0232] Computer system 700 can send and receive messages, including program code, via a network, network link 720, and communication interface 718. In the Internet example, server 730 transmits request codes for the application via the Internet 728, ISP 726, local area network 722, and communication interface 718.

[0233] The received code may be executed by processor 704 upon receipt and / or stored in storage device 710 or other non-volatile memory for later execution.

[0234] 20. Equivalents, extensions, substitutions, and miscellaneous items

[0235] In the foregoing description, embodiments of the invention have been described with reference to a number of specific details that may vary between implementations. Therefore, the applicant's intention regarding what is proprietary and exclusive to the invention is indicated by a set of claims, including any subsequent amendments, which are issued from this application in the specific form in which they are made. Any definitions expressly set forth herein for terms included in these claims should be understood in the sense that the terms are used in the claims. Consequently, any limitations, elements, properties, features, advantages, or attributes not expressly detailed in the claims should not in any way limit the scope of these claims. Therefore, the description and drawings should be considered interpretative rather than restrictive.

Claims

1. A method for gain adjustment of an audio signal based on loudness metadata generated by an encoder, the method comprising: An audio signal for a playback channel configuration that operates in a playback channel configuration different from the reference channel configuration is received by an audio decoder. The audio signal includes audio sampling data for each channel of the reference channel configuration and loudness metadata generated by an encoder. The loudness metadata generated by the encoder includes loudness metadata for multiple channel configurations, including the playback channel configuration and the reference channel configuration. The audio sample data is downmixed into downmixed audio sample data for the audio channel configured for the playback channel. Select loudness metadata for the playback channel configuration from the loudness metadata used for the plurality of channel configurations; The loudness adjustment gain is determined based on the loudness metadata used for the playback channel configuration; as well as The loudness adjustment gain is applied as part of the total gain applied to the downmixed audio sample data to generate the output audio sample data for each channel in the playback channel configuration. The loudness adjustment gain depends on a reference loudness level and a loudness level indicated by loudness metadata used for the playback channel configuration. The number of audio channels configured in the playback channel configuration differs from that in the reference channel configuration.

2. The method according to claim 1, wherein, The total gain includes one or more of the following: Gain related to downmixing The gain associated with restoring the original dynamic range is the input dynamic range of the audio sample data, which is derived from the original dynamic range. Gain related to gain limitation Gain related to gain smoothing, or Gain related to dialogue loudness normalization.

3. The method according to claim 1, wherein, The total gain includes the gain to be applied partially / individually, the gain to be applied serially, the gain to be applied in parallel, or the gain to be applied partially serially and partially in parallel.

4. A non-transitory computer-readable storage medium storing software instructions that, when executed by one or more processors, cause the method according to any one of claims 1 to 3 to be performed.

5. An audio signal processing apparatus for gain adjustment of an audio signal based on loudness metadata generated by an encoder, wherein the audio signal processing apparatus: An audio signal for a playback channel configuration that operates in a playback channel configuration different from the reference channel configuration is received by an audio decoder. The audio signal includes audio sampling data for each channel of the reference channel configuration and loudness metadata generated by an encoder. The loudness metadata generated by the encoder includes loudness metadata for multiple channel configurations, including the playback channel configuration and the reference channel configuration. The audio sample data is downmixed into downmixed audio sample data for the audio channel configured for the playback channel. Select loudness metadata for the playback channel configuration from the loudness metadata used for the plurality of channel configurations; The loudness adjustment gain is determined based on the loudness metadata used for the playback channel configuration; as well as The loudness adjustment gain is applied as part of the total gain applied to the downmixed audio sample data to generate the output audio sample data for each channel of the playback channel configuration. The loudness adjustment gain depends on a reference loudness level and a loudness level indicated by loudness metadata used for the playback channel configuration. The number of audio channels configured in the playback channel configuration differs from that in the reference channel configuration.

6. An apparatus for gain adjustment of an audio signal based on loudness metadata generated by an encoder, comprising: One or more processors, and A non-transitory computer-readable storage medium storing software instructions that, when executed by the one or more processors, cause the method according to any one of claims 1 to 3 to be performed.

7. An apparatus for gain adjustment of an audio signal based on loudness metadata generated by an encoder, comprising a component for performing the method according to any one of claims 1 to 3.

8. A computer program product having instructions that, when executed by a computing device or system, cause the computing device or system to perform the method according to any one of claims 1 to 3.

9. A method comprising: Receive an encoded audio bitstream, the encoded audio bitstream including audio data and metadata, the metadata including one or more sets of submixed loudness parameters; Determine whether the first submix loudness parameter indicates the existence of submix loudness offset data, and if submix loudness offset data does exist, adjust the difference between the expected loudness of the submixed audio signal and the measured loudness of the submixed audio signal based on the second submix loudness parameter; as well as The second lower mixing loudness parameter indicates whether there is a difference between the expected loudness of the mixed output under the two channels and the actual measured loudness.

10. An audio processing apparatus, comprising: A buffer is used to store at least a portion of an encoded audio bitstream, which includes audio data and metadata. A demultiplexer is used to parse this portion of the encoded audio bitstream; as well as An audio decoder is used to decode audio data. The metadata includes a first parameter indicating the presence of downmixed loudness offset data, and if downmixed loudness offset data is indeed present, a second parameter indicating the difference between the expected loudness of the downmixed audio signal and the measured loudness of the downmixed audio signal.

Images