Audio object clustering system

Abstract

Systems and methods for dynamically controlling cluster determination and object-to-cluster gains calculations in an audio object clustering system. One example provides an audio object clustering system that processes audio objects. The system includes an onset / offset detection module configured to set a flag based on an excitation of an audio object in a current frame. The system also includes a clustering decision module configured to determine whether to enable a cluster determination and object-to-cluster gains calculation module for the current frame based on the flag.

Description

AUDIO OBJECT CLUSTERING SYSTEMCROSS-REFERENCE

[0001] This application claims the benefit of priority from International Patent Application No. PCT / CN2024 / 101401, filed on 25 June 2024, U.S. Provisional Application No. 63 / 683,505 filed on 15 August 2024, International Patent Application No. PCT / CN2025 / 102000, filed on 19 June 2025, and United States Provisional Patent Application No. 63 / 827,201, filed on June 20, 2025, each of which is incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] This application relates generally to object-based audio signal processing and, more specifically, dynamically controlling cluster determination and object-to-cluster gains calculations in an audio object clustering system.BACKGROUND

[0003] Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted as prior art by inclusion in this section.

[0004] In interactive entertainment systems, object-based audio signals are transmitted between system endpoints, where the object-based audio signals may contain hundreds of individual audio objects. Consequently, transmitting these audio signals may be challenging. For example, some system endpoints may have limited resources (e.g., limited processing power, limited memory, etc.) that result in strict requirements as to the maximum number of audio objects supported at the endpoint. The present disclosure appreciates these and other system limitations, and therefore, an efficient object-based audio scene management approach is desired.

[0005] To reduce the complexity of a given audio scene, clustering techniques may be employed to reduce the number of input audio objects and beds into a small set of output objects (hereafter called clusters), and with minimum impact on audio quality. In general, an object clustering method may include two steps: 1) cluster determination, and 2) clustering generation. Cluster determination is used to determine the cluster position and associated metadata, while cluster generation is used to calculate the object to cluster gains and generate the clusters.

[0006] In an example clustering method, a first step may include determining the clusters by selecting the most perceptually important objects. Both the object loudness and the spatialposition of the object may be considered to identify the measure of perceptual importance. In a second step, the clusters may be generated by calculating the object-to-cluster gains and applying the calculated gains to the input audio objects. In a further example, the gains calculations may include a method that minimizes a cost function, where position correctness, distance and amplitude preservation may be jointly considered.

[0007] In gaming and the other interactive use-cases, computational complexity is a primary concern for a good user experience, which may lead to strict requirements for all running algorithms. The present disclosure thus appreciates that audio object clustering methods are expected to be optimized in terms of 1) complexity reduction by removing unnecessary processing intentionally, and 2) quickly adapted to temporal changes in terms of audio loudness / energy and positional and non-positional metadata.

[0008] It is with respect to these and other considerations that the disclosure made herein is presented.BRIEF SUMMARY OF THE DISCLOSURE

[0009] Techniques are described for processing audio signals. Examples found herein provide for systems, devices, and methods to process audio objects and, more specifically, to dynamically control whether cluster determination and object-to-cluster gains calculation operations are performed.

[0010] Techniques described herein relate to a low complexity audio objects clustering framework. The proposed audio object clustering framework operates on a frame-by-frame basis. As referred to herein, audio frames refer to fixed samples of audio signal. In each audio frame, the audio blocks are associated metadata may be generated at run-time. For example, in an operating frame, there may be M audio objects generated at runtime. A fixed number of clusters, denoted by N, is generated using examples described herein.

[0011] Particularly, examples described herein provide for a clustering decision module that controls whether cluster determination and object-to-cluster gains operations are performed. If a clustering decision flag from the clustering decision module is true, cluster determination and object-to-cluster gains operations are executed. If the clustering decision flag is false, cluster determination and object-to-cluster gains calculations are skipped, and previous clusters’ position and previous object-to-cluster gains are used for generating clusters.

[0012] One example described herein provides an audio object clustering system that processes audio objects. The audio objects may be generated at runtime. The system includes an onset / offset detection module configured to set a flag based on an excitation of an audio object in a current frame. The system also includes a clustering decision module configured to determine whether to enable a cluster determination and object-to-cluster gains calculation module for a current frame based on the flag.

[0013] In some instances, the excitation of the audio object indicates a loudness of the audio object.

[0014] In some instances, the onset / offset detection module sets the flag based on both the excitation of the audio object in the current frame and the excitation of the audio object in a previous frame. The previous excitation may be stored in an excitation buffer. For example, both the current excitation and the previous excitation are compared to identify a change in the excitation of the audio object. The onset / offset detection module may set the flag based on whether the change in the excitation of the audio object exceeds an excitation threshold. The onset / offset detection module may further set the flag based on changes in position of the audio object between frames.

[0015] In some instances, the onset / offset detection module sets the flag based on whether the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least an excitation change threshold. For example, the excitation change threshold may be an increasing function of the excitation of the audio object in the previous frame.

[0016] In some instances, the onset / offset detection module sets the flag to a first value (e.g., TRUE) or a second value (e.g., FALSE), wherein a condition for setting the flag to the first value includes that the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least the excitation change threshold, and wherein otherwise the flag is set to the second value. The condition for setting the flag to the first value may further include that the excitation of the audio object in the current frame is above an excitation threshold, and wherein otherwise the flag is set to the second value. The onset / offset detection module may set the flag based on a change in position of the audio object between the previous and current frame, wherein the condition for setting the flag to the first value further includes that the change in position is above a distance threshold.

[0017] In some instances, the clustering decision module determines whether to enable the cluster determination and object-to-cluster gains calculation module based on a frame index. For example, the clustering decision module may perform a modulo operation between the frame index and a predetermined modulo operator. A clustering decision flag is set by the clustering decision module based on the result of the modulo operation (e.g., set to TRUE on a condition that the result is zero). The clustering decision module may additionally or alternatively determine whether to enable the cluster determination and object-to-cluster gains calculation module based on whether there is any new cluster candidate in the current frame which is not included in a previous cluster candidates list. For example, the clustering decision flag may be set based on whether a new cluster candidate is present and / or the onset / offset flag indicate (e.g., by being set to the first value) that the new cluster candidate has either onset or offset (e.g., a change in loudness during the frame). In some instances, the clustering decision module determines that a given audio object is a new cluster candidate on a condition that the excitation of the given audio object is above a cluster candidate threshold, and is not included in a previous cluster candidates list for the previous frame. The previous cluster candidates list may include audio objects included in the previous frame having an excitation (in the previous frame) above the cluster candidate threshold.

[0018] When enabled, the cluster determination and object-to-cluster gains calculation module generates current cluster centroids and current object-to-cluster gains for the audio objects associated with the frame. The current cluster centroids and current object-to-cluster gains are used by a cluster generation module to generate output clusters. The current cluster centroids and current object-to-cluster gains may also be stored in a clusters and gains buffer. When the cluster determination and object-to-cluster gains calculation module is skipped, the cluster generation module generates output clusters using previous cluster centroids (e.g., previous cluster positions) and previous object-to-cluster gains stored in the clusters and gains buffer.

[0019] Another example described herein provides a method for clustering audio objects. The method includes receiving an input audio object included in a current frame. The audio object may include a block of pulse-code modulation (PCM) samples and corresponding metadata. The method includes calculating an excitation of the audio object in the current frame. The method includes determining whether to enable a cluster determination and object-to-cluster gains calculation module based on the excitation of the audio object in the current frame. The method may further include generating output clusters based on whether the cluster determination and object-to-cluster gains calculation module is enabled.

[0020] In some instances, generating output clusters based on whether the cluster determination and object-to-cluster gains calculation module is enabled includes: performing, when the cluster determination and object-to-cluster gains calculation module is enabled, cluster determination and object-to-cluster gains calculations on the plurality of audio objects to obtain current cluster centroids and object-to-cluster gains; and generating output clusters using the current cluster centroids and the current object-to-cluster gains.

[0021] In some instances, the method comprises storing the current cluster centroids and the current object-to-cluster gains in a clusters and gains buffer.

[0022] In some instances, generating output clusters based on whether the cluster determination and object-to-cluster gains calculation module is enabled includes generating, when the cluster determination and object-to-cluster gains calculation module is not enabled, output clusters using previous cluster centroids and previous object-to-cluster gains stored in a clusters and gains buffer.

[0023] Various aspects of the present disclosure provide for processing of audio signals, and effect improvements in at least the technical fields of audio processing, audio encoding, audio decoding, virtual reality, object-based audio, and the like.

[0024] The embodiments described herein may be generally described as techniques, where the term “technique” may refer to system(s), device(s), method(s), computer-readable instruction(s), module(s), component(s), hardware logic, and / or operation(s) as suggested by the context as applied herein.

[0025] Features and technical benefits other than those explicitly described above will be apparent from a reading of the following Detailed Description and a review of the associate drawings. This Summary is provided to introduce a selection of techniques in a simplified form, and not intended to identify key or essential features of the claimed subject matter, which are defined by the appended claims.DESCRIPTION OF THE DRAWINGS

[0026] These and other more detailed and specific features of various embodiments are more fully disclosed in the following description, reference being had to the accompanying drawings, in which:

[0027] FIG. 1 illustrates a block diagram of an example audio processing system in which various aspects of the present disclosure can be practiced.

[0028] FIG. 2 illustrates a block diagram of an example spatial coding method that can be implemented in the audio system of FIG. 1 according to some aspects of the present disclosure.

[0029] FIGS. 3A-3B illustrate a block diagram of a framework for audio object clustering according to some aspects of the present disclosure.

[0030] FIG. 4 illustrates a block diagram of various example methods for clustering audio objects, which may be performed by the audio processing system of FIG. 1, in accordance with various aspects of the present disclosure.

[0031] FIG. 5A illustrates a schematic block diagram of an example device architecture that may be used to implement various aspects of the present disclosure.

[0032] FIG. 5B illustrates a schematic block diagram of an example CPU implemented in the device architecture of FIG. 5A that may be used to implement various aspects of the present disclosure.DETAILED DESCRIPTION

[0033] In the following description, numerous details are set forth, such as audio device configurations, timings, operations, and the like, in order to provide an understanding of one or more aspects of the present disclosure. It will be readily apparent to one skilled in the art that these specific details are merely examples and not intended to limit the scope of this application.

[0034] As used herein, the term “includes” and its variants are to be read as open-ended terms that mean “includes, but is not limited to.” The term “or” is to be read as “and / or” unless the context clearly indicates otherwise. Such terms are to be read as having an inclusive meaning. For example, “A and B” may mean at least the following: “both A and B”, “at least both A and B”. As another example, “A or B” may mean at least the following: “at least A”, “at least B”, “both A and B”, “at least both A and B”. As another example, “A and / or B” may mean at least the following: “A and B”, “A or B”. When an exclusive-or is intended, such will be specifically noted (e.g., “either A or B”, “at most one of A and B”). The term “based on” is to be read as “based at least in part on.” The term “one example implementation” and “an example implementation” are to be read as “at least one example implementation.” The term “another implementation” is to be read as “at least one other implementation.” The terms “determined,” “determines,” or “determining” are to be read as obtaining, receiving, computing, calculating, estimating, predicting, or deriving. In addition, in the following description and claims, unlessdefined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

[0035] For interactive entertainment content, transmitting the original object-based audio signal, which may contain hundreds of individual objects, can be challenging because endpoints that support object-based audio typically have limitations with respect to the maximum number of audio objects that can be supported, e.g., due to the limited amounts of computational resources and / or memory. In such cases, application of efficient object-based audio-scene management is desirable.

[0036] In some cases, “spatial coding” may be used to reduce the complexity of an audio scene. Some spatial coding methods employ clustering techniques that aim to reduce the number of input objects and / or beds to a smaller set of output objects (hereafter referred to as clusters) with minimal impact on the audio quality.

[0037] Various Acronyms that may appear throughout this disclosure and in the associated claims and / or drawings are listed below. Other commonly used acronyms and terms of art may be excluded from this list in the interest of brevity. Thus, a short list of acronyms is provided below as an easy reference for the reader.FOA - First Order AmbisonicsHOA - Higher Order AmbisonicsPCM - Pulse-Code ModulationXR - Extended RealityVR - Virtual RealityAR - Augmented RealityMR - Mixed RealityRMS - Root Mean SquareERB - Equivalent Rectangular Bandwidth LFE - Low Frequency Enhancement ROM - Read Only MemoryRAM - Random Access Memory CPU - Central Processing Unit I / O - Input / OutputLCD - Liquid Crystal DisplayEPROM - Erasable Programmable Read-Only Memory CD-ROM - Compact Disk Read-Only Memory

[0038] Herein, the terms “clustering,” “grouping,” and “combining” may be used interchangeably to describe combinations of objects and / or beds (or channels) configured to reduce the amount of data in a unit of adaptive audio content for transmission and rendering in an audio playback system. The terms “compression” and “reduction” may be used to refer to an act of performing audio scene simplification, e.g., via such clustering. The terms “clustering,” “grouping,” and “combining” throughout this description are not limited to a strictly unique assignment of an audio object or bed to a single cluster only. In some cases, an audio object or bed may be distributed over more than one output bed or cluster using weights or gain vectors that determine the relative contribution of an object or bed signal to the output cluster or output bed signal.

[0039] FIG. 1 illustrates a block diagram of an example audio processing system 100 in which various aspects of the present disclosure can be practiced. The audio processing system 100 includes an audio object clustering module 110 coupled to a game engine 120. The audio object clustering module 110 receives input signals 102 which include a plurality of audio objects (e.g., audio object 1 through audio object N). The audio object clustering module 110 clusters the plurality of audio objects into a second plurality of audio objects (e.g., M audio objects) 106. The audio object clustering module 110 provides the second plurality of audio objects 106 to the game engine 120.

[0040] The game engine 120 receives the second plurality of audio objects 106 (e.g., M audio objects) from the audio object clustering module 110. The game engine 120 also receives input signals 104, which may include channel-based audio, first order Ambisonics (FOA), higher order Ambisonics (HOA), and stereo audio content. The game engine 120 processes the input signals 104 and the second plurality of audio objects 106 to generate output signals 108. In some instances, the game engine 120 merely passes the input signals 104 through as the output signals 108, such as when the input signals 104 include background music in stereo format. In other instances, the game engine 120 may mix different input sounds (e.g., apply different channelbased audio effects into one channel-based audio output), may perform panning of the audio objects (e.g., panning audio objects to 7.1.4), may perform 3D audio processing (e.g., render audio objects to binaural format), or perform other audio processing such as reflection, reverberation, attenuation, and similar types of processing operations. The output signals 108 may be provided to an audio rendering component (for example, a plurality of speakers, a game console, a headset device, a home theater, one or more E-media devices, a TV set, a sound reproduction system, and the like), or may be stored in memory for future rendering.

[0041] The audio processing system 100 may implement object-based audio formats that include both beds and objects. Audio beds refer to audio channels that are meant to be reproduced in predefined, fixed locations. Audio objects refer to individual audio elements that may exist for a defined duration in time but also have spatial information of each object, such as position, size, and the like. Audio objects may include a block of pulse-code modulation (PCM) samples and corresponding metadata. During transmission, beds and objects may be sent separately and then used by a spatial reproduction system to recreate the artistic intent. The reproduction system may include a variable number of speakers or headphones. Object-based audio as referred to herein may include interactive content that is typically generated at runtime as the user interacts with the corresponding virtual scene, and may also include music content.

[0042] FIG. 2 illustrates a block diagram of an example spatial coding method 200 that can be implemented in the audio processing system 100 according to some aspects of the present disclosure. The method 200 may include a seed selection module 210 and a cluster generation module 220. The input of the seed selection module 210 corresponds to a first path 202. The seed selection module 210 is configured to receive audio objects via the first path 202. The output of the seed selection module 210 corresponds to a second path 212. The seed selection module 210 may be configured to perform operations for cluster-seed selection, where input audio objects received via the first path 202 may be evaluated to identify a set of seed objects for output clusters. Once the set of seed objects is identified by the seed selection module 210, the cluster positions may be determined (e.g., calculated or estimated), and the associated metadata may be generated. The set of seed objects, the cluster positions, and the associated metadata are transmitted by the seed selection module 210 to the cluster generation module 220 via the second path 212.

[0043] The input of the cluster generation module 220 corresponds to the second path 212. The output of the cluster generation module 220 corresponds to a third path 222. The cluster generation module 220 is configured to receive the set of seed objects, the cluster positions, and the associated metadata from the seed selection module 210 via the second path 212. The cluster generation module 220 may be configured to perform cluster generation, where object to cluster gains are calculated and the corresponding clusters are generated from the set. The output clusters are provided by the cluster generation module 220 via the third path 222.

[0044] In some examples, operations of the seed selection module 210 may include evaluating the input audio objects and selecting perceptually most-important objects as cluster seeds. Both the object excitation (e.g., loudness) and spatial position may be considered as factors in theprocess of determining the object’s importance. Operations of the cluster generation module 220 may include calculating the object-to-cluster gains and applying the gains to the input audio objects to generate the corresponding output clusters. In some examples, the object-to-cluster gains may be calculated in the cluster generation module 220 based on minimizing a cost function, which may consider one or more factors such as the position correctness, distance preservation, amplitude preservation, or any combinations thereof.

[0045] Examples, aspects, and instances described herein provide a low (or reduced) complexity audio objects clustering framework. With the framework described herein, audio object clustering may be implemented by using the current (and / or the previous) audio frames and, optionally, the associated metadata. Frames as referred to herein are short time intervals of audio samples. An audio frame may include multiple blocks, wherein each block is associated with an audio object. The term audio block may refer to a fixed number of samples of an audio signal (for example, 256, 512, 1024, 1536, or 2048 samples per block or packet).

[0046] One of the challenges identified by the present disclosure is to reduce redundant operations performed during clustering of audio objects. For example, cluster determination and calculation of object-to-cluster gains are time consuming operations. In gaming and extended reality (XR) (for examples, virtual reality [VR], augmented reality [AR], and mixed reality [MR]) situations, a user may stand still for an extended period of time, causing the audio scene to remain stationary. In such a scenario, time-consuming steps may be skipped to reduce rendering complexity. To achieve these and other goals, a clustering decision module 308 (see FIG. 3A) may be employed to determine (e.g., based on calculation or estimation) whether an audio scene has changed significantly, and then set a clustering decision flag accordingly. The clustering decision flag may be used to determine whether to skip or perform cluster determination and object-to-cluster gains calculations operations.

[0047] FIGS. 3A-3B illustrate a block diagram of a framework 300 for audio object clustering according to some aspects of the present disclosure. The framework 300 can be implemented in the audio processing system 100 according to some aspects of the present disclosure, e.g., as part of the audio clustering module 110. The framework 300 includes an excitation calculation module 302, an excitation buffer 304, an onset / offset detection module 306, and a clustering decision module 308 (shown in FIG. 3A). The framework 300 also includes a decision block 310, a cluster determination and object-to-cluster gains calculation module 312, a clusters and gains buffer 314, and a cluster generation module 316 (shown in FIG. 3B).

[0048] The proposed audio object clustering framework 300 may be operated in a frame-by- frame basis. In each frame, the audio block(s) and the associated metadata may be generated at run-time. In an example operating frame, there may be M audio objects generated at runtime. A fixed number of clusters, denoted by N clusters, may be generated through the proposed approach.

[0049] Beginning with reference to FIG. 3A, the input of the excitation calculation module 302 corresponds to a first path 301. The outputs of the excitation calculation module 302 correspond to a second path 303 and a third path 305. The excitation calculation module 302 may as shown further comprise an output corresponding to a fourth path 331. The excitation calculation module 302 is coupled to the excitation buffer 304 via the second path 303. The excitation calculation module 302 is coupled to the onset / offset detection module 306 via the third path 305. The excitation calculation module 302 may receive input audio objects and associated metadata via the first path 301. The input audio objects may for example correspond to the audio objects 102 of FIG. 1. The excitation calculation module 302 is configured to process the input audio objects and, optionally the associated metadata, to determine (e.g., calculate or estimate) an excitation (or energy) of the audio object. The excitation may herein also be referred to as an excitation measure. For example, the excitation per band (Exband) may be calculated according to Equation 1 :Exband= X.* conjugate(X') Equation (1) where X is a frequency transform of one frame of samples of the audio object (e.g., the block of samples of the frame associated with the audio object). The excitation per band Exbanddefines a power spectrum for the object in the frame. The samples of the audio object may be pulsecode modulation (PCM) samples. The frequency transform may be a Fourier Frequency Transform or a cosine transform, for example. A window function, such as a Hann window, may be applied to the frame to reduce spectral leakage, prior to performing the frequency transform. As another example, the excitation may be the sum of all energy bands (Exau) for the audio object and may be calculated according to Equation 2:Equation (2)In another example, Exatlmay be a root mean square (RMS) value.As another example, an excitation for the audio object may be calculated as the average of the excitation per band Exbandacross all bands, or a selected subset of bands. In another example, an excitation value for the audio object may be calculated as the maximum excitation per bandExbandamong all bands, or a selected subset of bands. As another example, the power spectrum / excitation per band Exband(e.g., determined according to Equation 1) may define an initial excitation measure and in turn be converted into a final excitation measure in the form of an excitation per critical band. Such a conversion may involve performing auditory filtering (e.g., using a filtering bank of gammatone filters or equivalent rectangular bandwidth (ERB) filters) of the power spectrum, calculate the threshold in quiet for each frequency band of the auditory filtered power spectrum, and calculating the excitation pattern (e.g., involving accounting for spectral masking and applying compression to model the non-linear relationship between excitation and intensity).

[0050] The excitation calculation module 302 may optionally determine the excitation to be, or be indicative of, a loudness of the audio object. The loudness may be determined from the excitation per critical band using any suitable psychoacoustic loudness model, such as the Zwicker loudness model or the Moore- Glasberg Loudness model.

[0051] When determining the excitation measure of an input audio object, the excitation calculation module 302 may optionally further take metadata associated with the input audio object into account, such as a channel type associated with the input audio object and / or a state of the input audio object. For example, the excitation calculation module 302 may be configured to: skip (exclude) determining an excitation, or set an excitation value to zero (e.g., for each band), for any input audio object associated with a predefined channel type or set to a predefined state. The predefined channel type may for instance be a Low Frequency Enhancement (LFE) channel, or some other channel type that may be suitable to exclude from the audio object clustering and / or clustering decision method. A predefined state may for instance be a silent state, or an inactive state. In accordance with either of these examples, input audio objects may thus, based on associated object metadata, be excluded or be set to a zero excitation, thus reducing the amount of calculations performed by the excitation calculation module 302. It is further noted that excluded input audio objects, or objects set to zero excitation, may not have any impact on the output of the onset / offset detection module 306 or the clustering decision module 308. Further, such input audio objects may be excluded from cluster candidate selection, as discussed below with reference to a cluster candidate selection module 332.

[0052] As may be appreciated from the above, the excitation (excitation measure) of the audio object may be indicated as a single excitation value (e.g., loudness) for the audio object, or a perband excitation value (e.g., excitation per critical band). Thus, although the following description of the illustrated example of the audio object clustering framework 300 refers to an excitationvalue such as loudness, it is noted that any reference to an “excitation value” in the following more generally may refer to an “excitation” or “excitation measure”.

[0053] The excitation calculation module 302 is configured to provide the excitation value (or more generally, excitation metric) to the excitation buffer 304 via the second path 303. The excitation calculation module 302 is also configured to provide the excitation value (as a current excitation value) to the onset / offset detection module 306 via the third path 305.

[0054] The input of the excitation buffer 304 corresponds to the second path 303. The excitation buffer 304 is coupled to the excitation calculation module 302 via the second path 303. The output of the excitation buffer 304 corresponds to a fifth path 307. The excitation buffer 304 is coupled to the onset / offset detection module 306 via the fifth path 307. The excitation buffer 304 may receive the excitation value from the excitation calculation module 302 via the second path 303. The excitation buffer 304 is configured to store the excitation value as a previous excitation value (e.g., the excitation of the audio object in a previous frame). The excitation buffer is configured to provide the previous excitation value to the onset / offset detection module 306 via the fifth path 307.

[0055] The inputs of the onset / offset detection module 306 corresponds to the third path 305 and the fifth path 307. The onset / offset detection module 306 is coupled to the excitation calculation module 302 via the third path 305. The onset / offset detection module 306 receives current excitation values from the excitation calculation module 302 via the third path 305. The onset / offset detection module 306 is coupled to the excitation buffer 304 via the fifth path 307. The onset / offset detection module 306 receives previous excitation values from the excitation buffer 304 via the fifth path 307. The output of the onset / offset detection module 306 corresponds to a sixth path 309. The onset / offset detection module 306 is coupled to a clustering decision module 308 via the sixth path 309.

[0056] The onset / offset detection module 306 is configured to process the current excitation value (e.g., current excitation metric, such as current loudness) and the previous excitation value (e.g., previous excitation metric, such as previous loudness) for an audio object to detect and indicate if the abrupt change in terms of excitation (e.g., loudness) occurred for the audio object in the current frame. Thus, the change in the audio object may be a temporal change, such as a temporal change in excitation value (loudness) between frames. The onset / offset detection module 306 may indicate the abrupt change by setting an onset / offset flag (e.g., a transition flag). The onset / offset flag may herein refer to a single (i.e., one-bit) flag or a two-bit flag. In the case of a two-bit flag, the onset / offset flag may refer to an onset flag and an offset flag, wherein theonset flag may be set to indicate an abrupt increase in excitation (e.g., loudness) and the offset flag may be set to indicate an abrupt decrease in excitation (e.g., loudness). In the case of a single / one-bit flag, the onset / offset flag may be implemented to indicate an abrupt change in excitation (e.g., loudness) for the audio object between the previous and current frames. The previous and current frames may herein refer to consecutive frames.

[0057] Regardless of whether a one- or two-bit onset / offset flag is used, the onset / offset detection module 306 may for example set the flag based on whether the excitation (e.g., loudness) of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least an excitation change threshold. The excitation change threshold may be an increasing function of the excitation of the audio object in the previous frame. The excitation change threshold may for example be proportional to the excitation of the audio object in the previous frame.

[0058] The onset / offset detection module 306 may be configured to set the flag to a first value (e.g., TRUE) or a second value (e.g., FALSE), wherein a condition for setting the flag to the first value includes that the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least the excitation change threshold, and wherein otherwise the flag is set to the second value. The flag may thus indicate whether excitation of the audio object has changed significantly between the previous and current frame, wherein the degree of change considered as significant is defined by the excitation change threshold. The specific magnitude of the excitation change threshold (e.g., and how it may depend on the excitation of the audio object in the previous frame) may vary depending on implementation, and on the desired responsiveness of the dynamic clustering of the framework 300. For example, a smaller excitation change threshold may result in the flag being set to the first value (TRUE) in response to smaller variations of excitation between frames.

[0059] The condition for setting the onset / offset flag to the first value may further include that the excitation of the audio object in the current frame is above an excitation threshold, and wherein otherwise the flag is set to the second value. In case the excitation of an audio object is small, changes in excitation may have a limited perceptual impact for a listener, even in case the relative change in excitation of the audio object between the previous and current frame is large. Also, in case the relative change in excitation of the audio object between the previous and current frame is large (i.e., exceeding the excitation change threshold), but the excitation of the audio object in the current frame is small, the change in excitation need not motivate determining updated clusters and object- to-cluster gains for the current frame. Appreciating this,thresholding the excitation of the audio object in the current frame by the excitation threshold allows the flag to be set to the second value (FALSE) where the excitation in the current frame is below the excitation threshold, also where the change in excitation of the audio object between the previous and current frame exceeds the excitation change threshold. That is, the flag may be set to the first value (TRUE) only on a condition that the excitation of the audio object in the current frame is above the excitation threshold.

[0060] The onset / offset detection module 306 may further be configured to set the flag based on a change in position of the audio object between the previous and current frame. In this case, the condition for setting the flag to the first value (TRUE) may further include that the change in position is above a distance threshold. That is, the flag may be set to the first value (TRUE) only on a condition that the change in position is above the distance threshold. In an example combining each of the above conditions, it follows that the flag will be set to the first value (TRUE) for an audio object in a current frame only on a condition that each of the following are true: the excitation of the audio object in the current frame is above the excitation threshold, the change in position of the audio object between the previous and current frame is above the distance threshold, and the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least the excitation change threshold. Otherwise, the flag will be set to the second value (FALSE) for the audio object in the current frame.

[0061] In case of the excitation measure includes a per-band excitation value, the excitation change threshold, and / or the excitation threshold, may be applied on a per-band basis. The threshold(s) may in this case be common thresholds, meaning that a same threshold is used for each band. However, it is also possible to define individual thresholds per-band. For example, a per-band excitation change threshold may be defined as an increasing function of the excitation of the audio object in the previous frame for the respective band. The change in excitation level per-band may in this case be compared to the excitation change threshold for the respective bands. As a further example, a per-band excitation threshold may be defined, wherein the excitation of the audio object in the current frame may be compared to the excitation threshold for each respective band. In either case, the respective comparisons may evaluate to true in case the change in excitation between the previous and current frame, or the excitation in the current frame, exceeds the respective threshold in at least a predefined number of bands (e.g., at least one band).

[0062] As one example that takes changes in loudness (e.g., volume) into account, the onset / offset flag for the audio object may be set based on the following rules: 1 ) the onset flag is set to TRUE, if and only if the excitation of the audio object in the current frame is at least C times larger than the excitation of the audio object in the previous frame (otherwise, the onset flag is set to FALSE); 2) the offset flag is set to TRUE, if and only if the excitation of the audio object in the current frame is no larger than 1 / C times the excitation of the audio object in the previous frame (otherwise, the offset flag is set to FALSE); or 3) if the excitation of the audio object in the current frame is lower than a predefined excitation threshold, both the onset flag and the offset flag are set to FALSE. In the example, C is a preset constant (for example, C = 1000). The predefined excitation threshold may be, for example, -80dB. Values of C and the predefined excitation threshold may be obtained via other methods, including statistical calculation, experimental data collection, lookup tables, and the like. The onset / offset detection module 306 provides the onset / offset flags to the clustering decision module 308 via the sixth path 309.

[0063] Evaluating whether excitation of the audio object in the current frame is at least C times larger than the excitation of the audio object in the previous frame, or no larger than 1 / C times the excitation of the audio object in the current frame is at least C times larger than the excitation of the audio object in the previous frame, is equivalent to determining whether the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least an excitation change threshold, wherein the excitation change threshold is an increasing function of the excitation of the audio object in the previous frame. More specifically, the excitation change threshold is in this example proportional to the excitation of the audio object in the previous frame, wherein the C defines the proportionality factor. Referring to the constant C as a scaling factor or multiplicative factor, the evaluation may equivalently be defined as evaluating whether the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least a scaling factor C.

[0064] As another example that takes both loudness and position change into account, the onset / offset flag may be set based on the following rules: 1) the onset flag is set to TRUE, if and only if the excitation of the audio object in the current frame is at least C times larger than the excitation of the audio object in the previous frame and the distance from the current frame position to the last frame position is greater than D (otherwise, the onset flag is set to FALSE); 2) the offset flag is set to TRUE, if and only if the excitation of the audio object in the current frame is no larger than 1 / C times the excitation of the audio object in the previous frame and thedistance from the current frame position to the last frame position is greater than D (otherwise, the offset flag is set to FALSE); or 3) if the excitation of the audio object in the current frame is lower than a predefined excitation threshold, both the onset flag and the offset flag are set to FALSE. In the example, C and D are both preset constants (for example, C = 1000 and D = 0. 1), and the predefined excitation threshold is approximately -80dB.

[0065] The above examples set an onset flag and an offset flag independently of one another and hence exemplify a two-bit onset / offset flag. In these examples, the onset flag set to TRUE and the offset flag set to FALSE may be referred to as a first value of the (two-bit) onset / offset flag, the onset flag set to FALSE and offset flag set to TRUE may be referred to as a third value of the onset / offset flag, and the onset and offset flags both set to FALSE may be referred to as a second value of the onset / offset flag. A setting of both the onset flag and the offset flag to TRUE would, according to these examples, not correspond to a valid setting state of the onset / offset flag. A benefit of configuring both an onset flag and an offset flag in this manner is that different thresholds may be used for flagging onset and offset. For example, the onset flag may be set to TRUE on a condition that the audio object in the current frame has increased relative to the excitation of the audio object in the previous frame by at least a first excitation change threshold (and otherwise be set to FALSE). The offset flag may be set to TRUE on a condition that the audio object in the current frame has decreased relative to the excitation of the audio object in the previous frame by at least a second excitation change threshold (and otherwise be set to FALSE). It is noted that these first and second excitation change thresholds may be combined with the excitation threshold and / or the distance threshold, discussed above.

[0066] While the above examples refer to a two-bit flag in the form of individually set onset and offset flags, an analogous set of rules may be applied to set a one-bit onset / offset flag:

[0067] In an example that takes loudness into account, a one-bit onset / offset flag may be set based on the following rules: 1) the onset / offset flag is set to TRUE (first value), if and only if the excitation of the audio object in the current frame is at least C times larger than the excitation of the audio object in the previous frame, or the excitation of the audio object in the current frame is no larger than 1 / C times the excitation of the audio object in the previous frame (otherwise, the offset flag is set to FALSE); or 2) if the excitation of the audio object in the current frame is lower than a predefined excitation threshold, the onset / offset flag is set to FALSE.

[0068] In an example that takes both loudness and position change into account, a one-bit onset / offset flag may be set based on the following rules: 1) the onset / offset flag is set to TRUE,if and only if the excitation of the audio object in the current frame is at least C times larger than the excitation of the audio object in the previous frame, or the excitation of the audio object in the current frame is no larger than 1 / C times the excitation of the audio object in the previous frame, and the distance from the current frame position to the last frame position is greater than D (otherwise, the offset flag is set to FALSE); or 2) if the excitation of the audio object in the current frame is lower than a predefined excitation threshold, the onset / offset flag is set to FALSE.

[0069] The input of the clustering decision module 308 corresponds to the sixth path 309. The clustering decision module 308 is coupled to the onset / offset detection module 306 via the sixth path 309. The clustering decision module 308 receives the onset / offset flag(s) from the onset / offset detection module 306 via the sixth path 309. The input of the clustering decision module 308 also corresponds to a candidate list path, in Fig. 3A represented by first and second candidate list paths 335, 337. The clustering decision module 308 may receive cluster candidate lists (for example, a current cluster candidate list and / or a previous cluster candidate list) via the first and second candidate list paths 335, 337. The output of the clustering decision module 308 corresponds to a clustering decision output path 311. The clustering decision module 308 is coupled to the decision block 310 via the clustering decision output path 311 (see FIG. 3B).

[0070] The clustering decision module 308 is configured to determine (e.g., based on estimation or calculation) whether the framework 300 should implement the cluster determination and object-to-cluster gains calculation module 312 (shown in FIG. 3B) based on how much an audio scene has changed from the previous frame. If the audio scene has changed significantly, as determined by the clustering decision module 308, the cluster determination and object-to-cluster gains calculation module 312 is implemented by the framework 300. The clustering decision module 308 may indicate whether to implement the cluster determination and object-to-cluster gains calculation module 312 by setting a clustering decision flag.

[0071] As further set out below, the clustering decision module 308 may set the clustering decision flag 311 for the current frame based on whether any audio object in the current frame is a new cluster candidate. For this purpose, the framework 300 may as shown in FIG. 3A further include a cluster candidate selection module 332. An input of the cluster candidate selection module 332 corresponds to the fourth path 331. The cluster candidate selection module 332 is connected to the excitation calculation module 302 via the fourth path 331. The cluster candidate selection module 332 has first and second outputs, corresponding to a candidate list buffer path 333 and the first candidate list path 335, respectively.

[0072] The excitation calculation module 302 may be configured to provide the excitation value to the cluster candidate selection module 332 also via the fourth path 331 . The cluster candidate selection module 332 may be configured to identify (i.e., determine) an audio object in the current frame as a cluster candidate for the current frame on a condition that the excitation value (e.g., as received via the fourth path 331) exceeds a cluster candidate threshold. The cluster candidate threshold may be a predefined threshold. The value of the cluster candidate threshold may be set to exclude audio objects in the current frame that may be considered perceptually insignificant for the purpose of the clustering process. The value of the cluster candidate threshold may for example be set to exclude audio objects estimated to be inaudible. The cluster candidate threshold may be set independently from the above-mentioned excitation threshold. In case the excitation calculation module 302 provides an excitation measure in the form of a perband excitation level, the excitation level per-band may be compared to a cluster candidate threshold and any audio object for which the cluster candidate threshold is exceeded for a predefined number of bands (e.g., at least one band) may be identified as a cluster candidate. Either a common cluster candidate threshold or a respective cluster candidate threshold may be applied to each band. Additionally, where the excitation calculation module 302 further takes metadata associated with input audio objects into account to determine whether to skip determining an excitation for any input audio objects, any audio object skipped by the excitation calculation module 302 may be excluded as cluster candidate by the cluster candidate selection module 332. In either case, any audio object that is determined as a cluster candidate in the current frame may be added to (e.g., recorded in) a current cluster candidate list for the current frame. Adding audio objects to the current cluster candidate list may include adding a respective identifier of the audio objects to the current cluster candidate list. The audio objects may for example be stored in an audio object array, wherein the identifier may be an index or a pointer to the audio object in the array.

[0073] The framework 300 further includes a cluster candidate list buffer 334. An input of the cluster candidate list buffer 334 corresponds to a candidate list buffer path 333. An output of the cluster candidate list buffer 334 corresponds to the second candidate list path 337. The cluster candidate list buffer 334 is connected to the cluster candidate selection module 332 via the candidate list buffer path 333. The cluster candidate list buffer 334 is connected to a further input of the clustering decision module 308 via the second candidate list path 337. The cluster candidate selection module 332 is configured to provide the current cluster candidate list to the clustering decision module 308 via the first candidate list path 335. The cluster candidate selection module 332 is further configured to provide the current cluster candidate list to thecluster candidate list buffer 334 via the candidate list buffer path 333. The cluster candidate list buffer 334 is configured to receive and store the current cluster candidate list received from the cluster candidate selection module 332 via the candidate list buffer path 333 as a previous cluster candidate list for a next frame. The cluster candidate list buffer 334 is configured to provide the previous cluster candidate list for the current frame (i.e., the current cluster candidate list determined for the previous frame) to the clustering decision module 308 via the second candidate list path 337.

[0074] The clustering decision module 308 may in turn be configured to compare the current cluster candidate list and the previous cluster candidate list received via the first and second candidate list paths 335 and 337 to determine whether the current cluster candidate list (i.e., the cluster candidate list for the current frame) includes any new cluster candidates not included in the previous cluster candidate list (i.e., the cluster candidate list for the previous frame).

[0075] In some instances, the clustering decision module 308 may set the clustering decision flag 311 based on the result of a modulo operation using the current frame index and a modulo operator K. The modulo operator K may be a predefined value, such as 2, 3, or the like. The clustering decision module 308 may also set the clustering decision flag based on changes in cluster candidates. Cluster candidates are as discussed above audio objects identified as audible according to some metric (for example, loudness). In such an example, the clustering decision flag may be set based on the following rules: 1) if the result of current frame index modulo K is equal to 0, the clustering decision flag is set to TRUE (otherwise, the clustering decision flag is set to FALSE); or 2) if the result of current frame index modulo K is not 0, but a new cluster candidate for the current frame exists that was not included in a previous cluster candidates list, and the new cluster candidate has either onset or offset (based on the onset and offset flags, e.g., corresponding to a first or third value of a two-bit onset / offset flag), then the clustering decision flag is set to TRUE (otherwise, the clustering decision flag is set to FALSE). The clustering decision module 308 provides the clustering decision flag to the decision block 310 via the clustering decision output path 311.

[0076] As one example, the result of current frame index modulo K is not equal to 0. A previous cluster candidate list includes an audio object 1 and an audio object 2. A current cluster candidate list includes the audio object 1 and an audio object 3. As the audio object 3 is not included in the previous cluster candidate list and the result of current frame index modulo K is not equal to 0, the clustering decision module 308 sets the clustering decision flag based on the onset flag and the offset flag for the audio object 3. If the onset flag or the offset flag are set toTRUE for the audio object 3, the clustering decision module 308 sets the clustering decision flag to TRUE. Otherwise, if the onset flag and the offset flag are set to FALSE for the audio object 3, the clustering decision module 308 sets the clustering decision flag to FALSE.

[0077] As another example, the result of current fame index modulo K is not equal to 0. A previous cluster candidate list includes an audio object 1 and an audio object 2. A current cluster candidate list also includes the audio object 1 and the audio object 2. As there is no new cluster candidate in this frame, the clustering decision module 308 sets the clustering decision flag to FALSE.

[0078] As yet another example, the result of current frame index modulo K is equal to 0. In response, the clustering decision module 308 sets the clustering decision flag to TRUE. By performing a modulo operation between the index frame and the modulo operator K, implementing the cluster determination and object-to-cluster gains calculation module 312 may be avoided in subsequent frames for which the result of current frame index modulo K is not equal to 0, unless a new cluster candidate is introduced (and, in some examples, the onset or offset flag is TRUE). Moreover, in case the result of current frame index modulo K is equal to 0, the cluster determination and object-to-cluster gains calculation module 312 will be enabled for the current frame, meaning that cluster determination and object-to-cluster gains calculation will be performed for the audio objects in the current frame.

[0079] While the above examples describe the clustering decision module 308 with reference to a two-bit flag in the form of an onset flag and an offset flag, an analogous set of rules may be applied based on a one-bit onset / offset flag. In such an example, the clustering decision flag may be set based on the following rules: 1) if the result of current frame index modulo K is equal to 0, the clustering decision flag is set to TRUE (otherwise, the clustering decision flag is set to FALSE); or 2) if the result of current frame index modulo K is not 0, but a new cluster candidate for the current frame exists that was not included in a previous cluster candidates list, and the new cluster candidate has the onset / offset flag set to TRUE (first value), then the clustering decision flag is set to TRUE (otherwise, the clustering decision flag is set to FALSE).

[0080] Referring now to FIG. 3B, the input of the decision block 310 corresponds to the clustering decision output path 311. The decision block 310 receives the clustering decision flag from the clustering decision module 308 via the clustering decision output path 311. The outputs of the decision block 310 correspond to a seventh path 313 and an eighth path 315. The decision block 310 is coupled to the cluster determination and object-to-cluster gains calculation module 312 via the seventh path 313. The decision block 310 is coupled to the cluster generationmodule 316 via the eighth path 315. While illustrated as a separate processing block, the operation of the decision block 310 may alternatively be performed by the clustering decision module 308.

[0081] The decision block 310 is configured to determine whether to utilize the cluster determination and object-to-cluster gains calculation module 312 based on the cluster decision flag provided by the clustering decision module 308. For example, when the cluster decision flag is set to TRUE (“YES” at decision block 310), the framework 300 enables the cluster determination and object-to-cluster gains calculation module 312 using the seventh path 313. When the cluster decision flag is set to FALSE (“NO” at decision block 310), the framework 300 skips the cluster determination and object-to-cluster gains calculation module 312 by skipping to the cluster generation module 316 using the eighth path 315.

[0082] The inputs of the cluster determination and object-to-cluster gains calculation module 312 correspond to the seventh path 313 and a ninth path 317. The cluster determination and object-to-cluster gains calculation module 312 is coupled to the decision block 310 via the seventh path 313. The cluster determination and object-to-cluster gains calculation module 312 may receive input audio objects (e.g., the plurality of audio objects of the frame) and associated metadata via the ninth path 317. The outputs of the cluster determination and object-to-cluster gains calculation module 312 correspond to a tenth path 319 and an eleventh path 321. The cluster determination and object-to-cluster gains calculation module 312 is coupled to the clusters and gains buffer 314 via the tenth path 319. The cluster determination and object-to- cluster gains calculation module 312 is coupled to the cluster generation module 316 via the eleventh path 321. The cluster determination and object-to-cluster gains calculation module 312 is configured to process the input audio objects and the associated metadata to generate object- to-cluster gains. In some instances, both the current and historical audio blocks and associated metadata are used for analysis, where no advanced audio block and metadata may be available.

[0083] The audio object clustering performed by the cluster determination and object-to-cluster gains calculation module 312 may comprise two steps. As a first step, the cluster determination and object-to-cluster gains calculation module 312 may select the most important N objects as cluster centroids. More specifically, the N objects may be selected from the current cluster candidate list output by the cluster candidate selection module 332, that also may be received by the cluster determination and object-to-cluster gains calculation module 312 via a candidate list input path 318 corresponding to the first candidate list path 335. Secondly, the cluster determination and object-to-cluster gains calculation module 312 may calculate the object-to-cluster gains for each object to the N clusters. Therefore, the object-to-cluster gains for each object can be represented by an N-dimensional vector. Optionally, to enable a further reduction of computational complexity, object-to-cluster gains for any audio object not included in the current cluster candidate list may be set to zero. The cluster determination and object-to-cluster gains calculation module 312 is configured to provide the calculated object-to-cluster gains for each object and the cluster centroids to the clusters and gains buffer 314 via the tenth path 319. The cluster determination and object-to-cluster gains calculation module 312 is also configured to provide the calculated object-to-cluster gains for each object and the cluster centroids to the cluster generation module 316 via the eleventh path 321.

[0084] The input of the clusters and gains buffer 314 corresponds to the tenth path 319. The clusters and gains buffer 314 is coupled to the cluster determination and object-to-cluster gains calculation module 312 via the tenth path 319. The clusters and gains buffer 314 receives the calculated object-to-cluster gains for each object and the cluster centroids from the cluster determination and object-to-cluster gains calculation module 312. The output of the clusters and gains buffer 314 corresponds to a twelfth path 323. The clusters and gains buffer 314 is coupled to the cluster generation module 316 via the twelfth path 323. The clusters and gains buffer 314 may be configured to store the calculated object-to-cluster gains and cluster centroids in the operating / current frame, where the object-to-cluster gains and the cluster centroids may be retrieved and implemented in a later frame (for example, in the next frame). The clusters and gains buffer 314 is configured to provide the stored object-to-cluster gains and the stored cluster centroids to the cluster generation module 316 via the twelfth path 323.

[0085] The inputs of the cluster generation module 316 correspond to the eighth path 315, the eleventh path 321 , and the twelfth path 323. The cluster generation module 316 is coupled to the decision block 310 via the eighth path 315. The cluster generation module 316 is coupled to the cluster determination and object-to-cluster gains calculation module 312 via the eleventh path 321. The cluster generation module 316 is configured to receive current object-to-cluster gains and current cluster centroids from the cluster determination and object-to-cluster gains calculation module 312 via the eleventh path 321. The cluster generation module 316 is coupled to the clusters and gains buffer 314 via the twelfth path 323. The cluster generation module 316 is configured to receive the stored (e.g., previous) object-to-cluster gains and stored cluster centroids from the clusters and gains buffer 314 via the twelfth path 323. The output of the cluster generation module 316 corresponds to a thirteenth path 325.

[0086] When the cluster decision flag is set to TRUE (at decision block 310) and the cluster determination and object-to-cluster gains calculation module 312 is enabled, the cluster generation module 316 processes the current object-to-cluster gains and current cluster centroids to generate output clusters. That is, the audio objects in the current frame are clustered using the current object-to-cluster gains and current cluster-centroids to generate output clusters. Alternatively, when the cluster decision flag is set to FALSE (at decision block 310) and the cluster determination and object-to-cluster gains calculation module 312 is skipped, the cluster generation module 316 processes the previous object-to-cluster gains and previous clustercentroids to generate output clusters. That is, the audio objects in the current frame are clustered using the previous object-to-cluster gains and previous cluster-centroids to generate output clusters. The cluster generation module 316 provides the output clusters via the thirteenth path 325. The output clusters may correspond to the M audio objects 106 output by the audio clustering module 110 in FIG. 1.

[0087] The illustrated blocks and modules in the spatial coding method 200 and the framework 300 are merely examples for clustering audio objects. In other examples, the spatial coding method 200 and / or the framework 300 may include additional blocks, may omit blocks, may combine the functions of blocks, or may divide portions of the blocks into additional blocks.

[0088] FIG. 4 illustrates a block diagram of various example methods 400 for clustering audio objects, which may be performed by the framework 300 of FIGS. 3A-3B. The methods 400 may be performed by a processor, which may be configured to perform methods 400 via machineexecutable instructions. The methods 400 may be broken into various blocks or partitions, such as blocks 405, 410, 415, 420, and 425. The various process blocks illustrated in FIG. 4 provide examples of various methods disclosed herein, and it is understood that some blocks may be removed, added, combined, or modified without departing from the spirit of the present disclosure. For some examples, processing of the various blocks, which may be described as processes, methods, steps, blocks, operations, or functions, may commence at block 405.

[0089] At block 405, “Receiving An Input Audio Block Included In A Frame”, an example method 400 may include receiving an input audio object included in a current frame. The audio object may include a block of PCM samples and corresponding metadata. For example and with reference to FIG. 3A, an input audio object and associated metadata is received by the framework 300 via the first path 301. Processing may proceed from block 405 to block 410.

[0090] At block 410, “Calculating Excitation Of The Audio Object In The Current Frame,” an example method 400 may include calculating the excitation of the audio object, in the currentframe, as previously described with respect to the excitation calculation module 302. In some instances, the calculated excitation of the audio object is stored in a buffer, such as the excitation buffer 304. Processing may proceed from block 410 to block 415.

[0091] At block 415, “Enabling Cluster Determination And Object-To-Cluster Gains Calculation?”, an example method 400 may include determining whether to enable cluster determination and object-to-cluster gains calculation operations, as previously described with respect to the onset / offset detection module 306, the clustering decision module 308, and / or the decision block 310.

[0092] When cluster determination and object-to-cluster gains calculation operations are determined to be enabled (“YES” at block 415), processing may proceed from block 415 to block 420. At block 420, “Performing Cluster Determination and Object-To-Cluster Gains Calculation”, an example method 400 may include performing cluster determination and object- to-cluster gains calculation operations, as previously described with respect to the cluster determination and object-to-cluster gains calculation module 312. Processing may proceed from block 420 to block 425.

[0093] At block 425, “Generating Output Clusters Using Current Cluster Centroids And Object- To-Cluster Gains”, an example method 400 may include generating output clusters using current cluster centroids and current object-to-cluster gains, as previously described with respect to the cluster generation module 316.

[0094] Returning to block 415, when cluster determination and object-to-cluster gains calculation operations are determined to not be enabled (“NO” at block 415), processing may proceed from block 415 to block 430. At block 430, “Generating Output Clusters Using Previous Cluster Centroids And Object-To-Cluster Gains,” an example method 400 may include generating output clusters using previous cluster centroids and previous object-to-cluster gains, as previous described with respect to the clusters and gains buffer 314 and the cluster generation module 316.

[0095] FIG. 5A illustrates a schematic block diagram of an example device architecture 500 (e.g., an apparatus 500) that may be used to implement various aspects of the present disclosure. Architecture 500 includes but is not limited to servers and client devices, systems, and methods as described in reference to FIGS. 1-4. As shown, the architecture 500 includes central processing unit (CPU) 501 which is capable of performing various processes in accordance with a program stored in, for example, read only memory (ROM) 502 or a program loaded from, forexample, storage unit 508 to random access memory (RAM) 503. The CPU 501 may be, for example, an electronic processor 501 . Tn RAM 503, the data required when CPU 501 performs the various processes is also stored, as required. CPU 501, ROM 502, and RAM 503 are connected to one another via bus 504. Input / output interface 505 is also connected to bus 504.

[0096] The following components are connected to I / O interface 505: input unit 506, that may include a keyboard, a mouse, or the like; output unit 507 that may include a display such as a liquid crystal display (LCD) and one or more speakers; storage unit 508 including a hard disk, or another suitable storage device; and communication unit 509 including a network interface card such as a network card (e.g., wired or wireless).

[0097] In some implementations, input unit 506 includes one or more microphones in different positions (depending on the host device) enabling capture of audio signals in various formats (e.g., mono, stereo, spatial, immersive, and other suitable formats).

[0098] In some implementations, output unit 507 include systems with various number of speakers. Output unit 507 (depending on the capabilities of the hose device) can render audio signals in various formats (e.g., mono, stereo, immersive, binaural, and other suitable formats).

[0099] In some embodiments, communication unit 509 is configured to communicate with other devices (e.g., via a network). Drive 510 is also connected to I / O interface 505, as required.Removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a flash drive or another suitable removable medium is mounted on drive 510, so that a computer program read therefrom is installed into storage unit 508, as required. A person skilled in the art would understand that although apparatus 500 is described as including the above-described components, in real applications, it is possible to add, remove, and / or replace some of these components and all these modifications or alteration all fall within the scope of the present disclosure.

[0100] In accordance with example embodiments of the present disclosure, the processes described above may be implemented as computer software programs or on a computer-readable storage medium. For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied on a machine readable medium, the computer program including program code for performing methods. In such embodiments, the computer program may be downloaded and mounted from the network via the communication unit 509, and / or installed from the removable medium 51 1, as shown in FIG. 5 A.

[0101] FIG. 5B illustrates a schematic block diagram of an example CPU 501 implemented in the device architecture 500 of FIG. 5A that may be used to implement various aspects of the present disclosure. The CPU 501 includes an electronic processor 520 and a memory 521. The electronic processor 520 is electrically and / or communicatively connected to the memory 521 for bidirectional communication. The memory 521 stores encoding software 522 and / or decoding software 523. In some examples, memory 521 may be located internal to the electronic processor 520, such as for an internal cache memory or some other internally located ROM, RAM, or flash memory. In other examples, memory 521 may be located external to the electronic processor 520, such as in a ROM 502, a RAM 503, flash memory or a removable medium 511 , or another non-transitory computer readable medium that is contemplated for device architecture 500. In some instances, the electronic processor 520 may implement the spatial coding software 522 stored in the memory 521 to perform, among other things, any of the methods 400 of FIG. 4.

[0102] Generally, various example embodiments of the present disclosure may be implemented in hardware or special purpose circuits (e.g., control circuitry), software, logic or any combination thereof. For example, the units and modules discussed above can be executed by control circuitry (e.g., CPU 501 in combination with other components of FIG. 5A), thus, the control circuitry may be performing the actions described in this disclosure. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device (e.g., control circuitry). While various aspects of the example embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representation, it will be appreciated that the blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0103] Additionally, various blocks shown in the flowcharts may be viewed as method steps, and / or as operations that result from operation of computer program code, and / or as a plurality of coupled logic circuit elements constructed to carry out the associated function(s). For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied on a machine readable medium, the computer program containing program codes configured to carry out the methods as described above.

[0104] In the context of the disclosure, a machine-readable medium may be any tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine- readable signal medium or a machine-readable storage medium. A machine-readable medium may be non-transitory and may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] Computer program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These computer program codes may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus that has control circuitry, such that the program codes, when executed by the processor of the computer or other programmable data processing apparatus, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may execute entirely on a computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer or entirely on the remote computer or server or distributed over one or more remote computers and / or servers.

[0106] A person skilled in the art realizes that the present invention by no means is limited to the embodiments described above. On the contrary, many modifications and variations are possible and considered within the scope of the appended claims. Various aspects and implementations of the present disclosure may also be appreciated from the following enumerated example embodiments (EEEs), which are not claims, and which may represent systems, methods, and devices, all arranged in accordance with aspects of the present disclosure.

[0107] EEE-A1. An audio object clustering system that processes audio objects, the system comprising: an onset / offset detection module (306) configured to set a flag based on an excitation of an audio object in a current frame; and a clustering decision module (308) configured to determine whether to enable a cluster determination and object-to-cluster gains calculation module (312) for the current frame based on the flag.

[0108] EEE-A2. The system of EEE-A1, wherein the excitation of the audio object indicates a loudness of the audio object.

[0109] EEE- A3. The system of any of EEE-A1 to EEE-A2, wherein the onset / offset detection module (306) is further configured to set the flag based on a change in the excitation of the audio object relative to an excitation of the audio object in a previous frame.

[0110] EEE-A4. The system of EEE- A3, wherein the excitation of the audio object in the previous frame is stored in an excitation buffer (304).

[0111] EEE-A5. The system of any of EEE-A3 to EEE-A4, wherein the onset / offset detection module (306) is further configured to set the flag based on whether the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least an excitation change threshold.

[0112] EEE-A6. The system of EEE-A5, wherein the excitation change threshold is an increasing function of the excitation of the audio object in the previous frame.

[0113] EEE-A7. The system of any of EEE-A5 to EEE-A6, wherein the onset / offset detection module (306) is configured to set the flag to a first value or a second value, wherein a condition for setting the flag to the first value includes that the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least the excitation change threshold, and wherein otherwise the flag is set to the second value.

[0114] EEE-A8. The system of EEE-A7, wherein the condition for setting the flag to the first value further includes that the excitation of the audio object in the current frame is above an excitation threshold, and wherein otherwise the flag is set to the second value.

[0115] EEE-A9. The system of any of EEE-A7 to 8, wherein the onset / offset detection module (306) is further configured to set the flag based on a change in position of the audio object between the previous and current frame, and wherein the condition for setting the flag to the first value further includes that the change in position is above a distance threshold.

[0116] EEE- Al 0. The system of any of EEE-A7 to EEE-A9, wherein the clustering decision module (308) is configured to enable the cluster determination and object-to-cluster gains calculation module (312) on a condition that the audio object is a new cluster candidate for the current frame and that the flag for the audio object is set to the first value.

[0117] EEE-A11. The system of EEE-A10, wherein the onset / offset detection module (306) is configured to set a respective flag for each of the plurality of audio objects in the current frame to the first or second value based on its respective excitation value, and wherein the clustering decision module (308) is configured to enable the cluster determination and object-to- cluster gains calculation module (312) on a condition that any of the audio objects is a new cluster candidate for the current frame and that the flag for the audio object being a new cluster candidate is set to the first value.

[0118] EEE-A12. The system of any of EEE-A10 to EEE-A11, wherein the clustering decision module (308) is configured to determine that a given audio object is a new cluster candidate on a condition that the excitation of the given audio object is above a cluster candidate threshold, and is not included in a previous cluster candidates list for the previous frame.

[0119] EEE-A13. The system of EEE-A12, wherein the previous cluster candidates list includes audio objects included in the previous frame having an excitation above the cluster candidate threshold.

[0120] EEE-A14. The system of any of EEE-A1 to EEE-A13, wherein the clustering decision module (308) further is configured to enable the cluster determination and object-to- cluster gains calculation module (312) on a condition that a modulo operation performed between the frame index and a predetermined modulo operator is equal to zero.

[0121] EEE-A15. The system of any of EEE-A1 to EEE-A14, wherein the cluster determination and object-to-cluster gains calculation module (312) is configured to select the most perceptually important audio object or audio objects among the plurality of audio objects as cluster centroids.

[0122] EEE-A16. The system of any of EEE-A1 to EEE-A15, wherein the cluster determination and object-to-cluster gains calculation module (312) is configured to calculate the object-to-cluster gains for each object to each cluster.

[0123] EEE-A17. The system of any of EEE-A1 to EEE-A16, further comprising: a cluster generation module (316) configured to: receive and process, when the cluster determination and object-to-cluster gains calculation module (312) is enabled, current cluster centroids and current object-to-cluster gains from the cluster determination and object-to-cluster gains calculation module (312) to generate output clusters, and receive and process, when the cluster determination and object-to-cluster gains calculation module (312) is not enabled,previous cluster centroids and previous object-to-cluster gains from a clusters and gains buffer (314).

[0124] EEE-A18. A method (400) for clustering audio objects, the method (400) comprising: receiving (405) an input audio object included in a current frame; calculating (410) an excitation of the audio object in the current frame; and determining (415) whether to enable a cluster determination and object-to-cluster gains calculation module (312) based on the excitation of the audio object in the current frame.

[0125] EEE- Al 9. The method according to EEE- Al 8, further comprising generating(425, 430) output clusters based on whether the cluster determination and object-to-cluster gains calculation module (312) is enabled, wherein generating (425, 430) output clusters based on whether the cluster determination and object-to-cluster gains calculation module (312) is enabled includes: performing (420), when the cluster determination and object-to-cluster gains calculation module (312) is enabled, cluster determination and object-to-cluster gains calculations on the plurality of audio objects to obtain current cluster centroids and object-to- cluster gains; and generating (425) output clusters using the current cluster centroids and the current object-to-cluster gains.

[0126] EEE-A20. The method (400) of EEE-A19, further comprising: storing the current cluster centroids and the current object-to-cluster gains in a clusters and gains buffer (314).

[0127] EEE-A21. The method (400) of any of EEE-A19 to EEE-A20, wherein generating (425, 430) output clusters based on whether the cluster determination and object-to-cluster gains calculation module (312) is enabled includes: generating (430), when the cluster determination and object-to-cluster gains calculation module (312) is not enabled, output clusters using previous cluster centroids and previous object-to-cluster gains stored in a clusters and gains buffer (314).

[0128] EEE-A22. The method (400) of any of EEE-A18 to EEE-A21, wherein the excitation of the audio object in the current frame indicates a loudness of the audio object.

[0129] EEE-A23. The method (400) of any of EEE-A18 to EEE-A22, further comprising: setting a flag based on the excitation of the audio object in the current frame, wherein determining (415) whether to enable the cluster determination and object-to-cluster gains calculation module (312) is based on the flag.

[0130] EEE-A24. The method (400) of EEE-A23, wherein setting the flag includes setting the flag based on a change in the excitation of the audio object relative to an excitation of the audio object in a previous frame.

[0131] EEE-A25. The method (400) of any of EEE-A23 to EEE-A24, further comprising: storing the excitation of the audio object in an excitation buffer (304).

[0132] EEE-A26. The method of any of EEE-A23 to EEE-A25, wherein setting the flag includes setting the flag based on whether the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least an excitation change threshold.

[0133] EEE-A27. The method of EEE-A26, wherein setting the flag includes setting the flag to a first value or a second value, wherein a condition for setting the flag to the first value includes that the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least the excitation change threshold, and wherein otherwise the flag is set to the second value.

[0134] EEE-A28. The method of EEE-A27, wherein the condition for setting the flag to the first value further includes that the excitation of the audio object in the current frame is above an excitation threshold, and wherein otherwise the flag is set to the second value.

[0135] EEE-A29. The method of any of EEE-A27 to EEE- A28, wherein setting the flag further includes setting the flag based on a change in position of the audio object between the previous and current frame, and wherein the condition for setting the flag to the first value further includes that the change in position is above a distance threshold.

[0136] EEE-A30. The method of any of EEE-A27 to EEE-A29, wherein the cluster determination and object-to-cluster gains calculation module (312) is enabled on a condition that the audio object is a new cluster candidate for the current frame and that the flag for the audio object is set to the first value.

[0137] EEE- A31. The method of EEE-A30, further comprising setting a respective flag for each of the plurality of audio objects in the current frame to the first or second value based on its respective excitation value, and wherein the cluster determination and object-to-cluster gains calculation module (312) is enabled on a condition that any of the audio objects is a new cluster candidate for the current frame and that the flag for the audio object being a new cluster candidate is set to the first value.

[0138] EEE-A32. The method of any of EEE-A30 to EEE- A31 , wherein a given audio object is determined as a new cluster candidate on a condition that the excitation of the given audio object is above a cluster candidate threshold, and is not included in a previous cluster candidates list for the previous frame.

[0139] EEE-A33. The method of EEE-A32, wherein the previous cluster candidates list includes audio objects included in the previous frame having an excitation above the cluster candidate threshold.

[0140] EEE-A34. The method (400) of any of EEE-A18 to EEE-A33, wherein generating (425, 430) output clusters includes selecting the perceptually most important audio object or objects among the plurality of audio objects as cluster centroids.

[0141] EEE-A35. The method (400) of any of EEE-A18 to EEE-A34, wherein the audio object includes a block of pulse-code modulation, PCM, samples and metadata.

[0142] EEE-A36. An apparatus comprising: an electronic processor (520) configured to perform operations including the method (400) of any one of EEE-A18 to EEE- A35.

[0143] EEE-A37. A non-transitory computer-readable storage medium (521) recording a program of instructions that is executable by a device to perform the method (400) of any one of EEE- Al 8 to EEE-A35.

[0144] EEE-B1. An audio object clustering system that processes audio objects generated at runtime, the system comprising: an onset / offset detection module (306) configured to set a flag based on an excitation of an audio object in a current frame; and a clustering decision module (308) configured to determine whether to enable a cluster determination and object-to- cluster gains calculation module (312) for the current frame based on the flag.

[0145] EEE-B2. The system according to EEE-B1, wherein the excitation of the audio object indicates a loudness of the audio object.

[0146] EEE-B3. The system according to any one of EEE-B1 to EEE-B2, wherein the onset / offset detection module (306) is further configured to set the flag based on an excitation of the audio object in a previous frame.

[0147] EEE-B4. The system according to EEE-B3, wherein the excitation of the audio object in the previous frame is stored in an excitation buffer (304).

[0148] EEE-B5. The system according to any one of EEE-B1 to EEE-B4, wherein the onset / offset detection module (306) is configured to set a flag based on the excitation of the audio object in the current frame by determining whether a temporal change occurs in the current frame.

[0149] EEE-B6. The system according to EEE-B5, wherein the temporal change is calculated based on at least one of a received audio block including the audio object and a received metadata associated with the audio object.

[0150] EEE-B7. The system according to EEE-B6, wherein the received audio block, the received metadata, or both are associated with one of a previous frame or a current frame.

[0151] EEE-B8. The system according to any one of EEE-B1 to EEE-B7, wherein the clustering decision module (308) is configured to determine whether to enable the cluster determination and object-to-cluster gains calculation module (312) based on a frame index and based on whether a new cluster candidate having either onset or offset is included in a previous cluster candidates list.

[0152] EEE-B9. The system according to EEE-B8, wherein the clustering decision module (308) is configured to enable the cluster determination and object-to-cluster gains calculation module (312) based on a modulo operation performed between the frame index and a predetermined modulo operator.

[0153] EEE-B10. The system according to any one of EEE-B1 to EEE-B9, wherein the cluster determination and object-to-cluster gains calculation module (312) is configured to select the most important audio objects as cluster centroids.

[0154] EEE-B11. The system according to any one of EEE-B1 to EEE-B10, wherein the cluster determination and object-to-cluster gains calculation module (312) is configured to calculate the object-to-cluster gains for each object for a selected cluster.

[0155] EEE-B 12. The system according to any one of EEE-B 1 to EEE-B 11 , wherein the onset / offset detection module (306) is configured to: set the flag to a first value when the excitation of the audio object is above a threshold; and set the flag to a second value when the excitation of the audio object is below the threshold.

[0156] EEE-B 13. The system according to EEE-B 12, wherein the threshold is based on an audio object included in a previous frame.

[0157] EEE-B14. The system according to any one of EEE-B1 to EEE-B13, further comprising: a cluster generation module (316) configured to: receive, when the cluster determination and object-to-cluster gains calculation module (312) is enabled, current cluster centroids and current object-to-cluster gains from the cluster determination and object-to-cluster gains calculation module (312), and receive, when the cluster determination and object-to-cluster gains calculation module (312) is not enabled, previous cluster centroids and previous object-to- cluster gains from a clusters and gains buffer (314).

[0158] EEE-B15. A method (400) for clustering audio objects, the method (400) comprising: receiving (405) an input audio object included in a current frame, the audio object including a block of pulse-code modulation (PCM) samples and corresponding metadata; calculating (410) an excitation of the audio object in the current frame; determining (415) whether to enable a cluster determination and object-to-cluster gains calculation module (312) based on the excitation of the audio object in the current frame; and generating (425, 430) output clusters based on whether the cluster determination and object-to-cluster gains calculation module (312) is enabled.

[0159] EEE-B16. The method (400) according to EEE-B15, wherein generating (425, 430) output clusters based on whether the cluster determination and object-to-cluster gains calculation module (312) is enabled includes: performing (420), when the cluster determination and object-to-cluster gains calculation module (312) is enabled, cluster determination and object- to-cluster gains calculations on the plurality of audio objects to obtain current cluster centroids and object-to-cluster gains; and generating (425) output clusters using the current cluster centroids and the current object-to-cluster gains.

[0160] EEE-B17. The method according to EEE-B16, further comprising: storing the current cluster centroids and the current object-to-cluster gains in a clusters and gains buffer (314).

[0161] EEE-B 18. The method according to any one of EEE-B 15 to EEE-B 17, wherein generating (425, 430) output clusters based on whether the cluster determination and object-to- cluster gains calculation module (312) is enabled includes: generating (430) output clusters using previous cluster centroids and previous object-to-cluster gains stored in a clusters and gains buffer (314).

[0162] EEE-B 19. The method according to any one of EEE-B 15 to EEE-B 18, wherein the excitation of the audio object in the current frame indicates a loudness of the audio object.

[0163] EEE-B20. The method according to any one of EEE-B15 to EEE-B19, further comprising: setting a flag based on the excitation of the audio object in the current frame, wherein determining (415) whether to enable the cluster determination and object-to-cluster gains calculation module (312) is based on the flag.

[0164] EEE-B21. The method according to EEE-B20, wherein setting the flag includes setting the flag based on the excitation of the audio object in a previous frame.

[0165] EEE-B22. The method according to any one of EEE-B15 to EEE-B21, further comprising: storing the excitation of the audio object in an excitation buffer (304).

[0166] EEE-B23. The method according to any one of EEE-B 15 to EEE-B22, wherein determining (415) whether to enable the cluster determination and object-to-cluster gains calculation module (312) is further based on a frame index and based on whether a new cluster candidate is included in a previous cluster candidates list.

[0167] EEE-B24. The method according to EEE-B23, wherein determining (415) whether to enable the cluster determination and object-to-cluster gains calculation module (312) includes performing a modulo operation between the frame index and a predetermined modulo operator.

[0168] EEE-B25. The method according to any one of EEE-B 15 to EEE-B24, wherein generating (425, 430) output clusters includes selecting the most important audio objects as cluster centroids.

[0169] EEE-B26. An apparatus comprising: an electronic processor (520) configured to perform operations including the method (400) of any one of EEE-B 15 to EEE-B25.

[0170] EEE-B27. A non-transitory computer-readable storage medium (521) recording a program of instructions that is executable by a device to perform the method of any one of EEE- B15 to EEE-B25.

[0171] With regard to the processes, systems, methods, heuristics, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be replaced, amended, or omitted. In other words, thedescriptions of processes herein are provided for the purpose of illustrating certain embodiments and should in no way be construed so as to limit the claims.

[0172] Accordingly, it is to be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided would be apparent upon reading the above description. The scope should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the technologies discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In sum, it should be understood that the application is capable of modification and variation.

[0173] All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those knowledgeable in the technologies described herein unless an explicit indication to the contrary in made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.

[0174] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.

Claims

CLAIMSWhat is claimed is:

1. An audio object clustering system that processes audio objects, the system comprising: an onset / offset detection module (306) configured to set a flag based on an excitation of an audio object in a current frame; and a clustering decision module (308) configured to determine whether to enable a cluster determination and object-to-cluster gains calculation module (312) for the current frame based on the flag.

2. The system of claim 1, wherein the excitation of the audio object indicates a loudness of the audio object.

3. The system of any of claims 1-2, wherein the onset / offset detection module (306) is further configured to set the flag based on a change in the excitation of the audio object relative to an excitation of the audio object in a previous frame.

4. The system of claim 3, wherein the excitation of the audio object in the previous frame is stored in an excitation buffer (304).

5. The system of any of claims 3-4, wherein the onset / offset detection module (306) is further configured to set the flag based on whether the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least an excitation change threshold.

6. The system of claim 5, wherein the excitation change threshold is an increasing function of the excitation of the audio object in the previous frame.

7. The system of claim any of claims 5-6, wherein the onset / offset detection module (306) is configured to set the flag to a first value or a second value, wherein a condition for setting the flag to the first value includes that the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least the excitation change threshold, and wherein otherwise the flag is set to the second value.

8. The system of claim 7, wherein the condition for setting the flag to the first value further includes that the excitation of the audio object in the current frame is above an excitation threshold, and wherein otherwise the flag is set to the second value.

9. The system of any of claims 7-8, wherein the onset / offset detection module (306) is further configured to set the flag based on a change in position of the audio object between the previous and current frame, and wherein the condition for setting the flag to the first value further includes that the change in position is above a distance threshold.

10. The system of any of claims 7-9, wherein the clustering decision module (308) is configured to enable the cluster determination and object-to-cluster gains calculation module (312) on a condition that the audio object is a new cluster candidate for the current frame and that the flag for the audio object is set to the first value.

11. The system of claim 10, wherein the onset / offset detection module (306) is configured to set a respective flag for each of the plurality of audio objects in the current frame to the first or second value based on its respective excitation value, and wherein the clustering decision module (308) is configured to enable the cluster determination and object-to-cluster gains calculation module (312) on a condition that any of the audio objects is a new cluster candidate for the current frame and that the flag for the audio object being a new cluster candidate is set to the first value.

12. The system of any of claims 10-11, wherein the clustering decision module (308) is configured to determine that a given audio object is a new cluster candidate on a condition that the excitation of the given audio object is above a cluster candidate threshold, and is not included in a previous cluster candidates list for the previous frame.

13. The system of claim 12, wherein the previous cluster candidates list includes audio objects included in the previous frame having an excitation above the cluster candidate threshold.

14. The system of any of claims 1-13, wherein the clustering decision module (308) further is configured to enable the cluster determination and object-to-cluster gains calculation module (312) on a condition that a modulo operation performed between the frame index and a predetermined modulo operator is equal to zero.

15. The system of any of claims 1-14, wherein the cluster determination and object-to-cluster gains calculation module (312) is configured to select the most perceptually important audio object or audio objects among the plurality of audio objects as cluster centroids.

16. The system of any of claims 1-15, wherein the cluster determination and object-to-cluster gains calculation module (312) is configured to calculate the object-to-cluster gains for each object to each cluster.

17. The system of any of claims 1-16, further comprising: a cluster generation module (316) configured to: receive and process, when the cluster determination and object-to-cluster gains calculation module (312) is enabled, current cluster centroids and current object-to-cluster gains from the cluster determination and object-to-cluster gains calculation module (312) to generate output clusters, and receive and process, when the cluster determination and object-to-cluster gains calculation module (312) is not enabled, previous cluster centroids and previous object-to-cluster gains from a clusters and gains buffer (314).

18. A method (400) for clustering audio objects, the method (400) comprising: receiving (405) an input audio object included in a current frame; calculating (410) an excitation of the audio object in the current frame; and determining (415) whether to enable a cluster determination and object-to-cluster gains calculation module (312) based on the excitation of the audio object in the current frame.

19. The method of claim 18, further comprising generating (425, 430) output clusters based on whether the cluster determination and object-to-cluster gains calculation module (312) is enabled, wherein generating (425, 430) output clusters based on whether the cluster determination and object-to-cluster gains calculation module (312) is enabled includes: performing (420), when the cluster determination and object-to-cluster gains calculation module (312) is enabled, cluster determination and object-to-cluster gains calculations on the plurality of audio objects to obtain current cluster centroids and object-to-cluster gains; and generating (425) output clusters using the current cluster centroids and the current object- to-cluster gains.

20. The method (400) of claim 19, further comprising:storing the current cluster centroids and the current object-to-cluster gains in a clusters and gains buffer (314).

21. The method (400) of any of claims 19-20, wherein generating (425, 430) output clusters based on whether the cluster determination and object-to-cluster gains calculation module (312) is enabled includes: generating (430), when the cluster determination and object-to-cluster gains calculation module (312) is not enabled, output clusters using previous cluster centroids and previous object- to-cluster gains stored in a clusters and gains buffer (314).

22. The method (400) of any of claims 18-21, wherein the excitation of the audio object in the current frame indicates a loudness of the audio object.

23. The method (400) of any of claims 18-22, further comprising: setting a flag based on the excitation of the audio object in the current frame, wherein determining (415) whether to enable the cluster determination and object-to-cluster gains calculation module (312) is based on the flag.

24. The method (400) of claim 23, wherein setting the flag includes setting the flag based on a change in the excitation of the audio object relative to an excitation of the audio object in a previous frame.

25. The method (400) of any of claims 23-24, further comprising: storing the excitation of the audio object in an excitation buffer (304).

26. The method of any of claims 23-25, wherein setting the flag includes setting the flag based on whether the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least an excitation change threshold.

27. The method of claim 26, wherein setting the flag includes setting the flag to a first value or a second value, wherein a condition for setting the flag to the first value includes that the excitation of the audio object in the current frame has changed relative to the excitation of the audio object in the previous frame by at least the excitation change threshold, and wherein otherwise the flag is set to the second value.

28. The method of claim 27, wherein the condition for setting the flag to the first value further includes that the excitation of the audio object in the current frame is above an excitation threshold, and wherein otherwise the flag is set to the second value.

29. The method of any of claims 27-28, wherein setting the flag further includes setting the flag based on a change in position of the audio object between the previous and current frame, and wherein the condition for setting the flag to the first value further includes that the change in position is above a distance threshold.

30. The method of any of claims 27-29, wherein the cluster determination and object-to-cluster gains calculation module (312) is enabled on a condition that the audio object is a new cluster candidate for the current frame and that the flag for the audio object is set to the first value.

31. The method of claim 30, further comprising setting a respective flag for each of the plurality of audio objects in the current frame to the first or second value based on its respective excitation value, and wherein the cluster determination and object-to-cluster gains calculation module (312) is enabled on a condition that any of the audio objects is a new cluster candidate for the current frame and that the flag for the audio object being a new cluster candidate is set to the first value.

32. The method of any of claims 30-31, wherein a given audio object is determined as a new cluster candidate on a condition that the excitation of the given audio object is above a cluster candidate threshold, and is not included in a previous cluster candidates list for the previous frame.

33. The method of claim 32, wherein the previous cluster candidates list includes audio objects included in the previous frame having an excitation above the cluster candidate threshold.

34. The method (400) of any of claims 18-33, wherein generating (425, 430) output clusters includes selecting the perceptually most important audio object or objects among the plurality of audio objects as cluster centroids.

35. The method (400) of any of claims 18-34, wherein the audio object includes a block of pulse-code modulation, PCM, samples and metadata.

36. An apparatus comprising:an electronic processor (520) configured to perform operations including the method (400) of any one of claims 18-35.

37. A non-transitory computer-readable storage medium (521) recording a program of instructions that is executable by a device to perform the method (400) of any one of claims 18- 35.

Images