Hybrid priority-based rendering system and method for adaptive audio

Abstract

The present invention relates to a hybrid priority-based rendering system and method for adaptive audio. Embodiments are directed to a method for rendering adaptive audio by the following steps: receiving input audio including channel-based audio, audio objects, and dynamic objects, wherein the dynamic objects are classified into a group of low-priority dynamic objects and a group of high-priority dynamic objects; rendering the channel-based audio, audio objects, and low-priority dynamic objects in a first rendering processor of an audio processing system; and rendering the high-priority dynamic objects in a second rendering processor of the audio processing system. The rendered audio then undergoes virtualization and post-processing steps for playback through sound bars and other similar speakers with limited height capabilities.

Description

[0001] This application is a divisional application of the invention patent application with application number 202010452760.1, application date February 4, 2016, and invention name “Hybrid priority-based rendering system and method for adaptive audio”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 113,268, filed on February 6, 2015, which is hereby incorporated by reference in its entirety. Technical Field

[0004] One or more implementations relate generally to audio signal processing and, more particularly, to a hybrid priority-based rendering strategy for adaptive audio content. Background Art

[0005] The introduction of digital cinema and the development of true three-dimensional ("3D") or virtual 3D content has created new sound standards, such as the incorporation of multiple channels of audio to allow greater creativity for content creators and a more immersive and realistic listening experience for audiences. As a means for distributing spatial audio, extending beyond traditional speaker feeds and channel-based audio is critical, and there has been considerable interest in model-based audio descriptions that allow listeners to select a desired playback configuration, rendering the audio specifically for their selected configuration. The spatial rendering of sound utilizes audio objects, which are audio signals with associated parametric source descriptions of apparent source positions (e.g., 3D coordinates), apparent source widths, and other parameters. Further developments include that next-generation spatial audio (also known as "adaptive audio") formats have been developed, which include a mix of audio objects and traditional channel-based speaker feeds, along with positional metadata for the audio objects. In a spatial audio decoder, the channels are transmitted directly to their associated speakers, or downmixed to an existing set of speakers, and the audio objects are rendered by the decoder in a flexible (adaptive) manner. A parameterized source description (such as a position track in 3D space) associated with each object is taken as input along with the number and position of speakers connected to the decoder. The renderer then uses certain algorithms (such as panning laws) to distribute the audio associated with each object across the attached set of speakers. The authored spatial intent of each object is thus best rendered on the specific speaker configuration present in the listening room.

[0006] The advent of advanced object-based audio has significantly increased the nature of the audio content transmitted to various speaker arrays and the complexity of the rendering process. For example, a cinema soundtrack may include many different sound elements corresponding to images on the screen, dialogue, noises, and sound effects emanating from different places on the screen, and combined with background music and ambient effects to create an overall auditory experience. Accurate playback requires that the sound be reproduced in a way that corresponds as closely as possible to the content displayed on the screen in terms of sound source location, intensity, movement, and depth.

[0007] Although advanced 3D audio systems such as Atmos TM Most of the systems are designed and deployed for cinema applications, but consumer-level systems are being developed to bring cinema-level, adaptive audio experiences to home and office environments. Compared with cinemas, these environments are subject to significant constraints in terms of venue size, acoustic characteristics, system power, and speaker configuration. Current professional-level spatial audio systems therefore need to be suitable for rendering advanced object audio content to listening environments characterized by different speaker configurations and playback capabilities. To this end, certain virtualization technologies have been developed to expand the capabilities of traditional stereo or surround sound speaker arrays, thereby reconstructing spatial sound cues by using complex rendering algorithms and technologies (such as content-related rendering algorithms, reflected sound transmission, etc.). Such rendering technologies have led to the development of DSP-based renderers and circuits optimized for rendering different types of adaptive audio content (such as object audio metadata content (OAMD) beds and ISF (intermediate spatial format) objects). Different DSP circuits have been developed to utilize the different characteristics of adaptive audio about rendering specific OAMD content. However, such a multi-processor system needs to be optimized for the memory bandwidth and processing power of each processor.

[0008]

[0006] What is needed is a system for providing scalable processor loading for two or more processors in a multi-processor rendering system for adaptive audio.

[0009] The increasing adoption of surround sound and cinema-based audio in the home has also led to the development of different types and configurations of speakers beyond the standard two-way or three-way upright or bookshelf speakers. Different speakers have been developed to play back specific content, such as soundbar speakers as part of a 5.1 or 7.1 system. A soundbar refers to a class of speakers in which two or more drivers are juxtaposed in a single housing (speaker cabinet) and are typically arranged along a single axis. For example, a popular soundbar typically includes 4-6 speakers arranged in a row in a rectangular cabinet that is designed to be mounted on top of, below, or directly in front of a television or computer monitor to transmit sound directly out of the screen. Due to the configuration of the soundbar, certain virtualization techniques may be difficult to implement compared to speakers that provide height cues through physical placement (e.g., height drivers) or other techniques.

[0010] There is therefore a further need for a system that optimizes adaptive audio virtualization techniques for playback through a soundbar speaker system.

[0011] The subject matter discussed in the Background section should not be assumed to be prior art simply because it is mentioned in the Background section. Similarly, the problems mentioned in the Background section or problems associated with the subject matter in the Background section should not be assumed to have been previously recognized in the prior art. The subject matter in the Background section merely represents different approaches, which may also be inventions in their own right. Dolby, Dolby TrueHD, and Atmos are trademarks of Dolby Laboratories Licensing Corporation. Summary of the invention

[0012] An embodiment of a method for rendering adaptive audio is described by receiving input audio including channel-based audio, audio objects, and dynamic objects, wherein the dynamic objects are classified into a set of low priority dynamic objects and a set of high priority dynamic objects; rendering the channel-based audio, the audio objects, and the low priority dynamic objects in a first rendering processor of an audio processing system; and rendering the high priority dynamic objects in a second rendering processor of the audio processing system. The input audio may be formatted according to a digital bitstream format of object-based audio including audio content and rendering metadata. The channel-based audio includes a surround sound audio bed, and the audio objects include objects conforming to an intermediate spatial format. The low priority dynamic objects and the high priority dynamic objects are distinguished by a priority threshold, which may be defined by one of: an author of the audio content including the input audio, a user selected value, and an automated process performed by the audio processing system. In an embodiment, the priority threshold is encoded in an object audio metadata bitstream. The relative priority of the audio objects of the low priority audio objects and the high priority audio objects may be determined by their respective positions in the object audio metadata bitstream.

[0013] In an embodiment, the method further comprises: passing the high priority audio object through the first rendering processor to a second rendering processor during or after the channel-based audio, audio objects, and low priority dynamic objects are rendered in the first rendering processor to generate rendered audio; and post-processing the rendered audio for transmission to the speaker system. The post-processing step includes at least one of: upmixing, volume control, equalization, bass management, and a virtualization step for facilitating rendering of height cues present in the input audio for playback through the speaker system.

[0014] In an embodiment, the speaker system includes a soundbar speaker having a plurality of collocated drivers that transmit sound along a single axis, and the first rendering processor and the second rendering processor are embodied in separate digital signal processing circuits coupled together by a transmission link. The priority threshold is determined by at least one of: a relative processing capability of the first rendering processor and the second rendering processor, a memory bandwidth associated with each of the first rendering processor and the second rendering processor, and a transmission bandwidth of the transmission link.

[0015] Embodiments are further directed to a method for rendering adaptive audio by the following steps: receiving an input audio bitstream including audio components and associated metadata, each of the audio components having an audio type selected from the following: channel-based audio, audio objects, and dynamic objects; determining a decoder format for each audio component based on the respective audio type; determining the priority of each audio component according to a priority field in the metadata associated with each audio component; rendering the audio components of the first priority type in a first rendering processor; and rendering the audio components of the second priority type in a second rendering processor. The first rendering processor and the second rendering processor are implemented as separate rendering digital signal processors (DSPs) coupled to each other via a transmission link. The audio components of the first priority type include low priority dynamic objects, and the audio components of the second priority type include high priority dynamic objects, and the method further includes rendering the channel-based audio and audio objects in the first rendering processor. In an embodiment, the channel-based audio includes a surround sound audio bed, the audio objects include objects conforming to an intermediate spatial format (ISF), and the low priority dynamic objects and the high priority dynamic objects include objects conforming to an object audio metadata (OAMD) format. The decoder format of each audio component produces at least one of: an OAMD formatted dynamic object, a surround sound audio bed, and an ISF object. The method may further include applying virtualization processing to at least the high priority dynamic object to facilitate rendering of height cues present in the input audio for playback through a speaker system, and the speaker system may include a soundbar speaker having multiple collocated drivers that transmit sound along a single axis.

[0016] Embodiments are further directed to digital signal processing systems implementing the aforementioned methods and / or loudspeaker systems including circuits implementing at least some of the aforementioned methods.

[0017] Incorporation by Reference

[0018] Each publication, patent, and / or patent application mentioned in this specification is herein incorporated by reference in its entirety, to the same extent as if each individual publication and / or patent application was specifically and individually indicated to be incorporated by reference. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In the following figures, like reference numerals are used to refer to like elements.Although the following figures depict various examples, one or more implementations are not limited to the examples depicted in the figures.

[0020] Figure 1 Exemplary speaker placement in a surround system (eg, 9.1 surround) that provides height speakers for playback of height channels is illustrated.

[0021] Figure 2Combining channel-based data and object-based data to generate an adaptive audio mix is ​​illustrated under one embodiment.

[0022] Figure 3 is a table illustrating the types of audio content processed in a hybrid priority-based system, under one embodiment.

[0023] Figure 4 is a block diagram of a multi-processor rendering system for implementing a hybrid priority-based rendering strategy, under one embodiment.

[0024] Figure 5 In one embodiment Figure 4 A more detailed block diagram of a multi-processor rendering system.

[0025] Figure 6 A method for implementing priority-based rendering for playback of adaptive audio content through a soundbar is illustrated under one embodiment.

[0026] Figure 7 A soundbar speaker that may be used with embodiments of a hybrid priority-based rendering system is illustrated.

[0027] Figure 8 The use of a priority-based adaptive audio rendering system in an exemplary television and soundbar consumer use case is illustrated.

[0028] Fig. 9 The use of a priority-based adaptive audio rendering system in an exemplary full surround sound home environment is illustrated.

[0029] Fig.10 is a table illustrating some exemplary metadata definitions in an adaptive audio system utilizing priority-based rendering for a soundbar, under one embodiment.

[0030] Fig.11 An intermediate spatial format for use with a rendering system under some embodiments is illustrated.

[0031] Fig.12 An arrangement of rings in a stacked-ring format translation space is illustrated for use with an intermediate space format under one embodiment.

[0032] Fig.13 Illustrate speaker arcs that illustrate how audio objects are panned to angles used in an ISF processing system, under one embodiment.

[0033] Fig.14A -C illustrates the decoding of the stacked ring intermediate space format under different embodiments. DETAILED DESCRIPTION

[0034] Systems and methods for a hybrid priority-based rendering strategy are described, wherein an object audio metadata (OAMD) bed or an intermediate spatial format (ISF) object is rendered using a temporal object audio renderer (OAR) component on a first DSP component, while an OAMD dynamic object is rendered by a virtual renderer in a post-processing chain on a second DSP component. The output audio can be optimized for playback through a soundbar speaker by one or more post-processing and virtualization techniques. Aspects of one or more embodiments described herein can be implemented in an audio or audio-visual system that processes source audio information in a mixing, rendering, and playback system including one or more computers or processing devices that execute software instructions. Any of the described embodiments can be used alone or in any combination with each other. Although various embodiments may have been inspired by various defects of the prior art that may be discussed or implied in one or more places in this specification, the embodiments do not necessarily solve any of these defects. In other words, different embodiments may solve different defects that may be discussed in this specification. Some embodiments may only partially solve some of the defects that may be discussed in this specification or only one defect, and some embodiments may not solve any of these defects.

[0035] For the purposes of this description, the following terms have associated meanings: the term "channel" means an audio signal plus metadata in which the position is encoded as a channel identifier, e.g., left front or right upper surround; "channel-based audio" is audio formatted for playback through a predefined set of speaker zones with associated nominal locations (e.g., 5.1, 7.1, etc.); the term "object" or "object-based audio" means one or more audio channels with a parameterized source description such as apparent source position (e.g., 3D coordinates), apparent source width, etc.; "adaptive audio" means a channel-based and / or object-based audio signal plus metadata that renders the audio signal based on the playback environment, using an audio stream plus metadata in which the position is encoded as a 3D position in space; and "listening environment" means any open, partially enclosed, or fully enclosed area, such as a room that can be used for playback of audio content alone or with video or other content, and can be embodied in a home, a cinema, a theater, an auditorium, a studio, a gaming console, etc. Such an area may have one or more surfaces disposed therein, such as walls or baffles, that may directly or indirectly reflect sound waves.

[0036] Adaptive audio formats and systems

[0037] In an embodiment, the interconnect system is implemented as part of an audio system configured to work with a sound format and processing system, which may be referred to as a "spatial audio system" or an "adaptive audio system". Such a system is based on audio formats and rendering techniques to allow for enhanced audience immersion, better artistic control, and system flexibility and scalability. The entire adaptive audio system generally includes an audio encoding, distribution, and decoding system that is configured to generate one or more bitstreams containing both conventional channel-based audio elements and audio object coding elements. Such a combined approach provides better coding efficiency and rendering flexibility than using a channel-based approach or an object-based approach separately.

[0038] An exemplary implementation of an adaptive audio system and associated audio formats is Atmos TM This system includes a height (upper / lower) dimension that can be implemented as a 9.1 surround system or similar surround sound configuration. Figure 1 1 illustrates speaker placement in a current surround system (e.g., 9.1 surround) that provides height speakers for playback of height channels. The speaker configuration of the 9.1 system 100 consists of five speakers 102 in the floor plane and four speakers 104 in the height plane. In general, these speakers can be used to generate sounds designed to emanate more or less accurately from any position in the room. Predefined speaker configurations (such as Figure 1 Those shown) can naturally limit the ability to accurately represent the location of a given sound source. For example, a sound source cannot be panned further to the left than the left speaker itself. This applies to each speaker, thus forming a one-dimensional (e.g., left-right), two-dimensional (e.g., front-back), or three-dimensional (e.g., left-right, front-back, top-bottom) geometry in which the downmix is ​​constrained. A variety of different speaker configurations and types can be used in such a speaker configuration. For example, some enhanced audio systems can use speakers with 9.1, 11.1, 13.1, 19.4 or other configurations. Speaker types can include full-range direct speakers, speaker arrays, surround speakers, subwoofers, tweeters, and other types of speakers.

[0039] Audio objects can be thought of as groups of sound elements that can be perceived as emanating from a specific physical location or locations in the listening environment. Such objects can be static (stationary) or dynamic (moving). Audio objects are controlled by metadata that defines the location of the sound at a given point in time, as well as other functions. When objects are played back, they are rendered using the existing speakers, according to the position metadata, and not necessarily output to predefined physical channels. Tracks in a session can be audio objects, and standard panning data is similar to position metadata. In this way, content placed on the screen can be effectively panned in the same way as channel-based content, but if necessary, content placed around can be rendered to individual speakers. While the use of audio objects provides the desired control over discrete effects, other aspects of soundtracks can work effectively in a channel-based environment. For example, many environmental effects or reverberations actually benefit from being fed to a speaker array. Although these can be viewed as objects with a width sufficient to fill the array, it is beneficial to retain some channel-based functionality.

[0040] The adaptive audio system is configured to support audio beds in addition to audio objects, where a bed is effectively a channel-based sub-mix or stem. These can either be delivered separately for final playback (rendering) or combined into a single bed, depending on the content creator's intent. These beds can be created in different channel-based configurations (such as 5.1, 7.1, and 9.1) and arrays including overhead speakers (such as Figure 1 shown). Figure 2 Combining channel-based data and object-based data to generate an adaptive audio mix is ​​illustrated under one embodiment. As shown in process 200, channel-based data 202 (e.g., 5.1 or 7.1 surround sound data provided in the form of pulse code modulation (PCM) data) is combined with audio object data 204 to generate an adaptive audio mix 208. The audio object data 204 is generated by combining elements of the original channel-based data with associated metadata that specifies certain parameters related to the location of the audio object. Figure 2 As conceptually illustrated in , the authoring tool provides the ability to simultaneously create an audio program that contains a combination of speaker channel groups and object channels. For example, an audio program may contain one or more speaker channels, optionally organized into groups (or tracks, e.g., stereo or 5.1 tracks), descriptive metadata for the one or more speaker channels, one or more object channels, and descriptive metadata for the one or more object channels.

[0041] In an embodiment, Figure 2 The bed audio component and the object audio component may include content conforming to a specific formatting standard. Figure 3 exemplifies the types of audio content that may be processed in a hybrid priority-based rendering system under one embodiment. Figure 3 As shown in Table 300 of , there are two main types of content, channel-based content that is relatively static in terms of trajectory and dynamic content that moves between speakers or drivers in the system. Channel-based content can be embodied in an OAMD bed, and dynamic content is prioritized as OAMD objects of at least two priority levels (low priority and high priority). Dynamic objects can be formatted according to certain object formatting parameters and classified into certain types of objects, such as ISF objects. The ISF format is described in more detail later in this description.

[0042] The priority of a dynamic object reflects certain characteristics of the object, such as content type (e.g., dialogue vs. effects vs. ambient sounds), processing requirements, memory requirements (e.g., high bandwidth vs. low bandwidth), and other similar characteristics. In an embodiment, the priority of each object is defined along a scale and encoded in a priority field that is included as part of the bitstream that encapsulates the audio object. The priority can be set to a scalar value, such as an integer value of 1 (lowest) to 10 (highest), or to a binary flag (0 low / 1 high) or other similar encodable priority setting mechanism. The priority level is generally set once for each object by the content creator, who can determine the priority of each object based on one or more of the characteristics mentioned above.

[0043] In alternative embodiments, the priority levels of at least some objects may be set by a user, or by automated dynamics processing that may modify the default priority level of objects based on certain runtime criteria (such as dynamics processor load, object loudness, environmental changes, system failures, user preferences, acoustic customizations, etc.).

[0044] In an embodiment, the priority level of a dynamic object determines the processing of the object in a multi-processor rendering system. The encoded priority level of each object is decoded to determine which processor (DSP) of a dual DSP or multi-DSP system will be used to render that particular object. This enables the use of a priority-based rendering strategy when rendering adaptive audio content. Figure 4 is a block diagram of a multi-processor rendering system for implementing a hybrid priority-based rendering strategy, under one embodiment. Figure 4A multi-processor rendering system 400 is shown that includes two DSP components 406 and 410. The two DSPs are contained within two separate rendering subsystems, a decoding / rendering component 404 and a rendering / post-processing component 408. These rendering subsystems generally include processing blocks that perform traditional object and channel audio decoding, object rendering, channel remapping, and signal processing before the audio is sent to further post-processing and / or amplification stages and speaker stages.

[0045] The system 400 is configured to render and play back audio content generated by one or more capture components, preprocessing components, creation components, and encoding components that encode input audio into a digital bit stream 402. The adaptive audio component can be used to automatically generate appropriate metadata by analyzing the input audio by checking factors such as source interval and content type. For example, position metadata can be derived from a multi-channel recording by analyzing the relative levels of the relevant inputs between channel pairs. The detection of content types (such as speech or music) can be achieved, for example, by feature extraction and classification. Certain creation tools allow the creation of audio programs by optimizing the input and arrangement of the recording engineer's creation intent, so that he can create a final audio mix optimized for playback in almost any playback environment at one time. This can be achieved by using audio objects and position metadata associated with and encoded together with the original audio content. Once the adaptive audio content has been created and encoded in an appropriate codec device, it is decoded and rendered to be played back through a speaker 414.

[0046] like Figure 4 As shown, object audio including object metadata and channel audio including channel metadata are input as input audio bitstreams to one or more decoder circuits within the decoding / rendering subsystem 404. The input audio bitstream 402 contains information related to various audio components (such as Figure 3 4 (shown in the figure), including OAMD beds, low priority dynamic objects, and high priority dynamic objects. The priority assigned to each audio object determines which of the two DSPs 406 or 410 performs the rendering process for that particular object. The OAMD bed and low priority objects are rendered in DSP 406 (DSP1), while the high priority objects are passed through the rendering subsystem 404 for rendering in DSP 410 (DSP 2). The rendered bed, low priority objects, and high priority objects are then input to a post-processing component 412 in subsystem 408 to generate an output audio signal 413, which is transmitted for playback through speakers 414.

[0047] In an embodiment, a priority level that distinguishes low priority objects from high priority objects is set within the priority of the bitstream that encodes the metadata for each associated object. The cutoff or threshold between low priority and high priority can be set to a value along the priority range, such as a value of 5 or 7 along a priority scale of 1 to 10, or a simple detector for a binary priority flag of 0 or 1. The priority level of each object can be decoded in a priority determination component within the decoding subsystem 402 to route each object to the appropriate DSP (DPS1 or DSP2) for rendering.

[0048] Figure 4 The multi-processing architecture of the DSP facilitates efficient processing of different types of adaptive audio beds and objects based on the specific configuration and capabilities of the DSPs and the bandwidth / processing capabilities of the network and processor components. In an embodiment, DSP1 is optimized to render OAMD beds and ISF objects, but may not be configured to optimally render OAMD dynamic objects, while DSP2 is optimized to render OAMD dynamic objects. For this application, OAMD dynamic objects in the input audio are assigned a high priority level so that they are passed to DPS2 for rendering, while beds and ISF objects are rendered in DSP1. This allows the appropriate DSP to render the audio component or components that it can render best.

[0049] In addition to or in lieu of the type of audio components being rendered (e.g., bed / ISF objects vs. OAMD dynamic objects), routing and distributed rendering of audio components can be performed based on certain performance-related metrics, such as based on the relative processing power of the two DSPs and / or the bandwidth of the transmission network between the two DSPs. Thus, if one DSP is significantly more powerful than the other, and the network bandwidth is sufficient to transmit the unrendered audio data, the priority level can be set so that the more powerful DSP is required to render more of the audio components. For example, if DSP2 is much more powerful than DSP1, it can be configured to render all OAMD dynamic objects, or all objects regardless of format, assuming it is capable of rendering these other types of objects.

[0050] In an embodiment, certain application-specific parameters (such as room configuration information, user selections, processing / network constraints, etc.) may be fed back to the object rendering system to allow for dynamic changes in object priority levels. The prioritized audio data is then processed through one or more signal processing stages such as equalizers and limiters before being output for playback through the speakers 414.

[0051] It should be noted that system 400 represents an example of a playback system for adaptive audio, and other configurations, components, and interconnections are possible. Figure 32 illustrates two rendering DSPs for processing dynamic objects divided into two types of priorities. Additional numbers of DSPs may be included to provide greater processing power and more priority levels. Thus, N DSPs may be used for N different priority levels, such as three DSPs for high, medium, low priority, and so on.

[0052] In an embodiment, Figure 4 DSP 406 and 410 shown in are implemented as separate devices coupled together through a physical transmission interface or network. Each DSP can be included in a separate component or subsystem (such as subsystem 404 and 408 shown), or they can be separate components included in the same subsystem (such as an integrated decoder / renderer component). Alternatively, DSP 406 and 410 can be separate processing components in a monolithic integrated circuit device.

[0053] Example Implementation

[0054] As described above, the initial implementation of the adaptive audio format was in the context of digital cinema, including content capture (objects and channels), which was created using novel authoring tools, encapsulated using adaptive audio cinema encoders, and distributed using PCM or a proprietary lossless codec using the existing Digital Cinema Initiative (DCI) distribution mechanism. In this case, the audio content is intended to be decoded and rendered in the digital cinema to create an immersive spatial audio cinema experience. However, it is now imperative to deliver the enhanced user experience provided by the adaptive audio format directly to consumers at home. This requires that certain features of the format and system are suitable for use in more restricted listening environments. For the purpose of description, the term "consumer-based environment" is intended to include any non-cinema environment, including listening environments used by ordinary consumers or professionals, such as houses, studios, rooms, console areas, auditoriums, etc.

[0055] Current authoring and distribution systems for consumer audio create and deliver audio intended for reproduction to predefined and fixed speaker locations, with limited knowledge of the type of content conveyed in the audio essence (i.e., the actual audio played back by the consumer reproduction system). However, the adaptive audio system provides a new hybrid approach to audio creation that includes options for both audio specific to fixed speaker locations (left channel, right channel, etc.) and object-based audio elements with generalized 3D spatial information including position, size, and velocity. The hybrid approach provides a method that balances the fidelity (provided by fixed speaker locations) and flexibility of rendering (generalized audio objects). The system also provides additional useful information about the audio content via new metadata that is paired with the audio essence by the content creator at content creation / authoring time. This information provides detailed information about the properties of the audio that can be used during rendering. Such attributes may include content type (e.g., dialogue, music, effects, voiceover, background / ambience, etc.) as well as audio object information such as spatial properties (e.g., 3D position, object size, velocity, etc.) and useful rendering information (e.g., alignment to speaker locations, channel weights, gains, bass management information, etc.). Audio content and rendering intent metadata may be created either manually by the content creator or by using automated media intelligence algorithms that may run in the background during the authoring process and may be reviewed by the content creator during a final quality control stage, if desired.

[0056] Figure 5 is a block diagram of a priority-based rendering system for rendering different types of channel-based components and object-based components, and is a block diagram of a priority-based rendering system for rendering different types of channel-based components and object-based components according to an embodiment Figure 4 A more detailed illustration of the system shown. Figure 5 As shown, the system 500 processes an encoded input bitstream 506 carrying both a mixed object stream (one or more) and a channel-based audio stream (one or more). The bitstream is processed by rendering / signal processing blocks as indicated by 502, 504, both of which represent or are implemented as separate DSP devices. The rendering functions performed in these processing blocks implement various rendering algorithms for adaptive audio as well as certain post-processing algorithms (such as upmixing), etc.

[0057] The priority-based rendering system 500 includes two main components, a decoding / rendering stage 502 and a rendering / post-processing stage 504. An input bitstream 506 is provided to the decoding / rendering stage via an HDMI (High Definition Multimedia Interface), but other interfaces are possible. A bitstream detection component 508 parses the bitstream and directs the different audio components to the appropriate decoder, such as a Dolby Digital Plus decoder, a MAT 2.0 decoder, a TrueHD decoder, etc. The decoder generates various formatted audio signals, such as OAMD bed signals and ISF or OAMD dynamic objects.

[0058] The decoding / rendering stage 502 includes an OAR (object audio renderer) interface 510, which includes an OAMD processing component 512, an OAR component 514, and a dynamic object extraction component 516. The dynamic object extraction component 516 takes the output from all decoders and separates out the bed, ISF objects and any low priority dynamic objects and high priority dynamic objects. The bed, ISF objects and low priority dynamic objects are sent to the OAR component 514. For the example embodiment shown, the OAR component 514 represents the core of the processor (e.g., DSP) circuit of the decoding / rendering stage 502 and renders to a fixed 5.1.2 channel output format (e.g., standard 5.1+2 height channels), but other surround sound plus height configurations are also possible, such as 7.1.4, etc. The rendered output 513 of the OAR component 514 is then transmitted to the digital audio processor (DAP) component of the rendering / post-processing stage 504. This stage performs functions such as: upmixing, rendering / virtualization, volume control, equalization, bass management, and other possible functions. In an example embodiment, the output 522 of the rendering / post-processing stage 504 comprises a 5.1.2 speaker feed.The rendering / post-processing stage 504 may be implemented as any suitable processing circuitry, such as a processor, DSP or similar device.

[0059] In an embodiment, the output signal 522 is transmitted to a sound bar or a sound bar array. Figure 5As shown in the specific use case example shown in , the soundbar also utilizes a priority-based rendering strategy to support the use case of MAT2.0 input with 31.1 objects without overlapping the memory bandwidth between the two stages 502 and 504. In the exemplary implementation, the memory bandwidth allows up to 32 audio channels to be read and written from the external memory at 48kHz. Because 8 channels are required for the 5.1.2-channel rendering output 513 of the OAR component 514, a maximum of 24 OAMD dynamic objects can be rendered by the virtual renderer in the rendering / post-processing stage 504. If there are more than 24 OAMD dynamic objects in the input bitstream 506, the additional lowest priority objects must be rendered by the OAR component 514 on the decoding / rendering stage 502. The priority of the dynamic objects is determined based on their position in the OAMD stream (e.g., highest priority objects first, lowest priority objects last).

[0060] although Figure 4 and Figure 5 The embodiments are described with respect to beds and objects conforming to the OAMD and ISF formats, but it should be understood that the priority-based rendering scheme using a multi-processor rendering system can be used with any type of adaptive audio content including channel-based audio and two or more types of audio objects, where the object types can be distinguished based on relative priority levels. An appropriate rendering processor (e.g., DSP) can be configured to optimally render all types or only one type of audio object type and / or channel-based audio components.

[0061] Figure 5 The system 500 exemplifies a rendering system that adapts the OAMD audio format to work with a specific rendering application that involves channel-based beds, ISF objects, and OAMD dynamic objects and renders for playback on a soundbar. The system implements a priority-based rendering strategy that addresses certain implementation complexity issues for recreating adaptive audio content through a soundbar or similar collocated speaker system. Figure 6 is a flow chart illustrating a method for implementing priority-based rendering for playback of adaptive audio content through a soundbar, under one embodiment. Figure 6 The processing 600 is generally indicated in Figure 5Method steps performed in a priority-based rendering system 500 of the system. After receiving an input audio bitstream, audio components including a channel-based bed and audio objects of different formats are input to appropriate decoder circuits for decoding, 602. The audio objects include dynamic objects that may be formatted using different format schemes and may be distinguished based on the relative priority encoded with each object, 604. The processing determines the priority level of the object compared to a defined priority threshold by reading the appropriate metadata field within the bitstream for each dynamic audio object. The priority threshold that distinguishes low priority objects from high priority objects may be programmed into the system as a hardwired value set by the content creator, or it may be dynamically set by user input, automated means, or other adaptive mechanisms. The channel-based bed and low priority dynamic objects are then rendered in a first DSP of the system along with any objects that are optimized to be rendered in the first DSP, 606. The high priority dynamic objects are passed along to a second DSP where they are then rendered, 608. The rendered audio components are then passed through some optional post-processing steps for playback through a soundbar or soundbar array, 610 .

[0062] Soundbar Implementation

[0063] like Figure 4 As shown in , the prioritized rendered audio output generated by the two DSPs is transmitted to the sound bar for playback to the user. Considering the popularity of flat screen televisions, sound bar speakers have become increasingly popular. Such televisions have become very thin and relatively light to optimize portability and installation options, despite providing ever-increasing screen sizes at affordable prices. However, considering space, power and cost constraints, the sound quality of these televisions is usually very poor. Sound bars are typically fashionable powered speakers that are placed below a flat screen television to improve the quality of the television audio and can be used alone or as part of a surround sound speaker setup. Figure 7 A soundbar speaker that can be used with an embodiment of a hybrid priority-based rendering system is illustrated. As shown in system 700, the soundbar speaker includes a cabinet 701 that houses a number of drivers 703, which are arranged along a horizontal (or vertical) axis to drive sound directly out of the front of the cabinet. Any practical number of drivers 703 can be used depending on the size and system constraints, with a typical number in the range of 2-6 drivers. The drivers can be the same size and shape, or they can be an array of different drivers, such as a larger central driver for lower frequency sounds. An HDMI input interface 702 can be provided to allow direct interface with a high-definition audio system.

[0064] The soundbar system 700 can be a passive speaker system without onboard power and amplification and with minimal passive circuits. It can also be a powered system in which one or more components are mounted in a cabinet or tightly coupled by external components. Such functions and components include power supply and amplification 704, audio processing (e.g., EQ, bass control, etc.) 706, A / V surround sound processor 708, and adaptive audio virtualization 710. For the purpose of description, the term "driver" means a single electroacoustic transducer that generates sound in response to an electrical audio input signal. The driver can be implemented as any appropriate type, geometry, and size, and can include a speaker, a cone, a ribbon transducer, etc. The term "speaker" means one or more drivers within an overall housing.

[0065] The virtualization functionality provided in component 710 for the soundbar 700 or as a component of the rendering / post-processing stage 504 allows for the implementation of an adaptive audio system in a local application such as a television, computer, gaming console, or similar device, and allows for spatial playback of the audio through speakers arranged in a plane corresponding to the viewing screen or monitor surface. Figure 8 The use of a priority-based adaptive rendering system in an exemplary TV and soundbar consumer use case is illustrated. In general, the TV use case presents challenges in creating an immersive consumer experience based on the potentially limited speaker locations / configurations in terms of spatial resolution (i.e., no surround or rear speakers) and the generally reduced quality of the devices (TV speakers, soundbar speakers, etc.). Figure 8 The system 800 includes speakers in the left and right locations of a standard television set (TV-L and TV-R) and possibly optional left up-firing drivers and right up-firing drivers (TV-LH and TV-RH). The system also includes Figure 7 As previously mentioned, the size and quality of television speakers are reduced due to cost constraints and design choices compared to standalone or home theater speakers. However, the use of dynamic virtualization in conjunction with the soundbar 700 can help overcome these drawbacks. Figure 8 The sound bar 700 is shown with forward-firing drivers and possibly side-firing drivers, all of which are aligned along the horizontal axis of the sound bar cabinet. Figure 8, a dynamic virtualization effect is instantiated for the soundbar speakers so that a person at a particular listening position 804 will hear horizontal elements associated with appropriate audio objects rendered individually in the horizontal plane. Height elements associated with the appropriate audio objects can be rendered by dynamic control of speaker virtualization algorithm parameters based on object spatial information provided by the adaptive audio content to provide at least a partially immersive user experience. For collocated speakers of the soundbar, this dynamic virtualization can be used to create the perception of objects moving along the sides of the room or other horizontal planar sound track effects. This allows the soundbar to provide spatial cues that would otherwise not be present due to the absence of surround or rear speakers.

[0066] In an embodiment, the soundbar 700 may include non-colocated drivers, such as upward firing drivers that utilize sound reflections to allow for virtualization algorithms that provide height cues. Certain drivers may be configured to radiate sound in different directions to other drivers, for example, one or more drivers may implement a steerable sound beam with individually controlled sound zones.

[0067] In an embodiment, the soundbar 700 can be used as part of a full surround sound system with height speakers or height-enabled floor-standing speakers. Such an implementation would allow the soundbar to virtually expand the immersive sound provided by the surround speaker array. Fig. 9 The use of a priority-based adaptive audio rendering system in an exemplary full surround sound home environment is illustrated. As shown in system 900, a sound bar 700 associated with a television or monitor 802 is used in conjunction with a surround sound array of speakers 904, such as in a 5.1.2 configuration as shown. For this case, the sound bar 700 can include an A / V surround sound processor 708 to drive the surround speakers and provide at least a portion of the rendering and virtualization processing. Fig. 9 The system merely illustrates a possible set of components and functionality that may be provided by an adaptive audio system, and certain aspects may be reduced or removed based on the needs of the user while still providing an enhanced experience.

[0068] Fig. 9 The use of dynamic speaker virtualization is illustrated to provide an immersive user experience in a listening environment in addition to the immersive user experience provided by the soundbar. A separate virtualizer can be used for each relevant object, and the combined signal can be sent to the L speaker and the R speaker to create a multi-object virtualization effect. As an example, the dynamic virtualization effect is shown for the L speaker and the R speaker. These speakers can be used together with the audio object size and position information to create a diffuse or point source near-field audio experience. Similar virtualization effects can also be applied to any or all of the other speakers in the system.

[0069] In an embodiment, the adaptive audio system includes a component that generates metadata from the original spatial audio format. The method and components of the system 500 include an audio rendering system that is configured to process one or more bitstreams containing both conventional channel-based audio elements and audio object coding elements. A new extension layer containing audio object coding elements is defined and added to any one of the channel-based audio codec bitstream or the audio object bitstream. The method can implement a bitstream including an extension layer that will be processed by a renderer for existing speaker and driver designs or next-generation speakers that utilize individually addressable drivers and driver definitions. The spatial audio content from the spatial audio processor includes audio objects, channels, and position metadata. When the object is rendered, it is assigned to one or more drivers of a soundbar or soundbar array based on the position metadata and the location of the playback speaker. Metadata is generated in an audio workstation in response to an engineer's mixing input to provide a rendering queue that controls spatial parameters (e.g., position, speed, intensity, timbre, etc.) and specifies which (which) drivers or speakers in the listening environment play their respective sounds during the presentation. The metadata is associated with the respective audio data in the workstation for packaging and shipping of the spatial audio processor. Fig.10 is a table illustrating some exemplary metadata definitions used in an adaptive audio system utilizing priority-based rendering for a soundbar, under one embodiment. Fig.10 As shown in Table 1000 of , some metadata may include elements that define the type of audio content (e.g., dialogue, music, etc.) and certain audio characteristics (e.g., direct, diffuse, etc.). For a priority-based rendering system that plays through a soundbar, the driver definitions included in the metadata may include configuration information (e.g., driver type, size, power, built-in A / V, virtualization, etc.) for the playback soundbar and other speakers that may be used with the soundbar (e.g., other surround speakers or virtualization-enabled speakers). Figure 5 , the metadata may also include fields and data defining the decoder type (e.g., Digital Plus, TrueHD, etc.), from which specific formats of channel-based audio and dynamic objects (e.g., OAMD beds, ISF objects, dynamic OAMD objects, etc.) may be derived. Alternatively, the format of each object may be explicitly defined by a specific associated metadata element. The metadata also includes a priority field for dynamic objects, and the associated metadata may be expressed as a scalar value (e.g., 1 to 10) or a binary priority flag (high / low). Fig.10 The metadata elements shown are intended to illustrate only some of the possible metadata elements that may be encoded in a bitstream transmitting an adaptive audio signal, and many other metadata elements and formats are possible.

[0070] Intermediate Space Format

[0071] As described above for one or more embodiments, some of the objects processed by the system are ISF objects. ISF is a format that optimizes the operation of an audio object panner by dividing the panning operation into two parts: a time-varying part and a static part. In general, an audio object panner separates a monophonic object (e.g., Object i ) is panned to N loudspeakers to perform the operation, whereby the panning gain is calculated according to the loudspeaker locations (x1, y1, z1),…, (x N ,y N ,z N ) and the target location XYZ i (t). These gain values ​​will change continuously over time because the object locations will be time-varying. The goal of the intermediate spatial format is simply to divide this panning operation into two parts. The first part (which will be time-varying) uses the object locations. The second part (which uses a fixed matrix) will be configured based solely on the speaker locations. Fig.11 An intermediate spatial format for use with a rendering system in some embodiments is illustrated. As shown in diagram 1100, a spatial panner 1102 receives object and speaker location information for decoding by a speaker decoder 1106. Between these two processing blocks 1102 and 1106, an audio object scene is represented in a K-channel intermediate spatial format (ISF) 1104. A plurality of audio objects (1 <= i <= N i ) can be processed by separate spatial shifters, the outputs of which are added together to form the ISF signal 1104, so that a K-channel ISF signal set can contain N i In some embodiments, the encoder may also be given information about the height of the speakers via elevation restriction data so that detailed knowledge of the altitude of the playback speakers can be used by the spatial panner 1102 .

[0072] In an embodiment, the spatial panner 1102 is not given detailed information about the locations of the playback speakers. However, it is assumed that the locations of a series of "virtual speakers" are limited to a number of levels or layers and that the distribution within each level or layer is approximate. Therefore, although the spatial panner is not given detailed information about the locations of the playback speakers, some reasonable assumptions can generally be made about the approximate number of speakers and the approximate distribution of those speakers.

[0073] The quality of the resulting playback experience (i.e., its Fig.11The matching closeness of the audio object panners can be improved either by increasing the number of channels K or by gathering more knowledge about the most likely playback speaker placement. Specifically, in an embodiment, Fig.12 As shown, the speaker heights are divided into several planes. The desired constituent sound field can be considered as a series of sound events emanating from arbitrary directions around the listener. The locations of the sound events can be considered to be confined to the surface of a sphere 1202 centered on the listener. Sound field formats (such as High Order Ambisonics) are defined in a way that allows the sound field to be further rendered in a (fairly) arbitrary speaker array. However, the typical playback system envisioned is likely to be constrained in the sense that the heights of the speakers are fixed in 3 planes (the ear height plane, the ceiling plane, and the ground). Therefore, the concept of an ideal spherical sound field can be modified, in which the sound field consists of sound-emitting objects located in a ring at various heights on the surface of a sphere around the listener. For example, Fig.12 , which has a vertex ring, an upper ring, an intermediate ring, and a lower ring. If necessary, an additional ring at the bottom of the sphere (the lowest point, which is strictly speaking also a point rather than a ring) may also be included for the purpose of completeness. In addition, there may be more or fewer rings in other embodiments.

[0074] In an embodiment, the stacked ring format is named BH9.5.0.1, where four numbers indicate the number of channels in the middle ring, upper ring, lower ring, and apex ring, respectively. The total number of channels in a multi-channel bundle will be equal to the sum of these four numbers (so the BH9.5.0.1 format contains 15 channels). Another example format that uses all four rings is BH15.9.5.1. For this format, the channel naming and ordering will be as follows: [M1, M2, ... M15, U1, U2 ... U9, L1, L2, ... L5, Z1], where the channels are arranged in rings (in the order M, U, L, Z), and within each ring they are simply numbered in ascending cardinality order. Each ring can be considered to be filled by a set of nominal speakers spread evenly around the ring. Therefore, the channels in each ring will correspond to specific decode angles, starting with channel 1 (which will correspond to 0° azimuth (straight ahead)) and enumerating in counter-clockwise order (so channel 2 will be to the left of center from the listener's perspective). Therefore, the azimuth angle of channel n will be (where N is the number of channels in the loop, and n ranges from 1 to N).

[0075] Regarding certain use cases of object_priority related to ISF, OAMD generally allows each ring in the ISF to have an object_priority value separately. In an embodiment, these priority values ​​are used in a variety of ways to perform additional processing. First, the height ring and the lower plane ring are rendered by a minimum / suboptimal renderer, while the important listener plane ring can be rendered by a more complex / higher precision high-quality renderer. Similarly, in the encoding format, more bits (i.e., higher quality encoding) can be used for the listener plane ring, and fewer bits can be used for the height ring and the ground ring. This is possible in ISF because it uses rings, which is generally not possible in traditional high-order high-fidelity stereo formats because each different channel is a polar-pattern that interacts in a way that impairs the overall audio quality. In general, a slight decrease in the rendering quality of the height ring or the ground ring is not overly harmful because the content in these rings generally only contains atmosphere content.

[0076] In an embodiment, a rendering and sound processing system encodes a spatial audio scene using two or more rings, wherein different rings represent different spatially separated components of a sound field. Audio objects are translated within a ring according to a reusable translation curve, and audio objects are translated between rings using a non-reusable translation curve. The different spatially separated components are separated based on their vertical axes (i.e., as vertically stacked rings). Sound field elements are transmitted in the form of "nominal speakers" within each ring; and sound field elements within each ring are transmitted in the form of spatial frequency components. For each ring, a decoding matrix is ​​generated by concatenating pre-computed sub-matrices representing the segments of the ring. If there is no speaker in the first ring, the sound from one ring to another can be redirected.

[0077] In the ISF processing system, the location of each speaker in the playback array can be expressed in terms of coordinates (x,y,z) (which is the location of each speaker relative to a candidate listening position near the center of the array). Furthermore, the (x,y,z) vector can be converted to a unit vector to effectively project each speaker location onto the surface of a unit sphere:

[0078] Speaker Location:

[0079] Speaker unit vector:

[0080] Fig.131300 illustrates a speaker arc of angles used in an ISF processing system to which an audio object is panned, under one embodiment. Diagram 1300 illustrates a scenario where an audio object (o) is sequentially panned through a number of speakers 1302 such that a listener 1304 experiences the illusion that the audio object is moving through a trajectory that passes through each speaker sequentially. Without loss of generality, it is assumed that the unit vectors of the speakers 1302 are arranged along a ring in the horizontal plane such that the location of an audio object can be defined as a function of its azimuth angle φ. Fig.13 In the example, the audio object passes through the speakers A, B and C at an angle φ (where these speakers are placed at azimuth angles φ respectively). A ,φ B and φ C ). Audio object panners (e.g. Fig.11 The panner 1102 in will typically use speaker gains to pan the audio object to each speaker, where the speaker gains are functions of an angle φ. The audio object panner may use a panning curve having the following properties: (1) when the audio object is panned to a position that coincides with a physical speaker location, the coincident speaker is used to exclude all other speakers; (2) when the audio object is panned to an angle φ between two speaker locations, only those two speakers are active, thereby providing a minimal amount of "spreading" of the audio signal across the speaker array; and (3) the panning curve may exhibit a high level of "discreteness," meaning that the portion of the panning curve energy is constrained to the region between a speaker and its nearest neighbor. Thus, with reference to Fig.13 , for speaker B:

[0081]

[0082] Therefore, d B ≤1, and when d B = 1, which implies that the panning curve for speaker B is only A and φ C (the angular positions of speakers A and C, respectively) are completely constrained (spatially) to be non-zero. In contrast, the above-mentioned “discreteness” property (i.e., d B <1) may exhibit one other important property: the translation curves are spatially smoothed so that they are constrained in spatial frequency in order to satisfy the Nyquist sampling theorem.

[0083] Any panning curve that is spatially band-limited cannot be compact in its spatial support. In other words, these panning curves will spread over a wide angular range. The term "stopband ripple" refers to the (undesirable) non-zero gain that appears in the panning curve. These panning curves have the problem of not being too "discrete" by satisfying the Nyquist sampling theorem. By being appropriately "Nyquist sampled", these panning curves can be moved to alternative speaker locations. This means that a set of speaker signals that have been created for a specific arrangement of N speakers (these speakers are evenly spaced in a circle) can be remixed to an alternative set of N speakers at different angular locations (remixed with an N×N matrix); that is, the speaker array can be rotated to a new set of angular speaker locations, and the original N speaker signals can be repurposed to this new set of N speakers. In general, this "repurposeable" property allows the system to remap N speaker signals to S speakers via an S×N matrix, provided that for the case of S>N, it is acceptable that the new speaker feeds are no longer "discrete" than the original N channels.

[0084] In an embodiment, the intermediate spatial format of the stacked rings represents each object according to its (time-varying) (x, y, z) location by:

[0085] 1. Place object i at (x i ,y i ,z i ) and assume that the location is in the cube (so |x i |≤1,|y i |≤1 and -|z i |≤1) or in the unit sphere Inside.

[0086] 2. Use vertical location (z i ) to pan the audio signal of object i to each of several (R) spatial regions according to a non-repurposable panning curve.

[0087] 3. With N r Each spatial region (i.e., region r: 1≤r≤R) is represented in the form of a nominal loudspeaker signal (according to Figure 4 , which represents the audio component located in the annular region of the space), the N r The nominal loudspeaker signals are created using a transformable panning curve that is the azimuth (φ) of object i. i ) function.

[0088] Note that for a ring of size zero (according to Fig.12, vertex ring), step 3 above is unnecessary because the ring will contain at most one channel.

[0089] like Fig.11 As shown, the ISF signals 1104 for K channels are decoded in the speaker decoder 1106 . Fig.14A -C illustrates the decoding of the intermediate spatial format of the stacked rings under different embodiments. Fig.14A The stacked ring format is illustrated to be decoded into separate rings. Fig. 14B The stacked ring format is illustrated for decoding without a vertex speaker. Fig. 14C The stacked ring format is illustrated for decoding without apex or ceiling speakers.

[0090] Although the embodiments are described above with respect to ISF objects as one type of object in contrast to dynamic OAMD objects, it should be noted that audio objects formatted in a different format but distinguishable from dynamic OAMD objects may also be used.

[0091] The various aspects of the audio environment described herein represent the playback of audio or audio / visual content through appropriate speakers and playback devices, and can represent any environment in which the listener is experiencing the playback of the captured content, such as a theater, concert hall, amphitheater, home or room, listening booth, car, game console, headphone or headset system, public address (PA) system, or any other playback environment. Although the embodiments have been described primarily with respect to examples and implementations in a home theater environment in which spatial audio content is associated with television content, it should be noted that the embodiments can also be implemented in other consumer-based systems, such as games, projection systems, and any other monitor-based A / V systems. Spatial audio content, including object-based audio and channel-based audio, can be used in conjunction with any related content (associated audio, video, graphics, etc.), or it can constitute independent audio content. The playback environment can be any appropriate listening environment from headphones or near-field monitors to small or large rooms, cars, amphitheaters, concert halls, etc.

[0092] The various aspects of the system described herein can be implemented in a suitable computer-based processing network environment for processing digital or digitized audio files. The parts of the adaptive audio system can include one or more networks, which include any desired number of individual machines, including one or more routers (not shown) for buffering and routing data transmitted between computers. Such a network can be built on a variety of different network protocols and can be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof. In an embodiment where the network includes the Internet, one or more machines can be configured to access the Internet through a web browser program.

[0093] One or more of the components, blocks, processes, or other functional components may be implemented by a computer program that controls the execution of a processor-based computing device of the system. It should also be noted that the various functions disclosed herein may be described using any number of combinations of hardware, firmware, and / or data and / or instructions contained in various machine-readable or computer-readable media, with respect to their behavior, register transfers, logical components, and / or other characteristics. Computer-readable media in which such formatted data and / or instructions may be contained include, but are not limited to, various forms of physical (non-transitory) non-volatile storage media, such as optical, magnetic, or semiconductor storage media.

[0094] Unless the context clearly requires otherwise, throughout the specification and claims, the words "comprises," "including," and the like are to be interpreted in an inclusive sense, as distinct from an exclusive or exhaustive sense; that is, in the sense of "including, but not limited to." Words using the singular or plural number also include the plural or singular number, respectively. In addition, the words "herein," "herein below," "above," "below," and words of similar meaning refer to this application as a whole and not to any particular part of this application. When the word "or" is used in reference to a list of two or more items, the word encompasses all of the following interpretations of the word: any one of the items in the list, all of the items in the list, and any combination of the items in the list.

[0095] Reference throughout this specification to "one embodiment," "some embodiments," or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosed system(s) and method(s). Thus, the appearances of the phrases "in one embodiment," "in some embodiments," or "in an embodiment" in various places throughout this specification may or may not necessarily refer to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner apparent to one of ordinary skill in the art.

[0096] Although one or more implementations have been described with respect to specific embodiments by way of example, it is to be understood that one or more implementations are not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and similar arrangements that are apparent to those skilled in the art. Therefore, the scope of the appended claims should be given the broadest interpretation so as to include all such modifications and similar arrangements.

Claims

1. A method for rendering adaptive audio of an input audio bitstream, comprising: receiving an input audio bitstream, the input audio bitstream comprising at least a dynamic object, wherein the dynamic object is related to an audio object capable of moving, wherein the dynamic object is classified into a low priority dynamic object and a high priority dynamic object based on a priority value; determining whether the dynamic object is a low priority dynamic object or whether the dynamic object is a high priority dynamic object; and When the dynamic object is a low priority dynamic object, rendering the dynamic object based on a first rendering process, or when the dynamic object is a high priority dynamic object, rendering the dynamic object based on a second rendering process, wherein the first rendering process uses a different memory bandwidth than the second rendering process, wherein the determining comprises classifying the dynamic object as a low priority dynamic object or a high priority dynamic object based on a comparison of the priority value with a priority threshold, and Wherein the first rendering process or the second rendering process is selected based on the classification, and the rendering of the channel-based audio is independent of the classification.

2. The method of claim 1 further comprising post-processing the rendered audio for transmission to a speaker system.

3. The method of claim 2, wherein: The post-processing includes at least one of: upmixing, volume control, equalization, and bass management.

4. The method of claim 3, wherein: The post-processing also includes a virtualization step to facilitate rendering of height cues present in the input audio bitstream for playback through a speaker system.

5. The method of claim 1, wherein: The first rendering process is performed in a first rendering processor that is optimized to render channel-based audio and static objects; and The second rendering process is performed in a second rendering processor optimized to render the high priority object through at least one of increased performance capability, increased memory bandwidth, and increased transmission bandwidth of the second rendering processor relative to the first rendering processor.

6. The method of claim 5, wherein: The first rendering processor and the second rendering processor are implemented as separate rendering digital signal processors DSP coupled to each other via a transmission link.

7. The method of claim 1, wherein: The priority threshold is defined by one of: a preset value, a user-selected value, and an automated process.

8. The method of claim 1, wherein: High priority dynamic objects can be determined by their positions in the input audio bitstream.

9. A non-transitory computer-readable storage medium comprising instructions which, when executed by a processor, perform the method of any one of claims 1-8.

10. A system for rendering adaptive audio of an input audio bitstream, comprising: An interface for receiving an input audio bitstream, the input audio bitstream comprising at least a dynamic object, wherein the dynamic object is related to an audio object capable of moving, wherein the dynamic object is classified as a low priority dynamic object or a high priority dynamic object based on a priority value; a decoding / rendering stage, the decoding / rendering stage determining whether the dynamic object is a low priority dynamic object or whether the dynamic object is a high priority dynamic object, and rendering the dynamic object based on a first rendering process when the dynamic object is a low priority dynamic object, or rendering the dynamic object based on a second rendering process when the dynamic object is a high priority dynamic object, wherein the first rendering process uses a different memory bandwidth than the second rendering process to render the dynamic object, Wherein the decoding / rendering stage classifies the dynamic object as a low priority object or a high priority object based on a comparison of the priority value with a priority threshold, and wherein the first rendering process or the second rendering process is selected based on the classification.

11. The system of claim 10, wherein: The decoding / rendering stage is also configured to post-process the rendered audio for transmission to a speaker system.

12. The system of claim 11, wherein: The post-processing includes at least one of: upmixing, volume control, equalization, and bass management.

13. The system of claim 12, wherein: The post-processing also includes a virtualization step to facilitate rendering of height cues present in the input audio for playback through a speaker system.

14. The system of claim 10, wherein: The decoding / rendering stage also includes a first rendering process is performed in a first rendering processor, the first rendering processor being optimized to render channel-based audio and static objects; and The second rendering process is performed in a second rendering processor optimized to render the high priority object through at least one of increased performance capability, increased memory bandwidth, and increased transmission bandwidth of the second rendering processor relative to the first rendering processor.

15. The system of claim 14, wherein: The first rendering processor and the second rendering processor are implemented as separate rendering digital signal processors DSP coupled to each other via a transmission link.

16. The system of claim 10, wherein: The priority threshold is defined by one of: a preset value, a user-selected value, and an automated process.

17. The system of claim 10, wherein: High priority audio objects can be determined by their positions in the input audio bitstream.

Images