Rendering of audio objects with apparent size to arbitrary loudspeaker layouts

Abstract

To provide a rendering method for an audio object with an apparent size to arbitrary loudspeaker layouts.SOLUTION: A run time process for computing a gain value regarding a received audio object in accordance with a previously computed gain value regarding a virtual source location includes the steps of: receiving audio reproduction data including one or more audio objects; computing contributions from virtual sources within an area or volume defined by audio object position data and audio object size data; and computing a set of audio object gain values for each of a plurality of output channels based on the computed contributions.SELECTED DRAWING: Figure 5C

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to Spanish Patent Application No. P201330461, filed March 28, 2013, and U.S. Provisional Patent Application No. 61 / 833,581, filed June 11, 2013, the contents of each of which are incorporated herein by reference in their entirety.

[0002] Technical Field FIELD OF THE DISCLOSURE This disclosure relates to authoring and rendering audio reproduction data, and in particular to authoring and rendering audio reproduction data for reproduction environments such as movie theater sound reproduction systems. [Background technology]

[0003] Since the introduction of sound to motion pictures in 1927, there has been a steady advancement in the technology used to capture the artistic intent of the film soundtrack and reproduce it in a cinema environment. In the 1930s, synchronized sound on disc gave way to variable-area sound on film, which was further improved in the 1940s with consideration of theatre acoustics and improved speaker design, along with the early introduction of multitrack recording and steerable playback (using control tones to move the sound). In the 1950s and 1960s, magnetic stripes on film allowed for multichannel playback in theatres, introducing surround channels and up to five screen channels in premium theatres.

[0004] In the 1970s, Dolby introduced noise reduction, both in post-production and on film, along with a cost-effective means of encoding and distributing a mix of three screen channels and a mono surround channel. Cinema sound quality was further improved in the 1980s with Dolby Spectral Recording (SR) noise reduction and certification programs such as THX. Dolby brought digital sound to cinemas in the 1990s with the 5.1 channel format, which gave discrete left, center and right screen channels, left and right surround arrays, and a subwoofer channel for low-frequency effects. Dolby Surround 7.1, introduced in 2010, increased the number of surround channels by splitting the existing left and right surround channels into four "zones".

[0005] As channel counts increase and speaker layouts transition from planar, two-dimensional (2D) arrays to three-dimensional (3D) arrays that include height, the task of authoring and rendering sound becomes increasingly complex. Improved methods and apparatus would be desirable. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] V. Pulkki, Compensating Displacement of Amplitude-Panned Virtual Sources, Audio Engineering Society (AES) International Conference on Virtual, Synthetic and Entertainment Audio [Non-Patent Document 2] D. de Vries, Wave Field Synthesis, AES Monograph 1999 Summary of the Invention [Means for solving the problem]

[0007] Some aspects of the subject matter described in this disclosure can be implemented in a tool for rendering audio playback data including audio objects that are generated without reference to any particular playback environment. As used herein, the term "audio object" may refer to a stream of audio signals and associated metadata. The metadata may indicate at least the position and apparent size of the audio object. However, the metadata may also indicate rendering constraint data, content type data (e.g., dialogue, effects, etc.), gain data, trajectory data, etc. Some audio objects may be static, while other audio objects may have time-varying metadata: such audio objects may move, change size, and / or have other attributes that change over time.

[0008] When the audio object is monitored or played in a playback environment, the audio object may be rendered according to at least the position and size metadata. The rendering process may involve computing a set of audio object gain values ​​for each channel of a set of output channels. Each output channel may correspond to one or more playback speakers of the playback environment.

[0009] Some implementations described herein involve a "setup" process that may occur prior to rendering any particular audio object. This setup process, sometimes referred to herein as first stage or stage 1, may involve defining multiple virtual source positions within a volume in which the audio object can move. As used herein, a "virtual source position" is a position of a static point source. According to such implementations, the setup process may involve receiving playback speaker position data and pre-calculating virtual source gain values ​​for each of the virtual sources according to the playback speaker position data and the virtual source positions. As used herein, "speaker position data" may include position data that indicates the positions of some or all of the speakers of the playback environment. The position data may be provided as absolute coordinates of the playback speaker positions, e.g., Cartesian coordinates, spherical coordinates, etc. Alternatively or additionally, the position data may be provided as coordinates (e.g., Cartesian or angular coordinates) relative to other playback environment positions, such as an acoustic "sweet spot" of the playback environment.

[0010] In some implementations, the virtual source gain values ​​may be stored and used during "runtime" when audio playback data is rendered for speakers of the playback environment. During runtime, for each audio object, a contribution from a virtual source position within an area or volume defined by the audio object position data and the audio object size data may be calculated. The process of calculating the contribution from the virtual source position may involve calculating a weighted average of multiple pre-calculated virtual source gain values ​​determined during the setup process for virtual source positions that are within the audio object area or volume defined by the audio object size and position. A set of audio object gain values ​​for each output channel of the playback environment may be calculated based, at least in part, on the calculated virtual source contributions. Each output channel may correspond to at least one playback speaker of the playback environment.

[0011] Thus, some methods described herein involve receiving audio playback data including one or more audio objects. The audio objects may include an audio signal and associated metadata. The metadata may include at least audio object position data and audio object size data. The methods may involve calculating contributions from virtual sources within an audio object region or volume defined by the audio object position data and the audio object size data. The methods may involve calculating a set of audio object gain values ​​for each of a plurality of output channels based, at least in part, on the calculated contributions. For example, the playback environment may be a cinema sound system environment.

[0012] The process of calculating the contributions from the virtual sources may involve calculating a weighted average of the virtual source gain values ​​from the virtual sources within the audio object region or volume, where the weights for the weighted average may depend on the position of the audio object, the size of the audio object and / or each virtual source position within the audio object region or volume.

[0013] The methods may also involve receiving playback environment data including playback speaker position data. The methods may also involve defining a plurality of virtual source positions according to the playback environment data, and calculating, for each virtual source position, a virtual source gain value for each of the plurality of output channels. In some implementations, each of the virtual source positions may correspond to a position within the playback environment. However, in some implementations, at least some of the virtual source positions may correspond to positions outside the playback environment.

[0014] In some implementations, the virtual source positions may be uniformly spaced along the x, y and z axes. However, in some implementations, the spacing may not be the same in all directions. For example, the virtual source positions may have a first uniform spacing along the x and y axes and a second uniform spacing along the z axis. The process of computing a set of audio object gain values ​​for each of the multiple output channels may involve independent computation of contributions from virtual sources along the x, y and z axes. In alternative implementations, the virtual source positions may be non-uniformly spaced.

[0015] In some implementations, the process of calculating an audio object gain value for each of the plurality of output channels comprises: o ,y o ,z o The gain value (g) for an audio object of size (s) to be rendered in l (x o ,y o ,z o For example, the audio object gain value (g l (x o ,y o ,z o ;s)) is

number

[0016] According to some such implementations, g l (x vs ,y vs ,z vs )=g l (x vs )g l (y vs )g l (z vs ), where g l (x vs ), g l (y vs ) and g l (z vs ) represents an independent gain function of x, y, and z. In some such implementations, the weighting function may be factorized as follows:

[0017] w(x vs ,y vs ,z vs ;x o ,y o ,z o ;s)=w x (x vs ;x o ;s)w y (y vs ;y o ;s)w z (z vs ;z o ;s) Here, w x (x vs ;x o ;s), w y (y vs ;y o ;s) and wz (z vs ;z o ;s) is x vs、 y vs and z vs In some such implementations, p may be a function of the audio object size (s).

[0018] Some such methods may involve storing the calculated virtual source gain values ​​in a memory system. The process of calculating contributions from virtual sources within an audio object region or volume may involve retrieving the calculated virtual source gain values ​​corresponding to the audio object positions and sizes from the memory system, and interpolating between the calculated virtual source gain values. The process of interpolating between the calculated virtual source gain values ​​may involve: determining a plurality of neighboring virtual source positions near an audio object position; determining a calculated virtual source gain value for each of said neighboring virtual source positions; determining a plurality of distances between said audio object position and each of said neighboring virtual source positions; and interpolating between the calculated virtual source gain values ​​according to said plurality of distances.

[0019] In some implementations, the playback environment data may include playback environment boundary data. The method may involve determining that an audio object region or volume includes an outer region or volume outside a playback environment boundary and applying a fade-out factor based at least in part on the outer region or volume. Some methods may involve determining that an audio object may be within a threshold distance from a playback environment boundary and not providing speaker feed signals to playback speakers on an opposite boundary of the playback environment. In some implementations, the audio object region or volume may be a rectangle, a cuboid, a circle, a sphere, an ellipse, and / or an ellipsoid.

[0020] Some methods may involve decorrelating at least a portion of the audio playback data, for example, the methods may involve decorrelating the audio playback data for audio objects having an audio object size above a certain threshold.

[0021] Alternative methods are described herein. Some such methods involve receiving playback environment data including playback speaker position data and playback environment boundary data, and receiving audio playback data including one or more audio objects and associated metadata. The metadata may include audio object position data and audio object size data. The methods may involve determining that an audio object region or volume defined by the audio object position data and the audio object size data includes an outer region or volume outside the playback environment boundary, and determining a fade-out factor based at least in part on the outer region or volume. The methods may involve calculating a set of gain values ​​for each of a plurality of output channels based at least in part on the associated metadata and the fade-out factors. Each output channel may correspond to at least one playback speaker of the playback environment.

[0022] These methods may involve determining that an audio object may be within a threshold distance from one playback environment boundary, and not providing speaker feed signals to playback speakers on the opposite boundary of the playback environment.

[0023] The methods may involve computing contributions from virtual sources within an audio object region or volume. The methods may involve defining a plurality of virtual source positions according to playback environment data and computing, for each of the virtual source positions, a virtual source gain for each of a plurality of output channels. The virtual source positions may or may not be uniformly spaced, depending on the particular implementation.

[0024] Some implementations may be embodied in one or more non-transitory media on which software is stored. The software may include instructions for controlling one or more devices to receive audio playback data including one or more audio objects. The audio objects may include audio signals and associated metadata. The metadata may include at least audio object position data and audio object size data. The software may include instructions for calculating, for an audio object from the one or more audio objects, a contribution from a virtual source within a region or volume defined by the audio object position data and the audio object size data, and calculating a set of audio object gain values ​​for each of a plurality of output channels based at least in part on the calculated contributions. Each output channel may correspond to at least one playback speaker of the playback environment.

[0025] In some implementations, the process of calculating the contributions from the virtual sources may involve calculating a weighted average of the virtual source gain values ​​from the virtual sources within the audio object region or volume, where the weights for the weighted average may depend on the position of the audio object, the size of the audio object and / or each virtual source position within the audio object region or volume.

[0026] The software may include instructions for receiving playback environment data including playback speaker position data. The software may include instructions for defining a plurality of virtual source positions according to the playback environment data, and for each virtual source position, calculating a virtual source gain value for each of the plurality of output channels. Each of the virtual source positions may correspond to a position within the playback environment. In some implementations, at least some of the virtual source positions may correspond to positions outside the playback environment.

[0027] According to some implementations, the virtual source positions may be uniformly spaced. In some implementations, the virtual source positions may have a first uniform spacing along the x-axis and the y-axis and a second uniform spacing along the z-axis. The process of computing a set of audio object gain values ​​for each of the plurality of output channels may involve independent computation of contributions from virtual sources along the x-axis, the y-axis, and the z-axis.

[0028] Various devices and apparatus are described herein. Some such devices may include an interface system and a logic system. The interface system may include a network interface. In some implementations, the devices may include a memory device. The interface system may include an interface between the logic system and the memory device.

[0029] The logic system may be adapted to receive audio playback data from the interface system, the audio objects including one or more audio objects. The audio objects may include an audio signal and associated metadata. The metadata may include at least audio object position data and audio object size data. The logic system may be adapted to calculate, for an audio object from the one or more audio objects, a contribution from a virtual source within an audio object region or volume defined by the audio object position data and the audio object size data. The logic system may be adapted to calculate a set of audio object gain values ​​for each of a plurality of output channels based at least in part on the calculated contributions. Each output channel may correspond to at least one playback speaker of a playback environment.

[0030] The process of calculating contributions from virtual sources may involve calculating a weighted average of virtual source gain values ​​from virtual sources within an audio object region or volume. The weights for the weighted average may depend on the position of the audio object, the size of the audio object and / or each virtual source position within the audio object region or volume. The logic system may be adapted to receive playback environment data from the interface system, including playback speaker position data.

[0031] The logic system may be adapted to define a plurality of virtual source positions according to playback environment data, and to calculate, for each virtual source position, a virtual source gain value for each of the plurality of output channels. Each of the virtual source positions may correspond to a position within the playback environment. However, in some implementations, at least some of the virtual source positions may correspond to positions outside the playback environment. Depending on the specific implementation, the virtual source positions may or may not be uniformly spaced. In some implementations, the virtual source positions may have a first uniform spacing along the x-axis and the y-axis, and a second uniform spacing along the z-axis. The process of calculating a set of audio object gain values ​​for each of the plurality of output channels may involve independent calculation of contributions from virtual sources along the x-axis, the y-axis, and the z-axis.

[0032] The apparatus may include a user interface. The logic system may be adapted to receive user input, such as audio object size data, via the user interface. In some implementations, the logic system may be adapted to scale the input audio object size data.

[0033] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. It is noted that the relative dimensions of the following drawings may not be drawn to scale. [Brief description of the drawings]

[0034] [Figure 1] This figure shows an example of a playback environment with Dolby Surround 5.1 configuration. [Diagram 2] This figure shows an example of a playback environment with Dolby Surround 7.1 configuration. [Diagram 3] This figure shows an example of a playback environment with a Hamasaki 22.2 surround sound configuration. [Figure 4A] FIG. 1 shows an example of a graphical user interface (GUI) for depicting speaker zones at various heights in a virtual playback environment. [Figure 4B] FIG. 13 is a diagram illustrating another example of a playback environment. [Figure 5A] 1 is a flow chart giving an overview of an audio processing method; [Figure 5B] A flow chart giving an example of the setup process. [Figure 5C] 11 is a flow diagram providing an example of a run-time process for computing gain values ​​for a received audio object according to pre-computed gain values ​​for a virtual source position. [Figure 6A] FIG. 2 shows an example of a virtual source position relative to the playback environment. [Figure 6B] FIG. 13 illustrates alternative virtual source positions for the playback environment. [Figure 6C] FIG. 13 illustrates an example of applying near-field and far-field panning techniques to audio objects in different positions. [Figure 6D] FIG. 13 illustrates an example of applying near-field and far-field panning techniques to audio objects in different positions. [Figure 6E] FIG. 13 illustrates an example of applying near-field and far-field panning techniques to audio objects in different positions. [Figure 6F] FIG. 13 illustrates an example of applying near-field and far-field panning techniques to audio objects in different positions. [Figure 6G] FIG. 1 shows an example of a playback environment with one speaker in each corner of a square with side length equal to 1. [Figure 7] FIG. 13 shows an example of a contribution from a virtual source within a region defined by audio object position data and audio object size data. [Figure 8] A and B show an audio object in two positions within a playback environment. [Figure 9]1 is a flow diagram outlining a method for determining a fade-out factor based, at least in part, on how much of an audio object's area or volume extends outside the boundaries of the playback environment. [Figure 10] FIG. 2 is a block diagram providing examples of components of an authoring and / or rendering device. [Figure 11] FIG. 1A is a block diagram illustrating some components that may be used for audio content generation, and FIG. 1B is a block diagram illustrating some components that may be used for audio playback in a playback environment. Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] The following description is directed to certain implementations for purposes of describing some novel aspects of the disclosure and examples of contexts in which these novel aspects may be implemented. However, the teachings of the present disclosure may be applied in a variety of different ways. For example, while various implementations are described using a specific playback environment, the teachings of the present disclosure are broadly applicable to other known and future playback environments. Similarly, the described implementations may be implemented in a variety of authoring and / or rendering tools, which may be implemented in a variety of hardware, software, firmware, etc. Thus, the teachings of the present disclosure are not intended to be limited to the implementations shown in the drawings and / or described herein, but rather have broad applicability.

[0036] 1 shows an example of a playback environment with a Dolby Surround 5.1 configuration. Although Dolby Surround 5.1 was developed in the 1990s, this configuration is still widely deployed in movie theater sound system environments. A projector 105 may be configured to project a video image, for example for a movie, onto a screen 150. Audio playback data may be synchronized with the video image and processed by a sound processor 110. A power amplifier 115 may provide speaker feed signals to speakers in the playback environment 100.

[0037] The Dolby Surround 5.1 configuration includes a left surround array 120, a right surround array 125, each of which includes a group of speakers collectively driven by a single channel. The Dolby Surround 5.1 configuration also includes separate channels for the left screen channel 130, the center screen channel 135, and the right screen channel 140. A separate channel for the subwoofer 145 is provided for low-frequency effects (LFE).

[0038] In 2010, Dolby provided an improvement to digital cinema sound by introducing Dolby Surround 7.1. FIG. 2 shows an example of a playback environment with a Dolby Surround 7.1 configuration. Digital projector 205 may be configured to receive digital video data and project a video image onto screen 150. Audio playback data may be processed by sound processor 210. Power amplifiers 215 may provide speaker feed signals to speakers in playback environment 200.

[0039] The Dolby Surround 7.1 configuration includes a left lateral surround array 220 and a right lateral surround array 225, each of which may be driven by a single channel. Similar to Dolby Surround 5.1, the Dolby Surround 7.1 configuration includes separate channels for the left screen channel 230, the center screen channel 235, the right screen channel 240, and the subwoofer 245. However, Dolby Surround 7.1 increases the number of surround channels by splitting the left and right surround channels of Dolby Surround 5.1 into four zones: in addition to the left lateral surround array 220 and the right lateral surround array 225, separate channels are included for the left rear surround speaker 224 and the right rear surround speaker 226. Increasing the number of surround zones in the playback environment 200 can significantly improve sound localization.

[0040] In an effort to create a more immersive environment, some playback environments may be configured with an increased number of speakers driven by an increased number of channels. Additionally, some playback environments may include speakers that are deployed at various heights, some of which may be above the seating area of ​​the playback environment.

[0041] Figure 3 shows an example of a playback environment with a Hamasaki 22.2 surround sound configuration. Hamasaki 22.2 was developed at the NHK Science and Technology Research Laboratories in Japan as a surround sound component for ultra-high definition television. Hamasaki 22.2 provides 24 speaker channels that can be used to drive speakers arranged in three tiers. The upper speaker tier 310 of the playback environment 300 can be driven by nine channels. The middle speaker tier 320 can be driven by ten channels. The lower speaker tier 330 can be driven by five channels, two of which are for the subwoofers 345a and 345b.

[0042] Thus, the current trend is not only to include more speakers and more channels, but also to include speakers of different heights. As the number of channels increases and speaker layouts transition from 2D to 3D arrays, the task of positioning and rendering the sound becomes more difficult. The assignee of the present application has therefore developed various tools and related user interfaces that enhance the functionality and / or reduce the authoring complexity for 3D audio sound systems. Some of these tools are described in detail with reference to Figures 5A-19D in U.S. Provisional Patent Application No. 61 / 636,102, filed April 20, 2012, and entitled "Systems and Tools for Improved 3D Audio Creation and Presentation" (the "Creation and Presentation" application), the contents of which are incorporated herein by reference.

[0043] 4A shows an example of a graphical user interface (GUI) depicting speaker zones at various heights in a virtual playback environment. GUI 400 may be displayed on a display device, for example, according to instructions from a logic system, signals received from a user input device, etc. Some such devices are described below with reference to FIG.

[0044] As used herein to refer to a virtual playback environment such as virtual playback environment 404, the term "speaker zone" refers generally to a logical construct that may or may not have a one-to-one correspondence with a playback speaker in a real playback environment. For example, a "speaker zone location" may or may not correspond to a specific playback speaker location in a movie theater playback environment. Instead, the term "speaker zone location" may generally refer to a zone in a virtual playback environment. In some implementations, a speaker zone in a virtual playback environment may correspond to a virtual speaker through the use of a virtualization technology such as, for example, Dolby Headphone™ (sometimes referred to as Mobile Surround™), which generates a virtual surround sound environment in real time using a set of two-channel stereo headphones. GUI 400 has seven speaker zones 402a at a first elevation and two speaker zones 402b at a second elevation, for a total of nine speaker zones in virtual playback environment 404. In this example, speaker zones 1-3 are in a front region 405 of virtual playback environment 404. The front area 405 may for example correspond to the area in a cinema playback environment where the screen 150 is located, or in a home where a television screen is located, etc.

[0045] Here, speaker zone 4 generally corresponds to speakers in left region 410, and speaker zone 5 corresponds to speakers in right region 415 of virtual playback environment 404. Speaker zone 6 corresponds to left rear region 412, and speaker zone 7 corresponds to right rear region 414 of virtual playback environment 404. Speaker zone 8 corresponds to speakers in top region 420a, and speaker zone 9 corresponds to speakers in top region 420b, which may be a virtual ceiling region such as the region of virtual ceiling 520 shown in Figures 5D and 5E. Thus, as discussed in more detail in the "Making and Representing" application, the locations of speaker zones 1-9 shown in Figure 4A may or may not correspond to the locations of playback speakers in the actual playback environment. Additionally, other implementations may include more or fewer speaker zones and / or heights.

[0046] In various implementations described in the Creating and Representing application, a user interface such as GUI 400 may be used as part of an authoring tool and / or a rendering tool. In some implementations, the authoring tool and / or the rendering tool may be implemented via software stored on one or more non-transitory media. The authoring tool and / or the rendering tool may be implemented (at least in part) by hardware, firmware, etc., such as the logic systems and other devices described below with reference to FIG. 10. In some authoring implementations, an associated authoring tool may be used to generate metadata about associated audio data. The metadata may include, for example, data indicating the position and / or trajectory of audio objects in three-dimensional space, speaker zone constraint data, etc. The metadata may be generated with respect to speaker zones 402 of a virtual playback environment 404, rather than with respect to a particular speaker layout of a real playback environment. The rendering tool may receive the audio data and associated metadata and may calculate audio gain and speaker feed signals for the playback environment. Such audio gain and speaker feed signals may be calculated according to an amplitude panning process. The amplitude panning process can create the perception that a sound comes from a position P in the playback environment. For example, a speaker feed signal can be expressed as x i (t)=g i x(t) i=1,…,N (Equation 1) may be provided to playback speakers 1 to N of the playback environment according to

[0047] In formula (1), x i (t) represents the speaker feed signal applied to speaker i, and g irepresents the gain factor of the corresponding channel, x(t) represents the audio signal, and t represents time. The gain factor may be determined according to the amplitude panning methods described in, for example, Section 2, pp. 3-4 of "Phases 1 and 2", incorporated herein by reference. In some implementations, the gain may be frequency dependent. In some implementations, a time delay may be introduced by replacing x(t) with x(t-Δt).

[0048] In some rendering implementations, the audio playback data generated with reference to speaker zones 402 may be mapped to speaker locations of a wide range of playback environments, which may be in a Dolby Surround 5.1 configuration, a Dolby Surround 7.1 configuration, a Hamasaki 22.2 configuration, or other configurations. For example, with reference to FIG. 2, the rendering tool may map the audio playback data for speaker zones 4 and 5 to the left lateral surround array 220 and the right lateral surround array 225 of a playback environment having a Dolby Surround 7.1 configuration. The audio playback data for speaker zones 1, 2, and 3 may be mapped to the left screen channel 230, the right screen channel 240, and the center screen channel 235, respectively. The audio playback data for speaker zones 6 and 7 may be mapped to the left rear surround speaker 224 and the right rear surround speaker 226.

[0049] 4B shows another example of a playback environment. In some implementations, the rendering tool may map audio playback data for speaker zones 1, 2, and 3 to corresponding screen speakers 455 of the playback environment 450. The rendering tool may map audio playback data for speaker zones 4 and 5 to left lateral surround array 460 and right lateral surround array 465, and may map audio playback data for speaker zones 8 and 9 to left overhead speaker 470a and right overhead speaker 470b. Audio playback data for speaker zones 6 and 7 may be mapped to left rear surround speaker 480a and right rear surround speaker 480b.

[0050] In some authoring implementations, the authoring tool may be used to generate metadata about audio objects. As used herein, the term "audio object" may refer to a stream of audio data and associated metadata. The metadata may indicate the 3D position of the audio object, the apparent size of the audio object, rendering constraints, and content type (e.g., dialogue, effects, etc.). Depending on the implementation, the metadata may include other types of data, such as gain data, trajectory data, etc. Some audio objects may be static, while others may move. Details of an audio object may be authored or rendered according to associated metadata, which may indicate, for example, the location of the audio object in three-dimensional space at a given time. When the audio object is monitored or played in a playback environment, the audio object may be rendered according to its position and size metadata in accordance with the playback speaker layout of the playback environment.

[0051] FIG. 5A is a flow diagram providing an overview of audio processing methods. More detailed examples are described below with reference to FIG. 5B et seq. The methods may include more or fewer blocks than shown and described herein, and are not necessarily performed in the order presented herein. The methods may be performed, at least in part, by apparatus such as those shown in FIGS. 10-11 and described below. Software may include instructions for controlling one or more apparatus to perform the methods described herein.

[0052] In the example shown in FIG. 5A, the method 500 begins with a setup process that determines virtual source gain values ​​for virtual source positions for a particular playback environment (step 505). FIG. 6A illustrates an example of virtual source positions for a playback environment. For example, block 505 may involve determining virtual source gain values ​​for virtual source positions 605 for playback speaker positions 625 of playback environment 600a. The virtual source positions 605 and playback speaker positions 625 are merely examples. In the example shown in FIG. 6A, the virtual source positions 605 are uniformly spaced along the x, y, and z axes. However, in alternative implementations, the virtual source positions 605 may be spaced differently. For example, in some implementations, the virtual source positions 605 may have a first uniform spacing along the x and y axes and a second uniform spacing along the z axis. In other implementations, the virtual source positions 605 may be non-uniformly spaced.

[0053] 6A, the reproduction environment 600a and the virtual source volume 602a are coextensive, such that each of the virtual source positions 605 corresponds to a position within the reproduction environment 600a. However, in alternative implementations, the reproduction environment 600 and the virtual source volume 602 may not be coextensive. For example, at least some of the virtual source positions 605 may correspond to positions outside the reproduction environment 600.

[0054] 6B shows an alternative example of a virtual source position relative to the reproduction environment, where the virtual source volume 602b extends outside the reproduction environment 600b.

[0055] Returning to FIG. 5A, in this example, the setup process of block 505 occurs prior to rendering any particular audio object. In some implementations, the virtual source gain values ​​determined in block 505 may be stored in a storage system. The stored virtual source gain values ​​may be used during a "run-time" process of calculating audio object gain values ​​for received audio objects according to at least some of the virtual source gain values ​​(block 510). For example, block 510 may involve, at least in part, calculating audio object gain values ​​based on virtual source gain values ​​corresponding to virtual source positions that are within an audio object region or volume.

[0056] In some implementations, the method 500 may include an optional block 515 involving decorrelating the audio data. Block 515 may be part of a run-time process. In some such implementations, block 515 may involve convolution in the frequency domain. For example, block 515 may involve applying a finite impulse response ("FIR") filter for each speaker feed signal.

[0057] In some implementations, the process of block 515 may or may not be performed depending on the audio object size and / or the artistic intent of the author. According to some such implementations, an authoring tool may link audio object size to decorrelation by indicating (e.g., via a decorrelation flag included in associated metadata) that decorrelation should be turned on when the audio object size is equal to or greater than a certain size threshold, and that decorrelation should be turned off if the audio object size is less than the size threshold. In some implementations, decorrelation may be controlled (e.g., increased, decreased, or disabled) according to user input regarding the size threshold and / or other input values.

[0058] FIG. 5B is a flow diagram providing an example of a setup process. Thus, all blocks shown in FIG. 5B are examples of processes that may be performed in block 505 of FIG. 5A. Here, the setup process begins with receipt of playback environment data (block 520). The playback environment data may include playback speaker position data. The playback environment data may include data representing boundaries of the playback environment, such as walls, ceilings, etc. If the playback environment is a movie theater, the playback environment data may also include an indication of the movie screen position.

[0059] The playback environment data may also include data indicative of the correlation of the output channels with the playback speakers of the playback environment. For example, the playback environment may have a Dolby Surround 7.1 configuration as shown in FIG. 2 and described above. Thus, the playback environment data may also include data indicative of the correlation between the Lss channel and the left side surround speaker 220, between the Lrs channel and the left rear surround speaker 224, etc.

[0060] In this example, block 525 involves defining a virtual source position 605 according to the playback environment data. The virtual source position 605 may be defined within a virtual source volume. In some implementations, the virtual source volume may correspond to a volume within which an audio object can move. As shown in Figures 6A and 6B, in some implementations, the virtual source volume 602 may be coextensive with the volume of the playback environment 600, while in other implementations, at least a portion of the virtual source position 605 may correspond to a position outside the playback environment 600.

[0061] Furthermore, the virtual source positions 605 may or may not be uniformly spaced within the virtual source volume 602, depending on the particular implementation. In some implementations, the virtual source positions 605 may be uniformly spaced in all directions. For example, the virtual source positions 605 are spaced apart from each other by N x Multiply N y Multiply N z, forming a rectangular grid of virtual source positions 605. In some implementations, the value of N may range from 5 to 100. The value of N may depend, at least in part, on the number of playback speakers in the playback environment: it may be desirable to include more than one virtual source position 605 between each playback speaker position.

[0062] In other implementations, the virtual source positions 605 may have a first uniform spacing along the x-axis and y-axis and a second uniform spacing along the z-axis. x Multiply N y Kakeru M z The number of virtual source positions 605 may form a rectangular grid of virtual source positions 605. For example, in some implementations there may be fewer virtual source positions 605 along the z-axis than along the x-axis or y-axis. In some such implementations, the value of N may range from 10 to 100, while the value of M may range from 5 to 10.

[0063] In this example, block 530 involves calculating a virtual source gain value for each of the virtual source positions 605. In some implementations, block 530 involves calculating a virtual source gain value for each of the multiple output channels of the playback environment for each virtual source position 605. In some implementations, block 530 may involve applying a vector-based amplitude panning (VBAP) algorithm, a pairwise panning algorithm, or a similar algorithm to calculate a gain value for a point source located at each virtual source position 605. In other implementations, block 530 may involve applying a separable algorithm to calculate a gain value for a point source located at each virtual source position 605. As used herein, a "separable" algorithm is one in which the gain of a given speaker can be expressed as a product of two or more factors that can be calculated separately for each coordinate of a virtual source position. Examples include algorithms implemented in various existing mixing console panners, including but not limited to the panners implemented in Pro Tools™ software and digital film consoles offered by AMS Neve. Several two-dimensional examples are given below.

[0064] 6C-6F show examples of application of near-field and far-field panning techniques to audio objects at various positions. Referring first to FIG. 6C, the audio object is substantially outside the virtual playback environment 400a. Thus, one or more far-field panning methods are applied in this example. In some implementations, the far-field panning method may be based on vector-based amplitude panning (VBAP) formulas known to those skilled in the art. For example, the far-field panning method may be based on the VBAP formulas described in Section 2.3, p. 4, of "Analyzing Near-Field Audio Objects in Virtual Reality," IEEE Transactions on Audio Engineering, Vol. 13, No. 1, 2003, which is incorporated herein by reference. In alternative implementations, other methods may be used to pan the far-field and near-field audio objects, for example methods involving the synthesis of corresponding acoustic plane or spherical waves. Related methods are described in "Analyzing Near-Field Audio Objects in Virtual Reality," IEEE Transactions on Audio Engineering, Vol. 13, 2003, 2003, which is incorporated herein by reference.

[0065] 6D, an audio object 610 is inside the virtual playback environment 400a. Therefore, one or more near-field panning methods are applied in this example. Some such near-field panning methods use several speaker zones that surround the audio object 610 within the virtual playback environment 400a.

[0066] FIG. 6G shows an example of a playback environment with one speaker at each corner of a square with side lengths equal to 1. In this example, the origin (0,0) of the xy axis coincides with the left (L) screen speaker 130. Thus, the right (R) screen speaker 140 has coordinates (1,0), the left surround (Ls) speaker 120 has coordinates (0,1), and the right surround (Rs) speaker 125 has coordinates (1,1). The audio object position 615 (x,y) is x units to the right of the L speaker and y units from the screen 150. In this example, each of the four speakers receives a factor cos / sin proportional to their distance along the x and y axes. According to some implementations, the gain may be calculated as follows:

[0067] G_l(x)=cos(pi / 2*x) where l=L,Ls G_l(x)=sin(pi / 2*x) where l=R,Rs G_l(x)=cos(pi / 2*y) where l=L,R G_l(x)=sin(pi / 2*y) where l=Ls,Rs .

[0068] The overall gain is the product: G_l(x,y)=G_l(x)G_l(y). In general, these functions depend on all coordinates of all speakers. However, G_l(x) does not depend on the y position of the source and G_l(y) does not depend on its x position. To illustrate simple calculations, consider the audio object position 615 to be (0,0), i.e. the position of the L speaker. G_L(x)=cos(0)=1 and G_L(y)=cos(0)=1. The overall gain is the product G_L(x,y)=G_L(x)G_L(y)=1. Similar calculations give us G_Ls=G_Rs=G_R=0.

[0069] It may be desirable to blend between different pan modes when an audio object enters or exits the virtual playback environment 400a. For example, when an audio object 610 moves from the audio object position 615 shown in FIG. 6C to the audio object position 615 shown in FIG. 6D or vice versa, a blend of gains calculated according to the near-field panning method and the far-field panning method may be applied. In some implementations, a pair-wise panning law (e.g., an energy-preserving sine or power law) may be used to blend between gains calculated according to the near-field panning method and the far-field panning method. In alternative implementations, the pair-wise panning law may be amplitude-preserving rather than energy-preserving. Thus, rather than the sum of the squares being equal to one, the sum is equal to one. It is also possible to process an audio signal using both panning methods independently and blend the resulting processed signals, for example to cross-fade the two resulting audio signals.

[0070] Returning now to FIG. 5B, regardless of the algorithm used in block 530, the resulting gain values ​​may be stored in a memory system for use during run-time operation (block 535).

[0071] 5C is a flow diagram providing an example of a run-time process for calculating gain values ​​for a received audio object according to pre-calculated gain values ​​for virtual source positions. All of the blocks shown in FIG. 5C are examples of processes that may be performed in block 510 of FIG. 5A.

[0072] In this example, the runtime process begins with receipt of audio playback data including one or more audio objects (block 540). The audio objects include an audio signal and associated metadata, which in this example includes at least audio object position data and audio object size data. With reference to FIG. 6A, for example, audio object 610 is defined, at least in part, by audio object position 615 and audio object volume 620a. In this example, the received audio object size data indicates that audio object volume 620a corresponds to the volume of a rectangular prism. However, in the example shown in FIG. 6B, the received audio object size data indicates that audio object volume 620b corresponds to the volume of a sphere. These sizes and shapes are merely examples. In alternative implementations, the audio objects may have a variety of other sizes and / or shapes. In some alternative examples, the area or volume of the audio object may be a rectangle, a circle, an ellipse, an ellipsoid, or a sector of a sphere.

[0073] In this implementation, block 545 involves calculating the contribution from the virtual source within a region or volume defined by the audio object position data and the audio object size data. In the example shown in Figures 6A and 6B, block 545 may involve calculating the contribution from the virtual source at a virtual source position 605 that is within the audio object volume 620a or the audio object volume 620b. If the metadata of the audio object changes over time, block 545 may be executed again according to the new metadata values. For example, if the audio object size and / or audio object position changes, a different virtual source position 605 may fall within the audio object volume 620 and / or the virtual object position 605 used in the previous calculation may be a different distance from the audio object position 615. In block 545, the corresponding virtual source contribution is calculated according to the new object size and / or position.

[0074] In some examples, block 545 may involve retrieving from a memory system the calculated virtual source gain values ​​for virtual source positions corresponding to the audio object position and size, and interpolating between the calculated virtual source gain values. The process of interpolating between the calculated virtual source gain values ​​may involve determining a plurality of neighboring virtual source positions near the audio object position, determining a calculated virtual source gain value for each of the neighboring virtual source positions, determining a plurality of distances between the audio object position and each of the neighboring virtual source positions, and interpolating between the calculated virtual source gain values ​​according to the plurality of distances.

[0075] The process of calculating the contributions from the virtual sources may involve calculating a weighted average of the calculated virtual source gain values ​​for virtual source positions within a region or volume defined by the size of the audio object, where the weights for the weighted average may for example depend on the position of the audio object, the size of the audio object and each virtual source position within said region or volume.

[0076] FIG. 7 shows an example of contributions from virtual sources within a region defined by audio object position data and audio object size data. FIG. 7 illustrates a cross section of audio environment 200a taken perpendicular to the z-axis. Thus, FIG. 7 is illustrated from the perspective of an observer looking down on audio environment 200a along the z-axis. In this example, audio environment 200a is a cinema sound system environment with a Dolby Surround 7.1 configuration as shown in FIG. 2 and described above. Thus, playback environment 200a includes left side surround speaker 220, left rear surround speaker 224, right side surround speaker 225, right rear surround speaker 226, left screen channel 230, center screen channel 235, right screen channel 240, and subwoofer 245.

[0077] The audio object 610 has a size indicated by an audio object volume 620b, the rectangular cross-sectional area of ​​which is shown in Figure 7. Given the audio object position 615 at the time depicted in Figure 7, the area encompassed by the audio object volume 620b in the xy plane includes twelve virtual source positions 605. Depending on the extent of the audio object volume 620b in the z direction and the spacing of the virtual source positions 605 along the z axis, additional virtual source positions 605 may or may not be encompassed within the audio object volume 620b.

[0078] FIG. 7 shows the contributions from virtual source positions 605 within an area or volume defined by the size of the audio object 610. In this example, the diameter of the circle used to depict each of the virtual source positions 605 corresponds to the contribution from the corresponding virtual source position 605. The virtual source positions 605a closest to the audio object position 615 are shown largest, indicating the largest contribution from the corresponding virtual source. The second largest contribution is from the virtual source at virtual source position 605b, which is the second closest to the audio object position 615. A smaller contribution is made by virtual source position 605c, which is further away from the audio object position 615 but still within the audio object volume 620b. Virtual source position 605d, which is outside the audio object volume 620b, is shown smallest, indicating that in this example the corresponding virtual source makes no contribution.

[0079] Referring to FIG. 5C, in this example, block 550 may involve calculating a set of audio object gain values ​​for each of a plurality of output channels based at least in part on the calculated contributions. Each output channel may correspond to at least one playback speaker of the playback environment. Block 550 may involve normalizing the resulting audio object gain values. For the implementation shown in FIG. 7, for example, each output channel may correspond to a single speaker or a group of speakers.

[0080] The process of calculating an audio object gain value for each of the plurality of output channels includes: o ,y o ,z o The gain value (g) for an audio object of size (s) to be rendered in l size (x o ,y o ,z oThis audio object gain value is sometimes referred to as the "audio object size contribution" in this document. In some implementations, the audio object gain value (g l size (x o ,y o ,z o ;s)) may be expressed as follows:

[0081]

number

[0082] In some examples, the exponent p may have a value between 1 and 10. In some implementations, p may be a function of the audio object size s. For example, if s is relatively larger, in some implementations p may be relatively smaller. According to some such implementations, p may be determined as follows:

[0083] When p=6 s≦0.5 p=6+(-4)(s-0.5) / (s max -0.5) if s>0.5 Here, s max is the internal scaled up size s internal (see below), an audio object size s=1 may correspond to an audio object with a size (e.g., diameter) equal to one length of the boundary of the playback environment (e.g., equal to the length of one wall of the playback environment).

[0084] Depending in part on the algorithm or algorithms used to calculate the virtual source gain values, it may be possible to simplify equation (2) if the virtual source positions are uniformly distributed along an axis and if the weighting and gain functions are separable, e.g., as described above. If these conditions are met, then g l (x vs ,y vs ,z vs ) is g lx (x vs )g ly (y vs )g lz (z vs ), where g lx (x vs ), g ly (y vs ) and g lz (z vs ) represents the x, y and z coordinate independent gain function for the position of the virtual source.

[0085] Similarly, w(x vs ,y vs ,z vs;x o ,y o ,z o ;s) is w x (x vs ;x o ;s)w y (y vs ;y o ;s)w z (z vs ;z o ;s), where w x (x vs ;x o ;s), w y (y vs ;y o ;s) and w z (z vs ;z o ;s) is the x 、 represents independent weighting functions for the y and z coordinates. One such example is shown in Figure 7. In this example, w x (x vs ;x o The weighting function 710, denoted as w y (y vs ;y o ;s) from weighting function 720. In some implementations, weighting functions 710 and 720 may be Gaussian functions, while weighting function w z (z vs ;z o ;s) may be a product of a cosine and a Gaussian function.

[0086] w(x vs ,y vs ,z vs ;x o ,y o ,z o ;s) is w x (x vs ;x o ;s)w y (y vs ;y o ;s)w z (z vs ;z o ;s), equation (2) can be simplified to:

[0087]

number

[0088] In some implementations, the audio object size contribution g l size may be combined with an "audio object near gain" for the audio object position. As used herein, "audio object near gain" is the calculated gain based on the audio object position 615. The gain calculation may be done using the same algorithm used to calculate each of the virtual source gain values. According to some such implementations, a cross-fade calculation may be performed between the audio object size contribution and the audio object near gain result, for example as a function of the audio object size. Such an implementation may provide smooth panning and smooth growth of the audio object and may allow smooth transitions between minimum and maximum audio object sizes. In one such implementation:

[0089]

number

[0090] According to some implementations, the audio object size value may be scaled up in the larger portion of its range of possible values. In some authoring implementations, for example, the user may specify an audio object size value s user ∈[0,1], which may be used for a larger range, e.g., the range [0,s max ] where s max >1. This mapping can ensure that the gain is truly independent of the object's position when the size is set to maximum by the user. According to some such implementations, such a mapping can be used to map pairs of points (s user ,s internal ), where s user represents the user selected audio object size, and s internal represents the corresponding audio object size determined by the algorithm. According to some such implementations, the mapping is done for pairs of points (0,0), (0.2,0.3), (0.5,0.9), (0.75,1.5) and (1,s max ) according to a piecewise linear function that connects s max =2.8.

[0091] 8A and 8B show the audio object at two positions in the playback environment. In these examples, the audio object volume 620b is a sphere with a radius less than half the length or width of the playback environment 200a. The playback environment 200a is configured according to Dolby 7.1. At the time of depiction in FIG. 8A, the audio object position 615 is relatively closer to the center of the playback environment 200a. At the time of depiction in FIG. 8B, the audio object position 615 has moved closer to the boundary of the playback environment 200a. In this example, the boundary is the left wall of the theater, which coincides with the position of the left side surround speaker 220.

[0092] For aesthetic reasons, it may be desirable to modify the audio object gain calculation for audio objects approaching a boundary of the playback environment. For example, in Figures 8A and 8B, when the audio object position 615 is within a certain threshold distance from the left boundary 805 of the playback environment, no speaker feed signal is provided to the speaker at the opposite boundary of the playback environment (here, the right side surround speaker 225). In the example shown in Figure 8B, when the audio object position 615 is within a certain threshold distance (which may be a different threshold distance) from the left boundary 805 of the playback environment, no speaker feed signal is provided to the left screen channel 230, the center screen channel 235, the right screen channel 240 or the subwoofer 245 if the audio object position 615 is further away from the screen than a certain threshold distance.

[0093] In this example, shown in FIGURE 8B, the audio object volume 620b includes an area or volume outside the left boundary 805. According to some implementations, a fade-out factor for the gain calculation may be based, at least in part, on how much of the left boundary 805 is within the audio object volume 620b and / or how much of the audio object's area or volume extends outside such boundary.

[0094] 9 is a flow diagram outlining a method for determining a fade-out factor based, at least in part, on how much of an area or volume of an audio object extends outside the boundaries of a playback environment. At block 905, playback environment data is received. In this example, the playback environment data includes playback speaker position data and playback environment boundary data. Block 910 involves receiving audio playback data including one or more audio objects and associated metadata. The metadata, in this example, includes at least audio object position data and audio object size data.

[0095] In this implementation, block 915 involves determining that the audio object region or volume defined by the audio object position data and the audio object size data includes an outer region or volume outside the playback environment boundary. Block 915 may also involve determining what percentage of the audio object region or volume is outside the playback environment boundary.

[0096] At block 920, a fade-out factor is determined. In this example, the fade-out factor may be based at least in part on the outer region. For example, the fade-out factor may be proportional to the outer region.

[0097] In block 925, a set of audio object gain values ​​may be calculated for each of a number of output channels based at least in part on the associated metadata (in this example, audio object position data and audio object size data) and the fade-out factor, each output channel may correspond to at least one playback speaker of the playback environment.

[0098] In some implementations, the audio object gain calculation may involve calculating contributions from virtual sources within an audio object region or volume. The virtual sources may correspond to a number of virtual source positions, which may be defined with reference to playback environment data. The virtual source positions may or may not be uniformly spaced. For each of the virtual source positions, a virtual source gain value may be calculated for each of said plurality of output channels. As noted above, in some implementations, these virtual source gain values ​​may be calculated and stored during a setup process and retrieved for use during runtime operation.

[0099] In some implementations, a fade-out factor may be applied to all virtual source gain values ​​corresponding to virtual source positions in the playback environment. l size may be modified as follows:

[0100]

number

[0101] In an alternative implementation, g l size may be modified as follows:

[0102]

number

[0103] 10 is a block diagram providing an example of components of an authoring and / or rendering device. In this example, device 1000 includes an interface system 1005. Interface system 1005 may include a network interface, such as a wireless network interface. Alternatively or additionally, interface system 1005 may include a Universal Serial Bus (USB) interface or other such interface.

[0104] The device 1000 includes a logic system 1010. The logic system 1010 may include a processor, such as a general purpose single-chip or multi-chip processor. The logic system 1010 may include a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, or discrete hardware components, or a combination thereof. The logic system 1010 may be configured to control other components of the device 1000. Although interfaces between the components of the device 1000 are not shown in FIG. 10, the logic system 1010 may be configured to have interfaces for communication with other components. The other components may or may not be configured for communication with each other, as appropriate.

[0105] The logic system 1010 may be configured to perform audio authoring and / or rendering functions, including but not limited to audio authoring and / or rendering functions of the type described herein. In some such implementations, the logic system 1010 may be configured to operate (at least in part) according to software stored on one or more non-transitory media. The non-transitory media may include memory associated with the logic system 1010, such as random access memory (RAM) and / or read-only memory (ROM). The non-transitory media may include memory of the memory system 1015. The memory system 1015 may include one or more suitable types of non-transitory storage media, such as flash memory, hard drives, etc.

[0106] Display system 1030 may include one or more suitable types of displays, depending on the implementation of device 1000. For example, display system 1030 may include a liquid crystal display, a plasma display, a bi-stable display, etc.

[0107] The user input system 1035 may include one or more devices configured to accept input from a user. In some implementations, the user input system 1035 may include a touch screen overlaying a display of the display system 1030. The user input system 1035 may include a mouse, a trackball, a gesture detection system, a joystick, one or more GUIs and / or menus presented on the display system 1030, buttons, keyboards, switches, etc. In some implementations, the user input system 1035 may include a microphone 1025: a user may provide voice commands for the device 1000 via the microphone 1025. A logic system may be configured for voice recognition and for controlling at least some operations of the device 1000 according to such voice commands.

[0108] The power system 1040 may include one or more suitable energy storage devices, such as nickel-cadmium batteries or lithium-ion batteries. The power system 1040 may be configured to receive power from an electrical outlet.

[0109] FIG. 11A is a block diagram depicting some components that may be used for audio content generation. The system 1100 may be used for audio content generation, for example, in a mixing studio and / or a dubbing stage. In this example, the system 1100 includes an audio and metadata authoring tool 1105 and a rendering tool 1110. In this implementation, the audio and metadata authoring tool 1105 and the rendering tool 1110 include audio connection interfaces 1107 and 1112, respectively, which may be configured for communication via AES / EBU, MADI, analog, etc. The audio and metadata authoring tool 1105 and the rendering tool 1110 include network interfaces 1109 and 1117, respectively, which may be configured to send and receive metadata via TCP / IP or any other suitable protocol. The interface 1120 is configured to output the audio data to a speaker.

[0110] The system 1100 may include, for example, an existing authoring system, such as a Pro Tools™ system, that runs a metadata generation tool (i.e., a panner as described herein) as a plug-in. The panner may run on a standalone system (e.g., a PC or a mixing console) connected to the rendering tool 1110, or may run on the same physical device as the rendering tool 1110. In the latter case, the panner and the renderer may use a local connection, e.g., through shared memory. The panner GUI may also be provided on a tablet device, laptop, etc. The rendering tool 1110 may have a rendering system including a sound processor configured to perform a rendering method such as the one described in Figures 5A-C and 9. The rendering system may include, for example, a personal computer, laptop, etc., including interfaces for audio input and output and appropriate logic systems.

[0111] 11B is a block diagram illustrating some components that may be used for audio playback in a playback environment (e.g., a movie theater). System 1150, in this example, includes a cinema server 1155 and a rendering system 1160. Cinema server 1155 and rendering system 1160 include network interfaces 1157 and 1162, respectively, which may be configured to transmit and receive audio objects via TCP / IP or any other suitable protocol. Interface 1164 is configured to output audio data to speakers.

[0112] Various modifications to the implementations described in this disclosure may be readily apparent to those skilled in the art. The general principles defined herein may be applied to other implementations without departing from the spirit or scope of the disclosure. Thus, the claims are not intended to be limited to the implementations shown herein, but are to be accorded the widest scope consistent with the disclosure, principles and novel features disclosed herein.

[0113] Some numbering examples are given below. [Numbering Example 1] receiving audio playback data including one or more audio objects, the audio objects including an audio signal and associated metadata, the metadata including at least audio object position data and audio object size data; calculating, for an audio object from the one or more audio objects, a contribution from a virtual source within an audio object region or volume defined by the audio object position data and the audio object size data; calculating a set of audio object gain values ​​for each of a plurality of output channels based, at least in part, on said calculated contributions, each output channel corresponding to at least one playback speaker of the playback environment; method. [Numbering Example 2] 2. The method of claim 1, wherein calculating contributions from virtual sources comprises calculating a weighted average of virtual source gain values ​​from virtual sources within the audio object region or volume. [Numbered Example 3] The method of Example 2, numbered as above, wherein weights for the weighted average depend on a position of the audio object, a size of the audio object and each virtual source position within the audio object region or volume. [Numbered Example 4] receiving playback environment data including playback speaker position data; Method described in Example 1. [Numbered Example 5] defining a plurality of virtual source positions according to the playback environment data; for each virtual source position, calculating a virtual source gain value for each of the plurality of output channels; Method described in Example 4. [Numbered Example 6] 6. The method of embodiment 5, wherein each virtual source position corresponds to a position within the playback environment. [Numbered Example 7] The method of embodiment 5, wherein at least some of the virtual source positions correspond to positions outside the playback environment. [Numbered Example 8] The method of Example 5, wherein the virtual source positions are uniformly spaced along the x, y, and z axes. Numbered Example 9 The method of embodiment 5, wherein the virtual source positions have first uniform spacing along the x-axis and y-axis and a second uniform spacing along the z-axis. Numbered Example 10 10. The method of claim 8 or 9, wherein calculating a set of audio object gain values ​​for each of the plurality of output channels includes independent calculation of contributions from virtual sources along x, y and z axes. Numbered Example 11 The method of embodiment 5, wherein the virtual source positions are non-uniformly spaced. [Numbered Example 12] The step of calculating an audio object gain value for each of the plurality of output channels comprises: o ,y o ,z o The gain value (g) for an audio object of size (s) to be rendered in l (x o ,y o ,z o ;s)) and determining a gain value (g l (x o,y o ,z o ;s)) is

number

Claims

1. 1. A method of rendering input audio including audio objects and metadata, the metadata including audio object size metadata and audio object position metadata corresponding to the audio objects, the method comprising: receiving said audio object size metadata and said audio object position metadata; receiving content type metadata associated with the audio object, the content type metadata indicating a dialogue related to the audio object; determining at least a virtual audio object based on the input audio, the audio object size metadata, and the audio object position metadata; determining a position of the virtual audio object based on at least one of the audio object size metadata and the audio object position metadata; and rendering the audio object to the one or more speaker feeds based on the content-based metadata, the rendering also including rendering the virtual object based on at least the position of the virtual audio object. method.

2. A non-transitory medium storing software including instructions for performing the method of claim 1.

3. 1. An apparatus for rendering input audio comprising audio objects and metadata, the metadata including audio object size metadata and audio object position metadata corresponding to the audio objects, the apparatus comprising: a receiver configured to receive the audio object size metadata and the audio object position metadata and to receive content type metadata associated with the audio object, the content type metadata indicating a dialogue related to the audio object; a first processor for determining at least a virtual audio object based on the input audio, the audio object size metadata, and the audio object position metadata; a second processor for determining a position of the virtual audio object based on at least one of the audio object size metadata and the audio object position metadata; a renderer for rendering the audio object to the one or more speaker feeds based on the content-type metadata, the rendering also including rendering the virtual audio object based on at least the position of the virtual audio object. Device.

Images