Method, apparatus and system for encoding and decoding directional sound sources - Patents.com

Abstract

To provide a method for decoding audio data for achieving representation and efficient encoding of a complex radiation pattern.SOLUTION: An audio encoding method includes: a process for receiving mono (monophonic) audio signal corresponding to an audio object and representation of a radiation pattern corresponding to the audio object; and a process for encoding the mono audio signal, encoding a source radiation pattern and determining radiation pattern metadata. The radiation pattern may include sound levels corresponding to a plurality of sample times, a plurality of frequency bands and a plurality of directions. Encoding of the radiation pattern may involve determining spherical harmonic function transform of representation of the radiation pattern, compressing the spherical harmonic function transform and obtaining encoded radiation pattern metadata.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Application No. 62 / 658,067, filed August 16, 2018; U.S. Application No. 62 / 681,429, filed June 6, 2018; and U.S. Application No. 62 / 741,419, filed October 4, 2018, the contents of which are hereby incorporated by reference in their entireties.

[0002] Technical Field The present disclosure relates to encoding and decoding of directional sound sources and auditory scenes based on multiple dynamic and / or moving directional sound sources. [Background technology]

[0003] Real-world sound sources, whether natural or man-made (speakers, musical instruments, voices, mechanical devices), radiate sound in a non-isotropic manner. Characterizing the radiation pattern (or "directivity") of a sound source can be crucial for proper rendering, especially in the context of interactive environments such as video games and virtual / augmented reality (VR / AR) applications. In these environments, users typically interact with directional audio objects by walking around them, thereby changing the auditory perspective of the generated sound (also known as six degrees of freedom [DoF] rendering). Users can also grab and dynamically rotate virtual objects, which also requires the rendering of different directions in the radiation pattern of the corresponding sound source(s). In addition to more realistic rendering of direct propagation effects from the source to the listener, radiation characteristics also play a major role in higher-order acoustic coupling between the source and its environment (e.g., the virtual environment in a game), thus affecting reverberation (i.e., waves going back and forth as in echoes). As a result, such reverberation can affect other spatial cues, such as perceived distance.

[0004] Most audio game engines provide some way to represent and render directional sound sources, but are generally limited to simple directional gains that rely on the definition of a simple first order cosine function or "sound cone" (e.g., a power cosine function) and a simple high frequency roll-off filter. These representations are inadequate for representing real-world radiation patterns, and are also poorly suited for simplified / combined representations of many directional sound sources. Summary of the Invention [Means for solving the problem]

[0005] Various audio processing methods are disclosed herein. Some such methods may involve encoding directional audio data. For example, some methods may involve receiving a mono audio signal corresponding to an audio object and a representation of a radiation pattern corresponding to the audio object. The radiation pattern may include, for example, sound levels corresponding to multiple sample times, multiple frequency bands, and multiple directions. Some such methods may involve encoding the mono audio signal and encoding the source radiation pattern to determine radiation pattern metadata. Encoding the radiation pattern may involve determining a spherical harmonic transform of the representation of the radiation pattern and compressing the spherical harmonic transform to obtain the encoded radiation pattern metadata.

[0006] Some such methods may involve encoding multiple directional audio objects based on a cluster of audio objects. The radiation pattern may represent a centroid reflecting an average sound level value for each frequency band. In some such implementations, multiple directional audio objects are encoded as a single directional audio object with a directivity corresponding to a time-varying, energy-weighted average of the spherical harmonic coefficients of each audio object. The encoded radiation pattern metadata may indicate a position of the cluster of audio objects that is the average of the positions of each audio object.

[0007] Some methods may involve encoding group metadata regarding radiation patterns of a group of directional audio objects. In some examples, the source radiation pattern may be rescaled relative to the amplitude of the input radiation pattern in a direction per frequency to determine a normalized radiation pattern. According to some implementations, compressing the spherical harmonics transform may involve singular value decomposition techniques, principal component analysis, discrete cosine transforms, data-independent bases, and / or eliminating spherical harmonics coefficients of the spherical harmonics transform above a threshold order of the spherical harmonics coefficients.

[0008] Several alternative methods may involve decoding audio data. For example, some such methods may involve receiving an encoded core audio signal, encoded radiation pattern metadata, and encoded audio object metadata, and decoding the encoded core audio signal to determine a core audio signal. Some such methods may involve decoding the encoded radiation pattern metadata to determine a decoded radiation pattern, decoding audio object metadata, and rendering the core audio signal based on the audio object metadata and the decoded radiation pattern.

[0009] In some instances, the audio object metadata may include at least one of time-varying three degrees of freedom (3DoF) or six degrees of freedom (6DoF) source orientation information. The core audio signal may include a plurality of directional objects based on a cluster of objects. The decoded radiation pattern may represent a centroid reflecting an average value for each frequency band. In some examples, the rendering may be based at least in part on applying sub-band gains based on the decoded radiation data to the decoded core audio signal. The encoded radiation pattern metadata may correspond to a time- and frequency-varying set of spherical harmonic coefficients.

[0010] According to some implementations, the encoded radiation pattern metadata may include audio object type metadata. The audio object type metadata may, for example, indicate parametric directivity pattern data. The parametric directivity pattern data may include a cosine function, a sine function, and / or a cardioid function. In some examples, the audio object type metadata may indicate database directivity pattern data. Decoding the encoded radiation pattern metadata to determine the decoded radiation pattern may involve querying a directivity data structure that includes an audio object type and corresponding directivity pattern data. In some examples, the audio object type metadata may indicate dynamic directivity pattern data. The dynamic directivity pattern data may correspond to a time- and frequency-varying set of spherical harmonic coefficients. Some methods may involve receiving the dynamic directivity pattern data prior to receiving the encoded core audio signal.

[0011] Some or all of the methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices such as those described herein, including, but not limited to, random access memory (RAM) devices, read-only memory (ROM) devices, and the like. Thus, various innovative aspects of the subject matter described in this disclosure may be implemented in one or more non-transitory media having software stored thereon. The software may include, for example, instructions for controlling at least one device to process audio data. The software may be executable, for example, by one or more components of a control system as disclosed herein. The software may include, for example, instructions for performing one or more of the methods disclosed herein.

[0012] At least some aspects of the present disclosure may be implemented via an apparatus. For example, one or more apparatus may be configured to at least partially perform the methods disclosed herein. In some implementations, the apparatus may include an interface system and a control system. The interface system may include one or more network interfaces, one or more interfaces between the control system and a memory system, one or more interfaces between the control system and another device, and / or one or more external device interfaces. The control system may include at least one of a general-purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, or discrete hardware components. Thus, in some implementations, the control system may include one or more processors and one or more non-transitory storage media operatively coupled to the one or more processors.

[0013] According to some such examples, the control system may be configured to receive, via the interface system, audio data corresponding to at least one audio object. In some examples, the audio data may include a monophonic audio signal, audio object position metadata, audio object size metadata, and rendering parameters. Some such methods may involve determining whether the rendering parameters indicate a positional mode or a directional mode, and upon determining that the rendering parameters indicate a directional mode, rendering the audio data for playback through at least one loudspeaker according to a directional pattern indicated by the position metadata and / or the size metadata.

[0014] In some instances, rendering the audio data may involve interpreting audio object position metadata as audio object orientation metadata. The audio object position metadata may include, for example, x, y, z coordinate data, spherical coordinate data, and / or cylindrical coordinate data. In some instances, the audio object orientation metadata may include yaw, pitch, and roll data.

[0015] According to some examples, rendering the audio data may involve interpreting audio object size metadata as directional metadata corresponding to a directional pattern. In some implementations, rendering the audio data may include interrogating a data structure including a plurality of directional patterns and mapping position metadata and / or size metadata to one or more of the directional patterns. In some instances, the control system may be configured to receive the data structure via an interface system. In some examples, the data structure may be received prior to the audio data. In some implementations, the audio data may be received in a Dolby Atmos format. The audio object position metadata may correspond to, for example, world coordinates or model coordinates.

[0016] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Please note that the relative dimensions of the figures below may not be drawn to scale. Like reference numbers and symbols in the various drawings generally indicate like elements. [Brief description of the drawings]

[0017] [Figure 1A] 1 is a flow diagram illustrating blocks of an audio encoding method according to an example.

[0018] [Figure 1B] 1 illustrates process blocks that may be implemented by an encoding system for dynamically encoding per-frame directional information for directional audio objects, according to one example.

[0019] [Figure 1C] 1 illustrates process blocks that may be implemented by a decoding system according to an example.

[0020] [Diagram 2] Figures 2A and 2B represent the radiation pattern of an audio object in two different frequency bands.

[0021] [Figure 2C] 1 is a graph illustrating an example of normalized and non-normalized radiation patterns, according to an example.

[0022] [Diagram 3] An example hierarchy containing audio data and various types of metadata is shown below.

[0023] [Figure 4] 1 is a flow diagram illustrating blocks of an audio decoding method according to an example.

[0024] [Figure 5A] Drum cymbals are depicted.

[0025] [Figure 5B] An example of a speaker system is shown below.

[0026] [Figure 6] 1 is a flow diagram illustrating blocks of an audio decoding method according to an example.

[0027] [Figure 7] Here is an example of encoding multiple audio objects.

[0028] [Figure 8] FIG. 1 is a block diagram illustrating example components of an apparatus that may be configured to perform at least some of the methods disclosed herein.

[0029] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Certain aspects of the present disclosure relate to the representation and efficient encoding of complex radiation patterns. Some such implementations may include one or more of the following: 1. Representation of a general sound radiation pattern as time- and frequency-dependent Nth order coefficients (N≧1) of a real-valued spherical harmonics (SPH) decomposition. This representation can also be extended depending on the level of the reproduced audio signal. Contrary to the case where the directional source signal itself is a PCM representation like HOA, a mono object signal can be encoded separately from its directional information, which is represented as a set of time-dependent scalar SPH coefficients in the subbands. 2. An efficient encoding scheme to reduce the bitrate required to represent this information 3. A solution that dynamically combines radiation patterns so that a scene consisting of several radiating sources can be represented, at rendering time, by an equivalent reduced number of sources while preserving its perceptual quality.

[0031] One aspect of the present disclosure relates to representing a common radiation pattern to complement metadata about each mono audio object by a set of time / frequency dependent coefficients representing the directivity of the mono audio object projected in an Nth order spherical harmonics basis (N≧1).

[0032] The primary radiation pattern can be represented by a set of four scalar gain coefficients for a predefined set of frequency bands (e.g., 1 / 3 octave). The set of frequency bands is also known as a bin or subband. The bins or subbands may be determined based on a Short-Time Fourier Transform (STFT) or a perceptual filter bank on a single data frame (e.g., 512 samples as in Dolby Atmos). The resulting pattern can be rendered by evaluating a spherical harmonic decomposition in the required directions around the object.

[0033] In general, this radiation pattern is a property of the source and may be constant over time. However, to represent dynamic scenes where objects rotate or change, or to ensure that the data is randomly accessible, it may be beneficial to update this set of coefficients at regular time intervals. In the context of dynamic auditory scenes with moving objects, the results of object rotations can be directly encoded in the time-varying coefficients without requiring an explicit separate encoding of object orientation.

[0034] Each type of sound source has a characteristic radiation / emission pattern that typically varies across frequency bands. For example, a violin may have a very different radiation pattern than a trumpet, drum or bell. Furthermore, sound sources such as musical instruments may radiate differently at pianissimo and fortissimo performance levels. As a result, the radiation pattern may be a function not only of the direction around the sound-emitting object, but also of the pressure level of the radiating audio signal, which may also vary in time.

[0035] Thus, instead of simply representing the sound field at a point in space, some implementations involve encoding audio data corresponding to the radiation pattern of an audio object so that it can be rendered from different vantage points. In some cases, the radiation pattern may be a time- and frequency-varying radiation pattern. The audio data input to the encoding process may, in some cases, include multiple channels (e.g., 4, 6, 8, 20 or more channels) of audio data from directional microphones. Each channel may correspond to data from a microphone at a particular position in the space around the sound source, from which a radiation pattern can be derived. Given that the relative direction from each microphone to the sound source is known, this can be achieved by numerically fitting a set of spherical harmonic function coefficients such that the resulting spherical functions best match the observed energy levels in the various subbands of each input microphone signal. See, for example, the methods and systems described in connection with International Application No. PCT / US2017 / 053946 to Nicolas Tsingos and Pradeep Kumar Govindaraju entitled "Method, Systems and Apparatus for Determining Audio Representations," which is incorporated herein by reference. In another example, the radiation pattern of an audio object may be determined by numerical simulation.

[0036] Instead of simply encoding audio data from directional microphones at the sample level, some implementations involve encoding monophonic audio object signals along with corresponding radiation pattern metadata that describes radiation patterns for at least some of the encoded audio objects. In some implementations, the radiation pattern metadata may be represented as spherical harmonics data. Some such implementations may involve smoothing and / or compression / data reduction processes.

[0037] 1A is a flow diagram illustrating blocks of an audio encoding method, according to one example. Method 1 may be implemented, for example, by a control system (such as control system 815, described below with reference to FIG. 8) including one or more processors and one or more non-transitory memory devices. As with other disclosed methods, not all blocks of method 1 are necessarily performed in the order shown in FIG. 1A. Additionally, alternative methods may include more or fewer blocks.

[0038] In this example, block 5 is involved in receiving a mono audio signal corresponding to an audio object and receiving a representation of a radiation pattern corresponding to the audio object. According to this implementation, the radiation pattern includes sound levels corresponding to multiple sample times, multiple frequency bands, and multiple directions. According to this example, block 10 is involved in encoding the mono audio signal.

[0039] In the example shown in FIG. 1A, block 15 involves encoding the source radiation pattern to determine radiation pattern metadata. According to this implementation, encoding the representation of the radiation pattern involves determining a spherical harmonic transform of the representation of the radiation pattern and compressing the spherical harmonic transform to obtain the encoded radiation pattern metadata. In some implementations, the representation of the radiation pattern may be rescaled relative to the amplitude of the input radiation pattern in a direction for each frequency to determine a normalized radiation pattern.

[0040] In some cases, compressing the spherical harmonics transform may involve discarding some higher order spherical harmonics coefficients. Some such examples involve removing spherical harmonics coefficients of the spherical harmonics transform that are above a threshold order of the spherical harmonics coefficients, e.g., above order 3, above order 4, above order 5.

[0041] However, some implementations may involve alternative and / or additional compression methods. According to some such implementations, compressing the spherical harmonics transform may involve singular value decomposition, principal component analysis, discrete cosine transform, data independent bases, and / or other methods.

[0042] According to some examples, method 1 may also involve encoding multiple directional audio objects as a group or "cluster" of audio objects. Some implementations may involve encoding group metadata regarding a radiation pattern of the group of directional audio objects. In some instances, multiple directional audio objects may be encoded as a single directional audio object with a directivity corresponding to a time-varying, energy-weighted average of the spherical harmonic coefficients of each audio object. In some such examples, the encoded radiation pattern metadata may represent a centroid corresponding to an average sound level value for each frequency band. For example, the encoded radiation pattern metadata (or related metadata) may indicate a position of a cluster of audio objects that is the average of the positions of each directional audio object in the cluster.

[0043] 1B illustrates blocks of a process that may be implemented by encoding system 100 to dynamically encode per-frame directionality information for directional audio objects, according to an example. This process may be implemented via a control system, such as control system 815 described below with reference to FIG. 8. Encoding system 100 may receive a mono audio signal 101, which may correspond to a mono object signal as discussed above. Mono audio signal 101 may be encoded in block 111 and provided to serialization block 112.

[0044] In block 102, static or time-varying directional energy samples at different sound levels in a set of frequency bands may be processed relative to a reference coordinate system, which may be determined in some coordinate space, such as a model coordinate space or a world coordinate space.

[0045] In block 105, a frequency-dependent rescaling of the time-varying directional energy samples from block 102 may be performed. In one example, the frequency-dependent rescaling may be performed according to the example shown in Figures 2A-2C, as described below. The normalization may be based on an amplitude rescaling, for example, for the high frequency direction versus the low frequency direction.

[0046] The frequency-dependent rescaling may be renormalized based on the assumed capture direction of the core audio, which may represent a listening direction relative to the sound source. For example, this listening direction may be referred to as the gaze direction, where the gaze direction may be a direction (e.g., forward or backward) relative to the coordinate system.

[0047] In block 106, the rescaled directional output of 105 may be projected into a spherical harmonics basis, resulting in spherical harmonics coefficients.

[0048] In block 108, the spherical coefficients of block 106 are processed based on the instantaneous sound level 107 and / or information from the rotation block 109. The instantaneous sound level 107 may be measured in a certain direction and at a certain time. The information from the rotation block 109 may indicate an (optional) rotation of the time-varying source orientation 103. In one example, the spherical coefficients can be adjusted in block 109 to take into account time-dependent modifications in the source orientation relative to the originally recorded input data.

[0049] A target level determination may further be performed in block 108 based on the equalization determined for a direction of the assumed capture direction of the core audio signal. Block 108 may output a set of rotated spherical coefficients that are equalized based on the target level determination.

[0050] In block 110, the encoding of the radiation pattern may be based on a projection of the spherical coefficients associated with the source radiation pattern onto a smaller subspace, resulting in encoded radiation pattern metadata. As shown in FIG. 1A, in block 110, an SVD decomposition and compression algorithm may be performed on the spherical coefficients output by block 108. In one example, the SVD decomposition and compression algorithm of block 110 may be performed according to the principles described in connection with Equations 11-13 below.

[0051] Alternatively, block 110 may convert the spherical harmonics representation into a space that leads to lossy compression.

number

[0052] Such a syntax may include different sets of coefficients for different pressure / intensity levels of the sound source. Alternatively, a single set of coefficients may be dynamically generated if directional information is available at different signal levels and the level of the source cannot be determined further at playback time. For example, such coefficients may be generated by interpolating between low-level and high-level coefficients based on the time-varying level of the object audio signal at the time of encoding.

[0053] The input radiation pattern for a mono audio object signal may be "normalized" to a given direction, such as the principal response axis (which may be the direction in which it was recorded or the average of multiple recordings), and the encoded directivity and final rendering may need to be consistent with this "normalization." In one example, this normalization may be specified as metadata. In general, it is desirable to encode a core audio signal that would convey a good representation of the object's timbre if no directional information was applied.

[0054] Directional Encoding Certain aspects of the present disclosure are directed to implementing an efficient encoding scheme for the directional information since the number of coefficients grows quadratically with the order of decomposition. An efficient encoding scheme for the directional information may be implemented for final delivery of the auditory scene, for example over a limited bandwidth network, to an end-point rendering device.

[0055] If 16 bits are used to represent each coefficient, then a fourth-order spherical harmonic representation in 1 / 3 octave bands would require 25 × 31 = 12 kbits per frame. Refreshing this information at 30 Hz would require a transmission bit rate of at least 400 kbps, more than current object-based audio codecs currently require to transmit both audio and object metadata. In one example, the radiation pattern is G(θ i ,φi ,ω) Equation (1) It may be expressed by:

[0056] In the formula (1), (θ i ,φ i ), i∈{1…P} represent discrete co-latitude angles θ∈[0,π] and azimuth angles φ∈[0,2π) relative to the acoustic source, P represents the total number of discrete angles, and ω represents the spectral frequency. Figures 2A and 2B represent the radiation pattern of an audio object in two different frequency bands. Figure 2A may represent the radiation pattern of an audio object in a frequency band of, for example, 100-300 Hz, and Figure 2B may represent the radiation pattern of the same audio object in a frequency band of, for example, 1 kHz-2 kHz. Since low frequencies tend to be relatively closer to omnidirectional, the radiation pattern shown in Figure 2A is relatively closer to circular than the radiation pattern shown in Figure 2B. In Figure 2A, G(θ0,φ0,ω) represents the radiation pattern in the direction of the main response axis 200, while G(θ1,φ1,ω) represents the radiation pattern in an arbitrary direction 205.

[0057] In some examples, the radiation pattern may be captured and determined by multiple microphones physically positioned around a sound source corresponding to the audio object, while in other examples, the radiation pattern may be determined via numerical simulation. In multiple microphone examples, the radiation pattern may vary in time, for example reflecting a live recording. The radiation pattern may be captured at a variety of frequencies, including low frequencies (e.g., <100 Hz), mid-frequency (100 Hz < and >1 kHz), and high frequencies (>10 kHz). The radiation pattern may also be known as a spatial representation.

[0058] In another example, the radiation pattern can be expressed as the captured radiation pattern G(θ i ,φ i ,ω), for example:

number

[0059] In equation (2), G(θ,φ,ω) represents the radiation pattern in the direction of the main response axis. Referring again to FIG. 2B, in one example, the radiation pattern G(θ i ,φ i ,ω) and the normalized radiation pattern H(θ i ,φ i ,ω). FIG. 2C is a graph showing an example of normalized and unnormalized radiation patterns according to an example. In this example, the normalized radiation pattern in the direction of the principal response axis, represented as H(θ0,φ0,ω) in FIG. 2C, has substantially the same amplitude over the illustrated range of the frequency band. In this example, the normalized radiation pattern in the direction 205 (shown in FIG. 2A), represented as H(θ1,φ1,ω) in FIG. 2C, has relatively higher amplitude at higher frequencies than the unnormalized radiation pattern, represented as G(θ1,φ1,ω) in FIG. 2C. For a given frequency band, the radiation pattern may be assumed to be constant for notational convenience, but in reality may change over time due to, for example, different bowing techniques used in stringed instruments.

[0060] The radiation pattern, or a parametric representation thereof, may be transmitted. Pre-processing of the radiation pattern may be performed prior to its transmission. In one example, the radiation pattern or the parametric representation may be pre-processed by a computational algorithm, an example of which is shown in relation to FIG. 1A. After pre-processing, the radiation pattern may be, for example,

number

[0061] In equation (3), H(θ i ,φ i ,ω) represents the spatial representation,

number

number

number

[0062] In equation (4), P n m (x) is an associated Legendre polynomial with order m∈{-N…N} and degree n∈{0…N},

number

[0063] Other spherical bases may be used. Any technique for performing a spherical harmonic transform on discrete data may be used. In one example, the matrix transform

number

number

[0064] In the formula (7),

number

[0065] pseudo inverse matrix

number

number

[0066] The regularized solution may also be applicable when the distribution of spherical samples contains a large amount of missing data. The missing data may correspond to regions or directions for which directional samples are unavailable (e.g., due to non-uniform microphone coverage). In many cases, the distribution of spatial samples is sufficiently uniform that an identity weighting matrix W yields acceptable results. Also, it is often the case that P≫(N+1) 2 is assumed to be the spherical harmonic representation

number

number

[0067] Here, the discrete frequency band ω k , k∈{1…K}. The matrix H(ω) can be stacked so that each frequency range is represented by a column of the matrix.

number

[0068] That is, the spatial representation H(ω) may be determined based on the frequency bin / band / set. As a result, the spherical harmonic representation is:

number

[0069] In formula (10),

number

number

number

number

[0070] Some embodiments may involve performing a singular value decomposition (SVD), where:

number

number

number

[0071] O=(N+1) 2 In some examples, to achieve compression, the encoder may:

number

[0072] In the formula (12),

number

number

number

number

[0073] Below are three examples for transmitting the truncated decomposition vectors and the truncated right singular vectors: 1. The transmitter may transmit the encoded radiation T and the truncated right singular vectors V' for each object independently. 2. Objects may be grouped, for example according to a similarity metric, and U and V may be calculated as a representative basis for multiple objects. Thus, an encoded radiance may be transmitted for each object, and U and V may be transmitted for each group of objects. 3. The left and right singular matrices U and V may be pre-computed on a large database of representative data (e.g., training data) and information about V may be stored at the receiver. In some such examples, only the encoded radiation may be transmitted for each object. The DCT is another example of a basis that may be stored at the receiver.

[0074] Spatial encoding of directional objects When complex auditory scenes containing multiple objects are encoded and transmitted, spatial coding techniques can be applied in which individual objects are replaced by a smaller number of representative clusters in a way that best preserves the auditory perception of the scene. In general, replacing a group of sound sources by a representative "centroid" requires computing an aggregate / average value for each metadata field. For example, the location of a cluster of sound sources can be the average of the locations of each source. By representing the radiation pattern of each source using a spherical harmonics decomposition as described above (see, for example, Eqs. 1-12), it is possible to linearly combine the set of coefficients in each subband for each source to construct an average radiation pattern for the cluster of sources. By computing an energy-weighted average of the loudness or spherical harmonics coefficients over time, it is possible to construct a time-varying perceptually optimized representation that better preserves the original scene.

[0075] FIG. 1C illustrates process blocks that may be implemented by a decoding system according to an example. The blocks illustrated in FIG. 1C may be implemented, for example, by a control system (such as control system 815, described below with reference to FIG. 8) of a decoding apparatus including one or more processors and one or more non-transitory memory devices. In block 150, metadata and the encoded core mono audio signal may be received and deserialized. The deserialized information may include object metadata 151, the encoded core audio signal, and the encoded spherical coefficients. In block 152, the encoded core audio signal may be decoded. In block 153, the encoded spherical coefficients may be decoded. The encoded radiation pattern information may include the encoded radiation pattern T and / or a matrix V. The matrix V may be

number

[0076] Object metadata 151 may include information regarding the relative direction from the source to the listener. In one example, metadata 151 may include information regarding the distance and direction of the listener and the distance and direction of one or more objects relative to the 6DoF space. For example, metadata 151 may include information regarding the relative rotation, distance, and direction of the source in the 6DoF space. In the example of multiple objects in a cluster, the metadata field may reflect information regarding a representative "centroid" that reflects the aggregate value / average value of the cluster of objects.

[0077] The decoded core audio signal and the decoded spherical harmonics coefficients may then be rendered by a renderer 154. In one example, the renderer 154 may render the decoded core audio signal and the decoded spherical harmonics coefficients based on the object metadata 151. The renderer 154 may determine sub-band gains for the spherical coefficients of the radiation patterns based on information from the metadata 151, e.g., a relative direction from the source to the listener. The renderer 154 may then render the core audio object signal based on the determined sub-band gains of the corresponding decoded radiation pattern(s), source and / or listener pose information (e.g., x, y, z, yaw, pitch, roll) 155. The listener pose information may correspond to the user's position and viewing direction in the 6DoF space. The listener pose information may be received from a source local to the VR playback system, e.g., an optical tracking device. The source pose information corresponds to the position and orientation in space of the sound-emitting object. It can also be inferred from a local tracking system, for example when the user's hands are tracked to interact with virtual sound-emitting objects, or when tracked physical prop / proxy objects are used.

[0078] FIG. 3 shows an example hierarchy including audio data and various types of metadata. As with other figures provided herein, the number and types of audio data and metadata shown in FIG. 3 are provided by way of example only. Some encoders may provide the complete set of audio data and metadata shown in FIG. 3 (data set 345), while other encoders may provide only a portion of the metadata shown in FIG. 3, e.g., only data set 315, only data set 325, or only data set 335.

[0079] In this example, the audio data includes a monophonic audio signal 301. The monophonic audio signal 301 is an example of what is sometimes referred to herein as a "core audio signal," although in some examples the core audio signal may include audio signals corresponding to multiple audio objects included in a cluster.

[0080] In this example, the audio object position metadata 305 is expressed as Cartesian coordinates. However, in alternative examples, the audio object position metadata 305 may be expressed via other types of coordinates, such as spherical coordinates or polar coordinates. Thus, the audio object position metadata 305 may include three degrees of freedom (3DoF) position information. According to this example, the audio object metadata includes audio object size metadata 310. In alternative examples, the audio object metadata may include one or more other types of audio object metadata.

[0081] In this implementation, the data set 315 includes a monophonic audio signal 301, audio object position metadata 305, and audio object size metadata 310. The data set 315 may be provided, for example, in the Dolby Atmos™ audio data format.

[0082] In this example, the dataset 315 also includes optional rendering parameters R. According to some disclosed implementations, the optional rendering parameters R may indicate whether at least a portion of the audio object metadata of the dataset 315 should be interpreted in its "normal" sense (e.g., as position or size metadata) or as directional metadata. In some disclosed implementations, the "normal" mode may be referred to herein as a "positional mode" and the alternate mode may be referred to herein as a "directional mode." Some examples are described below with reference to FIGS. 5A-6.

[0083] According to this example, orientation metadata 320 includes angular information to represent the yaw, pitch, and roll of an audio object. In this example, orientation metadata 320 indicates yaw, pitch, and roll as Φ, Θ, and Ψ. Data set 325 includes sufficient information to orient the audio object for six degrees of freedom (6 DoF) applications.

[0084] In this example, the data set 335 includes audio object type metadata 330. In some implementations, the audio object type metadata 330 may be used to indicate corresponding radiation pattern metadata. The encoded radiation pattern metadata may be used (e.g., by a decoder or a device receiving audio data from a decoder) to determine a decoded radiation pattern. In some examples, the audio object type metadata 330 may indicate, essentially, "I am a trumpet," "I am a violin," etc. In some examples, the decoding device may have access to a database of audio object types and corresponding directivity patterns. According to some examples, the database may be provided along with the encoded audio data or prior to transmission of the audio data. Such audio object type metadata 330 may be referred to herein as "data directivity pattern data."

[0085] According to some examples, the audio object type metadata may indicate parametric directional pattern data. In some examples, the audio object type metadata 330 may indicate a directional pattern corresponding to a cosine function of a specified power, a cardioid function, etc.

[0086] In some examples, audio object type metadata 330 may indicate that the radiation pattern corresponds to a set of spherical harmonics coefficients. For example, audio object type metadata 330 may indicate that spherical harmonics coefficients 340 are provided in dataset 345. In some such examples, spherical harmonics coefficients 340 may be a time- and / or frequency-varying set of spherical harmonics coefficients, e.g., as described above. Such information may require the largest amount of data compared to the rest of the metadata hierarchy shown in FIG. 3. Thus, in some such examples, spherical harmonics coefficients 340 may be provided separately from monophonic audio signal 301 and corresponding audio object metadata. For example, spherical harmonics coefficients 340 may be provided at the start of transmission of audio data before real-time operations (e.g., real-time rendering operations such as games, movies, music performance, etc.) begin.

[0087] According to some implementations, a decoder-side device, such as a device that provides audio to a playback system, may determine the capabilities of the playback system and provide directional information according to those capabilities. For example, even if the entire data set 345 is provided to the decoder, only a usable portion of the directional information may be provided to the playback system in some such implementations. In some examples, the decoding device may determine which type(s) of directional information to use according to the capabilities of the decoding device.

[0088] 4 is a flow diagram illustrating blocks of an audio decoding method according to one example. Method 400 may be implemented, for example, by a control system of a decoding apparatus (such as control system 815 described below with reference to FIG. 8) that includes one or more processors and one or more non-transitory memory devices. As with other disclosed methods, not all blocks of method 400 are necessarily performed in the order shown in FIG. 4. Additionally, alternative methods may include more or fewer blocks.

[0089] In this example, block 405 involves receiving an encoded core audio signal, encoded radiation pattern metadata, and encoded audio object metadata. The encoded radiation pattern metadata may include audio object type metadata. The encoded core audio signal may include, for example, a monophonic audio signal. In some examples, the audio object metadata may include 3DoF position information, 6DoF position information and source orientation information, audio object size metadata, etc. The audio object metadata may be time-varying in some instances.

[0090] In this example, block 410 involves decoding the encoded core audio signal to determine a core audio signal, where block 415 involves decoding the encoded radiation pattern metadata to determine a decoded radiation pattern, where block 420 involves decoding at least a portion of the other encoded audio object metadata, where block 430 involves rendering the core audio signal based on the audio object metadata (e.g., audio object position, orientation and / or size metadata) and the decoded radiation pattern.

[0091] Block 415 may involve various types of operations, depending on the particular implementation. In some instances, the audio object type metadata may indicate database directivity pattern data. Decoding the encoded radiation pattern metadata to determine the decoded radiation pattern may involve querying a directivity data structure that includes an audio object type and corresponding directivity pattern data. In some instances, the audio object type metadata may indicate parametric directivity pattern data, such as directivity pattern data corresponding to a cosine function, a sine function, or a cardioid function.

[0092] According to some implementations, the audio object type metadata may indicate dynamic directivity pattern data, such as a time- and / or frequency-varying set of spherical harmonics coefficients. Some such implementations may involve receiving the dynamic directivity pattern data prior to receiving the encoded core audio signal.

[0093] In some instances, the core audio signal received at block 405 may include audio signals corresponding to multiple audio objects included in a cluster. According to some such examples, the core audio signal may be based on a cluster of audio objects, which may include multiple directional audio objects. The decoded radiation pattern determined at block 415 may correspond to a centroid of the cluster and may represent an average value for each frequency band of each of the multiple directional audio objects. The rendering process of block 430 may involve applying sub-band gains based at least in part on the decoded radiation data to the decoded core audio signal. In some examples, after decoding the core audio signal and decoding and applying directional processing thereto, the signal may be further virtualized to its intended location relative to the listener position using audio object position metadata and known rendering processes such as binaural rendering through headphones, rendering using loudspeakers in the playback environment, etc.

[0094] As described above with reference to FIG. 3, in some implementations, the audio data may be accompanied by rendering parameters (depicted as R in FIG. 3). The rendering parameters may indicate whether at least some audio object metadata, such as Dolby Atmos metadata, should be interpreted in a normal manner (e.g., as position or size metadata) or as directional metadata. The normal mode may be referred to as a "positional mode," and the alternate mode may be referred to herein as a "directional mode." Thus, in some examples, the rendering parameters may indicate whether at least some audio object metadata should be interpreted as a direction relative to a speaker or as a position relative to a room or other playback environment. Such implementations may be particularly useful, for example, for directional rendering using smart speakers with multiple drivers, as described below.

[0095] 5A depicts a drum cymbal. In this example, a drum cymbal 505 is shown emitting a sound having a directional pattern 510 with a substantially vertical primary response axis 515. The directional pattern 510 itself is also primarily vertical, with some divergence from the primary response axis 515.

[0096] FIG. 5B illustrates an example of a speaker system. In this example, speaker system 525 includes multiple speakers / transducers configured to radiate sound in various directions, including upward. The top speaker can in some cases be used in a traditional Dolby Atmos manner ("positional mode") to render a position such that sound is reflected from the ceiling to simulate a height / ceiling speaker (z=1). In some such cases, the corresponding Dolby Atmos rendering may include additional height virtualization processing to enhance the perception of audio objects with specific positions.

[0097] In other use cases, the same upward firing speaker(s) may be operated in a "directional mode", for example to simulate the directional pattern of a drum, symbol, or other audio object having a directional pattern similar to the directional pattern 510 shown in FIG. 5A. Some speaker systems 525 may be capable of beamforming, which can aid in the construction of a desired directional pattern. In some examples, no virtualization processing is included, to reduce the perception of audio objects having a specific location.

[0098] 6 is a flow diagram illustrating blocks of an audio decoding method according to one example. Method 600 may be implemented, for example, by a control system of a decoding apparatus (such as control system 815 described below with reference to FIG. 8) including one or more processors and one or more non-transitory memory devices. As with other disclosed methods, not all blocks of method 600 are necessarily performed in the order shown in FIG. 6. Additionally, alternative methods may include more or fewer blocks.

[0099] In this example, block 605 is involved in receiving audio data corresponding to at least one audio object. The audio data includes a monophonic audio signal, audio object position metadata, audio object size metadata, and rendering parameters. In this implementation, block 605 is involved in receiving this data via an interface system (such as interface system 810 of FIG. 8) of a decoding device. In some instances, the audio data may be received in Dolby Atmos™ format. The audio object position metadata may correspond to world coordinates or model coordinates, depending on the particular implementation.

[0100] In this example, block 610 involves determining whether the rendering parameters indicate a positional mode or a directional mode. In the example shown in FIG. 6, if it is determined that the rendering parameters indicate a directional mode, then in block 615 the audio data is rendered for playback (e.g., playback via at least one loudspeaker, headphones, etc.) according to a directional pattern indicated by at least one of the position metadata or the size metadata. For example, the directional pattern may be similar to that shown in FIG. 5A.

[0101] In some examples, rendering of the audio data may involve interpreting audio object position metadata as audio object orientation metadata. The audio object position metadata may be Cartesian / x,y,z coordinate data, spherical coordinate data, or cylindrical coordinate data. The audio object orientation metadata may be yaw, pitch, and roll metadata.

[0102] According to some implementations, rendering the audio data may involve interpreting audio object size metadata as directional metadata corresponding to a directional pattern. In some such examples, rendering the audio data may involve querying a data structure that includes a plurality of directional patterns and mapping at least one of the position metadata or the size metadata to one or more of the directional patterns. Some such implementations may involve receiving the data structure via an interface system. According to some such implementations, the data structure may be received prior to the audio data.

[0103] FIG. 7 shows an example of encoding multiple audio objects. In one example, information 701, 702, 703, etc. of objects 1-n may be encoded. In one example, in block 710, a representative cluster for the audio objects 701-703 may be determined. In one example, a group of sound sources may be aggregated and represented by a representative "centroid". This involves calculating an aggregate / average value for a metadata field. For example, the location of a cluster of sound sources may be the average of the locations of each sound source. In block 720, a radiation pattern for the representative cluster may be encoded. In some examples, the radiation pattern for the cluster may be encoded according to the principles described above with reference to FIG. 1A or FIG. 1B.

[0104] FIG. 8 is a block diagram illustrating example components of a device that may be configured to perform at least a portion of the methods disclosed herein. For example, device 805 may be configured to perform one or more of the methods described above with reference to FIGS. 1A-1C, 4, 6, and / or 7. In some examples, device 805 may be or include a personal computer, desktop computer, or other local device configured to provide audio processing. In some examples, device 805 may be or include a server. According to some examples, device 805 may be a client device configured to communicate with a server via a network interface. The components of device 805 may be implemented via hardware, via software stored on a non-transitory medium, via firmware, and / or by a combination thereof. The types and numbers of components shown in FIG. 8 and other figures disclosed herein are provided by way of example only. Alternative implementations may include more, fewer, and / or different components.

[0105] In this example, the device 805 includes an interface system 810 and a control system 815. The interface system 810 may include one or more network interfaces, one or more interfaces between the control system 815 and a memory system, and / or one or more external device interfaces (such as one or more universal serial bus (USB) interfaces). In some implementations, the interface system 810 may include a user interface system. The user interface system may be configured to receive input from a user. In some implementations, the user interface system may be configured to provide feedback to a user. For example, the user interface system may include one or more displays having a corresponding touch and / or gesture detection system. In some examples, the user interface system may include one or more microphones and / or speakers. According to some examples, the user interface system may include a device that provides tactile feedback, such as a motor, a vibrator, or the like. The control system 815 may include, for example, a general purpose single-chip or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASICS), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, and / or discrete hardware components.

[0106] In some examples, the device 805 may be implemented in a single device. However, in some implementations, the device 805 may be implemented in multiple devices. In some such implementations, the functionality of the control system 815 may be included in multiple devices. In some examples, the device 805 may be a component of another device.

[0107] Various exemplary embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. In general, the present disclosure is also understood to encompass an apparatus suitable for performing the above-described methods, such as an apparatus having a memory and a processor coupled to the memory (spatial renderer), the processor configured to execute instructions and perform the methods according to the embodiments of the present disclosure.

[0108] Although various aspects of the exemplary embodiments of the present disclosure have been illustrated and described as block diagrams, flow charts, or using some other pictorial representations, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in, by way of non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controllers, or other computing devices, or any combination thereof.

[0109] Additionally, the various blocks illustrated in the flowcharts may be viewed as method steps and / or as operations resulting from computer program code operations and / or as multiple combined logic circuit elements configured to perform the associated function(s). For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied on a machine-readable medium. The computer program includes program code configured to perform the above-described method.

[0110] In the context of this disclosure, a machine-readable medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of machine-readable storage media include an electrical connection with one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0111] The computer program codes for carrying out the methods of the present disclosure may be written in any combination of one or more programming languages. These computer program codes may be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, and when executed by the processor of the computer or other programmable data processing apparatus, the program codes cause the processor to perform the functions / operations specified in the flowcharts and / or block diagrams. The program codes may run entirely on the computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer, or entirely on a remote computer or server.

[0112] Furthermore, although operations are depicted in a particular order, this should not be understood as requiring such operations to be performed in the particular order depicted or in sequential order, or that all of the depicted operations be performed to achieve a desired result. In certain situations, multitasking and parallel processing may be advantageous. Similarly, while some specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features that are described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination.

[0113] It should be noted that the specification and drawings merely illustrate the principles of the proposed method and apparatus. Thus, it will be understood that those skilled in the art will be able to devise various configurations that, although not explicitly described or shown herein, embody the principles of the present invention and are included within its spirit and scope. Furthermore, all examples described herein are expressly intended primarily for educational purposes only to aid the reader in understanding the principles of the proposed method and apparatus, as well as the concepts contributed by the inventor to further the art, and are to be construed as being limited to the examples and conditions specifically described as such. Furthermore, all statements herein describing the principles, aspects, and embodiments of the present invention, as well as specific examples thereof, are intended to encompass their equivalents.

[0114] Several aspects will be described. [Aspect 1] 1. A method of encoding directional audio data, comprising: receiving a mono audio signal corresponding to an audio object and a representation of a radiation pattern corresponding to the audio object, the radiation pattern including sound levels corresponding to a plurality of sample times, a plurality of frequency bands and a plurality of directions; encoding the mono audio signal; and encoding the source radiation pattern to determine radiation pattern metadata; and encoding the radiation pattern includes determining a spherical harmonic transform of a representation of the radiation pattern and compressing the spherical harmonic transform to obtain encoded radiation pattern metadata. method. [Aspect 2] The method of claim 1, further comprising encoding a plurality of directional audio objects based on clusters of audio objects, the radiation patterns representing centroids reflecting average sound level values ​​for each frequency band. [Aspect 3] 3. The method of claim 2, wherein the multiple directional audio objects are encoded as a single directional audio object with directionality corresponding to a time-varying, energy-weighted average of spherical harmonic coefficients of each audio object. Aspect 4 4. The method of claim 2 or 3, wherein the encoded radiation pattern metadata indicates a location of a cluster of audio objects that is an average of a location of each audio object. Aspect 5 5. The method of any one of aspects 1 to 4, further comprising encoding group metadata regarding a radiation pattern of a group of directional audio objects. Aspect 6 6. The method of any one of aspects 1 to 5, wherein the source radiation pattern is rescaled relative to the amplitude of the input radiation pattern in a direction for each frequency to determine a normalized radiation pattern. Aspect 7 7. The method of any one of aspects 1 to 6, wherein compressing the spherical harmonic transform includes at least one of a singular value decomposition method, a principal component analysis, a discrete cosine transform, a data independent basis, or eliminating spherical harmonic coefficients of the spherical harmonic transform above a threshold order of spherical harmonic coefficients. Aspect 8 1. A method of decoding audio data, comprising: receiving an encoded core audio signal, encoded radiation pattern metadata, and encoded audio object metadata; decoding the encoded core audio signal to determine a core audio signal; decoding the encoded radiation pattern metadata to determine a decoded radiation pattern; decoding the audio object metadata; and rendering the core audio signal based on the audio object metadata and the decoded radiation pattern. method. Aspect 9 The method of claim 8, wherein the audio object metadata includes at least one of time-varying three degrees of freedom (3DoF) or six degrees of freedom (6DoF) source orientation information. Aspect 10 The method of claim 8 or 9, wherein the core audio signal includes a plurality of directional objects based on clusters of objects, and the decoded radiation patterns represent centroids reflecting average values ​​for each frequency band. Aspect 11 A method according to any one of aspects 8 to 10, wherein the rendering is based at least in part on applying subband gain to the decoded core audio signal based on the decoded radiation data. Aspect 12 12. The method of any one of aspects 8-11, wherein the encoded radiation pattern metadata corresponds to a time- and frequency-varying set of spherical harmonic coefficients. Aspect 13 13. The method of any one of aspects 8 to 12, wherein the encoded radiation pattern metadata includes audio object type metadata. Aspect 14 The method of claim 13, wherein the audio object type metadata indicates parametric directional pattern data, the parametric directional pattern data including one or more functions selected from a list of functions consisting of a cosine function, a sine function, or a cardioid function. Aspect 15 The method of claim 13, wherein the audio object type metadata indicates database directional pattern data, and decoding the encoded radiation pattern metadata to determine a decoded radiation pattern includes querying a directional data structure containing audio object types and corresponding directional pattern data. Aspect 16 14. The method of claim 13, wherein the audio object type metadata indicates dynamic directional pattern data, the dynamic directional pattern data corresponding to a time- and frequency-varying set of spherical harmonics coefficients. Aspect 17 17. The method of claim 16, further comprising receiving the dynamic directional pattern data before receiving the encoded core audio signal. Aspect 18 interface systems; and 1. An audio decoding device having a control system, comprising: The control system comprises: receiving, via the interface system, audio data corresponding to at least one audio object, the audio data including a monophonic audio signal, audio object position metadata, audio object size metadata, and rendering parameters; determining whether the rendering parameters indicate a positional mode or a directional mode; and upon determining that the rendering parameters indicate a directional mode, rendering the audio data for playback through at least one loudspeaker in accordance with a directional pattern indicated by at least one of the positional metadata or the size metadata. Device. Aspect 19 20. The apparatus of claim 18, wherein rendering the audio data includes interpreting the audio object position metadata as audio object orientation metadata. Aspect 20 The apparatus of claim 19, wherein the audio object position metadata includes at least one of x, y, z coordinate data, spherical coordinate data, or cylindrical coordinate data, and the audio object orientation metadata includes yaw, pitch, and roll data. Aspect 21 21. The apparatus of any one of aspects 18 to 20, wherein the rendering of the audio data includes interpreting the audio object size metadata as directional metadata corresponding to the directional pattern. Aspect 22 22. The apparatus of any one of aspects 18 to 21, wherein rendering the audio data includes querying a data structure including a plurality of directional patterns and mapping at least one of the position metadata or the size metadata to one or more of the directional patterns. Aspect 23 23. The apparatus of claim 22, wherein the control system is configured to receive the data structure via the interface system. Aspect 24 24. The apparatus of claim 23, wherein the data structure is received prior to the audio data. Aspect 25 25. The apparatus of any one of aspects 18 to 24, wherein the audio data is received in a Dolby Atmos format. Aspect 26 26. The apparatus of any one of aspects 18 to 25, wherein the audio object position metadata corresponds to world coordinates or model coordinates.

Claims

1. an interface system; and 1. An audio decoding device having a control system, comprising: The control system comprises: receiving, via the interface system, audio data corresponding to at least one audio object, the audio data including a monophonic audio signal, audio object position metadata, audio object size metadata and rendering parameters, the audio object position metadata including 6DoF source orientation information; determining whether the rendering parameters indicate a positional mode or a directional mode; and upon determining that the rendering parameters indicate a directional mode, rendering the audio data for playback through at least one loudspeaker in accordance with a directional pattern indicated by at least one of the audio object position metadata or the audio object size metadata. Device.

2. A method for decoding audio data, performed by an audio decoding device, comprising: receiving, via an interface system, audio data corresponding to at least one audio object, the audio data including a monophonic audio signal, audio object position metadata, audio object size metadata and rendering parameters, the audio object position metadata including 6DoF source orientation information; determining whether the rendering parameters indicate a positional mode or a directional mode; and upon determining that the rendering parameters indicate a directional mode, rendering the audio data for playback through at least one loudspeaker in accordance with a directional pattern indicated by at least one of the audio object position metadata or the audio object size metadata. method.

3. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 2.

Images