Omnidirectional coding and decoding for ambisonics

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is related to and claims priority to United States Provisional Application No. 62 / 855,202, filed on May 31, 2019, and entitled "OMNI-DIRECTIONAL DIRECT ENCODING AND DECODING,".BACKGROUND

[0002] A spatial audio signal decoder performs operations to convert spatial audio signals from an input spatial audio format to an output spatial audio format. Known spatial audio signal format decoding techniques include passive decoding and active decoding. A passive signal decoder performs decoding operations that are based upon the input spatial audio signal format and the output spatial audio signal format and, in some examples, can use external parameters such as frequency but may not depend upon spatial characteristics of the audio input signal itself, such as the direction of arrival of audio sources in the audio input signal, for example. In other words, a passive signal decoder performs one or more operations independent of the spatial characteristics of the input signal.

[0003] An active signal decoder, on the other hand, performs decoding operations that are based upon the input spatial audio signal format, the output spatial audio signal format and perhaps external parameters such as frequency, for example, as well as spatial characteristics of the audio input signal. An active signal decoder can perform operations that are adapted to the spatial characteristics of the audio input signal.

[0004] Active and passive signal decoders lack universality. Passive signal decoders often blur directional audio sources. For example, passive signal decoders sometimes render a discrete point source in an input audio signal format to all of the channels of an output spatial audio format (corresponding to an audio playback system) instead of to a subset localized to the point-source direction. Active signal decoders, on the other hand, often focus diffuse sources by modeling such sources as directional, for example, as a small number of acoustic plane waves. As a result, an active signal decoder sometimes imparts directionality to nondirectional audio signals. For example, an active signal decoder sometimes renders nondirectional reverberations from a particular direction in an output spatial audio format (corresponding to an audio playback system) such that the spatial characteristics of the reverberation are not preserved by the decoder.

[0005] VILKAMO JUHA ET AL, "Directional Audio Coding: Virtual Microphone-Based Synthesis and Subjective Evaluation", JAES, AES, 60 EAST 42ND STREET, ROOM 2520 NEW YORK 10165-2520, USA, (20090901), vol. 57, no. 9, pages 709 - 724, XP040508924 discusses B-format audio having four microphone channels with different directional characteristics: one omnidirectional, and three channels directed towards orthogonal axis. The document further discusses analyzing diffuseness and direction of the B-formal signal for a plurality of subbands. Diffuseness is discussed as being a fraction of the total sound field energy that is due to the diffuse sound. The remaining portion, that is, the non-diffuse part, is approximated as a plane wave from the analyzed direction using the B-formal signal. The non-diffuse and the diffuse sound streams are calculated by multiplying each virtual microphone signal by specific coefficients. Synthesis of the loudspeakers signals is performed by distributing the omnidirectional microphone signal to the loudspeakers according to the analyzed direction and diffuseness in frequency bands.

[0006] Document US 10 231 073 B2 discloses methods and apparatus that optimally represent full 3D audio mixes (e.g., azimuth, elevation, and depth) as "sound scenes" in which the decoding process facilitates head tracking. Sound scene rendering can be modified for the listener's orientation (e.g., yaw, pitch, roll) and 3D position (e.g., x, y, z). The document also discusses ambisonics.BRIEF SUMMARY

[0007] The invention provides for a method with the features of claim 1, a computer-readable storage medium with the features of claim 15 and a computing apparatus with the features of claim 16. Preferred embodiments of the invention are identified in the dependent claims.

[0008] The present inventors have recognized, among other things, that a problem to be solved includes optimizing audio signal rendering for omnidirectional components of an ambisonic mix, such as using active and passive decoders. In an example, some decoders may not correctly handle directionless or zero-distance information in ambisonic signals, such as "in-head" or "voice-of-god" type directionless signals. Some examples of potentially problematic signals include a 0th (zeroth) order (e.g., W-component only) ambisonic signals, or ambisonics that exhibit symmetry about one or more axes in three-dimensional space. In an example, ambisonic signals where the same signal is symmetrically panned in multiple directions or across nulls of the steering signals can be decoded improperly, such as when an active decoder makes assumptions about directionality. The present inventors have recognized that a solution can include, among other things, separating omnidirectional or middle components from an ambisonic signal, and using separate active and passive decoders for directional and omnidirectional components, respectively.

[0009] In an example, a 0th order or directionless signal, such as can be intended for diffuse or middle-panned rendering, can be added to a W-component of an ambisonic signal. To later decode the signal and recover the 0th order component, information about an energy ratio in the X, Y, and Z components can be used to renormalize the W component, such as with the remainder extracted as the 0th order or omnidirectional component. The renormalized directional part can then be decoded, such as using Harpex-like analysis or other techniques. In an example, the extracted part (which, for example, may have been intentionally added to the W-component) can be passively decoded or can be rendered using a volumetric method to diffuse and distribute the sound energy. In an example, directional but volumetric sources can be included in a mix by applying regular ambisonic panning and processing on multiple decorrelated signals.

[0010] This Brief Summary is intended to provide a summary of some of the present subject matter. It is not intended to provide an exclusive or exhaustive explanation of the invention. The detailed description includes further information and explanation of the present subject matter.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0011] To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced. The drawings illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in the present document. FIG. 1 illustrates generally a block diagram showing an input audio signal in a first spatial audio format, a decoder, and output audio signal in a second spatial audio format. FIG. 2 illustrates generally an example of a spatial audio signal decoder. FIG. 3 illustrates generally an example of a system with active and passive decoders. FIG. 4 illustrates generally a first example of a signal decomposition. FIG. 5 illustrates generally a second example of a signal decomposition. FIG. 6 illustrates generally an example of a method of processing an input ambisonic signal. FIG. 7 illustrates generally an example of a method of generating a secondary ambisonic signal. FIG. 8 illustrates generally an example of a method of preparing ambisonic or other signals for further processing by an active or passive decoder. FIG. 9 illustrates generally a diagram of a machine in the form of a computer system within which a set of instructions may be executed for causing the machine to perform any one or more of the methods discussed herein. DETAILED DESCRIPTION

[0012] Terms such as spatial encoding, spatial coding, or spatial audio coding can refer to representing a sound scene or soundfield in terms of audio signals and side information. The terms spatial format or spatial audio format or spatial audio signal can refer to audio signals and side information that represent a sound scene or soundfield. In an example, side information can include a definition of a format, such as directional characteristics corresponding to each of the audio channels in the format, and in some cases, can include signal-dependent information such as directions of sources present in audio signals. A spatial audio signal includes one or more constituents that may be referred to as audio signal components, or audio channels or sources. In some examples, a spatial audio signal may be referred to as an audio signal in a spatial format.

[0013] Spatial decoding, or spatial audio decoding, can include processing an input spatial audio signal in a specified spatial audio format to generate an output spatial audio signal in a specified spatial audio format. In an example, decoding can correspond to transcoding from an input spatial audio format to a different spatial audio format or decoding can include generating signals for playback over a specified audio reproduction system, such as a multichannel loudspeaker layout. An audio reproduction system can, in some examples, correspond to a spatial audio format or immersive format, such as a surround sound format.

[0014] In an example, active or parametric spatial audio systems can be configured to make determinations or assumptions about a direction of one or more audio signal sources. Some sources, however, can be intended to be rendered or reproduced with substantially no distance, that is, they can be considered to originate from essentially all directions simultaneously. In some examples, such signals or sources can be sourced from various panned objects. Examples discussed herein can encode such sources into an ambisonic mix such that the sources can be properly decoded downstream by active or parametric decoders.

[0015] In an example, information about one or more of an omnidirectional, "middle," or non-diegetic component can be added to an ambisonic source signal, or ambisonic source mix. That is, such signals or components can be encoded together with other directional components in an ambisonic mix, such as without using extra channels or metadata. A decoder receiving such a mix can identify residual energy, or 0th (zeroth) order energy, such as before or after accounting for diffuse or directional components in the same mix. In this example, the residual energy can be rendered as either a middle channel (e.g., having non-directional components, having zero distance, or being a "voice-of-god" type channel) or as an oiniii-voluinetric signal.

[0016] In another example, information from an ambisonic source can be pre-rendered as a volumetric source. In this example, omnidirectional components can be extracted by a parametric decoder as a diffuse component in a desired or intended number of different directions.

[0017] FIG. 1 illustrates generally a first block diagram 100 that includes a representation of operation of a spatial audio decoder 106 to decode an input spatial audio signal 102 in an input spatial audio format 104. The spatial audio decoder 106 can be configured to decode the input spatial audio signal 102 to an output spatial audio signal 108 such as in an output spatial audio format suitable for a multichannel audio reproduction system 110.

[0018] In an example, the spatial audio decoder 106 can transform an input signal, such as can be received in a first-order ambisonics B-format or other spatial audio format, to an output signal in a multichannel audio format suitable for playback using the multichannel audio reproduction system 110. In an example, the spatial audio decoder 106 can be implemented as an active or passive decoder.

[0019] In the case of a passive decoder, the decoder can perform transformation operations from the input spatial format to the output spatial format independent of spatial characteristics of the audio input signal, such as direction of arrival of the audio input signal. In an example, the spatial audio decoder 106 can be implemented as an active decoder that performs transformation operations from the input spatial format to the output spatial format based at least in part upon spatial characteristics of the audio input signal, or the input spatial audio signal 102. In an example, a rendered output using the multichannel audio reproduction system 110 can include information about a point source 112, such as can be decoded using an active decoder, can include information about a volumetric source 114, such as can be decoded using an active or passive decoder, or can include information about a middle source (e.g., at the reference listening position), such as can be decoded using an active or passive decoder.

[0020] FIG. 2 illustrates generally a spatial audio signal decoder example 200. The decoder example 200 can represent a configuration of the spatial audio decoder 106 from the example of FIG. 1. In an example, the decoder can be configured to map an input spatial audio signal in an input spatial format to an output spatial audio signal in an output spatial format. As further described below with refence to FIG. 3, one example decoder can be configured as an active signal decoder 308, and another example decoder can be configured as a passive signal decoder 310. It can be appreciated that each input spatial audio signal can include multiple audio signal components and that each output spatial audio signal can include multiple audio signal components. The respective audio signal components may be referred to as channels. The example decoder of FIG. 2 can include or use one or more mapping operations to map M input spatial audio signal components to N spatial audio output signal components. More particularly, an example mapping operation includes an M-by-N spatial decoder matrix to map M input spatial audio signal components in an input spatial format to N spatial audio output signal components in an output spatial format. The mapping operations can be used as a basis to configure the decoder as an active signal decoder or a passive signal decoder.

[0021] In the example of the spatial audio decoder 106, the value of M is four since the input spatial format is a first-order ambisonics B-format signal that has four signal components, and the value of N depends, for example, on the number of speakers in the multichannel audio reproduction system. The spatial format of the input spatial audio signal received by the example signal decoder can include B-format audio input signal components W, X, Y, and Z, such as having directivity patterns given by the respective elements in the vector (d)(Q) defined as d ⇀ Ω = 1 2 cos θ cos ϕ sin θ cos ϕ sin ϕ where Ω corresponds to an angular pair consisting of an azimuth angle θ and an elevation angle ϕ with respect to a reference point for measurement. A spatial audio scene or soundfield can be encoded in the W, X, Y, and Z components in accordance with the directivity patterns defined in the above vector. For instance, a point source S at azimuth angle θ and elevation angle ϕ is encoded in B-format components as W X Y Z = 1 2 cos θ cos ϕ sin θ cos ϕ sin ϕ S

[0022] Thus ambisonics, such as B-format ambisonics, can be used to represent a soundfield by capturing or encoding a fixed set of signals corresponding to a single point in the soundfield. Each of the fixed set of signals in an ambisonic representation has a defined directivity pattern. The directivity patterns are designed such that ambisonic-encoded signals carry directional information for all of the sounds in an entire soundfield. An ambisonic encoder (not shown) encodes a soundfield in an ambisonic format, and the format can be independent from a specific loudspeaker layout which may be used to reconstruct an encoded soundfield. An ambisonic decoder decodes ambisonic format signals for a specific loudspeaker layout. Eric Benjamin, Richard Lee, and Aaron Heller, in "Is My Decoder Ambisonic?" published in 125th AES Convention, San Francisco 2008, provides a general explanation of ambisonics.

[0023] In some examples, a signal decoder transforms an input audio signal in an input spatial format to an output audio signal in an output spatial format suitable for a five-loudspeaker layout as depicted in FIG. 1. The examples are not limited to the multichannel loudspeaker layout depicted in FIG. 1, however. Example signal decoders can be configured to decode to a 5.1 loudspeaker layout, a 7.1 loudspeaker layout, an 11.1 loudspeaker layout, or other loudspeaker layout, for example. In other examples, the signal decoder transforms an input audio signal in an input spatial format to an output audio signal in a two-channel binaural format. The examples are not limited to input audio signals in the first-order ambisonics B-format. In other examples, the signal decoder transforms an input audio signal in a higher-order ambisonics format to an output audio signal in an output spatial format. In an example, a decoder or other processor can include or use headtracking information to further enhance a playback experience for a listener.

[0024] FIG. 3 illustrates generally a schematic block diagram of a spatial audio signal decoder system 300. The spatial audio signal decoder system 300 can include a computer or processor system that includes one or more processor devices configured to be operatively coupled to one or more computer memory storage devices (e.g., a non-transitory storage device) that store instructions to configure the processor devices to provide the processing blocks described with reference to FIG. 3. In the example of FIG. 3, the spatial audio signal decoder system 300 can include one or more of a time-frequency transformation block 304, an active / passive decomposition block 306, an active signal decoder 308, a passive signal decoder 310, a combiner block 312, and an inverse time-frequency transformation block 314.

[0025] The time-frequency transformation block 304 can be configured to receive a time-domain input spatial audio signal, such as the input signal 302 in an input spatial audio format, and can convert the input signals to a time-frequency domain input signal 318. Subsequent processing is carried out in a corresponding time-frequency domain. An alternative example first spatial audio signal decoder system (not shown) omits the time-frequency transformation block so that subsequent processing is carried out in the time domain.

[0026] The active / passive decomposition block 306 can be configured to decompose the time-frequency domain input signal 318, such as in the time-frequency domain, to produce an active input component 320 and passive input component 322. In an example, the active input component 320 and the passive input component 322 can sum or add up to the time-frequency domain input signal 318. In an example, each of the active input component 320 and the passive input component 322 can have respective multiple components or component signals. In the spatial audio signal decoder system 300, the components can be in the same spatial audio format as the input signal 302 as received in the time domain.

[0027] In an example, the input signal 302 can include an ambisonic signal such as can include X, Y, Z, and W components. In an example, the input signal 302 can comprise an ambisonic signal with an excess W component, such as can result from a multidirectional source signal having symmetry along one or more axes (e.g., axes X, Y, and Z of a three-dimensional space). For example, a phantom center signal constructed from identical sources that are evenly panned on either side of a listener can be such a source. Such a phantom center signal can have a primary direction but can have a more ambiguous or less certain position.

[0028] In the example spatial audio signal decoder system 300, the active signal decoder 308 and the passive signal decoder 310 can transform their respective inputs from an input spatial format to respective decoder output spatial audio signals, such as including active decoder output signals 324 and passive decoder output signals 326, that can have a common output spatial format such as a common multichannel loudspeaker layout. In another example decoder (not shown), different respective ones of the decoders can transform respective decoder input audio signals to respective decoder output audio signals having different spatial formats.

[0029] In an example, the active signal decoder 308 can receive the active input component 320 and, in response, provide the active decoder output signals 324. The output format of the active decoder output signals 324 can be determined by a configuration of the active signal decoder 308. For example, a feature of ambisonics and other spatial audio encoding methods is to be agnostic to the output format, meaning the input spatial audio signal can be decoded to whatever format the decoder is configured to provide. The active signal decoder 308 can transform the active input component 320, such as can have a respective input spatial format, to active decoder output signals 324 having a particular or designated active signal output spatial format. The passive signal decoder 310 can receive the passive input component 322 and, in response, provide the passive decoder output signals 326. The output format of the passive decoder output signals 326 can be determined by a configuration of the passive signal decoder 310. The passive signal decoder 310 can transform the passive input component 322, such as having a respective input spatial format, to the passive decoder output signals 326, such as can have a specified passive signal output spatial format.

[0030] In an example, the passive signal decoder 310 can partition the passive input component 322 into one or more frequency bands such that different processing can be applied to each frequency band. For instance, an example passive signal decoder 310 can be configured to perform a lower frequency range transformation operation for a frequency range of the passive input component 322 below a cutoff frequency and is configured to perform an upper frequency range transformation operation for a frequency range of the passive input component 322 above the cutoff frequency.

[0031] In an example, the passive signal decoder 310 can be configured to apply decorrelation processing or filtering to at least a portion of the passive input component 322 to thereby provide a decorrelated passive decoder output signals 326. In an example, the passive signal decoder 310 can render the decorrelated signals as an envelopment source. An envelopment source can include sound source information that originates, or is perceived by a listener to originate, from essentially all directions, and may or may not change with listener movement in any direction. In an example, a reverb signal can be considered to be enveloping source or an envelopment source.

[0032] In an example, the combiner block 312 can include a summation circuit to sum the respective output spatial audio signals from the decoders, such as the active decoder output signals 324 and the passive decoder output signals 326. The combiner block 312 can provide summed output signals 328 representing the combination of the active decoder output signals 324 and the passive decoder output signals 326. In an example, the combiner block 312 can perform other processing such as filtering or decorrelation. Filters, such as all-pass filters or others, can be applied to one or more channels of the passive decoder output signals 326 to decorrelate the channels prior to the combination with the active decoder output signals 324 by the combiner block 312. Decorrelation of the channels can lead to a more diffuse and less directional rendering, which is generally preferable for the passive or omnidirectional components. In an example, additional processing of the decoded signal components can be carried out before combining the decoded signal components. For example, different filters may be applied to the active and passive components. In another example, additional processing of the decoded signal components can be carried out after combining the decoded signal components, for example, equalization or other filtering can be performed or applied.

[0033] In an example, the inverse time-frequency transformation block 314 can receive the summed output signals 328 from the combiner block 312. The inverse time-frequency transformation block 314 can be configured to convert the summed output signals 328, such as in the time-frequency domain, to a time-domain output, or time-domain spatial audio output signals 316. The time-domain spatial audio output signals 316 can be provided to a sound reproduction system, such as the multichannel audio reproduction system 110 from the example of FIG. 1, or can be provided to another signal processor or network for distribution.

[0034] FIG. 4 illustrates generally an example 400 of a first decomposition block 406. In an example, the active / passive decomposition block 306 from the example of FIG. 3 comprises or corresponds to the first decomposition block 406. The first decomposition block 406 can include a 0th order determination block 410 and a normalization block 412.

[0035] In an example, the 0th order determination block 410 is configured to receive the time-frequency domain input signal 318 such as comprising an ambisonic signal. The 0th order determination block 410 can be configured to perform various processing operations, as discussed elsewhere herein, to provide a directional component 402 and an omnidirectional component 404. The directional component 402 can include, for example, a first order ambisonic signal such as comprising X, Y, Z, and W components. In some examples, the W component of the directional component 402 can be unbalanced relative to the other components of the directional component 402. The normalization block 412 can be configured to analyze the X, Y, and Z components and re-scale or normalize the W component to thereby provide a normalized output 408 that comprises balanced W, X, Y, and Z components. That is, the normalization block 412 can be used to prepare an ambisonic signal for output by correcting a perceived imbalance in energy with respect to the X, Y, Z, and W components of the signal.

[0036] In an example, the normalized output 408 includes or comprises the active input component 320 such as can be provided to the active signal decoder 308. In another example, the normalization block 412 can be configured to prepare or provide an auxiliary ambisonic signal at its output by subtracting the omnidirectional component 404 from the directional component 402. In this example, the resulting auxiliary ambisonic signal can include one or more non-zero directional components.

[0037] The omnidirectional component 404 can include or represent information about an omnidirectional signal, or less-directional signal, that is present in the time-frequency domain input signal 318. The omnidirectional component 404 can include or comprise the passive input component 322 such as can be provided to the passive signal decoder 310. In an example, the omnidirectional component 404 can include a middle channel, or in-head channel, or "voice of god" channel that can be rendered by the passive signal decoder 310.

[0038] In an example, the omnidirectional component 404 can comprise an ambisonic signal with a W component and, in some examples, can omit X, Y, or Z components. In an example, a multidirectional source with perfect symmetry about all directional axes can comprise an ambisonic signal with only a W component. Generally, in the discussions herein, a "W-only" signal can be considered a 0th order ambisonic signal.

[0039] FIG. 5 illustrates generally an example 500 of a second decomposition block 506. In an example, the active / passive decomposition block 306 from the example of FIG. 3, or the first decomposition block 406 from the example of FIG. 4, comprises or corresponds to the second decomposition block 506.

[0040] The second decomposition block 506 can be configured to receive the time-frequency domain input signal 318 and route its various components to one or more of a direction estimation block 502, a subspace determination block 504, or the 0th order determination block 410. The direction estimation block 502 can provide an estimate 510 of a number and direction of arrival (DOA) of directional audio sources in the time-frequency domain input signal 318, in accordance with an input spatial audio format. The subspace determination block 504 can determine the active input component 320 based upon the estimate 510 of the number and DOAs of directional sound sources in the time-frequency domain input signal 318. An example subspace determination block 504 determines the active input component 320 by projecting the active signal component onto a subspace that can be determined based on the number and DOAs of directional sound sources and the time-frequency domain input signal 318.

[0041] The 0th order determination block 410 can be used to determine a value or values that can be used to scale a residual, passive, or omnidirectional component of the time-frequency domain input signal 318. In an example, the 0th order determination block 410 can determine or provide a scale factor for scaling information between the active input component 320 and the passive input component 322. The residual determination block 508 can determine the passive input component 322, for example, based on a difference between the time-frequency domain input signal 318 and the active input component 320 as-determined by the subspace determination block 504.

[0042] In an alternative example decomposition block, the passive input component 322 can be determined first, and the active input component 320 can be determined thereafter based upon a difference between the received time-frequency domain input signal 318 and the passive input component 322.

[0043] As a result of processing using the second decomposition block 506, active signals or the active input component 320 can include functionally directional source signals. The passive input component 322 can include a combination of non-directional or omnidirectional sources, such as can be detected using the 0th order determination block 410. In an example, the passive input component 322 can include diffuse or residual error-based signals as identified by the subspace determination block 504 and the residual determination block 508. Other examples of determining residual error and allocating residual signals are discussed by Goodwin et al. in "Spatial Audio Signal Decoder," PCT application PCT / US2019 / 046936, filed August 16, 2019 (herein "Goodwin et al.'').

[0044] FIG. 6 illustrates generally an example of a first method 600 that can include processing an input ambisonic signal to separate omnidirectional components that can be rendered using a passive decoder from other, directional components that can be rendered using an active decoder. At block 602, the example first method 600 can include receiving a B-format primary ambisonic signal such as including W, X, Y, and Z components. In an example, block 602 can include receiving the input spatial audio signal 102 such as comprising the input signal 302. In an example, block 602 can include receiving the input signal 302 using one or more of the active / passive decomposition block 306, the first decomposition block 406, or the second decomposition block 506.

[0045] At block 604, the first method 600 can include determining, such as for a first signal band or frequency band of the primary ambisonic signal, a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal. In an example, the active / passive decomposition block 306 can be configured to receive the input signal 302 and analyze the X, Y, Z, and W components of the input signal 302 such as to identify any imbalance of the W component relative to the X, Y, and Z components. If a mismatch or imbalance exists in the input signal 302 then the input signal 302 can be considered to have an omnidirectional component 404 that can be extracted from the input signal 302 such as for subsequent processing that can be different than the processing used for the other, directional component 402.

[0046] At block 606, the first method 600 can include determining a secondary ambisonic signal using information about a difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal. In an example, the active / passive decomposition block 306 can include a signal processor configured to determine or generate the secondary ambisonic signal to represent the omnidirectional component 404. In an example, the secondary ambisonic signal can include a 0th order ambisonic signal, or can include a single channel signal that includes exclusively omnidirectional signal information.

[0047] In an example, block 606 can include determining a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the input signal 302. For example, determining the total energy can include determining whether a sum of squared magnitudes of the X, Y, and Z components is substantially equal to a square magnitude of the W component. In other words, (X^2 + Y^2 + Z^2) / 2w^2 should be identically 1. In another example, determining the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the input signal 302 can include determining whether component attributes of the input signal 302 are mathematically equivalent to solutions of assumed panning equations and normalizations for the input signal 302, such as can have different values or requirements depending on a format used.

[0048] At block 608, the first method 600 can include generating an output signal that is based on the secondary ambisonic signal, such as using the passive signal decoder 310. In an example, the output can be a middle channel, or in-head type signal, sometimes referred to a "voice of god" signal because it can be interpreted by a listener as either originating inside the listener's head or as originating from all directions simultaneously.

[0049] At block 610, the first method 600 can include determining or generating a re-balanced primary ambisonic signal based on a difference between the primary ambisonic signal as-received and the secondary ambisonic signal such as determined at block 606. In an example, block 610 can include using the normalization block 412 to scale the primary ambisonic signal or to generate a W component that corresponds to the X, Y, and Z components of the primary ambisonic signal, such as received at block 602. At block 612, the first method 600 can include generating an output signal that is based on the re-balanced primary ambisonic signal, such as using the active signal decoder 308.

[0050] FIG. 7 illustrates generally an example of a second method 700 that can include generating a secondary ambisonic signal, such as using the active / passive decomposition block 306. The second method 700 can begin at block 606, which is described above in the discussion of FIG. 6. At block 702, the second method 700 can include identifying one or more first-order or higher-order ambisonic steering components for the secondary ambisonic signal. The one or more directional steering components can be based on respective steering components from the primary ambisonic signal.

[0051] For example, at block 704, the second method 700 can include computing a residual error projection matrix for the W, X, Y, and Z components of the primary ambisonic signal, such as received at block 602. A residual error projection matrix can be computed or determined according to the systems and methods discussed by Goodwin et al. The residual can represent a passive signal component as a difference between an input signal and the primary ambisonic signal.

[0052] At block 706, the second method 700 can include determining an omni-signal return factor. The omni-signal return factor can be based on the determined difference between the total energy of X, Y, and Z components and the total energy of the W component of the input signal.

[0053] At block 708, the second method 700 can include updating the residual error projection matrix using the determined omni-signal return factor from block 706. That is, the omni-signal return factor from block 706 can indicate a total energy difference (e.g., an excess) of the W component relative to the X, Y, and Z components. Updating the projection matrix can include rebalancing the values such that the residual or excess W component is removed. In an example, the matrix can be rebalanced such that any excess W component is allocated to the residual or passive component.

[0054] At block 710, the second method 700 can include applying the updated residual error projection matrix to the primary ambisonic signal to thereby generate the secondary ambisonic signal. The generated secondary ambisonic signal can be provided to the passive signal decoder 310 for further processing or rendering.

[0055] Goodwin et al. explains generally various systems and methods for determining a residual signal, or portions of an input signal that may not be adequately represented by directional components that can be detected in the input. For example, an input signal including a front-left source with some volumetric attribute (e.g., not a pure point source) may have a residual. The decoder described by Goodwin et al. can, in an example, detect a center of the source and designate directionality based on the center (e.g., 45 degrees left). The decoder can then test the directionality hypothesis by forming a pure point source at the given direction (e.g., 45 degrees left). A difference between the pure point source and the original volumetric signal can, in theory, be designated as the residual and can thus be passively decoded (or, non-point source rendered), while the center or point source information can be actively decoded.

[0056] In an example that includes a phantom center signal (e.g., a source having substantially equal parts at -45 degrees and +45 degrees from front center), an ambisonic signal that includes the phantom signal will cancel for left-right balance. Since the source is intended to be front center, the decoder of Goodwin et al. can produce a strong front center signal. The residual can be minimized because the left-right balance was cancelled out. Any information that the source was a center image made of respective sources on the left and right can be hidden in the relationship to the magnitude, rather than the direction, of the signal. In an example, the present systems and methods can help identify or determine a spaciousness or breadth of the source and can move a different portion of the source to the passive decoder stream, which in turn can render the source more as a phantom center (as in the original input signal).

[0057] FIG. 8 illustrates generally an example of a third method 800 that can include preparing different ambisonic or other signals for further processing by an active or passive decoder, such as by the active signal decoder 308 or the passive signal decoder 310. The third method 800 can be performed at least in part using the active / passive decomposition block 306 and can begin at block 802 with receiving a primary ambisonic signal having information about a volumetric source. The primary ambisonic signal can include or comprise the input signal 302. The volumetric source can comprise a source with various directional cues and can have ambiguous or less-directional characteristics.

[0058] At block 804, the third method 800 can include determining a secondary ambisonic signal having one or more non-zero directional components. The non-zero directional components can be based on the directional cues associated with volumetric source. In an example, the secondary ambisonic signal can represent predominantly the less-directional characteristics from the volumetric source.

[0059] At block 806, the third method 800 can include generating a tertiary ambisonic signal. In an example, the tertiary ambisonic signal can be generated by subtracting the secondary ambisonic signal from the primary ambisonic signal such that the tertiary ambisonic signal includes one or more of the non-zero directional components.

[0060] At block 808, the third method 800 can include outputting the one or more non-zero directional components of the tertiary ambisonic signal to an active decoder, such as to the active signal decoder 308. Block 810 can include outputting the secondary ambisonic signal to a passive decoder, such as the passive signal decoder 310.

[0061] One or more of the methods discussed herein can be used to process other-order ambisonic signals, or to process single-plane ambisonic signals. For example, an example of the input signal 302 can include or comprise a horizontal-only ambisonic signal comprising W, X, and Y components. In this case, any extracted or determined omnidirectional imbalance can be attributed to the "missing" Z component. A secondary ambisonic signal can be generated for W and Z, which could in turn be rendered as the missing height component.

[0062] FIG. 9 is a diagrammatic representation of a machine 900 within which instructions 908 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 900 to perform any one or more of the methodologies discussed herein can be executed. For example, the instructions 908 can cause the machine 900 to execute any one or more of the methods described herein. The instructions 908 can transform the general, non-programmed machine 900 into a particular machine 900 programmed to carry out the described and illustrated functions in the manner described.

[0063] In an example, the machine 900 can operate as a standalone device or can be coupled (e.g., networked) to other machines or devices or processors. In a networked deployment, the machine 900 can operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 900 can comprise a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 908, sequentially or otherwise, that specify actions to be taken by the machine 900. Further, while only a single machine 900 is illustrated, the term "machine" can be taken to include a collection of machines that individually or jointly execute the instructions 908 to perform any one or more of the methodologies discussed herein. In an example, the instructions 908 can include instructions stored using a memory circuit, and the machine 900 can include or use a processor circuit such as can be associated with any one or more of the various blocks, modules, processors, or other processing hardware or software discussed herein.

[0064] The machine 900 can include various processors and processor circuitry, such as represented in the example of FIG. 9 as processors 902, memory 904, and I / O components 942, which can be configured to communicate with each other via a bus 944. In an example, the processors 902 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an ASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) can include, for example, a processor 906 and a processor 910 that execute the instructions 908. The term "processor" is intended to include multi-core processors that can comprise two or more independent processors (sometimes referred to as "cores") that can execute instructions contemporaneously. Although FIG. 9 shows multiple processors, the machine 900 can include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

[0065] The memory 904 can include a main memory 912, a static memory 914, or a storage unit 916, such as can be accessible to the processors 902 via the bus 944. The memory 904, the static memory 914, and storage unit 916 can store the instructions 908 embodying any one or more of the methods or functions or processes described herein. The instructions 908 can also reside, completely or partially, within the main memory 912, within the static memory 914, within the machine-readable medium 918 within the storage unit 916, within at least one of the processors (e.g., within a processor's cache memory), or any suitable combination thereof, during execution thereof by the machine 900.

[0066] The I / O components 942 can include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I / O components 942 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones can include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I / O components 942 can include many other components that are not shown in FIG. 9. In various example embodiments, the I / O components 942 can include output components 928 and input components 930. The output components 928 can include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The input components 930 can include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and / or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

[0067] In an example, the I / O components 942 can include biometric components 932, motion components 934, environmental components 936, or position components 938, among a wide array of other components. For example, the biometric components 932 include components configured to detect a presence or absence of humans, pets, or other individuals or objects, or configured to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram-based identification), and the like. The motion components 934 can include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope), and so forth.

[0068] The environmental components 936 can include, for example, illumination sensor components (e.g., photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors to detection concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components 938 include location sensor components (e.g., a GPS receiver component, an RFID tag, etc.), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude can be derived), orientation sensor components (e.g., magnetometers), and the like.

[0069] The I / O components 942 can include communication components 940 operable to couple the machine 900 to a network 920 or devices 922 via a coupling 924 and a coupling 926, respectively. For example, the communication components 940 can include a network interface component or another suitable device to interface with the network 920. In further examples, the communication components 940 can include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth ®< components (e.g., Bluetooth ®< Low Energy), Wi-Fi ®< components, and other communication components to provide communication via other modalities. The devices 922 can be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

[0070] Moreover, the communication components 940 can detect identifiers or include components operable to detect identifiers. For example, the communication components 940 can include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information can be derived via the communication components 940, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi ®< signal triangulation, or location via detecting an NFC beacon signal that can indicate a particular location, and so forth.

[0071] The various memories (e.g., memory 904, main memory 912, static memory 914, and / or memory of the processors 902) and / or storage unit 916 can store one or more instructions or data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions 908), when executed by processors or processor circuitry, cause various operations to implement the embodiments discussed herein.

[0072] The instructions 908 can be transmitted or received over the network 920, using a transmission medium, via a network interface device (e.g., a network interface component included in the communication components 940) and using any one of a number of well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions 908 can be transmitted or received using a transmission medium via the coupling 926 (e.g., a peer-to-peer coupling) to the devices 922.

[0073] This disclosure has been described in detail and with reference to exemplary embodiments thereof, it will be apparent to one skilled in the art that various changes and modifications can be made.

[0074] The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show specific embodiments by way of illustration. These embodiments are also referred to herein as "examples." Such examples can include elements in addition to those shown or described. Moreover, the subject matter may include any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0075] In this document, the terms "a" or "an" are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of "at least one" or "one or more." In this document, the term "or" is used to refer to a nonexclusive or, such that "A or B" includes "A but not B," "B but not A," and "A and B," unless otherwise indicated. In this document, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein." Also, in the following claims, the terms "including" and "comprising" are open-ended, that is, a system, device, article, composition, formulation, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms "first," "second," and "third," etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0076] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with each other. Other embodiments can be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, the subject matter may lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment, and it is contemplated that such embodiments can be combined with each other in various combinations or permutations.

[0077] The scope of the invention is defined by the appended claims.

Claims

1. A method comprising: receiving (602) a B-format primary ambisonic signal including W, X, Y, and Z components; determining (604), for a first signal band of the primary ambisonic signal, a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal; determining (606) a secondary ambisonic signal using information about the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal; generating (608) an output signal based on the secondary ambisonic signal, the generating including scaling the secondary ambisonic signal for one or more of independent output, decoding, and spatial rendering; wherein the method is characterised by: generating (610) a re-balanced primary ambisonic signal based on a difference between the primary ambisonic signal as-received and the secondary ambisonic signal; and generating (612) an output signal based on the re-balanced primary ambisonic signal.

2. The method of claim 1, wherein determining the secondary ambisonic signal includes determining a single-channel signal that includes exclusively omnidirectional signal information.

3. The method of claim 2, further comprising rendering the single-channel signal as a middle channel audio signal, wherein the middle channel audio signal is a signal configured to be heard as originating inside a listener's head or as originating from all directions simultaneously.

4. The method of claim 2, further comprising applying decorrelation processing to at least a portion of the secondary ambisonic signal to provide a decorrelated signal, and rendering the decorrelated signal as an envelopment source.

5. The method of claim 1, wherein the secondary ambisonic signal includes one or more first-order or higher-order ambisonic steering components, wherein the one or more first-order or higher-order ambisonic steering components are based on respective steering components from the primary ambisonic signal.

6. The method of claim 5, wherein determining the secondary ambisonic signal includes: computing a residual error projection matrix for the W, X, Y, and Z components of the received primary ambisonic signal; determining an omni-signal return factor based on the determined difference between the total energy of X, Y, and Z components and the total energy of the W component of the primary ambisonic signal, wherein the omni-signal return factor indicates a total energy difference of the W component relative to the X, Y, and Z components; updating the residual error projection matrix using the determined omni-signal return factor; and applying the updated residual error projection matrix to the primary ambisonic signal to thereby generate the secondary ambisonic signal.

7. The method of claim 1, wherein the secondary ambisonic signal consists of 1st or higher-order ambisonic signals including an omni-directional channel and one or more directional steering channels based on the received primary ambisonic signal.

8. The method of claim 1, further comprising generating a tertiary ambisonic signal comprising the primary ambisonic signal less the determined difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal.

9. The method of claim 1, further comprising preparing the primary ambisonic signal for output by scaling the W component of the primary ambisonic signal to correct a perceived imbalance in energy with respect to the X, Y, and Z components of the primary ambisonic signal.

10. The method of claim 1, further comprising generating a tertiary ambisonic signal by subtracting the secondary ambisonic signal from the primary ambisonic signal such that the tertiary ambisonic signal includes one or more non-zero directional components.

11. The method of claim 10, wherein receiving the primary ambisonic signal includes receiving information about a volumetric source having directional cues and less-directional characteristics; wherein determining the secondary ambisonic signal includes generating a signal having one or more non-zero directional components based on the directional cues while representing predominately the less-directional characteristics; wherein the one or more non-zero directional components of the tertiary ambisonic signal predominately represent the directional cues; and the method further comprising: outputting, to an active decoder, the one or more non-zero directional components of the tertiary ambisonic signal; and outputting, to a passive decoder or a less-directional renderer, the secondary ambisonic signal; wherein the active decoder performs operations that are adapted to the spatial characteristics of the audio input signal and wherein the passive decoder performs operations independent of the spatial characteristics of the input signal.

12. The method of claim 10, further comprising providing the tertiary ambisonic signal to an active decoder and providing the secondary ambisonic signal to a passive decoder, wherein the active decoder performs operations that are adapted to the spatial characteristics of the audio input signal and wherein the passive decoder performs operations independent of the spatial characteristics of the input signal.

13. The method of claim 1, wherein the output signal based on the secondary ambisonic signal represents omnidirectional components and is rendered using a passive decoder for the omnidirectional components and wherein the output signal based on the re-balanced primary ambisonic signal represents directional components and is rendered using an active decoder for the directional components, wherein the active decoder performs operations that are adapted to the spatial characteristics of the audio input signal and wherein the passive decoder performs operations independent of the spatial characteristics of the input signal.

14. The method of claim 1, wherein a directionless signal has been added to the W- component of the received B-format primary ambisonic signal, or wherein the W-component of the received B-format primary ambisonic signal is an excess W-component resulting from a multidirectional source signal having symmetry along one or several axes, or wherein the W-component of the received B-format primary ambisonic signal is unbalanced relative to the other components; and determining (604) a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal comprises identifying any imbalance of the W component relative to the X, Y, and Z components.

15. A computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform the method of claim 1.

16. A computing apparatus, the computing apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, configure the apparatus to: perform the method of claim 1.