Sound field related rendering

Abstract

To provide apparatuses and methods for sound-field related audio representation and rendering.SOLUTION: There is provided an apparatus including means configured to: obtain a defocus direction; process a spatial audio signal that represents an audio scene to generate a processed spatial audio signal that represents a modified audio scene based on the defocus direction, so as to control deemphasis in a portion of the spatial audio signal in the defocus direction relative to at least in part other portions of the spatial audio signal; and output the processed spatial audio signal based on the defocus direction. The modified audio scene based on the defocus direction enables the deemphasis in, at least in part, the portion of the spatial audio signal in the defocus direction relative to at least in part other portions of the spatial audio signal.SELECTED DRAWING: Figure 1a

Description

[Technical field]

[0001] The present application relates to an apparatus and method for sound field related audio representation and rendering, although not limited to audio representation for audio decoders. [Background technology]

[0002] Spatial audio playback is known for presenting media with multiple viewing directions. Examples of this playback include (at least) the visual content of the media, including playback on a head-mounted display (or a phone in a head mount) with head orientation tracking, or on a phone screen without a head mount where the view direction can be tracked by changing the phone position / orientation, or by any user interface gesture, or on a surrounding screen.

[0003] Videos related to "media with multiple viewing directions" can be, for example, 360-degree videos, 180-degree videos, or other videos with a substantially wider viewing angle than traditional videos. Traditional videos typically refer to video content that is viewed in its entirety on a screen without the option (or any particular need) to change viewing direction.

[0004] Audio associated with videos with multiple gaze directions can be presented over headphones or using a surround loudspeaker setup, where the gaze directions are tracked and affect the spatial audio playback.

[0005] Spatial audio associated with videos with multiple viewing directions may come from a microphone array (e.g., an array attached to a VR camera such as an OZO, or a handheld mobile device), or spatial audio capture from other sources such as a studio mix. The audio content may be a mixture of several content types, such as microphone-captured sound and added commentary tracks.

[0006] Spatial audio associated with a video with multiple viewing directions can be in various forms, e.g., an Ambisonic signal (of any order) consisting of spherical harmonic audio signal components. The spherical harmonic functions can be thought of as a set of spatially selective beam signals. Ambisonics are currently utilized, e.g., in the YouTube 360 ​​VR video service. The advantage of Ambisonics is that it is a simple and well-defined signal representation. Surround loudspeaker signals, e.g., 5.1. Currently, typical cinematic spatial audio is conveyed in this format. The advantage of surround loudspeaker signals is their simplicity and legacy compatibility. Some audio formats similar to surround loudspeaker signal formats contain audio objects, which can be considered as audio channels with time-varying positions. The position can inform both the direction and distance of the audio object, or the orientation, i.e., parametric spatial audio, such as two audio channel audio signals and associated spatial metadata in perceptually relevant frequency bands. Several state-of-the-art audio encoding and spatial audio capture methods apply such signal representations. The spatial metadata essentially determines how the audio signal should be spatially reproduced at the receiver (e.g., in which directions at different frequencies). The advantages of parametric spatial audio are its versatility, quality, and ability to use low bit rates for encoding. Summary of the Invention

[0007] According to a first aspect, there is provided an apparatus comprising means configured to obtain a defocus direction; process a spatial audio signal representing an audio scene to generate a processed spatial audio signal representing a modified audio scene based on the defocus direction to at least partially control a relative de-emphasis of a portion of the spatial audio signal in the defocus direction relative to at least some other portions of the spatial audio signal; and output the processed spatial audio signal relative to at least some other portions of the spatial audio signal, wherein the modified audio scene based on the defocus direction at least partially enables de-emphasis of a portion of the spatial audio signal in the defocus direction relative to at least some other portions of the spatial audio signal.

[0008] The means may be further configured to obtain a defocus amount, and the means configured to process the spatial audio signal may be configured to at least partially control a relative de-emphasis of a part of the spatial audio signal in the defocus direction relative to other parts of at least a part of the spatial audio signal in accordance with the defocus amount.

[0009] The means configured to process the spatial audio signal may be configured to at least one of: reduce, for at least part of the spatial audio signal, an emphasis in parts of the spatial audio signal in a defocus direction relative to an emphasis in at least partly other parts of the spatial audio signal; and at least partly increase, an emphasis in other parts of the spatial audio signal relative to parts of the spatial audio signal in the defocus direction.

[0010] The means configured to process the spatial audio signal may be configured to at least one of: reduce a sound level in the portion of the spatial audio signal in accordance with an amount of defocus in at least a portion of the spatial audio signal relative to another portion; and increase a sound level in the portion of the spatial audio signal in accordance with the amount of defocus in at least a portion of the spatial audio signal relative to another portion.

[0011] The means may be further configured to obtain a defocus shape, and the means configured to process the spatial audio signal may be configured to control a relative de-emphasis in at least a portion of the spatial audio signal in the defocus direction and within the defocus shape relative to at least partially other portions of the spatial audio signal.

[0012] The means configured to process the spatial audio signal may be configured to at least one of: reduce emphasis in at least a portion of the spatial audio signal in portions from within a defocus direction and defocus shape of the spatial audio signal relative to other portions in at least a portion of the spatial audio signal; and increase emphasis in at least a portion of the other portions of the spatial audio signal relative to portions from within a defocus direction and defocus shape of the spatial audio signal.

[0013] The means configured to process the spatial audio signal may be configured to at least one of: reduce a sound level in at least a portion of the spatial audio signal, in a portion of the spatial audio signal in a defocus direction and from within the defocus shape, in accordance with a defocus amount relative to other portions of the at least a portion of the spatial audio signal; and increase a sound level in the other portions of the spatial audio signal, in a portion of the spatial audio signal, in a defocus direction and from the defocus shape according to the defocus amount.

[0014] The means may be configured to obtain playback control information for controlling at least one aspect of outputting the processed spatial audio signal, and the means configured to output the processed spatial audio signal may be configured to perform one of: processing the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate an output spatial audio signal in accordance with the playback control information; processing the spatial audio signal in accordance with the playback control information prior to the means configured to process the spatial audio signal representing the audio scene to generate a processed spatial audio signal representing the audio scene modified based on the defocus direction; and outputting the processed spatial audio signal as the output spatial audio signal.

[0015] The spatial audio signal and the processed spatial audio signal may comprise respective Ambisonic signals, and the means configured to process the spatial audio signal into the processed spatial audio signal may be configured to extract, for one or more frequency subbands, from the spatial audio signal a single-channel target audio signal representative of sound components arriving from a focus direction and generate a focused spatial audio signal, where the focused audio signal is located at a spatial location defined by the defocus direction, and to generate the processed spatial audio signal as a linear combination of the focused spatial audio signal subtracted from the spatial audio signal, and at least one of the focused spatial audio signal and the spatial audio signal is scaled by a respective scaling factor derived based on the amount of defocus to reduce the relative level of sound in the defocus direction.

[0016] The means configured to extract the single channel target audio signal may be configured to apply a beamformer to derive a beamformed signal representative of sound components arriving from a defocused direction from the spatial audio signal, and to apply a post filter to derive a processed audio signal based on the beamformed signal, thereby adjusting the spectrum of the beamformed signal to approximate the spectrum of sound arriving from the defocused direction.

[0017] The spatial audio signal and the processed spatial audio signal may include respective first order Ambisonic signals.

[0018] The spatial audio signal and the processed spatial audio signal may include a respective parametric spatial audio signal, the parametric spatial audio signal may include one or more audio channels and spatial metadata, the spatial metadata may include respective directional indications and energy ratio parameters for a plurality of frequency subbands, and the means configured to process the spatial audio signal to generate the processed spatial audio signal may include: calculating, for the one or more frequency subbands, respective angular differences between a defocus direction and an indicated direction for the respective frequency subbands of the spatial audio signal; deriving respective gain values ​​for the one or more frequency subbands based on the calculated angular differences for the respective frequency subbands by using a predefined function of the angular differences and a scaling factor derived based on the amount of defocus; and calculating, for the one or more frequency subbands of the processed spatial audio signal, respective updated ambient energy values ​​based on the energy ratio parameters of the respective frequency subbands of the spatial audio signal and the gain values ​​for the one or more frequency bands of the processed spatial audio signal. the spatial audio signal may be configured to: calculate a modified energy ratio parameter for each of the one or more frequency subbands of the processed spatial audio signal based on the updated directional energy divided by the sum of the updated direct and ambient energies; calculate a spectral adjustment factor for each of the one or more frequency subbands of the processed spatial audio signal based on the updated sum of the direct and ambient energies; and configure the processed spatial audio signal comprising one or more audio channels of the spatial audio signal, a directional indication of the spatial audio signal, the modified energy ratio parameter, and the spectral adjustment factor and to calculate the respective updated directional energy values. The spatial audio signal and the processed spatial audio signal may be configured to:The spatial audio signal may include a respective parametric spatial audio signal, the parametric spatial audio signal may include one or more audio channels and spatial metadata, the spatial metadata may include respective directional indications and energy ratio parameters for a plurality of frequency subbands, and the means configured to process the spatial audio signal to generate a processed spatial audio signal may include: calculating, for the one or more frequency subbands, respective angular differences between a defocus direction and an indicated direction for the respective frequency subbands of the spatial audio signal; deriving respective gain values ​​for the one or more frequency subbands based on the calculated angular differences for the respective frequency subbands by using a predefined function of the angular differences and a scaling factor derived based on the defocus amount; calculating, for the one or more frequency subbands of the processed spatial audio signal, respective updated directional energy values ​​based on the energy ratio parameters for the respective frequency subbands; calculating respective updated ambient energy values ​​for the processed spatial audio signal based on an energy ratio parameter and a scaling factor for each frequency subband of the spatial audio signal; calculating respective modified energy ratio parameters for one or more frequency subbands of the processed spatial audio signal based on the updated directional energy divided by a sum of the updated direct energy and ambient energy; calculating respective spectral adjustment coefficients for the one or more frequency subbands of the processed spatial audio signal based on the sum of the updated direct energy and ambient energy; deriving one or more extended audio channels by multiplying a respective one of the one or more audio channels of the spatial audio signal in the one or more frequency subbands by the derived spectral adjustment coefficients for the respective frequency subbands;and configuring the processed spatial audio signal.

[0019] The spatial audio signal and the processed spatial audio signal may include respective multi-channel loudspeaker signals according to a first predetermined loudspeaker configuration, and the means configured to process the spatial audio signal to generate the processed spatial audio signal may be configured to calculate respective angular differences between the defocus direction and the loudspeaker directions indicated for each channel of the spatial audio signal, derive respective gain values ​​for each channel of the spatial audio signal based on the calculated angular differences for the respective channels by using a predetermined function of the angular difference and a scaling factor derived based on the defocus amount, derive one or more modified audio channels by multiplying each channel of the spatial audio signal by the derived gain value for the respective channel, and provide the modified audio channels as the processed spatial audio signal. The predetermined function of the angular difference may result in gain values ​​that decrease as the angular difference value decreases and that increase as the angular difference value increases. The means configured to process the spatial audio signal may be configured to at least one of: reduce emphasis in at least a portion of the spatial audio signal in portions from within a defocus direction and defocus shape of the spatial audio signal relative to other portions in at least a portion of the spatial audio signal; and increase emphasis in at least a portion of the other portions of the spatial audio signal relative to portions from within a defocus direction and defocus shape of the spatial audio signal.

[0020] The means configured to process the spatial audio signal may be configured to at least one of: reduce a sound level in at least a portion of the spatial audio signal, in a portion of the spatial audio signal in a defocus direction and from within the defocus shape, in accordance with a defocus amount relative to other portions of the at least a portion of the spatial audio signal; and increase a sound level in the other portions of the spatial audio signal, in a portion of the spatial audio signal, in a defocus direction and from the defocus shape according to the defocus amount.

[0021] The means may be configured to obtain playback control information for controlling at least one aspect of outputting the processed spatial audio signal, and the means configured to output the processed spatial audio signal may be configured to perform one of: processing the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate an output spatial audio signal in accordance with the playback control information; processing the spatial audio signal in accordance with the playback control information prior to the means configured to process the spatial audio signal representing the audio scene to generate a processed spatial audio signal representing the audio scene modified based on the defocus direction; and outputting the processed spatial audio signal as the output spatial audio signal.

[0022] The spatial audio signal and the processed spatial audio signal may comprise respective Ambisonic signals, and the means configured to process the spatial audio signal into the processed spatial audio signal may be configured to extract, for one or more frequency subbands, from the spatial audio signal a single-channel target audio signal representative of sound components arriving from a focus direction and generate a focused spatial audio signal, where the focused audio signal is located at a spatial location defined by the defocus direction, and to generate the processed spatial audio signal as a linear combination of the focused spatial audio signal subtracted from the spatial audio signal, and at least one of the focused spatial audio signal and the spatial audio signal is scaled by a respective scaling factor derived based on the amount of defocus to reduce the relative level of sound in the defocus direction.

[0023] The means configured to extract the single channel target audio signal may be configured to apply a beamformer to derive a beamformed signal representative of sound components arriving from a defocused direction from the spatial audio signal, and to apply a post filter to derive a processed audio signal based on the beamformed signal, thereby adjusting the spectrum of the beamformed signal to approximate the spectrum of sound arriving from the defocused direction.

[0024] The spatial audio signal and the processed spatial audio signal may include respective first order Ambisonic signals.

[0025] The spatial audio signal and the processed spatial audio signal may include a respective parametric spatial audio signal, the parametric spatial audio signal may include one or more audio channels and spatial metadata, the spatial metadata may include respective directional indications and energy ratio parameters for a plurality of frequency subbands, and the means configured to process the spatial audio signal to generate the processed spatial audio signal may include: calculating, for the one or more frequency subbands, respective angular differences between a defocus direction and an indicated direction for the respective frequency subbands of the spatial audio signal; deriving respective gain values ​​for the one or more frequency subbands based on the calculated angular differences for the respective frequency subbands by using a predefined function of the angular differences and a scaling factor derived based on the amount of defocus; and calculating, for the one or more frequency subbands of the processed spatial audio signal, respective updated ambient energy values ​​based on the energy ratio parameters of the respective frequency subbands of the spatial audio signal and the gain values ​​for the one or more frequency bands of the processed spatial audio signal. the spatial audio signal may be configured to: calculate a modified energy ratio parameter for each of the one or more frequency subbands of the processed spatial audio signal based on the updated directional energy divided by the sum of the updated direct and ambient energies; calculate a spectral adjustment factor for each of the one or more frequency subbands of the processed spatial audio signal based on the updated direct and ambient energies; configure the processed spatial audio signal comprising one or more audio channels of the spatial audio signal, a directional indication of the spatial audio signal, the modified energy ratio parameter, and the spectral adjustment factor; and calculate each updated directional energy value. The spatial audio signal and the processed spatial audio signal may be configured to:The spatial audio signal may include a respective parametric spatial audio signal, the parametric spatial audio signal may include one or more audio channels and spatial metadata, the spatial metadata may include respective directional indications and energy ratio parameters for a plurality of frequency subbands, and the means configured to process the spatial audio signal to generate a processed spatial audio signal may include: calculating, for the one or more frequency subbands, respective angular differences between a defocus direction and an indicated direction for the respective frequency subbands of the spatial audio signal; deriving respective gain values ​​for the one or more frequency subbands based on the calculated angular differences for the respective frequency subbands by using a predefined function of the angular differences and a scaling factor derived based on the defocus amount; calculating, for the one or more frequency subbands of the processed spatial audio signal, respective updated directional energy values ​​based on the energy ratio parameters for the respective frequency subbands; calculating respective updated ambient energy values ​​for the processed spatial audio signal based on an energy ratio parameter and a scaling factor for each frequency subband of the spatial audio signal; calculating respective modified energy ratio parameters for one or more frequency subbands of the processed spatial audio signal based on the updated directional energy divided by a sum of the updated direct energy and ambient energy; calculating respective spectral adjustment coefficients for the one or more frequency subbands of the processed spatial audio signal based on the sum of the updated direct energy and ambient energy; deriving one or more extended audio channels by multiplying a respective one of the one or more audio channels of the spatial audio signal in the one or more frequency subbands by the derived spectral adjustment coefficients for the respective frequency subbands;and configuring the processed spatial audio signal.

[0026] The spatial audio signal and the processed spatial audio signal may include respective multi-channel loudspeaker signals according to a first predetermined loudspeaker configuration, and the means configured to process the spatial audio signal to generate the processed spatial audio signal may be configured to calculate respective angular differences between the defocus direction and the loudspeaker directions indicated for each channel of the spatial audio signal, derive respective gain values ​​for each channel of the spatial audio signal based on the calculated angular differences for the respective channels by using a predetermined function of the angular difference and a scaling factor derived based on the defocus amount, derive one or more modified audio channels by multiplying each channel of the spatial audio signal by the derived gain value for the respective channel, and provide the modified audio channels as the processed spatial audio signal. The predetermined function of the angular difference may result in gain values ​​that decrease as the angular difference value decreases and that increase as the angular difference value increases.

[0027] The processed spatial audio signal may comprise an Ambisonic signal, and the output spatial audio signal may comprise a two-channel binaural signal, and the playback control information may comprise an indication of a playback direction defining a listening direction with respect to the audio scene, and the means configured to process the processed spatial audio signal representing the modified audio scene based on a defocus direction to generate the output spatial audio signal in accordance with the playback control information may be configured to: generate a rotation matrix in response to the indicated playback direction, multiply the channels of the processed spatial audio signal with the rotation matrix to derive a rotated spatial audio signal, filter the channels of the rotated spatial audio signal using a predefined set of filter pairs generated based on a predefined dataset of finite impulse response (FIR), head related transfer function (HRTF), or head related impulse response (HRIR), and generate the left and right channels of the binaural signal as a sum of the filtered channels of the rotated spatial audio signal derived for the left and right channels, respectively.

[0028] The output spatial audio signal may comprise a two-channel binaural audio signal, and the playback control information may comprise an indication of a playback direction defining a listening direction relative to the audio scene, and the means configured to process the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate the output spatial audio signal in accordance with the playback control information may be configured to derive one or more enhanced audio channels by multiplying, in said one or more frequency sub-bands, a respective frequency band of a respective one of the one or more audio channels of the processed spatial audio signal by a spectral adjustment factor received for the respective frequency sub-band, and to convert the one or more enhanced audio channels into a two-channel binaural audio signal in accordance with the indicated playback direction.

[0029] The output spatial audio signal may comprise a two-channel binaural audio signal, the playback control information may comprise an indication of a playback direction defining a listening direction relative to the audio scene, and the means configured to process the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate the output spatial audio signal in accordance with the playback control information may be configured to convert one or more enhanced audio channels into a two-channel binaural audio signal in accordance with the indicated playback direction.

[0030] The output spatial audio signal may comprise a two-channel binaural signal, the playback control information may comprise an indication of a playback direction defining a listening direction with respect to the audio scene, and the means configured to process the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate the output spatial audio signal in accordance with the playback control information may be configured to select a set of head related transfer functions (HRTFs) in response to the indicated playback direction and to convert channels of the processed spatial audio signal into a two-channel binaural signal carrying the audio scene rotated using the selected set of HRTFs.

[0031] The playback control information may include an indication of a second predetermined loudspeaker configuration and the output spatial audio signal may include a multi-channel loudspeaker signal with the second predetermined loudspeaker configuration, and the means configured to process the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate the output spatial audio signal in accordance with the playback control information may be configured to derive a transformation matrix including amplitude panning gains that provide a mapping from the first predetermined loudspeaker configuration to the second predetermined loudspeaker configuration, thereby deriving the channels of the output spatial audio signal based on the channels of the processed spatial audio signal using amplitude panning, and multiplying the channels of the output spatial audio signal with the channels of the output spatial audio signal using the transformation matrix.

[0032] The means may be further configured to obtain a defocus input from a sensor arrangement including at least one directional sensor and at least one user input, and the defocus input may include an indication of the defocus direction based on the at least one directional sensor direction.

[0033] The defocus input may further include an indicator of the amount of defocus.

[0034] The defocus input may further include an indicator of the defocus shape.

[0035] The defocus shape can include at least one of a defocus shape width, a defocus shape height, a defocus shape radius, a defocus shape distance, a defocus shape depth, a defocus shape range, a defocus shape diameter, and a defocus shape characterizer.

[0036] The defocus direction may be an arc defined by a range of defocus directions.

[0037] According to a second aspect, a method is provided comprising the steps of obtaining a defocus direction, processing a spatial audio signal representing an audio scene to generate a processed spatial audio signal representing an audio scene modified based on the defocus direction so as to control, at least in part, a relative deemphasis of the defocus direction relative to other parts of the spatial audio signal, and outputting the processed spatial audio signal, wherein the modified audio scene based on the defocus direction enables the deemphasis, at least in part, of the parts of the spatial audio signal in the defocus direction, at least in part, in other parts of the spatial audio signal.

[0038] The method may further include obtaining a defocus amount, and processing the spatial audio signal may include controlling de-emphasis of a portion of the spatial audio signal in a defocus direction relative to at least partially other portions of the spatial audio signal according to the defocus amount.

[0039] Processing the spatial audio signal may include at least one of: reducing emphasis of portions of the spatial audio signal, at least in part in a defocus direction relative to other portions of the spatial audio signal; and increasing emphasis of other portions of the spatial audio signal, at least in part, relative to portions of the spatial audio signal in the defocus direction.

[0040] Processing the spatial audio signal may include at least one of: reducing a sound level in portions of the spatial audio signal in a defocus direction, at least in part depending on the amount of defocus relative to other portions of the spatial audio signal; and increasing a sound level in other portions of the spatial audio signal, at least in part depending on the amount of defocus, relative to the portions of the spatial audio signal in the defocus direction.

[0041] The method may further include obtaining a defocus shape, and processing the spatial audio signal may include, at least in part, controlling a relative de-emphasis within a defocus direction and defocus shape of the spatial audio signal relative to at least a portion of other portions of the spatial audio signal.

[0042] Processing the spatial audio signal may include at least one of reducing emphasis for portions of the spatial audio signal in the defocus direction relative to other portions of the spatial audio signal, at least partially from within the defocus shape, and increasing emphasis for portions of the spatial audio signal in the defocus direction and within the defocus shape, at least partially to other portions of the spatial audio signal.

[0043] Processing the spatial audio signal may include at least one of: decreasing a sound level within a portion of the spatial audio signal in a defocus direction from within the defocus shape, at least in part depending on the amount of defocus relative to at least some other portions of the spatial audio signal; and increasing a sound level within other portions of the spatial audio signal from the defocus shape, at least in part, for the portion of the spatial audio signal in the defocus direction and in accordance with the amount of defocus.

[0044] The method includes a step of obtaining playback control information for controlling at least one aspect of outputting a processed spatial audio signal, and the step of outputting the processed spatial audio signal may include one of the steps of: processing the processed spatial audio signal representing the modified audio scene based on the defocus direction in accordance with the playback control information to generate an output spatial audio signal; processing the spatial audio signal in accordance with the playback control information before processing the spatial audio signal representing an audio scene to generate the processed spatial audio signal representing the modified audio scene based on the defocus direction; and outputting the processed spatial audio signal as an output spatial audio signal.

[0045] The spatial audio signal and the processed spatial audio signal may include respective Ambisonic signals, and processing the spatial audio signal into the processed spatial audio signal may include extracting, for one or more frequency subbands, from the spatial audio signal a single channel target audio signal representative of sound components arriving from a focus direction, generating a focused spatial audio signal where the focused audio signal is located at a spatial location defined by the defocus direction, and generating the processed spatial audio signal as a linear combination of the focused spatial audio signal subtracted from the spatial audio signal. At least one of the focused spatial audio signal and the spatial audio signal is scaled by a respective scaling factor derived based on the amount of defocus to reduce a relative level of sound in the defocus direction.

[0046] The step of extracting the single channel target audio signal may include applying a beamformer to derive a beamformed signal from the spatial audio signal, the beamformed signal representing sound components arriving from a defocused direction, and applying a post-filter to derive a processed audio signal based on the beamformed signal, thereby adjusting the spectrum of the beamformed signal to approximate the spectrum of sound arriving from the defocused direction.

[0047] The spatial audio signal and the processed spatial audio signal may include respective first order Ambisonic signals.

[0048] The spatial audio signal and the processed spatial audio signal may include a respective parametric spatial audio signal, the parametric spatial audio signal may include one or more audio channels and spatial metadata, the spatial metadata may include respective directional indications and energy ratio parameters for a plurality of frequency subbands. Processing the spatial audio signal to generate the processed spatial audio signal may include: calculating, for the one or more frequency subbands, respective angular differences between a defocus direction and an indicated direction for the respective frequency subbands of the spatial audio signal; deriving respective gain values ​​for the one or more frequency subbands based on the calculated angular differences for the respective frequency subbands by using a scaling factor derived based on a predefined function of the angular differences and the defocus amount; calculating, for the one or more frequency subbands of the processed spatial audio signal, respective updated directional energy values ​​based on the energy ratio parameters and the gain values ​​for the respective frequency subbands of the spatial audio signal; calculating updated ambient energy values ​​based on the respective frequency sub-band energy ratio parameters and a scaling factor of the spatial audio signal; calculating respective modified energy ratio parameters for one or more frequency sub-bands of the processed spatial audio signal based on the updated directional energy divided by a sum of the updated directional and ambient energies; calculating respective spectral adjustment factors for the one or more frequency sub-bands of the processed spatial audio signal based on the sum of the updated directional energy and ambient energies; and configuring the processed spatial audio signal comprising the one or more audio channels of the spatial audio signal, a directional indication of the spatial audio signal, the modified energy ratio parameters, and the spectral adjustment factors.The spatial audio signal and the processed spatial audio signal may include respective parametric spatial audio signals, where the parametric spatial audio signals may include one or more audio channels and spatial metadata, where the spatial metadata may include respective directional indications and energy ratio parameters for a plurality of frequency sub-bands.Processing the spatial audio signal to generate a processed spatial audio signal includes: calculating, for one or more frequency subbands, respective angular differences between a defocus direction and a direction indicated for each frequency subband of the spatial audio signal; deriving respective gain values ​​for the one or more frequency subbands based on the angular differences calculated for each frequency subband by using a predefined function of the angular differences and a scaling factor derived based on the amount of defocus; calculating, for one or more frequency subbands of the processed spatial audio signal, respective updated directional energy values ​​based on an energy ratio parameter and a gain value of the respective frequency subband of the spatial audio signal; and calculating, for one or more frequency bands of the processed spatial audio signal, respective updated ambient energy values ​​based on the energy ratio parameter and the scaling factor of the respective frequency subband of the spatial audio signal; calculating respective modified energy ratio parameters for one or more frequency subbands of the processed spatial audio signal based on the updated direct energy divided by a sum of the updated direct and ambient energies; calculating respective spectral adjustment factors for one or more of the processed spatial audio signal based on the sum of the updated direct and ambient energies; deriving one or more enhanced audio channels by multiplying respective frequency bands of one or more audio channels of the spatial audio signal in the one or more frequency subbands by the derived spectral adjustment factors for the respective frequency subbands; and providing the processed spatial audio signal including the one or more enhanced audio channels, a directional indication of the spatial audio signal, and the modified energy ratio parameter.

[0049] The spatial audio signal and the processed spatial audio signal may include respective multi-channel loudspeaker signals according to a first predetermined loudspeaker configuration, and processing the spatial audio signal to generate the processed spatial audio signal may include calculating respective angular differences between a defocus direction and a loudspeaker direction indicated for each channel of the spatial audio signal, deriving a respective gain value for each channel of the spatial audio signal based on the calculated angular difference for the respective channel by using a scaling factor derived based on a predetermined function of the angular difference and an amount of defocus, deriving one or more modified audio channels by multiplying each channel of the spatial audio signal by the derived gain value for the respective channel, and providing the modified audio channels as the processed spatial audio signal.

[0050] The predetermined function of the angular difference may result in a gain value that decreases as the angular difference value decreases and increases as the angular difference value increases.

[0051] The processed spatial audio signal may comprise an Ambisonic signal and the output spatial audio signal may comprise a two-channel binaural signal, where the playback control information may comprise an indication of a playback direction defining a listening direction with respect to the audio scene, and generating the processed spatial audio signal representing the modified spatial audio signal according to the playback control information may comprise: generating a rotation matrix according to the indicated playback direction, multiplying the channels of the processed spatial audio signal by the rotation matrix to derive the rotated spatial audio signal, filtering the channels of the rotated spatial audio signal with a filter pair generated based on a predefined set of finite impulse responses (FIRs), head related impulse response functions (HRTFs), or head related impulse response (HRIR) data sets, and generating left and right channels of the binaural signal as a sum of the filtered channels of the rotated spatial audio signal derived for the left and right channels, respectively.

[0052] The output spatial audio signal may comprise a two-channel binaural audio signal, and the playback control information may comprise an indication of a playback direction defining a listening direction with respect to the audio scene, and processing the processed spatial audio signal representing the audio scene based on / modified on the defocus direction to generate the output spatial audio signal in accordance with the playback control information may comprise deriving one or more enhanced audio channels by multiplying, in the one or more frequency sub-bands, respective frequency bands of the one or more audio channels of the processed spatial audio signal by the spectral adjustment coefficients received for the respective frequency sub-bands, and converting the one or more enhanced audio channels into a two-channel binaural audio signal in accordance with the indicated playback direction.

[0053] The output spatial audio signal may comprise a two-channel binaural audio signal, and the playback control information defining a listening direction relative to the audio scene may comprise an indication of a playback direction, and processing the processed spatial audio signal representing the modified audio scene based on the defocus direction to generate the output spatial audio signal in accordance with the playback control information may comprise converting one or more enhanced audio channels into a two-channel binaural audio signal in accordance with the indicated playback direction.

[0054] The output spatial audio signal may comprise a two-channel binaural signal, where the playback control information may comprise an indication of a playback direction defining a listening direction with respect to the audio scene, and processing the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate the output spatial audio signal in accordance with the playback control information may comprise selecting a set of head-related transfer functions HRTFs in dependence on the indicated playback direction, and converting the channels of the processed spatial audio signal into a two-channel binaural signal conveying the rotated audio scene using the selected set of HRTFs.

[0055] The playback control information may include an indication of a second predefined loudspeaker configuration, and the output spatial audio signals may include multi-channel loudspeaker signals according to the second predefined loudspeaker configuration, and processing the processed spatial audio signals representing the audio scene modified based on the defocus direction to generate the processed spatial audio signals in accordance with the playback control information may include deriving channels of the output spatial audio signal based on the channels of the processed spatial audio signal using amplitude panning by being configured to derive a transformation matrix including amplitude panning gains that provide a mapping from the first predefined loudspeaker configuration to the second predefined loudspeaker configuration, and may include using the transformation matrix to multiply the channels of the output spatial audio signal with the channels of the processed spatial audio signal.

[0056] The method may further include obtaining a defocus input from a sensor arrangement including at least one directional sensor and at least one user input, the defocus input including an indication of a defocus direction based on the at least one directional sensor direction.

[0057] The defocus input may further include an indicator of the amount of defocus.

[0058] The defocus input may further include an indicator of the defocus shape.

[0059] The defocus shape can include at least one of a defocus shape width, a defocus shape height, a defocus shape radius, a defocus shape distance, a defocus shape depth, a defocus shape range, a defocus shape diameter, and a defocus shape characterizer.

[0060] The defocus direction may be an arc defined by a range of defocus directions.

[0061] According to a third aspect, there is provided an apparatus comprising at least one processor and at least one memory including computer program code configured, using the at least one processor, to cause the apparatus to at least obtain a defocus direction, process a spatial audio signal representing an audio scene to generate a spatial audio signal representing an audio scene based on the defocus direction so as to control a relative de-emphasis in a portion of the spatial audio signal in the defocus direction in at least some other portions of the spatial audio signal, and output the processed spatial audio signal, where the modified audio scene based on the defocus direction enables de-emphasis in at least a portion of the spatial audio signal in a defocus direction relative to at least some other portions of the spatial audio signal.

[0062] The apparatus may further be adapted to obtain a defocus amount, and the apparatus for processing the spatial audio signal may be adapted to control relative de-emphasis in one part of the spatial audio signal in a defocus direction relative to another part of the spatial audio signal, at least in part, in accordance with the defocus amount.

[0063] An apparatus adapted to process a spatial audio signal may at least perform one of: at least partially reducing emphasis of portions of the spatial audio signal in a defocus direction relative to at least partially other portions of the spatial audio signal; and at least partially increasing emphasis of portions of the spatial audio signal in a defocus direction relative to other portions of the spatial audio signal.

[0064] The device for processing a spatial audio signal may perform at least one of: reducing a sound level in at least a portion of the spatial audio signal in a defocus direction relative to the other portion of the spatial audio signal, at least in part according to the amount of defocus; and increasing a sound level in the other portion of the spatial audio signal relative to the portion of the spatial audio signal in the defocus direction, at least in part according to the amount of defocus.

[0065] The apparatus may further be adapted to obtain a defocus shape, and the apparatus for processing the spatial audio signal may be adapted to control relative de-emphasis within the defocus shape, at least in part, for a portion of the spatial audio signal in the defocus direction and at least in part for other portions of the spatial audio signal.

[0066] The device for processing the spatial audio signal may perform at least one of: reducing emphasis at least partially in portions of the spatial audio signal in the defocus direction and at least partially from within the defocus shape relative to other portions of the spatial audio signal; and increasing emphasis at least partially in portions of the spatial audio signal in the defocus direction and within the defocus shape relative to other portions of the spatial audio signal.

[0067] An apparatus for processing a spatial audio signal may at least one of: reducing sound levels from within the defocus shape in portions of the spatial audio signal in a defocus direction at least in part according to the amount of defocus relative to at least partial other portions of the spatial audio signal; and increasing sound levels from the defocus shape in other portions of the spatial audio signal at least in part according to the amount of defocus relative to the portion of the spatial audio signal in the defocus direction.

[0068] The device can obtain playback control information for controlling at least one aspect of outputting the processed spatial audio signal, and the device causing the output of the processed spatial audio signal can perform one of: processing the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate an output spatial audio signal in accordance with the playback control information; processing the processed spatial audio signal representing the audio scene in accordance with the playback control information prior to processing of the spatial audio signal representing the audio scene to generate a processed spatial audio signal representing the audio scene modified based on the defocus direction; and outputting the processed spatial audio signal as an output spatial audio signal.

[0069] The spatial audio signal and the processed spatial audio signal may include respective Ambisonic signals, and the apparatus for processing the spatial audio signal to a processed spatial audio signal may extract, for one or more frequency subbands, from the spatial audio signal a single-channel target audio signal representative of sound components arriving from a focus direction, and generate a focused spatial audio signal, where the focused audio signal is located at a spatial location defined by the defocus direction, and generate the processed spatial audio signal as a linear combination of the focused spatial audio signal subtracted from the spatial audio signal, and at least one of the focused spatial audio signal and the spatial audio signal is scaled by a respective scaling factor derived based on the amount of defocus to reduce a relative level of sound in the defocus direction.

[0070] The apparatus for extracting a single-channel target audio signal applies a beamformer to derive, from the spatial audio signal, a beamformed signal representative of sound components arriving from a defocused direction, and applies a post-filter to derive a processed audio signal based on the beamformed signal, thereby adjusting the spectrum of the beamformed signal to approximate the spectrum of sounds arriving from a defocused direction.

[0071] The spatial audio signal and the processed spatial audio signal may include respective first order Ambisonic signals.

[0072] The spatial audio signal and the processed spatial audio signal may include respective parametric spatial audio signals, the parametric spatial audio signals may include one or more audio channels and spatial metadata, the spatial metadata may include respective directional indications and energy ratio parameters for a plurality of frequency subbands. An apparatus for processing a spatial audio signal to generate a processed spatial audio signal may include: calculating, for one or more frequency subbands, respective angular differences between a defocus direction and an indicated direction for each frequency subband of the spatial audio signal; deriving respective gain values ​​for the one or more frequency subbands based on the calculated angular differences for each frequency subband by using a scaling factor derived based on a predetermined function of the angular differences and the defocus amount; calculating, for one or more frequency subbands of the processed spatial audio signal, respective updated directional energy values ​​based on the energy ratio parameters, and gain values ​​for each frequency subband of the spatial audio signal; to calculate respective updated ambient energy values ​​based on the energy ratio parameter and the scaling factor for each frequency subband of the spatial audio signal; to calculate modified energy ratio parameters for each of the one or more frequency subbands of the processed spatial audio signal based on the updated directional energy divided by the sum of the updated direct and ambient energies; and to calculate spectral adjustment coefficients for each of the one or more frequency subbands of the processed spatial audio signal based on the sum of the updated direct and ambient energies; and to configure the processed spatial audio signal comprising one or more audio channels of the spatial audio signal, a directional indication of the spatial audio signal, the modified energy ratio parameter, and the spectral adjustment coefficients.The spatial audio signal and the processed spatial audio signal may include a respective parametric spatial audio signal, the parametric spatial audio signal may include one or more audio channels and spatial metadata, the spatial metadata may include a respective directional indication and an energy ratio parameter for a plurality of frequency subbands. The apparatus for processing the spatial audio signal to generate the processed spatial audio signal may calculate, for one or more frequency subbands, a respective angular difference between a defocus direction and a direction indicated for each frequency subband of the spatial audio signal, derive a respective gain value for the one or more frequency subbands based on the calculated angular difference for each frequency subband by using a scaling factor derived based on a predetermined function of the angular difference and the defocus amount, and calculate, for one or more frequency subbands of the processed spatial audio signal, a respective updated directional energy value based on the energy ratio parameter and the gain value for the respective frequency subband of the spatial audio signal.calculating, for one or more frequency bands of the processed spatial audio signal and gain values, respective updated ambient energy values ​​based on the energy ratio parameters and scaling factors for the respective frequency subbands of the spatial audio signal; calculating respective modified energy ratio parameters for the one or more frequency subbands of the processed spatial audio signal based on the updated directional energy values ​​divided by the sum of the updated direct and ambient energies; calculating respective spectral adjustment coefficients for the one or more frequency subbands of the processed spatial audio signal based on the sum of the updated direct and ambient energies; deriving one or more extended audio channels by multiplying, in the one or more frequency subbands, a respective frequency band of a plurality of respective ones of the audio channels of the spatial audio signal by the spectral adjustment coefficient derived for the respective frequency subband; and constituting a processed spatial audio signal comprising the one or more extended audio channels, a directional indication of the spatial audio signal, and the modified energy ratio parameters.

[0073] The spatial audio signal and the processed spatial audio signal may include respective multi-channel loudspeaker signals according to a first predetermined loudspeaker configuration, and the apparatus for processing the processed spatial audio signal to generate the processed spatial audio signal may calculate respective angular differences between a defocus direction and a loudspeaker direction indicated for each channel of the spatial audio signal, derive respective gain values ​​for each channel of the spatial audio signal based on the calculated angular differences for the respective channels by using a predetermined function of the angular differences and a scaling factor derived based on the amount of defocus, derive one or more modified audio channels by multiplying each channel of the spatial audio signal by the derived gain value for the respective channel, and provide the modified audio channels as the processed spatial audio signal.

[0074] The predetermined function of the angular difference may result in a gain value that decreases as the angular difference value decreases and increases as the angular difference value increases.

[0075] The processed spatial audio signal may comprise an Ambisonic signal and the output spatial audio signal may comprise a two-channel binaural signal, where the playback control information may comprise an indication of a playback direction defining a listening direction with respect to the audio scene, and an apparatus adapted to process the processed spatial audio signal representing the modified audio scene based on the defocus direction to generate an output spatial audio signal according to the playback control information may generate a rotation matrix in dependence on the indicated playback direction, multiply the channels of the processed spatial audio signal with the rotation matrix to derive a rotated spatial audio signal, filter the channels of the rotated spatial audio signal with a filter pair generated based on a predetermined set of finite impulse responses, FIR, head-on impulse response functions, HRTFs, or head-on impulse responses, HRIRs, and generate left and right channels of the binaural signal as a sum of the filtered channels of the rotated spatial audio signal for each of the left and right channels.

[0076] The output spatial audio signal may comprise a two-channel binaural audio signal, and the playback control information may comprise an indication of a playback direction defining a listening direction relative to the audio scene, and the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate the output spatial audio signal in accordance with the playback control information; and the processing device may derive one or more enhanced audio channels in the one or more frequency sub-bands by multiplying respective frequency bands of one or more audio channels of the processed spatial audio signal by the received spectral adjustment coefficients for the respective frequency sub-bands, and convert the one or more enhanced audio channels into a two-channel binaural audio signal in accordance with the indicated playback direction.

[0077] The output spatial audio signal may consist of a two-channel binaural audio signal and the playback control information may consist of an indication of a playback direction defining a listening direction for the audio scene. The device for processing the processed spatial audio signal representative of the modified audio scene based on the defocus direction to generate an output spatial audio signal in accordance with the playback control information may be adapted to convert the one or more extended audio channels into the two-channel binaural audio signal in accordance with the indicated playback direction.

[0078] The output spatial audio signal may comprise a two-channel binaural signal, where the playback control information may comprise an indication of a playback direction defining a listening direction with respect to the audio scene, and a device adapted to process the processed spatial audio signal representing the modified audio scene based on the defocus direction may select a set of Head Related Transfer Functions (HRTFs) dependent on a specified playback direction to generate an output spatial audio signal responsive to the playback control information, and may convert the channels of the processed spatial audio signal into a two-channel binaural signal conveying a rotated audio scene using the selected set of HRTFs.

[0079] The playback control information may include an indication of a second predetermined loudspeaker configuration and the output spatial audio signal may include a multi-channel loudspeaker signal with the second predetermined loudspeaker configuration, and the means adapted to process the processed spatial audio signal representing the audio scene modified based on the defocus direction to generate the output spatial audio signal in accordance with the playback control information may be configured to derive a transformation matrix including amplitude panning gains providing a mapping from the first loudspeaker configuration to the second loudspeaker configuration, thereby deriving channels of the output spatial audio signal based on the channels of the processed spatial audio signal using amplitude panning, and multiplexing the channels of the processed spatial audio signal into the channels of the output spatial audio signal using the transformation matrix.

[0080] The apparatus may be adapted to obtain a defocus input from a sensor arrangement including at least one directional sensor and at least one user input, where the defocus input may include an indication of a defocus direction based on an orientation of the at least one directional sensor.

[0081] The defocus input may further include an indicator of the amount of defocus.

[0082] The defocus input may further include an indicator of the defocus shape.

[0083] The defocus shape can include at least one of a defocus shape width, a defocus shape height, a defocus shape radius, a defocus shape distance, a defocus shape depth, a defocus shape range, a defocus shape diameter, and a defocus shape characterizer.

[0084] The defocus direction may be an arc defined by a range of defocus directions.

[0085] According to a fourth aspect, there is provided an apparatus including a circuit configured to obtain a defocus direction, the spatial audio signal processing circuit configured to process an audio scene based on the defocus direction to generate a processed spatial audio signal representing a modified audio scene based on the defocus direction to control at least partially relative de-emphasis of a partial spatial audio signal in the defocus direction relative to at least some other parts of the defocus direction, an output circuit configured to control an output of the processed spatial audio signal, and an output circuit configured to control an output of the processed spatial audio signal, wherein the audio scene modified based on the defocus direction enables de-emphasis of a portion of the spatial audio signal in the defocus direction relative to at least some other parts of the spatial audio signal in at least some parts of the defocus direction.

[0086] According to a fifth aspect, there is provided a computer program comprising instructions [or a computer readable medium comprising program instructions] for causing an apparatus to at least obtain a defocus direction; process the spatial audio signal representing an audio scene to generate a processed spatial audio signal representing an audio scene modified based on the defocus direction to control a relative de-emphasis of the defocus direction in at least some portions of the spatial audio signal; and output the processed spatial audio signal, wherein the audio scene modified based on the defocus direction enables de-emphasis of portions of the defocus direction relative to at least some other portions of the spatial audio signal, in at least some portions of the spatial audio signal.

[0087] According to a sixth aspect, there is provided a non-transitory computer-readable medium comprising program instructions to cause an apparatus to at least obtain a defocus direction; process a spatial audio signal representing an audio scene to generate a processed spatial audio signal representing an audio scene modified based on a defocus direction to control a relative de-emphasis of the defocus direction in at least some portions of the spatial audio signal; and output the processed spatial audio signal, wherein the audio scene modified based on the defocus direction enables de-emphasis of portions of the defocus direction relative to at least some other portions of the spatial audio signal in at least some portions of the spatial audio signal.

[0088] According to a seventh aspect there is provided an apparatus comprising: means for obtaining a defocus direction; means for processing a spatial audio signal representing an audio scene to generate a processed spatial audio signal representing a modified audio scene based on the defocus direction so as to control de-emphasis of a part of the spatial audio signal in at least a part of the spatial audio signal relative to the defocus direction with respect to another part of the spatial audio signal; and means for outputting the processed spatial audio signal, wherein the modified audio scene based on the defocus direction enables de-emphasis of a part of the spatial audio signal in at least a part of the spatial audio signal relative to the defocus direction with respect to at least a part of the other part of the spatial audio signal.

[0089] According to an eighth aspect, there is provided a computer-readable medium comprising program instructions for causing an apparatus to at least: obtain a defocus direction; and process a spatial audio signal representing the audio scene to generate a processed spatial audio signal representing the audio scene modified based on the defocus direction to control relative de-emphasis of portions of the spatial audio signal in the defocus direction relative to at least some other portions of the spatial audio signal.

[0090] An apparatus comprising means for performing the operations set forth above.

[0091] An apparatus configured to perform the operations of the methods described above.

[0092] A computer program comprising program instructions for causing a computer to carry out the method described above.

[0093] A computer program product stored on the medium can cause an apparatus to perform the methods described herein.

[0094] The electronic device may comprise an apparatus as described herein.

[0095] The chipset may comprise an apparatus as described herein.

[0096] SUMMARY OF THE PRESENT APPLICATION The embodiments of the present application aim to address problems associated with the state of the art. [Brief description of the drawings]

[0097] For a better understanding of the present application, reference is made, by way of example, to the accompanying drawings, in which: [Figure 1a] 1a, 1b and 1c show exemplary sound scenes illustrating audio focus regions or areas. [Figure 1b] 1a, 1b and 1c show exemplary sound scenes illustrating audio focus regions or areas. [Figure 1c] 1a, 1b and 1c show exemplary sound scenes illustrating audio focus regions or areas. [Figure 2a] 2a and 2b illustrate generally an exemplary playback device and method for operating the playback device, according to some embodiments. [Figure 2b] 2a and 2b illustrate generally an exemplary playback device and method for operating the playback device, according to some embodiments. [Figure 3a] 3a and 3b illustrate generally the example focus processor shown in FIG. 2a having a higher order Ambisonic audio signal input and a method of operating the example focus processor according to some embodiments. [Figure 3b] 3a and 3b illustrate generally the example focus processor shown in FIG. 2a having a higher order Ambisonic audio signal input and a method of operating the example focus processor according to some embodiments. [Figure 4a] 4a and 4b illustrate generally the exemplary focus processor shown in FIG. 2a having a parametric spatial audio signal input and a method of operating the exemplary focus processor according to some embodiments. [Figure 4b] 4a and 4b illustrate generally the exemplary focus processor shown in FIG. 2a having a parametric spatial audio signal input and a method of operating the exemplary focus processor according to some embodiments. [Figure 5a] 5a and 5b illustrate generally the exemplary focus processor shown in FIG. 2a having a multi-channel and / or audio object signal input and a method of operating the exemplary focus processor according to some embodiments. [Figure 5b]5a and 5b illustrate generally the exemplary focus processor shown in FIG. 2a having a multi-channel and / or audio object signal input and a method of operating the exemplary focus processor according to some embodiments. [Figure 6a] 6a and 6b illustrate generally an exemplary playback processor as shown in FIG. 2a having a higher order Ambisonic audio signal input and a method of operating the exemplary playback processor according to some embodiments. [Figure 6b] 6a and 6b illustrate generally an exemplary playback processor as shown in FIG. 2a having a higher order Ambisonic audio signal input and a method of operating the exemplary playback processor according to some embodiments. [Figure 7a] 7a and 7b illustrate generally an exemplary playback processor as shown in FIG. 2a having a parametric spatial audio signal input and a method of operating the exemplary playback processor according to some embodiments. [Figure 7b] 7a and 7b illustrate generally an exemplary playback processor as shown in FIG. 2a having a parametric spatial audio signal input and a method of operating the exemplary playback processor according to some embodiments. [Figure 8] FIG. 8 illustrates an example implementation of some embodiments. [Figure 9] FIG. 9 illustrates an exemplary controller for controlling the focus direction, focus amount, and focus width according to an embodiment. [Figure 10] FIG. 10 illustrates an example of a processing output based on processing of a high-order Ambisonic audio signal according to some embodiments. [Figure 11] FIG. 11 illustrates an exemplary apparatus suitable for implementing the illustrated apparatus. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0098] In the following, suitable apparatus and possible mechanisms for providing efficient rendering and playback of spatial audio signals are described in further detail.

[0099] In the previous example of spatial audio signal reproduction, the user can control the focus direction and the focus amount. However, in some situations, such focus direction / amount control may not be sufficient. The concept described below is an apparatus and method featuring further focus control that can indicate the elimination or de-emphasis of sounds in a particular direction. For example, in a sound field, there may be several different features, such as multiple dominant sound sources in a particular direction, as well as ambient sounds. Some users may prefer to remove certain features of the sound field, while some users may prefer to hear the complete audio scene, or to remove alternative features of the sound field. In particular, a user may wish to remove undesired sounds so that the rest of the spatial sound scene is reproduced as originally intended.

[0100] Figures 1a to 1c, described below, show what a user would perceive when listening to a reproduced spatial audio signal.

[0101] As an example, Fig. 1a shows a user 101 with a defined direction. Within the audio scene there is a source of interest 105, e.g. a speaker. Additionally there may be other ambient audio content 107 surrounding the user.

[0102] Additionally, the user may identify interfering sound sources such as an air conditioner 103. Traditionally, the user may control the playback to focus on sources of interest 105 and emphasize these over the interfering sources 103. However, the concept described in the embodiment instead attempts to improve sound quality by performing a "removal" (or defocusing or negative focus) of the identified sources, as shown in Figure 1a by defocusing or negative focus identified sources 103.

[0103] As another example, as shown in FIG. 1b, a user may wish to defocus or negatively focus any source within a shape or region within a sound scene. Thus, for example, FIG. 1b shows a user 101 positioned with a defined orientation within an audio or sound scene having a source of interest 105, such as a speaker, other ambient audio content 107, such as environmental audio content, and an interference source 155 within a defined region 153. In this example, the region of defocus or negative focus is represented by a defocus arc 151 of a defined width and direction relative to the user 101. The defocus arc 151 of a defined width and direction relative to the user 101 covers the interference source 155 within the interference source region 153.

[0104] A further way in which regions of defocus or negative focus may be represented is shown in Figure 1c, where a defocus region or volume (for a 3D region) 161 covers an interferer 155 within an interferer region 153. In this example, the defocus region may be defined by distance as well as direction and "width."

[0105] Thus, the embodiments described herein attempt to provide control of the defocus shape (in addition to the defocus direction and amount). The concept as described with respect to the embodiments described herein relates to spatial audio reproduction, enabling audio playback with control means for reducing / removing audio elements (or regions or volumes) originating from selectable spatial directions (or regions or volumes) by a desired amount (e.g., 0%-100%) to elements (or regions or volumes) outside these determined defocus shapes, so as to de-emphasize the audibility of audio elements in selected spatial directions (or regions or volumes) while maintaining the audibility of desired audio elements in the selected spatial directions (or regions or volumes), while allowing the spatial audio signal format to be the same.

[0106] This embodiment provides at least one defocus (or negative focus) parameter corresponding to a selectable direction and amount. Furthermore, in some embodiments, this defocus (or negative focus) parameter can define a defocus (or negative focus) shape, which can be defined by any (or a combination of two or more) of the following parameters corresponding to direction, width, height, radius, distance, and depth. This parameter set in some embodiments includes parameters that define any defocus shape.

[0107] In some embodiments, at least one defocus parameter is provided to emphasize the audibility of a further selected spatial direction (or shape, area, or volume).

[0108] Spatial audio signal processing may be performed in some embodiments by obtaining a spatial audio signal associated with media having multiple viewing directions, obtaining focus / defocus direction and amount parameters (which may optionally include obtaining at least one focus / defocus shape information), modifying the spatial audio signal to have desired (focus) and defocus characteristics, and playing (over headphones or loudspeakers) the modified spatial audio signal.

[0109] The resulting spatial audio signals may be in a parametric spatial audio format, such as, for example, an Ambisonic signal, a loudspeaker signal, an audio channel configuration and associated spatial metadata.

[0110] Focus / defocus information can be defined as follows: focus refers to increasing the relative prominence of audio coming from a selectable direction (or shape or region), while defocus refers to decreasing the relative prominence of audio coming from that direction (or shape or region).

[0111] The focus / defocus amount determines how much to focus or defocus, which can be, for example, 0% to 100%, where 0% means to keep the original sound scene unmodified and 100% means to maximally focus / defocus in the desired orientation or within a specified range.

[0112] The focus / defocus control in some embodiments may be a switch control to determine whether to focus or defocus, or may be controlled in other ways, for example, by extending the focus amount range from -100% to 100%, where negative values ​​indicate a defocus (or negative focus) effect and positive values ​​indicate a focus effect.

[0113] It should be noted that different users may wish to have different focus / defocus characteristics: the original spatial audio signal may be modified and played back individually for each user based on the user's personal preferences.

[0114] 2a shows a block diagram of some components and / or entities of a spatial audio processing arrangement 250 according to an example. It is understood that the two separate steps (focus / defocus processor+playback processor) shown in this figure and further detailed later can be implemented as an integrated process or in some examples can be implemented in the reverse order (playback processor operation followed by focus / defocus processor operation) as described herein. The spatial audio processing arrangement 250 comprises an audio focus processor 201 configured to receive an input audio signal and further focus / defocus parameters 202 and derive an audio signal having a focus / defocus sound component 204 based on the input audio signal 200 and depending on the focus / defocus parameters 202 (which may include focus / defocus direction, focus / defocus amount, focus / defocus height, focus / defocus radius, focus / defocus distance, and focus depth for the focus / defocus element). The spatial audio processing arrangement 250 may further comprise an audio playback processor 207 configured to receive an audio signal having focused / defocused sound components 204 and playback control information 206. And configured to derive an output audio signal 208 in a predefined audio format based on the audio signal having focused / defocused sound components 204, further depending on the playback control information 206 which functions to control at least one aspect related to the processing of the spatial audio signal having focused / defocused components in the audio playback processor 207. The playback control information 206 may include an indication of a playback direction (or playback orientation) and / or an indication of an applicable speaker configuration.In view of the methods for processing spatial audio signals described above, the audio focus processor 201 may be configured to implement an aspect of processing the spatial audio signal by modifying the audio scene to control emphasis or de-emphasis in at least a portion of the spatial audio signal in a received focus region or direction according to a received focus / defocus amount. The audio playback processor 207 may output the processed spatial audio signal as a modified audio scene based on the observed direction and / or position, the modified audio scene exhibiting emphasis according to the received focus amount for at least said portion of the spatial audio signal within the focus region.

[0115] In Fig. 2a, for each of the input audio signals, the audio signal with focused / defocused sound components and the output audio signal are provided as respective spatial audio signals in a predefined spatial audio format. These signals can therefore be referred to as the input spatial audio signal, the spatial audio signal with focused / defocused sound components, and the output spatial audio signal, respectively. In line with the above description, typically, the spatial audio signal conveys an audio scene including both one or more directional sound sources at each specific position of the audio scene and the ambience of the audio scene. However, in some scenarios, the spatial audio scene may include one or more directional sound sources without bidirectional sound sources, or bidirectional sound sources without any directional sound sources. In this regard, the spatial audio signal comprises information conveying one or more directional sound components representing distinct sound sources with a fixed position (e.g., a fixed direction of arrival and a fixed relative intensity with respect to the listening point) in the audio scene and / or ambient sound components representing environmental sounds in the audio scene. It should be noted that the division of an audio scene into a directional sound component and an ambient component is typically only a representation or approximation, but a real sound scene may contain more complex features such as wide sound sources and coherent acoustic reflections. Nevertheless, even with such complex acoustic features, the conceptualization of an audio scene as a combination of direct and ambient components is usually a fair representation or approximation, at least in a perceptual sense.

[0116] Typically, the input audio signal and the audio signal with focused / defocused tonal components are provided in the same predefined spatial format, while the output audio signal may be provided in the same spatial format applied for the input audio signal (and the audio signal with focused / defocused tonal components) or may adopt a different predefined spatial format for the output audio signal. The spatial audio format of the output audio signal is selected taking into account the characteristics of the sound reproduction hardware applied for the reproduction of the output audio signal. In general, the input audio signal may be provided in a first predefined spatial audio format and the output audio signal may be provided in a second predefined spatial audio format. Non-limiting examples of spatial audio formats suitable for use as the first and / or second spatial audio format include Ambisonic, surround loudspeaker signals according to a predefined loudspeaker configuration, predefined parametric spatial audio formats. More detailed non-limiting examples of the use of these spatial audio formats as the first and / or second spatial audio formats in the framework of the spatial audio processing arrangement 250 are provided later in this disclosure.

[0117] The spatial audio processing arrangement 250 is applied to process the input spatial audio signal 200, typically as a sequence of input frames, into a respective sequence of output frames, each input (output) frame comprising a respective segment of a digital audio signal for each channel of the input (output) spatial audio signal, provided as a respective time series of input (output) samples at a predefined sampling frequency. In some embodiments, the input signal to the spatial audio processing arrangement 250 may be in an encoded format, e.g. AAC, or AAC+embedded metadata. In such embodiments, the encoded audio input may initially be a decoder. Similarly, in some embodiments, the output from the spatial audio processing arrangement 250 may be encoded in any suitable manner.

[0118] In a typical example, the spatial audio processing device 250 uses a fixed, predefined frame length such that each frame contains a respective L number of samples for each channel of the input spatial audio signal, and maps to a corresponding duration at a predefined sampling frequency. As an example in this regard, the fixed frame length is 20 milliseconds (ms), which results in frames of L=160, L=320, L=640, and L=960 samples per channel, respectively, at sampling frequencies of 8, 16, 32, or 48 kHz. The frames may be non-overlapping or partially overlapping, depending on whether the processor applies filter banks and how these filter banks are configured. However, these values ​​serve as non-limiting examples, and frame lengths and / or sampling frequencies different from these examples may be used instead, depending on, for example, the desired audio bandwidth, the desired framing delay, and / or the available processing capacity.

[0119] In the spatial audio processing device 250, focus / defocus refers to a user-selectable direction / amount parameter (or spatial region of interest). Focus / defocus may be, for example, a certain direction, distance, radius, arc of the audio scene in general. Another example is a focus / defocus region where a (directional) sound source of interest is currently located. In the former scenario, the user-selectable focus / defocus may indicate a region that remains constant or changes infrequently since the focus is mainly in a certain direction (or spatial region), while in the latter scenario, the user-selected focus / defocus may change more frequently since the focus / defocus is set on a particular sound source that may (or may not) change its position (or shape / size) in the audio scene over time. In one example, focus / defocus may be defined as, for example, an azimuth angle that defines a direction.

[0120] The functionality described above with reference to the components of the spatial audio processing device 250 may for example be provided according to a method 260 illustrated by the flow chart shown in FIG. 2b. The method 260 may for example be provided by an apparatus configured to implement the spatial audio processing system 250 described in the present disclosure through some examples. The method 260 serves as a method for processing an input spatial audio signal representing an audio scene into an output spatial audio signal representing a modified audio scene. The method 260 includes receiving an indication of a focus / defocus direction and an indication of a focus / defocus intensity or amount, as shown in block 261. The method 260 further includes processing the input spatial audio signal into an intermediate spatial audio signal representing a modified audio scene, in which the relative levels of sounds arriving from said focus / defocus direction are modified according to said focus / defocus intensity, as shown in block 263. The method 260 further includes receiving playback control information for controlling the processing of the intermediate spatial signal into an output spatial audio signal, as shown in block 265. The playback control information may, for example, define at least one of a playback direction (e.g., listening direction or line of sight direction) or a speaker configuration of the output spatial audio signals. Method 260 further includes processing the intermediate spatial audio signals into output spatial audio signals according to the playback control information, as shown in block 267.

[0121] The method 260 may be modified in a number of ways, for example according to the examples of the functionality of each of the components of the spatial audio processing unit 250 provided above and below.

[0122] In the following examples, the defocus operation is described in further detail, but it should be understood that the same operation can be applied to further focus operations as well as further defocus operations.

[0123] In some embodiments, the input to the spatial audio processing arrangement 250 is an Ambisonic signal. The apparatus can be configured to receive (and apply the method to) Ambisonic signals of any order. The Ambisonic audio signal could be a First Order Ambisonic (FOA) signal consisting of an omnidirectional signal and three orthogonal first order patterns along the y, z, x coordinate axes. The y, z, x coordinate order is chosen here because it is the same order as the first order coefficients of a typical ACN (Ambisonics Channel Numbering) channel ordering of Ambisonic signals.

[0124] It should be noted that the Ambisonics audio format can represent spatial audio signals in terms of spatial beam patterns, and it would be straightforward for one skilled in the art to illustrate and design alternative sets of spatial beam patterns to represent spatial audio. Furthermore, the Ambisonics audio format is a particularly relevant audio format since it is a typical way of representing spatial audio in the context of 360 video. Typical sources of Ambisonics audio signals include microphone arrays and content from VR video streaming services (such as YouTube® 360).

[0125] With reference to Fig. 3a, the focus processor 350 is shown in the context of Ambisonic input and output. The diagram assumes a first-order Ambisonic (FOA) signal (4 channels), but higher-order Ambisonic (HOA) may be applied instead of FOA. In an embodiment implementing an HOA input format, the number of channels instead of 4 channels may be, for example, 9 channels (second-order Ambisonics) or 16 channels (third-order Ambisonics).

[0126] An example ambisonic signal x FOA (t) 300 and (de)focus direction 304 , (de)focus amount and (de)focus control 310 are inputs to a focus processor 350 .

[0127] In some embodiments, the focus processor 350 comprises a filter bank 301. The filter bank 301 is configured in some embodiments to transform an Ambisonic (FOA) signal 300 (corresponding to Ambisonic or spherical harmonic patterns) to generate a time-frequency domain version of the time-domain input audio signal. The filter bank 301 in some embodiments may be any other suitable filter bank for spatial audio processing, such as a short-time Fourier transform (STFT) or complex modulated quadrature mirror filter (QMF) bank. The output of the filter bank 301 is a frequency-banded time-frequency domain Ambisonic audio signal 302. The frequency bands can be one or more frequency bins (individual frequency components) of the applied filter bank 301. The frequency bands can approximate a perceptually relevant resolution, such as Bark frequency bands, which are spectrally more selective at low frequencies than at high frequencies. Alternatively, the frequency bands can correspond to frequency bins in some implementations.

[0128] The (unfocused) time-frequency domain Ambisonic audio signal 302 is output to a mono focuser 303 and to a mixer 311 .

[0129] The focus processor 301 may further comprise a mono focuser 303 arranged to receive the transformed (unfocused) time-frequency domain Ambisonic signal 302 from the filter bank 301 and further to receive a (de)focus direction parameter 304.

[0130] The mono (de)focuser 303 can implement any known method for generating a mono focused audio output based on the FOA input. In this example, the mono focuser 303 implements a minimum variance distortion-free response (MVDR) mono focused audio output. Due to this constraint, it adaptively finds beamforming weights that try to minimize the output energy (in other words, suppress the interference energy) while the MVDR beamforming operation tries to obtain the target signal from the desired focus direction without distortion.

[0131] In some embodiments, the mono focuser 303 focuses a frequency band signal (e.g., four channels in the case of FOA) into

number

[0132] A mono-focuser 303 implementing an MVDR beamformer can be used for each frequency band k. An estimate of the covariance matrix of the signal x(b,n) in the bins in band k (and possibly also time-averaged over some time index n). A steering vector according to the focus direction. In the example of an FOA signal, the steering vector may be generated based on a unit vector oriented in the focus direction. For example, the steering vector for FOA may be

number

[0133] Based on the covariance matrix estimates and the steering vectors, the weights w(k,n) can be generated using the known MVDR formula.

[0134] Thus, the mono focuser 303 can provide a single channel focused output signal 306 which in some embodiments is provided to an Ambisonic panner 305 .

[0135] In some embodiments, an Ambisonics panner 305 is configured to receive the channel (de)focused output signals 306 and the (de)focus direction 304 and generate an Ambisonic signal, where the mono focused signal is positioned in the focus direction. The focused time-frequency Ambisonic signal 308 output by the Ambisonics panner 305 is

number

[0136] In some embodiments, the (de)focused time-frequency Ambisonic signal y FOA The (b,n) 308 may then be output to a mixer 311.

[0137] In some embodiments, the output of a beamformer such as MVDR can be cascaded with a post-filter. A post-filter is a process that adaptively changes the gain or energy of the beamformer output, typically within a frequency band. For example, MVDR is known to be effective at suppressing strong individual interfering sources, but only performs moderately in ambient acoustic scenes, such as outdoor recordings with traffic noise. This is because MVDR effectively aims to steer the beam pattern minimum in the direction where the interferer is located. If the interfering sound is spatially spread, such as traffic noise, MVDR will not effectively suppress the interference.

[0138] Thus, a post filter can be implemented in some embodiments to estimate the sound energy in a frequency band in the focus direction. The beamformer output energy is then measured in the same frequency band, and a gain is applied in the frequency band to correct the sound spectrum to improve the estimated target spectrum. In such an embodiment, the post filter can further suppress interference sounds.

[0139] An example of a post filter is described in Delikaris Manias, Symeon, and Ville Pulkki, "Cross-Spectral Coherence Algorithm for Spatial Filtering Applications Utilizing Microphone Arrays," IEEE Transactions on Audio, Speech, and Language Processing 21, No. 11 (2013): 2356-2367, where the target energy in the look direction is estimated using a cross-spectral energy estimate between the 1st and 2nd order spherical harmonic signals. Cross-spectral estimates can also be obtained for other patterns, such as between the zeroth (omnidirectional) and 1st (dipole) order spherical harmonic signals. The cross-spectral estimate provides an energy estimate for the target direction.

[0140] If post-filtering is implemented, a gain g(k,n) can be added to the beamforming equation.

number

[0141] This gain g(k,n) can be derived using the cross-spectral energy estimation method as follows: First, we formulate the cross-correlation between the omnidirectional FOA signal component with a positive lobe toward the focus direction and the figure-of-eight signal,

number

number

number

number

[0142] In other words, if the energy estimate C(k,n) is less than the beamform output energy D(k,n), then the beamform output energy in band k is reduced by the spatial filter. Thus, the function of the spatial filter is to further adjust the spectrum of the beamformer output to more closely resemble the spectrum of the sound arriving from the focus direction.

[0143] In some embodiments, the (de)focus processor may utilize this post-filtering. The beamformed output y(b,n) of the monofocuser 303 may be processed in frequency bands with a post-filter gain to generate a post-filtered beamformed output y'(b,n), where y'(b,n) is applied instead of y(b,n). It will be appreciated that there are a variety of suitable beamformers and post-filters that may be applied other than those described as examples above.

[0144] In some embodiments, the focus processor 350 comprises a mixer 311. The mixer receives the (de)focused time-frequency Ambisonic signal y FOA The mixer 311 is configured to receive the (de)focus amount and focus / defocus control parameters 310. The mixer 311 is configured to receive the (b,n) 308 and the unfocused time-frequency Ambisonic signal x(b,n) 302 (with potential delay adjustment where MVDR estimation and processing includes look-ahead processing).

[0145] In this example, the (de)focus control parameter is a binary switch of "focus" or "defocus". The (de)focus amount parameter a(n), expressed as a factor between 0..1 (where 1 is maximum focus), is utilized to describe either the amount of focus or the amount of defocus, depending on which mode is used.

[0146] In some embodiments, when the defocus parameter is in “focus” mode, the output of the mixer 311 is:

number

[0147] In some embodiments, the mixer, when the defocus parameter is in the “defocus” mode:

number

[0148] In other words, when a(n) is 0, the defocus process is also zero, however, when a(n) is greater than or equal to 1, the blending procedure blends the spatial FOA signal x(b,n) into a spatialized focus signal, signal y FOA (b,n) is subtracted. The subtraction reduces the amplitude of the signal components from the focus direction. In other words, the defocusing process is performed, and the resulting Ambisonic spatial audio signal has reduced amplitude for sounds from the focus direction. In some configurations, y MIX (b,n) 312 can be amplified on a principle basis as a function of a(n) to account for the average loss of loudness due to the defocus process.

[0149] The output of the mixer 311, the mixed time-frequency Ambisonic audio signal 312, is passed to an inverse filter bank 313.

[0150] In some embodiments, the focus processor 350 includes an inverse filter bank 313 configured to receive the mixed time-frequency Ambisonic audio signal 312 and convert the audio signal to the time domain. The inverse filter bank 313 generates a suitable pulse code modulated Ambisonic audio signal with added focus / defocus.

[0151] FIG. 3b shows a flow chart of the operation 360 of the FOA focus processor shown in FIG. 3a.

[0152] The initial operation is by step 361 receiving an Ambisonic (FOA) audio signal (and focus parameters such as direction, width, amount or other control information), as shown in FIG. 3b.

[0153] The next operation is to generate a converted Ambisonic audio signal in the time-frequency domain, as shown in FIG. 3b, according to step 363.

[0154] Having generated a time-frequency domain Ambisonic audio signal, the next operation is one of generating a mono-focused Ambisonic audio signal from the time-frequency domain Ambisonic audio signal based on a focus direction (e.g., using beamforming), as shown in FIG. 3b by step 365.

[0155] Then, by step 367, Ambisonic spanning is performed on the mono-(de)focused Ambisonic audio signal based on the focus direction as shown in FIG. 3b.

[0156] The panned Ambisonic audio signal (the (de)focused time-frequency Ambisonic signal) is then mixed with the unfocused time-frequency Ambisonic signal based on the (de)focus amount and the (de)focus control parameters, as shown in Figure 3b, by step 369.

[0157] The mixed Ambisonic audio signal may then be inverse transformed as shown in FIG. 3b, by step 371.

[0158] A time domain Ambisonic audio signal is then output, as shown in FIG. 3b, by step 373.

[0159] Referring to FIG. 4a, a focus processor is shown configured to receive a parametric spatial audio signal as an input. The parametric spatial audio signal includes an audio signal and spatial metadata such as direction(s) in frequency bands and direct-to-total energy ratio(s). The structure and generation of parametric spatial audio signals is known and their generation has been described from microphone arrays (e.g., mobile phones, VR cameras). Furthermore, parametric spatial audio signals can be generated from loudspeaker signals and Ambisonic signals. In some embodiments, the parametric spatial audio signal may be generated from an IVAS (Immersive Voice and Audio Services) audio stream, which may be decoded and demultiplexed into the form of spatial metadata and audio channels. A typical number of audio channels in such a parametric spatial audio stream is a two audio channel audio signal, but in some embodiments the number of audio channels can be any number of audio channels.

[0160] In some examples, the parametric information includes depth / distance information that can be implemented in six degrees of freedom (6DOF) playback, where distance metadata is used (along with other metadata) to determine how sound energy and direction should change as a function of the user's movement.

[0161] In this example, each spatial metadata directional parameter is associated with both a direct-to-total energy ratio and a distance parameter. The estimation of distance parameters associated with parametric spatial audio capture has been detailed in previous applications such as GB patent documents GB1710093.4 and GB1710085.0 and will not be explored further for clarity.

[0162] A focus processor 450 configured to receive the parametric spatial audio 400 is configured to use (de)focus parameters to determine how much the direct and ambient components of the parametric spatial audio signal should be attenuated or emphasized to enable a (de)focus effect. The focus processor 450 is described in two configurations: the first uses (de)focus parameters, i.e. direction and amount, and further includes width resulting in a focus / defocus arc. In this configuration, the 6DOF distance parameter is optional. The second uses parameters (de)focus direction and amount as well as distance and radius, which results in a focus / defocus sphere at a position. In this configuration, the 6DOF distance parameter is required. The difference between these configurations is expressed only where necessary in the following description.

[0163] In the following examples, the methods (and formulas) are presented as not varying over time, however, it should be understood that all parameters may vary over time.

[0164] In some embodiments, the focus processor comprises a ratio modifier and spectral adjustment coefficient determiner 401 configured to receive focus parameters 408 and further spatial metadata consisting of direction 402 (and in some embodiments distance 422) and direct-to-total energy ratio 404 in frequency bands.

[0165] The ratio modifier and spectral adjustment coefficient determiner 401 is configured to receive the focus parameters and further receive spatial metadata consisting of the direction 402, the direct-to-total energy ratio in the frequency band 404 (and in some embodiments the distance 422).

[0166] In the following description, unless otherwise specified, it is considered that the focus parameters include a direction, a width, and an amount. In some embodiments, the ratio modifier and spectral adjustment factor determiner 401 is configured to determine the angle difference between the focus direction (one for every frequency band k) and the spatial metadata direction (potentially different in different frequency bands k). In some embodiments, v m (k) is determined as a column-wise vector pointing to the direction parameter of the spatial metadata in band k and as a column-wise vector pointing to the focus direction. The angular distance β(k) is

number

[0167] The ratio modifier and spectral adjustment coefficient determiner 401 is then configured to determine the direct gain parameter f(k). The focus amount parameter a may be expressed as a normalized number between 0..1 (where 0 means zero focus / defocus and 1 is maximum focus / defocus) and a focus width β0, which may be, for example, 20 degrees at one point in time.

[0168] When the ratio modifier and spectral adjustment factor determiner 401 is configured to perform focusing (as opposed to defocusing), an example gain formula is:

number

number

[0169] The exemplary formula indicates that in some embodiments, the constant c may have different values ​​for focus imperfections and defocus. Furthermore, in practice, it may be desirable to smooth the above function so that the focus gain function smoothly transitions from high values ​​in focus regions to low values ​​in unfocused regions.

[0170] In the following description, unless otherwise specified, it is assumed that the focus parameters include direction, distance, radius, and amount. In some embodiments, the ratio modifier and spectral adjustment factor determiner 401 is formulated as follows: f and metadata position p m (k). In some embodiments, v m (k) is determined as a column-wise vector pointing to the directional parameters of the spatial metadata in band k, and as a column-wise vector pointing to the focus direction. The focus position is determined by p f =v f d f where d f is the focus distance. The spatial metadata position is formulated as being a distance parameter in the spatial metadata in band k. In some embodiments, the ratio modifier and spectral adjustment factor determiner 401 is configured to determine the difference between the focus positions (one for every frequency band k) and the spatial metadata positions, potentially different positions in different frequency bands k. The position difference is

number

[0171] The ratio modifier and spectral adjustment coefficient determiner 401 is then configured to determine the direct gain parameter f(k). The focus amount parameter may be represented as a normalized value r0 between 0..1 (where 0 means zero focus / defocus and 1 means maximum focus / defocus), and the focus radius may be, for example, 1 meter at a certain time instance.

[0172] When the ratio modifier and spectral adjustment factor determiner 401 is configured to perform focusing (as opposed to defocusing), an example gain formula is:

number

number

[0173] In some embodiments, the constant c may have different values ​​for the defocused and focused cases. Furthermore, in practice, it may be desirable to smooth the above focus gain function so that it transitions smoothly from high values ​​in focused regions to low values ​​in unfocused regions.

[0174] The remaining description is applicable to both focus parameter configurations described above. In some embodiments, the ratio modifier and spectral adjustment factor determiner 401 is further configured to determine new direct part values ​​of the parametric spatial audio signal as follows:

number

number

[0175] In one embodiment, the ratio modifier and spectral adjustment factor determiner 401 is configured to determine the new ambient components in the defocusing process using A(k)=(1-r(k)), which means that the defocusing process does not affect the ambient energy spatially.

[0176] The ratio modifier and spectral adjustment coefficient determiner 401 is then configured to determine the spectral correction coefficients that are output to the spectral adjustment processor 403, which are then formulated based on the overall modification of the acoustic energy. For example,

number

[0177] In some embodiments, the ratio modifier and spectral adjustment factor determiner 401 is

number

[0178] If it is not numerically determined, D(k)=A(k)=0, then r'(k) can be set to zero.

[0179] The direction values ​​402 (and distance values ​​422) in the spatial metadata may be passed through and output unmodified in some embodiments.

[0180] The focus processor in some embodiments comprises a spectral adjustment processor 403. The spectral adjustment processor 403 is configured to receive an audio signal (which in some embodiments is a time-frequency representation, or alternatively, which is first transformed to the time-frequency domain) 406 and spectral adjustment coefficients 412. In some embodiments, the output audio signal 414 may also be in the time-frequency domain and may be transformed back to the time domain before being output. The domains of the input and output may depend on the implementation.

[0181] The spectral adjustment processor 403 is configured to, for each band k, multiply the frequency bins (of the time-to-frequency transform) of all channels in band k by a spectral adjustment factor s(k). In other words, the spectral adjustment processor 403 is configured to perform a spectral adjustment. To avoid processing artifacts, the multiplications / spectral corrections can be smoothed over time.

[0182] In other words, the focus processor 450 is configured to modify the spectrum of the audio signal and the spatial metadata such that the procedure results in a parametric spatial audio signal modified according to the (de)focus parameters.

[0183] With reference to FIG. 4b, there is shown a flow diagram 460 of the operation of a parametric spatial audio input processor as shown in FIG. 4a.

[0184] The initial operation begins with step 461 receiving a parametric spatial audio signal (and focus / defocus parameters or other control information), as shown in FIG. 4b.

[0185] The next action is to modify the parametric metadata and generate spectral adjustment coefficients, as shown in FIG. 4b by step 463.

[0186] The next action is to perform a spectral adjustment on the audio signal, as shown in FIG. 4b by step 465.

[0187] The spectrally adjusted audio signal and the modified (and unmodified) metadata may then be output, via step 467, as shown in FIG. 4b.

[0188] Referring to FIG. 5a, a focus processor 550 is shown configured to receive a multi-channel or object audio signal as an input 500. The focus processor in such an example may comprise a focus gain determiner 501. The focus gain determiner 501 is configured to receive focus / defocus parameters 508 and channel / object position / orientation information, which may be static or time-varying. Based on the (de)focus parameters 508 (such as (de)focus direction, (de)focus amount, (de)focus control, and optionally (de)focus distance and radius or (de)focus width) and spatial metadata information 502 from the input signal 500, in some embodiments a channel signal direction is signaled and in some embodiments a channel signal direction is assumed. For example, if 6 channels are present, the direction may be assumed to be a 5.1 audio channel direction. In some embodiments, there may be a look-up table used to determine the channel direction as a function of the number of channels.

[0189] In some embodiments, there is no filter bank, i.e. there is only one frequency band k. The direct gain f(k) for each audio channel is output to the focus gain processor 503 as a focus gain.

[0190] In some embodiments, a focus gain processor 503 is configured to receive the audio signal and a focus gain value 512 and process the audio signal 506 based on the focus gain value 512 (per channel), potentially with some temporal smoothing. The processing based on the focus gain value 512 may in some embodiments be a multiplication of the focus gain value with the channel / object signal.

[0191] The output of the focus gain processor 503 is the focus processed audio channel. The channel direction / position information is unchanged and is also provided as output 510.

[0192] In some embodiments, the defocus process can be configured to be wider than one direction. For example, a focus width may be included as an input parameter. In these embodiments, the user may also generate a defocus arc. In another example, a focus distance and a focus radius may be included as input parameters. In these embodiments, the user may generate a defocus sphere at a determined location. Similar procedures may be employed for other input spatial audio signal types.

[0193] In some embodiments, the audio objects (spatial metadata) may include distance parameters, which may also be taken into account. For example, the focus / defocus parameters may determine the focus location (direction and distance), and the radius parameter may control the focus / defocus area around that location. In such an embodiment, the user may generate a defocus pattern as shown in FIG. 1c and described above. Similarly, another spatially related parameter may be defined to allow the user to control a different shape of the defocus area. In some embodiments, the attenuation of the audio objects within the defocus area may be an attenuation by a fixed number of decibels (e.g., 10 dB) multiplied by a desired amount of defocus between 0 and 1, leaving the audio objects outside the defocus direction. It is configured to generate the gain f(k) parameter 512 directly, without gain modification (or without applying any gain or attenuation related to the focus operation to the audio objects outside the defocus direction). In the formulation of the direct gain (output as focus gain), the focus gain determiner 501 can utilize the same formula as described in the context of the ratio modifier and spectral adjustment coefficient determiner 401 of Fig. 4a to determine the direct gain f(k). The exception is in the case of audio objects / channels, where typically only one frequency band is present and the spatial metadata typically indicates only the object direction / distance and not the ratio. If the distance is not available, a fixed distance can be assumed, e.g. 2 meters.

[0194] FIG. 5b shows a flow chart 560 of the operation of the multi-channel / object audio input processor shown in FIG. 5a.

[0195] The initial operation is to receive a multi-channel / object audio signal and, in some embodiments, to receive channel information such as the number of channels and / or distribution of channels (and focus / defocus parameters or other control information), as shown in FIG. 5b by step 561.

[0196] The next operation produces a focus gain factor as shown in FIG. 5b by step 563.

[0197] The next action is to apply a focus gain to each channel audio signal, as shown in FIG. 5b by step 565.

[0198] The processed audio signal and the unaltered channel direction (and distance) may then be output by step 567, as shown in Figure 5b.

[0199] With reference to Figure 6a, an example of a playback processor 650 based on Ambisonic audio input is shown (which may for example be configured to receive the output from a sample focus processor as shown in Figure 3a).

[0200] In these examples, the playback processor may comprise an Ambisonic rotation matrix processor 601. The Ambisonic rotation matrix processor 601 is configured to receive an Ambisonic signal with focus / defocus processing 600 and a view direction 602. The Ambisonic rotation matrix processor 601 is configured to generate a rotation matrix based on the view direction parameters 602. This may in some embodiments use any suitable method, such as that applied in head-tracked Ambisonic binauralization (or more generally such rotations of spherical harmonics are used in many fields including outside of audio). The rotation matrix is ​​then applied to the Ambisonic audio signal. The result is a rotated Ambisonic signal plus focus / defocus 604, which is output to the Ambisonic to Binaural Filter 603.

[0201] The Ambisonic-to-Binaural Filter 603 is configured to receive the rotated Ambisonic signal with added focus / defocus 604. The Ambisonic-to-Binaural Filter 603 may include a pre-formulated 2xK matrix of Finite Impulse Response (FIR) filters to be applied to the K Ambisonic signals to generate two binaural signals 606. In this example where a four-channel FOA audio signal is shown, K=4. The FIR filters may be generated by a least-squares optimization method on a set of Head-Based Impulse Responses (HRIRs). An example of such a design procedure is to transform (e.g., by FFT) the HRIR dataset into frequency bins to obtain an HRTF dataset, and to determine for each frequency bin, in a least-squares manner, a complex-valued processing matrix that approximates the HRTF dataset available at the data points of the HRTF dataset. When the complex-valued matrix for all frequency bins is determined in this way, the result can be inverted (e.g., by inverse FFT) as a time-domain FIR filter. The FIR filters can also be windowed, for example, by using a Hann window.

[0202] In some embodiments, the rendering is to a loudspeaker rather than to a headphone. There are many known methods that can be used to render an Ambisonic signal to a loudspeaker output. One example may be a linear decoding of the Ambisonic signal to a target loudspeaker configuration. This can be applied with good expected spatial fidelity if the order of the Ambisonic signal is sufficiently high, e.g., at least third order, preferably fourth order. In a specific example of such linear decoding, an Ambisonic decoding matrix may be designed that, when applied to the Ambisonic signal (corresponding to an Ambisonic beam pattern), generates loudspeaker signals corresponding to a beam pattern that, in a least squares sense, approximates a vector-based amplitude panning (VBAP) beam pattern suitable for the targeted loudspeaker configuration. Processing the Ambisonic signal with such a designed Ambisonic decoding matrix can be configured to generate a loudspeaker audio output. In such an embodiment, the playback processor is configured to receive information about the speaker configuration and no rotation processing is required.

[0203] FIG. 6b shows a flow chart 660 of the operation of the Ambisonic input playback processor shown in FIG. 6a.

[0204] The initial operation begins with step 661 receiving a focused / defocused Ambisonic audio signal (and a view direction), as shown in FIG. 6b.

[0205] The next operation is one of generating a rotation matrix based on the view direction as shown in FIG. 6b by step 663.

[0206] The next operation is to apply a rotation matrix to the Ambisonic audio signal to generate a rotated Ambisonic audio signal with focus / defocus processing as shown in FIG. 6b by step 665.

[0207] The next operation is then to convert the signal into a suitable audio output format, for example a binaural format (or a multi-channel audio format or a loudspeaker format), as shown in FIG. 6b by step 667.

[0208] Next, the output audio format is outputted, as shown in FIG. 6b, via step 669.

[0209] With reference to FIG. 7a, an example of a playback processor 750 based on a parametric spatial audio input (which may be configured to receive, for example, the output from the exemplary focus processor shown in FIG. 4a) is shown.

[0210] In some embodiments, the playback processor comprises a filter bank 701 configured to receive the audio channel 700 audio signal (unless the input is already in the appropriate time-frequency domain) and transform the audio channel into frequency bands. Examples of suitable filter banks include short-time Fourier transform (STFT) and complex quadrature mirror filter (QMF) banks. The time-frequency audio signal 702 can be output to a parametric binaural synthesizer 703.

[0211] In some embodiments, the playback processor comprises a parametric binaural synthesizer 703 configured to receive the time-frequency audio signal 702, the modified (and unmodified) metadata 704, and the view direction 706 (or suitable playback related control or tracking information). In the context of 6DOF playback, the user position may be provided along with the view direction parameter.

[0212] The parametric binaural synthesizer 703 can be configured to implement any suitable known parametric spatial synthesis method configured to generate a binaural audio signal (in frequency bands) 708, since focus correction has already been performed on the signal and metadata before the parametric binauralization block. One known method for parametric binaural synthesis is to split the time-frequency audio signal 702 into frequency band direct and ambient part signals based on the frequency band direct-to-total ratio parameter, process the frequency band direct part with HRTF corresponding to the frequency band directional parameter, process the ambient part with a decorrelator to obtain binaural diffuse sound field coherence, and combine the processed direct and ambient parts. The binaural audio signal (in frequency bands) 708 has two channels, regardless of how many channels the time-frequency audio signal 702 has. The binauralized time-frequency audio signal 708 can then be passed to an inverse filter bank 705. The embodiment may further feature a playback processor including an inverse filter bank 705 configured to receive the binauralized time-frequency audio signal 708 and apply the inverse to the applied forward filter bank, thus generating a time-domain binauralized audio signal 710 having focusing characteristics suitable for playback by headphones (not shown in FIG. 7a).

[0213] In an embodiment, the binaural audio signal output is replaced by loudspeaker channel audio signals output from the parametric spatial audio signal using a suitable loudspeaker synthesis method. Any suitable approach can be used, for example the view direction parameters are replaced with information of the loudspeaker positions and the parametric binaural synthesizer 703 is replaced with a parametric loudspeaker synthesizer based on a suitable known method. One known method for parametric loudspeaker synthesis is to split the time-frequency audio signal 702 into direct and ambient part signals in frequency bands based on a direct-to-sum ratio parameter in frequency bands, process the direct part in frequency bands with a Vector-Based Amplitude Panning (VBAP) gain corresponding to the loudspeaker configuration and the direction parameter in frequency bands, process the ambient part with a decorrelator to obtain an incoherent loudspeaker signal, and combine the processed direct and ambient parts. The loudspeaker audio signals (per frequency band) have a number of channels that is determined by the loudspeaker configuration, regardless of how many channels the time-frequency audio signal 702 has.

[0214] With reference to FIG. 7b, there is shown a flow diagram 760 of the operation of a parametric spatial audio input reproduction processor as shown in FIG. 7a.

[0215] The first action is to receive a focus / defocus processed parametric spatial audio signal (and view direction or other playback related control or tracking information), as illustrated in FIG. 7b by step 761.

[0216] The next operation is one of time-frequency transforming the audio signal as shown in FIG. 7b by step 763.

[0217] The next operation is to apply a parametric binaural (or loudspeaker channel format) processor based on the time-frequency transformed audio signal, the metadata and the gaze direction (or other information), as shown in FIG. 7b by step 765.

[0218] The next operation then, by step 767, is to inverse transform the generated binaural or loudspeaker channel audio signals, as shown in FIG. 7b.

[0219] Next, step 769 outputs the output audio format as shown in FIG. 7b.

[0220] Considering the speaker output for the playback processor when the audio signal is in the form of multi-channel audio and the focus processor 550 of FIG. 5a is applied, in some embodiments the playback processor can comprise a pass-through where the output loudspeaker configuration is the same as the format of the input signal. In some embodiments where the output loudspeaker configuration is different from the input loudspeaker configuration, the playback processor can comprise a vector-based amplitude panning (VBAP) processor. Each of the focus-processed audio channels can then be processed using VBAP, a known amplitude panning technique, to spatially play them back using the target loudspeaker configuration. In this way the output audio signal matches the output loudspeaker setup.

[0221] In some embodiments, the conversion from the first loudspeaker configuration to the second loudspeaker configuration may be performed using any suitable amplitude panning technique. For example, the amplitude panning technique may include deriving an N×M matrix of amplitude panning gains that defines the conversion from the M channels of the first loudspeaker configuration to the N channels of the second loudspeaker configuration, and then using the matrix to multiply the channels of an intermediate spatial audio signal that is provided as a multi-channel loudspeaker signal according to the first loudspeaker configuration. The intermediate spatial audio signal can be understood to be similar to an audio signal with focused / defocused sound components 204, as shown in FIG. 2a. As a non-limiting example, the derivation of the VBAP amplitude panning gains is provided in Pulkki, Ville: "Virtual Sound Source Localization with Vector-Based Amplitude Panning", Journal of Audio Engineering 45, no. 6 (1997), pp. 456-466.

[0222] For binaural output, any suitable binauralization of multi-channel loudspeaker signal formats (and / or objects) may be implemented. For example, a typical binauralization may include processing the audio channels with Head Related Transfer Functions (HRTFs) and adding synthetic room reverberation to generate an auditory impression of a listening room. Distance + direction (i.e., position) information of audio object sounds may be utilized for 6DOF reproduction with user movement, for example by employing the principles outlined in UK Patent Application GB1710085.0.

[0223] An exemplary apparatus suitable for implementation is shown in Figure 8 in the form of a mobile phone or handheld device 901 running suitable software 903. Video can be played, for example, by attaching the mobile phone 901 to a Daydream view type device (for clarity, video processing is not described here).

[0224] The audio bitstream obtainer 923 is configured to obtain, for example, an audio bitstream 924 received / retrieved from a memory. In some embodiments, the mobile device comprises a decoder 925 configured to receive compressed audio and decode it. An example of a decoder is an AAC decoder in the case of AAC decoding. The resulting decoded (e.g., Ambisonic (Ambisonic) implementing the examples shown in Figures 3a and 6a) audio signal 926 can be forwarded to a focus processor 927.

[0225] The mobile phone 901 receives controller data 900 from an external controller at a controller data receiver 911 (e.g., via Bluetooth) and passes the data to a focus parameter (from controller data) determiner 921. The focus parameter (from controller data) determiner 921 determines the focus parameters based, for example, on the orientation of the controller device and / or button events. The focus parameters can include any type of combination of proposed focus parameters (e.g., focus / defocus direction, focus / defocus amount, focus / defocus height, and focus / defocus width). The focus parameters 922 are forwarded to a focus processor 927.

[0226] Based on the Ambisonic audio signal and the focus parameters 927 is configured to generate modified Ambisonic signals 928 having desired focus characteristics. These modified Ambisonic signals 928 are forwarded to the Ambisonic processor 929. The Ambisonic binaural processor 929 is also configured to receive head orientation information 904 from the orientation tracking device 913 of the mobile phone 901. Based on the modified Ambisonic signal 928 and the head direction information 904, the Ambisonic / binaural processor 929 is configured to generate a head tracked binaural signal 930 that can be output from the mobile phone and played back using, for example, headphones.

[0227] 9 shows an example device (or focus / defocus parameter control) 1050 that may be configured to control or generate appropriate focus / defocus parameters such as focus / defocus direction, focus / defocus amount, and focus / defocus width. A user of the device may be configured to select a focus direction by pointing the controller in a desired direction 1009 and pressing a focus direction selection button 1005. The control has an orientation tracker 1001, and the orientation information may be used to determine the focus / defocus direction (e.g., in a focus parameter determiner 921 (from control unit data) as shown in FIG. 8). The focus / defocus direction in some embodiments may be visualized in a visual display while selecting the focus / defocus direction.

[0228] In some embodiments, the focus amount can be controlled using focus amount buttons (shown as + and - in FIG. 9) 1007. Each press increases or decreases the focus amount by, for example, an amount of 10 percentage points. In some embodiments, the focus amount is set to 0% and the user presses the minus button, the focus amount is set to 10% and the focus / defocus control is set to "defocus" mode; correspondingly, if the focus amount is set to 0% and the user presses the plus button, the focus amount is set to 10% and the focus / defocus control is set to "focus" mode.

[0229] In some embodiments, it may be desirable to further specify the focus or defocus processing, for example, by determining a desired frequency range or spectral characteristics of the focus signal. In particular, it may be useful to emphasize or de-emphasize the audio spectrum in the audio frequency range to improve intelligibility or to block out a talker, for example, by attenuating low frequency content (e.g., below 200 Hz) and high frequency content (e.g., above 8 kHz), thus leaving a particularly useful frequency range associated with the audio.

[0230] Similarly, when a user indicates a direction to be defocused, the audio processing system can analyze the spectrum or type of interference (e.g., speech, noise) in the direction to be attenuated. Then, based on this analysis, the system could determine a frequency range or amount of defocus per frequency that is well suited to that interferer. For example, the interferer may be a device that generates high frequency noise, and the high frequencies for that defocus direction would be attenuated more than, for example, mid-low frequencies. In another example, the defocus direction has a talker, and therefore the amount of defocus can be configured per frequency to suppress primarily typical audio frequency ranges.

[0231] It will be appreciated that the focus processed signal may be further processed with any known audio processing techniques, such as automatic gain control or enhancement techniques (eg, bandwidth expansion, noise suppression).

[0232] In some further embodiments, the focus / defocus parameters (including direction, amount, and control) are generated by the content creator, and the parameters are transmitted together with the spatial audio signal. For example, in a documentary of a VR video / audio nature with an on-site commentator, instead of the user having to select the direction of the commentator to be defocused, a dynamic focus parameter preset can be selected. The preset may be fine-tuned by the content creator to follow the commentator's movements. For example, defocus is enabled only when the commentator speaks. In other words, the content creator can generate several expected or estimated preference profiles as focus / defocus parameters. This approach is beneficial because it only requires conveying one spatial audio signal, but different preference profiles can be added. Legacy players that are not focus-enabled can be configured to simply decode Ambisonic or other signal types without applying focus / defocus processing.

[0233] An exemplary processing output is shown in FIG. 10 based on the implementation described for Ambisonic signals. In this example, three sound sources are in the audio scene: a talker in front, a talker at -90 degrees to the right, and a white noise interferer at 110 degrees to the left. FIG. 10 shows how, with the focus / defocus control set to "focus", the focus process is used to broadly emphasize the direction in which the noise source is located, and how, with the focus / defocus control set to "defocus", the focus process is used to broadly de-emphasize the direction in which the noise source is located while preserving the two talker signals in the spatial audio output. Thus, the Ambisonic signals are shown in three columns (omni W 1101, horizontal dipole Y 1103 and X 1105) in an exemplary situation shown by the Ambisonic signal in row 1111 with a talker in front (indicated specifically by signal X), a talker at -90 degrees to the right (indicated specifically by signal Y), and a noise interferer at 110 degrees to the left (indicated by all signals). The next column 1113 shows an Ambisonic audio signal with full focus towards the noise source. The bottom row 1115 shows an Ambisonic audio signal with full defocus towards the noise source (i.e. de-emphasizing the noise), leaving most of the speech source active.

[0234] 11, an exemplary electronic device that can be used as an analysis or synthesis device is shown. The device may be any suitable electronic device or apparatus. For example, in some embodiments, the device 1700 is a mobile device, user equipment, tablet computer, computer, audio playback device, etc. In some embodiments, the device 1200 comprises at least one processor or central processing unit 1207. The processor 1207 may be configured to execute various program codes, such as the methods described herein.

[0235] In some embodiments, the apparatus 1200 comprises a memory 1211. In some embodiments, at least one processor 1207 is coupled to the memory 1211. The memory 1211 may be any suitable storage means. In some embodiments, the memory 1211 includes program code sections for storing program code implementable on the processor 1207. Furthermore, in some embodiments, the memory 1211 may further comprise a stored data section for storing data, e.g., data that has been processed or is to be processed according to the embodiments described herein. The implemented program code stored in the program code section and the data stored in the stored data section may be retrieved by the processor 1207 whenever needed via the memory-processor coupling.

[0236] In some embodiments, the device 1200 comprises a user interface 1205. The user interface 1205 may be coupled to a processor 1207 in some embodiments. In some embodiments, the processor 1207 may control the operation of the user interface 1205 and receive input from the user interface 1205. In some embodiments, the user interface 1205 may allow a user to input commands to the device 1200, for example, via a keypad. In some embodiments, the user interface 1205 may allow a user to obtain information from the device 1200. For example, the user interface 1205 may include a display configured to display information from the device 1200 to the user. The user interface 1205 may comprise a touch screen or touch interface in some embodiments that can both allow information to be input into the device 1200 and also display information to the user of the device 1200.

[0237] In some embodiments, the apparatus 1200 comprises an input / output port 1209. The input / output port 1209 comprises a transceiver in some embodiments. The transceiver in such embodiments may be coupled to the processor 1207 and configured to enable communication with other apparatuses or electronic devices, for example, via a wireless communication network. The transceiver or any suitable transceiver or transmitter and / or receiver means may in some embodiments be configured to communicate with other electronic devices or apparatuses via a wire or wired coupling.

[0238] The transceiver may communicate with the further device by any suitable known communication protocol, for example, in some embodiments, the transceiver may use a suitable Universal Mobile Telecommunications System (UMTS) protocol, a Wireless Local Area Network (WLAN) protocol such as IEEE 802.X, a suitable short-range radio frequency communication protocol such as Bluetooth, or an Infrared Data Path (IRDA).

[0239] The transceiver input / output port 1209 can be configured to receive signals, in some embodiments, to obtain focus parameters, as described herein.

[0240] In some embodiments, device 1200 may be used to generate appropriate audio signals by using processor 1207 executing appropriate code. Input / output port 1209 may be coupled to any suitable audio output, for example a multi-channel speaker system and / or headphones (which may be head tracked or non-tracked headphones) or the like.

[0241] In general, various embodiments of the present invention can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, and other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and illustrated as block diagrams, flow charts, or using some other pictorial representations, it should be well understood that these blocks, devices, systems, techniques, or methods contemplated herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controller, or other computing device, or some combination thereof, as non-limiting examples.

[0242] The embodiments of the present invention may be realized by computer software executable by a data processor of a mobile device, such as in a processor entity, or by computer software executable by hardware, or by a combination of software and hardware. Furthermore, it should be noted in this regard that any block of the logic flow as illustrated may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips, or memory blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media, e.g. DVDs and data variants thereof.

[0243] The memory may be of any type suitable for the local technology environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed and removable memory, etc. The data processor may be of any type suitable for the local technology environment and may include, by way of non-limiting examples, one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), gate level circuits, and processors based on multi-core processor architectures.

[0244] Embodiments of the present invention can be implemented in a variety of components, such as integrated circuit modules. The design of integrated circuits is a highly automated process and is large scale. Complex and powerful software tools are available to convert logic level designs into complete semiconductor circuit designs that are ready to be etched and formed on semiconductor substrates.

[0245] Programs such as those offered by Synopsys, Inc. of Mountain View, California and Cadence Design, San Jose, California, use well-established rules of design and a library of pre-stored design modules to automatically route conductors and identify the locations of components on a semiconductor chip. Once the design of a semiconductor circuit is complete, the resulting design in a standardized electronic format (e.g., Opus, GDSII, etc.) may be sent to a semiconductor manufacturing facility or "fab" for fabrication.

[0246] The foregoing description has provided a complete and informative description of exemplary embodiments of the present invention by way of illustrative and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the art in view of the foregoing description upon perusal of the accompanying drawings and the appended claims. However, all such similar modifications of the teachings of the present invention will still fall within the scope of the present invention as defined in the appended claims.

Claims

1. 1. An apparatus comprising: at least one processor; When executed by the at least one processor, the device includes at least: determining at least one region within an audio scene represented by a spatial audio signal; Obtaining at least one focus / defocus information; processing the spatial audio signal based at least in part on the at least one focus / defocus information to generate a processed spatial audio signal representing a modified audio scene in which the determined at least one region is de-emphasized relative to at least some other portions of the spatial audio signal; - outputting the processed spatial audio signal, wherein the modified audio scene comprises at least the determined at least one region de-emphasized with respect to at least some of the other portions of the spatial audio signal according to the at least one focus / defocus information; at least one memory storing instructions for executing the An apparatus comprising:

2. The focus / defocus information is Defocus amount, Focus amount, Defocus shape, Focus shape, Defocus direction, or Focus direction, The apparatus of claim 1 , comprising at least one of:

3. Processing the spatial audio signal, when performed by the at least one processor, causes the device to: - reducing emphasis in at least a portion of the determined at least one region relative to at least a portion of the other portions of the spatial audio signal; or increasing emphasis in the other portion of at least a portion of the spatial audio signal for the determined at least one region; 2. The apparatus of claim 1, further comprising the instructions to perform at least one of:

4. Processing the spatial audio signal, when performed by the at least one processor, causes the device to: - reducing the sound level in at least a part of at least one region determined according to the at least one focus / defocus information relative to at least a part of the other part of the spatial audio signal, or increasing a sound level in the other portion of at least a part of the spatial audio signal for the at least one determined region according to the at least one focus / defocus information; 2. The apparatus of claim 1, further comprising the instructions to perform at least one of:

5. the at least one focus / defocus information includes at least a focus / defocus shape; Processing the spatial audio signal, when performed by the at least one processor, may cause the device to perform at least: reducing emphasis in at least a portion of the at least one determined region within the focus / defocus shape relative to at least some of the other portions of the spatial audio signal; or increasing emphasis in the other portion of at least a portion of the spatial audio signal for the determined at least one region within the focus / defocus shape; The apparatus of claim 1 , further comprising instructions to cause at least one of:

6. the at least one focus / defocus information includes at least a focus / defocus shape; Processing the spatial audio signal, when performed by the at least one processor, may cause the device to perform at least: - reducing the sound level in at least a portion of the at least one determined region within the focus / defocus shape according to the at least one focus / defocus information for the at least some other portion of the spatial audio signal; or increasing a sound level in the other portion of at least a part of the spatial audio signal for the determined at least one region within the focus / defocus shape according to the at least one focus / defocus information; 2. The apparatus of claim 1, further comprising the instructions to perform at least one of:

7. the at least one focus / defocus information includes at least a focus / defocus shape; The focus / defocus shape is width of the focus / defocus feature, height of focus / defocus features, Radius of focus / defocus shape, distance of focus / defocus features, Depth of focus / defocus features, range of focus / defocus shapes, the diameter of the focus / defocus feature, or Focus / defocus shape characterizer, The apparatus of claim 1 , comprising at least one of:

8. the spatial audio signal and the processed spatial audio signal each comprise an Ambisonic signal; Processing the spatial audio signal, when performed by the at least one processor, may cause the device to, for one or more frequency subbands, at least: extracting from the spatial audio signal a single-channel target audio signal representative of sound components coming from the determined at least one region; or generating focused spatial audio signals, the focused spatial audio signals being positioned at defined spatial locations based at least in part on the at least one focus / defocus information; generating the processed spatial audio signal as a linear combination of the focused spatial audio signal subtracted from the spatial audio signal, wherein at least one of the focused spatial audio signal or the spatial audio signal is scaled by a respective scaling factor derived at least in part based on the at least one focus / defocus information to reduce the relative level of sound in the determined at least one region; The apparatus of claim 1 , further comprising the instructions to perform at least one of:

9. the spatial audio signal and the processed spatial audio signal each comprise a parametric spatial audio signal, the parametric spatial audio signal each comprising one or more audio channels and spatial metadata, the spatial metadata including directional and energy ratio parameters corresponding to a plurality of frequency subbands, and processing the spatial audio signal, when performed by the at least one processor, causes the apparatus to: - calculating, for one or more frequency sub-bands respectively, an angular difference between the determined direction of the at least one region and a direction indicated for the corresponding frequency sub-band of the spatial audio signal; deriving respective gain values ​​for the one or more frequency sub-bands based at least in part on the angular difference calculated for the corresponding frequency sub-band using a predetermined function of the angular difference and a scaling factor derived at least in part based on the at least one focus / defocus information; calculating, for one or more frequency subbands of the processed spatial audio signal, respective updated directional energy values ​​based on the energy ratio parameters and the gain values ​​of the corresponding frequency subbands of the spatial audio signal; calculating, for the one or more frequency subbands of the processed spatial audio signal, respective updated ambient energy values ​​based on the energy ratio parameter and the scaling factor of the corresponding frequency subband of the spatial audio signal; calculating a respective modified energy ratio parameter for the one or more frequency subbands of the processed spatial audio signal based at least in part on the updated directional energy value divided by the sum of the updated directional energy value and the updated ambient energy value; calculating respective spectral adjustment factors for the one or more frequency subbands of the processed spatial audio signal based at least in part on the sum of the updated directional energy value and the updated ambient energy value; constructing the processed spatial audio signal including the one or more audio channels of the spatial audio signal, the directional indication of the spatial audio signal, the modified energy ratio parameter, and the spectral adjustment coefficients; The apparatus of claim 1 , further comprising the instructions to:

10. the spatial audio signal and the processed spatial audio signal each comprise a multi-channel loudspeaker signal according to a first predetermined loudspeaker configuration, and processing the spatial audio signal, when performed by at least one processor, causes the apparatus to: - calculating a respective angular difference between the determined direction of the at least one region and the direction of a loudspeaker indicated for each channel of the spatial audio signal; deriving a respective gain value for each channel of the spatial audio signal based at least in part on the calculated angular difference for each channel using a predetermined function of the angular difference and a scaling factor derived at least in part based on the at least one focus / defocus information; deriving one or more modified audio channels by multiplying each channel of the spatial audio signal by the gain value derived for each channel; providing the one or more modified audio channels as the processed spatial audio signal; The apparatus of claim 1 , further comprising the instructions to:

11. The acquiring of the at least one focus / defocus information, when executed by the at least one processor, causes the at least one memory to obtaining the at least one focus / defocus information based at least in part on information from a sensor arrangement, the information from the sensor arrangement comprising: at least one sensor direction, or at least one user input; obtaining, 10. The apparatus of claim 1, further comprising: storing instructions for causing a

12. determining at least one region within an audio scene represented by a spatial audio signal; Obtaining at least one focus / defocus information; processing the spatial audio signal based at least in part on the at least one focus / defocus information to generate a processed spatial audio signal representing a modified audio scene in which the determined at least one region is de-emphasized relative to at least some other portions of the spatial audio signal; - outputting the processed spatial audio signal, wherein the modified audio scene comprises at least the determined at least one region de-emphasized with respect to at least some of the other portions of the spatial audio signal according to the at least one focus / defocus information; A method comprising:

13. The focus / defocus information is Defocus amount, Focus amount, Defocus shape, Focus shape, Defocus direction, or Focus direction, The method of claim 12 , comprising at least one of:

14. - reducing emphasis in at least a portion of the determined at least one region relative to at least a portion of the other portions of the spatial audio signal; or increasing emphasis in the other portion of at least a portion of the spatial audio signal for the determined at least one region; The method of claim 12 , further comprising at least one of:

15. - reducing the sound level in at least a portion of the at least one region determined according to the at least one focus / defocus information relative to at least a portion of the other portion of the spatial audio signal, or increasing a sound level in the other portion of at least a part of the spatial audio signal for the at least one determined region according to the at least one focus / defocus information; The method of claim 12 , further comprising at least one of:

16. the at least one focus / defocus information includes at least a focus / defocus shape; The focus / defocus shape is width of the focus / defocus feature, height of focus / defocus features, Radius of focus / defocus shape, distance of focus / defocus features, Depth of focus / defocus features, range of focus / defocus shapes, the diameter of the focus / defocus feature, or Focus / defocus shape characterizer, The method of claim 12 , comprising at least one of:

17. The obtaining of the at least one focus / defocus information includes: obtaining the at least one focus / defocus information based at least in part on information from a sensor arrangement, the information from the sensor arrangement comprising: at least one sensor direction, or at least one user input; obtaining, 13. The method of claim 12, comprising:

18. A non-transitory computer-readable medium comprising at least: determining at least one region within an audio scene represented by a spatial audio signal; Obtaining at least one piece of focus / defocus information; processing the spatial audio signal based at least in part on the at least one focus / defocus information to generate a processed spatial audio signal representing a modified audio scene in which the determined at least one region is de-emphasized relative to at least some other portions of the spatial audio signal; - outputting the processed spatial audio signal, wherein the modified audio scene comprises at least the determined at least one region de-emphasized with respect to at least some of the other portions of the spatial audio signal according to the at least one focus / defocus information; A non-transitory computer-readable medium containing program instructions stored thereon for carrying out the method.

19. The focus / defocus information is Defocus amount, Focus amount, Defocus shape, Focus shape, Defocus direction, or Focus direction, 20. The non-transitory computer-readable medium of claim 18, comprising at least one of:

20. - reducing emphasis in at least a portion of the determined at least one region relative to at least a portion of the other portions of the spatial audio signal; or increasing emphasis in the other portion of at least a portion of the spatial audio signal for the determined at least one region; 20. The non-transitory computer-readable medium of claim 18, further comprising program instructions stored thereon for causing at least one of:

Images