Spatial audio rendering

Abstract

Examples of the disclosure relate to spatial audio rendering that allows for 6DoF of movement of a listener. A position of a listener and audio sources corresponding to the position of the listener are determined. The audio sources comprise a first audio source and one or more second audio sources. The first audio source is used for audio signal interpolation and the second audio sources are used for spatial metadata interpolation. A first audio content set and one or more second audio content sets are received. The first audio content set comprises at least one audio signal representing the first audio source and the second audio content sets comprise spatial metadata associated with the one or more second audio sources. A spatial audio scene comprising multiple audio sources is rendered using the first audio content set for signal interpolation and the second audio content sets for spatial metadata interpolation.

Description

[0001] TITLE

[0002] Spatial Audio Rendering

[0003] TECHNOLOGICAL FIELD

[0004] Examples of the disclosure relate to spatial audio rendering. Some relate to spatial audio rendering that allows for six degrees of freedom of movement of a listener.

[0005] BACKGROUND

[0006] Audio signal sets, such as Higher Order Ambisonics (HOAs) can be used to enable spatial rendering of a listening space. This can enable a listener to perceive accurate spatial aspects of audio scenes within the listening space. Such systems can be computationally intensive and can require large bandwidth for the transmission of the audio signals.

[0007] BRIEF SUMMARY

[0008] According to various, but not necessarily all, examples of the disclosure there may be provided an apparatus for six degrees of freedom audio rendering comprising:

[0009] at least one processor; and

[0010] at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:

[0011] determining a position of a listener;

[0012] determining audio sources corresponding to the position of the listener, the determined audio sources comprise at least one first audio source and one or more second audio sources, wherein audio signal interpolation for the position of the listener is configured to use the at least one first audio source and spatial metadata interpolation for the position of the listener is configured to use the one or more second audio sources;

[0013] receiving at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source;

[0014] receiving one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources; and

[0015] rendering a spatial audio scene comprising multiple audio sources using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

[0016] The processor and memory may also be arranged to cause the apparatus to perform generating spatial metadata from the at least one audio signal representing the at least one first audio source.

[0017] The at least one first audio content set and the one or more second audio content sets may be received in the same stream.

[0018] The one or more second audio content sets may be received as at least one of:

[0019] part of a renderer payload packet; or part of a scene update packet.

[0020] The one or more second audio content sets may be provided in a different stream to the at least one first audio content set.

[0021] The processor and memory may also be arranged to cause the apparatus to perform:

[0022] determining a change in position of the listener;

[0023] determining audio sources corresponding to the new position of the listener wherein the determined audio sources comprise at least one first audio source to be used for audio signal interpolation for the new position of the listener and one or more second audio sources to be used for spatial metadata interpolation for the new position of the listener;

[0024] receiving a further first audio content set and one or more further second audio content sets based on the change in position of the listener; and

[0025] rendering a spatial audio scene comprising multiple audio sources for the new position of the listener using the at least one further first audio content set for signal interpolation and the one or more further second audio content sets for spatial metadata interpolation.

[0026] The audio sources may comprise higher order ambisonics sources.

[0027] The position of the listener may comprise a location and an orientation.

[0028] According to various, but not necessarily all, examples of the disclosure there may be provided a method comprising:

[0029] determining a position of a listener;

[0030] determining audio sources corresponding to the position of the listener, the determined audio sources comprise at least one first audio source and one or more second audio sources, wherein audio signal interpolation for the position of the listener is configured to use the at least one first audio source and spatial metadata interpolation for the position of the listener is configured to use the one or more second audio sources;

[0031] receiving at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source;

[0032] receiving one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources; and

[0033] rendering a spatial audio scene comprising multiple audio sources using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

[0034] According to various, but not necessarily all, examples of the disclosure there may be provided a computer program comprising instructions which, when executed by an apparatus, cause the apparatus to perform:

[0035] determining a position of a listener; determining audio sources corresponding to the position of the listener, the determined audio sources comprise at least one first audio source and one or more second audio sources, wherein audio signal interpolation for the position of the listener is configured to use the at least one first audio source and spatial metadata interpolation for the position of the listener is configured to use the one or more second audio sources;

[0036] receiving at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source;

[0037] receiving one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources; and

[0038] rendering a spatial audio scene comprising multiple audio sources using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

[0039] According to various, but not necessarily all, examples of the disclosure there may be provided an apparatus for providing audio content for six degrees of freedom audio rendering comprising:

[0040] at least one processor;

[0041] and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:

[0042] obtaining audio signals representing multiple audio sources wherein the audio sources represent a spatial audio scene;

[0043] generating spatial metadata for the respective audio signals; and

[0044] providing a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener such that the at least one first audio content set comprises at least one audio signal representing a first audio source to be used for audio signal interpolation for the position of the listener and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources to be used for spatial metadata interpolation for the position of the listener.

[0045] The processor and memory may also be arranged to cause the apparatus to perform providing an interface indicating an availability of audio signals and corresponding spatial metadata and enabling the interface to be accessed by the playback device.

[0046] According to various, but not necessarily all, examples of the disclosure there may be provided a method comprising:

[0047] obtaining audio signals representing multiple audio sources wherein the audio sources represent a spatial audio scene;

[0048] generating spatial metadata for the respective audio signals; and

[0049] providing a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener such that the at least one first audio content set comprises at least one audio signal representing a first audio source to be used for audio signal interpolation for the position of the listener and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources to be used for spatial metadata interpolation for the position of the listener.

[0050] According to various, but not necessarily all, examples of the disclosure there may be provided a computer program comprising instructions which, when executed by an apparatus, cause the apparatus to perform:

[0051] obtaining audio signals representing multiple audio sources wherein the audio sources represent a spatial audio scene;

[0052] generating spatial metadata for the respective audio signals; and

[0053] providing a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener such that the at least one first audio content set comprises at least one audio signal representing a first audio source to be used for audio signal interpolation for the position of the listener and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources to be used for spatial metadata interpolation for the position of the listener.

[0054] According to various, but not necessarily all, embodiments there is provided an apparatus comprising

[0055] at least one processor; and

[0056] at least one memory including computer program code;

[0057] the at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus to perform at least a part of one or more methods described herein.

[0058] According to various, but not necessarily all, embodiments there is provided an apparatus comprising means for performing at least part of one or more methods described herein. The description of a function and / or action should additionally be considered to also disclose any means suitable for performing that function and / or action. Functions and / or actions described herein can be performed in any suitable way using any suitable method.

[0059] According to various, but not necessarily all, embodiments there is provided examples as claimed in the appended claims.

[0060] While the above examples of the disclosure and optional features are described separately, it is to be understood that their provision in all possible combinations and permutations is contained within the disclosure. It is to be understood that various examples of the disclosure can comprise any or all the features described in respect of other examples of the disclosure, and vice versa. Also, it is to be appreciated that any one or more or all the features, in any combination, may be implemented by / comprised in / performable by an apparatus, a method, and / or computer program instructions as desired, and as appropriate. The description of a function should additionally be considered to also disclose any means suitable for performing that function BRIEF DESCRIPTION

[0061] Some examples will now be described with reference to the accompanying drawings in which:

[0062] FIG. 1 shows an example audio scene;

[0063] FIG. 2 shows triangulation of an audio scene;

[0064] FIGS. 3A and 3B show an example method;

[0065] FIGS. 4A and 4B shows example audio content sets;

[0066] FIG. 5 shows an example immersive audio renderer;

[0067] FIG. 6 shows an example switch module;

[0068] FIG. 7 shows an example method;

[0069] FIG. 8 shows an example system;

[0070] FIG. 9 shows an example system; and

[0071] FIG. 10 shows an example controller.

[0072] The figures are not necessarily to scale. Certain features and views of the figures can be shown schematically or exaggerated in scale in the interest of clarity and conciseness. For example, the dimensions of some elements in the figures can be exaggerated relative to other elements to aid explication. Corresponding reference numerals are used in the figures to designate corresponding features. For clarity, all reference numerals are not necessarily displayed in all figures.

[0073] DETAILED DESCRIPTION

[0074] Fig. 1 shows an example audio scene 100. The audio scene 100 comprises multiple audio sources 102. In this example the audio sources comprise Higher Order Ambisonics (HOA) sources. Other types of audio sources could be used in other examples.

[0075] The audio scene 100 in Fig. 1 comprises eight audio sources 102. Other numbers of audio sources 102 could be used in other examples of the disclosure. Each of the audio sources 102 has a position and orientation within the audio scene 100.

[0076] The audio scene 100 could represent a virtual reality environment. For example, it could be a gaming environment or any other suitable type of environment.

[0077] A listener 104 is positioned within the audio scene 100. The listener 104 can move freely within the audio scene 100. The listener 104 can move with six degrees of freedom (6DoF) within the audio scene 100. That is, the listener 104 can change both their position and orientation within the audio scene 100. The arrows 106 show an example trajectory for the listener 104 through the audio scene 100. This changes the position and / or orientation of the listener 104 relative to the audio sources 102.

[0078] In order to enable 6DoF rendering for an audio scene 100 with multiple audio sources 102 the audio data, or source signals, for all of the audio sources 102 needed for the rendering is required.

[0079] The audio sources 102 needed for the rendering for a particular position of the listener 104 can be determined using triangulation, or any other suitable process. Fig. 2 schematically shows triangulation of an audio scene 100 that can be used. In this example the listener 104 is at position pi. The microphones that capture the audio signals and provide the audio sources 102 are provided at positions m₁ to m₅. The triangulation partitions the audio scene 100 into triangular sections. The vertices of the respective triangular sections are defined by the positions of the audio sources. When a listener is positioned in a triangular section the audio sources 102 needed for the rendering are the audio sources 102 provided at the vertices of the triangular section.

[0080] For example, in Fig. 2 the listener 104 is located in a triangle section which has the first audio source 102_1, the second audio source 102_2 and the third audio source 102_3 at the respective vertices. The audio sources 102 needed for the rendering for this position of the listener 104 are therefore the first audio source 102_1, the second audio source 102_2 and the third audio source 102_3.

[0081] Bitrate reduction for transport audio signals can be achieved by only retrieving and sending the audio sources 102 needed for the rendering for the position of the listener 104. So for the example position shown in Fig. 2 only the first audio source 102_1, the second audio source 102_2 and the third audio source 102_3. However this still comprises a significant amount of data. Examples of the disclosure provide apparatus and methods that can be used to further reduce the bitrate for audio signals that are used for 6DoF audio rendering.

[0082] Figs. 3A and 3B shows example methods that can be used in examples of the disclosure. These example methods can be used to reduce the bitrate for audio signals that are used for 6DoF audio rendering.

[0083] The method of Fig. 3A can be implemented by a rendering device. This can be any suitable device that receives audio signals and renders them for playback to a listener. The method of Fig. 3B can be performed by a corresponding encoding device. The encoding device can be configured to send the audio signals to the rendering device or to otherwise make the audio signals available to the rendering device.

[0084] The example method of Fig. 3A comprises, at block 300, determining a position of a listener 104. The position of the listener 104 can be determined as a position within an audio scene. The position of the listener 104 can be determined using any suitable coordinate system. The position of the listener 104 can comprise a location and an orientation. The orientation of the listener 104 can indicate the direction in which the listener 104 is facing. The orientation can indicate any rotation of the listener's 104 head.

[0085] At block 302 the method comprises determining audio sources 102 corresponding to the position of the listener 104. The determined audio sources 102 comprise at least one first audio source 102_1 and one or more second audio sources 102_2. The audio sources 102 can comprise higher order ambisonics sources (HOAs) or any other suitable type of audio sources.

[0086] Audio signal interpolation for the position of the listener 104 is configured to use the at least one first audio source 102_1. The first audio source 102_1 could be the audio source 102 that is closest to the position of the listener 104.

[0087] Spatial metadata interpolation for the position of the listener 102 is configured to use the one or more second audio sources 102_2. The one or more second audio sources 102_2 only need to be used for spatial metadata interpolation.

[0088] The one or more second audio sources 102_2 do not need to be used for audio signal interpolation. The first audio source 102_1 can be used for both audio signal interpolation and spatial metadata interpolation. That is, spatial metadata calculated from the first audio source 102_1 can be used for spatial metadata interpolation. In some examples the method can comprise generating spatial metadata from the at least one audio signal representing the at least one first audio source 102_1. Any suitable process can be used to generate the spatial metadata.

[0089] At block 304 the method comprises receiving at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source 102_1. An audio content set can comprise data relating to a specific audio source 102. The audio content can comprise audio signals and / or spatial metadata and / or any other suitable data.

[0090] At block 306 the method comprises receiving one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources 102_2.

[0091] In examples of the disclosure the second audio content sets only need to comprise spatial metadata associated with the one or more second audio sources 102_2. The second audio content sets do not need to comprise audio signals representing the one or more second audio sources 102_2. This reduces the bitrate needed. In some examples, this reduction in bitrate for the second audio content sets can enable a higher bitrate to be used for the first audio content set. This can enable high-quality audio signals to be provided in the first audio content set.

[0092] At block 308 the method comprises rendering a spatial audio scene 100 comprising multiple audio sources 102 using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation. In the example of Fig. 3A the block of receiving the one or more second audio content sets is shown as a separate block to the receiving of the at least one first audio content set. In some examples, the one or more second audio content sets can be provided in a different stream to the at least one first audio content set.

[0093] In other examples the at least one first audio content set and the one or more second audio content sets can be received in the same stream. For example, the one or more second audio content sets can be received as part of a renderer payload packet, or as part of a scene update packet.

[0094] Examples of the disclosure can be used for spatial audio rendering that allows for 6DoF movement of a listener 104. Therefore, in some use cases a change in position of the listener 104 can be determined. When a change in position of the listener 104 is determined audio sources 102 corresponding to the new position of the listener 104 are determined. The determined audio sources 102 for the new position of the listener comprise at least one first audio source 102_1 to be used for audio signal interpolation for the new position of the listener 104 and one or more second audio sources 102_2 to be used for spatial metadata interpolation for the new position of the listener 104. Once respective audio sources 102 have been determined the rendering device can receive the audio content sets corresponding to the audio sources 102 for the new position of the listener 104. This can be a different first audio content set and one or more different second audio content sets based on the change in position of the listener.

[0095] In some examples one or more of the audio content sets might not change as the listener 104 changes position. For instance, a listener 104 could change positions so that the first audio source 102_1 does not change but one or both of the second audio sources 102_2 changes. In some examples the listener 104 could change position so that the first audio source changes but at least one of the second audio sources does not change. In some examples an audio source 102 that was previously a first audio source 102_1 could become a second audio source 102_2 after the listener 104 has moved. In some examples an audio source 102 that was previously a second audio source 102_2 could become a first audio source 102_1 after the listener 104 has moved. The changes in the audio sources 102 that are used for the respective audio content sets is determined by the trajectory of the listener 104 and the positions of the respective audio sources 102.

[0096] Once the audio content sets for the new position of the listener 104 have been received a spatial audio scene 100 comprising multiple audio sources 102 can be rendered for the new position of the listener 104 using the at least one new first audio content set for signal interpolation and the one or more new second audio content sets for spatial metadata interpolation.

[0097] Fig. 3B shows a corresponding method to the method of Fig. 3A. The method of Fig. 3B can be performed by a device, such as an encoding device, that provides the audio content sets for the rendering device. The method of Fig 3B comprises, at block 310, obtaining audio signals representing multiple audio sources 102 wherein the audio sources 102 represent a spatial audio scene 102.

[0098] At block 312 the method comprises generating spatial metadata for the respective audio signals. Any suitable process can be used to generate the spatial metadata.

[0099] At block 314 the method comprises providing a at least one first audio content set and one or more second audio content sets to a playback device. The playback device could perform the method of Fig. 3A or can comprise an apparatus that performs the methos of Fig. 3A. The first audio content set and the one or more second audio content sets are provided based on a position of a listener 104 such that the at least one first audio content set comprises at least one audio signal representing a first audio source 102_1 to be used for audio signal interpolation for the position of the listener 104 and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources 102_2 to be used for spatial metadata interpolation for the position of the listener 104.

[0100] In some examples the device that implements the method of Fig. 3B could also provide an interface indicating an availability of audio signals and corresponding spatial metadata and enabling the interface to be accessed by the playback device. The playback device could then select or request the audio content sets based on the information in the interface.

[0101] Figs. 4A and 4B schematically show example audio content sets that could be used in examples of the disclosure. Fig.

[0102] 4A shows an example first audio content set 400 and Fig. 4B shows an example second audio content set 402.

[0103] The first audio content set 400 comprises audio signals 404_1 that represent a first audio source 102_1 where the first audio source 102_1 is used for signal interpolation. The first audio content set 400 can comprise full order audio data for the first audio source 102_1. The audio source that is used for signal interpolation could be the audio source 102 that is closest to the position of the listener 104 or any other suitable audio source 102. In example of the disclosure only one first audio content set 400 needs to be provided to enable 6DoF audio rendering.

[0104] In this example the first audio content 400 set also comprises spatial metadata 406_1 associated with the first audio source 102_1. The spatial metadata 406_1 can be obtained by analyzing the audio signals 404_1 that are comprised in the first audio content set 400. The analysis of the audio signals 404_1 to obtain the spatial metadata can be performed by an encoding device or any other suitable device.

[0105] The second audio content set 402 comprises spatial metadata 406_2 that is associated with a second audio source 102_2. The second audio source 102_2 can be used for spatial metadata interpolation. In examples of the disclosure one or more second audio content sets 402 corresponding to different second audio sources 102_2 can be used to enable the spatial metadata interpolation. Spatial metadata 406_1 for the first audio source 102_1 can also be used for the spatial metadata interpolation. The second audio content set 402 does not comprise audio signals 404_2. The audio signals 404_2 that represent the second audio source 102_2 are shown greyed out in Fig. 4B to indicate that they are not sent but are analyzed to obtain the spatial metadata 406.

[0106] The analysis of the audio signals 404 to obtain the spatial metadata can be performed by an encoding device or any other suitable device.

[0107] Other formats for the respective audio content sets 400, 402 could be used in other examples. For instance, the first audio content set 400 could comprise multiple audio signals 406 and no spatial metadata. In such examples the spatial metadata could be obtained after the audio signals have been decoded.

[0108] Providing second audio content sets 402 that comprise spatial metadata 406_2 but no audio signals 404_2 can significantly reduce the bitrate needed for the transport audio signals. For instance, an audio content set that just comprises spatial metadata 406 can have a bitrate that is up to 80% lower than an audio content set that comprises audio signals.

[0109] Further, the rendering device receives the spatial metadata 406_2 for the second audio sources in the second audio content sets 402. This means that the rendering device does not need to perform analysis of the audio signals to obtain the spatial metadata 406_2. This can reduce the computational complexity and power requirements needed at the rendering device. Further the spatial metadata 406_2 for the second audio sources 102_2 can be obtained by analyzing audio signals 404 before they have been encoded and decoded. This can improve the accuracy of the spatial metadata 406_2 compared to spatial metadata that is obtained from signals that have been encoded and decoded.

[0110] Fig. 5 shows an example immersive audio renderer 500 that could be used in some examples of the disclosure. The immersive audio renderer 500 could be configured to implement the method of Fig. 3A or any other suitable methods. The immersive audio renderer 500 shown in Fig. 5 provides 6DoF Multi-Point HOA rendering of audio scenes as described in MPEG-I immersive audio specification (ISO / IEC 23090-4). The audio scenes that are rendered can comprise HOA / FOA content that can be recorded or synthesized. Modifications to this implementation and other implementations can be used in examples of the disclosure.

[0111] The immersive audio Tenderer 500 comprises a pre-processing block 502. The pre-processing block 502 receives information of the audio scene 504 and head-related impulse responses (HRIRs) 506 as inputs.

[0112] The information of the audio scene 504 can comprise the positions of the audio sources 102 within an audio scene 100. The audio sources can be HOA or FOA audio sources or any other suitable type of sources. The positions of the audio sources 102 can be denoted pi. The pre-processing block 502 can be configured to preform triangulation on the positions of the audio sources 102. The pre-processing block 502 can perform Delaunay triangulation or any other suitable type of triangulation. The triangulation that is performed by the pre-processing block 502 provides a set of triangles T1..N508 as an output. The set of triangles T1..N508 partition the audio scene 100 into triangular sections. The triangulation can be as shown in Fig. 2.

[0113] The pre-processing block 502 is also configured to convert the HRIRs 506 to frequency domain head-related transfer functions (HRTFs) 510. The pre-processing block 502 can use a short-term Fourier transform to perform the conversion. For MPEG-I Audio cases, an alias-free STFT algorithm can be used. The HRTFs 510 can be used to calculate an Ambisonics-to-binaural transform matrix MH0A2binb), for each frequency band b. The HRTFs 510 can be provided as an output of the pre-processing block 502.

[0114] The immersive audio Tenderer 500 also comprises a position pre-processing block 512. The position pre-processing block 512 is configured to determine audio sources 102 that are close to the position of the listener 104.

[0115] During rendering, for each input frame j, the position pre-processing block 512 takes as input the listener position pl514, positions of the audio sources p1..Nor other information of the audio scene 504, and the set of triangles T1..N508 that were created in the pre-processing block 502. Based on these inputs, the position pre-processing block 512 determines interpolation weights wc(i,j) 516 for the audio sources pi. The position pre-processing block 512 can also determine the audio source 102 that is closest to the listener 104.

[0116] To determine the interpolation weights wc(i,j) an active triangle TA(j) is determined. The active triangle TA(j) is the triangle of the set of triangles T1..N508 that the listener 104 is in. Barycentric coordinates for the active triangle TA(j) and the listener position are then calculated. The barycentric coordinates are used as the interpolation weights wc(i,j) 516. The interpolation weights wc(i,j) 516 sum to one and the closer the listener 104 is to an audio source 102, the higher the weight for that audio source 102 will be. At an edge of a triangle, the interpolation weight for the audio source that is not part of that edge is 0.

[0117] The interpolation weights wc(i,j) 516 and the active triangle TA(j) 518 are provided as outputs of the position preprocessing block 512.

[0118] If the listener 104 moves from one triangle to another then the position pre-processing block can switch the active triangle TA(j). In some examples the switching of the active triangle TA(j) can be delayed, and only switched after a few frames of audio. The delay of the switching can be beneficial because processing (STFT) of the audio signals representing the audio sources 102 for the new triangle takes a few frames of audio to provide meaningful output. During the delayed switch, the interpolation weights wc(i,j) are not updated until the STFT is ready. After the STFT is ready, the interpolation weights wc(i,j) are calculated for the current listener position. A crossfade can be performed between the new interpolation weights wc(i,j) and the interpolation weights wc(i,j) used during the delayed switch of the active triangle TA(j). Therefore, the active triangle TA(j) is not always the triangle that the listener 104 is in.

[0119] Any suitable process can be used to perform the cross fade. In some examples, after a delayed switch of the active triangle TA(j), the interpolation weights wc(i,j) are cross-faded over a duration of 24 frames. On the first frame, the interpolation weights wc(i,j) used during the delayed switch are used. For the second frame, the interpolation weights wc(i,j) calculated based on the current listener position are used for a small subset of the frequency bands. For the rest of the frequency bands, the interpolation weights wc(i,j) used during the delayed switch are used. For the third frame interpolation weights wc(i,j) calculated based on the current listener position are used for more frequency bands than for the previous frame. This continues until after 24 frames, the interpolation weights wc(i,j) are all obtained based on the listener position for all frequency bands.

[0120] The immersive audio Tenderer 500 also comprises a switch module 520. The switch module 520 can receive audio content sets as inputs. The audio content sets can comprise a first audio content set 400 that comprises at least one audio signal sESD(i,j) 522 or a second audio content set 402 that comprises spatial metadata 524. The switch module 520 can determine whether to perform spatial analysis based on the audio content set that is input to the switch module 520. If the audio content set that is provided is a second audio content set that comprises spatial metadata then the switch module 520 does not need to perform spatial analysis and the spatial metadata 524 can be provided as an output. If the audio content set that is provided is a first audio content set 400 that comprises an audio signal sESD(i,f) 522 then the switch module 520 can perform spatial analysis so that the spatial metadata for the input audio signal sESD(i,f) 522 can be provided as an output. Fig. 6 shows an example switch module 520 in more detail.

[0121] As shown in Fig. 6 the switch module 520 can comprise a spatial analysis block 600 and a switch 602. The switch 602 is configured so that the spatial analysis block 600 can be used if the input audio content set is a first audio content set 400 and the spatial analysis block 600 can be bypassed if the input audio content set is a second audio content set 402 or other audio content set that comprise spatial metadata.

[0122] If the input comprises a first audio content set 400 that does not comprise the spatial metadata then the input comprises the audio signal sESD(i,j) 522. The audio signal sESD(i,j) 522 is provided to the spatial analysis block 600. The spatial analysis block 600 is configured to provide spatial metadata for the audio sources 102 for the first audio content set. The spatial metadata can then be used by a spatial metadata interpolation block 528, to estimate spatial metadata parameters at the listener position.

[0123] The spatial analysis block 600 takes as input a frame (for example, 256 samples) of audio signals sESD(i,f) 522 for each audio source i and frame j. In examples of the disclosure only one first audio content set 400 that comprises an audio signal might be provided and so the spatial analysis block might only perform the spatial analysis for a single audio source 102. The spatial analysis for the other audio sources 102 in the active triangle TA(j) can be performed by the encoder or other suitable device. This reduces the computational requirements of the immersive audio Tenderer 500.

[0124] The spatial analysis block 600 performs STFT processing to obtain time-frequency domain signals S(i,j, k) 526, where k refers to a sub-frame (for example, 128 samples). The spatial analysis block 600 calculates spatial metadata from the time-frequency domain signals 526. The spatial metadata can comprise energy parameters and directional parameters. In this example the spatial metadata can comprise the energy, direction information (azimuth and elevation) and diffuseness information (direct- to- total energy ratio). The spatial metadata can be obtained as follows:

[0125] First, a signal analysis vector s(i,j, k, ft) is calculated:

[0126] Sb1(i,j, k) * 1.0

[0127] Sb2(i,j, k) * 0.5774

[0128] S J’7’ ’ Sbi3(i, j, k) * 0.5774

[0129]

[0130] Sb4(i,j, k) * 0.5774

[0131] where Sb,c(i,j, k) is the value in matrix S(i,j, / c) corresponding to channel c and frequency bin b.

[0132] From the signal analysis vector, a signal intensity vector is calculated:

[0133] s1(i,j, k, b) * s4(i,j, k, b)*

[0134] i(i,j, k, b) = Re[s1(i,j, k, b) * s2(i,j, k, b)

[0135]

[0136] . s1(i,j, k, b) * s3(i,j, k, b)]

[0137] where sc*(i, j, k, b) denotes the complex conjugate of sc(i,j, k, b).

[0138] Signal energy is then calculated as follows:

[0139] 14

[0140] e(i,j, k, b) = (1 / 4)∑sc*(i,j,k,b) * sc(i,j, k, b)

[0141]

[0142] C = 1

[0143] An average intensity vector and average energy is calculated as follows:

[0144] Nsf

[0145] e(i,j, b) = (1 / Nsf) ∑ e(i,j, k, b)

[0146] Nsf

[0147] Nsf

[0148] I(i,j, b) = (1 / Nsf) ∑ i(i,j, k, b)

[0149]

[0150] The direction data (azimuth and elevation) are calculated as follows:

[0151] θ(i,j, b) = atan2(i2(i,j, b), i1(i,j, b))

[0152] i1(i,j, b),

[0153] φ(i,j, b) = atan2(i3(i,j, b), ...)

[0154]

[0155] where in(i,j, b) is the nth element of the average intensity vector i(i, j, b), θ(i, j, b) is the azimuth and φ(i,j, b) is the elevation.

[0156] The direct-to- total energy ratio is calculated as follows:

[0157] r(i,j, b)

[0158]

[0159] e(i,j, b)

[0160] The energy for the subframes k are then obtained as follows:

[0161] e(i,j, k, b) = e(i,j, b), k ∈ 1..Nsf

[0162] Other processes for calculating the spatial metadata can be used in other examples.

[0163] If the input comprises a second audio content set 402 then the input comprises the spatial metadata 524. The spatial metadata 524 can comprise direction-of-arrival (DOA) data including an azimuth direction θ(i, j, b) and an elevation angle φ(i,j, b), a direct- to- total energy ratio r(i,j, b) and an energy for the subframes e(i,j, k, b). Other parameters could be used in other examples.

[0164] The spatial metadata 524 is not provided to the spatial analysis block 600. The spatial metadata 524 can instead pass through the switch module and be provided as an input to a spatial metadata interpolation block 528.

[0165] The spatial metadata interpolation block 528 receives the spatial metadata 524 and the interpolation weights wc(i,j) as inputs. The spatial metadata interpolation block 528 can also receive orientations of the audio sources O; 530 and orientation of the listener's head ol532 as inputs.

[0166] The spatial metadata interpolation block 528 is configured to provide an estimate of the spatial metadata for the listener position. The spatial metadata for the listener position is estimated based on the spatial metadata 524 for the audio sources 102 comprising the active triangle TA(j) and the interpolation weights wc(i,j) 516 that have been calculated in the position pre-processing block 512.

[0167] Any suitable process can be used to perform the spatial metadata interpolation. As an example process, the spatial metadata can be converted into vector form:

[0168] −sin(θ(i,j, b)) cos(φ(i,j, b)) v(i,j, b) = sin(φ(i,j, b)) r(i,j, b)

[0169]

[0170] −cos(φ(i,j, b)) cos(φ(i,j, b))

[0171] The vectors are rotated according to the orientations of the audio sources o, 530 and orientation of the listener's head Ol v(j,j, b) = Rfiead DR source i, Dv i,j, D

[0172] An interpolated spatial metadata vector is then calculated by a weighted average of spatial metadata vectors:

[0173] Ns

[0174] v(j, k,b) = ∑ wCi(j, k) v(i,j, b)

[0175]

[0176] i = l

[0177] where the interpolation weights wCi(j,k) are the barycentric coordinates calculated in the position pre-processing block 512.

[0178] The interpolated spatial metadata vector is then converted to spatial metadata parameters as follows:

[0179] Azimuth:

[0180] θ(j, k, b) = atan2(−v2(j, k, b), −v1(j, k, b))

[0181] Elevation:

[0182] φ(j, k, b) = atan2(−v2(j, k, b), √((−v1(j, k, b))2+ (−v3(j, k, b))2))

[0183]

[0184] Direct-to- total energy ratio:

[0185] r(j, k, b) = √((v1(j, k, b))2+ (v2(j, k, b))2+ (v3(j, k, b))2) Energy:

[0186] Ns

[0187] e(j, k,b) = ∑ wCi(j, k) e(i,j, k, b)

[0188]

[0189] i = l

[0190] The spatial metadata interpolation block 528 provides the interpolated spatial metadata vector 534 as an output. The spatial metadata interpolation block 528 can also provide the energy parameter e(j, k, b) 536 as an output.

[0191] The immersive audio Tenderer 500 also comprises a signal interpolation block 538. The signal interpolation block 538 provides an interpolated signal Sb,c(j, k, b) 540 for the listener position. The interpolated signal Sb,c(j, k, b) 540 is an estimate of the signal at the listener position. The interpolated signal can be used later in conjunction with the interpolated metadata 534 at the listener position to provide the final binaural output.

[0192] The interpolated signal Sb,c(j, k, b) 540 is obtained by taking the audio signal for the audio source that is closest to the listener and applying equalisation the signal. The audio signal for the audio source that is closest to the listener could be obtained in a first audio content set 400.

[0193] Sb,c(j, k, b) = Geq(j, k, b) Sb,c(mc(j), j, k)

[0194] where mc(j) is index of the audio source chosen for interpolation (closest to listener in most cases) and: e(j,k,b) \

[0195] Geq(j, k, b) = min

[0196] e(mc(j), j, k, b) + ε

[0197]

[0198] The immersive audio Tenderer 500 also comprises a mixing block 542. The mixing block 542 receives inputs comprising the interpolated spatial metadata vector 534 at the listener position and the interpolated signal Sb,c(j, k, b) 540 at the listener position. The mixing block 542 also receives the HRTFs 510, orientations of the audio sources O; 530 and orientation of the listener's head ol532 as inputs. The mixing block 542 processes the inputs to provide the binaural time-frequency domain signal 544 as an output.

[0199] The mixing block 542 can create a signal covariance matrix from the interpolated spatial metadata vector 534. The covariance matrix describes the desired (or target) spatial characteristics of the signal at the listener position. An optimal mixing algorithm can then be used to obtain a mixing matrix that when multiplied with the interpolated signal Sb,c(j, k, b) 540, provides a resulting signal that is in accordance with the desired spatial characteristics.

[0200] First, a binaural prototype signal is created from the interpolated spatial metadata vector 534

[0201] B1,1(j,k) ... B1,N(j,k)

[0202] = MHOA2bin(b) * Rsh(j) *

[0203]

[0204] Sb,w(j,1) ... Sb,w(j,Nsf) Where Rsh(j) is a rotation matrix taking into account the orientation of the listener and the orientations of the audio sources and MH0A2bin(b) is the Ambisonics to binaural matrix.

[0205] Then, a signal covariance matrix Cxis calculated for the prototype signal:

[0206] Cxnew(j,b) = S(j,b)B(j,b)H

[0207] Recursive averaging is applied to get the signal covariance matrix for frame j:

[0208] Cx(j, b) = (1 - d)Cxnew(j, b) + dCx(j - 1, b)

[0209] Where d = 0.9.

[0210] Next, a signal covariance matrix Cyis calculated from the interpolated spatial metadata at the listener position. First the direct portion of Cyis calculated:

[0211] Nsf

[0212] Cydirect(j, b) = e(j, k, b)f(J, k, b)H(b, d)HH(b, d)

[0213]

[0214] k=l

[0215] where H(b, d) refers to the HRTF value at frequency bin b, in direction d.

[0216] Secondly the diffuse portion of Cyis calculated. Nsf

[0217] Cydiffuse(j, b) = ^(1 - f(j, k, b))e(j, k, b) Cdif(Jb)

[0218]

[0219] k=l

[0220] where:

[0221] Cdif(b) = (1 / Nd) ∑ H(b,d)H*(b,d)

[0222] c

[0223]

[0224] <“'m =- -

[0225] Cy is then obtained as follows:

[0226] C

[0227]

[0228] ”ew(j, b) = Cyirect(j, b) + Cylffuse(J, b)

[0229] And recursive averaging:

[0230] Cy(j, b) = (1 - d)Cynew(j, b) + dCy(j - 1, b)

[0231] where d = 0.9.

[0232] The signal covariance matrices are then used to obtain mixing matrices which are used to obtain the binaural output as follows:

[0233] O(j, k, b) = M(j, k, b) * B(j − 1, k, b) * D(j, b)

[0234] Where D(J, b) is a decorrelated time-frequency domain signal obtained from a buffer of previous binaural signals B.

[0235] The matrices M(j, k, b) and Mr(j, k, b) can be obtained from an optimal mixing procedure. After applying the mixing matrices on the binaural prototype signal B, the result output 0 is the binaural time-frequency domain signal 544 and has the spatial characteristics of the spatial metadata at the listener position.

[0236] The binaural time-frequency domain signal 544 is provided to an output block 546. The output block takes the obtained binaural time-frequency domain signal 544 and performs inverse STFT on it to produce the final time domain binaural output signal Ss(i, j) 548.

[0237] Fig. 7 shows another example method that can be used to implement examples of the disclosure. The method can be implemented by a rendering device or an apparatus that controls a rendering device or any other suitable means.

[0238] At block 700, the method comprises receiving information indicative of audio source 102 positions. The information can indicate the positions of multiple audio sources 102 in an audio scene 100. The information can comprise audio scene metadata or any other suitable information.

[0239] At block 702 the method comprises dividing the audio scene 100 into sections based on the positions of the audio sources 102 in the audio scene 100. In some examples triangulation can be used to divide the audio scene 100 into triangular sections. Each of the triangular sections can have an audio source 102 at the respective vertices. The sections of the audio scene 100 can be used to enable 6DoF audio rendering. At block 704 information indicative of the position of the listener 104 is received. At block 706 the audio sources 102 that form the active triangle around the listener 104 are identified. The active triangle is the triangle in which the listener 104 is located or which is being treated as the triangle in which the listener 104 is located. The sources that form the active triangle are the audio sources 102 at the vertices of the active triangle.

[0240] At block 708 it is determined which of the audio sources is to be used for signal interpolation and which is to be used for spatial metadata interpolation. This determines which audio sources 102 is to be the first audio source 102_1 for which a first audio content set 400 is obtained. The first audio source 102_1 could be the audio source 102 that is closest to the listener 104 or any other suitable criteria could be used to decide the first audio source 102_1. This also determines which of the audio sources 102 is to be a second audio source 102_2 for which a second audio content set 402 is obtained. The second audio sources 102_2 could be any of the audio sources 102_1 that form the active triangle that are not first audio sources 102_1.

[0241] The corresponding audio content sets for the determined audio sources 102 can be requested and retrieved to enable the rendering of the audio scene 100. The audio content sets can be requested and retrieved from an encoding device or any other suitable device.

[0242] Once the requested audio content sets have been received they can be used, at block 710, to perform spatial metadata interpolation. The spatial metadata from the second audio content set 402 can be used for spatial metadata interpolation. Spatial metadata for the first audio source 102_1 can also be used for the spatial metadata interpolation. This spatial metadata can be obtained in the first audio content set 400 or can be obtained by analyzing the audio signals obtained in the first audio content set 400.

[0243] At block 712 signal interpolation is performed. The audio signals from the first audio content set 400 are used for the signal interpolation. The data from the second audio content sets 402 is not used for the signal interpolation.

[0244] At block 714 the audio scene 100 is rendered based on the obtained interpolated spatial metadata and signal interpolation.

[0245] Fig. 8 shows an example system 800 that can be used to implement examples of the disclosure. The system 800 comprises a server 802 and a playback device 804. The playback device 804 could be configured to implement methods such as the method of Fig. 3A and the server 802 could be configured to implement method such as the method of Fig. 3B. In this example the system 800 is an MPEG (moving picture experts group) system. Other types of system 800 could be used in other examples.

[0246] The server 802 stores information that can be used for rendering the audio scene 100. The playback device 804 can request and retrieve the information needed based on the position of the listener 104. The server 802 comprises an MPEG-I bitstream 806 for providing immersive audio. The MPEG-I bitstream 806 can be stored in a memory within, or accessible by, the server 802. The MPEG-I bitstream 806 can comprise spatial metadata 808 and audio scene information 810. The audio scene information 810 can comprise information about the position and orientation of the audio sources 102 within the audio scene 100.

[0247] The spatial metadata 808 is obtained for the multiple audio sources 102 within the audio scene. This spatial metadata 808 is stored at the server 802 and can be accessed by the playback device 804. This means that the playback device 804 does not need to determine the spatial metadata 808. This can reduce the computational requirements for the playback device 804.

[0248] The server 802 also comprises MPEG-H audio coded audio data. This can comprise audio signals that represent the respective audio sources 102.

[0249] Audio content sets that comprise the audio signals and / or spatial metadata are provided to the playback device 804 based on the position of the listener 104.

[0250] The playback device 804 comprises a media retriever 814. The media retriever 814 is configured to retrieve an MPEG-H audio bitstream 816 and an MPEG-I audio bitstream 818.

[0251] The MPEG-H audio bitstream 816 is provided to a decoder 820. The decoder 820 is configured to decode the MPEG-H audio bitstream 816 and provide the decoded data to an audio Tenderer 822. The audio Tenderer 822 can be an MPEG-I audio Tenderer 822.

[0252] The audio Tenderer 822 also receives either scene information or spatial metadata from the MPEG-I audio bitstream 818. The playback device 804 comprises a switch 824 that enables switching between the scene information or spatial metadata. Whether scene information or spatial metadata is needed for an audio source is based on whether the audio source is a first audio source 102_1 or a second audio source 102_2.

[0253] The playback device 804 also comprises a controller 826. The controller 826 can comprise information that can be used by the audio render 822 or other components of the playback device 804. An interface is provided between the controller 826 and the audio Tenderer 822 to enable information to be exchanged between the controller 826 and the audio Tenderer 822.

[0254] The information comprised in the controller 826 can comprise local scene information 828. This can comprise local updates for the audio scene 100 such as changes in the position of the audio sources 102 or any other suitable information. The controller 826 can also comprise consumption environment information 830. The consumption environment information 830 can comprise information about the listening space of the listener. This can be provided in a listening space description file (LSDF) format. The consumption environment information 830 may be obtained by the playback device 804 during rendering. For example, the consumption environment information 830 may be obtained using sensing or measurement around the playback device 804, or some other means such as a file or data entry describing the listening space acoustics.

[0255] The controller 826 can also comprise listener position information 832. The listener position information 832 can be provided to the media retriever 814 to enable the media retriever 814 to retrieve the audio signals and / or spatial metadata based on the position of the listener 104.

[0256] The audio Tenderer 822 is configured to use the decoded data and the scene information and spatial metadata to generate spatial audio signals as an audio output 834. The spatial audio signals can enable 6DoF movement of the listener 104 through an audio scene 100. The spatial audio signals can comprise binaural signals or any other suitable type of signals.

[0257] Fig. 9 shows another example system 900 that can be used to implement examples of the disclosure. The system 900 of Fig. 9 can be used with HOA audio sources.

[0258] The system 900 comprises an EIF (encoder input format) source 902 that is configured to provide a description of the audio scene in EIF format to an encoder 906.

[0259] The system 900 comprises an MPEG-I audio source 904 that is configured to provide MPEG-I audio signals to an encoder 906. This provides the scene audio data.

[0260] The encoder 906 comprises an MPEG-I encoder 916 and an MPEG-H encoder 918. The encoder 906 is configured to encode the received HOA sources and MPEG-I audio signals to generate a bitstream that is provided to an MPEG-I content server 908.

[0261] The MPEG-I encoder 916 receives the description of the audio scene from the EIF source 902 and uses this to generate the bitstream for the scene metadata. The MPEG-I encoder 916 can also receive the audio signals corresponding to the HOA sources and create the spatial metadata.

[0262] The MPEG-H encoder 918 can receive the MPEG-I audio signals from the MPEG-I audio source 904 and use these signals to create the bitstream for the audio part. The MPEG-I content server 908 is configured to store the scene metadata, the spatial metadata and the audio bitstream. The MPEG-I renderer 914 can be configured to retrieve the audio signals and spatial metadata from the MPEG-I content server 908. The MPEG-I renderer 914 can retrieve full order audio data in a first audio content set for a first HOA source and can retrieve one or more sets of spatial metadata in a second audio content set for one or more second HOA sources.

[0263] The MPEG-I renderer 914 can comprise an availability interface 912. The availability interface 912 can provide an indication of which audio content sets are available from the MPEG-I content server 908. The MPEG-I renderer 914 can use this information to select the audio content sets to be retrieved.

[0264] The audio availability interface 912 provides information regarding the respective data availability for each HOA source.

[0265]

[0266] In order to efficiently use the spatial metadata as an alternative of audio data such as audio signals, the renderer should inform the player about which audio data or spatial metadata is being used currently by the renderer. This can indicate to the player to retrieve respective audio contents sets based on the listener position. Due to the dynamic nature of the data that is required based on the listener position in the audio scene 100, the player only needs to retrieve relevant audio content sets to avoid wasteful use of the network bitrate. The audio utilization interface can therefore comprise a flag to indicate if the renderer uses spatial metadata or audio signal data for a given HOA source. Syntax No. of bits Mnemonic RendererAudioUtilizationlnterface

[0267] for (int i=0;i<activeMPHOASources.size();i++) {

[0268] spatialMetaOrAudioStream blsbf if(spatialMetaOr AudioStream){ cstring spatialMetadataOraudioStreamld; cstring processingjevel; uimsbf

[0269] for (int j=0;j<notActiveSources.s!ze();j++) {

[0270] audioStreamld; cstring

[0271]

[0272] Semantics

[0273] spatialMetadataOrAudioStream shall indicate if audio stream or spatial metadata stream is used in the audio scene for this HOA source. A value equal to 0 indicates spatial metadata is used and a value equal to 1 indicates audio signal data is used.

[0274] activeMPHOASources.size() shall indicate the number of active HOA sources utilized for 6DoF HOA rendering in the audio scene.

[0275] spatialMetadataOrAudioStreamld shall indicate the identifier of the audio stream or spatial metadata stream used in the audio scene.

[0276] process! ngjevel shall indicate the order of the HOA audio element audio data being processed in the Tenderer. The maximum value is equal to the maximum HOA order.

[0277] notActiveStreams.size() shall indicate the number of audio streams that are not being utilized for rendering the audio scene. This excludes audio streams related to 6DoF HOA rendering.

[0278] spatialStreamld shall indicate the identifier of the spatial metadata for the HOA source used in the audio scene.

[0279] Following are the bitstream updates for example implementation support for MPEG-I immersive audio: The Scene configuration packet is updated to enable the Tenderer and the player to prepare for accepting the spatial metadata instead of audio data if a HOA source is a second audio source that is used for spatial metadata interpolation but not for signal interpolation.

[0280] Syntax No. of bits Mnemonic mpegiSceneConfig()

[0281] entityCount = GetCountOrlndexQ;

[0282] for (int i = 0; i < entityCount; i++) {

[0283] integerld = GetlD();

[0284] stringld; 8..* cstring

[0285] delayBufferSize; 3 uimsbf gainCullingThreshold; 3 uimsbf enableCullingReflectionRI 1 bslbf

[0286] overrideSpeedOfSound; 1 bslbf if (overrideSpeedOfSound) {

[0287] speedOf Sound; 13 uimsbf

[0288] overrideTemperature; 1 bslbf if (overrideTemperature) {

[0289] temperature; 5 uimsbf

[0290] overrideHumidity; 1 bslbf if (overrideHumidity) {

[0291] humidity; 4 uimsbf

[0292] isSmallScene; 1 bslbf FreqGridData()

[0293] hasEarlyTuningGain; 1 bslbf if (hasEarlyTuningGain) {

[0294] earlyTuningGain; 6 uimsbf

[0295] useLowComplexityER; 1 bslbf hasHeadphoneEQ; 1 bslbf

[0296]

[0297] Syntax No. of bits Mnemonic if (hasHeadphoneEQ) {

[0298] readHeadphoneEQQ;

[0299] }

[0300] hasLoudnessCullingThreshold; 1 bslbf if (hasLoudnessCullingThreshold) {

[0301] loudnessCullingThreshold; 6 Uimsbf

[0302] hasConsolidationThreshold; 1 bslbf if (hasConsolidationThreshold) {

[0303] consolidationThreshold; 5 uimsbf

[0304] enableLocallyCapturedAudio 1 bslbf enableSpatialMetadataModeForMPHOA 1 bslbf if (enableLocallyCapturedAudio) {

[0305] overrideListenerVoiceDirectivity; 1 bslbf

[0306] if (overrideListenerVoiceDirectivity) {

[0307] listenerVoiceDirectivityld = GetlDQ;

[0308]

[0309] enableSpatialMetadataModeForMPHOA Flag indicating the availability of spatial metadata as payload for rendering MPHOA (multi point HOA). In this mode, spatial metadata can be used as an alternative for HOA source audio data when the relevant HOA source is used for spatial metadata interpolation but not for signal interpolation. This can be used when MPHOA rendering is performed according to the clause 6.6.26 is ISO / IEC 23090-4 DIS.

[0310] The availability of the spatial metadata as part of the MPEG-I immersive audio bitstream can be indicated as described in the following update to the HOAGroups data structure in the scene plus payload of the ISO / IEC 23090-4.

[0311] Syntax No. of bits Mnemonic hoaGroupsQ

[0312] hoaGroupsCount = GetCountOrlndex();

[0313] for (int i = 0; i < hoaGroupsCount; i++) {

[0314] hoaGroupid = GetlDQ;

[0315]

[0316] Syntax No. of bits Mnemonic

[0317] hoaGroupHasRegion; 1 bslbf

[0318] if (hoaGroupHasRegion) {

[0319] hoaGroupRegionld = GetlD();

[0320] coSourceCount = GetCountOrlndexQ;

[0321] for (int j = 0; j < coSourceCount; j++) {

[0322] coSourceld = GetlD();

[0323] FreqBandConfig();

[0324] exteriorRenderingProjectionRadius

[0325] GetDistance(isSmallScene);

[0326] spatialMetadataModeEnabled; 1 bslbf hoaGroupHasinformedSources; 1 bslbf

[0327] if (hoaGroupHasinformedSources == True) {

[0328] informedSourceCount = GetCountOrlndexQ;

[0329] maxSimulInformedSources = GetCountOrlndexQ;

[0330] for (j = 0; j < informedSourceCount; j++) {

[0331] InformedSourcelnfoStructQ

[0332] adaptiveExteriorRenderingProjectionRadius; 1 bslbf

[0333] hoaGroupHasLowProfileConfig; 1 bslbf

[0334] if (hoaGroupHasLowProfileConfig == True) {

[0335] LowProfileConfigQ

[0336]

[0337] The spatial metadata frames for performing 6D0F HOA rendering can be included in the MHAS packet with MHASPacketType PACTYP_MPEGI_PLD or PACTYP_MPEGI_UPD. Each of the possible inclusions are described below. Syntax of mpegiSceneUpdate included in the PACTYP_MPEGI_UPD

[0338] Syntax No. of bits Mnemonic mpegiSceneUpdateQ

[0339] updatesCount = GetCountOrlndexQ;

[0340] for (int i = 0; i < updatesCount; i++) {

[0341] modificationsCount = GetCountOrlndexQ;

[0342] for (int j = 0; j < modificationsCount; j++) {

[0343] targetld = GetlDQ;

[0344] hasDuration; 1 bslbf if (hasDuration) {

[0345] Duration

[0346] GetDuration(isLongRangeMode=True);

[0347] changesCount; 5 uimsbf for (int k = 0; k < changesCount; k++) {

[0348] targetAttribute; 6 uimsbf switch (targetAttribute) {

[0349] case 0: {

[0350] for (int 1 = 0; 1 < 3; I++) {

[0351] 1 bslbf isPositionParameterVariable;

[0352] if

[0353] (isPositionParameterVariable) {

[0354] 4 uimsbf positionParameterVariablelndex;

[0355] else {

[0356] newPositionValue =

[0357] GetDistance(isSmallScene); 1 bslbf isNegative;

[0358] if (isNegative) {

[0359]

[0360] Syntax No. of bits Mnemonic

[0361] newPositionValue =

[0362] -newPositionValue

[0363] break;

[0364] case 1: {

[0365] for (int I = 0; I < 3; I++) {

[0366] 1 bslbf isOrientationParameterVariable;

[0367] if

[0368] (isOrientationParameterVariable) {

[0369] 4 uimsbf orientationParameterVariablelndex;

[0370] else {

[0371] 32 float newOrientationValue;

[0372] break;

[0373] case 2: {

[0374] newCoordSpaceValue; 1 bslbf break;

[0375] case 3: {

[0376] newActiveValue; 1 bslbf break

[0377] case 4: {

[0378] isGainDbParameterVariable; 1 bslbf if (isGainDbParameterVariable) {

[0379]

[0380] Syntax No. of bits Mnemonic 4 uimsbf gainDbParameterVariablelndex;

[0381] else {

[0382] newGainValue

[0383] GetGain(isHiPrecGain=True)

[0384] break;

[0385] case 5: {

[0386] newSignalld = GetlDQ;

[0387] break;

[0388] case 6: {

[0389] newExtentld = GetlDQ;

[0390] break;

[0391] case 7: {

[0392] newDirectivity Id = GetlDQ;

[0393] break;

[0394] case 8: {

[0395] newDirectivenessValue; 8 uimsbf break;

[0396] case 9: {

[0397] newPlayValue; 1 bslbf break;

[0398] case 10:{

[0399] newGroupId = GetlDQ;

[0400] break;

[0401] case 11: {

[0402] newRegionld = GetlDQ;

[0403] break;

[0404]

[0405]

[0406]

[0407]

[0408] Syntax No. of bits Mnemonic 4 uimsbf toleranceParameterVariablelndex;

[0409] else {

[0410] var vlclbf newToleranceParameterValue =

[0411] LUT(decimalCode);

[0412] break;

[0413] case 23: {

[0414] numjrames

[0415] GetCountOrlndexQ;

[0416] num_hoa_sources = GetCountOrIndex();

[0417] for(k=0;k<num_hoa_sources;k++){)

[0418] hoa_source_id = GetID();

[0419] for(int i = 0; i < num_frames; i++){

[0420] num_hoa_sources = GetCountOrIndex();

[0421] for(j = 0; j < num_hoa_sources; j++){;

[0422] spatial_metadata_frame_len = GetCountOrIndex();

[0423] SpatialMetadataFrame();

[0424] break;

[0425] updateType; uimsbf switch (updateType) {

[0426] case 0: {

[0427] timedUpdateId = GetID();

[0428] timedUpdateHasIndex; bslbf if (timedUpdateHasindex) {

[0429] timedUpdateIndex = GetCountOrIndex();

[0430]

[0431] Syntax No. of bits Mnemonic Time = GetDuration(isLongRangeMode=True);

[0432] break;

[0433] case 1: {

[0434] conditionalUpdateId = GetID();

[0435] conditionalHasindex; 1 bslbf

[0436] if (conditionalHasindex) {

[0437] conditionalUpdatelndex

[0438] GetCountOrlndexQ;

[0439] fireOn; 1 bslbf conditionalHasDelay; 1 bslbf

[0440] if (conditionalHasDelay) {

[0441] conditionalDelay =

[0442] GetDuration(isLongRangeMode=True);

[0443] conditionUpdateId = GetID();

[0444] break;

[0445] case 2: {

[0446] triggeredUpdateId = GetID();

[0447] triggeredUpdateIndex = GetCountOrIndex();

[0448] break;

[0449] case 3: {

[0450] dynamicUpdateId = GetID();

[0451] dynamicUpdateIndex = GetCountOrIndex();

[0452] break;

[0453]

[0454] num_frames spatial metadata for the number of rendering frames included in this MHAS packet.

[0455] num_hoa_sources spatial metadata for the number of HOA sources included in this MHAS packet.

[0456] num_hoa_sources HOA source identifier for which the spatial metadata is included in this MHAS packet. spatial_metadata_frame_len size in bytes for the spatial metadata frame.

[0457] SpatialMetadataFrame() the spatial metadata for the number of HOA sources included in this MHAS packet.

[0458] In some examples of the disclosure, the spatial metadata can be delivered as part of the payload packet (PACTYP_MPEGI_PLD). Another option is to deliver the spatial metadata as part of a new MHAS packet called PACTYP_MPEGI_PLD_UPD, this delivers rendering metadata while rendering is ongoing.

[0459] In some examples of the disclosure, the spatial metadata can be delivered as a new MHAS packet and ingested as a separate stream, analogous to audio stream in the MPEG-I immersive audio standard. However, in examples of the disclosure this stream does not comprise only audio signal data but also carries the spatial metadata required for rendering of HOA with six degrees of freedom.

[0460] Fig. 10 shows an example controller 1000. The controller 1000 could be provided within a Tenderer or any other suitable entity. Implementation of the controller 1000 may be as controller circuitry. The controller 1000 may be implemented in hardware alone, have certain aspects in software including firmware alone or can be a combination of hardware and software (including firmware). The controller 1000 can provide an apparatus for implementing the disclosure or could be provided as part of an apparatus that implements the disclosure.

[0461] As illustrated in Fig. 10 the controller 1000 can be implemented using instructions that enable hardware functionality, for example, by using executable instructions of a computer program 1006 in a general-purpose or special-purpose processor 1002 that may be stored on a computer readable storage medium (disk, memory etc.) to be executed by such a processor 1002.

[0462] The processor 1002 is configured to read from and write to the memory 1004. The processor 1002 may also comprise an output interface via which data and / or commands are output by the processor 1002 and an input interface via which data and / or commands are input to the processor 1002.

[0463] The memory 1004 stores a computer program 1006 comprising computer program instructions (computer program code) that controls the operation of the apparatus when loaded into the processor 1002. The computer program instructions, of the computer program 1006, provide the logic and routines that enables the apparatus to perform the methods illustrated in the Figs. The processor 1002 by reading the memory 1004 is able to load and execute the computer program 1006.

[0464] In some examples where the controller 1000 is provided within an apparatus, the controller therefore comprises means for:

[0465] determining 300 a position of a listener 104;

[0466] determining 302 audio sources 102 corresponding to the position of the listener 104, the determined audio sources comprise at least one first audio source 102_1 and one or more second audio sources 102_2, wherein audio signal interpolation for the position of the listener 104 is configured to use the at least one first audio source 102_1 and spatial metadata interpolation for the position of the listener 104 is configured to use the one or more second audio sources 102_2;

[0467] receiving 304 at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source 102_1;

[0468] receiving 306 one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources 102_2; and

[0469] rendering 308 a spatial audio scene 100 comprising multiple audio sources 102 using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

[0470] In some examples where the controller 1000 is provided within an apparatus, the controller therefore comprises means for:

[0471] obtaining 310 audio signals representing multiple audio sources 102 wherein the audio sources 102 represent a spatial audio scene 100;

[0472] generating 312 spatial metadata for the respective audio signals; and

[0473] providing 314 a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener 104 such that the at least one first audio content set comprises at least one audio signal representing a first audio source 102_1 to be used for audio signal interpolation for the position of the listener 104 and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources 102_2 to be used for spatial metadata interpolation for the position of the listener 104.

[0474] The computer program 1006 may arrive at the apparatus via any suitable delivery mechanism 1008. The delivery mechanism 1008 may be, for example, a machine-readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a memory device, a record medium such as a Compact Disc Read-Only Memory (CD-ROM) or a Digital Versatile Disc (DVD) or a solid-state memory, an article of manufacture that comprises or tangibly embodies the computer program 1006. The delivery mechanism may be a signal configured to reliably transfer the computer program 1006. The apparatus may propagate or transmit the computer program 1006 as a computer data signal.

[0475] The computer program 1006 can comprise computer program instructions for causing an apparatus to perform at least the following or for performing at least the following:

[0476] determining 300 a position of a listener 104;

[0477] determining 302 audio sources 102 corresponding to the position of the listener 104, the determined audio sources comprise at least one first audio source 102_1 and one or more second audio sources 102_2, wherein audio signal interpolation for the position of the listener 104 is configured to use the at least one first audio source 102_1 and spatial metadata interpolation for the position of the listener 104 is configured to use the one or more second audio sources 102_2;

[0478] receiving 304 at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source 102_1;

[0479] receiving 306 one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources 102_2; and

[0480] rendering 308 a spatial audio scene 100 comprising multiple audio sources 102 using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

[0481] The computer program 1006 can comprise computer program instructions for causing an apparatus to perform at least the following or for performing at least the following:

[0482] obtaining 310 audio signals representing multiple audio sources 102 wherein the audio sources 102 represent a spatial audio scene 100;

[0483] generating 312 spatial metadata for the respective audio signals; and

[0484] providing 314 a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener 104 such that the at least one first audio content set comprises at least one audio signal representing a first audio source 102_1 to be used for audio signal interpolation for the position of the listener 104 and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources 102_2 to be used for spatial metadata interpolation for the position of the listener 104.

[0485] The computer program instructions may be comprised in a computer program, a non-transitory computer readable medium, a computer program product, a machine-readable medium. In some but not necessarily all examples, the computer program instructions may be distributed over more than one computer program.

[0486] Although the memory 1004 is illustrated as a single component / circuitry it may be implemented as one or more separate components / circuitry some or all of which may be integrated / removable and / or may provide permanent / semi-permanent / dynamic / cached storage.

[0487] Although the processor 1002 is illustrated as a single component / circuitry it may be implemented as one or more separate components / circuitry some or all of which may be integrated / removable. The processor 1002 may be a single core or multi-core processor.

[0488] References to "computer-readable storage medium”, "computer program product”, "tangibly embodied computer program” etc. or a "controller”, "computer”, "processor” etc. should be understood to encompass not only computers having different architectures such as single / multi- processor architectures and sequential (Von Neumann) / parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGA), application specific circuits (ASIC), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device whether instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device etc.

[0489] As used in this application, the term "circuitry” can refer to one or more or all of the following:

[0490] (a) hardware-only circuitry implementations (such as implementations in only analog and / or digital circuitry) and

[0491] (b) combinations of hardware circuits and software, such as (as applicable):

[0492] (i) a combination of analog and / or digital hardware circuit(s) with software / fi rmware and

[0493] (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions and

[0494] (c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g. firmware) for operation, but the software might not be present when it is not needed for operation.

[0495] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit for a mobile device or a similar integrated circuit in a server, a cellular network device, or other computing or network device.

[0496] The blocks illustrated in the Figs, can represent steps in a method and / or sections of code in the computer program 1006. The illustration of a particular order to the blocks does not necessarily imply that there is a required or preferred order for the blocks and the order and arrangement of the block can be varied. Furthermore, it can be possible for some blocks to be omitted.

[0497] The above-described examples find application as enabling components of:

[0498] automotive systems; telecommunication systems; electronic systems including consumer electronic products; distributed computing systems; media systems for generating or rendering media content including audio, visual and audio visual content and mixed, mediated, virtual and / or augmented reality; personal systems including personal health systems or personal fitness systems; navigation systems; user interfaces also known as human machine interfaces; networks including cellular, non-cellular, and optical networks; ad-hoc networks; the internet; the internet of things; virtualized networks; and related software and services.

[0499] The apparatus can be provided in an electronic device, for example, a mobile terminal, according to an example of the present disclosure. It should be understood, however, that a mobile terminal is merely illustrative of an electronic device that would benefit from examples of implementations of the present disclosure and, therefore, should not be taken to limit the scope of the present disclosure to the same. While in certain implementation examples, the apparatus can be provided in a mobile terminal, other types of electronic devices, such as, but not limited to: mobile communication devices, hand portable electronic devices, wearable computing devices, portable digital assistants (PDAs), pagers, mobile computers, desktop computers, televisions, gaming devices, laptop computers, cameras, video recorders, GPS devices and other types of electronic systems, can readily employ examples of the present disclosure. Furthermore, devices can readily employ examples of the present disclosure regardless of their intent to provide mobility.

[0500] The term 'comprise' is used in this document with an inclusive not an exclusive meaning. That is any reference to X comprising Y indicates that X may comprise only one Y or may comprise more than one Y. If it is intended to use 'comprise' with an exclusive meaning then it will be made clear in the context by referring to ‘comprising only one...' or by using 'consisting.'

[0501] In this description, the wording 'connect', 'couple' and 'communication' and their derivatives mean operationally connected / coupled / in communication. It should be appreciated that any number or combination of intervening components can exist (including no intervening components), i.e., to provide direct or indirect connection / coupling / communication. Any such intervening components can include hardware and / or software components.

[0502] As used herein, the term "determine / determining" (and grammatical variants thereof) can include, not least: calculating, computing, processing, deriving, measuring, investigating, identifying, looking up (for example, looking up in a table, a database, or another data structure), ascertaining and the like. Also, "determining" can include receiving (for example, receiving information), accessing (for example, accessing data in a memory), obtaining and the like. Also, " determine / determining" can include resolving, selecting, choosing, establishing, and the like.

[0503] In this description, reference has been made to various examples. The description of features or functions in relation to an example indicates that those features or functions are present in that example. The use of the term 'example' or ‘for example' or 'can' or 'may' in the text denotes, whether explicitly stated or not, that such features or functions are present in at least the described example, whether described as an example or not, and that they can be, but are not necessarily, present in some of or all other examples. Thus 'example', ‘for example', 'can', or 'may' refers to a particular instance in a class of examples. A property of the instance can be a property of only that instance or a property of the class or a property of a sub-class of the class that includes some but not all the instances in the class. It is therefore implicitly disclosed that a feature described with reference to one example but not with reference to another example, can where possible be used in that other example as part of a working combination but does not necessarily have to be used in that other example. As used herein, "at least one of the following: ” and "at least one of ” and similar wording, where the list of two or more elements are joined by "and” or "or” mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0504] Although examples have been described in the preceding paragraphs with reference to various examples, it should be appreciated that modifications to the examples given can be made without departing from the scope of the claims.

[0505] Features described in the preceding description may be used in combinations other than the combinations explicitly described above.

[0506] Although functions have been described with reference to certain features, those functions may be performable by other features whether described or not.

[0507] The description of a feature, such as an apparatus or a component of an apparatus, configured to perform a function, or for performing a function, should additionally be considered to also disclose a method of performing that function. For example, description of an apparatus configured to perform one or more actions, or for performing one or more actions, should additionally be considered to disclose a method of performing those one or more actions with or without the apparatus.

[0508] Although features have been described with reference to certain examples, those features may also be present in other examples whether described or not.

[0509] The term 'a', 'an' or 'the' is used in this document with an inclusive not an exclusive meaning. That is any reference to X comprising a / an / the Y indicates that X may comprise only one Y or may comprise more than one Y unless the context clearly indicates the contrary. If it is intended to use 'a', 'an' or 'the' with an exclusive meaning then it will be made clear in the context. In some circumstances the use of ‘at least one' or ‘one or more' may be used to emphasis an inclusive meaning but the absence of these terms should not be taken to infer any exclusive meaning.

[0510] The presence of a feature (or combination of features) in a claim is a reference to that feature or (combination of features) itself and to features that achieve substantially the same technical effect (equivalent features). The equivalent features include, for example, features that are variants and achieve substantially the same result in substantially the same way. The equivalent features include, for example, features that perform substantially the same function, in substantially the same way to achieve substantially the same result.

[0511] In this description, reference has been made to various examples using adjectives or adjectival phrases to describe characteristics of the examples. Such a description of a characteristic in relation to an example indicates that the characteristic is present in some examples exactly as described and is present in other examples substantially as described. The above description describes some examples of the present disclosure however those of ordinary skill in the art will be aware of possible alternative structures and method features which offer equivalent functionality to the specific examples of such structures and features described herein above and which for the sake of brevity and clarity have been omitted from the above description. Nonetheless, the above description should be read as implicitly including reference to such alternative structures and method features which provide equivalent functionality unless such alternative structures or method features are explicitly excluded in the above description of the examples of the present disclosure.

[0512] Whilst endeavoring in the foregoing specification to draw attention to those features believed to be of importance the Applicant may seek protection via the claims in respect of any patentable feature or combination of features hereinbefore referred to and / or shown in the drawings whether or not emphasis has been placed thereon.

[0513] l / we claim:

Claims

CLAIMS1. An apparatus for six degrees of freedom audio rendering comprising:at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:determining a position of a listener;determining audio sources corresponding to the position of the listener, the determined audio sources comprise at least one first audio source and one or more second audio sources, wherein audio signal interpolation for the position of the listener is configured to use the at least one first audio source and spatial metadata interpolation for the position of the listener is configured to use the one or more second audio sources;receiving at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source;receiving one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources; andrendering a spatial audio scene comprising multiple audio sources using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

2. An apparatus as claimed in claim 1, wherein the processor and memory are also arranged to cause the apparatus to perform generating spatial metadata from the at least one audio signal representing the at least one first audio source.

3. An apparatus as claimed in any preceding claim, wherein the at least one first audio content set and the one or more second audio content sets are received in the same stream.

4. An apparatus as claimed in claim 3, wherein the one or more second audio content sets are received as at least one of:part of a renderer payload packet; orpart of a scene update packet.

5. An apparatus as claimed in any of claims 1 to 2, wherein the one or more second audio content sets are provided in a different stream to the at least one first audio content set.

6. An apparatus as claimed in any preceding claim, wherein the processor and memory are also arranged to cause the apparatus to perform:determining a change in position of the listener;determining audio sources corresponding to the new position of the listener wherein the determined audio sources comprise at least one first audio source to be used for audio signal interpolation for the new position of the listener and one or more second audio sources to be used for spatial metadata interpolation for the new position of the listener;receiving a further first audio content set and one or more further second audio content sets based on the change in position of the listener; andrendering a spatial audio scene comprising multiple audio sources for the new position of the listener using the at least one further first audio content set for signal interpolation and the one or more further second audio content sets for spatial metadata interpolation.

7. An apparatus as claimed in any preceding claim, wherein the audio sources comprise higher order ambisonics sources.

8. An apparatus as claimed in any preceding claim, wherein the position of the listener comprises a location and an orientation.

9. A method comprising:determining a position of a listener;determining audio sources corresponding to the position of the listener, the determined audio sources comprise at least one first audio source and one or more second audio sources, wherein audio signal interpolation for the position of the listener is configured to use the at least one first audio source and spatial metadata interpolation for the position of the listener is configured to use the one or more second audio sources;receiving at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source;receiving one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources; andrendering a spatial audio scene comprising multiple audio sources using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

10. A method as claimed in claim 9, further comprises generating spatial metadata from the at least one audio signal representing the at least one first audio source.

11. A method as claimed in claim 9, wherein the at least one first audio content set and the one or more second audio content sets are received in the same stream.

12. A method as claimed in claim 11, wherein the one or more second audio content sets are received as at least one of:part of a renderer payload packet; orpart of a scene update packet.

13. A method as claimed in any of claim 9 or 10, wherein the one or more second audio content sets are provided in a different stream to the at least one first audio content set.

14. A method as claimed in any of claims 9 to 13, wherein the processor and memory are also arranged to cause the apparatus to perform:determining a change in position of the listener;determining audio sources corresponding to the new position of the listener wherein the determined audio sources comprise at least one first audio source to be used for audio signal interpolation for the new position of the listener and one or more second audio sources to be used for spatial metadata interpolation for the new position of the listener;receiving a further first audio content set and one or more further second audio content sets based on the change in position of the listener; andrendering a spatial audio scene comprising multiple audio sources for the new position of the listener using the at least one further first audio content set for signal interpolation and the one or more further second audio content sets for spatial metadata interpolation.

15. A method as claimed in any of claims 9 to 14, wherein the audio sources comprise higher order ambisonics sources.

16. A method as claimed in any of claims 9 to 15, wherein the position of the listener comprises a location and an orientation.

17. A computer program comprising instructions which, when executed by an apparatus, cause the apparatus to perform:determining a position of a listener;determining audio sources corresponding to the position of the listener, the determined audio sources comprise at least one first audio source and one or more second audio sources, wherein audio signal interpolation for the position of the listener is configured to use the at least one first audio source and spatial metadata interpolation for the position of the listener is configured to use the one or more second audio sources;receiving at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source;receiving one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources; andrendering a spatial audio scene comprising multiple audio sources using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

18. An apparatus for providing audio content for six degrees of freedom audio rendering comprising:at least one processor;and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:obtaining audio signals representing multiple audio sources wherein the audio sources represent a spatial audio scene;generating spatial metadata for the respective audio signals; andproviding a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener such that the at least one first audio content set comprises at least one audio signal representing a first audio source to be used for audio signal interpolation for the position of the listener and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources to be used for spatial metadata interpolation for the position of the listener.

19. A method comprising:obtaining audio signals representing multiple audio sources wherein the audio sources represent a spatial audio scene;generating spatial metadata for the respective audio signals; andproviding a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener such that the at least one first audio content set comprises at least one audio signal representing a first audio source to be used for audio signal interpolation for the position of the listener and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources to be used for spatial metadata interpolation for the position of the listener.

20. A computer program comprising instructions which, when executed by an apparatus, cause the apparatus to perform:obtaining audio signals representing multiple audio sources wherein the audio sources represent a spatial audio scene;generating spatial metadata for the respective audio signals; andproviding a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener such that the at least one first audio content set comprises at least one audio signal representing a first audio source to be used for audio signal interpolation for the position of the listener and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources to be used for spatial metadata interpolation for the position of the listener.

21. An apparatus for six degrees of freedom audio rendering comprising means for:determining a position of a listener;determining audio sources corresponding to the position of the listener, the determined audio sources comprise at least one first audio source and one or more second audio sources, wherein audio signal interpolation for the position of the listener is configured to use the at least one first audio source and spatial metadata interpolation for the position of the listener is configured to use the one or more second audio sources;receiving at least one first audio content set wherein the first audio content set comprises at least one audio signal representing the at least one first audio source;receiving one or more second audio content sets wherein the second audio content sets comprise spatial metadata associated with the one or more second audio sources; andrendering a spatial audio scene comprising multiple audio sources using the at least one first audio content set at least for signal interpolation and the one or more second audio content sets for spatial metadata interpolation.

22. An apparatus for providing audio content for six degrees of freedom audio rendering comprising means for:obtaining audio signals representing multiple audio sources wherein the audio sources represent a spatial audio scene;generating spatial metadata for the respective audio signals; andproviding a at least one first audio content set and one or more second audio content sets to a playback device wherein the first audio content set and the one or more second audio content sets are provided based on a position of a listener such that the at least one first audio content set comprises at least one audio signal representing a first audio source to be used for audio signal interpolation for the position of the listener and the one or more second audio content sets comprise spatial metadata associated with one or more second audio sources to be used for spatial metadata interpolation for the position of the listener.

Images