Parametric spatial audio decoding with pass-through mode

Abstract

A system for decoding Metadata Assisted Spatial Audio (MASA) signals in a pass-through coding mode 140 obtains an audio metadata signal 116 which encodes channel signals 102 and object information 104

Description

Field The present application relates to apparatus and methods for spatial audio decoding, but not exclusively for an audio decoder. Background Parametric spatial audio processing is a field of audio signal processing where the spatial aspect of the sound is described using a set of parameters. For example, in parametric spatial audio capture from microphone arrays, it is a typical and an effective choice to estimate from the microphone array signals a set of parameters such as directions of the sound in frequency bands, and the ratios between the directional and non-directional parts of the captured sound in frequency bands. These parameters are known to well describe the perceptual spatial properties of the captured sound at the position of the microphone array. These parameters can be utilized in synthesis of the spatial sound accordingly, for headphones binaurally, for loudspeakers, or to other formats, such as Ambisonics. The directions and direct-to-total energy ratios in frequency bands are thus a parameterization that is particularly effective for spatial audio capture. A parameter set consisting of a direction parameter in frequency bands and an energy ratio parameter in frequency bands (indicating the directionality of the sound) can be also utilized as the spatial metadata (which may also include other parameters such as surround coherence, spread coherence, number of directions, distance etc) for an audio codec. For example, these parameters can be estimated from microphone-array captured audio signals, and for example a stereo or mono signal can be generated from the microphone array signals to be conveyed with the spatial metadata. The stereo signal could be encoded, for example, with an AAC encoder and the mono signal could be encoded with an EVS encoder. A decoder can decode the audio signals into PCM signals and process the sound in frequency bands (using the spatial metadata) to obtain the spatial output, for example a binaural output. Immersive audio codecs are being implemented supporting a multitude of operating points ranging from a low bit rate operation to transparency. An example of such a codec is the Immersive Voice and Audio Services (IVAS) codec which is being designed to be suitable for use over a communications network such as a 3GPP 4G / 5G network including use in such immersive services as for example immersive voice and audio for virtual reality (VR). This audio codec is expected to handle the encoding, decoding and rendering of speech, music and generic audio. It is furthermore expected to support channel-based audio and scene-based audio inputs including spatial information about the sound field and sound sources. The codec is also expected to operate with low latency to enable conversational services as well as support high error robustness under various transmission conditions. The aforementioned immersive audio codecs are particularly suitable for encoding captured spatial sound from microphone arrays (e.g., in mobile phones, VR cameras, stand-alone microphone arrays). However, such an encoder can have other input types, for example, loudspeaker signals, audio object signals, Ambisonic signals. Summary According to a first aspect there an apparatus for decoding an encoded spatial audio signal in a pass-through coding mode, wherein the encoded spatial audio signal comprises an encoded audio metadata signal, encoded audio channel signals and encoded information relating to a plurality of audio objects, wherein the apparatus comprises means configured to: decode the encoded spatial audio signal to produce audio metadata in a predefined format and audio channel signals; and decode information relating to the plurality of audio objects to produce a plurality of audio objects. The means configured to decode information relating to the plurality of audio objects to produce a plurality of audio objects may comprise means configured to: determine a number of the plurality of audio objects from the encoded audio metadata signal or from a frame header; and generate the number of null audio objects as a substitute for the plurality of audio objects; The means configured to decode information relating to the plurality of audio objects to produce a plurality of audio objects may further comprise means configured to: decode an encoded separate audio object to give a separate audio object; convert the separate audio object to separate audio object metadata in the predefined format and a separate audio object channel signal; combine the separate audio object metadata in the predefined format with the audio signal metadata in the predefined format to produce combined metadata in the predefined format; and combine the audio channel signals and the separate audio object channel signal. The means configured to decode information relating to the plurality of audio objects to produce a plurality of audio objects may further comprise means configured to: decode encoded metadata for a set of audio objects to produce energy ratios and metadata for the set of audio objects; convert the metadata for the set of audio objects to metadata in the predefined format for the set of audio objects; calculate an energy contribution of the set of audio objects from the metadata in the predefined format for the set of audio objects; calculate, based on the energy ratios, an energy contribution of the audio signal metadata in the predefined format and the audio channel signals; combine the audio signal metadata in the predefined format with the metadata in the predefined format for the set of audio objects to produce combined metadata in the predefined format, wherein the combining is based on the energy contribution of the audio signal metadata in the predefined format and the audio channel signals and the energy contribution of the set of audio objects; decode an encoded separate audio object to give a separate audio object; convert the separate audio object to separate audio object metadata in the predefined format and a separate audio object channel signal; combine the separate audio object metadata in the predefined format with the combined metadata in the predefined format to produce further combined metadata in the predefined format; and combine the channel audio signals and the separate audio object channel signal. A null audio object may comprise: an audio object channel signal having zero sample values; and an audio object direction which is a predetermined fixed value. The means configured to decode information relating to the plurality of audio objects to produce a plurality of audio objects may comprise means configured to: decode encoded metadata for a set of audio objects to produce metadata for the set of audio objects, audio object energy ratios and audio signal energy ratios; separate further audio channel signals from the audio channel signals, wherein the separation is based on the audio signal energy ratios; generate an audio object channel signal corresponding to each audio object of the set of audio objects from the audio channel signals, audio object energy ratios and audio signal energy ratios; create an audio object for each audio object channel signal by assigning a corresponding directional value to each of the audio object channel signals, wherein each corresponding directional value is from the metadata for the set of audio objects; and decode an encoded separate audio object to give a separate audio object. The means configured to generate an audio object channel signal corresponding to each audio object of the set of audio objects from the audio channel signals, audio object energy ratios and audio signal energy ratios may comprise means configured to: generate the audio object channel signal by applying a gain to a vector base amplitude panning generated signal corresponding to the audio object for which the audio object channel signal is generated, wherein the gain is based on audio object energy ratios associated with audio object and a factor of one minus the audio signal energy ratios, and wherein the vector base amplitude panning generated signal is based on the audio channel signals and the direction value associated with the audio object. The predefined format may be metadata assisted spatial audio (MASA) format. The audio object energy ratios may be independent stream with metadata (ISM) energy ratios and the audio signal energy ratios are MASA-to-total energy ratios. There is according to a second aspect a method for decoding an encoded spatial audio signal in a pass-through coding mode, wherein the encoded spatial audio signal comprises an encoded audio metadata signal, encoded audio channel signals and encoded information relating to a plurality of audio objects, wherein the method comprises: decoding the encoded spatial audio signal to produce audio metadata in a predefined format and audio channel signals; and decoding information relating to the plurality of audio objects to produce a plurality of audio objects. Decoding information relating to the plurality of audio objects to produce a plurality of audio objects may comprise: determining a number of the plurality of audio objects from the encoded audio metadata signal or from a frame header; and generating the number of null audio objects as a substitute for the plurality of audio objects; Decoding information relating to the plurality of audio objects to produce a plurality of audio objects may further comprise: decoding an encoded separate audio object to give a separate audio object; converting the separate audio object to separate audio object metadata in the predefined format and a separate audio object channel signal; combining the separate audio object metadata in the predefined format with the audio signal metadata in the predefined format to produce combined metadata in the predefined format; and combining the audio channel signals and the separate audio object channel signal. Decoding information relating to the plurality of audio objects to produce a plurality of audio objects may further comprise: decoding encoded metadata for a set of audio objects to produce energy ratios and metadata for the set of audio objects; converting the metadata for the set of audio objects to metadata in the predefined format for the set of audio objects; calculating an energy contribution of the set of audio objects from the metadata in the predefined format for the set of audio objects; calculating, based on the energy ratios, an energy contribution of the audio signal metadata in the predefined format and the audio channel signals; combining the audio signal metadata in the predefined format with the metadata in the predefined format for the set of audio objects to produce combined metadata in the predefined format, wherein the combining is based on the energy contribution of the audio signal metadata in the predefined format and the audio channel signals and the energy contribution of the set of audio objects; decoding an encoded separate audio object to give a separate audio object; converting the separate audio object to separate audio object metadata in the predefined format and a separate audio object channel signal; combining the separate audio object metadata in the predefined format with the combined metadata in the predefined format to produce further combined metadata in the predefined format; and combining the channel audio signals and the separate audio object channel signal. A null audio object may comprise: an audio object channel signal having zero sample values; and an audio object direction which is a predetermined fixed value. Decoding information relating to the plurality of audio objects to produce a plurality of audio objects may comprise: decoding encoded metadata for a set of audio objects to produce metadata for the set of audio objects, audio object energy ratios and audio signal energy ratios; separating further audio channel signals from the audio channel signals, wherein the separation is based on the audio signal energy ratios; generating an audio object channel signal corresponding to each audio object of the set of audio objects from the audio channel signals, audio object energy ratios and audio signal energy ratios; creating an audio object for each audio object channel signal by assigning a corresponding directional value to each of the audio object channel signals, wherein each corresponding directional value is from the metadata for the set of audio objects; and decoding an encoded separate audio object to give a separate audio object. Generating an audio object channel signal corresponding to each audio object of the set of audio objects from the audio channel signals, audio object energy ratios and audio signal energy ratios may comprise: generating the audio object channel signal by applying a gain to a vector base amplitude panning generated signal corresponding to the audio object for which the audio object channel signal is generated, wherein the gain is based on audio object energy ratios associated with audio object and a factor of one minus the audio signal energy ratios, and wherein the vector base amplitude panning generated signal is based on the audio channel signals and the direction value associated with the audio object. The predefined format may be a metadata assisted spatial audio (MASA) format. The audio object energy ratios may be independent stream with metadata (ISM) energy ratios and the audio signal energy ratios may be MASA-to-total energy ratios. There is according to a third aspect an apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to in a pass through-through coding mode: decode an encoded spatial audio signal to produce audio metadata in a predefined format and audio channel signals, wherein the encoded spatial audio signal comprises an encoded audio metadata signal, encoded audio channel signals and encoded information relating to a plurality of audio objects; and decoding information relating to the plurality of audio objects to produce a plurality of audio objects. An apparatus configured to perform the actions of the method as described above. A computer program comprising program instructions for causing a computer to perform the method as described above. A computer program product stored on a medium may cause an apparatus to perform the method as described herein. An electronic device may comprise apparatus as described herein. A chipset may comprise apparatus as described herein. Embodiments of the present application aim to address problems associated with the state of the art. Summary of the Figures For a better understanding of the present application, reference will now be made by way of example to the accompanying drawings in which: Figure 1 shows schematically a system or apparatus suitable for implementing some embodiments; Figure 2 shows schematically an example encoding mode selector as shown in the system of apparatus as shown in Figure 1 according to some embodiments; Figure 3 shows a flow diagram of the operation of the example encoding mode selector shown in Figure 2 according to some embodiments; Figure 4 shows a flow diagram of the operation of the example first, lowest, or only MASA bitrate encoding mode shown in Figure 3 according to some embodiments; Figure 5 shows a flow diagram of the operation of the example second, lower, or object information encoding mode shown in Figure 3 according to some embodiments; Figure 6 shows a flow diagram of the operation of the example third, higher, or single object encoding mode shown in Figure 3 according to some embodiments; Figure 7 shows a flow diagram of the operation of the example fourth, highest, or independent object and multi-input encoding mode shown in Figure 3 according to some embodiments; Figure 8 shows schematically an example audio encoder as shown in Figure 1 with respect to the second encoding mode according to some embodiments; Figure 9 shows schematically an example audio encoder as shown in Figure 1 with respect to the third encoding mode according to some embodiments; Figure 10 shows schematically an example of the audio decoder &renderer 129 shown in Figure 1 operating in a pass through mode with respect to an embodiment of the first encoding mode; Figure 11 shows schematically an example of the audio decoder &renderer 129 shown in Figure 1 operating in a pass through mode with respect to an embodiment of the second encoding mode; Figure 12 shows a flow diagram of the operation of the audio decoder &renderer 129 shown in Figure 1 operating in a pass through mode with respect to the embodiment of the second encoding mode; Figure 13 shows schematically an example of the audio decoder &renderer 129 shown in Figure 1 operating in a pass through mode with respect to an embodiment of the third encoding mode; Figure 14 shows a flow diagram of the operation of the decoder &renderer 129 shown in Figure 1 operating in a pass through mode with respect to the embodiment of the third encoding mode; Figure 15 shows schematically an example of the audio decoder &renderer 129 shown in Figure 1 operating in a pass through mode with respect to a further embodiment of the third encoding mode; Figure 16 shows a flow diagram of the operation of the audio decoder &renderer 129 shown in Figure 1 operating in a pass through mode with respect to the further embodiment of the third encoding mode; and Figure 17 shows an example device suitable for implementing the apparatus shown in previous figures. Embodiments of the Application The following describes in further detail suitable apparatus and possible mechanisms for the encoding of parametric spatial audio signals comprising transport audio signals and spatial metadata. As indicated above immersive audio codecs (such as 3GPP IVAS) support a multitude of operating points ranging from a low bit rate operation to transparency. It supports channel-based audio and scene-based audio inputs including spatial information about the sound field and sound sources. In the following the example codec is configured to be able to receive multiple input formats. In particular the codec is configured to obtain or receive a multi audio signal (for example received from a microphone array, or as a multi-channel audio format input, an ambisonics format input) and one or more audio object signal (these can also be called an independent stream with metadata - ISM format). Furthermore, in some situations the codec is configured to handle more than one input format at a time. This combined (input) format mode can, for example, enable simultaneous encoding of two different audio input formats. An example of two different audio input formats which IVAS currently supports is the combination of the MASA format with audio object format. Metadata-Assisted Spatial Audio (MASA) is an example of a parametric spatial audio format and representation suitable as an input format for IVAS. It can be considered as an audio representation consisting of ‘N channels + spatial metadata’. It is a scene-based audio format particularly suited for spatial audio capture on practical devices, such as smartphones. The idea is to describe the sound scene in terms of time- and frequency-varying sound source directions and, e.g., energy ratios. Sound energy that is not defined (described) by the directions, is described as diffuse (coming from all directions). As discussed above spatial metadata associated with the audio signals may comprise multiple parameters (such as multiple directions and associated with each direction (or directional value) a direct-to-total energy ratio, spread coherence, distance, etc.) per time-frequency tile. The spatial metadata may also comprise other parameters or may be associated with other parameters which are considered to be non-directional (such as surround coherence, diffuse-to-total energy ratio, remainder-to-total energy ratio) but when combined with the directional parameters are able to be used to define the characteristics of the audio scene. For example, a reasonable design choice which is able to produce a good quality output is one where the spatial metadata comprises one or more directions for each time-frequency subframe (and associated with each direction direct-to-total ratios, spread coherence, distance values etc) are determined. As described above, parametric spatial metadata representation can use multiple concurrent spatial directions. With MASA, the proposed maximum number of concurrent directions is two. For each concurrent direction, there may be associated parameters such as: Direction index; Direct-to-total ratio; Spread coherence; and Distance. In some embodiments other parameters such as Diffuse-to-total energy ratio; Surround coherence; and Remainder-to-total energy ratio are defined. The IVAS codec is also configured to operate in what is known as a “pass through” mode where the format of the (spatial) audio output at the decoder / renderer is of the same form as the input to the IVAS encoder. For instance, taking the above example of encoding the audio input combination of the MASA format with the audio object format. Then an IVAS codec operating in “pass through” mode would produce an output at the decoder / renderer which adheres to the same format as the audio input, which in this case would be an output combination of MASA format and audio object format. Furthermore, the IVAS coding model also provides for encoding of a combined format at various bitrates. This coding model enables a parametric encoding of the audio object input that includes variable rate encoding for an object direction parameter based. As will be seen below when the IVAS codec provides for the simultaneous encoding of audio object format and MASA format, this pairing of formats can be encoded using one of four separate modes depending on the available bit rate. For the two lowest bit rate coding modes the encoding of some or all audio objects are amalgamated with the encoding of the MASA format resulting in an audio output where audio objects are inseparable from the overall spatial audio mix. For the third coding mode the audio objects can be separated to some degree by using sent parametric information relating to the audio objects. In the fourth mode, which is the highest bit rate coding mode, all audio objects are separable from the overall spatial audio mix at the decoder / renderer. Therefore, when the IVAS codec is operating in a “pass through” mode the decoder / renderer is currently not able to recreate the same format of output as the input to the codec for cases when the audio object and MASA input have been encoded using one of the first three encoding modes This is in part due to the loss of information from the first three encoding modes, which ultimately does not allow for the same number of individual audio objects to be created at the decoder / renderer as found at the input to the codec. Embodiments of the present application aim to address this problem associated with the state of the art. The concept as discussed in further detail herein proceeds on the basis of providing a signalling mechanism which carries information relating to the number of audio objects which have subjected to combined encoding with the MASA metadata. This information can then be used at the decoder / renderer as part of the IVAS decoding process to recreate a same number of audio objects as that presented at the encoder. In this regard Figure 1 depicts a general apparatus 100 and system for providing the joint coding of the MASA format and the audio object format. The system is shown with an ‘analysis’ part. The ‘analysis’ part is the part from receiving the multichannel signals up to an encoding of the metadata and downmix signal. The input to the system ‘analysis’ part is the multi-channel audio signals 102. In the following examples a microphone channel signal input is described, however any suitable input (or synthetic multi-channel) format may be implemented in other embodiments. For example, in some embodiments the spatial analyser and the spatial analysis may be implemented external to the encoder. For example, in some embodiments the spatial (MASA) metadata associated with the audio signals may be provided to an encoder as a separate bit-stream. Additionally, Figure 1 also depicts multiple audio objects 104 (each audio object comprising an audio object signal and audio object metadata) as a further input to the analysis part. As mentioned above these multiple audio objects (or audio object stream) 104 may represent various sound sources within a physical space. Each audio object may be characterized by an audio (object) signal and accompanying metadata comprising directional data (in the form of azimuth and elevation values) which indicate the position or direction of the audio object within a physical space on an audio frame basis. The multi-channel signals 102 are passed to an analyser and encoder 101, and specifically a transport signal generator 105 and to a metadata generator 103. In some embodiments the metadata generator 103 is also configured to receive the multi-channel signals and analyse the signals to produce metadata 114 associated with the multi-channel signals and thus associated with the transport signals 106. The metadata generator103 may be configured to generate the metadata which can comprise, for each time-frequency analysis interval, a direction parameter and an energy ratio parameter and a coherence parameter (and in some embodiments a diffuseness parameter). The direction, energy ratio and coherence parameters may in some embodiments be considered to be MASA spatial audio parameters (or MASA metadata). In other words, the spatial audio parameters comprise parameters which aim to characterize the sound-field created / captured by the multi-channel signals (or two or more audio signals in general). In some embodiments the parameters generated can differ from frequency band to frequency band. Thus, for example in band X all of the parameters are generated and transmitted, whereas in band Y only one of the parameters is generated and transmitted, and furthermore in band Z no parameters are generated or transmitted. A practical example of this can be that for some frequency bands such as the highest band some of the parameters are not required for perceptual reasons. The transport signals 106 and the metadata 114 may be passed to a combined encoder core 109. In some embodiments the transport signal generator 105 is configured to receive the multi-channel signals and generate a suitable transport signal comprising a determined number of channels and output the transport signals 106 (MASA transport audio signals). For example, the transport signal generator 105 can be configured to generate a 2-audio channel downmix of the multi-channel signals. The determined number of channels may be any suitable number of channels. The transport signal generator in some embodiments is configured to otherwise select or combine, for example, by beamforming techniques the input audio signals to the determined number of channels and output these as transport signals. In some embodiments the transport signal generator 105 is optional and the multichannel signals are passed unprocessed to a combined encoder core 109 in the same manner as the transport signal are in this example. The audio objects 104 can be passed to the audio object analyser 107 for processing. In some embodiments the audio object analyser 107 analyses the object audio input stream 104 in order to produce suitable audio object transport signals and audio object metadata. For example, the audio object analyser can be configured to produce the audio object transport signals 128 by downmixing the audio signals of the audio objects into a stereo channel together using amplitude panning based on the associated audio object directions. Additionally, the audio object analyser can also be configured to produce the audio object metadata associated with the audio object input stream 104. The audio object metadata may comprise direction values which are applicable for all sub-bands. So, if there are 4 objects, there are 4 directions. In the examples described herein the direction values also apply across all of the subframes of the frame, but in some embodiments the temporal resolution of the direction values can differ and the directions values apply for one or more than one sub-frames of the frame. Furthermore, energy ratios (or ISM ratios) may be determined for each object. The energy ratio (ISM ratio) defines the contribution of the object within the object part of the total audio environment. In the following examples the energy ratios (or ISM ratios) are for each time-frequency tile for each object. The audio object analyser 107 can be additionally configured to determine an audio object for separation from the stream of audio objects 104 and create the associated separated object audio signal 114. In embodiments this can be a mono audio signal. The separated audio object audio signal may then be encoded by an audio signal encoder 110 (such as an EVS encoder) to form the encoded separated object audio signal 117. This encoded separated object audio signal 117 can then be forwarded to the Bitstream generator 113 to form part of the bitstream 118. In embodiments the audio signal encoder 110 may be a mono channel encoder of IVAS. Additionally, an identifier index of the separated audio object can form part of the bitstream 118. In some embodiments, the audio object analyser 107 may be sited elsewhere and the audio objects 104 input to the analyser and encoder 101 is audio object transport signals and audio object metadata. The analyser and encoder 101 can comprise a combined encoder core 109 which is configured to receive the transport audio (for example downmix) signals 106 and audio object transport signals 128 in order to generate a suitable encoding of these audio signals. The analyser and encoder 101 can also comprise an audio object metadata encoder 111 which is similarly configured to receive the audio object metadata 108 and output an encoded or compressed form of the input information as encoded audio object metadata 112. In some embodiments the combined encoder core 109 can be configured to implement a stream separation metadata determiner and encoder which can be configured to determine the relative contributory proportions of the multi-channel signals 102 (MASA audio signals) and audio objects 104 to the overall audio scene. This measure of proportionality produced by the stream separation metadata determiner and encoder can be used to determine the proportion of quantizing and encoding “effort” expended for the input multi-channel signals 102 and the audio objects 104. In other words, the stream separation metadata determiner and encoder may produce a metric which quantifies the proportion of the encoding effort expended on the multichannel audio signals 102 compared to the encoding effort expended on the audio objects 104. This metric can be used to drive the encoding of the audio object metadata 108 and the metadata 114. Furthermore, the metric as determined by the separation metadata determiner and encoder can also be used as an influencing factor in the process of encoding the transport audio signals 106 and audio object transport audio signal 128 performed by the combined encoder core 109. The output metric from the stream separation metadata determiner and encoder can furthermore be represented as encoded stream separation metadata and be combined into the encoded metadata stream from the combined encoder core 109. In some embodiments the analyser and encoder 101 comprises a bitstream generator 113 configured to obtain the encoded metadata 116, the encoded transport audio signals 138 and the encoded audio object metadata 112 and generate the bitstream 118 for potential transmission or storage. In some embodiments the analyser and encoder 101 comprises an encoder controller 115. The encoder controller 115 can in some embodiments control the encoding implemented by the audio object metadata encoder 111 and the combined encoder core 109. In some embodiments encoder controller 115 is configured to determine the bitrate for the bitstream 118 and based on the bitrate control the encoding. In some embodiments the encoder controller 115 is further configured to control at least one of the audio object analyser 107, transport signal generator 105 and metadata generator in generating parameters. The analyser and encoder 101 can in some embodiments be a computer or mobile device (running suitable software stored on memory and on at least one processor), or alternatively a specific device utilizing, for example, FPGAs or ASICs. The encoding may be implemented using any suitable scheme. In some embodiments the encoder 107 may further interleave, multiplex to a single data stream or embed the encoded MASA metadata, audio object metadata and stream separation metadata within the encoded (downmixed) transport audio signals before transmission or storage shown in Figure 1 by the dashed line. The multiplexing may be implemented using any suitable scheme. Furthermore, with respect to Figure 1 is shown an associated decoder and renderer 129 which is configured to obtain the bitstream 118 comprising encoded metadata 116, Encoded transport audio signals 138 and encoded audio object metadata 112 and from these generate suitable spatial audio output signals. The decoding and processing of such audio signals in relation to the problem of decoding in a “pass-through” mode of operation will be discussed later on. With respect to Figure 2 is shown in further detail the encoder controller 115 according to some embodiments. In this example the encoder controller 115 comprises a bitrate determiner / monitor 201 configured to determine and / or monitor the available bitrate for the bandwidth for the encoded audio and metadata. This could be determined based on a transmission path bandwidth estimation (and for example be based on an estimated signal strength) or a bandwidth storage determination to maintain the file for a determined time to be below a required size or by any suitable manner. The bitrate determiner / monitor 201 can furthermore be configured to control an encoding mode selector 203. The encoder controller 115 can comprise an encoding mode selector 203 configured to select an encoding mode, for example based on the determined bandwidth, bitrate and / or number of audio objects since the overall bit rate is distributed between the encoding of the MASA metadata &transport signals and the audio objects. Once the encoding mode has been determined the encoder controller 115 then controls the encoders, for example the combined encoder core 109 and audio object metadata encoder 111, according to the encoding mode. Bandwidth and bitrate information can also be passed onto the audio object analyser 107 so that this information can be used [by the audio object analyser 107] to determine whether audio objects are to be separated in accordance with the second and third modes of encoding. As will be seen later these modes are also referred to as mode B and mode C respectively. With respect to Figure 3 is shown a flow diagram of an example operation of the encoder controller shown in Figure 2 In this example there is an initial operation of receiving or obtaining or otherwise determining the bitrate or bandwidth for encoded parameters and audio data as shown in Figure 3 by step 301. Having obtained the available bandwidth or bitrate then a check can be made to determine whether the bitrate is below a first (or lowest) threshold limit as shown in Figure 3 by step 303. Where the available bandwidth or bitrate is below the first (or lowest) threshold limit then the encoders can be controlled to encode the transport channels and MASA metadata only (also shown as Mode A) as shown in Figure 3 by step 304. Where the available bandwidth or bitrate is above the first (or lowest) threshold limit then a further check can be made to determine whether the bitrate is below a second (or lower or one audio object) threshold limit as shown in Figure 3 by step 305. Where the available bandwidth or bitrate is below the second (or lower or one object) threshold limit then the encoders can be controlled to encode the combined input of MASA and audio objects in a mode B. To that end Figure 8 depicts an encoder suitable for mode B where it can be that the encoder has some functional blocks which are the same as found in Figure 1 . The encoder 1101 is arranged to receive the multichannel audio signals 102 and produce the respective transport audio signals 1106 and multichannel MASA metadata 1104. This is performed using the MASA meta generator 103 and transport signal generator 105 whose functionality is the same as the functional blocks 103 and 105 Figure 1. The encoder 1101 is also depicted as receiving the stream of audio objects 104 via the audio object separator &analyser 1107, which can be arranged for each frame to decide which audio object to separate from the audio object stream 104 in order to create a separated audio object 1114 comprising audio object metadata and an audio object signal. The separated audio object 114 may be selected based on the relative level of the audio objects, such as the loudest audio object. This is explained in detail in WO2022 / 214730. The audio object separator &analyser 1107 is also arranged to produce a suitable remaining audio objects transport audio signal 1126. This may be produced by downmixing the audio signals of the remaining audio objects as explained before using amplitude panning based on the associated audio object directions. Additionally, the audio object separator &analyser 1107 may also be configured to produce the audio object metadata associated with the remaining audio objects. In mode B this can take the form of generating an audio object based MASA stream 1108 from the remaining audio objects by the methods presented in WO2019086757. The separated audio object 1114 comprising the separated audio object audio signal and separated audio object direction) may be encoded by an audio object encoded 100 to form the encoded separated audio object 1117 which is then passed to the bitstream generator 1113 to form part of the bitstream 118. In addition, an identifier describing which audio object was separated may also be sent. In Figure 8, the combined encoder core 109 as described previously is shown as receiving the remaining audio objects MASA metadata 1108, the remaining audio objects transport audio signals 1126, the transport audio signals 1106 and the multichannel MASA metadata 1104. The combined encoder core 109 can be arranged to combine the multichannel transport audio signals 1106 and the audio signals / channels from the remaining audio objects transport audio signals 1126 into a single combined stream which is encoded to produce the encoded transport audio signals stream 1138. Similarly, the combined encoder core 109 can be arranged to combine the multichannel MASA metadata 1104 with the remaining audio objects MASA metadata 1108 to produce the encoded MASA metadata stream 1116. In Figure 8, the controller 115 is shown as controlling the encoding implemented by the combined encoder core 109. In some embodiments the controller 115 also controls the audio object encoder 1110 and audio object signal separator &analyser 1107. Therefore, in mode B the encoders can be controlled to encode in a combined manner the multichannel transport channels 1106, multichannel MASA metadata 104, remaining audio objects MASA metadata 1108 remaining audio objects transport audio signals 1126, one audio object data (separated audio object metadata and separated audio object signal) and some in embodiments the one audio object identifier as shown Figure 3 by step 306. Where the available bandwidth or bitrate is above the second (or lower or one object) threshold limit then a further check can be made to determine whether the bitrate is below a third, higher or full object threshold limit as shown in Figure 3 by step 307. Where the available bandwidth or bitrate is below the third, higher or full object threshold limit then the encoders can be controlled to encode transport channels, MASA metadata, ISM metadata (all objects), MASA to total ratios, ISM ratios, and 1 object audio data, with 1 object identifier (also shown as Mode C) as shown in Figure 3 by step 308. 5 Where the available bandwidth or bitrate is above the third, higher or full object threshold limit then the encoders can be controlled to encode transport channels, MASA metadata, ISM metadata (all objects), All objects audio data (also shown as Mode D) as shown in Figure 3 by step 310. 10 The encoding modes can for example be summarised by the following table: Mode Bitrate range / Format Encoded parameters A -32kbps -transport channels - MASA metadata - input format - number of input objects B 48-64kbps I SM_M AS A_M 0 D E_MAS A_O N E_O B J - Transport channels of mix of MASA and remaining audio objects - MASA metadata of mix of input MASA metadata and remaining audio objects equivalent MASA metadata - 1 separated object audio data 1 separated object metadata C 80-128kbps I SM_MAS A_M0 D E_PARAM_O N E_O B J - Transport channels of mix of MASA and remaining audio objects - MASA metadata - all objects metadata - MASA to total ratios - ISM ratios -1 object audio data -1 object identifier D 160- ISM_MASA_MODE_DISC - MASA Transport channels - MASA metadata - ISM metadata - all objects audio data The bitrates shown herein are examples and it would be understood that they can be other specific values. With respect to Figures 4 to 7 is shown flow diagrams showing a first (or lowest or combined) encoding mode as shown in Figure 3 by step 304, a second (or lower or object metadata) encoding mode as shown in Figure 3 by step 306, a third (or higher or one object) encoding mode as shown in Figure 3 by step 308 and a fourth (or highest or all objects) encoding mode as shown in Figure 3 by step 310 respectively. For example, Figure 4 the mode A encoding method, shows the first (or lowest or combined) encoding mode as shown in Figure 3 by step 304 in further detail. Thus, for very low total bitrates (for example less or equal to 32kbps) all encoding is implemented using a MASA representation. Thus, for example there is an operation of receiving / obtaining the object based streams (independent streams with metadata) and multichannel based (MASA stream) transport audio signals and metadata as shown in Figure 4 by step 401. Then, as shown in Figure 4 by step 403, there is an operation of generating an object based MASA stream from the object streams (independent streams with metadata). This object based MASA stream can in some embodiments be created from the object stream using, for example, the methods presented in WO2019086757A1. After this, as shown in Figure 4 by step 405, the object based MASA stream and multichannel based MASA stream are combined. In some embodiments the original MASA stream and the MASA stream created from the objects can be combined using the method presented in GB2574238. The decoder received the information about the audio objects and the MASA audio content in the MASA format. Then the combined stream is output as shown in Figure 4 by step 407. In such embodiments the object audio content (together with the MASA audio content) is present in the decoded audio scene, but the objects cannot be edited nor separated from the scene at the decoder based on the decoded parameters. Figure 5 shows the mode B encoding method, the second (or lower or object metadata) encoding mode as shown in Figure 3 by step 306. Thus, for low bit-rates (for example between 48kbps and 80kbps) and since there are a more bits available, there is a possibility to send the audio signal and metadata of one audio object independently. Thus, there is shown in Figure 5 501 which portrays the processing step of selecting an audio object for separation and encoding said audio object. In addition, a downmix formed from the transport audio signals / channels 1106 and the remaining audio object signals (from the remaining audio objects 1118) giving the encoded combined transport audio signals 1138 are sent together with a combined multichannel based MASA and audio object based MASA stream 1116 Thus, for example there is a method step of receiving / obtaining the remaining audio objects stream 1118, the multichannel transport audio signals 1106, remaining audio objects MASA metadata 1124 and the multichannel MASA metadata 1104 as shown in Figure 5 by step 503. Then, as shown by step 505 in Figure 5, generate a combined multichannel transport and audio object signals downmixed (channel pair element) audio signals. In other words, the audio content of MASA (1106) and the audio signal channels of the remaining audio objects (1118) are downmixed to 2 channels (channel pair element CPE). The remaining audio objects MASA metadata 1124 and the multichannel MASA metadata 1104 may be combined into a single combined MASA metadata stream by any suitable method. For example, such as the method presented in GB2574238. This step is shown in Figure 5 as step 507. Furthermore, the combined MASA metadata can then be encoded based on any suitable MASA metadata encoding method as shown in Figure 5 by step 509. The combined multichannel and audio object signals can then be encoded based on any suitable audio signal encoding method as shown in Figure 5 by step 511. The encoder can then output encoded combined MASA metadata 1116, encoded combined transport audio signals 1138 and encoded separated audio object 1117 as shown in Figure 5 by step 513. Figure 9 shows an encoder 1201 for operating in mode C method of encoding. It can be seen that the encoder and analyser have similar processing blocks to the analyser and encoder 1101 in Figure 8. For the sake of brevity, the following will describe functional blocks which differ from Figure 8. With respect to the audio object signal separator &analyser 1107, this may differ from mode B operation in the sense that 1107 may be arranged to produce the ISM ratios, MASA-to-total energy ratios and metadata for the remaining audio objects for each frame of the audio object stream 104. These have been collectively grouped together as 1208 in Figure 9 These parameters may be received by the audio object metadata encoder 111 and output encoded remaining audio object metadata 1112. With respect to the controller 115 this is shown as controlling both the combined encoder core 109 and audio object metadata encoder 111. Figure 6 shows the mode C encoding method, the third encoding mode as shown in Figure 3 by step 308. Thus, in medium or higher bitrates (for example bitrates larger or equal to 96kbps and lower than 160kbps), the audio content of one object is separated and sent independently. In addition, the downmix formed from the MASA transport channels and the rest of the objects are sent under MASA format with the additional parameters of the MASA-to-total energy ratios and ISM ratios. Moreover, the ISM metadata is sent, and an identifier describing which object was separated. At each frame it is decided which object is to be separated. The decision may, e.g., be based on the relative level of the objects with respect to other objects (e.g., separate the loudest object). This is explained in detail in WO2022 / 214730. MASA-to-total-energy may be defined as the proportion of MASA energies to total audio energies for a TF tile basis (total audio energies being the combined MASA and audio object energies). Details of MASA-to-total-energy may be found in WO2022 / 200666. Thus, for example there is a method step 601 of selecting one audio object and generating an object identifier based on the selected audio object. Furthermore, the audio signal associated with the selected audio object is encoded. Any suitable audio signal encoder may be used for encoding the audio signal of the selected object. For example, the same or similar audio signal encoder as used for encoding the MASA audio signal(s) can be employed. The remaining (or non-selected) audio object transport audio signals (or downmix) is generated as shown by Figure 6 by step 603. The object transport signals can be created in the same manner as presented in the previous mode, mode B, with the selected or separated object not included within the mix. For example, the multichannel MASA audio signals and the remaining audio object transport signals can be summed together to generate the combined transport audio signals. The MASA-to-total energy ratios and the ISM ratios can be determined as shown in Figure 6 by step 605. The object identifier, multichannel MASA metadata, remaining audio object metadata, MASA-to-total energy ratios and the ISM ratios can then be encoded based on any suitable encoding method as shown in Figure 6 by step 607. For example, the encoding can employ any scalar or vector quantizer followed or not by entropy coding. The encoding of the MASA-to-total energy ratios can be implemented in the manner as described in WO2022 / 200666. The encoding of the ISM ratios is described in GB 2304286.4. The combined transport audio signals can then be encoded based on any suitable MASA audio signal encoding method as shown in Figure 6 by step 609. The encoding of the combined transport audio signals can employ any suitable transport audio signal encoding, for example the MASA encoder. In other words, the separated object is determined, separated and encoded as described in WO2022 / 214730, and for the remaining objects and the MASA stream the processing works as was described in WO2022 / 200666. The encoder can then output the encoded object identifier, encoded MASA metadata, encoded MASA-to-total ratios, encoded ISM ratios, encoded object metadata (for all objects), selected single object audio signal and encoded combined transport audio signals as shown in Figure 6 by step 611. Figure 7 shows the mode D encoding method, the fourth (or highest or all objects) encoding mode as shown in Figure 3 by step 310. Thus, in the higher bitrates (for example bitrates above or equal to 160kbps) the two input audio formats, MASA and ISM are independently encoded and transmitted. Thus for example there is a method step of receiving / obtaining the object based streams (independent streams with metadata) and multichannel based (MASA stream) transport audio signals and metadata as shown in Figure 7 by step 701. Then, as shown by step 703 in Figure 7, there is encoded the multichannel based (MASA stream) transport audio signals and metadata based on any suitable MASA encoding method. The object (independent streams with metadata) and associated metadata can furthermore be encoded as shown in Figure 7 by step 705. Any suitable mono encoder (as part of the main encoder) can be employed to implement the encoding, for example an EVS based mono encoder. The encoder can then output the independently encoded object (independent streams with metadata) and associated metadata and independently encoded multichannel based (MASA stream) transport audio signals and metadata as shown in Figure 7 by step 707. With reference to decoder &renderer 129 it can be seen in Figures 1,8 and 9 that there may be a further input 140 which can be used to indicate that the decoder is to operate in a “pass through” EXT mode. As explained previously the decoder requires information relating to the number of audio objects at the encoder so that the decoder can operated in a “pass though” EXT mode. In some of the above combined encoding modes the number of audio objects is explicitly included into the frame header as part of the encoding process. However, other combined coding modes, especially the lower bit rate modes, may not have the bit rate capacity to include the number of audio objects in each frame header. In these cases, the encoder is arranged to repurpose other bits in the bitstream in order to convey information relating to the number of audio objects 104. For instance, when the lowest coding rate mode (mode A) is selected for encoding two bits may be reserved from the bit allocation for a frame of encoded MASA metadata stream 116. Therefore, when mode A is selected in the combined encoder core 109 two bits may be repurposed to convey the number of audio objects 104. In this instance the two reserved bits are used to signal the number of audio objects 104 according to the following allocation - ‘00’ indicates only a multichannel audio signal / MASA input - ‘01’ indicates a multichannel audio signal / MASA input with 1 audio object input - ‘10’ indicates a multichannel audio signal / MASA input with 2 audio object input - ‘11’ indicates a multichannel audio signal / MASA input with either 3 or 4 audio objects. Additionally, an extra bit may be used repurposed from the transport channels signalling bit to discern the difference between either 3 or 4 audio objects. As a note, when the IVAS encoder is operating as a MASA format encoder, the encoder is configured to operate with either one or two transport audio channel(s) and therefore a transport channels signalling bit is required. However, when the IVAS encoder is operating in the mode which supports the simultaneous encoding of the MASA format and audio object format the number of transport audio channels / signals is constantly two. Consequently, the transport channels signalling bit can be repurposed for the signalling of the number of audio objects. When modes B and C are selected as the method for combined encoding then information relating to the number of audio objects 104 is included in the frame header of the encoded bitstream. Thus, for these encoding modes, the combined encoder core 109 is not required to use the above reserved bits from the MASA metadata stream. With respect to Figure 10 there is shown a decoder for use in “pass through” mode (EXT mode) when combined encoding according to mode A has been used. Specifically for each frame, the demux 160 is arranged to separate the reserved two bits from the encoded metadata 1005 together with the repurposed signalling bit from the encoded transport audio signal 1006 from the bitstream 118 and feed them to the audio objects number determiner 1013. The audio objects number determiner 1013 is then arranged to determine the number of audio objects by using the above allocation table to give the number of audio objects value 1007. The number of audio objects value 1007 is then passed to the “null” audio object producer 1015. The “null” audio object producer is arranged to produce as an output the number of audio objects value of “null” audio objects. Where a “null” audio object is in effect a dummy audio object having a zero valued audio object signal and a zero valued audio object metadata. The “null” audio object serves as place holder for future development of the IVAS coding system. Additionally, for each IVAS frame, the demux 160 is arranged to extract the encoded MASA signal 1002 comprising the encoded MASA metadata and transport audio signals from the bitstream 118. These are then decoded by the MASA decoder 1003 to provide decoded MASA signal (comprising decoded MASA metadata 10041 and transport audio signals 10042) 1004. The output frame 150 from the decoder &renderer 129 when operating in EXT mode is the combination of decoded MASA signal 1004 and the requisite number of “null” audio objects 1009. With respect to Figure 11, the decoder &renderer 129 is shown in a form which is suitable for “pass though” decoding of the bitstream 118 when encoded using mode B and Figure 12 shows the corresponding processing steps of the decoder &renderer 129. For each frame the demux 160 is arranged to extract from the bitstream 118 the encoded MASA metadata 1116, the encoded transport audio signals 1138, the encoded separated audio object 1117 (comprising the audio object audio signal and the audio object directional metadata) and frame header 1121. This can be shown as the method step 1201 in Figure 12. The audio objects number determiner 1013, when in mode B operation, is arranged to read the IVAS frame header 1121 in order to determine the number of audio objects 1007 present in the audio object stream 104. This step is shown for example by processing step 1203 in Figure 12. The resulting value giving the number of audio objects 1007 may then be used by the “null” audio object processor 1015 as before to generate the requisite number of “null” audio objects 1009. This is shown as the processing step 1205 in Figure 12. These steps are basically the same steps as performed in mode A decoding in “pass through” mode. In a manner similar to Figure 10, the decoder &renderer 129 is arranged to decode the encoded MASA metadata 1116 and encoded transport audio signals 1138 extracted from the IVAS frame with the MASA decoder core 1003. This produces the decoded MASA signal 1004 (comprising the MASA metadata 10041 and transport audio signals 10042) which is passed to the MASA combiner 1181. This is shown as processing step 1207 in Figure 12. Additionally, the decoder &renderer 129 is also arranged to decode the encoded separated audio object 1117 by using an audio object signal decoder 1101. This produces as an output signal the decoded separated audio object 1061 comprising the audio object audio transport signal and corresponding directional metadata. This signal is passed to the audio object to MASA converter 1171 in the decoder &renderer 129. This process is shown by the step 1209 in Figure 12. The audio object to MASA converter 1171 is arranged to convert the decoded separated audio object 1061 to an object based MASA format. This is performed using techniques described in detail in WO2019086757A1. The resulting separated MASA audio object 1051 is passed to the MASA combiner 1181. This process is shown by the step 1211 in Figure 12. Also shown is a MASA combiner 1181 which is configured to combine two individual MASA format signals. A MASA format signal being a signal comprising a MASA metadata and at least one transport audio signal or channel. In this case the MASA combiner 1181 is configured to combine the separated MASA audio object 1051 and the decoded MASA signal 1004 and combine these two signals to give the combined MASA signal 1101 comprising the combined MASA metadata 11011 and combined audio transport signals 11012 . The combining of the two signals is performed according to the techniques presented in GB2574238. This combining step is shown as step 1213 in Figure 12. The output from the MASA metadata combiner 1181, the combined MASA metadata signal 1101 (comprising combined MASA metadata 11011 and combine transport audio channels 11012) forms part of the output 150 from the decoder &rendered 129 when operating with “Pass through” in mode B. The other component of the output 150 comprises the “null” audio objects 1009. With respect to Figure 13, the decoder &renderer 129 is shown in a form which is suitable for “pass though” decoding of the bitstream 118 when encoded using mode C and Figure 14 show the corresponding processing steps of the decoder &renderer 129 The demux 160 is arranged to extract from the bitstream 118, for each IVAS frame the encoded MASA metadata 1116, the encoded transport audio signals 1138, the encoded separated audio object 1117, the encoded remaining audio objects metadata 1112 and frame header 1121. This is shown as the method step 1401 in Figure 14. The audio objects number determiner 1013 when operating in mode C is also configured to operate in a similar manner to mode B by reading the IVAS frame header 1121 to determine the number of audio objects 1007. The resulting value is then used by the “null” audio object processor 1015 to generate the requisite number of “null” audio objects 1009. This is shown as the combined processing step 1403 in Figure 14, which is equivalent to the steps 1201 and 1203 from Figure 12. In a similar manner to mode B decoding, the decoder &renderer 129 operating in mode C is also arranged to decode the encoded MASA metadata 1116 and encoded transport audio signals 1138 extracted from the IVAS frame with the MASA decoder core 1003 to provide the decoded MASA signal 1004. This is represented as the processing step 1405 in Figure 14. In mode C operation an audio object metadata decoder 1303 is arranged to receive the encoded remaining audio objects metadata 1112 and decode said data to provide the remaining audio objects metadata 1310. The remaining audio objects metadata 1310 is passed to the audio object energy calculator 1320 which can be arranged to derive an energy value for each TF tile of the IVAS frame from the decoded remaining audio object metadata parameters such as the objects directions ( of the individual audio objects), the ISM ratios and the MASA-to-total energy ratios. These energy values represent the contribution of the remaining audio objects to the overall spatial audio mix. These steps can be represented for example as the processing steps 1407 and 1409. Additionally, the remaining audio objects metadata 1310 is passed to an audio object metadata to MASA converter 1318 which is arranged to convert the remaining audio objects metadata 1310 to the MASA format giving the remaining audio objects MASA metadata 1319. This can be shown, for example as the processing step 1411 in Figure 14. The ISM ratios and the MASA-to-total energy ratios can be grouped together with the audio object directions of the audio objects. Alternatively, the ISM ratios and the MASA-to-total energy ratios can also be considered to be separate from the audio object directions when it comes to encoding (and decoding) functionality. The audio object decoder 1101 is also used in Mode C for decoding the encoded separated audio object 1117 which is used to provide the decoded separated audio object 1061. This is shown as the processing step 1402 in Figure 14. Also shown in Figure 13 is the audio object to MASA converter 1171. When operating in mode C, the audio object to MASA converter 1171 is also configured to convert the decoded separated audio object 1061 (comprising audio object audio signal and audio object directional metadata) to the MASA format resulting in the separated MASA audio object 1051. This is essentially the same step as performed for by the audio object metadata to MASA converter when operating in Mode B. This is shown in Figure 14 as the processing step 1413. The audio object metadata decoder 1303 is also shown as producing the MASA-to-total energy ratios 1305 for use by the MASA energy calculator 1317. In some embodiments this signal is also part of the remaining audio object metadata signal 1319. The MASA-total energy ratios 1305 is used by the MASA energy calculator 1317 to provide energy values (on a per TF tile basis) corresponding to the proportion of MASA signal (from the multichannel audio signal) in the overall spatial audio mix. This is shown as the MASA energy signal 1309. With respect to Figure 14, this determining step is shown as 1415. The MASA combiner 1181 is then configured to combine the remaining audio objects MASA metadata 1319 and the decoded MASA signal 1004. This combination is performed in view of the relative contribution to the overall mix of the aforementioned signals by incorporating the MASA and remaining audio objects MASA energy values into the calculation. Details as to how the combination is performed can be found in GB2574238. This combining step is shown in Figure 14 as step 1417 combining (contribution of) decoded MASA signal 1004 and (contribution of) remaining audio objects metadata in MASA format. Finally, The MASA combiner 1181 is also configured when operating in mode C, to follow the above combining step with a further combining step to provide the overall combined MASA signal 1310. This is performed by combining the combined MASA and remaining audio objects metadata with the separated MASA (format) audio object 1051. Again, this combining step (step 1419 in Figure 14) can be performed according to the details in GB2574238. Combined MASA metadata signal 1310 (comprising the combined MASA metadata 13101 and combined audio transport signals 13102) form part of the output 150 from the decoder &rendered 129 when operating with “Pass through” in mode C. The other component of the output 150 being the “null” audio objects 1009. There is according to a further embodiment a method of “pass through” decoding when operating in C mode which determine the individual audio object signals in the mix by using their associated parametric object metadata and the encoded transport audio signals. The “pass though” output can then be formed as comprising the determined “individual” remaining audio object signals together with the separated object signal together with the decoded MASA metadata and accompanying transport audio signals (corresponding to the multichannel audio input.) Figure 15 depicts the further embodiment of the decoder &renderer 129 when operating in the C mode with “pass through” decoding and Figure 16 show the associated method steps. The demux 160 is arranged to extract from the bitstream 118, for each IVAS frame the encoded MASA metadata 1116, the encoded transport audio signals 1138, the encoded separated audio object 1117, the encoded remaining audio objects metadata 1112 and the IVAS frame header (not shown). This is shown as the method step 1601 in Figure 16. In the further embodiment the decoder &renderer 129 is be arranged to decode the encoded MASA metadata 1116 and encoded transport audio signals 1138 extracted from the IVAS frame with the combined decoder 1003 to provide the decoded MASA signal. This is shown in Figure 15 in the form of its constituent parts of decoded MASA metadata 1501 and the decoded transport audio signals 1503. The decoded MASA metadata 1501 comprises solely MASA metadata from the multichannel audio signals 102. The decoded transport audio signals 1503 comprise the transport audio signals from the multichannel audio signals mixed as a CPE with the remaining audio objects transport audio signals. The step of decoding the MASA signal is represented as the processing step 1603 in Figure 16. The audio object metadata decoder 1303 is arranged in this embodiment to decode the encoded remaining audio objects metadata 1112. This produces the MASA-to-total energy ratios 1305 and ISM ratios 1509 for each TF tile of the frame and a directions value for associated with each remaining audio objects. This is shown collectively in Figure 15 as the direction values 1513. The decoding of the encoded remaining audio objects metadata 1112 is shown in Figure 16 by the method step 1605. Also shown in Figure 15 is the transport audio signal separator 1502 whose function is to separate the decoded transport audio signals 1503 into the (MASA) transport audio signals 1505 (which are the transport audio signals associated with the multichannel audio signals 102). This separation can be performed on a TF tile basis using the MASA-to-total energy ratios 1305. The resulting (MASA) transport audio signals 1505 can then go onto form part of the EXT mode output 150. The step of separating the decoded transport audio signals 1503 is shown in Figure 16 as the processing step 1607. For example, the (MASA) transport audio signals 1505 can be formed from the decoded transport audio signals 1503 by multiplying the decoded transport audio signals on a TF tile basis by a gain factor based on the MASA-to-total energy ratios 1305. For instance if S(k,n,i) is a TF tile of a decoded transport audio signal / channel i , k and n being the frequency / time index of the tile, the (MASA) transport audio signals 1505 can be determined on a per TF tile basis as SMASA(k,n, i) = gMASA(J<,n)S(k,n,i) where the gain gMASA(.k,n) is given as the square root of the MASA-to-total energy ratio for the TF tile k,n. The decoded transport audio signals 1503 are also shown as being passed to the audio object signal signal generator 1506 whose function is to produce an audio signal for each of the remaining audio objects. This can be performed by taking the decoded transport audio signals 1503 and applying on a TF tile basis 1-MASA-to-total energy ratio, resulting in a transport audio signal for each remaining audio object. A prototype audio signal can be generated for each remaining audio object using vector-based amplitude panning (VBAP) using the audio direction and transport audio signal (associated with the remaining audio object). An audio object audio signal for each audio object can then be formed using the audio prototype signal, and the ISM ratios 1509 corresponding to the audio object. In Figure 15 the audio object signals (one for each audio object) are collectively referred to as the remaining audio objects audio signals 1511. With respect to Figure 16, this processing step is shown as 1611. For example, each audio object audio signal, Sobj(k,n, i), where i is the audio object index, can be determined as gISM(k,n,i)Sproto(k,n,i) . The audio object signal gain gISM(k,n, i) can be determined from the MASA-to-total-energy ratio for the TF tile n,k and the ISM ratio for the TF tile n,k and the audio object i. An example of the audio object signal gain may be expressed as gISM(k,n, i) = V((l -rMASAtototal(.k> n)) * rlSM^>n>0)- In some embodiments the gains gISM(k,n,i) and gMAsA(k>n) can be smoothed over time in order to reduce any artifacts. It is to be noted that that other methods can be used to generate the gains for the audio object audio signals 1511 and (MASA) transport audio signals 1503. Finally, each remaining audio object can be formed by associating or attaching the corresponding direction value from the direction values signal stream 1513. The direction values signals can comprise the direction values for all audio objects and is produced as a result of the decoding of the encoded remaining audio objects metadata 1112. The formation of the audio objects takes place in the audio object former 1508. The output signal is therefore the set of remaining audio objects 1515. The step of forming the audio objects is shown as 1613 in Figure 16. The decoder &renderer 129 when operating as the further embodiment in mode C is also arranged to decode the encoded separated audio object 1117 via the audio object decoder 1101 to produce the decoded separated audio object 1061. The output from the decoder &renderer 129 when operating as the further embodiment in mode C with “pass through” decoding comprises the following components: the decoded MASA metadata 1501; (MASA) transport audio signals 1505; remaining audio objects 1515; and the separated audio object 1061. With respect to Figure 17 an example electronic device which may be used as any of the apparatus parts of the system as described above. The device may be any suitable electronics device or apparatus. For example, in some embodiments the device 1400 is a mobile device, user equipment, tablet computer, computer, audio playback apparatus, etc. The device may for example be configured to implement the encoder / analyser part and / or the decoder part as shown in Figures 1,8, 10, 11, 13 and 15 or any functional block as described above. In some embodiments the device 1700 comprises at least one processor or central processing unit 1707. The processor 1707 can be configured to execute various program codes such as the methods such as described herein. In some embodiments the device 1700 comprises at least one memory 1711. In some embodiments the at least one processor 1707 is coupled to the memory 1711. The memory 1711 can be any suitable storage means. In some embodiments the memory 1711 comprises a program code section for storing program codes implementable upon the processor 1707. Furthermore, in some embodiments the memory 1711 can further comprise a stored data section for storing data, for example data that has been processed or to be processed in accordance with the embodiments as described herein. The implemented program code stored within the program code section and the data stored within the stored data section can be retrieved by the processor 1707 whenever needed via the memory-processor coupling. In some embodiments the device 1700 comprises a user interface 1705. The user interface 1705 can be coupled in some embodiments to the processor 1707. In some embodiments the processor 1707 can control the operation of the user interface 1705 and receive inputs from the user interface 1705. In some embodiments the user interface 1705 can enable a user to input commands to the device 1700, for example via a keypad. In some embodiments the user interface 1705 can enable the user to obtain information from the device 1700. For example the user interface 1705 may comprise a display configured to display information from the device 1700 to the user. The user interface 1705 can in some embodiments comprise a touch screen or touch interface capable of both enabling information to be entered to the device 1700 and further displaying information to the user of the device 1700. In some embodiments the user interface 1705 may be the user interface for communicating. In some embodiments the device 1700 comprises an input / output port 1709. The input / output port 1709 in some embodiments comprises a transceiver. The transceiver in such embodiments can be coupled to the processor 1707 and configured to enable a communication with other apparatus or electronic devices, for example via a wireless communications network. The transceiver or any suitable transceiver or transmitter and / or receiver means can in some embodiments be configured to communicate with other electronic devices or apparatus via a wire or wired coupling. The transceiver can communicate with further apparatus by any suitable known communications protocol. For example in some embodiments the transceiver can use a suitable radio access architecture based on long term evolution advanced (LTE Advanced, LTE-A) or new radio (NR) (or can be referred to as 5G), universal mobile telecommunications system (UMTS) radio access network (UTRAN or E-UTRAN), long term evolution (LTE, the same as E-UTRA), 2G networks (legacy network technology), wireless local area network (WLAN or Wi-Fi), worldwide interoperability for microwave access (WiMAX), Bluetooth®, personal communications services (PCS), ZigBee®, wideband code division multiple access (WCDMA), systems using ultra-wideband (UWB) technology, sensor networks, mobile ad-hoc networks (MANETs), cellular internet of things (loT) RAN and Internet Protocol multimedia subsystems (IMS), any other suitable option and / or any combination thereof. The transceiver input / output port 1409 may be configured to receive the signals. In some embodiments the device 1400 may be employed as at least part of the synthesis device. The input / output port 1409 may be coupled to headphones (which may be a headtracked or a non-tracked headphones) or similar and loudspeakers. In general, the various embodiments of the invention may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof. The embodiments of this invention may be implemented by computer software executable by a data processor of the mobile device, such as in the processor entity, or by hardware, or by a combination of software and hardware. Further in this regard it should be noted that any blocks of the logic flow as in the Figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD. The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The data processors may be of any type suitable to the local technical environment, and may include one or more of general-purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASIC), gate level circuits and processors based on multi-core processor architecture, as non-limiting examples. Embodiments of the inventions may be practiced in various components such as integrated circuit modules. The design of integrated circuits is by and large a highly automated process. Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate. Programs, such as those provided by Synopsys, Inc. of Mountain View, California and Cadence Design, of San Jose, California automatically route conductors and locate components on a semiconductor chip using well established rules of design as well as libraries of pre-stored design modules. Once the design for a semiconductor circuit has been completed, the resultant design, in a standardized electronic format (e.g., Opus, GDSII, or the like) may be transmitted to a semiconductor fabrication facility or "fab" for fabrication. As used in this application, the term “circuitry” may refer to one or more or all of the following: (a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and (b) combinations of hardware circuits and software, such as (as applicable): (i) a combination of analog and / or digital hardware circuit(s) with software / firmware and (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation. This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device. The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM). As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements The foregoing description has provided by way of exemplary and non-limiting examples a full and informative description of the exemplary embodiment of this invention. However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of this invention will still fall within the scope of this invention as defined in the appended claims.

Claims

1. An apparatus for decoding an encoded spatial audio signal, wherein the encoded spatial audio signal comprises an encoded audio metadata signal, encoded audio channel signals and encoded information relating to a plurality of audio objects, wherein the apparatus when operating in pass-through mode of decoding comprises means configured to:decode the encoded spatial audio signal to produce audio metadata in a predefined format and audio channel signals; anddecode information relating to the plurality of audio objects to produce a plurality of audio objects.

2. The apparatus as claimed in Claim 1, wherein the means configured to decode information relating to the plurality of audio objects to produce a plurality of audio objects comprises means configured to:determine a number of the plurality of audio objects from the encoded audio metadata signal or from a frame header; andgenerate the number of null audio objects as a substitute for the plurality of audio objects.

3. The apparatus as claimed in Claims 1 and 2, wherein the means configured to decode information relating to the plurality of audio objects to produce a plurality of audio objects further comprises means configured to:decode an encoded separate audio object to give a separate audio object;convert the separate audio object to separate audio object metadata in the predefined format and a separate audio object channel signal;combine the separate audio object metadata in the predefined format with the audio signal metadata in the predefined format to produce combined metadata in the predefined format; andcombine the audio channel signals and the separate audio object channelsignal.

4. The apparatus as claimed in Claims 1 to 3, wherein the means configured to decode information relating to the plurality of audio objects to produce a plurality of audio objects further comprises means configured to:decode encoded metadata for a set of audio objects to produce energy ratios and metadata for the set of audio objects;convert the metadata for the set of audio objects to metadata in the predefined format for the set of audio objects;calculate an energy contribution of the set of audio objects from the metadata in the predefined format for the set of audio objects;calculate, based on the energy ratios, an energy contribution of the audio signal metadata in the predefined format and the audio channel signals;combine the audio signal metadata in the predefined format with the metadata in the predefined format for the set of audio objects to produce combined metadata in the predefined format, wherein the combining is based on the energy contribution of the audio signal metadata in the predefined format and the audio channel signals and the energy contribution of the set of audio objects;decode an encoded separate audio object to give a separate audio object;convert the separate audio object to separate audio object metadata in the predefined format and a separate audio object channel signal;combine the separate audio object metadata in the predefined format with the combined metadata in the predefined format to produce further combined metadata in the predefined format; andcombine the channel audio signals and the separate audio object channel signal.

5. The apparatus as claimed in Claims 2 to 4, wherein a null audio object comprises:an audio object channel signal having zero sample values; andan audio object direction which is a predetermined fixed value.

6. The apparatus as claimed in Claim 1, wherein the means configured to decode information relating to the plurality of audio objects to produce a plurality of audio objects comprises means configured to:decode encoded metadata for a set of audio objects to produce metadata for the set of audio objects, audio object energy ratios and audio signal energy ratios;separate further audio channel signals from the audio channel signals, wherein the separation is based on the audio signal energy ratios;generate an audio object channel signal corresponding to each audio object of the set of audio objects from the audio channel signals, audio object energy ratios and audio signal energy ratios;create an audio object for each audio object channel signal by assigning a corresponding directional value to each of the audio object channel signals, wherein each corresponding directional value is from the metadata for the set of audio objects; anddecode an encoded separate audio object to give a separate audio object.

7. The apparatus as claimed in Claim 6, wherein the means configured to generate an audio object channel signal corresponding to each audio object of the set of audio objects from the audio channel signals, audio object energy ratios and audio signal energy ratios comprises means configured to:generate the audio object channel signal by applying a gain to a vector base amplitude panning generated signal corresponding to the audio object for which the audio object channel signal is generated, wherein the gain is based on audio object energy ratios associated with audio object and a factor of one minus the audio signal energy ratios, and wherein the vector base amplitude panning generated signal is based on the audio channel signals and the direction value associated with the audio object.

8. The apparatus as claimed in Claims 1 to 7, wherein the predefined format is metadata assisted spatial audio (MASA) format.

9. The apparatus as claimed in Claims 6 to 8, wherein the audio object energy ratios are independent stream with metadata (ISM) energy ratios and the audio signal energy ratios are MASA-to-total energy ratios.

10. A method for decoding an encoded spatial audio signal in a pass-through coding mode, wherein the encoded spatial audio signal comprises an encoded audio metadata signal, encoded audio channel signals and encoded information relating to a plurality of audio objects, wherein the method comprises:decoding the encoded spatial audio signal to produce audio metadata in a predefined format and audio channel signals; anddecoding information relating to the plurality of audio objects to produce a plurality of audio objects.

11. The method as claimed in Claim 10, wherein decoding information relating to the plurality of audio objects to produce a plurality of audio objects comprises:determining a number of the plurality of audio objects from the encoded audio metadata signal or from a frame header; andgenerating the number of null audio objects as a substitute for the plurality of audio objects.

12. The method as claimed in Claims 10 and 11, wherein decoding information relating to the plurality of audio objects to produce a plurality of audio objects further comprises:decoding an encoded separate audio object to give a separate audio object;converting the separate audio object to separate audio object metadata in the predefined format and a separate audio object channel signal;combining the separate audio object metadata in the predefined format with the audio signal metadata in the predefined format to produce combined metadata in the predefined format; andcombining the audio channel signals and the separate audio object channelsignal.

13. The method as claimed in Claims 10 to 12, wherein decoding information relating to the plurality of audio objects to produce a plurality of audio objects further comprises:decoding encoded metadata for a set of audio objects to produce energy ratios and metadata for the set of audio objects;converting the metadata for the set of audio objects to metadata in the predefined format for the set of audio objects;calculating an energy contribution of the set of audio objects from the metadata in the predefined format for the set of audio objects;calculating, based on the energy ratios, an energy contribution of the audio signal metadata in the predefined format and the audio channel signals;combining the audio signal metadata in the predefined format with the metadata in the predefined format for the set of audio objects to produce combined metadata in the predefined format, wherein the combining is based on the energy contribution of the audio signal metadata in the predefined format and the audio channel signals and the energy contribution of the set of audio objects;decoding an encoded separate audio object to give a separate audio object;converting the separate audio object to separate audio object metadata in the predefined format and a separate audio object channel signal;combining the separate audio object metadata in the predefined format with the combined metadata in the predefined format to produce further combined metadata in the predefined format; andcombining the channel audio signals and the separate audio object channel signal.

14. The method as claimed in Claims 11 to 13, wherein a null audio object comprises:an audio object channel signal having zero sample values; andan audio object direction which is a predetermined fixed value.

15. The method as claimed in Claim 10, wherein decoding information relating to the plurality of audio objects to produce a plurality of audio objects comprises:decoding encoded metadata for a set of audio objects to produce metadata for the set of audio objects, audio object energy ratios and audio signal energy ratios;separating further audio channel signals from the audio channel signals, wherein the separation is based on the audio signal energy ratios;generating an audio object channel signal corresponding to each audio object of the set of audio objects from the audio channel signals, audio object energy ratios and audio signal energy ratios;creating an audio object for each audio object channel signal by assigning a corresponding directional value to each of the audio object channel signals, wherein each corresponding directional value is from the metadata for the set of audio objects; anddecoding an encoded separate audio object to give a separate audio object.

16. The method as claimed in Claim 15, wherein generating an audio object channel signal corresponding to each audio object of the set of audio objects from the audio channel signals, audio object energy ratios and audio signal energy ratios comprises:generating the audio object channel signal by applying a gain to a vector base amplitude panning generated signal corresponding to the audio object for which the audio object channel signal is generated, wherein the gain is based on audio object energy ratios associated with audio object and a factor of one minus the audio signal energy ratios, and wherein the vector base amplitude panning generated signal is based on the audio channel signals and the direction value associated with the audio object.

17. The method as claimed in Claims 10 to 16, wherein the predefined format is metadata assisted spatial audio (MASA) format.

18. The method as claimed in Claims 15 to 17, wherein the audio object energy ratios are independent stream with metadata (ISM) energy ratios and the audio signal energy ratios are MASA-to-total energy ratios.

Images