Apparatuses and methods for implementing processing information based packets

Abstract

An apparatus comprising means configured to: obtain at least one audio signal; generate at least one audio signal data packet; obtain processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a renderer; generate at least one processing information packet comprising the processing information; and associate the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

Description

[0001] APPARATUSES AND METHODS FOR IMPLEMENTING PROCESSING INFORMATION BASED PACKETS

[0002] Field

[0003] The present application relates to apparatus and methods for implementing processing information based packets, but not exclusively for implementing audio processing information based packets in an immersive voice and audio services (IVAS) real-time transport protocol (RTP) payload.

[0004] Background

[0005] Immersive audio codecs are being implemented supporting a multitude of operating points ranging from a low bit rate operation to transparency. An example of such a codec is the immersive voice and audio services (IVAS) codec which is being designed to be suitable for use over a communications network such as a 3GPP 4G / 5G network. Such immersive services include uses for example in immersive voice and audio for applications such as virtual reality (VR), augmented reality (AR) and mixed reality (MR) as well as spatial voice communication including teleconferencing. This audio codec is expected to handle the encoding, decoding and rendering of speech, music and generic audio. It is furthermore expected to support channel-based audio and scene-based audio inputs including spatial information about the sound field and sound sources. The codec is also expected to operate with low latency to enable conversational services as well as support high error robustness under various transmission conditions.

[0006] The input signals are presented to the IVAS encoder in one of the supported formats (and in some allowed combinations of the formats). Similarly, it is expected that the decoder can output the audio in supported formats. A pass-through mode has been proposed, where the audio could be provided in its original format after transmission (encoding / decoding).

[0007] Additionally RTP (Real-Time Transport Protocol) is intended for an end-to- end, real-time transfer of streaming media and provides facilities for jitter compensation and detection of packet loss and out-of-order delivery. RTP allows data transfer to multiple destinations through IP multicast or to a specific destination through IP unicast. The majority of the RTP implementations are built on top of the User Datagram Protocol (UDP). Other transport protocols may also be utilized. RTP is used in together with other protocols such as H.323 and Real Time Streaming Protocol (RTSP).

[0008] The RTP specification describes two protocols: RTP and RTCP. RTP is used for the transfer of multimedia data, and its companion protocol (RTCP) is used to periodically send control information and QoS (Quality of Service) parameters.

[0009] RTP sessions are typically initiated between client and server or between client and another client (or a multi-party topology) using a signalling protocol, such as H.323, the Session Initiation Protocol (SIP), or RTSP. These protocols typically use the Session Description Protocol (SDP), such as defined by RFC 8866 to specify parameters for the sessions.

[0010] Summary

[0011] There is provided according to a first aspect an apparatus comprising means configured to: obtain at least one audio signal; generate at least one audio signal data packet; obtain processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a Tenderer; generate at least one processing information packet comprising the processing information; and associate the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0012] The means may be further configured to determine a size of the at least one processing information, wherein the means configured to generate at least one processing information packet comprising the processing information may be configured to generate the at least one processing information packet comprising a size field indicating the size of the at least one processing information. The means may be further configured to: determine a type of the processing information, wherein the means configured to generate at least one processing information packet comprising the processing information may be configured to generate the at least one processing information packet comprising a type field indicating a type of the at least one processing information.

[0013] The means may be further configured to: determine a size and a type of the processing information, wherein the means configured to generate at least one processing information packet comprising the processing information may be configured to generate the at least one processing information packet comprising: a size field indicating the size of the at least one processing information; and a type field indicating the type of the at least one processing information.

[0014] The type of the processing information may comprise one of: an orientation indicator parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0015] The means configured to generate at least one processing information packet comprising the processing information may be configured to generate at least one processing information packet comprising at least one of: a usage indicator configured to describe how the processing information should be used; and a validity indicator configured to describe how long the processing information is valid.

[0016] The processing information may comprise one of: an orientation parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0017] The means may be further configured to: generate at least one processing information packet header comprising an indicator identifying that the at least one processing information packet comprises processing information; and append the at least one processing information packet header with the at least one processing information packet in a manner that the at least one processing information packet is identified at a further apparatus as being at least one processing data packet comprising processing information rather than audio data.

[0018] The means configured to append the at least one processing information packet header with the at least one processing information packet may be configured to: append the at least one processing information packet header in a first block and the at least one processing information packet in a separate second block; and append the at least one processing information packet header immediately preceding an associated of the at least one processing information packet.

[0019] The means configured to associate the at least one processing information packet with the at least one audio signal data packet may be configured to append the at least one processing information packet with the at least one audio signal data packet.

[0020] The means configured to associate the at least one processing information packet with the at least one audio signal data packet may be configured to perform one of: append every one of the at least one processing information packet before the at least one audio signal data packet; and append the at least one at least one processing information packet immediately preceding an associated at least one audio signal data packet.

[0021] The at least one audio signal data packet may be an Immersive Voice and Audio Services data packet.

[0022] The means may be further configured to transmit the at least one audio signal data packet and the at least one processing information packet as a real-time transport protocol transmission.

[0023] The means may be further configured to transmit the at least one processing information packet as part of a real-time transport protocol header extension.

[0024] The means may be further configured to transmit the at least one processing information packet as a separate packet according to a real-time transport protocol. The means may be configured to determine whether to transmit the at least one processing information packet as part of a real-time transport protocol header extension or a separate packet according to a real-time transport protocol based on one of: session negotiated preferences; or an agreement indicated by out of band signalling.

[0025] The means may be further configured to signal to at least one further apparatus an inclusion of the at least one processing information packet.

[0026] The means may be further configured to signal to at least one further apparatus at least one supported processing information packet type.

[0027] The means may be further configured to negotiate with at least one further apparatus for at least a session at least one supported processing information packet type.

[0028] The means may be further configured to transmit the at least one processing information packet as part of a session description.

[0029] According to a second aspect there is provided an apparatus comprising means configured to: receive an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and process the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0030] The at least one processing information packet may comprise a size field indicating a size of the at least one processing information.

[0031] The at least one processing information packet may comprise a type field indicating a type of the processing information within the at least one processing information packet and wherein the means configured to process the at least one audio signal based on the processing information may be configured to identify the type of the processing information within at least one processing information packet based on the type field. The at least one processing information packet may comprise: a size field indicating a size of the at least one processing information packet; and a type field indicating a type of the processing information within the at least one processing information packet and wherein the means configured to process the at least one audio signal based on the processing information may be configured to identify the type of the processing information within at least one processing information packet.

[0032] The type of the processing information may comprise one of: an orientation indicator parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0033] The at least one processing information packet may further comprise at least one of: a usage indicator configured to describe how the processing information should be used; and a validity indicator configured to describe how long the processing information is valid with respect to the processing of the at least one succeeding audio signal data packet.

[0034] The processing information may comprise one of: an orientation parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0035] The audio packet stream may comprise further comprise at least one processing information packet header comprising an indicator identifying that the at least one processing information packet comprises processing information.

[0036] The at least one processing information packet header and the at least one processing information packet may be arranged in one of: the at least one processing information packet header in a first block and the at least one processing information packet in a separate second block; and the at least one processing information packet header immediately preceding an associated of the at least one processing information packet.

[0037] The at least one processing information packet and the at least one audio signal data packet may be arranged in one of: every one of the at least one processing information packet before the at least one audio signal data packet; and the at least one at least one processing information packet immediately preceding an associated at least one audio signal data packet.

[0038] The at least one audio signal data packet may be an Immersive Voice and Audio Services data packet.

[0039] The means may be configured to receive the audio packet stream as a realtime transport protocol transmission.

[0040] The means may be further configured to receive the at least one processing information packet as part of a real-time transport protocol header extension.

[0041] The means may be further configured to receive the at least one processing information packet as a separate packet according to a real-time transport protocol.

[0042] The means may be configured to determine whether to receive the at least one processing information packet as part of a real-time transport protocol header extension or a separate packet according to a real-time transport protocol based on one of: session negotiated preferences; or an agreement indicated by out of band signalling.

[0043] The means may be further configured to receive a signal from at least one further apparatus indicating an inclusion of the at least one processing information packet.

[0044] The means may be further configured to receive a signal from at least one further apparatus identifying at least one supported processing information packet type.

[0045] The means may further be configured to retain at least one of the identified supported processing information packet.

[0046] The means may be further configured to negotiate with at least one further apparatus for at least a session at least one supported processing information packet type. The means may be further configured to receive the at least one processing information packet as part of a session description.

[0047] According to a third aspect there is provided a method for an apparatus comprising: obtaining at least one audio signal; generating at least one audio signal data packet; obtaining processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a Tenderer; generating at least one processing information packet comprising the processing information; and associating the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0048] The method may further comprise determining a size of the at least one processing information, wherein generating at least one processing information packet comprising the processing information may comprise generating the at least one processing information packet comprising a size field indicating the size of the at least one processing information.

[0049] The method may further comprise determining a type of the processing information, wherein generating at least one processing information packet comprising the processing information may comprise generating the at least one processing information packet comprising a type field indicating a type of the at least one processing information.

[0050] The method may further comprise determining a size and a type of the processing information, wherein generating at least one processing information packet comprising the processing information may comprise generating the at least one processing information packet comprising: a size field indicating the size of the at least one processing information; and a type field indicating the type of the at least one processing information.

[0051] The type of the processing information may comprise one of: an orientation indicator parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0052] Generating at least one processing information packet comprising the processing information may comprise generating at least one processing information packet comprising at least one of: a usage indicator configured to describe how the processing information should be used; and a validity indicator configured to describe how long the processing information is valid.

[0053] The processing information may comprise one of: an orientation parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0054] The method may further comprise: generating at least one processing information packet header comprising an indicator identifying that the at least one processing information packet comprises processing information; and appending the at least one processing information packet header with the at least one processing information packet in a manner that the at least one processing information packet is identified at a further apparatus as being at least one processing data packet comprising processing information rather than audio data.

[0055] Appending the at least one processing information packet header with the at least one processing information packet may comprise: appending the at least one processing information packet header in a first block and the at least one processing information packet in a separate second block; and appending the at least one processing information packet header immediately preceding an associated of the at least one processing information packet.

[0056] Associating the at least one processing information packet with the at least one audio signal data packet may comprise appending the at least one processing information packet with the at least one audio signal data packet. Associating the at least one processing information packet with the at least one audio signal data packet may comprise one of: appending every one of the at least one processing information packet before the at least one audio signal data packet; or appending the at least one at least one processing information packet immediately preceding an associated at least one audio signal data packet.

[0057] The at least one audio signal data packet may be an Immersive Voice and Audio Services data packet.

[0058] The method may further comprise transmitting the at least one audio signal data packet and the at least one processing information packet as a real-time transport protocol transmission.

[0059] Transmitting the at least one processing information packet may comprise transmitting the at least one processing information packet as part of a real-time transport protocol header extension.

[0060] Transmitting the at least one processing information packet may comprise transmitting the at least one processing information packet as a separate packet according to a real-time transport protocol.

[0061] The method may further comprise determining whether to transmit the at least one processing information packet as part of a real-time transport protocol header extension or a separate packet according to a real-time transport protocol based on one of: session negotiated preferences; or an agreement indicated by out of band signalling.

[0062] The method may further comprise signalling to at least one further apparatus an inclusion of the at least one processing information packet.

[0063] The method may further comprise signalling to at least one further apparatus at least one supported processing information packet type.

[0064] The method may further comprise negotiating with at least one further apparatus for at least a session at least one supported processing information packet type.

[0065] The method may be further configured to transmit the at least one processing information packet as part of a session description. According to a fourth aspect there is provided a method for an apparatus comprising: receiving an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and processing the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0066] The at least one processing information packet may comprise a size field indicating a size of the at least one processing information.

[0067] The at least one processing information packet may comprise a type field indicating a type of the processing information within the at least one processing information packet and wherein processing the at least one audio signal based on the processing information may comprise identifying the type of the processing information within at least one processing information packet based on the type field.

[0068] The at least one processing information packet may comprise: a size field indicating a size of the at least one processing information packet; and a type field indicating a type of the processing information within the at least one processing information packet and wherein processing the at least one audio signal based on the processing information may comprise identifying the type of the processing information within at least one processing information packet.

[0069] The type of the processing information may comprise one of: an orientation indicator parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0070] The at least one processing information packet may further comprise at least one of: a usage indicator configured to describe how the processing information should be used; and a validity indicator configured to describe how long the processing information is valid with respect to the processing of the at least one succeeding audio signal data packet.

[0071] The processing information may comprise one of: an orientation parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0072] The audio packet stream may comprise further comprise at least one processing information packet header comprising an indicator identifying that the at least one processing information packet comprises processing information.

[0073] The at least one processing information packet header and the at least one processing information packet may be arranged in one of: the at least one processing information packet header in a first block and the at least one processing information packet in a separate second block; and the at least one processing information packet header immediately preceding an associated of the at least one processing information packet.

[0074] The at least one processing information packet and the at least one audio signal data packet may be arranged in one of: every one of the at least one processing information packet before the at least one audio signal data packet; and the at least one at least one processing information packet immediately preceding an associated at least one audio signal data packet.

[0075] The at least one audio signal data packet may be an Immersive Voice and Audio Services data packet.

[0076] The audio packet stream may be received as a real-time transport protocol transmission.

[0077] Receiving the audio packet stream comprising the at least one processing information packet may comprise receiving the at least one processing information packet as part of a real-time transport protocol header extension.

[0078] Receiving the at least one the audio packet stream comprising the at least one processing information packet may comprise receiving the at least one processing information packet as a separate packet according to a real-time transport protocol.

[0079] The method may further comprise determining whether to receive the audio packet stream comprising the at least one processing information packet as part of a real-time transport protocol header extension or a separate packet according to a real-time transport protocol based on one of: session negotiated preferences; or an agreement indicated by out of band signalling.

[0080] The method may further comprise receiving a signal from at least one further apparatus indicating an inclusion of the at least one processing information packet.

[0081] The method may further comprise receiving a signal from at least one further apparatus identifying at least one supported processing information packet type.

[0082] The method may further comprise retaining at least one of the identified supported processing information packet.

[0083] The method may further comprise negotiating with at least one further apparatus for at least a session at least one supported processing information packet type.

[0084] The method may further comprise receiving the at least one processing information packet as part of a session description.

[0085] According to a fifth aspect there is provided an apparatus comprising at least one processor and at least one memory including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: obtain at least one audio signal; generate at least one audio signal data packet; obtain processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a Tenderer; generate at least one processing information packet comprising the processing information; and associate the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet. The apparatus may be further caused to determine a size of the at least one processing information, wherein the apparatus caused to generate at least one processing information packet comprising the processing information may be caused to generate the at least one processing information packet comprising a size field indicating the size of the at least one processing information.

[0086] The apparatus may be further caused to: determine a type of the processing information, wherein the apparatus caused to generate at least one processing information packet comprising the processing information may be caused to generate the at least one processing information packet comprising a type field indicating a type of the at least one processing information.

[0087] The apparatus may be further caused to: determine a size and a type of the processing information, wherein the apparatus caused to generate at least one processing information packet comprising the processing information may be caused to generate the at least one processing information packet comprising: a size field indicating the size of the at least one processing information; and a type field indicating the type of the at least one processing information.

[0088] The type of the processing information may comprise one of: an orientation indicator parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0089] The apparatus caused to generate at least one processing information packet comprising the processing information may be caused to generate at least one processing information packet comprising at least one of: a usage indicator configured to describe how the processing information should be used; and a validity indicator configured to describe how long the processing information is valid.

[0090] The processing information may comprise one of: an orientation parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0091] The apparatus may be further caused to: generate at least one processing information packet header comprising an indicator identifying that the at least one processing information packet comprises processing information; and append the at least one processing information packet header with the at least one processing information packet in a manner that the at least one processing information packet is identified at a further apparatus as being at least one processing data packet comprising processing information rather than audio data.

[0092] The apparatus caused to append the at least one processing information packet header with the at least one processing information packet may be caused to: append the at least one processing information packet header in a first block and the at least one processing information packet in a separate second block; and append the at least one processing information packet header immediately preceding an associated of the at least one processing information packet.

[0093] The apparatus caused to associate the at least one processing information packet with the at least one audio signal data packet may be caused to append the at least one processing information packet with the at least one audio signal data packet.

[0094] The apparatus caused to associate the at least one processing information packet with the at least one audio signal data packet may be caused to perform one of: associate every one of the at least one processing information packet before the at least one audio signal data packet; or append the at least one at least one processing information packet immediately preceding an associated at least one audio signal data packet.

[0095] The at least one audio signal data packet may be an Immersive Voice and Audio Services data packet.

[0096] The apparatus may be further caused to transmit the at least one audio signal data packet and the at least one processing information packet as a real-time transport protocol transmission. The apparatus may be caused to transmit the at least one processing information packet as part of a real-time transport protocol header extension.

[0097] The apparatus may be caused to transmit the at least one processing information packet as a separate packet according to a real-time transport protocol.

[0098] The apparatus may be further caused to transmit the at least one processing information packet as part of a real-time transport protocol header extension.

[0099] The apparatus may be further caused to transmit the at least one processing information packet as a separate packet according to a real-time transport protocol.

[0100] The apparatus may be caused to determine whether to transmit the at least one processing information packet as part of a real-time transport protocol header extension or a separate packet according to a real-time transport protocol based on one of: session negotiated preferences; or an agreement indicated by out of band signalling.

[0101] The apparatus may be caused to signal to at least one further apparatus an inclusion of the at least one processing information packet.

[0102] The apparatus may be caused to signal to at least one further apparatus at least one supported processing information packet type.

[0103] The apparatus may be caused to negotiate with at least one further apparatus for at least a session at least one supported processing information packet type.

[0104] The apparatus may be caused to transmit the at least one processing information packet as part of a session description. According to a sixth aspect there is provided an apparatus comprising at least one processor and at least one memory including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: receive an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and process the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0105] The at least one processing information packet may comprise a size field indicating a size of the at least one processing information.

[0106] The at least one processing information packet may comprise a type field indicating a type of the processing information within the at least one processing information packet and wherein the apparatus caused to process the at least one audio signal based on the processing information may be caused to identify the type of the processing information within at least one processing information packet based on the type field.

[0107] The at least one processing information packet may comprise: a size field indicating a size of the at least one processing information packet; and a type field indicating a type of the processing information within the at least one processing information packet and wherein the apparatus caused to process the at least one audio signal based on the processing information may be caused to identify the type of the processing information within at least one processing information packet.

[0108] The type of the processing information may comprise one of: an orientation indicator parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0109] The at least one processing information packet may further comprise at least one of: a usage indicator configured to describe how the processing information should be used; and a validity indicator configured to describe how long the processing information is valid with respect to the processing of the at least one succeeding audio signal data packet.

[0110] The processing information may comprise one of: an orientation parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

[0111] The audio packet stream may comprise further comprise at least one processing information packet header comprising an indicator identifying that the at least one processing information packet comprises processing information.

[0112] The at least one processing information packet header and the at least one processing information packet may be arranged in one of: the at least one processing information packet header in a first block and the at least one processing information packet in a separate second block; and the at least one processing information packet header immediately preceding an associated of the at least one processing information packet.

[0113] The at least one processing information packet and the at least one audio signal data packet may be arranged in one of: every one of the at least one processing information packet before the at least one audio signal data packet; and the at least one at least one processing information packet immediately preceding an associated at least one audio signal data packet.

[0114] The at least one audio signal data packet may be an Immersive Voice and Audio Services data packet.

[0115] The apparatus may be caused to receive the audio packet stream as a realtime transport protocol transmission.

[0116] The apparatus may be further caused to receive the at least one processing information packet as part of a real-time transport protocol header extension.

[0117] The apparatus may be further caused to receive the at least one processing information packet as a separate packet according to a real-time transport protocol.

[0118] The apparatus may be caused to determine whether to receive the at least one processing information packet as part of a real-time transport protocol header extension or a separate packet according to a real-time transport protocol based on one of: session negotiated preferences; or an agreement indicated by out of band signalling. The apparatus may be further caused to receive a signal from at least one further apparatus indicating an inclusion of the at least one processing information packet.

[0119] The apparatus may be further caused to receive a signal from at least one further apparatus identifying at least one supported processing information packet type.

[0120] The apparatus may be further caused to retain at least one of the identified supported processing information packet.

[0121] The apparatus may be further caused to negotiate with at least one further apparatus for at least a session at least one supported processing information packet type.

[0122] The apparatus may be further caused to receive the at least one processing information packet as part of a session description.

[0123] According to a seventh aspect there is provided an apparatus comprising: obtaining circuitry configured to obtain at least one audio signal; generating circuitry configured to generate at least one audio signal data packet; obtaining circuitry configured to obtain processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a Tenderer; generating circuitry configured to generate at least one processing information packet comprising the processing information; and appending circuitry configured to associate the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0124] According to an eighth aspect there is provided an apparatus comprising receiving circuitry configured to receive an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and processing circuitry configured to process the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0125] According to a ninth aspect there is provided a computer program comprising instructions [or a computer readable medium comprising program instructions] for causing an apparatus to perform at least the following: obtaining at least one audio signal; generating at least one audio signal data packet; obtaining processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a Tenderer; generating at least one processing information packet comprising the processing information; and associating the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0126] According to a tenth aspect there is provided a computer program comprising instructions [or a computer readable medium comprising program instructions] for causing an apparatus to perform at least the following: receiving an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and processing the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0127] According to an eleventh aspect there is provided a non-transitory computer readable medium comprising program instructions for causing an apparatus to perform at least the following: obtaining at least one audio signal; generating at least one audio signal data packet; obtaining processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a Tenderer; generating at least one processing information packet comprising the processing information; and associating the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0128] According to a twelfth aspect there is provided a non-transitory computer readable medium comprising program instructions for causing an apparatus to perform at least the following: receiving an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and processing the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0129] According to a thirteenth aspect there is provided an apparatus comprising: means for obtaining at least one audio signal; means for generating at least one audio signal data packet; obtaining processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a Tenderer; means for generating at least one processing information packet comprising the processing information; and means for associating the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0130] According to a fourteenth aspect there is provided an apparatus comprising: means for receiving an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and means for processing the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0131] According to a fifteenth aspect there is provided a computer readable medium comprising program instructions for causing an apparatus to perform at least the following: obtaining at least one audio signal; generating at least one audio signal data packet; obtaining processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a Tenderer; generating at least one processing information packet comprising the processing information; and associating the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0132] According to a sixteenth aspect there is provided a computer readable medium comprising program instructions for causing an apparatus to perform at least the following: receiving an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and processing the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

[0133] An apparatus comprising means for performing the actions of the method as described above.

[0134] An apparatus configured to perform the actions of the method as described above.

[0135] A computer program comprising program instructions for causing a computer to perform the method as described above. A computer program product stored on a medium may cause an apparatus to perform the method as described herein.

[0136] An electronic device may comprise apparatus as described herein.

[0137] A chipset may comprise apparatus as described herein.

[0138] Embodiments of the present application aim to address problems associated with the state of the art.

[0139] Summary of the Figures

[0140] For a better understanding of the present application, reference will now be made by way of example to the accompanying drawings in which:

[0141] Figure 1 shows schematically example server and peer-to-peer teleconferencing systems within which embodiments may be implemented;

[0142] Figure 2 shows schematically an example packet format with Enhanced Voice Services (EVS) Header-Full payload structure;

[0143] Figure 3 shows schematically an example encoder configuration as shown in Figure 1 according to some embodiments;

[0144] Figure 4 shows a flow diagram showing a method of operation for the encoder as shown in Figure 3 according to some embodiments;

[0145] Figure 5 shows schematically example Header-Full IVAS payload structures according to some embodiments;

[0146] Figures 6a and 6b show schematically example Table of Content (ToC) byte structures in EVS Header-Full frame and Header-Full IVAS frame according to some embodiments;

[0147] Figure 7 shows schematically example payload structures of processing information (PI) frames with and without Header-Full IVAS frames according to some embodiments;

[0148] Figure 8 shows schematically example payload structures of multiple frames of Header-Full IVAS and PI frames in the same RTP packet according to some embodiments;

[0149] Figure 9 shows schematically examples of different PI frame structures according to some embodiments; Figures 10a and 10b show schematically example structures of a ‘Type’ identifier in an PI frame according to some embodiments;

[0150] Figure 11 shows schematically examples of orientation frame structures according to some embodiments;

[0151] Figure 12 shows schematically an example ISM audio object priority frame structure according to some embodiments;

[0152] Figure 13 shows schematically an example of a priority frame for IVAS input format according to some embodiments;

[0153] Figure 14 shows schematically an example of an active speaker label structure according to some embodiments;

[0154] Figure 15 shows schematically examples of ISM labels according to some embodiments;

[0155] Figure 16 shows schematically an example decoder configuration as shown in Figure 1 according to some embodiments;

[0156] Figure 17 shows a flow diagram showing a method of operation for the decoder as shown in Figure 16 according to some embodiments; and

[0157] Figure 18 shows an example device suitable for implementing the apparatus shown.

[0158] Embodiments of the Application

[0159] The following describes in further detail suitable apparatus and possible mechanisms for the provision of efficient IVAS audio.

[0160] An example system within which embodiments may be implemented is shown in Figure 1 .

[0161] Figure 1 , for example, shows an example teleconferencing system within which some embodiments can be implemented. In this example there is shown two sites or rooms, Room A 100 and Room B 102. Room A 100 comprises a ‘talker’ or user, Talker 1 103. Room B 102 comprises one ‘talker’ or user, Talker RX 141.

[0162] In the following example within room A is a suitable teleconference apparatus (or more generally telecommunications apparatus 110) configured to spatially capture and encode the audio environment and furthermore is configured to render a spatial audio signal to the room. Within each of the other rooms may be a suitable teleconference apparatus (or more generally telecommunications apparatus such as apparatus 120 within room B) configured to render a spatial audio signal to the room and furthermore is configured to capture and encode at least a mono audio and optionally configured to spatially capture and encode the audio environment.

[0163] In the following examples each room is provided with the means to spatially capture, encode spatial audio signals, receive spatial audio signals and render these to a suitable listener. It would be understood that there may be other embodiments where the system comprises some apparatus configured to only capture and encode audio signals (in other words the apparatus is a ‘transmit’ only apparatus), and other apparatus configured to only receive and render audio signals (in other words the apparatus is a ‘receive’ only apparatus). In such embodiments the system within which embodiments may be implemented may comprise apparatus with varying abilities to capture / render audio signals.

[0164] The teleconference apparatus (for each site or room) 110, 120 can be configured to call into a teleconference controlled by and implemented over a server 111.

[0165] In some embodiments the communications or teleconferencing system comprises a (peer-to-peer) communications system (rather than the server based system shown in Figure 1 ) within which some embodiments can be implemented. Thus, for example, two or more UEs can be configured to interact directly with each other (for example to implement an immersive audio phone call between users). In such a scenario one of the UEs can be configured to deliver spatial ambience as one stream and employ a close-up microphone (for example a Lavalier microphone) to capture the speech as an audio object or audio source. The sender UE can be configured to encode the spatial ambience audio signals in a MASA format stream and the close-up microphone audio signal as an object format stream. The two audio streams can then be delivered as separated IVAS streams. The sender UE, in addition, can be configured to encode processing information during the encoding to deliver the PI frames together with the IVAS frames to the receiver UE. The teleconference apparatus can be configured to spatially capture and encode the audio environment and furthermore can be configured to render a spatial audio signal to the room. In this example only the communications or signalling path from the Room A 100 to the Room B 102 is shown for simplicity but a duplex or multipoint communication system comprising multiple signalling paths can be implemented using the methods as described herein without significant inventive input.

[0166] The teleconference apparatus (for each site or room) 110, 120 is further configured to communicate with each other to implement a teleconference function.

[0167] As shown in Figure 1 , the apparatus 110, 120 and server 111 can comprise suitable encoder and decoder functionality. For example the apparatus 110 is shown comprising an (IVAS) encoder 101 , the server 111 is shown comprising a (IVAS) decoder and encoder 121 and the apparatus 120 is shown comprising an (IVAS) decoder 131. In such a manner an object 120 (the audio signals representing the user or talker 1 111 ) can be encoded by the encoder 101 which generates a bitstream 106 to be passed to a server 111. The server 111 can then decode, (optionally then mix with other objects and otherwise process the audio signals) and encode then to generate the bitstream 108 to be passed to the apparatus 120. The apparatus 120 can then decode the audio signals and present them to the user or talker ‘Talker RX’ 141.

[0168] Although this example shows a teleconference application the encoder / decoder functionality can be applied to the streaming of any suitable media.

[0169] The IVAS decoder / renderer for each of the teleconference apparatus 102 can be furthermore configured to handle multiple input streams that may each originate from a different encoder.

[0170] As discussed previously RTP is intended for an end-to-end, real-time transfer of streaming media and provides facilities for jitter compensation and detection of packet loss and out-of-order delivery. RTP is furthermore designed to carry a multitude of multimedia formats, which permit the transport of new formats without revising the RTP standard. To this end, the information required by a specific application of the protocol is not included in the generic RTP header. For a class of applications (e.g., audio, video), an RTP profile may be defined. For a media format (e.g., a specific video coding format), an associated RTP payload format may be defined. Every instantiation of RTP in a particular application may therefore require a profile and payload format specifications.

[0171] The profile is configured to define the codec used to encode the payload data and the mapping to payload format codes in the protocol field Payload Type (PT) of the RTP header.

[0172] For example, the RTP profile for audio and video conferences with minimal control is defined in RFC 3551. The profile defines a set of static payload type assignments, and a dynamic mechanism for mapping between a payload format, and a PT value using Session Description Protocol (SDP). The latter mechanism is used for newer video codec such as RTP payload format for H.264 Video defined in RFC 6184 or RTP Payload Format for High Efficiency Video Coding (HEVC) defined in RFC 7798.

[0173] An RTP session can be established for each multimedia stream. Audio and video streams may be implemented which use separate RTP sessions, enabling a receiver to selectively receive components of a particular stream. The RTP specification can furthermore be configured to recommend port numbers for RTP, and furthermore to recommend the use of the next odd port number for the associated RTCP session. A single port can be used for RTP and RTCP in applications that multiplex the protocols.

[0174] Each RTP stream can comprise RTP packets, and the RTP packet in turn can comprise a RTP header and payload pair.

[0175] Enhanced Voice Services (EVS) is a mono voice codec standardized in 3GPP and described in the TS 26.445 specification document. The codec can have two operating modes: EVS Primary and EVS AMR-WB IO (Adaptive Multi Rate Wideband Inter-Operable).

[0176] The IVAS codec can be considered to be an extension to the EVS codec and as such the IVAS and EVS codecs can have some similarities in terms of design and implementation. The RTP payload structure, while not having been specified for the IVAS codec, is envisioned to have similarities to the RTP payload structure in EVS.

[0177] The RTP payload format of EVS is described in 3GPP TS 26.445 Annex A. In EVS, the RTP payload format is divided into two different embodiments: a Compact format and a Header-Full format. In the EVS Compact payload format, a RTP packet includes only a single EVS speech frame for EVS Primary mode. For EVS AMR-WB IO mode, the compact RTP packet also includes a 3-bit Codec Mode Request (CMR) field in front of the speech frame. In the EVS Compact format, the different modes and bitrates for the speech frames are identified by the size of the RTP payload. For example, an RTP packet of size 328 bits is assigned for EVS Primary mode with 16.4 kbps bitrate, as is shown in Table A.1 in TS 26.445 Annex A.

[0178] In EVS Header-Full format, the RTP payload consists of the speech frame(s) accompanied by an optional CMR byte and Table of Content (ToC) bytes. The CMR byte is used to request a change in bitrate or coding mode that the receiver wants to receive. The request is sent as a CMR byte as part of the Header-Full EVS packet. In EVS AMR-WB IO Compact format, the CMR functionality is also present as a 3- bit signaling at the beginning of the packet.

[0179] The ToC bytes in EVS Header-Full format describe the mode and bitrate for the accompanied EVS coded speech frames. RTP payload structures for EVS Header-Full format are furthermore shown in Figure 2.

[0180] Figure 2 shows, for example, a payload structure of Header-Full format with ToC single frame 201 , a payload structure of Header-Full format with ToC multiple frames 203, a payload structure of Header-Full format with CMR and ToC single frame 205 and a payload structure of Header-Full format with CMR and ToC multiple frames 207.

[0181] Each of the payload structures comprises ToC / CMR bytes where for each of these bytes there is a first bit 211 , 221 , 231 at the beginning of the ToC / CMR bytes which can be used to differentiate the bytes between ToC (where the bit value is 0) and CMR (where the bit value is 1 ). In the payload structure 201 where there is a single speech frame in the payload, the speech data 215 is preceded by a single ToC byte 213 (with optional zero padding 217 at the end of the payload).

[0182] In examples where the single speech frame payload structure can have an optional CMR byte, such as shown in payload structure 205. The payload structure 205 differs from the payload structure 201 in that there is a CMR byte 232 (with associated indicator bit 231 value of 1 ) positioned before the ToC byte 213.

[0183] In some example structures there can be two speech frames in the payload structure 203, 207. In this example an additional ToC byte 223 (with associated indicator bit 221 value of 0) and the ToC byte 213 is positioned at the beginning of the payload followed by the speech data frames 215 and 225 respectively to the order of the ToC bytes.

[0184] Furthermore the multiple speech frame payload structure can have an optional CMR byte, such as shown in payload structure 207. The payload structure 207 differs from the payload structure 203 in that there is a CMR byte 232 (with associated indicator bit 231 value of 1 ) positioned before the ToC bytes 213, 223.

[0185] The IVAS codec supports stereo and several spatial / immersive formats at wide range of bitrates. Particularly at the lowest IVAS bitrates (as low as 13.2 kbps) the encoding quality of the audio signals is reduced due to insufficient bitrate. There is thus generally a very limited capability to provide audio processing, rendering related, or even non-audio related, auxiliary information within the codec payload.

[0186] There following embodiments thus describe apparatus and methods which enable delivery of information which is relevant for consuming the delivered audio bitstream, in addition to the coded IVAS bitstream. Such examples are aimed to precisely denote the related one or more coded audio frame(s) with the relevant processing information but allow for any subsequent extension based on user or application defined requirements.

[0187] Consequently in some embodiments there is the ability to send additional information as part of IVAS RTP packets / stream to facilitate rendering or consumption of the IVAS audio bitstream data. For example, the information can comprise information such as orientation of the sender, priority information for ISM objects, or other non-audio information. In some implementations, the processing information can be signalled as part of the RTP header extension, the use of either one or two byte header is decided based on the amount of data to be carried as part of the processing information packet.

[0188] The concept thus as discussed in the embodiments herein in further detail is one in which apparatus and methods for transmission of (audio) processing information (PI) comprise suitable means for providing the audio processing information as RTP payload packets in addition to the immersive audio coded data packets (IVAS frames) and furthermore in the same RTP stream carrying the IVAS frames. In such embodiments the described RTP payload structures aim to enable in band delivery of PI packets while achieving compact header size for delivery of immersive audio coded data (IVAS frames) for precise bitrate allocation networks and enabling generic extension support for over-the-top (OTT) delivery networks without precise bitrate allocation constraints. The compact header size for IVAS is retained by encapsulating the processing information as a separate RTP packet or carried as a separate packet in the RTP header extension. Thus, not extending the IVAS header which is required for all IVAS frames.

[0189] In some embodiments this can be achieved with respect to a suitable encoder which is configured to receive audio processing information (and furthermore in some embodiments receive update frequency information and type of audio processing information). The audio processing information can be employed by a suitable decoder or Tenderer apparatus to assist in the processing of the encoded and packetized audio signals (for example the audio signals within the IVAS audio data packets). The assistance can be as simple as being able to label sources and / or identify an ordering of audio sources or audio objects. Furthermore the assistance can enable or control a modification of the processing or rendering of the audio signals. Then, as discussed in further detail herein, for the RTP packet corresponding to the update frequency there is a packetizing of the audio processing information with the ToC header identifying the type of packet. Furthermore the PI type header comprising a predetermined number of bytes can then be appended. In some PI types, the payload size follows the PI type header. Then the PI payload can be appended to the PI pay load packet.

[0190] In some embodiments the inclusion of the processing information packet in the RTP stream is implicitly negotiated or signalled. For example, the IVAS codec can be configured to implicitly indicate the application or use of processing information packets. In some embodiments, the specific processing information packets are explicitly indicated as part of the session negotiation. Furthermore, whether the processing information is packetized as part of the RTP header extension or delivered as a separate packet can be decided by specific session negotiated preferences or agreed via any suitable out of band signalling.

[0191] In other embodiments, the sender UE or SDP offer creator can be configured to indicate the processing information packets supported for packetization. In some examples, the receiver can be configured to select which of the processing information packets are supported. The supported information packets can then be retained.

[0192] Similarly, in some embodiments a session negotiation can include whether the processing information packet is delivered as part of a RTP HE or delivered as a separate packet. Depending on the implementation preferences, the sender UE and the receiver UE can agree on a mutually acceptable approach.

[0193] In further embodiments, the processing information packet can be carried as part of the session description. This is more suitable for scenarios where the processing information is expected to be less dynamic or at some values are desired to be the default starting values.

[0194] For example, the session parameter can be an attribute used for the IVAS media stream. a=pi-frame-included; / / indicates PI frames are to be used a=pi-frame-included:mode; / / mode can be 0 equals separate RTP PI frames or PI frames appended to IVAS frames; mode value of 1 indicates the PI frame data is carried as part of RTP header extension. a=pi-frame-included:mode:<list of comma separated or semicolon separate PI packet types listed> / / <list of PI packet types where each packet type is listed>

[0195] With respect to the decoder, in some embodiments, the RTP packets are received and then based on the ToC byte the payload type is determined.

[0196] The embodiments described herein can be employed by sending the (audio) processing information (PI) frames as separate payload frames, in addition to the coded audio frames as part of the IVAS RTP packet stream. The frames can be identified or disambiguated from the coded audio frames with the help of a Table of Content (ToC) byte located prior to or in front of the frames. The PI frames can in some embodiments be sent as separate IVAS RTP packets (as in, no IVAS speech frames in the same packets with the PI frames) or PI frames together with other IVAS frames in the same IVAS RTP payload packet. The PI frames can in some embodiments be identified in a manner similar to that described above, where the identification is provided with the help of the ToC byte. Furthermore, as described earlier, the PI frames can be delivered as part of the RTP header extension for the IVAS frames.

[0197] As discussed above it is envisioned that IVAS, in a manner similar to EVS, will have two different formats for the RTP packets: Compact and Header-Full format.

[0198] In the EVS Compact format, the RTP packets include only the encoded EVS speech frames. In the IVAS Compact format, a similar behaviour is defined, where the RTP packets include only the encoded IVAS frames. Payload differentiation can in some embodiments be based on unique packet size defining the bit rate.

[0199] In the EVS Header-Full format, the speech frames are accompanied with Table of Content (ToC) bytes that describe the used mode and bitrate for the frame. Additionally, the packet can include Codec Mode Request (CMR) bytes which are used to change the coding at the receiver / decoder side. In the IVAS Header-Full format, a modified ToC byte is included in the RTP packet, as described hereafter.

[0200] Additionally in EVS, the CMR byte can be employed used to request a change in bitrate or coding mode that the receiver wants to receive. The request can be sent as a CMR byte as part of the Header-Full EVS packets. In EVS AMR- WB 10 Compact format, the CMR functionality can also present as a 3-bit signalling at the beginning of the packet such as described in EVS description [TS 26.445, Annex A],

[0201] In the following embodiments the ToC byte can be modified in IVAS applications to indicate the availability of any additional (audio) processing information (PI) within the data stream and furthermore identify the type of the additional (audio) processing information (PI) within the data stream.

[0202] The (IVAS) encoder according to some embodiments, such as the example encoder 101 , is shown in Figure 3.

[0203] The encoder in some embodiment is configured to receive or otherwise obtain the audio processing information 300, the update frequency 302 and the audio processing type information 304. Additionally the encoder comprises a RTP generator 301 which is configured to receive the audio processing information 300, the update frequency 302 and the audio processing type information 304 and generate the RTP packets comprising the PI payload.

[0204] The RTP generator 301 , in some embodiments, comprises an audio processing information packetizer (where the ToC header identifies the type of packet) 303.

[0205] The RTP generator 301 , in some embodiments, further comprises an PI type header appender 305 configured to append the PI type information to the packet.

[0206] The RTP generator 301 , in some embodiments, further comprises an PI payload appender 307. The PI payload appender 307 is configured to append the PI payload to the packet prior to sending the generated packets.

[0207] Figure 4 shows the operations of the example RTP generator shown in Figure 3.

[0208] The first operation is one of obtaining or receiving audio processing information, update frequency and type of audio processing information as shown in Figure 4 by step 401 .

[0209] Then for the RTP packet corresponding to the update frequency a packet is generated as shown in Figure 4 by step 403. The packet generation operation of step 403 can for example be divided into a first operation of packetizing the audio processing information with the ToC header identifying the type of packet as shown by step 405.

[0210] The packet generation operation of step 403 can further comprise appending a PI type header comprising a predetermined number of bytes. In some PI types, the payload size follows the PI type as shown by step 407.

[0211] The packet generation operation of step 403 further comprises appending PI payload to the PI payload packet as shown by step 409.

[0212] The Header-Full IVAS frames comprise one or more than one Table-of- Content (ToC) indicators and their associated IVAS speech frames. Figure 5 shows an example packet structure of Header-Full IVAS payload format.

[0213] A first example packet structure is shown in 501 , which represents a payload structure when a single IVAS frame is coded in the Header-Full format. In this example the IVAS speech frame 505 is preceded by a single ToC byte 503.

[0214] Furthermore example packet structures are shown 511 and 521 which represent alternative versions of the payload structure when two IVAS frames are coded in the Header-Full format.

[0215] Thus a first multiple IVAS frame packet structure 511 comprises one in which all the ToC bytes 513, 514 are positioned at the beginning of the payload and before the IVAS frames 515, 516.

[0216] A further multiple IVAS frame packet structure 521 comprises the ToC bytes 523, 527 can be positioned before each respective or associated IVAS frame 525 and 529.

[0217] The example packet structures 501 and 511 are similar to the packet structures for the EVS codec described in TS 26.445, where all the EVS ToC bytes precede the EVS speech frames. The ToC bytes in EVS and IVAS are different, but they both describe the content of the speech frames in the packet and have similar higher-level functionality in the RTP packet.

[0218] An example of the ToC byte structure in EVS and IVAS Header-Full frame ToC byte structure according to some embodiments is shown in Figures 6a and 6b. With respect to Figure 6a the EVS Header-Full frame ToC byte structure 601 according to some embodiments is shown. The Header Type (H) identification bit 603 is used to differentiate between CMR and ToC bytes, the (F) bit 605 is used to indicate if another ToC byte follows this frame and the Frame type (FT) index 607 is used to indicate the EVS bitrate and mode of the frame.

[0219] Similarly Figure 6b shows the ToC byte structure 651 for an IVAS frame. Compared to the EVS ToC byte structure the (H) bit is omitted because it is to be expected that the CMR byte is not part of the IVAS packets. The ToC indicators describe the data in the IVAS speech frames which can be used to derive the content and size of each frame.

[0220] The bits in the IVAS ToC byte structure 651 can therefore in some embodiments have the following structure:

[0221] F (1 bit) 653: If set to 1 , the bit indicates that the corresponding frame is followed by another speech or PI frame in this payload, implying that another ToC byte follows this entry. If set to 0, the bit indicates that this frame is the last frame in this payload and no further header entry follows this entry.

[0222] Extra (2-3 bits) 655 only when Extra is 2 bits or 655 and 657 when Extra is 3 bits: These can be bits reserved for future use. If 4 bits are reserved for the FT indicator, the extra 3 bits can be partly used to identify other frame content than bitrates / frame-size (e.g., SPEECH_LOST, NO_DATA, SID) as demonstrated below.

[0223] FT (4-5 bits) 659 only when FT is 4 bits or 659 and 657 when FT is 5 bits: Frame type index. These bits can be configured to indicate the bitrate or other frame content indication for the frame. From the content indication, the receiver can determine the size of the received frame either directly from the bitrate or from predetermined frame sizes, e.g., for the SPEECH_LOST and NO_DATA frames. The FT bits can indicate, for example, the bitrate of an IVAS speech frame, a SPEECH_LOST, NO_DATA or comfort noise (SID) frames.

[0224] An example for FT bit values (for 5 bits) are presented in the following table, this leads to more extension bits. At this point of the codec development, it is not decided if the aforementioned three types (SPEECH_LOST, NO_DATA, SID) are supported in the final codec. The SPEECH_LOST, NO_DATA and SID frames are part of the EVS codec and at it is likely that at least the SID and NO_DATA frame types are being incorporated into the IVAS specification. It is understood that the final IVAS codec specification, however, might have different frame types present than presented here.

[0225] In some embodiments, 4 bits are reserved for the FT part. The frame type bit values and content indications when using 4 bits are presented below. In these embodiments the FT part identifies the bitrate where the FT bits are bit values other than a defined value (for example 1111 ) and can be used to identify other aspects, for example SPEECH_LOST, NO_DATA and SID frames with the combination of the FT indicator OTHER (the defined value) and the extra bits.

[0226] As shown above there are 15 available bit allocations for the FT bits reserved for future use (bit allocations 10001 - 11111 ), when 5 bits are reserved for the FT indicator. In some embodiments when 4 bits are reserved for the FT indicator, there are 5 available bit allocations for future use, when FT value 1111 is used. FT value 1110 provides an additional 8 bit allocations for future use when combined with the extra bits. These available bit allocations can be used in the future to indicate frame contents that will be defined, such as PI frames described hereafter.

[0227] These bit allocation values are examples only and it would be appreciated that in some embodiments the bit allocation values can be otherwise configured.

[0228] In some embodiments, the name for the supplementary information frames can be something different than audio processing information frames.

[0229] In some embodiments additional (non-audio) data can be added to streamed IVAS RTP packets in order to enable better rendering or consumption of the IVAS audio bitstream data. For example, in some embodiments any external orientations from the sender UE side could be included and encoded into the (audio) processing information (PI) frames. In another example, in the ISM mode, the importance, and / or the content type, of the ISM audio objects could be included in the PI frames.

[0230] Figure 7 shows an example how the PI frames could be integrated to the IVAS RTP packets.

[0231] For example a first option 701 is shown where the PI frame 705 is sent as a separate IVAS RTP packet. In other words, in some embodiments the PI frame is sent without IVAS speech (or any coded audio) frames in the same RTP packet. This approach can be employed, if the bitrate limitation of a single RTP packet is very strict. For example in situations where adding the PI frame in the same packet as an IVAS speech frame would increase the size of the RTP packet too much (for example ending up with a frame greater than the Maximum Transmission Unit (MTU) size for the network being used).

[0232] A further option 711 is one where the PI frame is integrated to the same RTP packet with an IVAS speech frame. In this example packet structure 711 all the ToC bytes 713, 714 are positioned in the beginning of the payload comprising the PI frame 715 and the IVAS frame 716.

[0233] In some embodiments the ToC bytes associated with the payload frames can be located such that ToC bytes associated with the payload frame are adject to each other. Thus, as shown by the structure 721 a ToC byte 723 and associated PI frame 725 are located together and the ToC byte 727 and associated IVAS frame 729 are located together.

[0234] In some embodiments, the structure comprising the complete separation of the PI and IVAS frame payload can be implemented when the PI frame does not contain critically important data for rendering. Such packets can be configured to be transmitted at lower priority.

[0235] For example, in situations where important information that affects rendering and / or should be tightly time-aligned with the audio data (e.g., IVAS frame) is sent then the packet structures where there are combined PI and IVAS frame payload, can be used instead, as they guarantee simultaneous delivery of the audio frame (IVAS frame) and the PI frame data.

[0236] With respect to Figure 8 there are shown examples of how to integrate multiple IVAS speech frames or PI frames in the same RTP packet structure.

[0237] For example, in some embodiments such as shown in the example structures 801 , 811 and 821 the structure is such that the ToC bytes precede the data frames.

[0238] For example the structure 801 is a structure in which the ToC bytes 802, 803, 804 precede the data frames which are arranged such that the PI frame 805 is positioned before all the IVAS speech frames 806, 807. In this case the data in the PI frame 805 could apply to the first IVAS speech frame 806 or to all the IVAS frames 806, 807 in the RTP packet. For example, if the PI data contains orientation information, the orientations could be applied to the first IVAS frame or to all the IVAS frames. The interpretation on to which IVAS frames the PI data should be applied can depend on the content in the PI frame or what is specified in the codec specifications.

[0239] In a further example structure 811 the ToC bytes 812, 813, 814 precede the data frames which are arranged such that an PI frame 816 is placed in between two IVAS speech frames 815 and 817. In this case, the PI data can be interpreted to be applied to the latter IVAS speech frame 817 only.

[0240] An additional example structure 821 is one where the ToC bytes 822, 823, 824 precede the data frames which are arranged such that the RTP packet includes two PI frames 825, 826 before the IVAS frame 827. In some embodiments these two PI frames comprise different kinds of data. For example, the first PI frame 825 can comprise orientation information, and the second PI frame 826 comprises priority information for ISM audio objects.

[0241] In some embodiments such as shown in the example structures 831 , 841 and 851 the structure is such that the ToC bytes interleave the data frames and are located adjacent their associated frames.

[0242] For example, the structure 831 is a structure in which the ToC byte 832 is associated with and proceeds the PI frame 833, the ToC byte 834 is associated with and proceeds the IVAS frame 835 and the ToC byte 836 is associated with and proceeds the IVAS frame 837. In this case the data in the PI frame 833 could apply to the first IVAS speech frame 835 or to all the IVAS frames 835, 837 in the RTP packet (such as described above with respect to the example structure 801 ).

[0243] In a further example structure 841 is a structure in which the ToC byte 842 is associated with and proceeds the IVAS frame 843, the ToC byte 844 is associated with and proceeds the PI frame 845 and the ToC byte 846 is associated with and proceeds the IVAS frame 847. In this case the data in the PI frame 845 could only apply to the later IVAS speech frame 847.

[0244] An additional example structure 851 is one where the ToC byte 852 is associated with and proceeds the PI frame 853, the ToC byte 854 is associated with and proceeds the PI frame 855 and the ToC byte 856 is associated with and proceeds the IVAS frame 857. In this example these two PI frames comprise different kinds of data.

[0245] The PI frames are identified with help from the ToC byte, and can follow the same structure as shown with respect to Figure 6b. As mentioned before and presented in the above tables, the FT bits in some embodiments (with 5 bits) have 15 unused bit allocation slots that are able to be defined. One (or more than one) of these bit values can be used to identify PI frames, for example a defined bit allocation, such as 10010 can be employed as the PI frame identifier. In some embodiments when 4 bits is reserved for the FT indicator, one of the freely available combinations of extra bits and FT presented above could be used to identify PI frames, for example the bit values (011 + 1111 ),

[0246] In some embodiments, and shown with respect to Figure 9, the PI frame could be structured in a variety of ways.

[0247] For example an example PI frame structure 901 is one in which the frame comprises a size identifier or size field 902 at the beginning of the frame, which is followed by a frame type identifier or type field 903 which is then followed by the PI data 904. In some embodiments the size identifier information (or size field) is coded with N bits, where N is some fixed integer number (and can be pre-defined in the IVAS specification). The frame type identifier or type field can be configured to indicate the type of data the PI frame carries. The type could be, for example, orientation data, priority of audio objects in ISM mode, scene information, or any suitable non-audio information. The type identifier or field can also include usage and validity information, as explained later. The type identifier or field can in some embodiments be coded with a pre-defined number of bits.

[0248] A further example PI frame structure 911 is one in which the frame comprises a frame type identifier 913 which is then followed by the PI data 914. In this example the size information is not coded into the PI frame, but the size of an PI frame is predefined (for example within the IVAS specification). Thus the size of an PI frame is known to the receiver when reading the RTP packet, so that a correct number of bits is read for the frame.

[0249] An additionally example PI frame structure 921 is one in which the frame comprises a size identifier 922 at the beginning of the frame, which is followed by the PI data 924. The size identifier information is similarly coded with N bits, where N is some fixed integer number. In this example structure the type of the PI frame cannot be modified within the frame. However in some embodiments, different types could be defined using the FT bits in the ToC byte. For example, when allocating 5 bits for the FT indicator, or with the combination of the extra bits and the FT indicator, when allocating 4 bits for the FT indicator. Thus instead of having a single PI_FRAME frame content indication, there could be different FT bit allocations, or extra bits + FT combinations reserved for different types of PI frames.

[0250] An example FT bit usages (with 5 FT bits) is presented in the below table, where PI_FRAME type could be used for any generic PI data, ORIENTATION_DATA type could be used for orientation based data, SCENEJNFO type could be for scene information data and ISM_PRIORITY type could be used for frames indicating the priority audio objects in ISM streams.

[0251] In some embodiments combinations of extra 3 bits + FT indicator (4 bits) for the above frame types can be presented in the following table.

[0252] In some embodiments the PI frame structure 931 comprises only the PI data 934. In such embodiments the size of the PI frame is fixed and additionally the type of the data is pre-defined. In such embodiments the PI only frame structure reduces the total number of bits required for an PI frame but requires the use of the freely available FT bits or extra bits + FT bit combinations in the ToC byte. However this limits the possible size / type variations significantly as the they are predefined and fixed (for example by the specification) or are limited by the number of free FT bit or extra bits + FT bit combinations.

[0253] There is furthermore shown an example structure 941 , which is similar to the structure of 901 where the type indicator or field 943 is positioned before the size indicator or field 942. This approach makes it possible to reserve different amounts of bits for the size field for different PI frame types.

[0254] For example, some PI types might have long data sections, e.g., label names which might require more bits to be reserved for the size field. Additionally, the structure 941 makes it possible to implement arbitrary sized PI frames (for example in the future, if new PI frame types are developed.

[0255] With respect to Figures 10a and 10b are shown examples, according to some embodiments of the structure for the type indicator (or field or identifier) as described above.

[0256] With respect to Figure 10a an example structure of the type indicator 1001 (or field or identifier) in an PI frame is presented. The type fields for example can comprise the following:

[0257] Format 1003 - The format field describes the format and the semantics of the PI frame payload. In other words the format describes what kind of data is carried in the PI frame, e.g. orientation data.

[0258] Usage 1005 - The usage field describes how the PI data should be used. For example, the usage field 1005 can describe to apply the current PI data to the next IVAS frame or to all IVAS frames in the RTP packet. Alternatively, the usage field can describe that the PI data should not be taken account in the rendering, i.e. , that the data is more general information to the receiver.

[0259] Validity 1007 - The validity field describes how long the PI frame is valid in the receiver end. For example, the field could describe that for X amount of processing frames the receiver should apply the PI data described in the frame, and if no new PI data is received within X frames, stop applying the data. The validity could also be indicated in other time formats than number of processing frames, e.g., in milliseconds. Furthermore, some audio frame values may be “hold” whereas other values may be “instantaneous” only. This enables the Tenderer to perform rendering accordingly.

[0260] With respect to Figure 10b a further example structure 1051 for the type indicator where the type indicator further comprises size information 1053.

[0261] In some embodiments to minimize the number of additional bits introduced by the PI frames, the maximum size of the PI frames can be restricted. For example, the frames can be restricted to have a maximum size of K bits that is common for all IVAS bitrates. Alternatively, the maximum size of the PI frames could be linked to the used bitrate of the IVAS speech frames, e.g., by having the maximum number of bits for PI frames be some percentage share of the bits used for the speech frames. In addition, the use of PI frames could be restricted to be used only in the highest IVAS bitrates to reduce the risk of adding too much load to the sent packets.

[0262] With respect to Figure 11 is shown a first example application of the PI frame. The example shown in Figure 11 (and also the following Figures) show the size of the packet (nBits 1100) is outside the type indicator (in other words the type indicator is one similar to the example shown in Figure 10a). However, in some embodiments the size could be incorporated to the type indicator, such as shown in the example in Figure 10b. The examples show how the other fields (format, usage, validity and PI data) could be used in an PI frame.

[0263] Figure 11 shows examples how an PI frame could be used to transmit orientation data. In the first example 1101 the PI frame comprises a size (nbits) indicator 1103 and type indicator 1105.

[0264] The type indicator or field can indicate orientation format (ORIENTATION) 1121 and that the orientation data should be applied to all IVAS frames (ALL_FRAMES) 1123. The validity period for the orientation is set to infinity (INF) 1125. In other words the specified orientation should be applied to all incoming frames, until another orientation data frame appears.

[0265] The first example 1101 shows that the orientation data is transmitted as a spherical index 1107. The spherical index 1107 in some embodiments indicates a pre-defined direction in a spherical grid. The accuracy of the coded direction depends on the resolution of the spherical grid. In other words how many directions the grid has.

[0266] The second example 1111 encodes the direction directly as yaw 1116, pitch 1117, and roll 1118 angles.

[0267] In some other embodiments, only one or some of the direction angles could be transmitted. The combinations could be identified by an identifier before the angle values. For example the combination can be identified by a 2-bit identifier such as (00 - only yaw), (01 - only pitch), (10 - yaw and pitch), (11 - yaw, pitch and roll).

[0268] In some embodiments a higher number of bits for the identifier could be used to include more angle combinations.

[0269] With respect to Figure 12 there is shown an example how an PI frame could be used to transmit the priority of audio objects for a IVAS ISM format.

[0270] The number of bits nBits 1200 for the PI frame comprises a size (nbits) indicator 1203 and type indicator 1205.

[0271] The type indicator 1205 in this embodiments comprises the type identification, the ISM_Priority 1221 value identifying that the PI frame is a priority identifier.

[0272] The type 1205 indicator furthermore comprises a usage field or part with a THIS_FRAME 1223 value which indicates that the transmitted priority values should be applied only to the current IVAS frame, i.e., the IVAS frame following this PI frame.

[0273] Furthermore in this example the validity period is shown set to zero frames with a 0_FRAMES 1225 value indicating that the priority values are valid only for the current frame.

[0274] As the usage field of the type is shown set to THIS_FRAME, the priority values would be valid only to the current frame regardless of the value in the validity field.

[0275] The priority values (ISMJmpN) 1206, 1207, 1208, 1209 indicate the priority of the audio objects in an IVAS ISM stream (which can have maximum of 4 objects by definition). With respect to Figure 13 there is shown an example how an PI frame could be used to indicate priority between multiple IVAS input formats.

[0276] The number of bits nBits 1300 for the PI frame comprises a size (nbits) indicator 1303 and type indicator 1305 and PI data in the form of the IVAS_format_priority 1309 value.

[0277] In this example the type indicator 1305 comprises the type indication IVAS_IF_PRIORITY 1321 value. Furthermore the type indicator comprises a usage field or part with a ALL_FRAMES value 1323 which indicates that the transmitted priority values should be applied to all frames following this PI frame.

[0278] Furthermore in this example the validity period is shown set to 50 frames with a 50_FRAMES 1325 value indicating that the priority values are valid only for the IVAS formats present in the current frame. The validity period is thus set to 50 frames, which implies that the priority value for the attached IVAS input format remains for a period of 50 processing frames if a new priority value is not set during that time.

[0279] Figures 14 and 15 show further examples how an PI frame could be used to transmit more general data that is not directly linked to rendering. This kind of data could be, e.g., label for the active speaker (such as shown by Figure 14) or labels for the audio objects (such as shown by Figure 15). Other examples of more general data or information could be ideal loudspeaker setup for the playback or some information related to how the transmitted audio was captured. This capture information could be, for example, the constellation of the microphone array.

[0280] Figure 14 for example comprises, a size, nBits 1403 indicator and type indicator 1405.

[0281] The type indicator 1405 in this embodiments comprises the type identification, the SPEAKER_LABEL 1421 value identifying that the PI frame is a speaker label identifier.

[0282] The type 1405 indicator furthermore comprises a usage field or part with a THIS_FRAME value 1423 which indicates that the transmitted priority values should be applied only to the current IVAS frame, i.e., the IVAS frame following this PI frame. Furthermore in this example the validity period is shown set to zero frames with a 0_FRAMES 1425 value indicating that the priority values are valid only for the current frame.

[0283] Figure 15 for example comprises, a size, nBits 1503 indicator and type indicator 1505.

[0284] The type indicator 1505 in this embodiments comprises the type identification, the ISM_LABEL 1521 value identifying that the PI frame is a ISM label identifier.

[0285] The type 1505 indicator furthermore comprises a usage field or part with a THIS_FRAME value 1523 which indicates that the transmitted priority values should be applied only to the current IVAS frame, i.e., the IVAS frame following this PI frame.

[0286] Furthermore in this example the validity period is shown set to zero frames with a 0_FRAMES 1525 value indicating that the priority values are valid only for the current frame.

[0287] The (IVAS) decoder according to some embodiments, such as the example decoder 131 , is shown in Figure 16.

[0288] The decoder in some embodiment is configured to receive or otherwise obtain the RTP packets 1600. Additionally the decoder comprises a RTP extractor 1601 which is configured to receive the RTP packets 1600 and extract from these RTP packets the PI payload and enable the decoder to render the audio signals based on the RTP packet (IVAS) payload.

[0289] The RTP extractor 1601 , in some embodiments, comprises a payload type determiner 1603. The payload type determiner 1603 is configured to determine from the RTP packets the PI information type and thus be able to determine the PI information and apply the PI information based on the determined PI information type.

[0290] Figure 17 shows the operations of the example RTP extractor shown in Figure 16.

[0291] The first operation is one of obtaining or receiving RTP packets as shown in Figure 17 by step 1701. Then for the received RTP packet a payload type is determined as shown in Figure 17 by step 1703.

[0292] Then the payload is applied based on the determined payload type as shown in Figure 17 by step 1705.

[0293] With respect to Figure 18 an example electronic device is shown. The device may be any suitable electronics device or apparatus. For example in some embodiments the device 1800 is a mobile device, user equipment, tablet computer, computer, audio playback apparatus, etc.

[0294] In some embodiments the device 1800 comprises at least one processor or central processing unit 1807. The processor 1807 can be configured to execute various program codes such as the methods such as described herein.

[0295] In some embodiments the device 1800 comprises a memory 1811. In some embodiments the at least one processor 1807 is coupled to the memory 1811. The memory 1811 can be any suitable storage means. In some embodiments the memory 1811 comprises a program code section for storing program codes implementable upon the processor 1807. Furthermore in some embodiments the memory 1811 can further comprise a stored data section for storing data, for example data that has been processed or to be processed in accordance with the embodiments as described herein. The implemented program code stored within the program code section and the data stored within the stored data section can be retrieved by the processor 1807 whenever needed via the memory-processor coupling.

[0296] In some embodiments the device 1800 comprises a user interface 1805. The user interface 1805 can be coupled in some embodiments to the processor 1807. In some embodiments the processor 1807 can control the operation of the user interface 1805 and receive inputs from the user interface 1805. In some embodiments the user interface 1805 can enable a user to input commands to the device 1800, for example via a keypad. In some embodiments the user interface 1805 can enable the user to obtain information from the device 1800. For example the user interface 1805 may comprise a display configured to display information from the device 1800 to the user. The user interface 1805 can in some embodiments comprise a touch screen or touch interface capable of both enabling information to be entered to the device 1800 and further displaying information to the user of the device 1800.

[0297] In some embodiments the device 1800 comprises an input / output port 1809. The input / output port 1809 in some embodiments comprises a transceiver. The transceiver in such embodiments can be coupled to the processor 1807 and configured to enable a communication with other apparatus or electronic devices, for example via a wireless communications network. The transceiver or any suitable transceiver or transmitter and / or receiver means can in some embodiments be configured to communicate with other electronic devices or apparatus via a wire or wired coupling.

[0298] The transceiver can communicate with further apparatus by any suitable known communications protocol. For example in some embodiments the transceiver can use a suitable universal mobile telecommunications system (UMTS) protocol, a wireless local area network (WLAN) protocol such as for example IEEE 802. X, a suitable short-range radio frequency communication protocol such as Bluetooth, or infrared data communication pathway (IRDA).

[0299] The transceiver input / output port 1809 may be configured to receive the signals and in some embodiments obtain the focus parameters as described herein.

[0300] In some embodiments the device 1800 may be employed to generate a suitable audio signal using the processor 1807 executing suitable code. The input / output port 1809 may be coupled to any suitable audio output for example to a multichannel speaker system and / or headphones (which may be a headtracked or a non-tracked headphones) or similar.

[0301] In general, the various embodiments of the invention may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0302] The embodiments of this invention may be implemented by computer software executable by a data processor of the mobile device, such as in the processor entity, or by hardware, or by a combination of software and hardware. Further in this regard it should be noted that any blocks of the logic flow as in the Figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD.

[0303] The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The data processors may be of any type suitable to the local technical environment, and may include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASIC), gate level circuits and processors based on multi-core processor architecture, as non-limiting examples.

[0304] Embodiments of the inventions may be practiced in various components such as integrated circuit modules. The design of integrated circuits is by and large a highly automated process. Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.

[0305] Programs, such as those provided by Synopsys, Inc. of Mountain View, California and Cadence Design, of San Jose, California automatically route conductors and locate components on a semiconductor chip using well established rules of design as well as libraries of pre-stored design modules. Once the design for a semiconductor circuit has been completed, the resultant design, in a standardized electronic format (e.g., Opus, GDSII, or the like) may be transmitted to a semiconductor fabrication facility or "fab" for fabrication.

[0306] The foregoing description has provided by way of exemplary and non-limiting examples a full and informative description of the exemplary embodiment of this invention. However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of this invention will still fall within the scope of this invention as defined in the appended claims.

[0307] 3GPP 3rdGeneration Partnership Project

[0308] AMR-WB IO Adaptive Multi Rate Wideband Inter-Operable

[0309] PI Processing Information (Audio)

[0310] EVS Enhanced Voice Services

[0311] ISM Independent Streams with Metadata

[0312] (i.e. , type of Object-Based Audio)

[0313] IVAS Immersive Voice and Audio Services kbps kilobits per second

[0314] RTP Real-Time Transport Protocol

[0315] SDP Session Description Protocol

Claims

CLAIMS:1 . An apparatus comprising means configured to: obtain at least one audio signal; generate at least one audio signal data packet; obtain processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a renderer; generate at least one processing information packet comprising the processing information; and associate the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

2. The apparatus as claimed in claim 1 , wherein the means is further configured to determine a size of the at least one processing information, wherein the means configured to generate at least one processing information packet comprising the processing information is configured to generate the at least one processing information packet comprising a size field indicating the size of the at least one processing information.

3. The apparatus as claimed in claim 1 , wherein the means is further configured to: determine a type of the processing information, wherein the means configured to generate at least one processing information packet comprising the processing information is configured to generate the at least one processing information packet comprising a type field indicating a type of the at least one processing information.

4. The apparatus as claimed in claim 1 , wherein the means is further configured to: determine a size and a type of the processing information, wherein the means configured to generate at least one processing information packet comprising the processing information is configured to generate the at least one processing information packet comprising: a size field indicating the size of the at least one processing information; and a type field indicating the type of the at least one processing information.

5. The apparatus as claimed in any of claims 3 or 4, wherein the type of the processing information comprises one of: an orientation indicator parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

6. The apparatus as claimed in any of claims 2 to 5, wherein the means configured to generate at least one processing information packet comprising the processing information is configured to generate at least one processing information packet comprising at least one of: a usage indicator configured to describe how the processing information should be used; and a validity indicator configured to describe how long the processing information is valid.

7. The apparatus as claimed in any of claims 1 to 6, wherein the processing information comprises one of:an orientation parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

8. The apparatus as claimed in any of claims 1 to 7, wherein the means is further configured to: generate at least one processing information packet header comprising an indicator identifying that the at least one processing information packet comprises processing information; and append the at least one processing information packet header with the at least one processing information packet in a manner that the at least one processing information packet is identified at a further apparatus as being at least one processing data packet comprising processing information rather than audio data.

9. The apparatus as claimed in claim 8, wherein the means configured to append the at least one processing information packet header with the at least one processing information packet is configured to: append the at least one processing information packet header in a first block and the at least one processing information packet in a separate second block; and append the at least one processing information packet header immediately preceding an associated of the at least one processing information packet.

10. The apparatus as claimed in any of claims 1 to 9, wherein the means configured to associate the at least one processing information packet with the at least one audio signal data packet is configured to perform one of: append every one of the at least one processing information packet before the at least one audio signal data packet; orappend the at least one at least one processing information packet immediately preceding an associated at least one audio signal data packet.11 . The apparatus as claimed in any of claims 1 to 10, wherein the at least one audio signal data packet is an Immersive Voice and Audio Services data packet.

12. The apparatus as claimed in any of claims 1 to 11 , wherein the means are further configured to transmit the at least one audio signal data packet and the at least one audio processing data packet as a real-time transport protocol transmission.

13. An apparatus comprising means configured to: receive an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; and process the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

14. The apparatus as claimed in claim 13, wherein the at least one processing information packet comprises a size field indicating a size of the at least one processing information.

15. The apparatus as claimed in claim 13, wherein the at least one processing information packet comprises a type field indicating a type of the processing information within the at least one processing information packet and wherein the means configured to process the at least one audio signal based on the processinginformation is configured to identify the type of the processing information within at least one processing information packet based on the type field.

16. The apparatus as claimed in claim 13, wherein the at least one processing information packet comprises: a size field indicating a size of the at least one processing information packet; and a type field indicating a type of the processing information within the at least one processing information packet and wherein the means configured to process the at least one audio signal based on the processing information is configured to identify the type of the processing information within at least one processing information packet.

17. The apparatus as claimed in any of claims 15 or 16, wherein the type of the processing information comprises one of: an orientation indicator parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

18. The apparatus as claimed in any of claims 14 to 17, wherein the at least one processing information packet further comprises at least one of: a usage indicator configured to describe how the processing information should be used; and a validity indicator configured to describe how long the processing information is valid with respect to the processing of the at least one succeeding audio signal data packet.

19. The apparatus as claimed in any of claims 13 to 18, wherein the processing information comprises one of: an orientation parameter associated with the at least one audio signal; a scene information parameter associated with the at least one audio signal; an audio signal or stream priority parameter indicating the priority of audio signals within the at least one audio signal; an active speaker indicator; and a priority parameter indicating the priority audio objects in the at least one audio signal.

20. The apparatus as claimed in any of claims 13 to 19, wherein the audio packet stream comprising further comprises at least one processing information packet header comprising an indicator identifying that the at least one processing information packet comprises processing information; and21. The apparatus as claimed in claim 8, wherein the at least one processing information packet header and the at least one processing information packet is arranged in one of: the at least one processing information packet header in a first block and the at least one processing information packet in a separate second block; and the at least one processing information packet header immediately preceding an associated of the at least one processing information packet.

22. The apparatus as claimed in any of claims 13 to 21 , wherein the at least one processing information packet and the at least one audio signal data packet is arranged in one of: every one of the at least one processing information packet before the at least one audio signal data packet; and the at least one at least one processing information packet immediately preceding an associated at least one audio signal data packet.

23. The apparatus as claimed in any of claims 13 to 22, wherein the at least one audio signal data packet is an Immersive Voice and Audio Services data packet.

24. The apparatus as claimed in any of claims 13 to 23, wherein the means configured to receive the audio packet stream as a real-time transport protocol transmission.

25. A method for an apparatus, the method comprising: obtaining at least one audio signal; generating at least one audio signal data packet; obtaining processing information, the processing information comprising at least one parameter associated with the at least one audio signal and the processing information being for assisting the processing of the at least one audio signal at a renderer; generating at least one processing information packet comprising the processing information; and associating the at least one processing information packet with the at least one audio signal data packet in a manner that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

26. A method for an apparatus, the method comprising: receiving an audio packet stream, the audio packet stream comprising: at least one audio signal data packet comprising at least one audio signal; at least one processing information packet, the at least one processing information packet comprising processing information comprising at least one parameter associated with the at least one audio signal; andprocessing the at least one audio signal based on the processing information such that the at least one processing information packet is configured to assist a processing of at least one succeeding audio signal data packet.

Images