Audio decoder, audio encoder, decoding method, encoding method and bitstream using multiple packets including one or more scene configuration packets and one or more scene update packets having one or more update conditions - Patents.com

The implementation of scene configuration, update, and payload packets in MPEG-H MHAS definitions addresses the challenge of bandwidth-efficient, immersive audio rendering for VR and AR applications, ensuring high-definition and real-time updates.

JP2024543058AInactive Publication Date: 2024-11-19FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024527296
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-09
Filing Date
2022-11-09
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing technologies face challenges in providing high-definition, immersive auditory experiences for VR and AR applications while efficiently managing data transmission and decoding within bandwidth constraints.

Method used

The use of scene configuration, update, and payload packets, conforming to MPEG-H MHAS definitions, to efficiently transmit and render dynamic 6DoF audio scenes, allowing for spatial audio rendering and updates based on scene metadata.

Benefits of technology

Enables efficient transmission, storage, and rendering of high-definition, immersive audio scenes with real-time updates, reducing bandwidth requirements and ensuring smooth audio reconstruction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024543058000001_ABST
    Figure 2024543058000001_ABST
Patent Text Reader

Abstract

An embodiment according to the invention relates to an audio decoder for providing a decoded audio representation based on an encoded audio representation, the audio decoder configured to spatially render one or more audio signals, the audio decoder configured to receive a plurality of packets of different packet types, the packets including one or more scene configuration packets providing renderer configuration information defining use of scene objects and / or use of scene properties, the packets including one or more scene update packets defining updates of scene metadata for the rendering, the packets including one or more scene payload packets including definitions of one or more of the scene objects and / or one or more of the scene properties, the audio decoder configured to select for the rendering the one or more scene object definitions and / or the one or more scene property definitions contained in the scene payload packets in dependence on the renderer configuration information, the audio decoder configured to update one or more scene metadata in dependence on a content of the one or more scene update packets. Further embodiments relate to an encoder, a method and a bitstream. Further embodiments relate to a decoder, encoder, method and bitstream having a scene update packet with update conditions, the scene configuration packet providing renderer configuration information defining the temporal evolution of a rendering scenario together with timestamp information and / or sub-scene cell information, the cell information defining an association between one or more cells and a respective one or more data structures.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Embodiments in accordance with the present invention relate to dynamic VR (Virtual Reality) / AR (Augmented Reality) bitstreams, for example using three packet types, using scene update packets with update conditions, using timestamps, and / or using cell information. [Background technology]

[0002] To provide an immersive experience for VR and / or AR applications, it is not enough to provide not only a spatial viewing experience but also a spatial auditory experience. As an example, six degrees of freedom (6DoF) audio technology has been developed to meet such needs. In this regard, it is difficult to develop a bitstream and corresponding encoders and decoders that enable a high-definition, immersive auditory experience while also being available with a feasible bandwidth.

[0003] It is therefore desirable to obtain a concept that allows a better compromise between the achievable auditory impression of a rendered audio scene, the efficiency of the transmission of the data used to render the audio scene and the efficiency of decoding and / or rendering of the data.

[0004] This is achieved by the subject matter of the independent claims of the present application. Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application. Summary of the Invention

[0005] In the following, an embodiment according to a first aspect of the invention is described. The embodiment according to the first aspect of the invention may be based on using three packet types. The embodiment according to the first aspect of the invention may for example include scene update packets and / or scene payload packets. The embodiment according to the first aspect of the invention may comprise MPEG-H compatible packets or may provide or comprise an MPEG-H compatible decoder, encoder and / or bitstream.

[0006] An embodiment according to the invention comprises an audio decoder for providing a decoded and optionally rendered audio representation based on the encoded audio representation. The audio decoder is configured to spatially render one or more audio signals and receive a plurality of packets of different packet types, e.g. having packet types conforming to the MPEG-H MHAS packet definition, including one or more scene configuration packets, e.g. Scene ConfigPacket, e.g. mpegiSceneConfig[] (sometimes called "mpeghiSceneConfig[]"), providing renderer configuration information defining the use of scene objects and / or the use of scene properties, e.g. using a definition of a cell to define when or under what conditions different scene objects and / or scene properties should be used in the rendering process. The concept of a cell is particularly important for practical implementation of e.g. sub-scenes. A sub-scene is a portion of a scene that relates, e.g., to a particular point in the scene time or to a particular neighborhood / proximity of a given scene location. In some cases, the terms cell and sub-scene may be used synonymously.

[0007] Optionally, the scene configuration packets may define, for example, which scene payload packets are needed at a given point in space and time. As another optional feature, the scene configuration packets may define, for example, where the scene payload packets can be taken from.

[0008] Additionally, the packet may include one or more scene update packets, e.g., mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate[]"), that define updates, e.g., changes, to scene metadata for rendering (e.g., changes to one or more metadata values, changes to parameters of scene objects, or changes to scene characteristics, e.g., changes to scene metadata that occur during playback). Optionally, the one or more scene update packets may, e.g., define one or more conditions for the scene update.

[0009] Further, the packet includes one or more scene payload packets (e.g., mpeghiScenePayload, which may also be referred to as "mpeghiScenePayload") that include definitions of one or more of the scene objects and / or definitions of one or more of the scene characteristics (e.g., bulk metadata, e.g., metadata required for rendering one or more audio scenes, e.g., geometry metadata describing the audio scene for rendering, and / or parametric rendering instructions for rendering, and / or audio element metadata describing one or more audio elements in the audio scene for rendering, e.g., instructions, and / or geometry and / or audio effects metadata, e.g., reverberation metadata, and / or early reflections metadata, and / or diffraction metadata, e.g., mpegiScenePayload (also designated "mpeghiScenePayload") within an MHA PacketPayload()).

[0010] In addition, the audio decoder is configured to select for rendering one or more scene object definitions and / or one or more scene property definitions contained in the scene payload packet depending on renderer configuration information that may optionally be contained in the scene configuration packet. As another optional feature, cells may be used to select which scene objects and / or scene properties to use.

[0011] Further, the audio decoder is configured to update one or more scene metadata (e.g., one or more rendering parameters specified by "targetId", where new values ​​for the one or more rendering parameters may be specified by "attributes") depending on the contents of the one or more scene update packets.

[0012] As an example, an audio decoder may further comprise the functionality of a renderer and may comprise a renderer or a rendering unit. Thus, in the context of some embodiments, an audio decoder may be synonymous with a renderer, e.g. a renderer with decoding functionality.

[0013] The inventors have recognized that by using at least three different packet types, namely, scene configuration packets, scene update packets, and scene payload packets, metadata for complex dynamic audio coding applications, for example, for dynamic 6DoF audio scenes, can be efficiently provided, transmitted, stored, and / or updated.

[0014] As an example, a scene configuration packet may contain or provide relevant information for a decoder (or renderer) to configure itself. In particular, this may include, for example, instructions on which scene payload packets are needed at any given point in space and / or time (e.g., an audio scene or a spatial position in a rendering scenario), and optionally where they can be retrieved from, for example a dedicated client-server backchannel.

[0015] On the other hand, a scene payload packet may be, for example, a container for bulk metadata, which may be metadata that cannot be directly related to the timeline of the audio stream, but is necessary or useful for rendering a complex and dynamic audio scene, such as, for example, a 6 DoF audio scene. The payload packets may contain, for example, geometry information of acoustically relevant objects present in the audio scene, parametric rendering instructions, and / or further audio element metadata, such as reflection or diffraction properties. In other words, the payload packets may contain, for example, special metadata for directivity, geometry, and individual audio effects, such as reverberation, early reflections or diffraction.

[0016] Furthermore, the inventors have recognised that it may be advantageous to define a third class of packets, namely scene update packets, for updating the above-mentioned scene metadata, which may allow for specifying the conditions under which the update is performed (e.g. time-based, and / or location-based, and / or based on interactivity triggers) and the changes made to the scene.

[0017] Briefly, as an example, an embodiment according to the invention may be based on the idea of ​​providing scene metadata for defining acoustically relevant elements and / or attributes of a scene in scene payload packets. To provide information on how the respective payload information should be processed and / or when and / or where the respective payload information is used (e.g. in an audio scene or rendering scenario), scene configuration packets may be used to set a renderer or decoder in a corresponding configuration. To be able to update such metadata, scene update packets may be used to provide information on the update and optionally the update conditions.

[0018] Thus, a "measured" (e.g., real) audio scene may be accurately reconstructed using the payload information, or a virtual audio scene may be realistically rendered, and decoder or renderer configuration, as well as transmission, storage, distribution, and updating of information may be performed efficiently based on the separation into different packet types described above.

[0019] In other words, the use of three different packet types, including scene configuration packets, scene payload packets, and scene update packets, allows for particularly efficient transmission and evaluation of scene information (e.g., scene object information and scene property information), since an audio decoder can determine, based on one or more scene configuration packets, which scene payload packets are needed or which information from the scene payload packets is needed. Thus, an audio decoder can use information of the scene configuration packets, for example, to determine which scene payload packets to store and / or evaluate, and an audio decoder can also use the scene configuration packets, for example, to determine which scene payload packets to request from a scene payload packet provider (e.g., from a server), in some cases. In addition, the scene update packets can allow for efficient signaling of changes in scene information, thereby contributing to high efficiency in the transmission and processing of scene information.

[0020] According to a further embodiment of the present invention, the audio decoder is configured to determine a rendering configuration based on a scene configuration packet. Optionally, the scene configuration packet can reference a global payload packet, e.g. using a globalPayloadId, and the scene configuration packet can reference individual payloads (or individual payload packets), e.g. using a cell concept associating payloadIds with cells. Furthermore, the audio decoder is configured to determine a rendering configuration update based on one or more scene update packets.

[0021] A cell may provide a spatial and / or temporal segmentation of an audio scene, and the current position of the listener at which the audio scene is rendered may trigger the request or use of the respective payload packets, e.g., which may define metadata about acoustically related objects in the scene, e.g., spatially located within the area associated with the cell and, e.g., active at the current playtime.

[0022] According to further embodiments of the present invention, one or more scene update packets, e.g., mpegiSceneUpdate() (sometimes referred to as "mpeghiSceneUpdate()"), include an enumeration of scene metadata items to be modified, e.g., an enumeration having a variable number of scene metadata items to be modified and a variable order of the scene metadata items to be modified. Further, the enumeration includes a metadata identifier, e.g., targetId, and a metadata update value, e.g., attribute, for the one or more metadata items to be modified. Optionally, the audio decoder may be configured to selectively update the scene metadata included in the enumeration.

[0023] The inventors have realised that this allows one to efficiently indicate which metadata items are to be updated and then update them.

[0024] According to a further embodiment of the invention, the audio decoder is configured to obtain one or more definitions of the scene objects and / or one or more definitions of the scene properties from one or more scene payload packets.

[0025] As explained above, the inventors have recognized that a scene payload packet may enable efficient definition of acoustically relevant scene objects and / or scene characteristics, based on which a realistic sound environment rendering can be performed.

[0026] According to a further embodiment of the present invention, the one or more scene payload packets include an enumeration of payloads defining scene objects and / or scene characteristics, for example an enumeration having a variable number of payloads and a variable order of the payloads, and the audio decoder is further configured to evaluate the enumeration of payloads defining the scene objects and / or scene characteristics.

[0027] The enumeration of payloads can allow for simple selection of which payloads, e.g., payload elements, are currently considered. Furthermore, the enumeration used by the scene update packet can correspond to the enumeration of the scene payload packets, to simply select which payload elements to update.

[0028] According to a further embodiment of the present invention, a payload identifier, e.g., an ID, is associated with a payload in a scene payload packet, and the audio decoder is configured to evaluate the payload identifier of a given payload to determine whether the given payload, e.g., a definition of a particular given scene object and / or a particular scene property, should be used for rendering, e.g., using renderer configuration information defining the use of the scene object and / or the use of the scene property.

[0029] The inventors have realised that the use of payload identifiers allows for efficient selection of payloads to be considered for rendering.

[0030] According to a further embodiment of the present invention, one or more of the scene update packets define conditions for a scene update, and the audio decoder is configured to evaluate whether the conditions for a scene update defined in the scene update packets are satisfied to determine whether a scene update should be performed.

[0031] Thus, efficient scene updates can be performed. The use of conditional updates can allow the updates to be performed quickly after a condition is met, e.g., without having to first request the respective update information when the condition is met. Thus, such updates, including information about the update itself and the trigger condition, can be sent to the decoder before they are needed, so that the decoder can simply "wait" until the criteria are met.

[0032] According to a further embodiment of the invention, one or more of the scene update packets define an interactivity trigger condition, e.g., a condition that a user takes a particular action beyond mere movement within the scene, e.g., a condition that a user gives a predefined command or activates a predefined button. Further, the audio decoder is configured to evaluate whether the interactivity trigger condition is satisfied in order to determine whether a scene update should occur.

[0033] Thus, a realistic audio environment rendering can be provided, even taking into account real-world user interactions with the environment (e.g., pressing a button, e.g., activating a virtually simulated machine that causes a change in the acoustic characteristics of the scene).

[0034] According to a further embodiment of the invention, the one or more scene configuration packets, the one or more scene update packets and the one or more scene payload packets conform to an MPEG-H MHAS packet definition. Optionally, the audio decoder may be configured to identify the one or more scene configuration packets, the one or more scene update packets and the one or more scene payload packets in the stream of packets, for example using bitstream parsing conforming to an MPEG-H MHAS packet definition.

[0035] This allows for easy integration into existing coding frameworks.

[0036] According to further embodiments of the present invention, the one or more scene configuration packets, the one or more scene update packets, and the one or more scene payload packets each include a packet type identifier, e.g., MHASPacketType, a packet label, e.g., MHASPacketLabel, packet length information, e.g., MHASPacketLength, and a packet payload, e.g., MHASPacketPayload. Optionally, the audio decoder may be configured to evaluate the packet type identifier in order to distinguish between packets of different packet types.

[0037] As an example, the packet label may provide an indication of which packets belong together. For example, different labels may be used to assign different MPEG-H 3D audio composition structures to a particular sequence of MPEG-H 3D audio access units. The packet length information may indicate the length of the packet payload. The inventors have recognized that the use of such data structures may enable efficient processing of packets.

[0038] According to a further embodiment of the present invention, an audio decoder is configured to extract one or more scene configuration packets, one or more scene update packets, and one or more scene payload packets from a bitstream comprising a plurality of MPEG-H packets, including packets representing one or more audio channels to be rendered.

[0039] Optionally, the audio decoder may be configured to extract one or more scene configuration packets, one or more scene update packets, and one or more scene payload packets from an interleaved sequence of packets including packets of different types (e.g., from an interleaved sequence of MHAS packets), e.g., using packet type identifiers and / or packet labels included within the packets.

[0040] Embodiments according to the invention may be used in the context of MPEG-H audio streams. Thus, embodiments according to the invention may be compatible with existing audio streaming frameworks. Furthermore, embodiments may support a number of different options for, for example, providing (e.g., interleaved) packets. As an example, providing in an interleaved manner may allow the size of each packet to be kept low, which may be advantageous for some, e.g., broadcast, channels.

[0041] According to a further embodiment of the present invention, the audio decoder is configured to receive one or more scene configuration packets via a broadcast stream, for example via a low bitrate broadcast stream. The inventors have realised that for applications having a large number of users, for example, the scene configuration packets can be provided efficiently via the broadcast stream.

[0042] According to a further embodiment of the present invention, the audio decoder is configured to request one or more scene payload packets from the packet provider, e.g. using a backchannel to the packet provider, e.g. in response to a determination by the audio decoder that one or more scene payload packets, or content of one or more scene payload packets, are required for rendering.

[0043] For example, based on the scene configuration packets, the decoder can determine which payload packets it needs (e.g., by first determining which metadata elements are needed or the renderer configuration, which may indicate which payloads are needed) and can request them accordingly. This may, for example, make it possible to alleviate the burden on broadcast channels and reallocate individually related (e.g., for a particular decoder or listener for which the decoder is rendering an audio scene) information transmission to unicast and / or multicast channels.

[0044] According to a further embodiment of the invention, the audio decoder is configured to request one or more scene payload packets from the packet provider using a payload ID, e.g., an ID associated with a payload element, or the audio decoder is configured to request one or more scene payload packets from the packet provider using a packet ID, e.g., an ID associated with a scene payload packet.

[0045] Thus, according to an embodiment, scene payload packets may be identified, for example, using a payload ID representing the payload of the respective payload packet, or using a packet ID, which may allow efficient requesting of information.

[0046] According to further embodiments of the present invention, the audio decoder is configured to anticipate, e.g. using prediction, which one or more data structures, e.g. which one or more PayloadElements, are needed or expected to be needed, e.g. which Cells will be active next or have a defined likelihood of being active next, and to request the one or more data structures, or one or more scene payload packets containing the one or more data structures, before the data structures are actually needed.

[0047] The inventors have recognized that such predictions may be performed, for example, to ensure that no buffering time is required, e.g., to enable fluent or smooth audio scene reconstruction. For example, unexpected bandwidth drops may be mitigated based on timely prediction of required data structures and thus their transmission before they are needed.

[0048] According to a further embodiment of the present invention, the audio decoder is configured to provide information to the packet provider, e.g. position information, or play time information, or scene time, that at least implicitly indicates which one or more scene payload packets are required or are required within a predetermined period of time, thereby enabling the packet provider to selectively provide to the audio decoder, e.g. using point-to-point transmission, packets that are required by the audio decoder or are required by the audio decoder within a predetermined period of time.

[0049] Therefore, the bitstream provided below can be optimized to take into account the transmission of the required scene payload packets. Furthermore, in this way, a scheduling can be determined that allows for example to robustly meet timing constraints, such that it is guaranteed, or at least highly likely, that the required information is present at the decoder when needed.

[0050] It should be noted that in general, a packet provider may be, for example, an encoder according to any of the embodiments as disclosed herein.

[0051] According to a further embodiment of the present invention, one or more scene update packets, e.g., mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate[]"), define updates to scene metadata for rendering, e.g., changes to parameters of scene objects or changes to scene properties, and include a representation of one or more update conditions. Furthermore, the audio decoder is configured to evaluate whether the one or more update conditions are met, and, if the one or more update conditions are met, selectively update the one or more scene metadata depending on the content of the one or more scene update packets, e.g., thereby determine a rendering scenario corresponding to the timestamp.

[0052] Thus, the scene update packet may not only provide the update itself, but also the conditions of when and / or where (e.g., spatially within the acoustic scene and / or which payload, respectively) to update. Thus, update information may be provided before an update is needed, thus preventing buffer times and enabling real-time audio scene adaptation. As an example, a condition may be the opening of a door, so that when a user in the VR surroundings opens the door, the scene may be immediately updated with respect to acoustically changed characteristics.

[0053] In the following embodiments, embodiments are described relating to an apparatus for providing an encoded audio representation, e.g. an encoder. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments relating to a decoder. Thus, the following embodiments may include, individually and in combination, the same, similar or corresponding features, functions and details as the above-disclosed embodiments.

[0054] Thus, a further embodiment according to the invention comprises an apparatus for providing an encoded audio representation, such as an audio encoder or an audio server, configured to provide information for spatial rendering of one or more audio signals, the apparatus being configured to provide a plurality of packets of different packet types, for example having packet types conforming to an MPEG-H MHAS packet definition.

[0055] Additionally, the packet may include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information defining the usage of scene objects and / or the usage of scene properties, e.g., using cell definitions to define when or under what conditions different scene objects and / or scene properties should be used in the rendering process.

[0056] Additionally, the packet may include one or more scene update packets, such as mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate"), that define updates to scene metadata for rendering (e.g., changes to one or more metadata values, changes to parameters of scene objects, or changes to scene characteristics, e.g., changes to one or more metadata values).

[0057] Further, the packet includes one or more scene payload packets containing definitions of one or more of the scene objects and / or one or more of the scene properties, e.g., bulk metadata, e.g., metadata required for rendering one or more audio scenes, e.g., geometry metadata describing the audio scene for rendering, and / or parametric rendering instructions for rendering, and / or audio element metadata describing one or more audio elements in the audio scene for rendering.

[0058] According to a further embodiment of the invention, the apparatus is configured to provide renderer configuration information included in a scene configuration packet, such that for rendering, the renderer configuration information defines a selection of one or more scene object definitions and / or one or more scene property definitions included in the scene payload packet.

[0059] Thus, the audio scene may be or may be rendered in an immersive manner and / or with a high level of detail.

[0060] According to a further embodiment of the invention, an apparatus is configured to provide one or more scene update packets, such that a content of the one or more scene update packets defines an update to one or more scene metadata.

[0061] According to a further embodiment of the present invention, the apparatus is configured to provide scene configuration packets, such that the scene configuration packets determine a rendering configuration, and the apparatus is configured to provide scene update packets, such that the scene update packets define updates to the rendering configuration.

[0062] According to a further embodiment of the present invention, the apparatus is configured to provide one or more scene configuration packets, one or more scene update packets and one or more scene payload packets, such that the one or more scene configuration packets, one or more scene update packets and one or more scene payload packets conform to an MPEG-H MHAS packet definition.

[0063] According to a further embodiment of the present invention, the apparatus is configured to provide one or more scene configuration packets, one or more scene update packets, and one or more scene payload packets, such that the one or more scene configuration packets, one or more scene update packets, and one or more scene payload packets each include a packet type identifier, e.g., MHASPacketType, a packet label, e.g., MHASPacketLabel, packet length information, e.g., MHASPacketLength, and a packet payload, e.g., MHASPacketPayload.

[0064] According to a further embodiment of the invention, the apparatus is configured to provide one or more scene configuration packets and one or more scene update packets and one or more scene payload packets in a bitstream comprising multiple MPEG-H packets, e.g. in an interleaved manner, including packets representing one or more audio channels to be rendered.

[0065] Optionally, the audio decoder may be configured to extract one or more scene configuration packets, one or more scene update packets, and one or more scene payload packets from an interleaved sequence of packets including packets of different types (e.g., from an interleaved sequence of MHAS packets), e.g., using packet type identifiers and / or packet labels included within the packets.

[0066] Thus, the smaller data portions may be transmitted, for example, in an interleaved manner. For example, in a broadcast scenario, this may allow the broadcast bitstream data rate to be kept low. In such a case, as an example, the payload packets may optionally be provided using a separate client-server channel.

[0067] According to a further embodiment of the invention, the apparatus is configured to provide scene configuration packets via a broadcast stream, for example via a low bitrate broadcast stream, and optionally provide scene payload packets depending on the scene time and / or user position.

[0068] According to a further embodiment of the invention, the apparatus is configured to provide one or more scene payload packets in response to a request from an audio decoder, e.g. in response to a determination by the audio decoder that one or more scene payload packets, or content of one or more scene payload packets, are required for rendering, e.g. on-demand, thereby allowing the payload packets to be provided efficiently.

[0069] According to further embodiments of the invention, the apparatus is configured to provide one or more scene payload packets in response to a request from the audio decoder including a payload ID, e.g. using an ID associated with a payload element. Alternatively, the apparatus is configured to provide one or more scene payload packets in response to a request from the audio decoder including a packet ID, e.g. using an ID associated with a scene payload packet.

[0070] According to a further embodiment of the present invention, the apparatus is configured to provide one or more scene payload packets depending on information, e.g. position information, or play time information, or scene time, that at least implicitly indicates which one or more scene payload packets are required or are required within a predetermined period of time, e.g. such that the apparatus may selectively provide to the audio decoder, e.g. using point-to-point transmission, packets that are required by the audio decoder or packets that are required by the audio decoder within a predetermined period of time.

[0071] According to a further embodiment of the invention, the apparatus is configured to provide one or more scene update packets, e.g., mpegiSceneUpdate[ ] (sometimes referred to as “mpeghiSceneUpdate ”), defining updates to scene metadata for rendering, e.g., changes to parameters of scene objects or changes to scene characteristics, and including expressions of one or more update conditions.

[0072] According to further embodiments of the invention, the apparatus is configured to periodically repeat providing a scene configuration packet, or even a sequence of a scene configuration packet and one or more scene payload packets, and optionally one or more scene update packets, which may allow for simple adjustment of the respective decoder or renderer.

[0073] According to a further embodiment of the invention the apparatus is arranged to provide a scene configuration packet such that the scene configuration packet defines which scene payload packets are required at a given point in space and time.

[0074] According to a further embodiment of the invention, the apparatus is arranged to provide scene configuration packets such that the scene configuration packets define from where the scene payload packets can be taken from, such that each decoder or renderer can request each required payload packet individually.

[0075] According to a further embodiment of the invention the device is arranged for providing a scene update packet, such that the scene update packet defines a condition for a scene update.

[0076] According to a further embodiment of the invention, the device is configured to provide a scene update packet such that the scene update packet defines an interactivity trigger condition for a scene update (e.g., a condition in which the user takes a particular action beyond mere movement within the scene, e.g., the user gives a predefined command or activates a predefined button).

[0077] According to a further embodiment of the invention, the apparatus is configured to adapt an order of one or more definitions of scene objects and / or one or more definitions of scene properties in a scene payload packet depending on when and / or where the one or more definitions of scene objects and / or one or more definitions of scene properties are required by the renderer.

[0078] This allows efficient scheduling of the transmission of payloads describing scene objects or properties. As an example, based on scene time or play time, the relevance of acoustically related objects can be determined and the order of transmission can be set accordingly. As another example, based on the position of the listener in the virtual acoustic scene, acoustically related objects close to this position can be transmitted earlier based on the order, e.g., rendered in detail without interrupting the scene if the listener's spatial trigger conditions are met (simply put, if the listener moves to another position near the real location, the respective information about the acoustic properties of the new position can be available immediately, e.g.).

[0079] According to a further embodiment of the invention, the apparatus is configured to adapt an order of one or more definitions of scene objects and / or one or more definitions of scene properties in a scene payload packet depending on the importance of the one or more definitions of scene objects and / or one or more definitions of scene properties to a renderer.

[0080] Thus, the order may for example be selected according to the acoustic impact of each object or property, which may further enable incorporating the level of detail concept according to embodiments.

[0081] According to a further embodiment of the invention, the device is configured to adapt an order of one or more definitions of scene objects and / or one or more definitions of scene properties in a scene payload packet depending on a packet size limit.

[0082] Thus, embodiments may allow for distributing the definition of scene objects and / or scene characteristics in order to provide a desired packet size for its transmission, for example allowing for a small packet size to be achieved in order to provide it efficiently in an interleaved manner with other packets.

[0083] According to a further embodiment of the invention, the apparatus is arranged to initially provide payload packets containing a relatively low level of detail and to subsequently provide payload packets containing a relatively high level of detail.

[0084] This may make it possible to initially provide a minimal amount of information for the rendering of a scene that can be refined later, for example given a sufficiently good communication channel, thus allowing the audio scene to be provided robustly.

[0085] According to a further embodiment of the present invention, the apparatus is configured to separate definitions of one or more scene objects and / or one or more scene properties (e.g. bulk metadata, e.g. metadata required for rendering one or more audio scenes, e.g. geometry metadata describing the audio scene for rendering, and / or parametric rendering instructions for rendering, and / or audio element metadata describing one or more audio elements in the audio scene for rendering) into multiple scene payload packets, e.g. scene payload packets associated with different points in time (e.g. different playback times) and / or different locations in the space of the scene.

[0086] Further, the device may be configured to provide different scene payload packets, e.g. at different times, e.g. according to a determination at which playback time and / or at which position within the scene the scene characteristics contained in the respective scene payload packets are required.

[0087] Thus, the inventors have recognised that, simply put, spreading out smaller payload packets may enable an efficient bitstream to be provided.

[0088] According to a further embodiment of the invention, the apparatus is configured to provide a scene composition packet for decomposing a scene into a number of spatial regions, e.g. areas or volumes, e.g. spatial regions including shapes defined by geometry objects, for which different rendering metadata is valid.

[0089] The inventors have realised that scene decomposition in multiple spatial domains may allow for efficiently organising or scheduling or manipulating the activation of acoustically related elements, e.g. represented by payloads, e.g. with respect to time and / or space, to provide a realistic acoustic scene. As an example, cells, e.g. as described in the context of further embodiments, may be used for such scene decomposition as well as for the concept of level of detail, thus separating audio scenes in space and / or time in different cells, which may for example overlap, such that one cell may provide refinement information for acoustic elements related to another cell.

[0090] In the following embodiments, embodiments relating to a method for providing a decoding and encoding audio representation are described. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments related to the device. Thus, the following embodiments may include the same, similar or corresponding features, functions and details as the above-disclosed embodiments, individually and in combination.

[0091] Further, embodiments in accordance with the invention comprise a method for providing a decoded and optionally rendered audio representation based on an encoded audio representation, the method comprising spatially rendering one or more audio signals and receiving a plurality of packets of different packet types, for example having packet types conforming to an MPEG-H MHAS packet definition.

[0092] Additionally, the packet may include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information defining the usage of scene objects and / or the usage of scene properties, e.g., using cell definitions to define when or under what conditions different scene objects and / or scene properties should be used in the rendering process.

[0093] Optionally, a scene configuration packet may define, for example, which scene payload packets are needed at a given point in space and time, and the scene configuration packet may define, for example, where the scene payload packets may be taken from.

[0094] In addition, the packet includes one or more scene update packets, e.g., mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate"), defining updates, e.g., changes, e.g., changes in one or more metadata values, changes in parameters of scene objects, or changes in scene characteristics, e.g., changes in scene metadata occurring during playback, for rendering. Optionally, the one or more scene update packets may also define one or more conditions, e.g., for a scene update, and the packet further includes one or more scene payload packets including definitions of one or more of the scene objects and / or one or more of the scene characteristics, e.g., bulk metadata, e.g., metadata required for rendering one or more audio scenes, e.g., geometry metadata describing an audio scene for rendering, and / or parametric rendering instructions for rendering, and / or audio element metadata describing one or more audio elements in an audio scene for rendering, e.g., instructions, and / or geometry and / or audio effect metadata, e.g., reverberation metadata, and / or early reflections metadata, and / or diffraction metadata.

[0095] Further, the method includes selecting, for rendering, one or more scene object definitions and / or one or more scene property definitions included in the scene payload packet, e.g., in response to renderer configuration information that may be included in the scene configuration packet, e.g., a cell may be used to select which scene objects and / or scene properties should be used.

[0096] Further, the method includes updating one or more scene metadata, e.g., one or more rendering parameters specified by "targetId", where new values ​​of the one or more rendering parameters are specified by "attribute", in response to content of the one or more scene update packets.

[0097] A further embodiment according to the invention includes a method for providing an encoded audio representation, the method comprising providing information for spatial rendering of one or more audio signals and providing a plurality of packets of different packet types, for example having a packet type conforming to an MPEG-H MHAS packet definition.

[0098] The packets include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), that provide renderer configuration information defining the use of scene objects and / or the use of scene properties, e.g., using cell definitions to define when or under what conditions different scene objects and / or scene properties should be used in the rendering process.

[0099] Additionally, the packet may include one or more scene update packets, such as mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate"), that define updates to scene metadata for rendering (e.g., changes to one or more metadata values, changes to parameters of scene objects, or changes to scene characteristics).

[0100] Further, the packet includes one or more scene payload packets containing definitions of one or more of the scene objects and / or one or more of the scene properties, e.g., bulk metadata, e.g., metadata required for rendering one or more audio scenes, e.g., geometry metadata describing the audio scene for rendering, and / or parametric rendering instructions for rendering, and / or audio element metadata describing one or more audio elements in the audio scene for rendering.

[0101] A further embodiment according to the invention comprises a computer program for performing the method according to any of the embodiments disclosed herein, when the computer program runs on a computer.

[0102] In the following, embodiments related to bitstreams will be described. It should be noted that such embodiments may be based on the same, similar or corresponding considerations as the above-mentioned embodiments related to the device and / or method. Thus, the following embodiments may include the same, similar or corresponding features, functions and details as the above-disclosed embodiments, both individually and in combination.

[0103] A further embodiment according to the invention comprises a bitstream representing audio content, the bitstream comprising a number of packets of different packet types, for example with packet types conforming to the MPEG-H MHAS packet definition.

[0104] The packets include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), that provide renderer configuration information defining the use of scene objects and / or the use of scene properties, e.g., using cell definitions to define when or under what conditions different scene objects and / or scene properties should be used in the rendering process.

[0105] Optionally, a scene configuration packet may define, for example, which scene payload packets are needed at a given point in space and time, and the scene configuration packet may define, for example, where the scene payload packets may be taken from.

[0106] Additionally, the packet may include one or more scene update packets, e.g., mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate"), that define updates, e.g., changes, to scene metadata for rendering (e.g., changes to one or more metadata values, changes to parameters of scene objects, or changes to scene characteristics, e.g., changes to scene metadata that occur during playback). Optionally, the one or more scene update packets may define, e.g., one or more conditions for a scene update.

[0107] Further, the packets include one or more scene payload packets containing definitions of one or more of the scene objects and / or one or more of the scene properties, e.g., bulk metadata, e.g., metadata required for rendering one or more audio scenes, e.g., geometry metadata describing the audio scene for rendering, and / or parametric rendering instructions for rendering, and / or audio element metadata describing one or more audio elements in the audio scene for rendering, e.g., instructions, and / or geometry and / or audio effect metadata, e.g., reverberation metadata, and / or early reflections metadata, and / or diffraction metadata.

[0108] Furthermore, the bitstream may be optionally supplemented by any of the bitstream elements disclosed herein, both individually and in combination.

[0109] Further embodiments comprise an audio decoder for providing a decoded audio representation based on the encoded audio representation, the audio decoder configured to spatially render one or more audio signals and receive a plurality of packets of different packet types, the packets comprising one or more scene configuration packets providing renderer configuration information defining scene objects and / or scene properties, the packets comprising one or more scene update packets defining updates to scene metadata for the rendering, and the packets comprising one or more scene payload packets comprising definitions of one or more of the scene objects and / or one or more of the scene properties.

[0110] Further, the audio decoder is configured to select one or more scene object definitions and / or one or more scene characteristic definitions contained in the scene payload packet for rendering, and to update one or more scene metadata according to the content of the one or more scene update packets.

[0111] It should be noted that such a decoder of the present invention may include, both individually and in combination, the same, similar or corresponding features, functions and details of any of the embodiments disclosed above or any of the other embodiments disclosed herein.

[0112] In the following, an embodiment according to the second aspect of the present invention is described, which can be based on using a scene update packet with an update condition and / or using a scene update packet with an update condition and other features.

[0113] The embodiments according to the second aspect may include features and details of the embodiments of the first aspect of the invention either individually or in combination, and vice versa.

[0114] An embodiment according to the invention comprises an audio decoder for providing a decoded and optionally rendered audio representation based on the encoded audio representation, the audio decoder being configured to spatially render one or more audio signals, the audio decoder being configured to receive a plurality of packets of different packet types, for example having packet types conforming to an MPEG-H MHAS packet definition.

[0115] Further, the packet may include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), that provide renderer configuration information, e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the use of scene objects and / or the use of scene properties.

[0116] The packet further includes one or more scene update packets, e.g., mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate"), that define scene metadata updates for rendering, e.g., changes to parameters of scene objects or changes to scene characteristics, and include expressions of one or more update conditions.

[0117] In addition, the audio decoder is configured to evaluate whether one or more update conditions, e.g., as defined in a scene update packet, are met, and, if the one or more update conditions are met, selectively update one or more scene metadata according to the content of the one or more scene update packets.

[0118] The inventors have realised that audio scenes can be updated efficiently using metadata updates in the form of scene update packets, which thus comprise a representation of the update conditions, such that a decoder can, for example, receive the scene update packet before the update itself has to be performed, thus relaxing the transmission timing constraints, and the device can then perform the update when the respective condition is met.

[0119] This allows for efficient consideration of predefined triggered acoustic effects in the audio scene. For example, in a VR ambience, a user can open a door and thus change the acoustic properties of the virtual room. The opening of the door can thus trigger an update. Thus, information about the acoustic "dynamics" of the door opening may be transmitted before the door is opened.

[0120] Thus, virtual acoustic scenes can be provided with a high degree of realism and in real time, while simplifying the transmission of metadata updates.

[0121] It should further be emphasized that one advantage of this approach may be the aggregation of not only update information, but also update trigger information, for example in the form of update conditions within a scene update packet, thereby providing self-consistent and / or self-contained and / or autonomous update information.

[0122] Furthermore, the audio decoder can react to different events defined by the update conditions (e.g. triggering an update for a local event occurring at the audio decoder side) and is not bound to a given temporal evolution of the scene. The use of scene update packets therefore results in a particularly good auditory representation while keeping the required bit rate reasonably small.

[0123] According to a further embodiment of the present invention, the audio decoder is configured to evaluate a time condition, e.g. a temporal trigger condition, e.g. defined by startTimestamp, contained in the scene update packet in order to decide whether to update one or more scene metadata depending on the content of the one or more scene update packets, e.g. depending on the enumeration of scene metadata items to be changed, e.g. referenced by "targetId", and their corresponding new values, e.g. defined by "attribute".

[0124] The inventors have recognized that update triggers may not only be provided by events, for example, but also by timing. Thus, again, the scene update packet may include information about the update itself and a time or time window for determining when to perform the update. Thus, for example, acoustic changes over time may be efficiently taken into account.

[0125] According to a further embodiment of the invention, the time condition defines a start time, for example using the bitstream element startTimestamp, or alternatively, the time condition defines a time interval, for example using a start time and an end time.

[0126] Further, the audio decoder is configured to perform one or more scene metadata updates in response to the detection, e.g., in response to a detection that the current playback time, e.g., the scene time, has reached or is after the start time, e.g., immediately in response to the detection or with a time delay defined in the respective scene update packet, e.g., in accordance with a definition included in the respective scene update packet.

[0127] Alternatively, the audio decoder may be configured to perform one or more scene metadata updates in response to detecting that the current playback time, e.g., a scene time, is within the time interval, e.g., according to definitions included in a respective scene update package.

[0128] Thus, depending on the specific implementation, a "timer" for when an update must be performed, or simply until an update must be performed, may for example be provided, allowing efficient updating of metadata.

[0129] According to a further embodiment of the present invention, the audio decoder is configured to evaluate spatial conditions, e.g. spatial trigger conditions, e.g. spatial conditions defined by reference to a geometry definition, e.g. spatial conditions defined by geometryId, contained in the scene update packets in order to decide whether to update one or more scene metadata depending on the content of the one or more scene update packets, e.g. an enumeration of scene metadata items to be changed, e.g. referenced by 'targetId', and their corresponding new values, e.g. defined by 'attributes'.

[0130] Therefore, updates that depend on spatial location within the audio scene can be efficiently represented.

[0131] According to a further embodiment of the invention, the spatial conditions define the geometry elements, for example using a reference to a geometry definition, which may for example be included in a scene payload element.

[0132] Further, the audio decoder is configured to perform one or more scene metadata updates in response to the detection, e.g. immediately in response to the detection, or with a time delay defined in the respective scene update packet, e.g. that the current position has reached a geometry element (e.g. a one-dimensional boundary defined by a geometry element has been reached, or a two-dimensional boundary defined by a geometry object has been reached, or a three-dimensional boundary defined by a geometry object), or in response to the detection, e.g. immediately in response to the detection, or with a time delay defined in the respective scene update packet, e.g. that the current position is within a geometry element, e.g. within a two-dimensional geometry element or a three-dimensional geometry element, e.g. according to a definition included in the respective scene update packet.

[0133] As an example, a user in a VR ambience may move through a VR room separated within acoustic zones defined by geometric elements such as boxes. As the user enters a new acoustic zone, that zone may be updated to accurately describe the corresponding acoustic characteristics, thus enabling an immersive user experience. In this way, only the currently relevant acoustic zones may need to be updated.

[0134] According to a further embodiment of the present invention, the audio decoder is configured to evaluate whether an interactivity trigger condition is met, which may for example be defined in a scene update packet (e.g. a condition that the user takes a particular action beyond mere movement within the scene, e.g. a condition that the user gives a certain command or activates a certain button, defined for example by the flag "fireOn"), in order to determine whether one or more scene metadata should be updated depending on the content of the one or more scene update packets.

[0135] As mentioned above, for example in the context of the first aspect of the invention, the audio scene can thus be efficiently updated.

[0136] According to a further embodiment of the present invention, the audio decoder is configured to evaluate a combination of two or more update conditions, e.g. an AND combination or another Boolean combination, and the audio decoder is configured to selectively update one or more scene metadata depending on the content of one or more scene update packets if the combined update condition is met, e.g. if two or more update conditions are all met.

[0137] The inventors have recognised that the evaluation of interlinked or chained update conditions may enable efficient updating of acoustic scenes, eg audio scenes, eg rendering scenarios.

[0138] According to a further embodiment of the invention, the audio decoder is configured to evaluate both a temporal update condition and a spatial update condition, or alternatively, the audio decoder is configured to evaluate both a temporal update condition and an interactivity update condition.

[0139] Therefore, the update conditions can be expressed with high flexibility, and the efficiency and / or adaptability of the update can be improved.

[0140] According to a further embodiment of the invention, the audio decoder is configured to evaluate delay information, e.g. a delay, contained in the scene update packets, and the audio decoder is configured to delay updates of one or more scene metadata depending on the content of the one or more scene update packets in response to detecting that the one or more update conditions are met in accordance with the delay information.

[0141] Some changes in the audio scene may be triggered by events and followed by a delay, e.g., a time lag. Such changes can be efficiently incorporated into the audio scene updates based on the techniques described above.

[0142] According to a further embodiment of the present invention, the audio decoder is configured to evaluate a flag in the scene update packet indicating whether a temporal update condition is defined in the scene update packet, e.g. hasTemporalCondition. Alternatively or additionally, the audio decoder is configured to evaluate a flag in the scene update packet indicating whether a spatial update condition is defined in the scene update packet, e.g. hasSpatialCondition. Optionally, the audio decoder is configured to AND the conditions whose presence is indicated by the respective flags.

[0143] Thus, the decoder can be provided with information whether temporal and / or spatial update conditions should be taken into account using flags, thus with low signaling effort.

[0144] According to a further embodiment of the present invention, the audio decoder is configured to evaluate a flag in the scene update packet indicating whether delay information is defined in the scene update packet, e.g. a flag indicating whether there is a delay, e.g. "there is a delay".

[0145] Thus, an indication of whether a delay should be taken into account can be provided with low signaling effort.

[0146] According to further embodiments of the present invention, a scene update packet includes a representation (e.g., an enumeration, where the number of entries in the enumeration may be indicated, for example, by a bitstream parameter, e.g., numOfModifications), of multiple modifications of one or more parameters of one or more scene objects and / or one or more scene characteristics.

[0147] Optionally, the parameters or scene properties to be modified (or updated) are specified, for example by "targetId", and the respective new values ​​of the parameters or scene properties are specified, for example by "attribute".

[0148] In addition, the audio decoder is configured to apply the modifications in response to detecting that one or more update conditions are met.

[0149] According to a further embodiment of the present invention, the scene update packet includes trajectory information, which may for example be associated with parameters of the scene objects to be changed, or may for example be associated with scene properties to be changed, and includes for example isTrajectory, interpolationType, numPoints, time[n] and value[n].

[0150] Further, the audio decoder is configured to update each scene metadata with which the trajectory information is associated using parameter variation, e.g. smooth interpolation parameter variation, according to a trajectory, e.g. a temporal evolution, defined by the trajectory information, the audio decoder being able to determine the trajectory, e.g. based on a number of supporting points.

[0151] Thus, even complex updates can be parameterized or represented with low effort, for example using trajectories, reducing the signaling effort. Optionally, trajectories, and therefore metadata updates, can be interpolated based on the fulcrums.

[0152] According to a further embodiment of the present invention, the audio decoder is configured to evaluate information indicating whether trajectory-based updating of the scene metadata is used (e.g., a flag isTrajectory, e.g., information included in a scene update packet) to activate or deactivate trajectory-based updating of the scene metadata.

[0153] According to a further embodiment of the present invention, the audio decoder is configured to evaluate interpolation type information, e.g., interpolationType information, which may indicate, e.g., linear interpolation, and / or cubic interpolation, and / or sample-and-hold behavior, included in a scene update packet, to determine a type of interpolation between two or more fulcrums of a trajectory, the fulcrums being defined, e.g., by time information (e.g., time[n]) associated with each fulcrum and value information (e.g., value[n]) associated with each fulcrum.

[0154] Therefore, depending on the desired trajectory, an appropriate interpolation type can be selected. Furthermore, a good compromise between the computational complexity for the trajectory interpolation and the accuracy of the desired trajectory can be achieved based on the selection of the type of interpolation.

[0155] According to a further embodiment of the present invention, the audio decoder is configured to evaluate fulcrum information describing a trajectory (e.g. fulcrum information comprising information about several fulcrums, e.g. fulcrum information comprising one or more values ​​time[n] and value[n]), which may describe multiple fulcrums for temporal variations in the scene metadata, e.g. using pairs of fulcrum time information and fulcrum value information.

[0156] The inventors have realised that supporting points may enable trajectories, and therefore update rules, to be described at low computational and / or transmission cost.

[0157] In the following embodiments, embodiments are described relating to an apparatus for providing an encoded audio representation, e.g. an encoder. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments relating to a decoder. Thus, the following embodiments may include, individually and in combination, the same, similar or corresponding features, functions and details as the above-disclosed embodiments.

[0158] A further embodiment according to the invention comprises an apparatus for providing an encoded audio representation, such as an audio encoder or an audio server, configured to provide information for spatial rendering of one or more audio signals, the apparatus configured to provide a plurality of packets of different packet types, such as having a packet type conforming to an MPEG-H MHAS packet definition.

[0159] Further, the packet may include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), that provide renderer configuration information, e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the use of scene objects and / or the use of scene properties.

[0160] In addition, the packet includes one or more scene update packets, such as mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate"), that define scene metadata updates for rendering, such as changes to parameters of scene objects or changes to scene characteristics, and include expressions of one or more update conditions.

[0161] According to a further embodiment of the invention, the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of a time condition, e.g. a temporal trigger condition, defined by a startTimestamp, for updating one or more scene metadata items depending on the content of the scene update packet, e.g. a list of scene metadata items to be changed, referenced by 'targetId', and their corresponding new values, defined by 'attribute', for example.

[0162] According to a further embodiment of the invention, the time condition defines a start time, for example using the bitstream element startTimestamp, or alternatively, the time condition defines a time interval, for example using a start time and an end time.

[0163] According to a further embodiment of the invention the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of a spatial condition, e.g. a spatial trigger condition, e.g. a spatial condition defined by a reference to a geometry definition, e.g. a spatial condition defined by a geometryId, for updating one or more scene metadata items in response to the content of the scene update packet, e.g. an enumeration of scene metadata items to be changed, referenced by a 'targetId', and their corresponding new values, e.g. defined by an 'attribute'.

[0164] According to a further embodiment of the invention, the spatial conditions define the geometry elements, for example using a reference to a geometry definition, which may for example be included in a scene payload element.

[0165] According to a further embodiment of the invention, the device is configured to provide a scene update packet such that the scene update packet includes a representation of an interactivity trigger condition that may be defined within the scene update packet, e.g., in response to a list of scene metadata items to be changed, referenced by "targetId", for updating one or more scene metadata items in response to the content of the scene update packet (e.g., a condition that the user takes a particular action beyond mere movement within the scene, e.g., a condition that the user gives a certain command or activates a certain button, e.g., defined by the flag "fireOn"), and a corresponding new value, e.g., defined by "attribute".

[0166] According to a further embodiment of the invention, the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of a combination of two or more update conditions, such as an AND combination, or another Boolean combination.

[0167] According to a further embodiment of the invention, the apparatus is configured to provide a scene update packet such that the scene update packet includes delay information, e.g. a delay, defining a delay to delay updating one or more scene metadata depending on the content of the one or more scene update packets, in response to detecting that one or more update conditions are met.

[0168] According to a further embodiment of the present invention, the apparatus is configured to provide a scene update packet such that the scene update packet includes a flag indicating whether a temporal update condition is defined in the scene update packet, such as hasTemporalCondition. Alternatively or additionally, the apparatus is configured to provide a scene update packet such that the scene update packet includes a flag indicating whether a spatial update condition is defined in the scene update packet, such as a representation of hasSpatialCondition.

[0169] According to a further embodiment of the present invention, the apparatus is configured to provide a scene update packet such that the scene update packet includes a flag indicating whether delay information is defined in the scene update packet, e.g. having a delay, e.g. "with delay".

[0170] According to a further embodiment of the present invention, the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation (e.g., an enumeration, where the number of entries of the enumeration may be indicated, for example, by a bitstream parameter, e.g., numOfModifications, "enumeration") of a plurality of modifications of one or more parameters of one or more scene objects and / or one or more scene characteristics.

[0171] According to a further embodiment of the present invention, the apparatus is configured to provide a scene update packet such that the scene update packet includes trajectory information, which may for example be associated with a parameter of a scene object to be modified, or which may for example be associated with a scene property to be modified, and which may for example include isTrajectory, interpolationType, numPoints, time[n] and value[n].

[0172] Further, the trajectory information describes updating each scene metadata with which the trajectory information is associated using parameter variation, e.g. smooth interpolation parameter variation, according to a trajectory, e.g. a temporal evolution, defined by the trajectory information, and the audio decoder can determine the trajectory, e.g. based on multiple support points.

[0173] According to a further embodiment of the present invention, the apparatus is configured to provide a scene update packet such that the trajectory information includes information (e.g. a flag isTrajectory, e.g. information included in the scene update packet) indicating whether trajectory based updating of the scene metadata is used to activate or deactivate trajectory based updating of the scene metadata.

[0174] According to a further embodiment of the present invention, the apparatus is configured to provide a scene update packet, wherein the trajectory information includes interpolation type information, e.g., interpolationType information, which may indicate, e.g., linear interpolation, and / or cubic interpolation, and / or sample-and-hold behavior, included in the scene update packet to determine a type of interpolation between two or more fulcrums of the trajectory, and the fulcrums may be defined, e.g., by time information (e.g., time[n]) associated with each fulcrum and value information (e.g., value[n]) associated with each fulcrum.

[0175] According to a further embodiment of the present invention, the device is configured to provide a scene update packet such that the trajectory information includes fulcrum information describing a trajectory (e.g., fulcrum information including information about several fulcrums, e.g., fulcrum information including one or more values ​​time[n] and value[n]), where the fulcrum information can describe multiple fulcrums for temporal variations in the scene metadata, e.g. using pairs of fulcrum time information and fulcrum value information.

[0176] In the following embodiments, embodiments relating to a method for providing a decoding and encoding audio representation are described. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments related to the device. Thus, the following embodiments may include the same, similar or corresponding features, functions and details as the above-disclosed embodiments, individually and in combination.

[0177] A further embodiment according to the invention comprises a method for providing a decoded and optionally rendered audio representation based on an encoded audio representation, the method comprising spatially rendering one or more audio signals and receiving a plurality of packets of different packet types, e.g. having a packet type conforming to an MPEG-H MHAS packet definition.

[0178] Additionally, the packet may include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information (e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the use of scene objects and / or the use of scene properties).

[0179] In addition, the packet includes one or more scene update packets, e.g., mpegiSceneUpdate[ ] (sometimes referred to as "mpeghiSceneUpdate"), that define updates to scene metadata for rendering, e.g., changes to parameters of scene objects or changes to scene characteristics, and that include a representation of one or more update conditions, and the method further includes evaluating whether the one or more update conditions, e.g., as defined in the scene update packet, are satisfied, and selectively updating the one or more scene metadata depending on content of the one or more scene update packets if the one or more update conditions are satisfied.

[0180] A further embodiment according to the invention includes a method for providing an encoded audio representation, the method including providing a plurality of packets of different packet types, for example having a packet type conforming to an MPEG-H MHAS packet definition.

[0181] Additionally, the packet may include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information (e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the use of scene objects and / or the use of scene properties).

[0182] In addition, the packet includes one or more scene update packets, such as mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneUpdate"), that define scene metadata updates for rendering, such as changes to parameters of scene objects or changes to scene characteristics, and include expressions of one or more update conditions.

[0183] A further embodiment according to the invention comprises a computer program for performing the method according to any of the embodiments, when the computer program runs on a computer.

[0184] In the following, embodiments related to bitstreams will be described. It should be noted that such embodiments may be based on the same, similar or corresponding considerations as the above-mentioned embodiments related to the device and / or method. Thus, the following embodiments may include the same, similar or corresponding features, functions and details as the above-disclosed embodiments, both individually and in combination.

[0185] A further embodiment according to the invention comprises a bitstream representing audio content, the bitstream comprising a number of packets of different packet types, for example with packet types conforming to the MPEG-H MHAS packet definition.

[0186] Additionally, the packet may include one or more scene configuration packets, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information (e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the use of scene objects and / or the use of scene properties).

[0187] In addition, the packet includes one or more scene update packets, such as mpegiSceneUpdate[] (sometimes referred to as "mpeghiSceneConfigUpdate"), that define scene metadata updates for rendering, such as changes to parameters of scene objects or changes to scene characteristics, and include expressions of one or more update conditions.

[0188] For example, the bitstream may be optionally supplemented with any of the bitstream elements disclosed herein, both individually and in combination.

[0189] In the following, embodiments according to the third aspect of the invention are described, which may be based on the use of timestamps and / or evaluation of timestamp information, as well as other features.

[0190] Embodiments according to the third aspect may include, individually or in combination, feature features and details of the embodiments of the first and / or second aspects of the invention and vice versa.

[0191] An embodiment according to the invention comprises an audio decoder for providing a decoded and optionally rendered audio representation based on the encoded audio representation, the audio decoder being configured to spatially render one or more audio signals and to receive a plurality of packets of different packet types, e.g. having a packet type conforming to the MPEG-H MHAS packet definition, the packets including a plurality of scene configuration packets, e.g. Scene ConfigPackets, e.g. mpegiSceneConfig[] (sometimes called "mpeghiSceneConfig[]"), providing renderer configuration information defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g. using cell definitions, defining the temporal evolution of the rendering scenario, e.g. defining the usage of scene objects and / or the usage of scene properties, and including timestamp information.

[0192] Furthermore, the audio decoder is configured to evaluate the timestamp information and set the rendering configuration to the rendering scenario that corresponds to the timestamp, for example using the renderer configuration information, when the audio decoder tunes to the stream.

[0193] The inventors have realised that audio scenes having a temporal evolution can be rendered efficiently by adjusting the renderer configuration of the respective decoder. Thus, a decoder may be provided with, for example, a scene configuration packet providing renderer configuration information defining a rendering scenario, for example the temporal evolution of an audio scene.

[0194] Therefore, the decoder may be provided with information defining when to use each renderer configuration, provided for example in a scene configuration packet, to better render an audio scene or rendering scenario.

[0195] Thus, the scene configuration packet includes timestamp information. Thus, upon evaluation of the timestamp information, a decoder or renderer can, for example, set a rendering configuration, which is suitable for rendering a rendering scenario at a certain point in time, as defined, for example, by the timestamp information. For example, a renderer configuration can be, for example, activated when the play time matches or exceeds the time defined by the timestamp information.

[0196] A renderer configuration may for example define which acoustically relevant objects or scene properties should be taken into account for an accurate rendering of an audio scene.

[0197] Thus, for example, based on the time difference information extracted from the reference time and the timestamp information, or optionally directly, it can be determined when the respective renderer configuration provided by the scene configuration packet is used. Thus, the timestamp may be, for example, any entity of information suitable for defining when the respective renderer configuration is used.

[0198] This allows for efficient incorporation of point-in-time or time-lapse triggered audio scene changes.

[0199] Furthermore, it should be emphasized that one advantage of this approach may be the aggregation of trigger information, for example in the form of configuration information (e.g., how to set the renderer configuration) but also time information regarding adaptation (e.g., when to set the renderer configuration). This may make it possible to provide self-consistent and / or self-contained and / or autonomous update information.

[0200] In addition, it should be noted that the scene configuration packets may optionally contain information about which payload packets (e.g., mpegiScenePayload (sometimes called "mpeghiScenePayload")) may be needed at a given point in time and / or space, as explained above. Thus, information about the temporal evolution of the rendering scenario may optionally be used as well to adapt metadata of, e.g., acoustically related objects and / or scene characteristics, e.g., as defined in the payload packets, based on timing information.

[0201] Thus, the timestamp information ensures that the rendering configuration is properly set to match (or match) the actual play time.

[0202] As an example, the rendering configuration may include information or settings that are based on, for example, renderer configuration information, among other information, for example, user input information and / or previous input information and / or previous packet information.

[0203] According to a further embodiment of the invention, the audio decoder is configured to evaluate the timestamp information when the audio decoder misses one or more previous scene composition packets of the stream or when the audio decoder tunes into the stream.

[0204] In addition, the audio decoder is configured to set the playback time, or optionally the scene time, depending on the timestamp information contained in the scene configuration packets.

[0205] Thus, the decoder can efficiently determine and therefore set the time (e.g., relative or absolute time) at which the associated audio information should be displayed, e.g., based on the current playback time, e.g., after a disturbance, or when newly joining or rejoining a stream.

[0206] According to a further embodiment of the invention, the audio decoder is configured to perform, or optionally equivalently retrace or reconstruct, the temporal evolution of the rendered scene up to a playback time, or for example a scene time, defined by the timestamp information when the audio decoder has missed one or more previous scene composition packets of the stream or when the audio decoder has tuned into the stream.

[0207] Thus, as an example, a decoder or renderer may capture a series of updates, eg metadata updates, based on the scene composition packets, eg taking into account timestamp information.

[0208] According to a further embodiment of the present invention, the audio decoder is configured to obtain time scale information contained in a packet, for example a scene configuration packet, for example an mpegiSceneConfig() packet (sometimes referred to as "mpeghiSceneConfig").

[0209] Furthermore, the audio decoder may be configured to evaluate the timestamp information using timescale information, e.g. the audio decoder may be configured to evaluate the timestamp information on a timescale defined by timescale information, e.g. said timescale information configuring an interpretation of said timestamp information, and optionally also other temporal information, in relation to a clock source of said audio decoder or said renderer. Thus, temporal or timing information can be efficiently provided and determined.

[0210] According to a further embodiment of the invention, the audio decoder is arranged to determine which scene objects to use for rendering depending on the timestamp information, thus allowing for an efficient representation of time-dependent scene changes that can be modelled by objects.

[0211] According to a further embodiment of the invention, the audio decoder is configured to evaluate scene composition packets that define the evolution of a rendered scene starting from a point in time before the time defined by the time stamp information.

[0212] In addition, the audio decoder is configured to derive a scene composition associated with the point in time defined by the time stamp information based on information in the scene composition packet.

[0213] Thus, the evolution of audio scene updates prior to the time of the timestamp information can be taken into account by the audio decoder, for example, to determine the scene configuration present at the time of the timestamp information. Thus, tuning of the audio decoder in the audio stream is possible without having self-consistent scene configuration information associated with the time of the timestamp information. This allows for a reduction in bit rate while still providing a good auditory experience.

[0214] According to a further embodiment of the present invention, the audio decoder is configured to use one or more scene update packets, and preferably also use information in the scene configuration packets, to derive a scene configuration associated with a point in time defined by the time stamp information.

[0215] Thus, the embodiments provide good flexibility regarding the renderer configuration information provided packets.

[0216] According to a further embodiment of the present invention, the scene configuration packet, and optionally one or more scene update packets and one or more scene payload packets, conform to the MPEG-H MHAS packet definition.

[0217] Thus, embodiments in accordance with the present invention can be used in conjunction with or in accordance with existing audio coding frameworks or standards.

[0218] According to further embodiments of the present invention, the scene configuration packet, and optionally one or more scene update packets and one or more scene payload packets, each include a packet type identifier, e.g., MHASPacketType, a packet label, e.g., MHASPacketLabel, packet length information, e.g., MHASPacketLength, and a packet payload, e.g., MHASPacketPayload.

[0219] As an example, the packet label may provide an indication of which packets belong together. For example, different labels may be used to assign different MPEG-H 3D audio composition structures to a particular sequence of MPEG-H 3D audio access units. The packet length information may indicate the length of the packet payload. The inventors have recognized that the use of such data structures may enable efficient processing of packets.

[0220] According to a further embodiment of the present invention, the audio decoder is configured to extract one or more scene configuration packets, and optionally one or more scene update packets, and one or more scene payload packets, from a bitstream comprising a plurality of MPEG-H packets, including packets representing one or more audio channels to be rendered.

[0221] Optionally, the audio decoder may be configured to extract one or more scene configuration packets, one or more scene update packets, and one or more scene payload packets from an interleaved sequence of packets including packets of different types (e.g., from an interleaved sequence of MHAS packets), e.g., using packet type identifiers and / or packet labels included within the packets.

[0222] Thus, smaller data portions may be transmitted, for example, in an interleaved manner. For example, in a broadcast scenario, this may allow keeping the broadcast bitstream data rate low.

[0223] According to a further embodiment of the invention, the audio decoder is arranged to receive one or more scene composition packets via a broadcast stream, for example via a low bitrate broadcast stream.

[0224] This may allow, for example, distribution of renderer configuration information to multiple decoders or renderers with limited usage of transmission resources.

[0225] According to a further embodiment of the invention, the audio decoder is configured to tune to the broadcast stream and determine a playback time based on the timestamp of a first scene composition packet identified by the audio decoder after tuning.

[0226] The inventors have recognized that by enabling a timestamp determination based on the first scene configuration packet identified by the audio decoder after tuning, the audio decoder can quickly set the correct current play time in order to correctly render the audio information.

[0227] In the following embodiments, embodiments are described relating to an apparatus for providing an encoded audio representation, e.g. an encoder. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments relating to a decoder. Thus, the following embodiments may include, individually and in combination, the same, similar or corresponding features, functions and details as the above-disclosed embodiments.

[0228] A further embodiment according to the invention comprises an apparatus for providing an encoded audio representation, such as an audio encoder or an audio server, the apparatus being configured to provide information for spatial rendering of one or more audio signals.

[0229] The apparatus is further configured to provide a plurality of packets of different packet types, e.g. having a packet type conforming to an MPEG-H MHAS packet definition, the packets including a plurality of scene configuration packets, e.g. Scene ConfigPacket, e.g. mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig[]"), providing renderer configuration information defining the temporal evolution of a rendering scenario, e.g. defining the usage of scene objects and / or the usage of scene properties, e.g. defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g. using cell definitions, and including timestamp information.

[0230] According to a further embodiment of the invention, the device is configured to provide time scale information in one of the packets, for example in a scene composition packet, and the timestamp information is provided in a representation related to the time scale information.

[0231] According to a further embodiment of the present invention, the apparatus is configured to provide a scene configuration packet, and optionally one or more scene update packets and one or more scene payload packets, such that the scene configuration packet, and optionally one or more scene update packets and one or more scene payload packets, conform to an MPEG-H MHAS packet definition.

[0232] According to a further embodiment of the present invention, the apparatus is configured to provide a scene configuration packet, and optionally one or more scene update packets and one or more scene payload packets, such that the scene configuration packet, and optionally one or more scene update packets and one or more scene payload packets, each include a packet type identifier, e.g., MHASPacketType, a packet label, e.g., MHASPacketLabel, packet length information, e.g., MHASPacketLength, and a packet payload, e.g., MHASPacketPayload.

[0233] According to a further embodiment of the present invention, the apparatus is configured to provide a bitstream comprising a plurality of MPEG-H packets, also referred to as MHAS or MHAS stream, for example including packets representing one or more audio channels to be rendered, one or more scene configuration packets, optionally one or more scene update packets and one or more scene payload packets.

[0234] According to a further embodiment of the present invention, the apparatus is configured to provide a bitstream comprising a plurality of MPEG-H packets, also referred to as MHAS or MHAS stream, for example comprising packets representing one or more audio channels to be rendered, interleaved with one or more scene configuration packets, optionally one or more scene update packets and one or more scene payload packets.

[0235] According to a further embodiment of the present invention, the apparatus is configured to periodically repeat a scene configuration packet between two subsequent scene configuration packets, e.g. by changing only the timestamp, with one or more scene payload packets (e.g. payload 1, payload 2) and one or more packets representing one or more audio channels to be rendered (e.g. MPEG H3 DA FRAME) (and optionally also one or more scene update packets), e.g. the apparatus provides for periodically repeating a scene configuration packet between two subsequent scene configuration packets, e.g. by changing only the timestamp, with one or more packets representing one or more audio channels to be rendered (e.g. MPEG H3 DA FRAME) (and optionally also one or more scene update packets).

[0236] This may enable an efficient tuning procedure for decoders or renderers joining the stream.

[0237] According to a further embodiment of the invention, the device is configured to cyclically repeat, between two subsequent scene configuration packets, e.g. in a broadcast bitstream, one or more scene payload packets, e.g. payload 1, payload 2, one or more packets representing one or more audio channels to be rendered, e.g. MPEG H3 DAFRAME, and optionally again one or more scene update packets, optionally changing only the timestamp of the scene configuration packets.

[0238] According to a further embodiment of the invention, the device is configured to periodically repeat scene configuration packets, e.g. in a broadcast bitstream, between two subsequent scene configuration packets, including one or more packets representing one or more audio channels to be rendered, e.g. MPEG H3 DAFRAME, and optionally also one or more scene update packets, optionally with only a change or update of the timestamp.

[0239] Furthermore, the device is configured to provide or request one or more scene payload packets upon request, for example upon request of an audio decoder or renderer.

[0240] Optionally, the device may be configured to periodically repeat scene configuration packets with one or more packets representing one or more audio channels to be rendered (e.g., MPEG H3 DAFRAME) between two subsequent scene configuration packets (and optionally one or more scene update packets), e.g., the device provides scene payload packets only on request.

[0241] Thus, the scene configuration packets may, for example, be small packets, and each decoder or renderer may determine, for example, based on the scene configuration packets, which payload packets it needs, and the payload packets may, for example, request them via a backchannel from the above-mentioned device, which may be configured to provide it, thus keeping the broadcast bitstream data rate low.

[0242] According to a further embodiment of the invention the device is arranged to, for example periodically, provide a plurality of otherwise identical scene construction packets which differ only in the timestamp information.

[0243] Thus, for example, only the timestamp information may be updated, thus keeping the adaptation effort low while still providing information about the current play time.

[0244] According to a further embodiment of the invention the device is arranged to adapt the timestamp information to a playback time, e.g. a scene time.

[0245] According to a further embodiment of the invention, the device is configured to adapt the timestamp information to a playback time of the rendering scene information contained in the packet, such as an indendent playback time, such as an intended playback time, such as an independent playback time, such as a scene time, and the packets are provided by the device in the time context of the respective scene configuration packets in which the respective timestamp information is contained.

[0246] In the following embodiments, embodiments relating to a method for providing a decoding and encoding audio representation are described. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments related to the device. Thus, the following embodiments may include the same, similar or corresponding features, functions and details as the above-disclosed embodiments, individually and in combination.

[0247] Further embodiments of the present invention include a method for providing a decoded, optionally rendered, audio representation based on an encoded audio representation, the method comprising spatially rendering one or more audio signals and receiving a plurality of packets of different packet types, e.g. having a packet type conforming to an MPEG-H MHAS packet definition, the packets including a plurality of scene configuration packets, e.g. Scene ConfigPacket, e.g. mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig[]"), providing renderer configuration information defining the temporal evolution of a rendering scenario, e.g. defining the usage of scene objects and / or the usage of scene properties, e.g. defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g. using cell definitions, and including timestamp information.

[0248] Further, the method includes evaluating the timestamp information and setting a rendering configuration to a rendering scenario that corresponds to the timestamp, for example using the renderer configuration information, when an audio decoder tunes into the stream.

[0249] Further embodiments of the present invention include a method for providing an encoded audio representation, the method comprising providing information for spatial rendering of one or more audio signals and providing a plurality of packets of different packet types, e.g. having a packet type conforming to an MPEG-H MHAS packet definition, the packets including a plurality of scene configuration packets, e.g. Scene ConfigPacket, e.g. mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig[]"), providing renderer configuration information defining the temporal evolution of a rendering scenario, e.g. defining the usage of scene objects and / or the usage of scene properties, e.g. defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g. using cell definitions, and including timestamp information.

[0250] A further embodiment according to the invention comprises a computer program for performing the method according to any of the embodiments, when the computer program runs on a computer.

[0251] In the following, embodiments related to bitstreams will be described. It should be noted that such embodiments may be based on the same, similar or corresponding considerations as the above-mentioned embodiments related to the device and / or method. Thus, the following embodiments may include the same, similar or corresponding features, functions and details as the above-disclosed embodiments, both individually and in combination.

[0252] Further embodiments of the present invention include a bitstream representing audio content, the bitstream including a plurality of packets of different packet types, e.g. having packet types conforming to the MPEG-H MHAS packet definition, the packets including a plurality of scene configuration packets, e.g. Scene ConfigPackets, e.g. mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig[]"), providing renderer configuration information defining the temporal evolution of a rendering scenario, e.g. defining the usage of scene objects and / or the usage of scene properties, e.g. using cell definitions to define when or under what conditions different scene objects and / or scene properties should be used in the rendering process, and including timestamp information.

[0253] For example, the bitstream may be optionally supplemented with any of the bitstream elements disclosed herein, both individually and in combination.

[0254] According to a further embodiment of the present invention, there is provided an audio decoder for providing a decoded audio representation based on an encoded audio representation, the audio decoder configured to spatially render one or more audio signals and to receive a plurality of packets of different packet types, the packets including a plurality of scene configuration packets providing renderer configuration information.

[0255] Furthermore, the audio decoder is configured to evaluate the timestamp information and to set the rendering configuration to a rendering scenario that corresponds to the timestamp.

[0256] It should be noted that such a decoder of the present invention may include, both individually and in combination, the same, similar or corresponding features, functions and details of any of the embodiments disclosed above or any of the other embodiments disclosed herein.

[0257] In the following, an embodiment according to the fourth aspect of the invention will be described. The embodiment according to the fourth aspect of the invention may be based on the use of cell information.

[0258] Embodiments according to the fourth aspect may include, individually or in combination, feature features and details of the embodiments of the first, second and third aspects of the invention, and vice versa.

[0259] An embodiment according to the invention comprises an audio decoder for providing a decoded and optionally rendered audio representation based on the encoded audio representation, the audio decoder being configured to spatially render one or more audio signals, which may for example be encoded within the encoded audio representation.

[0260] Additionally, the audio decoder receives scene configuration packets, e.g., scene configuration packets conforming to an MPEG-H MHAS packet definition, e.g., Scene ConfigPacket, e.g., mpegiSceneConfig[ ] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information, e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the use of scene objects and / or the use of scene properties.

[0261] In addition, the scene configuration packet includes cell information (e.g., the number of cells, and for each cell, a definition of one or more cell conditions (e.g., a start timestamp and optionally an end timestamp, or a geometry identifier), as well as a definition of one or more scene payloads (e.g., payloadId[i][j] and / or a reference to a scene update packet (e.g., updateId[i])) defining one or more cells, e.g., preferably a plurality of cells (e.g., the cell information may be sub-scene cell information, although this is not necessarily the case), and optionally a definition of one or more audio streams, or information numCells indicating a definition defining one or more audio streams, for example.

[0262] Further, the cell information defines an association between one or more, e.g., temporal and / or spatial cells, e.g. having cell index i, and one or more respective data structures (e.g., payloads or payload packets (e.g., mpegiScenePayload (sometimes referred to as "mpeghiScenePayload")), e.g., payloadId[i][j], e.g., payloads defining scene objects and / or scene properties, e.g., sound sources and / or scattering objects and / or scattering surfaces and / or attenuating objects and / or attenuating surfaces, and / or material parameters and / or reverberation properties and / or portals and / or early reflections and / or late reverberation and / or diffraction properties and / or acoustic materials and / or geometric elements within a scene, e.g., a data structure identified by a data structure identifier or payload identifier, e.g., identified by payloadId) associated with the one or more cells, e.g., using a reference to a payload of a scene payload packet representing the one or more respective data structures, and defines a rendering scenario (e.g., a sub-scene rendering scenario, but not necessarily so).

[0263] In addition, the audio decoder may be configured to evaluate the cell information to determine which data structures, e.g., which scene payloads identified by payloadId[i][j], should be used for spatial rendering, e.g., at different times or different listener positions, which data structures may be included, for example, in scene payload packets.

[0264] The inventors have realised that based on the definition of a cell, a rendering scenario including a data structure can be efficiently defined. As explained above, a cell can be, for example, a temporal and / or spatial cell. In other words, an audio scene, for example aspects of an audio scene such as acoustically related objects, can be divided or separated or partitioned in time and / or space.

[0265] In other words, metadata may become needed, for example, (a) at a particular time, or (b) at a particular location in space. The "cell" concept provides a way to specify at what time and / or location, for example, a particular payload packet is needed to render a scene.

[0266] A cell may include a cell boundary that defines the geometry of the cell within the spatial audio scene. A cell may be active, for example, when a listener is within the cell boundary at a certain time and / or when a listener is within the cell boundary. A cell boundary may be, for example, any geometry, although a primitive geometry (such as an axis-aligned bounding box) may be preferred for efficient implementation.

[0267] Simply put, as an example, a cell may provide a "package" for a data structure, e.g., in the form of a payload defining scene objects and / or scene properties, which elements are associated with particular areas and / or times within the acoustic scene, and which elements may be activated or taken into account in a simple manner, e.g., by activating the respective cell.

[0268] Furthermore, the cells may be overlapping, such that a first cell provides coarse information about acoustically relevant elements of the audio scene and is refined by overlapping cells providing more detailed information, e.g. implementing a level of detail approach.

[0269] As an example, based on the cell information optionally included in the payload packets, the decoder can select the payloads required for rendering the audio scene, for example based on temporal or spatial conditions of the cell information, and thus the decoder can only request the selection of payloads that can be used and / or stored.

[0270] Thus, efficiency may be increased since less payload may have to be requested and evaluated, potentially reducing the corresponding required bitrate between the encoder and decoder.

[0271] Further, as an example, scene configuration packets can include cell information that can optionally be distributed as small bitstream packets in a broadcast channel, thus providing good availability of such basic rendering information, e.g. for efficient selection of payloads, which can increase the efficiency of the inventive concept.

[0272] According to a further embodiment of the invention, the cell information comprises a temporal definition, e.g. a time condition, e.g. start time information and / or stop time information, e.g. a start timestamp and / or an end timestamp, e.g. startTimestamp[i] and / or endTimestamp[i], of a given cell, e.g. a cell with cell index i.

[0273] Further, the audio decoder may be configured to evaluate a temporal definition of a given cell, e.g., to determine whether one or more data structures associated with a given cell, e.g., identified by payloadId[i][j], should be considered (e.g., used) in the spatial rendering, and the audio decoder may be configured to selectively use a data structure associated with a given cell, e.g., if the current playback time satisfies the temporal definition (e.g., a time condition) of the given cell.

[0274] Therefore, temporal information (eg, the temporal evolution) of an audio scene can be easily incorporated into the rendering procedure, eg, based on the cell concept.

[0275] According to a further embodiment of the invention, the cell information comprises a spatial definition of the given cell, e.g. a spatial condition, e.g. a direct geometry definition, or a reference to a definition of a geometry object, e.g. a geometry identifier, e.g. geometryId[i].

[0276] Further, the audio decoder is configured to evaluate the spatial definition of a given cell, e.g., identified by payloadId[i][j], to determine whether one or more data structures associated with the given cell should be taken into account (e.g., used) in spatial rendering.

[0277] Optionally, the audio decoder may be configured to selectively use a data structure associated with a given cell, for example, if the current listener position meets a spatial definition (e.g., a spatial condition) of the given cell.

[0278] Therefore, spatial information about the audio scene can be easily incorporated into the rendering procedure, for example based on the cell concept.

[0279] According to a further embodiment of the invention, the audio decoder is configured to evaluate number of cells information, eg numCells, contained in the scene configuration packet to determine the number of cells.

[0280] According to a further embodiment of the invention, the cell information comprises a flag, eg isTimed[i], indicating whether the cell information comprises a temporal definition of the cell or a spatial definition of the cell.

[0281] Further, the audio decoder is configured to evaluate a flag indicating whether the cell information includes a temporal definition of the cell or a spatial definition of the cell, e.g., to derive conditions when one or more data structures associated with each cell should be used for spatial rendering.

[0282] The flags can, for example, be transmitted and evaluated with low computational effort and low requirements regarding transmission resources.

[0283] According to a further embodiment of the invention, the cell information includes a reference to a geometry structure, eg geometryId[i], to define the cell, which may eg be defined in the payload packet.

[0284] Furthermore, the audio decoder is configured to evaluate a criterion of the geometric structure to obtain a geometric definition of the cell.

[0285] By using the reference information, for example, multiple complex geometries can be defined, the selection of which can be indicated by the reference information, for example using a list or look-up table that can be common to both the encoder and the decoder, so that the transmission of extensive information defining the geometry itself can be omitted.

[0286] It should be noted that in general, a cell may define a spatial and / or temporal division of an audio scene, including acoustically related elements that may be described using, for example, payload elements. Thus, a cell may be associated with a payload or a payload packet. However, the cells themselves may also optionally represent acoustically related objects, and their geometry may, for example, represent the geometry of the elements.

[0287] According to further embodiments of the present invention, the audio decoder is configured to obtain from a global payload packet a definition of a geometry structure, e.g. a geometry structure referenced by geometryId that defines a geometric boundary of a cell, a reference to a global payload packet may be included in, e.g. a scene configuration packet, the global payload packet may define data structures such as scene objects and / or scene properties that are used in multiple cells and / or that should be globally available.

[0288] The present inventors have recognised that geometry structures, for example particularly frequently used geometry structures, may be efficiently defined by global payload packets that may be globally defined and therefore globally available.

[0289] According to a further embodiment of the invention, the audio decoder is configured to identify one or more current, e.g. temporal and / or spatial, cells, e.g. using the current play time and / or the current position and optionally a temporal and / or spatial definition of the cells.

[0290] Further, the audio decoder is configured to perform spatial rendering, e.g., selectively using one or more data structures, e.g., payloads, associated with the one or more identified current cells, and optionally also using one or more globally required payloads.

[0291] As an example, the current cell may for example be the active cell, and therefore the relevant metadata, for example payload, may be taken into account for rendering the audio scene. Thus, only a limited amount of cells need to be activated, which may result in limited signaling and computational effort.

[0292] The inventors have recognised that for e.g. a particular listener (e.g. at a particular spatial location) and / or for e.g. a current playback time of an audio scene, only a portion of the available cells may need to be activated and / or taken into account.

[0293] According to a further embodiment of the invention, the audio decoder is configured to identify one or more current, e.g. temporal and / or spatial, cells, e.g. using the current play time and / or the current position and optionally a temporal and / or spatial definition of the cells.

[0294] Further, the audio decoder is configured to perform spatial rendering, e.g., selectively using one or more scene objects (e.g., audio sources and / or scattering objects and / or attenuating objects and / or obstacles) and / or scene properties (e.g., material properties, and / or propagation properties, and / or diffraction properties, and / or reflection properties) associated with one or more identified current cells, and optionally also using one or more globally required payloads.

[0295] According to a further embodiment of the invention the audio decoder is arranged to select scene objects and / or scene properties to be taken into account, eg used, in the spatial rendering depending on the cell information.

[0296] According to a further embodiment of the present invention, the audio decoder is configured to determine the current position of one or more, for example, spatially overlapping spatial cells, for example, the current listener position, using, for example, cell boundaries that may be defined in one or more globally required payloads, in order to obtain the cells identified thereby.

[0297] Furthermore, the audio decoder is configured to perform spatial rendering by selectively using one or more scene objects associated with one or more identified current cells, such as audio sources and / or scattering objects and / or attenuation objects and / or obstacles, and / or scene characteristics, such as material characteristics, and / or propagation characteristics, and / or diffraction characteristics, and / or reflection characteristics, and optionally also using one or more globally required payloads.

[0298] Thereby, the audio scene can be efficiently rendered.

[0299] According to a further embodiment of the present invention, the audio decoder is configured to determine, based on an enumeration of payload identifiers, for one or more payloads, such as a payload describing scene objects and / or scene characteristics, for example, included in the cell definition of a cell, (j = 0; j < numPayloads[i]; j++){payloadId[i][j];}, a payload mpegiPayloadElement() (which may also be referred to as "mpeghiPayloadElement") associated with one or more current cells, for example, having a cell index i.

[0300] Furthermore, the audio decoder is configured to perform spatial rendering using the determined one or more payloads, for example, without considering / ignoring other payloads associated with other cells.

[0301] Thus, by way of example, a cell may be associated with a payload, which may define metadata that may be used, for example, to incorporate acoustically relevant elements into an audio scene. Thus, based on the identity of the respective cell, the selection of which payloads to take into account (and which payloads to leave unconsidered) may be made, thus improving the efficiency of audio rendering.

[0302] According to a further embodiment of the present invention, the audio decoder is configured to perform spatial rendering using information from one or more scene update packets, e.g., mpegiSceneUpdate() (also referred to as "mpeghiSceneUpdate"), which are associated with one or more current cells and which may be identified, e.g., using the reference updateId[i] in the cell definition of the cell, where the scene update defined by the scene update packet may include, e.g., activation and / or deactivation of one or more scene objects.

[0303] As an example, updates may be associated with specific cells or with elements, e.g., payload elements, respectively associated with specific cells. Thus, scene updates may be efficiently indicated based on references to the corresponding cells.

[0304] According to further embodiments of the present invention, the audio decoder is configured to update the rendering scene using information from one or more scene update packets associated with the given cell, e.g., scene update packets specified by updateId[], in response to discovering that a given cell becomes active, e.g., in response to discovering that the position reaches or enters an area associated with the cell, and / or in response to discovering that the play time reaches or enters a time or time interval associated with the cell.

[0305] Thus, acoustically relevant elements of a scene can be updated, for example, when they are needed for rendering a scene, e.g., rendering an audio scene. The inventors have realized that based on the activation state of a cell, the need to provide an updated version of the acoustic metadata can be efficiently indicated.

[0306] According to a further embodiment of the invention, the cell information includes a scene update packet, e.g. mpegiSceneUpdate[] (also referred to as "mpeghiSceneUpdate"), and / or a reference to a scene update packet, e.g. updateId[i], that defines an update to scene metadata for rendering, e.g. a change to a parameter of a scene object or a change to a scene property, and optionally includes a representation of one or more update conditions.

[0307] Further, the audio decoder is configured to selectively perform scene metadata updates defined in a given scene update packet in response to detecting that a cell with a link to a given scene update packet has become active, e.g., the audio decoder uses an evaluation of the cell information to determine which scene objects and / or scene characteristics should be used for spatial rendering, and, via the link to the scene update packet, to determine which scene metadata updates should be performed in response to the activation of the cell.

[0308] Thus, based on the cell information, a scene update packet associated with each cell may be requested and / or obtained. As an example, upon activation of a cell, metadata associated with the cell may be updated based on the referenced scene update packet. Thus, the metadata may be updated efficiently.

[0309] According to a further embodiment of the present invention, the one or more scene update packets include a representation of one or more update conditions, and the audio decoder is configured to evaluate whether the one or more update conditions, e.g. one or more update conditions defined in the scene update packets, are met if the one or more update conditions are met, and to selectively update one or more scene metadata depending on the content of the one or more scene update packets, e.g. there are two mechanisms for triggering the updating of one or more scene metadata depending on the content of the one or more scene update packets: triggering using cells defined in the scene configuration packet and triggering using conditions defined in the scene update packet itself.

[0310] Thus, the acoustic scene can be updated efficiently.Embodiments may allow for defining and providing flexible trigger conditions, such that efficient update triggers can be provided for a number of different applications.

[0311] According to a further embodiment of the present invention, the audio decoder is configured to evaluate a time condition, e.g. a temporal trigger condition, e.g. defined by startTimestamp, contained in the scene update packet in order to decide whether to update one or more scene metadata depending on the content of the one or more scene update packets, e.g. depending on the enumeration of scene metadata items to be changed, e.g. referenced by "targetId", and their corresponding new values, e.g. defined by "attribute".

[0312] Furthermore, a time condition may define a start time, for example using the bitstream element startTimestamp, or a time condition may define a time interval, for example using a start time and an end time.

[0313] In addition, the audio decoder is configured to perform one or more scene metadata updates in response to the detection, e.g., in response to a detection that the current playback time, e.g., the scene time, has reached or is after the start time, e.g., immediately in response to the detection or with a time delay defined within the respective scene update packet, e.g., in accordance with a definition included in the respective scene update packet.

[0314] Alternatively, the audio decoder may be configured to perform one or more scene metadata updates in response to detecting that the current playback time, e.g., a scene time, is within the time interval, e.g., according to definitions included in a respective scene update package.

[0315] Alternatively or additionally, the audio decoder is configured to evaluate spatial conditions, e.g. spatial trigger conditions, e.g. spatial conditions defined by reference to a geometry definition, e.g. spatial conditions defined by geometryId, contained in the scene update packets to determine whether to update one or more scene metadata depending on the content of the one or more scene update packets, e.g. an enumeration of scene metadata items to be changed, e.g. referenced by 'targetId', and corresponding new values, e.g. defined by 'attribute'.

[0316] Thus, updates may for example be triggered by temporal and / or spatial conditions, offering good flexibility for the implementation of update conditions and thus allowing to update the acoustic scene efficiently.

[0317] According to a further embodiment of the present invention, the spatial conditions in the scene update packet define geometry elements, for example using a reference to a geometry definition, which may be included in, for example, a scene payload element.

[0318] In addition, the audio decoder is configured to perform one or more scene metadata updates in response to the detection, e.g. immediately in response to the detection, or with a time delay defined in the respective scene update packet, e.g. that the current position has reached a geometry element (e.g., has reached a one-dimensional boundary defined by a geometry element, or has reached a two-dimensional boundary defined by a geometry object, or has reached a three-dimensional boundary defined by a geometry object), or in response to the detection, e.g. immediately in response to the detection, or with a time delay defined in the respective scene update packet, e.g. that the current position is within a geometry element, e.g. within a two-dimensional geometry element or a three-dimensional geometry element, e.g. according to a definition included in the respective scene update packet.

[0319] Thus, the inventors have realized that the spatial condition can represent or define a geometric element, for example in the form of a cell. The geometric element can be, for example, a spatial portion of an audio scene, for example a rendering scenario. Acoustically relevant metadata can thus be determined, as an example, based on the position of the listener or user at which the audio scene is rendered, so that metadata associated with the geometric element, optionally describing acoustically relevant elements or properties of the scene, can be relevant when the listener or user is close to or within the geometric element. In this way, metadata associated with such geometric elements (for example metadata describing elements spatially located around or within the geometric element in the audio scene) can be updated in order to render the audio scene with high quality.

[0320] According to a further embodiment of the present invention, the audio decoder is configured to evaluate whether an interactivity trigger condition is met, which may for example be defined in a scene update packet (e.g. a condition that the user takes a particular action beyond mere movement within the scene, e.g. a condition that the user gives a certain command or activates a certain button, defined for example by the flag "fireOn"), in order to determine whether one or more scene metadata should be updated depending on the content of the one or more scene update packets.

[0321] Event-triggered updates may therefore be incorporated into the rendering of an audio scene. This may be particularly advantageous in the context of VR applications where, for example, a digital twin of an object such as a machine or building is simulated. The acoustic environment may change based on interaction with the object's digital twin, rather than just based on the relative position to the object or time in the simulation.

[0322] According to a further embodiment of the invention, the audio decoder is configured to evaluate the cell information in order to determine at what point in time, e.g. at what playback time and / or in what area of ​​listener position, which data structure, e.g. which payload specified by a payload identifier, e.g. payloadId, is required for spatial rendering or should e.g. be used.

[0323] Thus, cell information can be compact information that allows for determining, associating, or connecting acoustically relevant information of an audio scene (e.g., location (e.g., area), time (e.g., playback time), and acoustic characteristics at location and time (e.g., as defined by the payload)), thus enabling efficient rendering of the audio scene.

[0324] According to further embodiments of the present invention, the audio decoder is configured to spatially render one or more audio signals when the listener position is within a first spatial region, e.g. within a first cell, using a first set of scene objects and / or scene properties, e.g. using scene objects and / or scene properties referenced in cell information associated with the first cell, and optionally using global scene objects and / or scene properties that apply to all cells.

[0325] Further, the audio decoder is configured to spatially render the one or more audio signals when the listener position is within a second spatial region, e.g. within a second cell, using a second set of scene objects and / or scene properties, e.g. using scene objects and / or scene properties referenced in cell information associated with the second cell, and optionally using global scene objects and / or scene properties that apply to all cells.

[0326] Further, for example, the first set of scene objects and / or scene characteristics may include a larger number of scene objects and / or scene characteristics than the second set of scene objects and / or scene characteristics, e.g., the first second spatial region may be closer to the sound source than the first spatial region, so that the first set of scene objects and / or scene characteristics provides a more detailed spatial rendering when compared to the second set of scene objects and / or scene characteristics.

[0327] The inventors have recognized that the level of detail concept may enable efficient rendering of audio scenes. As an example, a level of detail (LOD) decomposition of an audio scene may be performed, e.g., when far away from a geometric structure, a coarse geometric representation of the structure may be sufficient (e.g., having a small number of reflective surfaces), while when close to the same structure, the reflection of the geometric structure and other effects related to the geometric acoustics should be rendered with a higher LOD (e.g., having a large number of reflective surfaces). This may be achieved, for example, by specifying one cell for the neighborhood of the considered geometry that contains a high LOD geometric representation and one cell for the rest of the scene that contains a low LOD geometric representation.

[0328] Thus, in general, cells may overlap each other and / or contain or include each other. It should be noted that, in general, cells may therefore be used to define different degrees of generalization or abstraction of an audio scene. For example, a first cell may be associated with coarse metadata for the cell's location and for example the rendering of the audio scene at the start-up time. A second overlapping cell may provide refined or additional metadata information for a finer rendering of the audio scene.

[0329] Thus, the level of detail may be scaled not only with respect to the distance to a particular element, but also with respect to available bandwidth and / or other considerations that may make scaling the level of detail necessary or beneficial.

[0330] Thus, an audio scene can be rendered with variable levels of granularity.

[0331] According to a further embodiment of the present invention, the audio decoder is configured to request, e.g., using a backchannel to the packet provider, one or more scene payload packets, which may include, e.g., data structures referenced in the cell information, from the packet provider, e.g., in response to a determination by the audio decoder that one or more scene payload packets, or content of one or more scene payload packets, are required for rendering.

[0332] Thus, traffic on, for example, a broadcast channel can be reduced since each decoder can individually request only those payload packets that it needs.

[0333] According to a further embodiment of the invention, the audio decoder is configured to use a payload identifier, e.g. payloadId[i], included in the cell information to identify one or more data structures used for spatial rendering.

[0334] Based on the payload identifier, for example, the data structure can be efficiently identified.

[0335] According to a further embodiment of the present invention, the audio decoder is configured to request one or more scene payload packets from a packet provider, e.g. using a back channel, based on cell information, e.g. based on cell information that one or more scene payload packets, or content of one or more scene payload packets, are required for rendering by the audio decoder.

[0336] The inventors have recognised that, for example, in a broadcast scenario, if a decoder or renderer is configured to identify and request missing scene payload packets, e.g., via a separate channel, and thus takes responsibility, traffic on the broadcast channel may be reduced.

[0337] According to a further embodiment of the invention, the audio decoder is configured to request one or more scene payload packets from a packet provider using a payload ID included in the cell information, e.g. using an ID associated with a payload element.

[0338] Alternatively, the audio decoder is configured to request one or more scene payload packets from a packet provider using a packet ID, for example an ID associated with a scene payload packet.

[0339] The payload and / or packet ID, for example, may be an efficient means to identify and request missing scene payload packets, thus keeping request transmission costs low.

[0340] According to further embodiments of the present invention, the audio decoder is configured to anticipate, e.g. using prediction, which one or more data structures, e.g. which one or more PayloadElements, are needed or expected to be needed, e.g. using cell information to anticipate which cells will be active next or have a defined likelihood of being active next, and to request the one or more data structures, or one or more scene payload packets containing the one or more data structures, before the data structures are actually needed.

[0341] Thus, the time constraints for the transmission of the respective data structures can be relaxed, which can be particularly advantageous, for example, in scenarios where an audio scene has to be rendered in real time and / or where the audio scene can change in a real-time but predictable way, e.g., how an event triggers subsequent further events that change the acoustic properties of the scene is known or at least can be modeled in advance.

[0342] According to a further embodiment of the invention the audio decoder is arranged to extract payloads identified by the cell information from the bitstream, for example from payload packets of the bitstream, for example from payload packets of a broadcast bitstream.

[0343] According to a further embodiment of the invention, the audio decoder is arranged to use the cell information to track the required data structure, for example the payload identified by the payload identifier payloadId[i].

[0344] Thus, the decoder may be, for example, at least nearly always up to date regarding, for example, acoustically relevant elements of a scene, for example up to the scene time or play time, which allows the use of small incremental updates that may, for example, reduce transmission costs.

[0345] According to further embodiments of the invention, the audio decoder is configured to selectively discard one or more data structures depending on the cell information, e.g. a payload identified by a payload identifier payloadId[i], in response to finding, e.g. using the cell information, that the current play time is after the time interval (defined in the cell information) for which a data structure is required, and / or in response to finding that the current listener position is sufficiently far from a geometric cell boundary of the cell for which a data structure is required.

[0346] Thus, the audio scene information can be efficiently updated based on separation of the acoustic scene or aspects thereof within a cell.

[0347] According to a further embodiment of the invention, the cell information defines a location-based and / or time-based subdivision of the rendered scene.

[0348] This allows for a reduction in the complexity of rendering scenarios.

[0349] According to a further embodiment of the invention, the audio decoder is configured to obtain e.g. definitions of temporal and / or spatial cells based on a scene configuration data structure, e.g. a scene configuration packet, e.g. SceneConfig, which may be located at the beginning of a file or stream and which may optionally be repeated within the stream, and the decoder may be configured to parse the file or stream, e.g. for the scene configuration packets.

[0350] The present inventors have realised that using the scene composition data structure of the present invention, cell definitions can be provided efficiently.

[0351] According to a further embodiment of the invention, the audio decoder is configured to request one or more data structures, e.g. payloads or payload packets, using a respective data structure identifier, e.g. a payload identifier or a payload packet identifier, e.g. payloadId.

[0352] In addition, the audio decoder is configured to derive data structure identifiers for the requested data structures using the cell information, e.g. by identifying in which one or more cells the current position is located and, optionally, by providing a request message including data structure identifiers associated with the one or more identified cells, e.g. by identifying one or more cells associated with the current time information and providing a request message including data structure identifiers associated with the one or more identified cells.

[0353] As previously mentioned, by way of example, traffic on the broadcast channel can be reduced by implementing on-demand data structure serving.

[0354] According to a further embodiment of the invention, the audio decoder is configured to anticipate, e.g. using prediction, which one or more data structures or are expected to be needed, e.g. which cells will be active next or have a defined likelihood of being active next, and to request one or more data structures before the data structures are actually needed.

[0355] As mentioned above, this can, for example, relax time constraints on the transmission of required data structures in real-time applications.

[0356] According to a further embodiment of the invention, the audio decoder is configured to extract one or more data structures, e.g. payloads or payload packets, using a respective data structure identifier, e.g. a payload identifier or a payload packet identifier, e.g. payloadId.

[0357] In addition, the audio decoder is configured to use the cell information to derive data structure identifiers for the extracted and optionally evaluated data structures.

[0358] According to a further embodiment of the invention, the audio decoder is configured to extract metadata required for rendering, e.g. rendering of complex and / or dynamic 6DoF audio scenes, from payload packets, e.g. "Scene Payload" packets.

[0359] Optionally, a scene payload packet may contain multiple payload elements, eg, mpegiPayloadElements (also called "mpeghiPayloadElements"), which are assigned payload element identifiers, eg, IDs.

[0360] In the following embodiments, embodiments are described relating to an apparatus for providing an encoded audio representation, e.g. an encoder. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments relating to a decoder. Thus, the following embodiments may include, individually and in combination, the same, similar or corresponding features, functions and details as the above-disclosed embodiments.

[0361] A further embodiment according to the invention comprises an apparatus for providing an encoded audio representation, e.g. an audio encoder or an audio server, configured to provide information for spatial rendering of one or more audio signals, and to provide a plurality of packets of different packet types, e.g. having packet types conforming to an MPEG-H MHAS packet definition.

[0362] Further, the device provides a scene configuration packet, e.g., a scene configuration packet conforming to an MPEG-H MHAS packet definition, e.g., a Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information, e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the use of scene objects and / or the use of scene properties.

[0363] In addition, the scene configuration packet includes cell information (e.g., information indicating the number of cells, numCells, and, for each cell, a definition of one or more cell conditions (e.g., a start timestamp and optionally an end timestamp, or a geometry identifier) ​​and a definition of one or more scene payloads (e.g., payloadId[i][j] and / or a reference to a scene update packet (e.g., updateId[i])), defining one or more cells, preferably multiple cells, and optionally also including a definition of one or more audio streams.

[0364] Further, the cell information defines an association between one or more, e.g., temporal and / or spatial cells, e.g. having cell index i, and one or more respective data structures (e.g. payloads or payload packets, e.g., payloadId[i][j], e.g. payloads defining scene objects and / or scene properties, e.g. data structures defining sound sources and / or scattering objects and / or scattering surfaces and / or attenuating objects and / or attenuating surfaces, and / or material parameters and / or reverberation properties and / or portals and / or early reflections and / or late reverberation and / or diffraction properties and / or acoustic materials and / or geometric elements in a scene, e.g. data structures identified by a data structure identifier or payload identifier, e.g., payloadId) associated with the one or more cells, e.g. using a reference to the payload of a scene payload packet representing the one or more respective data structures, to define a rendering scenario.

[0365] The audio decoder may optionally provide any of the packets disclosed herein, both individually and in combination, with respect to the audio encoder. Further, the cell information may include any of the characteristics disclosed herein, both individually and in combination, with respect to the audio decoder, for example.

[0366] According to further embodiments of the invention, the apparatus is configured to periodically repeat providing a scene configuration packet, or even a sequence of, for example, a scene configuration packet and one or more scene payload packets, and optionally one or more scene update packets. Alternatively or additionally, the apparatus is configured to provide one or more scene payload packets on request, for example upon request of an audio decoder or renderer.

[0367] Briefly, as an example, the role of distributing the scene configuration packets may be played by the encoder. Optionally, the encoder may also decide which payload packets to provide and when, e.g., broadcast the same or parts of them, e.g., periodically, together with the scene configuration packets. On the other hand, the decoder may explicitly request the required payloads, so that the respective payload packets are optionally provided only, e.g., via a unicast channel.

[0368] According to a further embodiment of the invention, the apparatus is configured to provide one or more scene payload packets including one or more data structures referenced in the cell information.

[0369] According to a further embodiment of the present invention, the device is configured to provide the scene payload packet taking into account when the data structures contained in the scene payload packet are required by an audio decoder in accordance with the cell information.

[0370] Thus, as an example, the encoder can ensure, or at least make it more likely, that the necessary information is present at the decoder or renderer in time, for example to prevent buffering times or audio lag.

[0371] According to a further embodiment of the invention, the audio encoder or decoder is configured to provide first cell information defining a first set of scene objects and / or scene properties for rendering the scene when the listener's position is within a first spatial region, such as within a first cell.

[0372] Further, the audio encoder or decoder is configured to provide second cell information defining a second set of scene objects and / or scene properties for rendering the scene when the listener's position is within a second spatial region, e.g. within a second cell.

[0373] For example, the first set of scene objects and / or scene characteristics may include a larger number of scene objects and / or scene characteristics than the second set of scene objects and / or scene characteristics, e.g., the first second spatial region may be closer to the sound source than the first spatial region, so that the first set of scene objects and / or scene characteristics provides a more detailed spatial rendering when compared to the second set of scene objects and / or scene characteristics.

[0374] According to further embodiments of the invention, the device may be configured to use different cell definitions to control spatial rendering at different levels of detail, e.g. depending on whether the listener's position is within a first cell or a second cell, e.g. a cell relatively closer to a sound source may contain more data structures (e.g. data structures describing reflective and / or absorbing surfaces, and / or scattering and / or absorbing objects, etc.) than a cell relatively farther from the sound source.

[0375] The inventors have realised that by using different cell definitions or different categories of cell definitions it may be possible to efficiently implement the concept of level of detail such that the quality of the rendering of an audio scene may be scalable.

[0376] In the following embodiments, embodiments relating to a method for providing a decoding and encoding audio representation are described. It should be noted that such embodiments may be based on the same or similar or corresponding considerations as the above-mentioned embodiments related to the device. Thus, the following embodiments may include the same, similar or corresponding features, functions and details as the above-disclosed embodiments, individually and in combination.

[0377] Further embodiments according to the invention include a method for providing a decoded and, optionally, rendered audio representation based on an encoded audio representation, the method comprising spatially rendering one or more audio signals, which may for example be encoded within the encoded audio representation.

[0378] Additionally, the method includes receiving a scene configuration packet, e.g., a scene configuration packet conforming to an MPEG-H MHAS packet definition, e.g., a Scene ConfigPacket, e.g., mpegiSceneConfig[ ] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information, e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the usage of scene objects and / or the usage of scene properties.

[0379] The scene configuration packet further includes cell information (e.g., the number of cells, and for each cell, a definition of one or more cell conditions (e.g., a start timestamp and optionally an end timestamp, or a geometry identifier), as well as a definition of one or more scene payloads (e.g., payloadId[i][j] and / or a reference to a scene update packet (e.g., updateId[i])) that define one or more cells, e.g., preferably a plurality of cells, and optionally information numCells indicating the definition of one or more audio streams).

[0380] In addition, the cell information defines an association between one or more, e.g., temporal and / or spatial cells, e.g. having cell index i, and one or more respective data structures (e.g., payloads or payload packets, e.g., payloadId[i][j], e.g., payloads defining scene objects and / or scene properties, e.g., data structures defining sound sources and / or scattering objects and / or scattering surfaces and / or attenuating objects and / or attenuating surfaces, and / or material parameters and / or reverberation properties and / or portals and / or early reflections and / or late reverberation and / or diffraction properties and / or acoustic materials and / or geometric elements in a scene, e.g., data structures identified by a data structure identifier or payload identifier, e.g., payloadId) associated with the one or more cells, e.g., using a reference to a payload of a scene payload packet representing the one or more respective data structures, to define a rendering scenario.

[0381] The method further includes evaluating the cell information to determine which data structures, e.g., which scene payloads identified by payloadId[i][j], should be used for spatial rendering, e.g., at different times or different listener positions, where the data structures may be included, e.g., in a scene payload packet.

[0382] Further embodiments according to the invention include a method for providing an encoded audio representation, such as an audio encoder or for an audio encoder, or an audio server or for an audio server, comprising providing information for spatial rendering of one or more audio signals and providing a plurality of packets of different packet types, such as having a packet type conforming to an MPEG-H MHAS packet definition.

[0383] Further, the method includes providing a scene configuration packet, e.g., a scene configuration packet conforming to an MPEG-H MHAS packet definition, e.g., a Scene ConfigPacket, e.g., mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig"), providing renderer configuration information, e.g., defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g., defining the usage of scene objects and / or the usage of scene properties.

[0384] The scene configuration packet further includes cell information (e.g., the number of cells, and for each cell, a definition of one or more cell conditions (e.g., a start timestamp and optionally an end timestamp, or a geometry identifier), as well as a definition of one or more scene payloads (e.g., payloadId[i][j] and / or a reference to a scene update packet (e.g., updateId[i])) that define one or more cells, e.g., preferably a plurality of cells, and optionally information numCells indicating the definition of one or more audio streams).

[0385] Further, the cell information defines an association between one or more, e.g., temporal and / or spatial cells, e.g. having cell index i, and one or more respective data structures (e.g. payloads or payload packets, e.g., payloadId[i][j], e.g. payloads defining scene objects and / or scene properties, e.g. data structures defining sound sources and / or scattering objects and / or scattering surfaces and / or attenuating objects and / or attenuating surfaces, and / or material parameters and / or reverberation properties and / or portals and / or early reflections and / or late reverberation and / or diffraction properties and / or acoustic materials and / or geometric elements in a scene, e.g. data structures identified by a data structure identifier or payload identifier, e.g., payloadId) associated with the one or more cells, e.g. using a reference to the payload of a scene payload packet representing the one or more respective data structures, to define a rendering scenario.

[0386] A further embodiment according to the invention comprises a computer program for performing the method according to any of the embodiments disclosed herein, when the computer program runs on a computer.

[0387] In the following, embodiments related to bitstreams will be described. It should be noted that such embodiments may be based on the same, similar or corresponding considerations as the above-mentioned embodiments related to the device and / or method. Thus, the following embodiments may include the same, similar or corresponding features, functions and details as the above-disclosed embodiments, both individually and in combination.

[0388] Further embodiments of the present invention include a bitstream representing audio content, the bitstream including a plurality of packets of different packet types, e.g. having a packet type conforming to an MPEG-H MHAS packet definition, the packets including scene configuration packets conforming to an MPEG-H MHAS packet definition, e.g. Scene ConfigPacket, e.g. mpegiSceneConfig[] (sometimes referred to as "mpeghiSceneConfig[]"), providing renderer configuration information defining when or under what conditions different scene objects and / or scene properties should be used in the rendering process, e.g. using cell definitions, e.g. defining the usage of scene objects and / or the usage of scene properties.

[0389] The scene configuration packet further includes cell information (e.g., the number of cells, and for each cell, a definition of one or more cell conditions (e.g., a start timestamp and optionally an end timestamp, or a geometry identifier), as well as a definition of one or more scene payloads (e.g., payloadId[i][j] and / or a reference to a scene update packet (e.g., updateId[i])) that define one or more cells, e.g., preferably a plurality of cells, and optionally information numCells indicating the definition of one or more audio streams).

[0390] Further, the cell information defines an association between one or more, e.g., temporal and / or spatial cells, e.g. having cell index i, and one or more respective data structures (e.g. payloads or payload packets, e.g., payloadId[i][j], e.g. payloads defining scene objects and / or scene properties, e.g. data structures defining sound sources and / or scattering objects and / or scattering surfaces and / or attenuating objects and / or attenuating surfaces, and / or material parameters and / or reverberation properties and / or portals and / or early reflections and / or late reverberation and / or diffraction properties and / or acoustic materials and / or geometric elements in a scene, e.g. data structures identified by a data structure identifier or payload identifier, e.g., payloadId) associated with the one or more cells, e.g. using a reference to the payload of a scene payload packet representing the one or more respective data structures, to define a rendering scenario.

[0391] For example, the bitstream may be optionally supplemented with any of the bitstream elements disclosed herein, both individually and in combination.

[0392] According to a further embodiment of the present invention, there is provided an audio decoder for providing a decoded audio representation based on an encoded audio representation, the audio decoder configured to receive a plurality of packets of different packet types, the packets including one or more scene configuration packets providing renderer configuration information and the packets including one or more scene update packets defining updates to scene metadata for rendering.

[0393] Further, the audio decoder is configured to evaluate whether one or more update conditions are met, and, if the one or more update conditions are met, selectively update one or more scene metadata according to content of the one or more scene update packets.

[0394] It should be noted that such a decoder of the present invention may include, both individually and in combination, the same, similar or corresponding features, functions and details of any of the embodiments disclosed above or any of the other embodiments disclosed herein.

[0395] The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.In the following description, various embodiments of the invention are described with reference to the following drawings, in which: [Brief description of the drawings]

[0396] [Figure 1] 1 is a schematic diagram of an audio decoder according to an embodiment of the first aspect of the present invention; [Diagram 2] 4 is a schematic diagram of an audio decoder according to an embodiment of the first aspect of the present invention, having further additional optional features; [Diagram 3] FIG. 2 is a schematic diagram of an encoder according to an embodiment of the first aspect of the present invention; [Figure 4] 1 is a schematic block diagram of a method for providing a decoded audio representation based on an encoded audio representation according to an embodiment of a first aspect of the present invention; [Diagram 5] 1 is a schematic block diagram of a method for providing an encoded audio representation according to an embodiment of a first aspect of the present invention; [Figure 6] FIG. 3 is a schematic diagram of an audio decoder according to an embodiment of the second aspect of the present invention; [Figure 7] FIG. 4 is a schematic diagram of an encoder according to an embodiment of the second aspect of the present invention; [Figure 8] 5 is a schematic block diagram of a method for providing a decoded audio representation based on an encoded audio representation according to an embodiment of the second aspect of the present invention; [Figure 9] 5 is a schematic block diagram of a method for providing an encoded audio representation according to an embodiment of the second aspect of the present invention; [Figure 10] 4 shows a schematic diagram of an audio decoder according to an embodiment of the third aspect of the present invention; [Figure 11] 4 shows a schematic diagram of an encoder according to an embodiment of the third aspect of the present invention; [Figure 12] FIG. 4 is a schematic block diagram of a method for providing a decoded audio representation based on an encoded audio representation according to an embodiment of the third aspect of the present invention; [Figure 13] FIG. 4 is a schematic block diagram of a method for providing an encoded audio representation according to an embodiment of the third aspect of the present invention; [Figure 14] FIG. 4 is a schematic diagram of an audio decoder according to an embodiment of the fourth aspect of the present invention; [Figure 15] 4 shows a schematic diagram of an encoder according to an embodiment of the third aspect of the present invention; [Figure 16] FIG. 10 is a schematic block diagram of a method for providing a decoded audio representation based on an encoded audio representation according to an embodiment of the fourth aspect of the present invention; [Figure 17] FIG. 4 is a schematic block diagram of a method for providing an encoded audio representation according to an embodiment of the fourth aspect of the present invention; [Figure 18] 2 shows a schematic diagram of a first bitstream according to an embodiment of the present invention; [Figure 19] 4 shows a schematic diagram of a second bitstream according to an embodiment of the present invention; [Figure 20] FIG. 2 shows a schematic diagram of a third bitstream according to an embodiment of the present invention; [Figure 21] FIG. 1 is a schematic block diagram of an architecture overview according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0397] Elements that are equal or similar or have equal or similar functionality are designated in the following description with equal or similar reference numbers even if they occur in different figures.

[0398] In the following description, a number of details are set forth to provide a more complete description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the embodiments of the present invention. In addition, the features of different embodiments described below can be combined with each other unless otherwise specified.

[0399] Fig. 1 is a schematic diagram of an audio decoder according to an embodiment of the first aspect of the present invention. Fig. 1 shows an audio decoder 100 configured to provide a decoded audio representation 106, optionally rendered as shown, based on an encoded audio representation 102. The audio decoder 100 comprises a rendering unit 110 configured to spatially render one or more audio signals. The rendering unit 110 may therefore optionally comprise a decoding unit, which may be configured to decode the encoded audio representation to obtain one or more audio signals. However, as an alternative, as indicated by dotted and dashed lines, the audio decoder 100 may comprise a decoding unit 120, which may comprise the encoded audio representation 102 and may provide one or more audio signals to the rendering unit 110.

[0400] Further, the audio decoder 100 is configured to receive multiple packets 104 of different packet types, including one or more scene configuration packets providing renderer configuration information defining the use of scene objects and / or the use of scene properties, one or more scene update packets defining updates to scene metadata 130 for rendering, and one or more scene payload packets comprising definitions of one or more of the scene objects and / or one or more of the scene properties.

[0401] Thus, based on the scene configuration packet, the renderer configuration of the rendering unit 110 can be set and / or adjusted. Based on the renderer configuration, the rendering unit 110 can determine which scene objects and / or scene characteristics to consider.

[0402] Such objects and / or properties may be defined, for example, using metadata 130. As previously described, based on scene update packets, such metadata 130 may be updated for rendering.

[0403] As an example, based on the scene payload packets, new definitions of scene objects and / or scene properties may be added to metadata 130 for rendering unit 110 and / or provided directly to rendering unit 110.

[0404] Thus, the rendering unit 110 is configured to select, for rendering, one or more scene object definitions and / or one or more scene property definitions contained in the scene payload packets in response to the renderer configuration information. In addition, as mentioned above, the decoder 100 is configured to update one or more scene metadata 130 in response to the content of one or more scene update packets.

[0405] Briefly, as an example, a decoder 100 may receive an encoded audio representation 102. This encoded audio representation may be decoded to obtain audio information. The audio information may include, for example, information about spectral coefficients of the audio signal.

[0406] However, for an accurate reconstruction of the audio scene or for an immersive listening experience, e.g. in a VR or AR environment, further effects may have to be taken into account, and therefore the decoder 100 may be configured to take into account metadata 130 for the rendering of the audio information.

[0407] This metadata may, for example, include, describe or relate to data objects that may further define characteristics or elements, e.g. objects, of the acoustic scene. The metadata may, for example, define elements in space and / or time that may cause acoustically related effects such as reverberation, reflections, etc.

[0408] In order to efficiently exploit the concept of audio metadata, packets 104 are provided to the decoder 100. The inventors have recognised that, for example, a distinction between at least three packet types may be advantageous.

[0409] Scene configuration packets may provide information about which acoustic elements and / or characteristics should be considered at a particular location or at a particular time within an audio scene, for example. To incorporate changes to acoustically relevant elements, scene update packets are introduced to allow, for example, modifying the respective metadata.

[0410] On the other hand, scene payload packets may contain information and / or definitions of acoustically relevant elements, e.g. objects or scene characteristics themselves that may be relevant for rendering an audio signal. The selection of payload elements may be performed based on scene construction information.

[0411] Moreover, it should be emphasized that the metadata 130 described and illustrated above is optional: the rendering unit may, for example, select and take into account only those acoustically relevant elements defined by the payload packets provided to the decoder 100. The selection may be performed by a rendering configuration, which may, for example, be adapted based on the scene configuration packets.

[0412] It is further noted that the separation of the input signal in packets 104 and coded audio representation 102 is an example. The coded audio representation may be provided as part of packets 104, e.g. in the form of packets containing information about spectral audio coefficients, e.g. as MPEG H3 DAFRAME. On the other hand, the decoder 100 may receive only the coded audio representation, e.g. including packets 104 as described above, containing configuration data, update data and metadata in addition to the audio information or audio signal. The optional decoding unit 120 may therefore alternatively or additionally be configured to decode coded packets.

[0413] As an optional feature, the decoder 100 is configured to determine a rendering configuration based on a scene configuration packet, e.g., using the rendering unit 110 or an optional evaluation unit (e.g., as described in the context of FIG. 2), and to determine an update of the rendering configuration of the rendering unit 110 based on one or more scene update packets.

[0414] Thus, a scene configuration packet may, for example, include a complete set of configuration parameters based on which, for example, incremental updates may be provided or performed based on the scene update packets.

[0415] Optionally, one or more scene update packets may include an enumeration of scene metadata items to be changed, where the enumeration may include, for one or more metadata items to be changed, a metadata identifier and a metadata update value. Thus, optionally, metadata 130 may be organized based on an identifier, e.g., a number, and one or more values. Such values ​​may be changed in response to the metadata update values.

[0416] As another optional feature, the audio decoder 100, e.g. the rendering unit 110 of the decoder 100, is configured to obtain definitions of one or more of the scene objects and / or definitions of one or more of the scene properties, e.g. as metadata definitions, as an example.

[0417] Reference is now made to Fig. 2, which is a schematic diagram of an audio decoder with further additional optional features according to an embodiment of the first aspect of the invention. Fig. 2 shows an audio decoder 200 comprising a rendering unit 210, an optional decoding unit 220 and metadata 230, as well as corresponding decoded and encoded audio representations 202, 206 and packets 204, as described in the context of Fig. 1.

[0418] As an optional feature, the audio decoder 200 comprises an evaluation unit 240. Optionally, one or more scene payload packets (e.g., included in packet 204) include an enumeration of payloads defining scene objects and / or scene characteristics. Furthermore, the audio decoder 100 is configured to evaluate, e.g., using the evaluation unit 240, the enumeration of payloads defining scene objects and / or scene characteristics.

[0419] As another optional feature, a payload identifier may be associated with a payload in a scene payload packet, and an audio decoder, e.g., its evaluation unit 240, may be configured to evaluate the payload identifier of a given payload to determine whether the given payload should be used for rendering in the rendering unit 210.

[0420] Thus, optionally, information regarding packets 204 may be provided to rendering unit 210 exclusively via evaluation unit 240 .

[0421] As another optional feature, one or more of the scene update packets define a condition for a scene update, and the audio decoder, e.g., its evaluation unit 240, is configured to evaluate whether the condition for a scene update defined in the scene update packet is satisfied to determine whether a scene update should be performed. Thus, by way of example, the metadata 230 may be adjusted and / or the rendering configuration of the rendering unit 210 may be adjusted.

[0422] As an optional feature, one or more of the scene update packets define an interactivity trigger condition, and the audio decoder, e.g. its evaluation unit 240, is configured to evaluate whether the interactivity trigger condition is satisfied to determine whether the scene update should be performed. Thus, by way of example, metadata 230 may be updated and / or the rendering unit 210 may be instructed to adjust rendering and / or rendering configuration. A trigger condition may for example be an event-based condition, alternatively or in addition to location-based and / or time-based conditions.

[0423] As another optional feature, the one or more scene configuration packets and the one or more scene update packets and the one or more scene payload packets, and thus the exemplary packets 204, conform to the MPEG-H MHAS packet definition.

[0424] As another optional feature, the one or more scene configuration packets, the one or more scene update packets, and the one or more scene payload packets each include a packet type identifier, e.g., MHASPacketType, a packet label, e.g., MHASPacketLabel, packet length information, e.g., MHASPacketLength, and a packet payload, e.g., MHASPacketPayload. As an example, the audio decoder 200 may be optionally configured to evaluate the packet type identifier, e.g., using the evaluation unit 240, to distinguish between packets of different packet types. Thus, the decoder can distinguish between different packets for further processing.

[0425] As another optional feature, the audio decoder 200 comprises an extraction unit 250. As another optional feature, the audio decoder 200, e.g. its extraction unit 250, is configured to extract the one or more scene configuration packets, the one or more scene update packets, and the one or more scene payload packets from a bitstream 208 that includes a plurality of MPEG-H packets, including packets representing one or more audio channels to be rendered.

[0426] As an example, the encoded audio representation 202 may include information about the audio channels to be rendered, or may be information about the audio channels to be rendered. As mentioned above, the encoded audio representation 202 may be in packets. Optionally, the extraction unit 250 may be configured to extract the encoded audio representation 202 separated from the packets 204 from the bitstream 208, as shown.

[0427] It should again be noted that the separation of the input signal in packets 204 and the coded audio representation 202 is an example. The coded audio representation may be provided as part of the packets 204, e.g. in the form of packets containing information about spectral audio coefficients, e.g. as an MPEG H3 DAFRAME. On the other hand, the decoder 200 may receive only the coded audio representation, e.g. including packets 204 as described above, containing configuration data, update data and metadata in addition to the audio information or audio signal.

[0428] As another optional feature, the bitstream 208 may be a broadcast bitstream. Thus, the decoder 200 may be configured to receive one or more scene composition packets via a broadcast stream. However, it should be noted that the packets 204 may be received by the decoder 204 via different bitstreams. The bitstreams may include broadcast and unicast bitstreams, for example, for dedicated servers or transmission requests to an encoder.

[0429] As another optional feature, the decoder 200 comprises a request unit 260. Optionally, the audio decoder 200 is configured to request one or more scene payload packets from a packet provider, for example using the request unit 260. Thus, the decoder 200 can provide the request 201. Thus, the scene payload packets may be received by the decoder via a separate bitstream (not shown), for example a unicast bitstream, via a channel used for transmitting the request 201.

[0430] To request one or more scene payload packets, the decoder 200 can optionally use a payload ID, e.g., using an ID associated with a payload element, or a packet ID, e.g., using an ID associated with a scene payload packet. Thus, the request 201 can include such an ID.

[0431] As another optional feature, the decoder 200 comprises an anticipation unit 270. As an optional feature, the audio decoder 200 is configured to anticipate, e.g., which one or more data structures are needed or expected to be needed, e.g., using the anticipation unit 270, and request the one or more data structures, or one or more scene payload packets containing the one or more data structures, before the data structures are actually needed.

[0432] Thus, the anticipation unit 270 may, for example, provide information for defining the request 201 to the request unit 260 .

[0433] As another optional feature, an audio decoder, such as, for example, a request unit 260, is configured to provide information, such as a request 201, to a packet provider, such as an encoder according to an embodiment, indicating which one or more scene payload packets are or will be needed within a given period of time.

[0434] As an optional feature, one or more scene update packets (e.g., packet 204) define updates to scene metadata for rendering, e.g., metadata 230, and include a representation of one or more update conditions. Further, as an optional feature, audio decoder 200, e.g., evaluation unit 240 thereof, is configured to evaluate whether the one or more update conditions are met, and, if the one or more update conditions are met, selectively update one or more scene metadata, e.g., metadata 230, depending on the content of the one or more scene update packets.

[0435] Reference is now made to Fig. 3. For the sake of brevity and as explained above, it is noted that an embodiment according to the invention comprises a decoder and thus an encoder having corresponding features according to the decoder shown in Fig. 1. The encoder may therefore comprise only the features necessary to provide a signal as received by the decoder 100 of Fig. 1 and to process it to provide a decoded audio representation 106, without any optional features. However, any of the optional features, functions and details disclosed above in the context of Figs. 1 and 2 may be present correspondingly (e.g. in a corresponding manner) in an encoder according to an embodiment, either individually or in combination. The same applies to the features, functions and details of the decoders and / or encoders of the other aspects of the invention.

[0436] Fig. 3 is a schematic diagram of an encoder according to an embodiment of the first aspect of the invention. The encoder 300 is configured to provide a bitstream 302 (e.g. similar, corresponding or identical to the bitstream 208 as shown in Fig. 2), which may for example comprise a coded audio representation. In particular, the encoder 300 is configured to provide information for spatial rendering of one or more audio signals, contained in the bitstream 302. The bitstream 302 thus comprises a number of packets 322 of different packet types.

[0437] To provide the bitstream and thus the above mentioned information entities, the encoder 300 comprises a bitstream provider 310 to which packets 322 are provided.

[0438] As shown, the encoder 300 may optionally comprise a packet providing unit 320. The encoder 300 is configured to provide packets 322, for example using the packet providing unit 320. The packets 322 include one or more scene configuration packets providing renderer configuration information defining use of scene objects and / or use of scene properties, one or more scene update packets defining updates to scene metadata for rendering, and one or more scene payload packets comprising definitions of one or more of the scene objects and / or one or more of the scene properties.

[0439] As an example, an audio signal 304 to be encoded may be provided to the encoder 300. The audio signal may comprise, for example, time domain samples and / or spectral values ​​of a speech signal and / or music signal. Optionally (not shown), if this signal has already been encoded, it may be provided directly to the bitstream provider 310, for example to be included in the bitstream together with the packets 322.

[0440] The packets 322 may be provided by the packet providing unit in different ways, for example, the packet providing unit 320 may provide the packets 322 based on scene information defining an acoustic scene and which may be predefined or may be obtained by the packet providing unit.

[0441] For example, in the case of a virtual reality application, based on a virtual model (e.g. of an acoustic scene), the packets may be used to determine acoustically related virtual objects to model or represent the acoustic scene (e.g. an acoustic scene including a virtual model having acoustically related virtual objects and audio signals). Optionally, the (optional) analysis unit 330 may support the determination of acoustically related virtual objects, or properties thereof (e.g. properties of the audio signals may be used to supplement and / or refine the virtual model of the acoustic scene, e.g. by providing information about the positions of sound sources).

[0442] Thus, the optional parsing unit 330 may provide information to the packet providing unit 320 to support the decision of each packet 322. For example, the packet providing unit 320 may manage a virtual model or may be provided with information about the virtual model.

[0443] As another example, in the context of an augmented reality application, the audio signal 304 to be encoded may be provided to the encoder 300 and may comprise, for example, time-domain samples and / or spectral values ​​of a speech signal and / or music signal. Optionally, however, the audio signal 304 may further comprise (or carry) spatial information of the real audio scene to be augmented (for example, in an implicit form), for example, position information of a measured audio source, for example of a speaking user, in the scene. Such information, and optionally additional virtual overlay information (for example, information for adding acoustically relevant virtual objects to the real scene to augment the real scene), may be extracted and / or analyzed and / or applied by the optional analysis unit 330 to model or represent the scene using packets. Thus, as mentioned above, information for determining each packet 322 may be provided to the packet providing unit 320. As an example, the analysis unit 330 may determine, based on the spatial information in the audio signal 304, which acoustically relevant objects of the scene should be considered or updated or rendered to provide a desired auditory experience.

[0444] However, it should be noted that the encoder 300 may optionally not comprise a parsing unit, e.g. so that information about the packets 322 may be provided to the encoder 300 from an external unit, e.g. a unit managing a virtual or augmented scene.

[0445] To further illustrate the above example, an audio signal 304 to be encoded may be provided to the encoder 300, which may be an audio signal from a spatial audio scene and may optionally further include spatial information regarding the audio scene. Thus, as an optional feature, the encoder 300 comprises an analysis unit 330. The analysis unit 330 is configured to analyze information provided from the audio scene in order to determine or approximate a representation of the audio scene. As an example, the audio scene may be modeled using metadata describing scene objects and / or scene properties that may be used, for example, together with the spectral coefficients of the audio signal to provide a listener with an immersive representation of the audio scene.

[0446] It should be noted that the metadata may be based, for example, on a digital model of the audio scene (e.g. in the case of virtual reality applications) and / or on an analysis of the actual scene from which the audio signal was recorded (e.g. for the case of augmented reality applications).

[0447] Based on this, corresponding scene configuration packets, scene update packets, and scene payload packets can be determined and provided, as an example.

[0448] Thus, as an example, the packet providing unit 320 may further provide packets containing said spectral information of the audio signal, for example in the form of packets representing one or more audio channels to be rendered (e.g., optionally packets such as MPEG H3D AFRAME), which may be part of the packets 322.

[0449] Optionally, the audio signal 304 may for example be provided directly to the packet providing unit 320 and / or to the bitstream provider 310. As an example, if the audio signal 304 has already been encoded, these packets may already include defined packets 322 such that they can only be extracted in the packet providing unit 322, for example to be encoded, for example re-encoded, in the bitstream provider 310. The audio signal information may for example be provided to the bitstream provider 310 separately from metadata information, for example in the form of packets or directly based on the audio signal 304.

[0450] It should further be noted that the analysis unit 330 may optionally be configured to determine or approximate a virtual acoustic scene, e.g. using the audio signal 304 representing only the acoustic signal itself (e.g. its spectral coefficients measured by the headset of the user in the VR room), and further spatial properties of the scene may be based on a virtual model of the surroundings in the virtual acoustic scene, e.g. based on the position of the user in the virtual surroundings. For example, in a virtual conference room, the reflection properties of the virtual walls and / or the attenuation properties of the virtual carpet may be incorporated as metadata describing the scene objects, e.g. not based on real measurements, but based on a virtual acoustic model of the walls or carpet with respect to the position of the listener.

[0451] As another optional feature, the encoder 300, e.g., the packet providing unit 320, is configured to provide the renderer configuration information included in the scene configuration packet such that, for the rendering, the renderer configuration information defines a selection of definitions of one or more scene objects and / or definitions of one or more scene characteristics (e.g., as defined by metadata, e.g., metadata 230 as shown in FIG. 2 ) included in the scene payload packet.

[0452] As another optional feature, the encoder 300, eg, the packet providing unit 320, is configured to provide one or more scene update packets, such that content of the one or more scene update packets defines one or more scene metadata updates.

[0453] As another optional feature, the encoder 300, e.g., the packet providing unit 320, is configured to provide scene configuration packets, such that the scene configuration packets determine the rendering configuration, and to provide scene update packets, such that the scene update packets define updates to the rendering configuration. Thus, as an example, updates to the rendering parameters, e.g., incremental updates, can be provided based on the scene configuration packet renderer parameters and based on the scene update packets.

[0454] Additionally, as another optional feature, the encoder 300, e.g., the packet providing unit 320, is configured to provide one or more scene configuration packets, one or more scene update packets, and one or more scene payload packets such that the one or more scene configuration packets, one or more scene update packets, and one or more scene payload packets conform to an MPEG-H MHAS packet definition.

[0455] As another optional feature, the encoder 300, e.g., the packet providing unit 320, is configured to provide the one or more scene configuration packets, the one or more scene update packets, and the one or more scene payload packets, such that the one or more scene configuration packets, the one or more scene update packets, and the one or more scene payload packets each include a packet type identifier, e.g., MHASPacketType, a packet label, e.g., MHASPacketLabel, packet length information, e.g., MHASPacketLength, and a packet payload, e.g., MHASPacketPayload.

[0456] Thus, as an optional feature, the bitstream 302 may include multiple MPEG-H packets, including, for example, packets representing one or more audio channels to be rendered. The encoder 300, e.g., the bitstream provider 310, may thus be configured to provide, in the bitstream, one or more scene configuration packets and one or more scene update packets and one or more scene payload packets, e.g., interleaved with the MPEG-H packets.

[0457] Furthermore, the encoder 300 (general encoding device) can optionally provide the bitstream via a broadcast stream. However, it should be noted that the encoder 300, e.g., the bitstream provider 310, can optionally be configured to provide some packets via a broadcast bitstream, e.g., via the optionally indicated broadcast bitstream 306, and some packets via a non-broadcast bitstream, e.g., 302. For example, the broadcast bitstream 306 may include scene configuration packets. On the other hand, the bitstream 302 may, e.g., be addressed to a specific user and thus include specific payload packets. Thus, the encoder 300 can, e.g., provide the broadcast bitstream 306 and multiple individual bitstreams 302 via multiple server-client channels.

[0458] As another optional feature, the encoder 300 comprises a request unit 340. The encoder 300 may receive a request 308 (e.g. corresponding to request 201 shown in Fig. 2), for example from a decoder, and thus provide one or more scene payload packets in response to the request 308. The request unit 340 may then forward such a request to the packet providing unit 320 and / or the bitstream provider 310 to provide packets, which may then be encoded into a bitstream.

[0459] The one or more scene payload packets may be identified using, for example, a payload ID and / or a packet ID. In other words, the encoder 300 may be configured to provide one or more scene payload packets in response to a request 308 from the audio decoder with a payload ID or in response to a request 308 from the audio decoder with a packet ID.

[0460] The request 308 may, as an optional feature, additionally or alternatively include information at least implicitly indicating which one or more scene payload packets are required or are required within a given time period, and thus the request unit 340 may optionally schedule the timely transmission of the requested packets.

[0461] As another optional feature, the encoder 300 is configured to provide, for example using the packet providing unit 320, one or more scene update packets for the rendering, such that the one or more scene update packets define updates to scene metadata and include representations of one or more update conditions.

[0462] As an example, the parsing unit 330 may determine scene metadata required to represent or approximate an audio scene. Based on the metadata currently used in the corresponding decoder, the parsing unit may provide information to the packet providing unit 320 to define or determine a scene update packet to provide metadata update information to the corresponding decoder. Furthermore, updates may be conditional, for example, with respect to time, location within the audio scene, and / or events (e.g., if a user in a VR room opens a window).

[0463] As another optional feature, the encoder 300 is configured to periodically repeat providing the scene configuration packets, for example in the broadcast bitstream 306, to allow for efficient training of, for example, new decoders.

[0464] As another optional feature, the encoder 300 is configured to provide, e.g. by means of the packet providing unit 320, a scene configuration packet, whereby the scene configuration packet defines which scene payload packets are required at a given point in space and time. In other words, as an example, based on an analysis of the acoustic scene, a configuration can be determined that defines at which points in time and space and / or for which conditions, e.g., which payloads representing metadata defining acoustically related objects and / or properties, may be required or are advantageous to define or reproduce an audio scene.

[0465] As another optional feature, the encoder 300 is configured to provide scene configuration packets, e.g., by means of the packet providing unit 320, whereby the scene configuration packets define where the scene payload packets can be retrieved from. Thus, based on the scene configuration packets, e.g., from a broadcast channel, the decoder can request the respective payload packets individually, e.g., via a unicast channel.

[0466] As mentioned above, as an optional feature, the encoder 300 is configured to provide a scene update packet, such that the scene update packet defines a condition for a scene update, for example using the packet providing unit 320. Optionally, the scene update packet can define an interactive trigger condition for the scene update.

[0467] Furthermore, as another optional feature, the encoder 300 is configured, for example using the packet providing unit 320, to adapt an order of one or more definitions of scene objects and / or one or more definitions of scene properties in a scene payload packet depending on when and / or where the one or more definitions of scene objects and / or one or more definitions of scene properties are required by a renderer or decoder.

[0468] As another optional feature, the apparatus 300 is configured, for example by means of the packet providing unit 320, to adapt an order of the one or more definitions of the scene objects and / or the one or more definitions of the scene properties in the scene payload packet depending on the importance of the one or more definitions of the scene objects and / or the one or more definitions of the scene properties to the renderer.

[0469] Optionally, the order of one or more definitions of scene objects and / or one or more definitions of scene properties within a scene payload packet may be set according to packet size limitations.

[0470] As another optional feature, the apparatus is configured to provide payload packets with a relatively low level of detail first and later with a relatively high level of detail. As an example, the analysis unit 330 may "decompose" the audio scene to be encoded at different levels of detail or granularity, whereby first coarse information about the audio scene may be provided and then more precise information may be provided.

[0471] Furthermore, as an optional feature, the device 300 is configured to separate, for example using the parsing unit 330, one or more definitions of the scene objects and / or one or more definitions of the scene properties into multiple scene payload packets and provide different scene payload packets at different times. In accordance with the above, some scene objects or properties may have a greater impact on the audio scene and therefore may be provided immediately for rendering of the audio scene. Other objects or properties may, for example, only be required for an improved acoustic experience and therefore may be transmitted if computational resources and / or bandwidth limitations allow it.

[0472] As another optional feature, the apparatus 300 is configured to provide scene configuration packets to decompose the scene into multiple spatial regions for which different rendering metadata are valid, for example using the parsing unit 330. Thus, a decoder addressing a particular user at a particular location in the audio scene can selectively request only valid metadata, increasing efficiency.

[0473] FIG. 4 is a schematic block diagram of a method for providing a decoded audio representation based on an encoded audio representation according to an embodiment of the first aspect of the invention.

[0474] The method 400 includes spatially rendering 410 one or more audio signals and receiving 420 a plurality of packets of different packet types, the packets comprising one or more scene configuration packets providing renderer configuration information defining use of scene objects and / or use of scene properties, the packets comprising one or more scene update packets defining updates to scene metadata for the rendering, and the packets comprising one or more scene payload packets comprising definitions of one or more of the scene objects and / or one or more of the scene properties.

[0475] Further, the method includes selecting 430 one or more scene object definitions and / or one or more scene property definitions included in the scene payload packet for rendering in response to the renderer configuration information, and updating 440 one or more scene metadata in response to content of the one or more scene update packets.

[0476] 5 is a schematic block diagram of a method for providing an encoded audio representation according to an embodiment of the first aspect of the present invention. The method 500 comprises providing 510 information for spatial rendering of one or more audio signals and providing 520 a plurality of packets of different packet types, the packets comprising one or more scene configuration packets providing renderer configuration information defining the use of scene objects and / or the use of scene properties, the packets comprising one or more scene update packets defining updates to scene metadata for the rendering, the packets comprising one or more scene payload packets comprising definitions of one or more of the scene objects and / or one or more of the scene properties.

[0477] Reference is now made to Figures 2 to 3. Bitstream 208 in Figure 2 and therefore bitstreams 302 and / or 306 in Figure 3 represent audio content. An embodiment in accordance with the invention includes a bitstream such as the bitstreams mentioned above. In summary, such a bitstream comprises a number of packets of different packet types, the packets comprising one or more scene configuration packets providing renderer configuration information defining the use of scene objects and / or the use of scene properties, the packets comprising one or more scene update packets defining updates to scene metadata for rendering, the packets comprising one or more scene payload packets comprising definitions of one or more of the scene objects and / or one or more of the scene properties.

[0478] Fig. 6 is a schematic diagram of an audio decoder according to an embodiment of the second aspect of the present invention. Fig. 6 shows an audio decoder 600 for providing an encoded audio representation 602 based on a decoded audio representation 606. The decoder 600 comprises, as an optional feature, a rendering unit 610 configured to spatially render one or more audio signals. The rendering unit 610 may therefore optionally comprise a decoding unit, which may be configured to decode the encoded audio representation to obtain one or more audio signals. However, as an alternative, as indicated by dotted and dashed lines, the audio decoder 600 may comprise a decoding unit 620, which may comprise the encoded audio representation 602 and may provide one or more audio signals to the rendering unit 610.

[0479] Additionally, the audio decoder 600 is configured to receive multiple packets 604 of different packet types, including one or more scene configuration packets providing renderer configuration information, the packets 604 including one or more scene update packets defining updates to scene metadata for rendering and including a representation of one or more update conditions.

[0480] Thus, based on the scene configuration packet, the renderer configuration of the rendering unit 610 can be set and / or adjusted. Based on the renderer configuration, the rendering unit 610 can determine which scene objects and / or scene characteristics to consider.

[0481] Such objects and / or properties may be defined, for example, using metadata 630. As discussed above, based on scene update packets, such metadata 630 may be updated for rendering.

[0482] Further, the audio decoder 600 is configured to evaluate, using an optional evaluation unit 640, whether one or more update conditions are met, and, if the one or more update conditions are met, selectively update one or more scene metadata 130 according to the content of the one or more scene update packets.

[0483] Thus, in other words, as an example, the decoder 600 may receive an encoded audio representation 602 that is, for example, decoded using a decoding unit 620 and rendered using a rendering unit 610. The rendering is performed based on a renderer configuration that is defined based on one or more scene configuration packets that are provided to the decoder 600 in addition to the audio representation 602.

[0484] Additionally, the packets 604 provided to the decoder 600 contain information regarding the scene update, including information regarding updates to the metadata used by the rendering unit 610. However, in addition to the update data itself, update conditions are provided so that the decoder can evaluate the conditions and update the metadata used for rendering when defined criteria are met.

[0485] 6, the evaluation unit 640 can trigger updates to the metadata 630, e.g., the definitions of scene objects and / or the scene characteristics themselves. Optionally, the evaluation unit 640 may cause adaptations in the rendering unit to select other metadata objects or to update the metadata via the rendering unit. However, such additional functions or signal paths may be optional.

[0486] It should be noted here again that the separation of the input signal in packets 604 and the coded audio representation 602 is an example. The coded audio representation may be provided as part of the packets 604, e.g. in the form of packets, e.g. as MPEG H3 DAFRAMEs, e.g. containing information on spectral audio coefficients. On the other hand, the decoder 600 may receive only the coded audio representation, e.g. containing packets 604 as explained above containing configuration data and update data in addition to the audio information or audio signal. The optional decoding unit 620 may therefore alternatively or additionally be configured to decode the coded packets.

[0487] Optionally, the decoder 600 is configured to evaluate, e.g. using an evaluation unit 640, a temporal condition included in the scene update packet to determine whether to update one or more scene metadata 630 depending on the content of the one or more scene update packets.

[0488] As an example, the time condition may define a start time or a time interval, and the decoder 600 may be configured, e.g., using the evaluation unit 640, to perform one or more scene metadata updates in response to detecting that the current playback time has reached or is after the start time, or to perform one or more scene metadata 630 updates in response to detecting that the current playback time is within the time interval.

[0489] As another optional feature, e.g. in addition to or instead of the above, the decoder 600 may be configured to evaluate, e.g. using evaluation unit 640, spatial conditions included in the scene update packets to determine whether to update one or more scene metadata depending on the content of the one or more scene update packets.

[0490] Optionally, the spatial condition defines a geometric element, and the audio decoder 600 is configured, for example using the evaluation unit 640, to perform one or more updates of the scene metadata 630 in response to detecting that the current position has reached the geometric element or in response to detecting that the current position is within the geometric element.

[0491] Therefore, as an optional feature, the decoder 600 may be configured to receive additional information 608 that may include information regarding such a current position, which may for example be the position of the listener within the scene at which the decoder is rendering the audio scene. It should be noted, however, that the decoder 600 may also determine such information based on the provided packets 604, for example using the evaluation unit 640.

[0492] Further, the decoder 600, e.g. the evaluation unit 640, may be configured to evaluate whether an interactivity trigger condition is met to determine whether one or more scene metadata should be updated depending on the content of one or more scene update packets. As an example, when a user opens a window in a virtual room, the acoustic properties of the room may be changed, and thus based on the trigger "open window" metadata, e.g. an element representing the acoustic properties of the wall, to a wall with a hole (= window). This event may be communicated to the decoder, as an example, via the additional information 608. However, the decoder may optionally include such information or derive such information itself.

[0493] As another optional feature, the audio decoder 600, e.g., the evaluation unit 640, is configured to evaluate a combination of two or more update conditions and, if the combined update condition is satisfied, selectively update one or more scene metadata 130 according to the content of one or more scene update packets.

[0494] Thus, as an optional feature, the audio decoder 600, e.g. the evaluation unit 640, is configured to evaluate both temporal and spatial update conditions, or to evaluate both temporal and interactive update conditions. According to an embodiment, any combination of spatial, temporal and / or event conditions may be taken into account.

[0495] As another optional feature, the audio decoder, e.g., the evaluation unit 640, is configured to evaluate delay information included in the scene update packets and, in accordance with the delay information, delay updates of one or more scene metadata 130 depending on the content of the one or more scene update packets in response to detecting that one or more update conditions are met.

[0496] Further, as an optional feature, the audio decoder 600, e.g. the evaluation unit 640, is configured to evaluate a flag in the scene update packet indicating whether a temporal update condition is defined in the scene update packet and / or to evaluate a flag in the scene update packet indicating whether a spatial update condition is defined in the scene update packet.

[0497] Thus, the audio decoder 600, eg the evaluation unit 640, is configured to evaluate a flag in the scene update packet that indicates whether delay information is defined in the scene update packet.

[0498] As another optional feature, the scene update packet includes a representation of a plurality of modifications of one or more parameters of one or more scene objects and / or one or more scene characteristics, and the audio decoder 600 is configured to apply the modifications, e.g., to metadata 630 defining such objects or characteristics, in response to detecting, e.g., using the evaluation unit 640, that one or more update conditions have been met.

[0499] As another optional feature, the scene update packets include trajectory information, and the audio decoder 600 is configured to update, e.g., using the evaluation unit 640, each scene metadata 130 with which the trajectory information is associated using parameter variations that follow a trajectory defined by the trajectory information.

[0500] It should be noted that the evaluation unit 640 may therefore be configured to perform any or all of the updates described above, e.g. metadata updates. The evaluation unit 640 may therefore comprise an update unit, which is not shown for simplicity. Optionally, the decoder 600 may comprise a separate update unit configured to receive the evaluation results from the evaluation unit 640 and perform the respective updates.

[0501] As another optional feature, the audio decoder 600, e.g., the evaluation unit 640, is configured to evaluate information indicating whether trajectory-based updates of the scene metadata 630 are used to activate or deactivate trajectory-based updates of the scene metadata.

[0502] As another optional feature, the audio decoder 600, for example the evaluation unit 640, is configured to evaluate the interpolation type information included in the scene update packet to determine a type of interpolation between two or more support points of the trajectory.

[0503] Therefore, as an optional feature, the audio decoder 600, for example the evaluation unit 640, is configured to evaluate the fulcrum information describing the trajectory.

[0504] It should be noted that the decoder 600 may include any or all of the features as described in the context of the decoders shown in Figures 1 and 2, individually and in combination. For example, the decoder 600 may optionally comprise an extraction unit, an expectation unit and / or a request unit (hence the respective functionality). Furthermore, the decoder 600 may be configured to receive and process a payload package as described in the context of Figures 1 and 2, for example as part of a packet 604. Vice versa, the decoders 100 and 200 from Figures 1 and 2 may include any or all of the features as described in the context of the decoders shown in Figure 6, for example including means for receiving additional information and using it in the evaluation unit.

[0505] Reference is now made to Fig. 7. For the sake of brevity and as explained above, it is noted that an embodiment according to the invention comprises a decoder as shown in Fig. 6 and therefore an encoder with corresponding features according to the decoder. The encoder may therefore comprise only the features necessary to provide a non-optional signal as received by the decoder 600 of Fig. 6 and to process it to provide a decoded audio representation 606, without any optional features. However, any of the optional features, functions and details disclosed above may be present, individually or in combination, corresponding to an encoder according to an embodiment. The same applies to the features, functions and details of the decoders and / or encoders of the other aspects of the invention.

[0506] Fig. 7 is a schematic diagram of an encoder according to an embodiment of the second aspect of the present invention. The encoder 700 is configured to provide a bitstream 702, which may for example comprise an encoded audio representation. In particular, the encoder 700 is configured to provide information for spatial rendering of one or more audio signals contained in the bitstream 702. The bitstream 702 thus comprises a number of packets 722 of different packet types.

[0507] In order to provide the bitstream 702 and thus the above mentioned information entities, the encoder 700 comprises an optional bitstream provider 710 to which packets 722 are provided.

[0508] As shown, the encoder 700 may optionally comprise a packet providing unit 720. The encoder 700 may be configured to provide packets 722, for example, using the packet providing unit 720. The packets 722 include one or more scene configuration packets that provide renderer configuration information, one or more scene update packets that define updates to scene metadata for rendering, and include a representation of one or more update conditions.

[0509] As an example, an audio signal 704 to be encoded may be provided to the encoder 700. The audio signal may comprise, for example, time domain samples and / or spectral values ​​of a speech signal and / or music signal. Optionally (not shown), if this signal has already been encoded, it may be provided directly to the bitstream provider 710, for example to be included in the bitstream together with the packets 722.

[0510] The packets 722 may be provided by the packet providing unit in different ways, for example, the packet providing unit 720 may provide the packets 722 based on scene information that defines an acoustic scene and may be predefined or may be obtained by the packet providing unit.

[0511] For example, in the case of a virtual reality application, based on a virtual model (e.g. of an acoustic scene), the packets may be used to determine acoustically related virtual objects to model or represent the acoustic scene (e.g., an acoustic scene including a virtual model having acoustically related virtual objects and audio signals). Optionally, the (optional) analysis unit 730 may support the determination of acoustically related virtual objects, or characteristics thereof (e.g., characteristics of the audio signals may be used to supplement and / or refine the virtual model of the acoustic scene, e.g., by providing information about the positions of sound sources).

[0512] Thus, information supporting the decision for each packet 722 may be provided to the packet providing unit 720 from the optional parsing unit 730. For example, the packet providing unit 720 may manage a virtual model or may be provided with information regarding the virtual model.

[0513] As another example, in the context of an augmented reality application, the audio signal 704 to be encoded may be provided to the encoder 700 and may comprise, for example, time-domain samples and / or spectral values ​​of a speech signal and / or music signal. Optionally, however, the audio signal 704 may further comprise (or carry) spatial information of the real audio scene to be augmented (e.g., in an implicit form), for example, position information of a measured audio source, for example of a speaking user, in the scene. Such information, and optionally additional virtual overlay information (e.g., information for adding acoustically relevant virtual objects to a real scene to augment the real scene), may be extracted and / or analyzed and / or applied by an optional analysis unit 730 to model or represent the scene using packets. Thus, as mentioned above, information for determining each packet 722 may be provided to the packet providing unit 720. As an example, the analysis unit 730 may determine, based on the spatial information in the audio signal 704, which acoustically relevant objects of the scene should be considered or updated or rendered to provide a desired auditory experience.

[0514] However, it should be noted that the encoder 700 may optionally not comprise a parsing unit, e.g. so that information about the packets 722 may be provided to the encoder 700 from an external unit, e.g. a unit managing a virtual or augmented scene.

[0515] To further illustrate the above example, an audio signal 704 to be encoded may be provided to the encoder 700, which may be an audio signal from a spatial audio scene and may optionally further include spatial information regarding the audio scene. Optionally, the encoder 700 may therefore comprise an analysis unit 730. The analysis unit 730 may be configured to analyze information provided from the audio scene in order to determine or approximate a representation of the audio scene. As an example, the audio scene may be represented using metadata describing, for example, scene objects and / or scene properties, which may be used together with the spectral coefficients to provide a listener with an immersive representation of the audio scene.

[0516] Based on this, as an example, in order to provide renderer configuration information and metadata updates for rendering of an audio scene, the packet providing unit 720 can be used to determine and provide scene update packets including corresponding scene configuration packets and update conditions, which can indicate the evolution of the audio scene or changes in the perception of the audio scene to a listener within the audio scene, for example with respect to space, time, and / or further conditions.

[0517] It should be noted that the metadata may be based, for example, on a digital model of the audio scene (e.g. in the case of virtual reality applications) and / or on an analysis of the actual scene from which the audio signal was recorded (e.g. for the case of augmented reality applications).

[0518] As an example, the packet providing unit 720 may further provide packets containing said spectral information of the audio signal, for example in the form of packets representing one or more audio channels to be rendered (e.g., optionally, packets such as MPEG H3D AFRAME) that are part of the packets 722.

[0519] Optionally, the audio signal 704 may for example be provided directly to the packet providing unit 720 and / or to the bitstream provider 710. As an example, the audio signal 704 may already include defined packets 722 such that these packets can only be extracted in the packet providing unit 722 to be encoded in the bitstream provider 710. The audio signal information may be provided to the bitstream provider 710 separately from, for example, metadata information, for example in the form of packets or directly based on the audio signal 304.

[0520] Furthermore, the analysis unit 730 may determine or approximate a virtual acoustic scene, e.g. using the audio signal 704 representing only the acoustic signal itself, and further spatial properties of the scene may be based on a virtual model of the surroundings in the virtual acoustic scene, e.g. using information about the position of the user within the virtual surroundings. For example, in a virtual conference room, the reflection properties of a virtual wall, or the attenuation properties of a virtual carpet, may be incorporated as metadata, e.g. not based on real measurements, but based on a virtual acoustic model of the wall or carpet with respect to the position of the listener.

[0521] As an optional feature, the encoder 700, e.g. the packet providing unit 720, is configured to provide a scene update packet such that said scene update packet includes a representation of a temporal condition for updating one or more scene metadata depending on a content of said scene update packet. For example, based on a result of the analysis unit 730, the encoder 700 can determine or estimate that the audio scene to be encoded may change according to a temporal development and can therefore communicate such information, e.g. modeling, via a transmission of the respective update data and the temporal condition.

[0522] As another optional feature, the time condition defines a start time or a time interval.

[0523] Thus, as another optional feature, the analysis unit 730 may be configured, by way of example, to determine that the audio scene to be encoded includes spatial dependencies and thus communicate such information via the transmission of the respective update data and spatial conditions. Thus, the device 700 is optionally configured to provide a scene update packet such that said scene update packet includes a representation of spatial conditions for updating one or more scene metadata depending on the content of said scene update packet.

[0524] As another optional feature, the spatial conditions define geometric elements, such that the analysis unit can efficiently model the audio scene using such geometric elements.

[0525] As another optional feature, the encoder 700, e.g., the packet providing unit 720, is configured to provide a scene update packet such that the scene update packet includes a representation of an interactivity trigger condition for updating one or more scene metadata depending on the content of the scene update packet.

[0526] Optionally, the apparatus 700, for example the packet providing unit 720, is configured to provide a scene update packet, such that the scene update packet includes a representation of a combination of two or more update conditions.

[0527] As another optional feature, the apparatus 700, e.g. the packet providing unit 720, is configured to provide a scene update packet such that the scene update packet includes delay information defining, in response to detecting that the one or more update conditions are satisfied, to delay updating one or more scene metadata depending on the content of the one or more scene update packets.

[0528] Optionally, the apparatus 700, e.g. the packet providing unit 720, is configured to provide a scene update packet such that the scene update packet includes a representation of a flag indicating whether a temporal update condition is defined in the scene update packet and / or a flag indicating whether a spatial update condition is defined in the scene update packet.

[0529] As another optional feature, the apparatus 700, e.g. the packet providing unit 720, is configured to provide a scene update packet such that the scene update packet includes a flag indicating whether delay information is defined in the scene update packet.

[0530] As another optional feature, the apparatus 700, e.g. the packet providing unit 720, is configured to provide a scene update packet such that the scene update packet includes a representation of a plurality of modifications of one or more parameters of one or more scene objects and / or one or more scene characteristics.

[0531] Optionally, the apparatus 700, e.g. the packet providing unit 720, is configured to provide a scene update packet, such that the scene update packet includes trajectory information, where the trajectory information describes updating respective scene metadata with which the trajectory information is associated using parameter variations following a trajectory defined by the trajectory information.

[0532] As an example, the analysis unit 730 may be configured to determine that changes in acoustically relevant objects or properties of the audio scene being encoded can be modeled or approximated using metadata updates according to trajectory information, and may therefore be configured to reduce the signaling effort of such updates by providing them in the form of trajectory information.

[0533] As another optional feature, the apparatus 700, e.g. the packet providing unit 720, is configured to provide a scene update packet such that the trajectory information includes information indicating whether trajectory-based updating of scene metadata is used to activate or deactivate trajectory-based updating of scene metadata.

[0534] As another optional feature, the apparatus 700, eg, the packet providing unit 720, is configured to provide a scene update packet such that the trajectory information includes the interpolation type information included in the scene update packet.

[0535] As another optional feature, the apparatus 700, eg, the packet providing unit 720, is configured to provide the scene update packets such that the trajectory information includes fulcrum information describing the trajectory.

[0536] 8 is a schematic block diagram of a method for providing a decoded audio representation based on an encoded audio representation according to an embodiment of the second aspect of the present invention. The method 800 comprises spatially rendering 810 one or more audio signals and receiving 820 a plurality of packets of different packet types, said packets including one or more scene configuration packets providing renderer configuration information, one or more scene update packets defining updates to scene metadata for said rendering, and including a representation of one or more update conditions.

[0537] Further, the method 800 includes evaluating 830 whether one or more update conditions are met, and, if the one or more update conditions are met, selectively updating one or more scene metadata according to content of the one or more scene update packets.

[0538] 9 is a schematic block diagram of a method for providing an encoded audio representation according to an embodiment of the second aspect of the present invention. The method 900 comprises providing 910 a plurality of packets of different packet types, said packets including one or more scene configuration packets providing renderer configuration information, one or more scene update packets defining updates to scene metadata for said rendering, and including a representation of one or more update conditions.

[0539] Reference is now made to Fig. 7. Bitstream 702 in Fig. 7 represents audio content. An embodiment in accordance with the invention includes a bitstream such as the bitstream described above. In summary, such a bitstream includes multiple packets of different packet types, including one or more scene configuration packets providing renderer configuration information, including one or more scene update packets defining updates to scene metadata for the rendering, including a representation of one or more update conditions.

[0540] Fig. 10 shows a schematic diagram of an audio decoder according to an embodiment of the third aspect of the present invention. Fig. 10 shows an audio decoder 1000 for providing an encoded audio representation 602 based on a decoded audio representation 1006. The decoder 1000 comprises a rendering unit 1010 configured to spatially render one or more audio signals. The rendering unit 1010 may therefore optionally comprise a decoding unit, which may be configured to decode the encoded audio representation to obtain one or more audio signals. However, as an alternative, as indicated by dotted and dashed lines, the audio decoder 1000 may comprise an encoded audio representation 1002 and a decoding unit 1020 capable of providing one or more audio signals to the rendering unit 1010.

[0541] Further, the audio decoder 1000 is configured to receive a number of packets 1004 of different packet types, the packets 1004 providing renderer configuration information defining the temporal evolution of a rendering scenario, including a number of scene configuration packets containing timestamp information.

[0542] Furthermore, the audio decoder 1000 is configured to evaluate the timestamp information using the evaluation unit 1020 and to set the rendering configuration of the rendering unit 1010 to a rendering scenario that corresponds to the timestamp using the renderer configuration information.

[0543] It is noted here again that the separation of the input signal in packets 1004 and coded audio representation 1002 is an example. The coded audio representation may be provided as part of packets 1004, e.g. in the form of packets containing information on spectral audio coefficients, e.g. as MPEG H3 DAFRAME. On the other hand, the decoder 1000 may receive only the coded audio representation, e.g. including packets 1004 as explained above containing configuration data, update data and metadata in addition to the audio information or audio signal. The optional decoding unit 1020 may therefore alternatively or additionally be configured to decode coded packets.

[0544] As an optional feature, the audio decoder 1000, e.g. the evaluation unit 1020, is configured to evaluate the timestamp information when the audio decoder misses one or more previous scene composition packets of a stream or when the audio decoder tunes into a stream. Further, the audio decoder 1000 is configured to set a playback time, e.g. in the rendering unit 1010, depending on the timestamp information contained in the scene composition packets.

[0545] As another optional feature, the audio decoder 1000, e.g. the rendering unit 1010, is configured to perform a temporal evolution of a rendering scene up to a playback time defined by the timestamp information when the audio decoder misses one or more preceding scene composition packets of a stream or when the audio decoder tunes into a stream.

[0546] As another optional feature, the audio decoder 1000 is configured to obtain the time scale information included in the packets and to evaluate the time stamp information using the time scale information, for example using the evaluation unit 1020 .

[0547] As another optional feature, the audio decoder 1000, e.g. the evaluation unit 1020, is configured to determine which scene objects to use for the rendering depending on the timestamp information. Based on such a determination, the renderer configuration of the rendering unit 1010 can be adapted accordingly.

[0548] Optionally, the rendering unit 1010 may receive or select scene objects or scene characteristics for rendering from a set of metadata elements, for example as shown in FIG. 1, FIG. 2 and / or FIG.

[0549] As another optional feature, the audio decoder 1000 is configured to evaluate, e.g., using the evaluation unit 1020, a scene composition packet that defines an evolution of a rendered scene starting from a point in time before the time defined by the time stamp information. Furthermore, the audio decoder 1000, e.g., its evaluation unit 1020, is configured to derive, based on information in the scene composition packet, a scene composition associated with the point in time defined by the time stamp information.

[0550] Such a scene composition may then be provided to, for example, a rendering unit 1010.

[0551] As another optional feature, the audio decoder 1000 is configured to derive, for example using the evaluation unit 1020, a scene composition associated with a point in time defined by the time stamp information using one or more scene update packets.

[0552] Thus, packets 1004 may optionally comprise one or more scene update packets.

[0553] As another optional feature, the scene composition packets conform to the MPEG-H MHAS packet definition.

[0554] As another optional feature, the scene configuration packets each include a packet type identifier, a packet label, packet length information, and a packet payload.

[0555] As another optional feature, the audio decoder 1000 is configured to extract one or more scene composition packets from a bitstream that includes a plurality of MPEG-H packets, the packets including which represent one or more audio channels to be rendered. Thus, the decoder 1000 may optionally comprise an extraction unit, for example as shown in FIG.

[0556] As an optional feature, the audio decoder 1000 is configured to receive one or more scene composition packets via the broadcast stream.

[0557] As another optional feature, the audio decoder 1000 is configured to tune to the broadcast stream and determine the play time based on a timestamp of the first scene composition packet identified by the audio decoder after tuning.

[0558] Reference is now made to Fig. 11. For the sake of brevity, and as explained above, it is noted that an embodiment according to the invention comprises an encoder with corresponding features according to the decoder shown in Fig. 10. The encoder may therefore comprise only features necessary to provide a non-optional signal as received by the decoder 1000 of Fig. 10 and to process this to provide a decoded audio representation 1006, and no optional features. However, any of the optional features, functions and details disclosed above may be present, individually or in combination, in a corresponding encoder according to an embodiment. The same applies to the features, functions and details of the decoders and / or encoders of the other aspects of the invention.

[0559] Fig. 11 shows a schematic diagram of an encoder according to an embodiment of the third aspect of the present invention. The encoder 1100 is configured to provide a bitstream 1102, which may for example comprise a coded audio representation. In particular, the encoder 1100 is configured to provide information for spatial rendering of one or more audio signals, which are comprised in the bitstream. Thus, the bitstream 1102 comprises a number of packets 1122 of different packet types.

[0560] To provide the bitstream 1102 and thus the above mentioned information entities, the encoder 1100 comprises a bitstream provider 1110 to which packets 1122 are provided.

[0561] As shown, the encoder 1100 may comprise a packet providing unit 1120. The encoder 1100 may be configured to provide, for example, packets 1122 using the packet providing unit 1120. The packets 1122 include a number of scene configuration packets that define the temporal evolution of a rendering scenario and provide renderer configuration information including timestamp information.

[0562] Therefore, as shown in FIG. 11, the time stamp information may be provided to the packet providing unit 1120 by an optional time information unit 1140 .

[0563] As an example, an audio signal 1104 to be encoded may be provided to the encoder 1100. The audio signal may comprise, for example, time domain samples and / or spectral values ​​of a speech signal and / or music signal. Optionally (not shown), if this signal has already been encoded, it may be provided directly to the bitstream provider 1110, for example to be included in the bitstream together with the packets 1122.

[0564] The packets 1122 may be provided by the packet providing unit in different ways, for example, the packet providing unit 1120 may provide the packets 322 based on scene information defining an acoustic scene and which may be predefined or may be obtained by the packet providing unit.

[0565] For example, in the case of a virtual reality application, based on a virtual model (e.g. of an acoustic scene), the packets may be used to determine acoustically related virtual objects to model or represent the acoustic scene (e.g. an acoustic scene including a virtual model having acoustically related virtual objects and audio signals). Optionally, the (optional) analysis unit 1130 may support the determination of acoustically related virtual objects, or properties thereof (e.g. properties of the audio signals may be used to supplement and / or refine the virtual model of the acoustic scene, e.g. by providing information regarding the position of sound sources).

[0566] Thus, information supporting the decision of each packet 1122 may be provided to the packet providing unit 1120 from the optional parsing unit 1130. For example, the packet providing unit 1120 may manage a virtual model or may be provided with information regarding the virtual model.

[0567] As another example, in the context of an augmented reality application, the audio signal 1104 to be encoded may be provided to the encoder 1100 and may comprise, for example, time-domain samples and / or spectral values ​​of a speech signal and / or music signal. Optionally, however, the audio signal 1104 may further comprise (or carry) spatial information of the real audio scene to be augmented (e.g., in an implicit form), for example, position information of a measured audio source, for example of a speaking user, in the scene. Such information, and optionally additional virtual overlay information (e.g., information for adding acoustically relevant virtual objects to a real scene to augment the real scene), may be extracted and / or analyzed and / or applied by the optional analysis unit 1130 to model or represent the scene using packets. Thus, as mentioned above, information for determining each packet 1122 may be provided to the packet providing unit 1120. As an example, the analysis unit 1130 may determine, based on the spatial information in the audio signal 1104, which acoustically relevant objects of the scene are to be considered or updated or rendered to provide a desired auditory experience.

[0568] However, it should be noted that the encoder 1100 may optionally not include a parsing unit, e.g., information about the packet 1122 may be provided to the encoder 1100 from an external unit, e.g., a unit managing a virtual or augmented scene.

[0569] To further illustrate the above example, the audio signal 1104 may be an audio signal from a spatial audio scene and may optionally further comprise spatial information regarding the audio scene. Optionally, the encoder 1100 may therefore comprise an analysis unit 1130. The analysis unit 1130 may be configured to analyze information provided from the audio scene, for example to determine or approximate a representation of the audio scene. As an example, the audio scene may be represented using metadata describing, for example, scene objects and / or scene properties, which may be used together with the spectral coefficients of the audio signal to provide a listener with an immersive representation of the audio scene.

[0570] It should be noted that the metadata may be based, for example, on a digital model of the audio scene (e.g. in the case of virtual reality applications) and / or on an analysis of the actual scene from which the audio signal was recorded (e.g. for the case of augmented reality applications).

[0571] Based on this, corresponding scene configuration packets, scene update packets, and scene payload packets can be determined and provided, as an example.

[0572] Thus, as an example, the packet providing unit 1120 may further provide packets containing said spectral information of the audio signal, for example in the form of packets representing one or more audio channels to be rendered, optionally included in the packets 1122.

[0573] Optionally, the audio signal 1104 may be provided directly to the packet providing unit 1120 and / or to the bitstream provider 1110. As an example, the audio signal 1104 may already comprise defined packets 1122 such that these packets can only be extracted in the packet providing unit 1122 to be encoded in the bitstream provider 1110. The audio signal information may be provided to the bitstream provider 1110 in the form of packets or directly based on the audio signal 1104, e.g. separately from metadata information.

[0574] It should further be noted that the analysis unit 1130 may also determine or approximate a virtual acoustic scene, e.g. using the audio signal 1104 which represents only the acoustic signal itself, and further spatial properties of the scene may be based on a virtual model of the surroundings in the virtual acoustic scene, e.g. based on the position of the user within the virtual surroundings. For example, in a virtual conference room, the reflection properties of a virtual wall, or the attenuation properties of a virtual carpet may be incorporated as metadata, e.g. not based on real measurements, but based on a virtual acoustic model of the wall or carpet with respect to the position of the listener.

[0575] Optionally, the timestamp information may be derived via analysis of the audio signal 1104, e.g., as indicated by the dashed line, or may be defined or set independently, for example. Optionally, the timestamp information may be provided as a separate input signal of the encoder 1100.

[0576] Optionally, the apparatus 1100 is configured to provide timescale information for one of the packets, for example using the time information unit 1140, and the timestamp information is provided in a representation related to the timescale information.

[0577] As another optional feature, the apparatus 1100, eg, the packet providing unit 1120, is configured to provide the scene composition packets such that the scene composition packets conform to the MPEG-H MHAS packet definition.

[0578] As another optional feature, the apparatus 1100, for example the packet providing unit 1120, is configured to provide scene configuration packets, such that each scene configuration packet includes a packet type identifier, a packet label, packet length information, and a packet payload.

[0579] As another optional feature, the apparatus 1100, e.g., the bitstream provider 1120, is configured to provide a bitstream 1102 including a plurality of MPEG-H packets including packets representing one or more audio channels to be rendered and one or more scene composition packets.

[0580] Thus, packets 1122 may include, for example, MPEG-H packets that may be provided by packet providing unit 1120 .

[0581] As another optional feature, the apparatus 1100, e.g., the bitstream provider 1110, is configured to provide a bitstream 1102 including a plurality of MPEG-H packets including packets representing one or more audio channels to be rendered and one or more scene composition packets in an interleaved manner.

[0582] As another optional feature, the apparatus 1100 is configured to periodically repeat the scene composition packets.

[0583] As another optional feature, the apparatus 1100 is configured to periodically repeat the scene configuration packets with one or more scene payload packets and one or more packets representing one or more audio channels to be rendered between two subsequent scene configuration packets.

[0584] Thus, the packet providing unit 1120 may optionally be configured to provide scene payload packets for the encoding thereof using the bitstream provider 1110.

[0585] As another optional feature, the apparatus 1100 is configured to periodically repeat the scene configuration packets with one or more packets representing one or more audio channels to be rendered between two subsequent scene configuration packets. Further, the apparatus 1100 is configured to provide one or more scene payload packets on request.

[0586] The apparatus 1100 therefore comprises, as an optional feature, a requesting unit 1150 configured to receive such a request 1108. The requesting unit 1150 can therefore provide information regarding the request to the packet providing unit 1120 to provide one or more scene payload packets, optionally to the bitstream provider 1110 for encoding in the bitstream 1102.

[0587] As another optional feature, the device is configured to provide multiple otherwise identical scene composition packets with different timestamp information.

[0588] As another optional feature, the apparatus 1100, e.g. the time information unit 1140, is configured to adapt the time stamp information to a play time. Optionally, the encoder 1100 may receive information about the play time as an input signal or may estimate or determine the play time based on the audio signal, e.g. using the analysis unit 1130.

[0589] As another optional feature, the apparatus 1100 is configured to adapt the timestamp information to a playback time of rendering scene information contained in packets, the packets being provided by the apparatus in the time environment of the respective scene configuration packets in which the respective timestamp information is contained.

[0590] 12 is a schematic block diagram of a method for providing a decoded audio representation based on an encoded audio representation according to an embodiment of the third aspect of the present invention. The method 1200 comprises spatially rendering 1210 one or more audio signals and receiving 1220 a plurality of packets of different packet types, the packets providing renderer configuration information defining a temporal evolution of a rendering scenario and comprising a plurality of scene configuration packets including timestamp information. The method further comprises evaluating 1230 the timestamp information and setting a rendering configuration to a rendering scenario corresponding to the timestamp using the renderer configuration information.

[0591] 13 is a schematic block diagram of a method for providing an encoded audio representation according to an embodiment of the third aspect of the present invention. The method 1300 comprises providing 1310 information for spatial rendering of one or more audio signals and providing 1320 a number of packets of different packet types, including a number of scene configuration packets that define the temporal evolution of the rendering scenario and provide renderer configuration information including timestamp information.

[0592] Reference is now made to Fig. 11. Bitstream 1102 in Fig. 11 represents audio content. An embodiment in accordance with the invention includes a bitstream such as the bitstreams described above. In summary, such a bitstream includes multiple packets of different packet types, including multiple scene configuration packets that provide renderer configuration information defining the temporal evolution of a rendering scenario and that include timestamp information.

[0593] Fig. 14 is a schematic diagram of an audio decoder according to an embodiment of the fourth aspect of the present invention. Fig. 14 shows an audio decoder 1400 for providing an encoded audio representation 1402 based on a decoded audio representation 1406. The decoder 1400 comprises, as an optional feature, a rendering unit 1410 configured to spatially render one or more audio signals. The rendering unit 1410 may thus optionally comprise a decoding unit, which may be configured to decode the encoded audio representation to obtain one or more audio signals. However, as an alternative, as indicated by dotted and dashed lines, the audio decoder 1400 may comprise a decoding unit 1420, which may comprise the encoded audio representation 1402 and may provide one or more audio signals to the rendering unit 1410.

[0594] Further, the audio decoder 1400 is configured to receive packets 1404, including a scene configuration packet providing renderer configuration information, the scene configuration packet including sub-scene cell information defining one or more cells, the cell information defining an association between the one or more cells and respective one or more data structures 1430 associated with the one or more cells and defining a rendering scenario.

[0595] Thus, based on the scene configuration packet, the renderer configuration of the rendering unit 1410 can be set and / or adjusted. Furthermore, the audio decoder 1400, e.g., using the illustrated optional evaluation unit 1440, is configured to evaluate the cell information to determine which data structures 1430 should be used for spatial rendering in the rendering unit 1410.

[0596] Thus, in other words, as an example, the decoder 600 may receive an encoded audio representation 1402 that is, for example, decoded using a decoding unit 1420 and rendered using a rendering unit 1410. The rendering is performed based on a renderer configuration that is defined based on a scene configuration packet that is provided to the decoder 1400 in addition to the audio representation 4102.

[0597] Further, the evaluation unit 1440 can be used to extract cell information from the configuration information based on the cell information, for example to define an audio scene or a rendering scene or a portion of a rendering scenario in space and / or time, and a data structure related to the rendering of the audio signal can be selected for and / or provided to the rendering unit 1410.

[0598] Thus, optionally, the evaluation unit 1440 can cause adaptation in the rendering unit to select other data structures, for example via a direct signal path to the rendering unit 1410.

[0599] It should be noted here again that the separation of the input signal in packets 1404 and coded audio representation 1402 is an example. The coded audio representation may be provided as part of packets 1404, e.g. as MPEG H3 DAFRAME, in the form of packets containing information about the spectral audio coefficients and / or audio channels to be rendered. On the other hand, the decoder 1400 may receive only the coded audio representation, e.g. including packets 1404 as explained above containing, in addition to the audio information or audio signal, configuration data and, optionally, further update data and metadata, which may be an example of a data structure, e.g. defining acoustically relevant objects and / or properties of the audio scene to be rendered. Furthermore, the optional decoding unit 1420 may therefore alternatively or additionally be configured to decode the coded packets.

[0600] As an optional feature, the cell information includes a temporal definition of the given cell, and the audio decoder 1400 is configured to evaluate, e.g. using the evaluation unit 1400, the temporal definition of the given cell to determine whether one or more data structures associated with the given cell should be taken into account (e.g., used) in the spatial rendering, e.g. in the rendering unit 1410.

[0601] As another optional feature, the cell information includes a spatial definition of the given cell, and the audio decoder 1400 is configured to evaluate, e.g., using the evaluation unit 1440, the spatial definition of the given cell to determine whether one or more data structures associated with the given cell should be taken into account (e.g., used) in the spatial rendering, e.g., in the rendering unit 1410.

[0602] As another optional feature, the audio decoder 1400 is configured to evaluate, for example using the evaluation unit 1400, cell number information included in the scene composition packets to determine the number of cells.

[0603] As another optional feature, the cell information includes a flag indicating whether the cell information includes a temporal definition of the cell or a spatial definition of the cell, and the audio decoder 1400 is configured, for example using the evaluation unit 1400, to evaluate the flag indicating whether the cell information includes a temporal definition of the cell or a spatial definition of the cell.

[0604] As another optional feature, the cell information includes a reference to a geometric structure for defining the cell, and the audio decoder is configured to evaluate, for example using the evaluation unit 1440, the reference to the geometric structure to obtain a geometric definition of the cell.

[0605] As another optional feature, the audio decoder 1400 is configured to obtain, from a global payload packet, a definition of a geometric structure that defines a geometric boundary of the cell, e.g., using the evaluation unit 1440. Thus, the packets 1404 may include such global payload packets that may be provided via a broadcast bitstream.

[0606] As another optional feature, the audio decoder 1400 is configured to identify one or more current cells, e.g. using the evaluation unit 1440, and the audio decoder 1400 is configured to perform spatial rendering, e.g. using the rendering unit 1410, e.g. using a plurality of data structures associated with the one or more identified current cells, e.g. one or more data structures from 1430.

[0607] As another optional feature, the audio decoder 1400 is configured to identify one or more current cells, e.g., using the evaluation unit 1440, and the audio decoder 1400 is configured to perform spatial rendering using one or more scene objects and / or scene properties associated with the one or more identified current cells, e.g., using the rendering unit 1410. As an example, a set of data objects may be associated with the one or more current cells, a data structure including, e.g., metadata defining acoustically related objects and / or properties of the scene to be rendered.

[0608] As another optional feature, the audio decoder 1400 is configured to select, eg using the evaluation unit 1440, scene objects and / or scene characteristics to be considered in the spatial rendering depending on the cell information.

[0609] As another optional feature, the audio decoder 1400 is configured to determine, e.g., using the evaluation unit 1440, within which one or more spatial cells the current position falls, and the audio decoder is configured to perform spatial rendering, e.g., using the rendering unit 1410, using one or more scene objects and / or scene properties associated with the one or more identified current cells.

[0610] As shown optionally, the decoder 1400 may be configured to receive additional information 1408. The additional information may include information regarding a current position. Optionally, the decoder 1400 may alternatively be configured to estimate or determine information regarding a current position of, for example, a listener or user, where a rendering scenario, such as an audio scene, is rendered internally, for example without receiving a dedicated input.

[0611] As another optional feature, the audio decoder 1400 is configured to determine one or more payloads associated with one or more current cells, e.g., payloads describing scene objects and / or scene properties, based on an enumeration of payload identifiers included in the cell definition of the cell, and the audio decoder 1400 is configured to perform spatial rendering using, e.g., the rendering unit 1410, using the determined one or more payloads.

[0612] Optionally, the payload may be provided as a payload packet, such that packet 1404 may optionally include a payload packet. As an example, the payload of a payload packet may be associated with or define metadata, which information may be represented by data structure 1430.

[0613] As another optional feature, the audio decoder is configured to perform spatial rendering using information from one or more scene update packets associated with the one or more current cells, e.g., using the rendering unit 1410. Thus, the packet 1404 may further comprise a scene update packet.

[0614] As another optional feature, the audio decoder 1400 is configured to, in response to discovering that a given cell becomes active, update the rendering scene, e.g., the rendered audio scenario, using information from one or more scene update packets associated with the given cell. As an example, a cell may become active when its associated metadata information and / or associated data structures are acoustically significantly relevant, which may be the case when the listener's spatial location is within or within the cell.

[0615] As another optional feature, the cell information includes a reference to and / or a reference to a scene update packet that defines an update of scene metadata, e.g., describing or comprising or being a data structure 1430, for rendering, and the audio decoder 1400 is configured to selectively perform the update of the scene metadata defined in a given scene update packet in response to detecting that a cell including a link to the given scene update packet has become active.

[0616] In other words, as an example, the evaluation unit 1440 can determine that a cell is active, e.g., because a user is spatially located within the rendered audio scene within the geometric boundaries of the cell, and can therefore perform an update of the scene metadata associated with the active cell based on an update packet referenced by the cell information corresponding to the active cell.

[0617] As another optional feature, the one or more scene update packets include a representation of one or more update conditions, and the audio decoder is configured to evaluate, e.g., using evaluation unit 1440, whether the one or more update conditions are satisfied, and, if the one or more update conditions are satisfied, selectively update one or more scene metadata, e.g., data structure 1430, depending on the content of the one or more scene update packets.

[0618] As another optional feature, the audio decoder 1400 is configured to evaluate, e.g., with evaluation unit 1440, a temporal condition included in the scene update packet to determine whether one or more scene metadata, such as, e.g., data structure 1430, should be updated depending on the content of one or more scene update packets that may be included in packet 1404, for example.

[0619] A time condition optionally defines a start time, or a time condition optionally defines a time interval.

[0620] The audio decoder 1400 is configured to perform updates to one or more scene metadata, such as data structure 1430, in response to detecting, for example, that the current play time has reached or is after the start time.

[0621] Alternatively, the audio decoder 1400 is optionally configured to perform an update of one or more scene metadata, such as the data structure 1430, in response to detecting that the current play time is within the time interval.

[0622] Alternatively or additionally, the audio decoder may optionally be configured to evaluate spatial conditions included in the scene update packets, for example using evaluation unit 1440, to determine whether to update one or more scene metadata in response to the content of the one or more scene update packets.

[0623] As another optional feature, the spatial conditions in a scene update packet, e.g., included in packet 1404, define a geometry element, and the audio decoder 1400 is configured to perform one or more scene metadata updates, e.g., represented at least in part using data structure 1430, in response to detecting that the current position has reached the geometry element or in response to detecting that the current position is within the geometry element.

[0624] As another optional feature, the audio decoder 1400 is configured to evaluate, e.g., using the evaluation unit 1440, whether an interactivity trigger condition is met to determine whether one or more scene metadata, e.g., the description data structure 1430, should be updated depending on the content of one or more scene update packets, e.g., included in packet 1404.

[0625] As another optional feature, the audio decoder 1400 is configured to evaluate the cell information, for example using an evaluation unit 1440, to determine which data structures are needed at which times and / or in which areas of the listener position.

[0626] As another optional feature, the audio decoder 1400 is configured to spatially render one or more audio signals using a first set of scene objects and / or scene characteristics when the listener position is in a first spatial region, e.g., with the rendering unit 1410, and the audio decoder is configured to spatially render the one or more audio signals using a second set of scene objects and / or scene characteristics when the listener position is in a second spatial region.

[0627] The first set of scene objects and / or scene characteristics optionally provide a more detailed spatial rendering compared to the second set of scene objects and / or scene characteristics.

[0628] The data structure 1430 may contain, describe, or represent information about scene objects and / or scene characteristics and may contain, describe, or represent such information with different levels of detail regarding their acoustic impact on the scene, for example, different levels of detail may be considered for rendering based on the spatial distance of the listener position from particular scene objects and / or scene characteristics.

[0629] As another optional feature, the audio decoder 1400 is configured to request one or more scene payload packets from a packet provider. Thus, as an optional feature, the encoder 1400 comprises a request unit 1460 for providing the request 1401.

[0630] As another optional feature, the audio decoder 1400 is configured to identify, for example using the evaluation unit 1440, one or more data structures to be used for spatial rendering using a payload identifier included in the cell information.

[0631] As another optional feature, the audio decoder 1400 is configured to request, eg, using the request unit 1460, one or more scene payload packets from a packet provider, eg, an encoder according to an embodiment.

[0632] As another optional feature, the audio decoder 1400 is configured to request one or more scene payload packets from the packet provider using a payload ID included in the cell information, e.g. using the requesting unit 1460, or the audio decoder is configured to request one or more scene payload packets from the packet provider using a packet ID, e.g. using the requesting unit 1460. Optionally, the respective payload IDs and / or packet IDs may be determined by the evaluation unit 1440 and provided to the requesting unit 1460.

[0633] As another optional feature, the audio decoder 1400 is configured to use the cell information, e.g. using the optionally shown anticipation unit 1470, to predict which one or more data structures are needed or expected to be needed, e.g. using the results of evaluation of the cell information by the evaluation unit 1440, and to request the one or more data structures, or one or more scene payload packets containing the one or more data structures, before the data structures are actually needed, e.g. by sending a request 1401 using the request unit 1460.

[0634] As another optional feature, the audio decoder 1400 is configured to extract payloads identified by the cell information from the bitstream. Thus, the decoder 1400 may optionally comprise an extraction unit, e.g. as shown in Fig. 2, which may be provided with extraction instructions from the evaluation unit 1440 based on the evaluation result of the cell information.

[0635] As another optional feature, the audio decoder 1400 is configured to use the cell information to track down the required data structures, for example based on the evaluation results of the evaluation unit 1440.

[0636] As another optional feature, the audio decoder 1400 is configured to selectively discard one or more data structures 1430 depending on the cell information. As an example, the discarded data structures of the plurality of data structures may not be considered for rendering or may be deleted, for example, in a memory of the decoder 1400.

[0637] As another optional feature, the cell information defines a location-based and / or time-based subdivision of the rendering scene (audio scene, rendering scenario).

[0638] As another optional feature, the audio decoder 1400 is configured to obtain the cell definitions based on a scene composition data structure, for example based on an evaluation thereof using the evaluation unit 1440.

[0639] As another optional feature, the audio decoder 1400 is configured to request one or more data structures, e.g. using the requesting unit 1460, and the audio decoder 1400 is configured to derive data structure identifiers for the requested data structures using the cell information, e.g. using the evaluation unit 1440.

[0640] As another optional feature, the audio decoder 1400 is configured to anticipate which one or more data structures are needed or expected to be needed, e.g., using the anticipation unit 1470, and to request one or more data structures before the data structures are actually needed, e.g., using the request unit 1460.

[0641] As another optional feature, the audio decoder 1400 is configured to extract one or more data structures using respective data structure identifiers, for example using the evaluation unit 1440, and the audio decoder 1400 is configured to derive data structure identifiers for the extracted data structures using the cell information.

[0642] As another optional feature, the audio decoder 1400 is configured to extract metadata required for rendering from the payload packets, for example using the evaluation unit 1440. Thus, the packets 1404 may optionally contain such payload packets. However, it is to be noted that the decoder 1400 may also be configured to receive multiple bitstreams, for example from broadcast and / or unicast channels. In the following, reference is made to FIG. 15. For the sake of brevity, and as explained above, it is to be noted that an embodiment according to the invention comprises an encoder having corresponding features according to the decoder shown in FIG. 14. Thus, the encoder may comprise only features necessary to provide a non-optional signal as received by the decoder 1400 of FIG. 14 and to process this to provide a decoded audio representation 1406, without any optional features. However, any of the optional features, functions and details disclosed above may be present in a corresponding encoder according to an embodiment, individually or in combination. The same applies to the features, functions and details of the decoders and / or encoders of other aspects of the invention.

[0643] Fig. 15 shows a schematic diagram of an encoder according to an embodiment of the fourth aspect of the present invention. The encoder 1500 is configured to provide a bitstream 1502, which may for example comprise an encoded audio representation. In particular, the encoder 1500 is configured to provide information for spatial rendering of one or more audio signals comprised in the bitstream 1502. The bitstream 1502 thus comprises a number of packets 1522 of different packet types.

[0644] To provide the bitstream 1502 and thus the above mentioned information entities, the encoder 1500 comprises a bitstream provider 1510 to which packets 1522 are optionally provided.

[0645] As shown, the encoder 1500 may comprise a packet providing unit 1520 for providing packets 1522. The packets 1522 include a scene configuration packet that provides renderer configuration information. Furthermore, the scene configuration packet includes cell information that defines one or more cells, the cell information defining an association between the one or more cells and one or more respective data structures associated with the one or more cells that define a rendering scenario.

[0646] As an example, an audio signal 1504 to be encoded may be provided to the encoder 1500. The audio signal may include, for example, time domain samples and / or spectral values ​​of a speech signal and / or music signal. Optionally (not shown), if this signal has already been encoded, it may be provided directly to the bitstream provider 1510, for example to be included in the bitstream together with the packets 1522.

[0647] The packets 1522 may be provided by the packet providing unit in different ways, for example, the packet providing unit 1520 may provide the packets 1522 based on scene information that defines an acoustic scene and may be predefined or may be obtained by the packet providing unit.

[0648] For example, in the case of a virtual reality application, based on a virtual model (e.g. of an acoustic scene), the packets may be used to determine acoustically related virtual objects to model or represent an acoustic scene (e.g. an acoustic scene including a virtual model having acoustically related virtual objects and audio signals). Optionally, the (optional) analysis unit 1530 may support the determination of acoustically related virtual objects, or characteristics thereof (e.g. characteristics of the audio signals may be used to supplement and / or refine the virtual model of the acoustic scene, e.g. by providing information regarding the position of sound sources).

[0649] Thus, information supporting the decision of each packet 1522 may be provided to the packet providing unit 1520 from the optional parsing unit 1530. For example, the packet providing unit 1520 may manage a virtual model or may be provided with information regarding the virtual model.

[0650] As another example, in the context of an augmented reality application, the audio signal 1504 to be encoded may be provided to the encoder 1500 and may comprise, for example, time-domain samples and / or spectral values ​​of a speech signal and / or music signal. Optionally, however, the audio signal 1504 may further comprise (or carry) spatial information of the real audio scene to be augmented (e.g., in an implicit form), for example, position information of a measured audio source, for example of a speaking user, in the scene. Such information, and optionally additional virtual overlay information (e.g., information for adding acoustically relevant virtual objects to a real scene to augment the real scene), may be extracted and / or analyzed and / or applied by the optional analysis unit 1530 to model or represent the scene using packets. Thus, as mentioned above, information for determining each packet 1522 may be provided to the packet providing unit 1520. As an example, the analysis unit 1530 may determine, based on the spatial information in the audio signal 1504, which acoustically relevant objects of the scene should be considered or updated or rendered to provide a desired auditory experience.

[0651] However, it should be noted that the encoder 1500 may optionally not include a parsing unit, e.g., information about the packet 1522 may be provided to the encoder 1500 from an external unit, e.g., a unit managing a virtual or augmented scene.

[0652] To further illustrate the above example, the audio signal 1504 may be an audio signal from a spatial audio scene and may optionally further include spatial information regarding the audio scene. Optionally, the encoder 1500 may therefore comprise an analysis unit 1530. The analysis unit 1530 may be configured to analyze information provided from the audio scene to determine or approximate a representation of the audio scene. As an example, the audio scene may be represented using metadata and / or data structures (e.g., data structures that include or are metadata) describing scene objects and / or scene characteristics that may be used together with the spectral coefficients of the audio signal to provide an immersive representation of the audio scene to a listener.

[0653] It should be noted that the metadata may be based, for example, on a digital model of the audio scene (e.g. in the case of virtual reality applications) and / or on an analysis of the actual scene from which the audio signal was recorded (e.g. for the case of augmented reality applications).

[0654] Based on this, as an example, corresponding scene configuration packets, scene update packets, and scene payload packets can be determined and provided, for example.

[0655] Thus, as an example, the packet providing unit 1520 may further provide packets containing said spectral information of the audio signal, for example in the form of packets representing one or more audio channels to be rendered (e.g., optionally packets such as MPEG H3D AFRAME), which may be part of the packets 1522.

[0656] As another optional feature, the encoder 1500 comprises a cell information unit 1540, which is configured to define or determine information regarding an association between one or more cells and the respective one or more data structures and provide this to the packet providing unit 1520. As optionally shown, such association may be performed based on an analysis of the audio signal 1504 using the analysis unit 1530. However, such information may be provided to the encoder 1500 optionally.

[0657] Optionally, the audio signal 1504 may be provided directly to the packet providing unit 1520 and / or to the bitstream provider 1510. As an example, the audio signal 1504 may already include defined packets 1522 such that these packets can only be extracted in the packet providing unit 1522 in order to be encoded in the bitstream provider 1510. The audio signal information may be provided to the bitstream provider 1510 separately from, for example, metadata information, for example in the form of packets or directly based on the audio signal 1504.

[0658] It should further be noted that the analysis unit 1530 may also determine or approximate a virtual acoustic scene, e.g. using the audio signal 1504 which represents only the acoustic signal itself, and further spatial properties of the scene may be based on a virtual model of the surroundings in the virtual acoustic scene, e.g. based on the position of the user within the virtual surroundings. For example, in a virtual conference room, the reflection properties of a virtual wall, or the attenuation properties of a virtual carpet may be incorporated as metadata, e.g. not based on real measurements, but based on a virtual acoustic model of the wall or carpet with respect to the position of the listener.

[0659] Again, it should be noted that audio encoder 1500 can optionally provide any of the packets disclosed herein, individually and in combination, for any of the audio decoders and / or encoders disclosed herein. Additionally, the cell information can include any of the characteristics disclosed herein, individually and in combination, for example, for the audio decoder.

[0660] As an optional feature, the apparatus 1500 is configured to repeat providing the scene configuration packets periodically, and / or to provide one or more scene payload packets on request.

[0661] The encoder 1500 therefore optionally comprises a requesting unit 1550 configured to receive the request 1508. The requesting unit 1550 can therefore instruct the packet providing unit 1520 to provide a packet 1522 including a scene payload packet, which the bitstream provider 1510 encodes into the bitstream 1502, as optionally shown. As mentioned above, the encoder 1500 may also be configured to provide multiple bitstreams, for example a broadcast and a unicast bitstream, where the global header data may be repeated in the broadcast bitstream and the requested data structures, payload packets, metadata or any information entities individually required for rendering may be provided via the unicast bitstream.

[0662] As another optional feature, the apparatus 1500 is configured to provide, for example using the packet providing unit 1520, one or more scene payload packets that include one or more data structures referenced in the cell information.

[0663] As another optional feature, the apparatus 1500 is configured to provide the scene payload packets taking into account when the data structures contained in the scene payload packets are needed by the audio decoder according to the cell information.

[0664] As another optional feature, the audio encoder 1500 is configured to provide first cell information defining a first set of scene objects and / or scene characteristics for rendering of the scene when a listener position, e.g., possibly provided to the encoder 1500 as an additional input signal or determined implicitly by the encoder based on the audio signal, is within a first spatial region (e.g., as determined by the analysis unit 1530), and the audio encoder is configured to provide second cell information defining a second set of scene objects and / or scene characteristics for rendering of the scene when the listener position is within a second spatial region. Furthermore, the first set of scene objects and / or scene characteristics provide a more detailed spatial rendering compared to the second set of scene objects and / or scene characteristics.

[0665] As another optional feature, the apparatus 1500 is configured to use different cell definitions to control spatial rendering at different levels of detail.

[0666] Fig. 16 is a schematic block diagram of a method for providing a decoded audio representation based on an encoded audio representation according to an embodiment of the fourth aspect of the present invention. The method 1600 comprises spatially rendering 1610 one or more audio signals and receiving 1620 a scene configuration packet providing renderer configuration information, the scene configuration packet comprising cell information defining one or more cells, the cell information defining an association between the one or more cells and respective one or more data structures associated with the one or more cells and defining a rendering scenario. Further, the method comprises evaluating 1630 the cell information to determine which data structure should be used for spatial rendering.

[0667] Fig. 17 is a schematic block diagram of a method for providing an encoded audio representation according to an embodiment of the fourth aspect of the present invention. The method 1700 comprises providing 1710 information for spatial rendering of one or more audio signals, providing a plurality of packets of different packet types, providing a scene update packet 1720, and providing a scene configuration packet 1730 providing renderer configuration information, said scene configuration packet comprising cell information defining one or more cells, said cell information defining an association between said one or more cells and respective one or more data structures associated with said one or more cells and defining a rendering scenario.

[0668] Reference is now made to Fig. 15. Bitstream 1502 in Fig. 15 represents audio content. An embodiment in accordance with the invention includes a bitstream such as the bitstream described above. In summary, such a bitstream includes multiple packets of different packet types providing scene configuration packets providing renderer configuration information, said scene configuration packets including cell information defining one or more cells, said cell information defining an association between said one or more cells and respective one or more data structures associated with said one or more cells and defining a rendering scenario.

[0669] It should be noted again that the embodiments according to the invention are described in four inventive aspects. It should be noted that such subdivision of the invention helps to facilitate the understanding of the gist of the invention and to limit redundancy. It should therefore be emphasized that any features, characteristics and / or details of the embodiments according to any optional aspect of the invention can be combined or individually incorporated in any other embodiment of the same or another aspect of the invention. It should be noted that in the same spirit, features, and details of the encoder may be used interchangeably for the decoder of the invention and vice versa. The same applies to the details of the bitstream and method of the invention.

[0670] It should further be noted that the embodiments according to the present invention include a decoder and / or a renderer. Although the decoder according to the embodiments may include the functionality of the renderer, the embodiments may include architectures in which these two functions are realized in different entities. In the above description, some embodiments have been described in which the decoder is used interchangeably with the renderer, e.g., further providing such functionality. However, it should be noted that this should be understood as an example. The functionality described above in one entity, e.g., the rendering unit of the decoder, may be split into two separate entities, i.e., the decoder and the renderer, with the renderer as an example comprising the rendering unit. Thus, the input and output signals can be split.

[0671] In the following further embodiments of the present invention are disclosed, which may be used in the context of or for dynamic VR / AR audio bitstreams, e.g. using three packet types, e.g. using scene update packets with update conditions, e.g. using timestamps, e.g. using cell information.

[0672] remarks: In the following, different inventive embodiments and aspects are described in the chapters "Overview", "New Approach" and "Description of Embodiments", as well as in the chapters "Application Examples" and "Aspects of the Invention". Further, optional details are provided in the appendices.

[0673] Further embodiments are defined by the appended claims.

[0674] It is noted that any embodiment defined by the claims can optionally be supplemented by any of the details (features and functions) described in the preceding paragraphs and in the remainder of the description.

[0675] Also, the embodiments described in the above sections can optionally be used individually or can be supplemented by any of the features of the other sections, or by any features included in the claims, or by any features as disclosed in the remainder of this specification.

[0676] It should also be noted that the individual aspects described herein can be used individually or in combination, and thus details can be added to each of the individual aspects without adding the details to another one of the aspects.

[0677] Moreover, features and functions disclosed herein in relation to a method can also optionally be used in an apparatus (configured to perform such functions). Moreover, any features and functions disclosed herein in relation to an apparatus can also optionally be used in a corresponding method. In other words, the methods disclosed herein can optionally be supplemented by any of the features and functions described in relation to the apparatus.

[0678] Additionally, any of the features and functionality described herein can be implemented in hardware or software, or using a combination of hardware and software, as described in the "Implementation Alternatives" section.

[0679] It should further be noted that the audio bitstream [or, equivalently, the encoded audio representation] may be optionally supplemented with any of the features, functions and details disclosed in this specification, both individually and in any combination.

[0680] Furthermore, as mentioned above, it should be noted that embodiments disclosed in the context of a decoder may include corresponding features, functionality and details of the corresponding encoder, either individually or in combination, and vice versa.

[0681] Further, in general, embodiments in accordance with the present invention may include a payload packet that includes one or more payloads, where a payload packet and a payload may be examples of a data structure, which may also be other types of packets.

[0682] The payload information may include metadata, e.g., information about an audio scene that is not directly related to the timeline of the corresponding audio stream, such as acoustically related geometry, parametric rendering instructions, and information about audio elements.

[0683] As an example, the metadata may include or be information about acoustically associated objects and / or properties.

[0684] Thus, the payload packets and respective payloads may contain definitions of one or more of the scene objects and / or one or more of the scene properties, and thus metadata.

[0685] 1. Overview One aspect of the present invention relates to a bitstream design, e.g., for six degree of freedom (6DoF) audio applications, such that the transmission of bitstream packets can be flexibly adapted to the application use case, e.g., in-situ standalone rendering, streaming, broadcast, or client-server approach.

[0686] Other aspects of the invention are also disclosed herein.

[0687] 2. Conventional Solutions and Their Shortcomings A traditional solution is the bitstream design found in MPEG audio codecs such as MPEG-H. In such a bitstream, data packets are transmitted, for example, primarily from a sender ("encoder") to a decoder / renderer. At the renderer site, user interactivity can generate additional packets that are used in a static way to render the audio content. Examples are head tracking data or level values ​​for a customized audio mix.

[0688] It has been found that six degrees of freedom (6DoF) audio applications sometimes require dynamic interactivity in VR / AR audio applications, and it has been found that transmitting 6DoF metadata should (or must) be taken into consideration for this to be efficient.

[0689] Sending all the required metadata from the sender to the receiver / renderer as bulk data up front is a simple and easy way to operate, but it has been found to involve huge data rate peaks and some transmission delay before the actual rendering can begin.

[0690] In certain applications, this simple solution proves to be even impossible, for example when the further evolution of the scene is unknown at the initial moment when the rendering starts.

[0691] According to one aspect, embodiments in accordance with the present invention can be used in such scenarios and applications.

[0692] However, according to another aspect, embodiments of the present invention may be used in different scenarios and applications.

[0693] 3. New methods According to one aspect, the inventive bitstream concept considers the increased level of interactivity inherent in, e.g., six degrees of freedom (6DoF) audio applications, and the large scale dynamic interactivity, e.g., especially in VR / AR audio applications, where the required rendering metadata is highly dependent on the user's local position, the user's interactions (e.g., pressing a virtual button, etc.), and / or the timeline of the virtual scene. Thus, according to one aspect, it is beneficial to design the bitstream in a flexible manner, such that, e.g., the transmission of bitstream packets can be flexibly adapted to the use case of the application, e.g., in-situ standalone rendering, streaming, broadcast, or client-server approach. [Mode for carrying out the invention]

[0694] Packet Types According to one aspect of the invention, the inventive bitstream is an extension of MHAS, i.e. an MPEG-H audio stream. In addition to the existing MHAS packet types, three new packet types are specified that allow for storing and transmitting metadata, e.g., for complex and dynamic 6DoF audio scenes.

[0695] Scene Config The Scene Config packet is, for example, the "header" of an MPEG-I bitstream, and may, for example, be repeated periodically in the bitstream for random access. It provides, for example, all the relevant information for a renderer to configure itself. In particular, it may contain, for example, which Scene Payload packets are needed at any given point in space and time, and (optionally) instructions on how they can be retrieved.

[0696] Scene Update Scene updates (i.e., changes to scene metadata that occur during playback) can be communicated using, for example, Scene Update packets, which make it possible to specify, for example, the conditions under which the update is performed (e.g., time-based triggers, and / or location-based triggers, and / or interactivity triggers) and the changes made to the scene.

[0697] Scene Payload The Scene Payload packet, for example, is a container for all the bulk metadata in the bitstream.

[0698] Bulk metadata is, for example, all metadata that cannot be directly associated with the timeline of the audio stream, but is needed (or useful) for rendering complex and dynamic 6DoF audio scenes, including, for example, (acoustically related) geometry, parametric rendering instructions, and audio element metadata. Packets may contain, for example, directivity, geometry, and special metadata for individual audio effects such as reverberation, early reflections, or diffraction.

[0699] The distribution of payloads into scene payload packets can be organized by the encoder, taking into account, for example, when and where metadata is needed by the renderer, what metadata is essential for rendering, and the maximum size of the packets.

[0700] Furthermore, the (optional) "level of detail" concept of individual modules allows spreading larger payloads (e.g., geometry) over a longer period in a broadcast scenario. In an (optional) server-client scenario, the payload can be loaded, for example, via a separate channel.

[0701] 4.2. Packet Syntax (Details are Optional)

[0702] [Table 1]

[0703] [Table 2] Notes: Updating the Encoder Input Format (EIF) by duration is just a special case of linear interpolation (optional) Optional interpolationType, which can be, for example, linear, sample-and-hold, cubic, etc. The optional "ListenerProximityCondition" from the EIF can be expressed as a spatial condition, e.g. For example, the combination of spatial and temporal conditions covers everything. Optional: The time condition can be updated to a time range at a later stage. · Parameter indicators may optionally be removed entirely (eg after CfP) since updates may for example consist of simply including modified entities and new values.

[0704] [Table 3]

[0705] [Table 4]

[0706] [Table 5] 4.3. Payload, Cell, and Subscene Decomposition (Optional Aspects and Examples) Scene Payload Packet (hereafter "Payload Packet") is a container for all bulk metadata that cannot be directly associated with the audio stream timeline, but is required (or useful) for rendering complex and dynamic 6DoF audio scenes, including, for example, (acoustically related) geometry, parametric rendering instructions, and audio element metadata.

[0707] In the simplest case, the entire bulk metadata for a scene can be contained in a single Payload packet that is sent to the decoder, e.g. before the start of scene playback. This is the preferred method for e.g. file-based transmission, where the Payload packet can be located, e.g., at the beginning of the file.

[0708] However, in streaming scenarios it may be beneficial to split bulk metadata into multiple payload packets to avoid large transmissions before scene playback can begin, for example.

[0709] Metadata may, for example, be needed (a) at a particular time, or (b) at a particular location in space. The optional "Cell" concept provides a way to specify at what time and / or location a particular payload packet is needed, for example, to render a scene.

[0710] For example, all Cells of a scene are defined in a Scene Config packet that may be located at the beginning of a file or repeated periodically in an MHAS stream, e.g. to allow random access. A decoder may, for example (or may have to) first parse the Scene Config packet to determine, for example, which payload packets are needed before scene playback can begin.

[0711] Geometry that specifies the volume in which, for example, a position-based Cell is active is called the "Cell bounds". A Cell is active if, for example, a listener is within the Cell bounds. Cell bounds can be, for example, any geometry, but primitive geometries (such as axis-aligned bounding boxes) are preferred for efficient implementations.

[0712] · Optionally, a set of zero or more globally required payloads, e.g., containing bulk metadata that is always required for proper rendering of the scene (e.g., including geometry specifying cell bounds).

[0713] For example, depending on the rendering profile, different payloads may be required for a given Cell. This way, for example, simplified geometry can be provided for a low complexity profile.

[0714] This document does not specify (or require) how the payload packets reach the decoder. There are several possibilities (which can be used, for example, in combination with embodiments of the present invention): a) According to an optional aspect, the decoder can request payload packets by ID, for example from a separate channel (e.g., TCP / IP based). In this case, for example, the decoder is responsible for fetching the payload packets so that the contained metadata is available when needed for scene playback.

[0715] b) According to an optional aspect, the payload packets are interleaved with the MHAS stream, e.g. in a broadcast scenario. Payloads required at a later time after the start of a scene can, e.g., be embedded in the stream before they are needed. However, if random access is allowed, this may require, e.g., repeated transmission of the required payload packets at regular intervals, and this method may not be suitable for scenes with a large amount of bulk metadata.

[0716] c) According to an optional aspect, in a "personalized" unicast stream, the sender may, for example, ensure that payload packets are embedded in the MHAS stream before they are needed.

[0717] (Possibly: According to an optional aspect, the decoder can track which payloads are obsolete by observing the end timestamps of time-based cells as well as location-based cells that are no longer active) For example, in extreme cases, to account for the conceptual possibilities of Cells, each acoustically relevant geometry and audio element can be packetized into a separate Payload packet with separate, possibly overlapping, location-based Cells, e.g., based on their perceptual relevance within the scene. For example, in this way, each geometry is only considered for geometry acoustic effects and / or all audio elements are only active if they are actually perceptually relevant at, e.g., the current listener position.

[0718] According to an optional aspect, this concept can also be exploited for Level of Detail (LOD) decomposition of a scene. For example, when far away from a geometric structure, a coarse geometric representation of the structure may be sufficient (e.g., with a small number of reflective surfaces), whereas when closer to the same structure, the reflections of the geometric structure and other effects related to geometric acoustics should be rendered with a higher LOD (e.g., with a large number of reflective surfaces). This can be achieved, for example, by specifying one Cell for the neighborhood of the considered geometry that contains a high LOD geometric representation and one Cell for the rest of the scene that contains a low LOD geometric representation.

[0719] For example, there are two options for sub-scene decomposition (i.e., deactivating certain elements if they are perceptually irrelevant).

[0720] a) Rendering of scene objects (e.g., audio elements and geometry) should only occur if they are currently in the associated payload packet.

[0721] b) A corresponding scene update allows for example to start and stop scene objects. For this, in addition to the EIF specification, the geometry may require for example an "isActive" flag.

[0722] For example, the acoustic environment (e.g. for a parametric description of late reverberation) may, but need not, be matched to a location-based cell.

[0723] 4.4. Notes on Scene Update Packets (Optional Aspects and Examples) For each parameter with a continuous value range, an interpolation type can be optionally specified. The interpolation type can be, for example, linear, sample-and-hold, cubic, etc., and the corresponding interpolation curve is constructed, for example, from a given support point.

[0724] 4.5. Scene Config Packet Notes (Optional Aspects and Implementations, Payload is Optional) · sceneSize (optional) - configures the scaling of positions within the scene, which can change the number of bits needed to encode a position if, for example, the highest expected value of any coordinate is smaller.

[0725] · timeScale (optional) - sets the interpretation of timestamps with respect to the renderer's clock source.

[0726] currentTime (optional) - timestamp of this config in the given timescale. In streaming scenarios, this can be used to set a new receiver time.

[0727] numSceneObjects (optional) - the number of total scene objects (e.g. audio elements + anchors / transforms + geometry), including all subscenes. This can be used, for example, to derive scene object id ranges and pre-allocate resources in the renderer.

[0728] globalPayloadId (optional) - the MHAS packet ID of the payload packet that contains artifacts that are globally needed to configure the renderer and render the scene. This includes, for example, geometry to describe cell boundaries. Can also be "none".

[0729] Audio Stream (optional) - a mapping from MPEG-I internal stream ID to a source, which can be for example either a physical PCM input channel (for "locally captured audio") or some kind of addressing of an MPEG-H 3D Audio frame channel. The optional "isLocallyCaptured" branch can be used for example for CfP channel configuration. The channel number should leave enough space for e.g. HOA. Furthermore, an audio stream can be (optionally) marked as for example "dynamic playback". In this case, the entire audio content should be preloaded and decoded for example, so that a dynamic playback trigger can be executed without delay. Each audio stream's data may also optionally contain information about delay compensation that may be required.

[0730] · Cells (optional) - Cells allow location and / or time based subdivision of a scene. For example, each cell (a) references geometry by ID describing the volume for which one or more payloads are required, and / or (b) references a timestamp for which a payload is required. Additionally, it can optionally reference the ID of a corresponding optional SceneUpdate that bundles all modifications to the scene that should occur when the cell trigger is executed. Geometries of location-based cells can optionally overlap. See also Section 4.3.

[0731] 5. Application examples (details optional) Below are three examples based on the preferred embodiment of the bitstream syntax. The examples address three different use cases where MHAS packets of the inventive MPEG-I metadata bitstream are combined with MHAS packets containing an MPEG-H audio stream ("MPEGH3DAFRAME"). For example, the MPEG-H audio stream transmits audio material (channels, objects, and HOA signals) to a 6DoF audio scene that is rendered by an MPEG-I decoder / renderer using the MPEG-I metadata.

[0732] The following embodiment is described as dealing with file-based MHAS streams (disclosed example, details optional).

[0733] Please refer to Fig. 18. Fig. 18 shows a schematic diagram of a first bitstream according to an embodiment of the present invention. The bitstream 1800 includes a scene configuration packet 1810, a number of scene payload packets 1820, 1830, and a number of packets 1840, 1850 containing an audio stream or parts of an audio stream, e.g., an MPEG-H audio stream. As shown in Fig. 18, the scene configuration packet 1810 may optionally include information 1812 about the state and / or settings of the respective decoder or renderer, and further information 1814 including cell information indicating which scene payload packets are required for which cells (e.g., using payload packet identifiers) and information such as Sphere count and ObjSrc count.

[0734] The first scene payload packet 1820 includes, by way of example, information about geometry 1822, animation information 1824, and information about audio elements 1826. The second scene payload packet 1830 includes, by way of example, information about three geometries 1832, 1834, 1836, and material information 1838.

[0735] Thus, as indicated in scene configuration packet 1810, cell 1 may be associated with multiple acoustically related information, e.g., geometry information as identified by indexes 12, 3, 4, and 5, animation information as identified by index 27, audio element information as identified by index 1, and material information as identified by index 1. Thus, in order to correctly render the audio scene, payload packets 1820 and 1830 containing information about these elements may be required when cell 1 is active. For example, an audio decoder may use payload packet identifiers associated with each cell to determine which payload packets should be evaluated (or retrieved and / or requested from a data source).

[0736] As another optional feature, payload packet 1 1820 may contain a coarse description of the geometry of the cell elements, which may be refined by the geometry information in payload packet 1830.

[0737] Thus, a decoder may only use refined geometry information in some cases (e.g., when a particular cell is active). For example, when cell 1 is active, the information in information packets 1820 and 1830 may be used, but when cell 2 is active, the decoder may use the information in payload packet 1820 but ignore the information in payload packet 1830.

[0738] As another example, the payload packet may define, for example, the geometry of the cell, and thus the geometric structure of the cell, as defined by, for example, information 1822, 1832, 1834, 1836.

[0739] In a stream application, for example, data is received from a single stream or data is read from a file. For example, MPEG-I scene configuration packets and payload packets are sent first, followed by several MPEG-H audio stream packets. At periodic intervals, e.g. reflected in updated timestamps included in the config packets, this packet sequence may be optionally repeated to allow random access or "tuning" to the MPEG-I scene. This application allows for example broadcasting, to transmit all the required data. On the downside, the broadcast bitstream data rate may be quite large in some cases, due to the large payload packets being transmitted and repeated periodically.

[0740] In the following embodiments, addressing a broadcast MHAS stream (eg, a client-server channel for payload packets) (eg, details are optional) is discussed.

[0741] Please refer to Figure 19. Figure 19 shows a schematic diagram of an inventive bitstream 1900 that includes a number of scene composition packets 1910, 1920 and a number of packets 1930, 1940 that include an audio stream, for example an MPEG-H audio stream.

[0742] In a client-server application, for example, only small scene composition packets, eg 1910, 1920, and MPEG-H audio stream packets, eg 1930, 1940, are received, eg from a broadcast stream or read from a file.

[0743] As a result, for example, only small MPEG-I scene configuration packets are transmitted, and are repeated at periodic intervals, for example with updated timestamps (e.g., packet 1910 is equal to 1920 apart from the timestamp information). The MPEG-I decoder / renderer can determine, for example, from a registry contained in the scene configuration packets, which payload packets 1950 need to enter the scene at a given timestamp or virtual location, and request only these payload packets once, for example, via a backchannel from a server, for example, an encoder. This keeps the broadcast bitstream data rate low, for example, compared to the first scenario.

[0744] In the following embodiment, we describe an embodiment that addresses a broadcast MHAS stream (eg, a client-server channel for payload packets) and involves dynamic scene changes (examples of which, details are optional).

[0745] Please refer to Fig. 20. Fig. 20 shows a schematic diagram of an inventive bitstream 2000 including a scene construction packet 2010, a number of packets 2020, 2030 including an audio stream, e.g. an MPEG-H audio stream, and a scene update packet 2040 including as an example timestamp information to indicate when e.g. the update itself, e.g. the association of entity indexes with modifications, should be performed.

[0746] Additionally, a payload packet 2050 is shown that may be requested, for example, over the backchannel.

[0747] For example, both types of broadcast streams may further include a scene update packet at any point in the packet sequence, e.g., 2040. The update packet itself may contain information regarding its execution, such as the time when it should be applied (e.g., immediately if the timestamp is less than or equal to the received scene time), or if any other triggers (e.g., user interaction such as manipulating a virtual object or entering a particular location in the virtual space) may activate this update.

[0748] All streams may contain MPEG-H Audio Stream MHAS synchronization packets, e.g. at selected bits in time or at regular time intervals, to make it possible to, e.g., cut the bitstream at any poin...

Claims

1. An audio decoder (600) for providing a decoded audio representation based on an encoded audio representation (602), comprising: the audio decoder is configured to spatially render one or more audio signals; the audio decoder is configured to receive a plurality of packets (604, 722) of different packet types; the packets include one or more scene configuration packets that provide renderer configuration information; the packets include one or more scene update packets that define updates to scene metadata for the rendering and include a representation of one or more update conditions; the audio decoder is configured to evaluate whether the one or more update conditions are met and, if the one or more update conditions are met, selectively update one or more scene metadata (630) in response to content of the one or more scene update packets. An audio decoder (600).

2. the audio decoder is configured to evaluate a temporal condition included in the scene update packets to determine whether to update one or more scene metadata (630) in response to a content of the one or more scene update packets.

2. An audio decoder according to claim 1.

3. The time condition defines a start time; or the time condition defines a time interval; the audio decoder is configured to perform one or more updates to scene metadata (630) in response to detecting that the current play time has reached or is after the start time; or the audio decoder is configured to perform one or more scene metadata updates in response to detecting that the current play time is within the time interval. An audio decoder (600) according to claim 2.

4. the audio decoder is configured to evaluate spatial conditions included in the scene update packets to determine whether to update one or more scene metadata (630) in response to a content of the one or more scene update packets. An audio decoder (600) according to any one of claims 1 to 3.

5. the spatial conditions define geometric elements; the audio decoder is configured to perform an update of one or more scene metadata (630) in response to detecting that the current position has reached the geometric element or in response to detecting that the current position is within the geometric element. An audio decoder (600) according to claim 4.

6. the audio decoder is configured to evaluate whether an interactivity trigger condition is satisfied to determine whether to update one or more scene metadata (630) in response to a content of the one or more scene update packets. An audio decoder (600) according to any one of the preceding claims.

7. the audio decoder is configured to evaluate a combination of two or more update conditions; the audio decoder is configured to selectively update one or more pieces of scene metadata (630) in response to content of the one or more scene update packets if a combined update condition is met. An audio decoder (600) according to any one of the preceding claims.

8. the audio decoder is configured to evaluate both a temporal update condition and a spatial update condition, or the audio decoder is configured to evaluate both a temporal update condition and an interactive update condition. An audio decoder (600) according to any one of the preceding claims.

9. the audio decoder is configured to evaluate delay information included in the scene update packets; the audio decoder is configured to delay updating one or more pieces of scene metadata (630) in response to detecting that the one or more update conditions are satisfied according to the delay information, depending on a content of the one or more scene update packets. An audio decoder (600) according to any one of the preceding claims.

10. the audio decoder is configured to evaluate a flag in the scene update packet indicating whether a temporal update condition is defined in the scene update packet; and / or the audio decoder is configured to evaluate a flag in the scene update packet indicating whether a spatial update condition is defined in the scene update packet. An audio decoder (600) according to any one of the preceding claims.

11. the audio decoder is configured to evaluate a flag in the scene update packet that indicates whether delay information is defined in the scene update packet. An audio decoder (600) according to any one of the preceding claims.

12. the scene update packet includes a representation of a plurality of modifications of one or more parameters of one or more scene objects and / or one or more scene characteristics; the audio decoder is configured to apply the modifications in response to detecting that the one or more update conditions are satisfied. An audio decoder (600) according to any one of the preceding claims.

13. the scene update packet includes trajectory information; the audio decoder is configured to update respective scene metadata (630) with which the trajectory information is associated using parameter variations that follow a trajectory defined by the trajectory information. An audio decoder (600) according to any one of the preceding claims.

14. the audio decoder is configured to evaluate information indicating whether trajectory-based updating of the scene metadata (630) is used to activate or deactivate trajectory-based updating of scene metadata. An audio decoder (600) according to claim 13.

15. the audio decoder is configured to evaluate interpolation type information included in the scene update packet to determine a type of interpolation between two or more support points of the trajectory. An audio decoder (600) according to claim 13 or 14.

16. the audio decoder is configured to evaluate fulcrum information describing the trajectory. An audio decoder (600) according to any one of claims 13 to 15.

17. An apparatus (700) for providing an encoded audio representation, comprising: the apparatus is configured to provide information for spatial rendering of one or more audio signals; the apparatus is configured to provide a plurality of packets (604, 722) of different packet types; the packets include one or more scene configuration packets that provide renderer configuration information; the packets include one or more scene update packets that define updates to scene metadata (630) for the rendering and that include a representation of one or more update conditions; Apparatus (700).

18. the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of a time condition for updating one or more scene metadata (630) in response to a content of the scene update packet.

20. The apparatus (700) of claim 17.

19. The time condition defines a start time; or The time condition defines a time interval.

20. The apparatus (700) of claim 18.

20. the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of a spatial condition for updating one or more scene metadata (630) in response to a content of the scene update packet.

20. Apparatus (700) according to any one of claims 17 to 19.

21. The spatial conditions define geometric elements.

21. The apparatus (700) of claim 20.

22. the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of an interactivity trigger condition for updating one or more scene metadata (630) in response to a content of the scene update packet.

22. Apparatus (700) according to any one of claims 17 to 21.

23. the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of a combination of two or more update conditions.

23. Apparatus (700) according to any one of claims 17 to 22.

24. the apparatus is configured to provide a scene update packet such that the scene update packet includes delay information defining, in response to detecting that the one or more update conditions are satisfied, a delay in updating one or more pieces of scene metadata (630) depending on a content of the one or more scene update packets.

24. Apparatus (700) according to any one of claims 17 to 23.

25. the device is configured to provide a scene update packet such that the scene update packet includes a flag indicating whether a temporal update condition is defined in the scene update packet; and / or the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of a flag indicating whether a spatial update condition is defined in the scene update packet; 25. Apparatus (700) according to any one of claims 17 to 24.

26. the apparatus is configured to provide the scene update packet such that the scene update packet includes a flag indicating whether delay information is defined in the scene update packet.

26. Apparatus (700) according to any one of claims 17 to 25.

27. the apparatus is configured to provide a scene update packet such that the scene update packet includes a representation of a plurality of modifications of one or more parameters of one or more scene objects and / or one or more scene characteristics.

27. Apparatus (700) according to any one of claims 17 to 26.

28. the apparatus is configured to provide a scene update packet such that the scene update packet includes trajectory information; the trajectory information describes updating each scene metadata (630) with which the trajectory information is associated using parameter variations that follow a trajectory defined by the trajectory information; 28. Apparatus (700) according to any one of claims 17 to 27.

29. the device is configured to provide a scene update packet such that the trajectory information includes information indicating whether trajectory-based updates of the scene metadata (630) are used to activate or deactivate trajectory-based updates of scene metadata.

30. The apparatus (700) of claim 28.

30. the apparatus is configured to provide a scene update packet such that the trajectory information includes interpolation type information included in the scene update packet to determine a type of interpolation between two or more support points of the trajectory; 30. Apparatus (700) according to claim 28 or 29.

31. the apparatus is configured to provide a scene update packet such that the trajectory information includes fulcrum information describing the trajectory.

31. Apparatus (700) according to any one of claims 28 to 30.

32. A method (800) for providing a decoded audio representation based on an encoded audio representation, comprising: The method includes spatially rendering (810) one or more audio signals; The method includes receiving (820) a plurality of packets (604, 722) of different packet types; the packets include one or more scene configuration packets that provide renderer configuration information; the packets include one or more scene update packets that define updates to scene metadata (630) for the rendering and include a representation of one or more update conditions; The method includes evaluating whether the one or more update conditions are met, and if the one or more update conditions are met, selectively updating (830) one or more scene metadata (630) in response to content of the one or more scene update packets. Method (800).

33. A method (900) for providing an encoded audio representation, comprising: The method includes providing (910) a plurality of packets (604, 722) of different packet types; the packets include one or more scene configuration packets that provide renderer configuration information; the packets include one or more scene update packets that define updates to scene metadata (630) for the rendering and that include a representation of one or more update conditions; Method (900).

34. 34. A computer program for carrying out the method according to claim 32 or 33 when said computer program is run on a computer.

35. A bitstream (702) for representing audio content, comprising: the bitstream includes a plurality of packets (604, 722) of different packet types; the packets include one or more scene configuration packets that provide renderer configuration information; the packets include one or more scene update packets that define updates to scene metadata (630) for the rendering and that include a representation of one or more update conditions; Bitstream (702).