Rendering of immersive audio

Abstract

A method for creating an association between audio and video media components. The method comprises obtaining the respective positions of an audio media component and a video media component before de

Description

Field The present application relates to apparatus and methods for rendering of immersive audio, but not exclusively for rendering of MPEG-I immersive audio signals reverberation with startup control in augmented reality and / or virtual reality apparatus. Background Moving Picture Experts Group - Immersive audio (MPEG-I) (MPEG-IIA) standard (International Organization for Standardization ISO / International Electrotechnical Commission IEC 23090-4) specifies immersive audio rendering with six degrees of freedom (6DoF) for the listener. This audio standard generates a MPEG-IIA bitstream that has the following main components: MPEG-IIA metadata bitstream - carries the audio scene representation and rendering metadata for performing 6DoF audio rendering; MPEG-IIA audio bitstreams - can utilize audio coded with MPEG-H 3DA codec (ISO / IEC 23008-3); and Computationally efficient renderer specification which utilizes the normative compressed bitstream optimized for performing the normative rendering. The MPEG-I Scene Description (ISO / IEC 23090-14) specifies a common scene description for different media. This enables common description of how the different media assets (volumetric visual content, haptics, etc.) together form the immersive scene. The scene description specification is defined as gITF (Graphics Library Transmission Format) extensions which extend the gITF functionality beyond that currently specified by the Khronos Group. gITF was formerly known as WebGL Transmissions Format or WebGL TF and is a standard file format for three-dimensional scenes and models. A gITF file uses one of two possible file extensions: .gltf (JSON / ASCII) or .gib (binary). Both gltf and .gib files may reference external binary and texture resources. Alternatively, both formats may be self-contained by directly embedding binary data buffers (as base64-encoded strings in .gltf files or as raw byte arrays in .gib files). gITF supports 3D model geometry, appearance, scene graph hierarchy, and animation. It is intended to be a streamlined, interoperable format for the delivery of 3D assets, while minimizing file size and runtime processing. The GL Transmission Format (gITF) as a JavaScript Object Notation (JSON) based rendering API agnostic runtime asset delivery format bridges the gap between 3D content creation tools and modern 3D applications by providing an efficient, extensible, interoperable format for the transmission and loading of 3D content. An example gITF asset comprises JSON files plus supporting external data. A specific example of a gITF asset is:: A JSON-formatted file (.gltf) containing a full scene description: node hierarchy, materials, cameras, as well as descriptor information for meshes, animations, and other constructs; Binary files (.bin) containing geometry and animation data, and other buffer-based data; and Image files (Jpg, .png) for textures. The JSON formatted file contains information about the binary files that describes how they may be used when uploaded to a graphics processing unit (GPU) with minimal processing. This makes gITF particularly well suitable for runtime delivery, as the assets may be directly copied into GPU memory for the rendering pipeline. Assets can be defined in other formats, such as images, may be stored in external files referenced via Uniform Resource Identifier (URI), stored side-by-side in GL binary (GLB) container, or embedded directly into the JSON using data URIs. gITF has furthermore been designed to allow extensibility. While the initial base specification supports a rich feature set, there will be many opportunities for growth and improvement. gITF defines a mechanism that allows the addition of both general-purpose and vendor-specific extension. The current scene description specification supports a basic audio extension which supports rendering audio objects and higher order ambisonics as audio sources in the scene. The listener is specified as a child node of the camera. Also, there is a possibility to specify reverberation properties for the scene. As part of ongoing work the use of scene description formats for enabling immersive augmented reality (AR) and 6 Degree-of-Freedom (6DoF) experiences has been studied. These investigations propose employing Khronos gITF as the starting point for MPEG scene description initiative for delivery of 3D and volumetric assets, as well as temporally changing audio-visual media. Another framework for scene description for interchange of 3D computer graphics data are, for example, Universal Scene Description (USD) or The Immersive Technologies Media Format (ITMF). Summary There is provided according to a first aspect a method for creating an association between audio and video media components, the method comprising: obtaining at least a position for a media component, the media component being one of an audio or video media component; obtaining at least a position for a further media component, the further media component being the other of the audio or video media component; determining a spatial relationship between the further media component and the media component; generating a scene description, the scene description defining at least one identifier associated with the determined spatial relationship. The method may further comprise adding to the scene description the further media component. The method may further comprise adding to the scene description the further media component as a bitstream. Determining the spatial relationship between the further media component and the media component may further comprise: determining a spatial or temporal offset between the further media component and the media component; and adding the spatial or temporal offset to the scene description. The media component and the further media component may be from a common scene. The media component and the further media component may be from different scenes. The scene description comprising the at least one media component and the at least one identifier may be represented within a Graphics Library Transmission Format file and the further media component may be represented by a Graphics Library Transmission Format file extension. The method may further comprise: obtaining a first media data stream associated with the media component; obtaining a further media data stream associated with the further media component; encoding the first media data stream, the further media data stream, the scene description and appended bitstream. The identifier associated with the determined spatial relationship may be a unique identifier. The scene description may further comprise a definition of the media component. According to a second aspect there is provided a method for processing associations between audio and video media components of a scene description, the method comprising: obtaining the scene description comprising: at least a position of a media component, the media component being one of an audio or video media component; and at least one identifier associated with a spatial relationship between a further media component and the media component; obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component; determining a spatial position update for the media component; updating a spatial position of the media component based on the spatial position update; and updating a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship. The scene description may further comprise a spatial or temporal offset between the further media component and the media component, and the updating the spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship may further comprise updating the spatial position of the further media component further based on the spatial or temporal offset. The media component and the further media component may be from a common scene. The media component and the further media component may be from different scenes. The scene description may be represented within a Graphics Library Transmission Format file and the further media component may be represented by a Graphics Library Transmission Format file extension. The method may further comprise: obtaining a first media data stream associated with the media component; obtaining a further media data stream associated with the further media component; wherein updating the spatial position of the media component based on the spatial position update and updating the spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship may comprise processing the first media data stream and the further media data stream based on the updating of the spatial position of the media component based on the spatial position update and based on the updating of the spatial position of the further media component as identified by the identifier based on the spatial position update and the spatial relationship when rendering an output audio signal. The media component or the further media component may be specified according to an encoder input format or MPEG-I Immersive audio encoder input format. The media component and the further media component may represent a single entity within a scene. According to a third aspect there is provided an apparatus for creating an association between audio and video media components, the apparatus comprising means configured to perform: obtaining at least a position for a media component, the media component being one of an audio or video media component; obtaining at least a position for a further media component, the further media component being the other of the audio or video media component; determining a spatial relationship between the further media component and the media component; generating a scene description, the scene description defining at least one identifier associated with the determined spatial relationship. The means may be further configured to perform adding to the scene description the further media component. The means may be further configured to perform adding to the scene description the further media component as a bitstream. The means configured to perform determining the spatial relationship between the further media component and the media component may further be caused to perform: determining a spatial or temporal offset between the further media component and the media component; and adding the spatial or temporal offset to the scene description. The media component and the further media component may be from a common scene. The media component and the further media component may be from different scenes. The scene description comprising the at least one media component and the at least one identifier may be represented within a Graphics Library Transmission Format file and the further media component may be represented by a Graphics Library Transmission Format file extension. The means may be further configured to perform: obtaining a first media data stream associated with the media component; obtaining a further media data stream associated with the further media component; encoding the first media data stream, the further media data stream, the scene description and appended bitstream. The identifier associated with the determined spatial relationship may be a unique identifier. The scene description may further comprise a definition of the media component. According to a fourth aspect there is provided an apparatus for processing associations between audio and video media components of a scene description, the apparatus comprising means configured to perform: obtaining the scene description comprising: at least a position of a media component, the media component being one of an audio or video media component; and at least one identifier associated with a spatial relationship between a further media component and the media component; obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component; determining a spatial position update for the media component; updating a spatial position of the media component based on the spatial position update; and updating a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship. The scene description may further comprise a spatial or temporal offset between the further media component and the media component, and the means configured to perform updating the spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship may further be configured to perform updating the spatial position of the further media component further based on the spatial or temporal offset. The media component and the further media component may be from a common scene. The media component and the further media component may be from different scenes. The scene description may be represented within a Graphics Library Transmission Format file and the further media component may be represented by a Graphics Library Transmission Format file extension. The means may be further configured to perform: obtaining a first media data stream associated with the media component; obtaining a further media data stream associated with the further media component; wherein the means configured to perform updating the spatial position of the media component based on the spatial position update and updating the spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship may be caused to further perform processing the first media data stream and the further media data stream based on the updating of the spatial position of the media component based on the spatial position update and based on the updating of the spatial position of the further media component as identified by the identifier based on the spatial position update and the spatial relationship when rendering an output audio signal. The media component or the further media component may be specified according to an encoder input format or MPEG-I Immersive audio encoder input format. The media component and the further media component may represent a single entity within a scene. According to a fifth aspect there is provided an apparatus, for creating an association between audio and video media components, the apparatus comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the system at least to perform: obtaining at least a position for a media component, the media component being one of an audio or video media component; obtaining at least a position for a further media component, the further media component being the other of the audio or video media component; determining a spatial relationship between the further media component and the media component; generating a scene description, the scene description defining the media component and at least one identifier associated with the determined spatial relationship. The apparatus may be further caused to perform adding to the scene description the further media component. The apparatus may be further caused to perform adding to the scene description the further media component as a bitstream. The apparatus caused to perform determining the spatial relationship between the further media component and the media component may further be caused to perform: determining a spatial or temporal offset between the further media component and the media component; and adding the spatial or temporal offset to the scene description. The media component and the further media component may be from a common scene. The media component and the further media component may be from different scenes. The scene description comprising the at least one media component and the at least one identifier may be represented within a Graphics Library Transmission Format file and the further media component may be represented by a Graphics Library Transmission Format file extension. The apparatus may be further caused to perform: obtaining a first media data stream associated with the media component; obtaining a further media data stream associated with the further media component; encoding the first media data stream, the further media data stream, the scene description and appended bitstream. The identifier associated with the determined spatial relationship may be a unique identifier. According to a sixth aspect there is provided an apparatus, for processing associations between audio and video media components, the apparatus comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the system at least to perform: obtaining the scene description comprising: at least a position for a media component, the media component being one of an audio or video media component; and at least one identifier associated with a spatial relationship between a further media component and the media component; obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component; determining a spatial position update for the media component; updating a spatial position of the media component based on the spatial position update; and updating a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship. The scene description may further comprise a spatial or temporal offset between the further media component and the media component, and the apparatus caused to perform updating the spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship may further be caused to perform updating the spatial position of the further media component further based on the spatial or temporal offset. The media component and the further media component may be from a common scene. The media component and the further media component may be from different scenes. The scene description may be represented within a Graphics Library Transmission Format file and the further media component may be represented by a Graphics Library Transmission Format file extension. The apparatus may be further caused to perform: obtaining a first media data stream associated with the media component; obtaining a further media data stream associated with the further media component; wherein the apparatus caused to perform updating the spatial position of the media component based on the spatial position update and updating the spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship may be caused to perform processing the first media data stream and the further media data stream based on the updating of the spatial position of the media component based on the spatial position update and based on the updating of the spatial position of the further media component as identified by the identifier based on the spatial position update and the spatial relationship when rendering an output audio signal. The media component or the further media component may be specified according to an encoder input format or MPEG-I Immersive audio encoder input format. The media component and the further media component may represent a single entity within a scene. According to a seventh aspect there is provided an apparatus for creating an association between audio and video media components, the apparatus comprising: obtaining circuitry configured to obtain at least a position of a media component, the media component being one of an audio or video media component; obtaining circuitry configured to obtain at least a position of a further media component, the further media component being the other of the audio or video media component; determining circuitry configured to determine a spatial relationship between the further media component and the media component; generating circuitry configured to generate a scene description, the scene description defining the media component and at least one identifier associated with the determined spatial relationship. According to an eighth aspect there is provided an apparatus for processing associations between audio and video media components of a scene description, the apparatus comprising: obtaining circuitry configured to obtain the scene description comprising: at least a position of a media component, the media component being one of an audio or video media component; and at least one identifier associated with a spatial relationship between a further media component and the media component; obtaining circuitry configured to obtain a bitstream defining the further media component, the further media component being the other one of the audio or video media component; determining circuitry configured to determine a spatial position update for the media component; updating circuitry configured to update a spatial position of the media component based on the spatial position update; and updating circuitry configured to update a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship. According to a ninth aspect there is provided a computer program comprising instructions [or a computer readable medium comprising instructions] for causing an apparatus, for creating an association between audio and video media components, the apparatus caused to perform at least the following: obtaining at least a position for a media component, the media component being one of an audio or video media component; obtaining at least a position for a further media component, the further media component being the other of the audio or video media component;; determining a spatial relationship between the further media component and the media component; generating a scene description, the scene description defining the media component and at least one identifier associated with the determined spatial relationship. According to a tenth aspect there is provided a computer program comprising instructions [or a computer readable medium comprising instructions] for causing an apparatus, for processing associations between audio and video media components of a scene description, caused to perform at least the following: obtaining the scene description comprising: at least a position of a media component, the media component being one of an audio or video media component; and at least one identifier associated with a spatial relationship between a further media component and the media component; obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component; determining a spatial position update for the media component; updating a spatial position of the media component based on the spatial position update; and updating a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship. According to a eleventh aspect there is provided a non-transitory computer readable medium comprising program instructions for causing an apparatus, for creating an association between audio and video media components, to perform at least the following: obtaining at least a position for a media component, the media component being one of an audio or video media component; obtaining at least a position for a further media component, the further media component being the other of the audio or video media component; determining a spatial relationship between the further media component and the media component; generating a scene description, the scene description defining the media component and at least one identifier associated with the determined spatial relationship. According to a twelfth aspect there is provided a non-transitory computer readable medium comprising program instructions for causing an apparatus, for processing associations between audio and video media components of a scene description, to perform at least the following: obtaining the scene description comprising: at least a position of a media component, the media component being one of an audio or video media component; and at least one identifier associated with a spatial relationship between a further media component and the media component; obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component; determining a spatial position update for the media component; updating a spatial position of the media component based on the spatial position update; and updating a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship. According to a thirteenth aspect there is provided an apparatus, for creating an association between audio and video media components, comprising: means for obtaining at least a position for a media component, the media component being one of an audio or video media component; means for obtaining at least a position for a further media component, the further media component being the other of the audio or video media component; means for determining a spatial relationship between the further media component and the media component; means for generating a scene description, the scene description defining the media component and at least one identifier associated with the determined spatial relationship. According to a fourteenth aspect there is provided an apparatus, for processing associations between audio and video media components of a scene description, comprising: means for obtaining the scene description comprising: at least a position for a media component, the media component being one of an audio or video media component; and at least one identifier associated with a spatial relationship between a further media component and the media component; means for obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component; means for determining a spatial position update for the media component; means for updating a spatial position of the media component based on the spatial position update; and means for updating a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship. According to a fifteenth aspect there is provided a computer readable medium comprising instructions for causing an apparatus, for creating an association between audio and video media components, to perform at least the following: obtaining at least a position for a media component, the media component being one of an audio or video media component; obtaining at least a position for a further media component, the further media component being the other of the audio or video media component; determining a spatial relationship between the further media component and the media component; generating a scene description, the scene description defining the media component and at least one identifier associated with the determined spatial relationship. According to an sixteenth aspect there is provided a computer readable medium comprising instructions for causing an apparatus, for processing associations between audio and video media components of a scene description, to perform at least the following: obtaining the scene description comprising: at least a position for a media component, the media component being one of an audio or video media component; and at least one identifier associated with a spatial relationship between a further media component and the media component; obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component; determining a spatial position update for the media component; updating a spatial position of the media component based on the spatial position update; and updating a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship. An apparatus comprising means for performing the actions of the method as described above. An apparatus configured to perform the actions of the method as described above. A computer program comprising program instructions for causing a computer to perform the method as described above. A computer program product stored on a medium may cause an apparatus to perform the method as described herein. An electronic device may comprise apparatus as described herein. A chipset may comprise apparatus as described herein. Embodiments of the present application aim to address problems associated with the state of the art. Summary of the Figures For a better understanding of the present application, reference will now be made by way of example to the accompanying drawings in which: Figure 1 shows schematically a system of example apparatus within which some embodiments may be implemented; Figure 2 shows a schematically a system of example apparatus with respect to the decoding and rendering within which some embodiments may be implemented; Figure 3 shows a flow diagram of the operation of the example encoding according to some embodiments; Figure 4 shows a flow diagram of the operation of the example presentation engine according to some embodiments; Figure 5 shows a schematically an example system of example apparatus with respect to integration of MPEG-IIA bitstream and extended scene description; Figure 6 shows a flow diagram of the operation of the example presentation engine according to some embodiments; and Figure 7 shows an example device suitable for implementing the apparatus shown in previous figures. Embodiments of the Application The following describes in further detail suitable apparatus and possible mechanisms for combined handling of different media components or entities of a single scene where the different media components or entities are processed according to different scene description approaches. Immersive audio inherently carries scene information because the different audio scene elements have different positions, orientations, etc. An example of such a technology is MPEG-H 3D Audio, specified as ISO / IEC 23008-3 (MPEG-H Part 3). MPEG-H 3D audio is an audio coding standard developed by the ISO / IEC Moving Picture Experts Group (MPEG) to support coding and rendering audio as audio channels, audio objects, or higher order ambisonics (HOA). MPEG-H 3D Audio can support up to 64 loudspeaker channels and 128 codec core channels. Similarly, Immersive Voice and Audio Services (IVAS) is currently being standardized by the 3rd Generation Partnership Project (3GPP), within 3GPP TS 26.250. IVAS is a new Enhanced Voice Services (EVS) based communication codec which was previously specified in 3GPP TS 26.441. IVAS enables a lifelike spatial acoustic experience, for example in conference calls and video conferences. As a conversational stereo and immersive codec, IVAS is configured to process signals captured by various microphone configurations in different conversational scenarios. Additionally just like EVS, IVAS is designed primarily as a mobile communications codec and can is configured to operate at bitrates as low as 13.2 or 24.4 kbit / s and with a very low latency. There are different scene description approaches such as Universal Scene Description (USD) for which further information can be found on https: / / developer.nvidia.com / usd. Universal Scene Description (USD) is a framework for interchange of 3D computer graphics data. The framework focuses on collaboration, non-destructive editing, and enabling multiple views and opinions about graphics data. Another example of scene description is MPEG-I Scene Description, ISO / IEC 23090-14 Scene Description support for immersive media codecs which specifies the coded representation of scene descriptions, The scene description of ISO / IEC 23090-14 is based on the GL Transmission Format (gITF), which lacks support for integrating multiple video bitstreams into a single 3D object. This MPEG-I scene description enables the association of multiple buffers with a single mesh object. Additionally, it enables these buffers to deliver decoded components of objects to the rendering engine so that the rendering engine can reconstruct the 3D point cloud or immersive video content. As the components of Visual-based Volumetric Visual coded (V3C) objects are encoded as 2D videos, support for YCbCr formats is also added. In scenes with multiple media components there is a need to control the spatio-temporal coordinates of the different media components via a common player or control entity such that the two or more media components are rendered with spatio-temporal consistency. For example, audio-visual media components, require an appropriate audiovisual temporal and spatial alignment while rendering by the player or any rendering entity to maintain the illusion of the audio and the visual media components being the same entity. However, in some cases, the audio and visual components do not have a common audio-visual scene representation. For example, a visual scene could be described as a single tree structure (i.e., the change in location of parent notes can affect the location of children nodes) whereas the audio scene may not be described as a single tree structure (or significantly hierarchical). In different embodiments, even if it is a tree structure, most of the nodes may be directly specified relative to the origin, thus producing a flat tree structure with most nodes at the same level, except for few nodes which may have child nodes. Currently MPEG-I Immersive audio standard (ISO / IEC / 23090-4) is not supported by the MPEG-I Scene description standard (ISO / IEC 23090-14). There is a need to specify with respect to current MPEG-I scene description standard such that the reliance on the normative bitstream specified in part 4 of MPEG-I can be retained while enabling the player or presentation engine or any entity that controls the playback of the different media types while maintaining consistency across the different media components. In order for this to be achieved the following requirements need to be addressed: • Enabling synchronization (spatial and temporal coordinates) between the different media components of a common audio scene description. This is important for 6DoF immersive media. • If the change in spatial coordinates occurs only based on predefined animation for visuals and audio, then the two could run in parallel without requiring the player to coordinate the playback other than the basic time synchronization, as is the case in case of O-DOF media. • A common mapping definition between audio bitstream components and the primary scene description which carries information for managing or orchestrating the rendering of the different media components. The concept as discussed in the embodiments in further detail herein relates to apparatus and methods for creating correspondence between the audio and visual media components of a primary scene description which enables a common presentation engine or a renderer entity or a player to jointly update the spatiotemporal coordinates of the audio and visual media components based on modification information for only one media component. In other words any changes in one media should be reflected in the other media for proper audio / visual rendering. This, in some embodiments, can be implemented by employing an extension which comprises the following: Correspondence information between the audio and visual entities in the primary scene description; Positional and rotational offset information between the audio and visual entities. This enables correct position and orientation transformation for corresponding entities which are not concentric with the primary scene description entity. In some embodiments, the scene description encoding, or creation phase can be summarized by the following (when the audio representation of the scene is modified based on changes in the visual representation of the scene): Determine the presence of entities that have run-time (or animated) spatiotemporal coordinate updates in the first scene or the first media component of a common scene (e.g., audio modality); Determine the associated entities in the second scene or second component of a common scene, corresponding to the run-time update entities in the first scene (or first media component of a common scene); and Apply run-time updates of the second scene (or second media component of a common scene) to the first scene (or first media component of a common scene). These updates can be implemented in both space and time. In some further embodiments (when the visual representation of the scene is modified based on changes to the audio representation of the scene) the scene description encoding, or creation phase can be summarized by the following: Determine presence of entities that have run-time (or animated) spatiotemporal coordinate updates in the first scene or the first media component of a common scene (e.g., audio); Determine the associated entities in the second scene or second media component of a common scene, corresponding to the update entities in the first scene (or first media component of a common scene); and Apply updates of the first scene (or first media component) to the second scene (or second media component). In some further embodiments, in a common scene, the first media component and the second media component can be represented in a gITF format. In these embodiments the signalling from the first media component to the second media component (or vice versa) of a common scene can be performed via an extension of the gITF. MPEG-I Immersive audio standard currently has support for 4 types of audio scene changes or updates (where the terminology can be used interchangeably). The scope of audio scene updates or changes also includes audio scene entity movement or motion or animation. These are as follows: • Audio scene changes such as audio scene entity animation which is included in the bitstream. This can be known to the presentation engine beforehand based on the bitstream information. • Audio scene changes that are specified in the bitstream but triggered based on user interaction. The presentation engine needs to track the interaction triggering to enable initiation of the corresponding visual entity changes. Thus correspondence information is required for handling such updates to ensure audio-visual consistency and alignment. • Audio scene changes that are specified in the bitstream but triggered based on user or listener position in the scene. The presentation engine needs to track the triggering of listener position based scene changes. Thus correspondence information is required for handling such updates. • Audio scene changes that are specified during run-time. Since the position information for the audio scene entities is available during runtime or during the scene consumption, there is a need for correspondence information between the audio scene entities and visual scene entities for consistent audio-visual scene changes. Thus, for many audio scene change types, correspondence information is required for the presentation engine in order to enable consistent audio-visual scene experience. In absence of such correspondence information, the audio scene changes are included in the visual scene as part of content authoring step. In this scenario the visual scene game or engine can be employed to drive the changes to the audio scene. Although because this approach has limited effectiveness it is not a flexible approach. Figure 1 shows schematically an example system within which some embodiments could be implemented. In this example implementation is shown the scenario where immersive audio rendering is performed in accordance with the MPEG-I Immersive audio standard, the audio is coded via MPEG-H 3DA audio standard and the first scene description is specified according to the MPEG-I Scene description standard. However, this example can be applied without loss of generality to any other implementations employing other standards or formats. The concept as shown herein is where a first scene representation is used as the basis for modifying the entities in a second scene, this is achieved by extending the first scene representation approach by including the correspondence information as well as the bitstream reference. There are three main entities as shown in Figure 1, which are the encoder input data 180, the server computer (or encoder) 190 and the playback device (or decoder or renderer) 192. The encoder input data 180 comprises the first scene representation 100 which can also be the master or main or first scene description, for example the visual scene. The encoder input data 180 can further comprise a second scene representation 104, the second or main scene description. The encoder input data 180 can further comprise the visual data 106, which can be visual object bitstreams, for example, volumetric video objects, 2D rectilinear videos. The encoder input data 180 can further comprise the audio data or signals 108, which can be in any suitable format or representation. For example the audio data can be any of: audio objects, one or more audio channels (for example a multichannel 5.1 channel format), or a First or higher order Ambisonics (FOA or HOA) representation. The encoder input data 180 is passed to the server implementing an encoding function to generate the correspondence information to be included in the first scene description (or main or primary) where the correspondence information is included as specified later herein. For example, the server 190 comprises a main scene description document generator (with extension for immersive audio) 101 configured to implement such functionality. The second (or main) scene representation 104 and rendering bitstream can furthermore be passed to an Immersive audio encoder 103. The immersive audio encoder (e.g. a MPEG-I Immersive audio coder) 103 is configured to encode the second scene description and passes the encoded sound scene information to the main scene description document generator 101 where the encoded second scene representation is included via reference or by a binary blob in the first scene description as specified later in further detail. The scene description bitstream 111 can then be passed to the bitstream encoder 107 and stored. In addition, the audio data 108 can be passed to a suitable MPEG-H 3D audio encoder 105 (or any suitable audio encoder) which generates an encoded audio bitstream 115 which can be passed to the bitstream encoder 107 and stored (for example as a bitstream on the server 190). Furthermore, the visual data can be stored as a bitstream 113 in any suitable format for retrieval on the server 190. The encoded visual bitstream 113 can be, for example, passed to the bitstream encoder 107 and stored. In this figure all the bitstreams (scene, visual and audio) are situated on the same server. However, in some embodiments, the bitstreams can be located on different servers or also stored a local storage device without the network interface. The server / encoder 190 can communicate via a suitable network 110 to the playback device / decoder / renderer 192. Although the example shown in Figure 1 is implemented with separate server 190 and playback devices 192 which communicate over the network 110, it would be understood that in some embodiments that the server 190 and playback device 192 are implemented on the same physical hardware and that the bitstreams 111, 113, 115, 107 or any combination of them are stored as a file or as a number of files and retrieved at a later time. In these common hardware examples the network 110 can be considered to represent any suitable storage medium on the apparatus or associated with the apparatus. The playback device 192 comprises a presentation engine (PE) 141. The PE retrieves the scene description document, from a scene description retriever 121. The scene description retriever 121 can request the scene from the server 190 based on a scene selection by the user for playback. The PE 141 in some embodiments comprises a scene description parser 125 configured to receive the (first) scene description from the server and parse the scene description. The scene parser can furthermore control or instantiate the media access function pipelines and initiate any suitable retrieval, decoding and buffer pipeline. Thus for example the scene description parser 125 controls the audio-visual media retriever 123 which retrieves the audio-visual bitstreams from the server via the network. Additionally the scene description parser 125 controls the audio-visual aligners 129, 133. The retrieved audio-visual bitstreams can then be decoded in an audiovisual media decoder 127, the output of which can be passed to the audio-visual aligners 129, 133. The first scene description carries information for retrieving the second scene description or immersive audio (e.g. MPEG-IIA bitstream). Subsequently, based on the changes to the one or more first scene entity position changes, the PE 141 can be configured to perform audio-visual alignment based on the correspondence information. In this example, the first scene representation is a visual scene, and the second scene representation is an audio scene. However, in practice, these could be any two different media components. Thus, rendered output audio is delivered to the output buffers with aligned audio-visual entities. This can be implemented by an Audio-visual aligner (implemented by renderer stages pipeline of audio renderer) 129 and Audio-visual aligner (implemented as part of a Visual renderer pipeline) 133. The audio-visual aligners 129, 133 can then output to output buffers 131, 133 which are configured to output to a suitable output device, such as a head mounted device (HMD) 170. Additionally, the HMD or suitable sensors provide information which can be passed to a suitable head pose generator 137 for establishing the position of the listener in the audio scene (and which can be 3DoF, 3DoF+, or 6DoF). In this example, the visual scene rendering is not impacted via changes in audio scene entities, but this audio-visual alignment could be implemented both ways in different examples. For example, if there is a scene where the change in the visual object “person” leads to change in the position of the audio object corresponding to “person” is reflected from the visual scene to the audio scene. Furthermore, in the same scene, the change in the audio object “TV” may lead to change in the visual object “TV” if there is two-way correspondence. The playback device / decoder / renderer 192 functionality can be implemented on an end-user-device, which can be a mobile device, personal computer, sound bar, tablet computer, car media system, home HiFi or theatre system, head mounted display for AR or VR, smart watch, or any suitable system for audio consumption. With respect to Figure 2 is shown in further detail a schematic visualization of the rendering and decoding functions (for audio) which are implemented in some embodiments. In this example is shown the input bitstreams, the MPEG-H 3DA Audio bitstream 201 and the MPEG-I bitstream 211. The MPEG-H 3DA Audio bitstream 201 can be passed to a suitable decoder, such as the MPEG-H 3DA Audio decoder 203 which can decode and pass the audio signals to the MPEG-I renderer 221. The MPEG-I renderer 221 can furthermore be configured to receive the MPEG-I bitstream 211. Additionally, the MPEG-I renderer 221 can receive via the interface 230 information such as user position and interaction 235, local scene updates 231 and consumption environment information 233. In some embodiments the consumption or listener space environment acoustic information can be in the form of Listener Space Description Format (LSDF) information), which is further described in MDS22224, N00183 MPEG-I immersive audio augmented reality Listener Space Description Format, Version 2. The MPEG-I renderer 221 can then generate the audio output 241 for a suitable output means such as headphones (though in some embodiments the output means can be any suitable audio transducer means). The implementation of the concept for the synchronization of the various medias as reflected in the following embodiments can be separated into three parts. A first part can relate to the server or encoder or encoding operations where an enhanced version of the scene description document is generated which can be used by a presentation engine or player or any other suitable entity for assisting rendering of an audio-visual scene such that the audio scene entities are rendered in accordance with the change in the spatiotemporal coordinates of visual scene entities. A second part can relate to the presentation engine or presentation engine operations where alignment of the audio scene entities is performed in accordance with the change in the spatiotemporal coordinates of the visual scene entities (or more generally alignment of the first scene entities or first media entities is performed in accordance with the change in the spatiotemporal coordinates of the second scene entities or second media entities alignment). A third part can relate to the enhanced scene description specification. This third part describes, as example, the extension to a gITF scene description and storage or reference to audio scene rendering metadata or bitstream. With respect to the first part, the encoder or encoding part, a flow diagram for an example implementation of an example encoder according to some embodiments can be shown in Figure 3. The first scene description can, as shown in figure 3 by 301, be received or otherwise obtained. A scene description document describes the position, orientation of the one or more media assets that comprise the entire scene. A media asset can be any media file or media component referenced by any entity (for example a node) in scene representation. This scene can consist of different scene representations for each media or modality. In different implementations, the scene can consist of different media components that constitute the complete scene. In some embodiments, the (first) master scene comprises a visual scene and the audio scene is represented via another scene description method. For example, the common scene description document (or primary scene description or the master scene description) which is implemented according to the gITF specification with extensions for MPEG media [ISO / IEC 23090-14]. The audio scene or audio media component can be specified according to the encoder input format [EIF in MPEG-I Immersive audio Encoder Input Format version 5, MDS22558_WG06_N00200] which is the input format for authoring the audio scene and generating the MPEG-I Immersive audio transmission bitstream representation comprising the audio scene as well as the rendering metadata according to the MPEG-I Immersive audio standard. The first scene description received by the encoder can represent the visual scene in the MPEG-I scene description compliant format. Then the second scene description, as shown in Figure 3 by 303, can be received or otherwise obtained by the encoder. In some embodiments the second scene description can be in the EIF format or directly in the form of MPEG-IIA bitstream. Subsequently, as shown in Figure 3 by 305, the encoder is configured to analyze the correspondence between the audio and visual scene entities to determine audio scene entities (or more generally second scene entities) that should be spatially aligned with the visual scene entities (or more generally first scene entities) such that any visual scene entity position or orientation change reflects in a spatial position or orientation change for the one or more audio scene entities. This can be, for example, a determination that there is a spatial relationship between the entities or media components. This can for example be determined following obtaining at least a position of a media component, the media component being one of an audio or video media component and further obtaining at least a position of a further media component, the further media component being the other of the audio or video media component. Then can be determined a spatial relationship between the further media component and the media component. These entities as indicated above could be audio objects corresponding to a visual object in the visual scene or in some cases may even correspond to scene elements such as a wall or door in the audio-visual scene. As a result of the analysis, unique identifiers can be added to audio scene entities (second scene entities) with the first scene as additional information to the first scene description as shown in Figure 3 by 307. Additionally in some embodiments, there is a check or determination as to whether there is an offset (spatial or temporal) required between the visual scene (first scene) entity and the one or more audio scene (second scene) entities as shown in Figure 3 by 309. In other words that the spatial relationship can be defined by at least one of: a spatial offset; and temporal offset. Where there is an offset (spatial or temporal) required between the visual scene (first scene) entity and the one or more audio scene (second scene) entities, this can is recorded as additional offset information. The adding of spatial position and / or orientation offset information for the determined corresponding second scene entity with respect to the first scene entity is shown in Figure 3 by 311. This spatial and / or temporal offset is added to the first scene description document. In the final step, as shown in Figure 3 by 313, following the offset information being added or determining no offset is required, the audio scene representation and bitstream or reference or indication to the audio bitstream is added to the visual (first) scene description in order to enable MPEG-I Immersive audio compliant rendering of the audio scene. With respect to the second part, the presentation engine (or the playback device more generally), a flow diagram for an example implementation of the presentation engine or playback device is shown in Figure 4. The player retrieves or otherwise obtains the first scene description, as shown in Figure 4 by 401, which in the current example is considered to be generated in accordance with the MPEG-I Scene description specification. The first scene description is primarily representing the visual scene in the current example description. In some embodiments the first scene description comprises correspondence information with second scene entities and second scene rendering bitstream or location for the second scene rendering bitstream. The second scene description is for the audio scene which is represented in accordance with MPEG-I Immersive specification, i.e., accordance with MPEG-IIA metadata bitstream. This second scene description is referred to by the first scene description. The renderer pipeline for the visual scene is initialized according to the first scene description information. The renderer pipeline for the audio scene is initialized according to the MPEG-IIA bitstream. The player can then, as shown in Figure 4 by 403 receive or otherwise obtain a second scene rendering bitstream from the scene description file or a URL for the second scene rendering bitstream. The player can then be configured to receive or obtain position update information regarding the first scene entities as shown in Figure 4 by 405. Then there can be a determining for (each) firs scene entity with a position update the presence of a corresponding second scene entity as shown in Figure 4 by 407. Finally there is an updating of the position and / or orientation for the second scene entities corresponding to the first scene entities with position updates as shown in Figure 4 by 409. In other words, there is a check for finding out any correspondence information between the visual scene and the audio scene. In case of audio and visual scene not having any correspondence, which is possible if both are specified to follow only the respective scene information (first scene and second scene respectively), the PE or player renders the audio and visual scene without any explicit need for correspondence. However, if the first scene description carries the correspondence information between the one or more visual scene entity and the one or more audio scene entities (e.g., spatial offset information and correspondence entity identifiers) as specified in the encoding steps, the PE or player needs to align the presentation of the audio scene (in this example) in accordance with the change in the visual scene entity spatial position or orientation. Figure 5 shows an overview of the functions of implementing synchronization and rendering. In this example the first scene description (referred to as Scene Description 515) and the second scene description and immersive audio rendering bitstream are shown. The MPEG-IIA metadata bitstream can either be included directly as a binary blob (shown as option B 513) in the first scene description file (shown as option B 515) or it can be referred to as a separate binary blob available as a track or a URI (shown as option A 507). The MPEG-IIA metadata bitstream can in some embodiments be delivered directly to the MPEG-I Decoder / Renderer 509 whereas the MPEG-IIA audio bitstreams (i.e., MPEG-H 3DA audio data) is delivered to the renderer via the MPEG-H 3DA media access function pipeline (receiving the audio elements 505i,...,505n and comprising the MPEG-H decoders 506i,...,506n). The MPEG-H 3DA audio data can be retrieved according to the URIs in the scene description document (for example from web storage 501 or local storage 503) and decoded via the MPEG-H 3DA decoder 506i,... ,506n before making it available to the buffer 507i,...,507n. The MPEG-I Immersive audio renderer 509 extracts the necessary audio frames at a specified or configured interval to the immersive audio renderer. With respect to Figure 6 are shown a flow diagram of the operations of an example presentation engine with respect to the rendering of the audio scene in accordance to changes in the visual scene entity positions such that the same change is reflected in the audio scene entity positions. Thus, for example, the presentation engine or immersive media player is started, as shown in Figure 6 by 601. In some embodiments this is started by the user consuming the immersive audio-visual scene. The presentation engine can be configured to retrieve or obtain the primary or common or visual scene description from the specific location (e.g., URI or local storage) and then parses the scene description document as shown in Figure 6 by 603. The PE can then be configured to parse the first scene, to obtain the visual scene information as shown in Figure 6 by 605 Additionally, the second scene, for example from the audio scene description is obtained as well as the immersive audio rendering metadata as shown in Figure 6 by 607. The audio scene description and metadata can be obtained from the parsed MPEG-I SD. In some embodiments the metadata can be MPEG-IIA or any other suitable immersive audio rendering technology. Subsequently, for all the audio scene elements, media access function pipelines are instantiated or initialized as shown in Figure 6 by 609, to retrieve the relevant data (e.g., audio data for audio elements and rendering metadata to facilitate rendering of audio acoustic effects). In other words, media access function pipelines can be initialized for the visual scene objects and the MPEG-I immersive audio bitstream or any other immersive audio rendering metadata. After initializing the media access functionality, the presentation engine starts fetching the data for audio scene as shown in Figure 6 by 611. The data for the audio scene, the audio data or audio streams, may be coded with MPEG-H 3DA codec, EVS, AAC or any other suitable coding technology. Furthermore the visual scene data (e.g., volumetric visual data for V3C assets) is fetched and rendering metadata for immersive audio rendering (if there are updates to the rendering metadata or other audio scene changes) is also fetched. Subsequently for every scene change instance, as shown in Figure 6 by 613, the PE determines if there is a corresponding audio scene entity. If there is, then the same change is applied in accordance with the spatiotemporal offsets described in the first scene description for the corresponding second scene entities as shown in Figure 6 by 615. Then the second scene (audio elements) is rendered, as shown in Figure 6 by 617, in accordance with the changes occurring for the visual scene entities. This is of specific importance where the animation information for audio and visual scene entities are known beforehand (i.e., during content creation and encoding). With respect to the third part the definition of the extension, in some embodiments, a new extension is proposed to extend the existing gITF Khronos specification and ISO / IEC 23090-14 specification, where the extension carries the following information. This extension defines that for each parent node in the visual scene or primary or master scene representation, one or more audio correspondence node identifiers are attached through the extension. Each of the audio correspondence nodes is the same as the unique identifier of the audio scene entity to uniquely identify the correspondence relationship. Furthermore the position and / or rotation offset information of the audio scene entity with respect to the visual or master scene objects is defined. 5 An example of the extension specification can be as follows: { "$schema":" https: / / ison-schema.orq / draft / 2020-12 / schema", "$id": "MPEG_audio_immersive.schema.json", "title": "MPEG_audio_immersive", "type": "object", "description": "gITF extension for specifying relation of the node to MPEG-I Immersive Audio entities ", "allOf": [ {"$ref": "gITFProperty.schema.json"} ], "properties": { "mpegi_audio_entities": { "type": "array", "items": {"$ref":"MPEG_audio_immersive.mpegi_audio_entity.schema.json"}, "minitems": 1 "extensions": {}, "extras": {} }. "required": ["mpegi_audio_entities"] { "$schema": "https: / / json-schema.org / draft / 2020-12 / schema", "$id": "MPEG_audio_immersive.audio_element.schema.json", "title": "MPEG_audio_immersive.audio_element", "type": "object", "description": "Audio entity that provides audio entity id and the transformation (translation + rotation or matrix) of the audio entity in respect to the parent node object in the visual scene." "allOf": [ {"$ref": "glTFChildOfRootProperty.schema.json"} ], "properties": { "source": { "type": ["number", "string", "object"], "description": "Is the source of the audio from which the audio entity originated." "audio_entity_id": { "type": ["number", "string", "object"], "description": "Audio entity identifier mapping with the visual parent node object." "matrix": { "type": "array", "description": "A floating-point 4x4 transformation matrix stored in column-major order, describing audio entity’s rotation and translation, in respect to the parent node object", "items": { "type": "number" }, "minitems": 16, "maxitems": 16, "default": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0], "gltf_webgl": "'uniformMatrix4fv()' with the transpose parameter equal to false" }, "rotation": { "type": "array", "description": "The audio entity’s quaternion rotation, in the order (x, y, z, w), where w is the scalar, in respect to the parent node object.", "items": { "type": "number", "minimum": -1.0, "maximum": 1.0 }, "minitems": 4, "maxitems": 4, "default": [0.0, 0.0, 0.0, 1.0] }, "translation": { "type": "array", "description": "The audio entity’s translation, along thex, y, and z axes, in respect to the parent node object.", "items": { "type": "number" }, "minitems": 3, "maxitems": 3, "default": [ 0.0, 0.0, 0.0 ] }, "extensions": {}, "extras": {} }, "not": { "anyOf": [ {"required": [ "matrix", "translation" ]}, {"required": [ "matrix", "rotation" ]}, In the MPEG_audio_immersive.audio_element source property can be: a media source to refer to the MPEG-I Immersive audio metadata bitstream information to be carried as part of MPEG_media extension in the common or primary scene representation; a URL to refer to the MPEG-I Immersive audio metadata bitstream; A reference to the accessor, by specifying the accessor's index in accessors array, that describes the buffer where: MPEG-H audio bitstream is provided which can be implicitly mapped to the audio entity ID in MPEG-IIA Metadata bitstream (in that case audio_entity_id would not need to be present in the extension); and MPEG-IIA Metadata bitstream is provided. In MPEG_audio_immersive.audio_element audio_entity_id property combined with the source property provides a unique identifier of MPEG-I audio entity id, audio_entity_id can be : A unique ID that is the same as audio entity ID in MPEG-I metadata bitstream describing the scene. For example, in combination with source element an URI can be created, i.e. combination of the path to the MPEG-I metadata bitstream and MPEG-I audio entity ID in that bitstream. For example file: / / path / to / mpeg_i_bitstream.bin / ?mpegi_audio_entity_id=123 As such in some embodiments, the following can be implemented: The PE reads the scene description document as an entry point to the immersive audio-visual application; The PE parses the JSON scene description to build a scene graph in memory; The PE then iterates through all objects in the node (audio and visual), determines the associated media sources and their corresponding buffer formats and timing information; In the example audio scene, the renderer is initialized based on the MPEG-IIA metadata bitstream data; In the example audio scene, the PE initializes the MAF for each audio source (object, channel, HOA) and can allocate a (circular) buffer for the audio data (MPEG-H 3DA audio); The PE then invoices startFetching for media pipeline ahead of the requested presentation time for the corresponding audio sources (objects, channels, HOA). In some embodiments similar extensions or functionality that map MPEG-I audio entities to the visual objects in USD or ITEF scene description, or any other visual scene description, is provided. With respect to Figure 7 an example electronic device which may be used as any of the apparatus parts of the system as described above. The device may be any suitable electronics device or apparatus. For example, in some embodiments the device 1600 is a mobile device, user equipment, tablet computer, computer, audio playback apparatus, etc. The device may for example be configured to implement the encoder or the renderer or any functional block as described above. In some embodiments the device 1600 comprises at least one processor or central processing unit 1607. The processor 1607 can be configured to execute various program codes such as the methods described herein. In some embodiments the device 1600 comprises a memory 1611. In some embodiments the at least one processor 1607 is coupled to the memory 1611. The memory 1611 can be any suitable storage means. In some embodiments the memory 1611 comprises a program code section for storing program codes implementable upon the processor 1607. Furthermore, in some embodiments the memory 1611 can further comprise a stored data section for storing data, for example data that has been processed or to be processed in accordance with the embodiments as described herein. The implemented program code stored within the program code section and the data stored within the stored data section can be retrieved by the processor 1607 whenever needed via the memory-processor coupling. In some embodiments the device 1600 comprises a user interface 1605. The user interface 1605 can be coupled in some embodiments to the processor 1607. In some embodiments the processor 1607 can control the operation of the user interface 1605 and receive inputs from the user interface 1605. In some embodiments the user interface 1605 can enable a user to input commands to the device 1600, for example via a keypad. In some embodiments the user interface 1605 can enable the user to obtain information from the device 1600. For example, the user interface 1605 may comprise a display configured to display information from the device 1600 to the user. The user interface 1605 can in some embodiments comprise a touch screen or touch interface capable of both enabling information to be entered to the device 1600 and further displaying information to the user of the device 1600. In some embodiments the user interface 1605 may be the user interface for communicating. In some embodiments the device 1600 comprises an input / output port 1609. The input / output port 1609 in some embodiments comprises a transceiver. The transceiver in such embodiments can be coupled to the processor 1607 and configured to enable a communication with other apparatus or electronic devices, for example via a wireless communications network. The transceiver or any suitable transceiver or transmitter and / or receiver means can in some embodiments be configured to communicate with other electronic devices or apparatus via a wire or wired coupling. The transceiver can communicate with further apparatus by any suitable known communications protocol. For example, in some embodiments the transceiver can use a suitable universal mobile telecommunications system (UMTS) protocol, a wireless local area network (WLAN) protocol such as for example IEEE 802.X, a suitable short-range radio frequency communication protocol such as Bluetooth, or infrared data communication pathway (IRDA). The input / output port 1609 may be configured to receive the signals. In some embodiments the device 1600 may be employed as at least part of the renderer. The input / output port 1609 may be coupled to headphones (which may be a headtracked or a non-tracked headphones) or similar. In general, the various embodiments of the invention may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof. The embodiments of this invention may be implemented by computer software executable by a data processor of the mobile device, such as in the processor entity, or by hardware, or by a combination of software and hardware. Further in this regard it should be noted that any blocks of the logic flow as in the Figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD. The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The data processors may be of any type suitable to the local technical environment, and may include one or more of general-purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASIC), gate level circuits and processors based on multi-core processor architecture, as non-limiting examples. Embodiments of the inventions may be practiced in various components such as integrated circuit modules. The design of integrated circuits is by and large a highly automated process. Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate. Programs, such as those provided by Synopsys, Inc. of Mountain View, California and Cadence Design, of San Jose, California automatically route conductors and locate components on a semiconductor chip using well established rules of design as well as libraries of pre-stored design modules. Once the design for a semiconductor circuit has been completed, the resultant design, in a standardized electronic format (e.g., Opus, GDSII, or the like) may be transmitted 5 to a semiconductor fabrication facility or "fab" for fabrication. The foregoing description has provided by way of exemplary and nonlimiting examples a full and informative description of the exemplary embodiment of this invention. However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, 10 when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of this invention will still fall within the scope of this invention as defined in the appended claims.

Claims

1. A method for creating an association between audio and video media components, the method comprising:obtaining at least a position for a media component, the media component being one of an audio or video media component;obtaining at least a position for a further media component, the further media component being the other of the audio or video media component;determining a spatial relationship between the further media component and the media component;generating a scene description, the scene description defining at least one identifier associated with the determined spatial relationship.

2. The method as claimed in claim 1, further comprising adding to the scene description the further media component.

3. The method as claimed in claim 2, further comprising adding to the scene description the further media component as a bitstream.

4. The method as claimed in any of claims 1 to 3, wherein determining the spatial relationship between the further media component and the media component further comprises:determining a spatial or temporal offset between the further media component and the media component; andadding the spatial or temporal offset to the scene description.

5. The method as claimed in any of claims 1 to 4, wherein the media component and the further media component are from a common scene.

6. The method as claimed in any of claims 1 to 3, wherein the media component and the further media component are from different scenes.

7. The method as claimed in any of claims 1 to 6, wherein the scene description comprising the at least one media component and the at least one identifier is represented within a Graphics Library Transmission Format file and the further media component represented by a Graphics Library Transmission Format file extension.

8. The method as claimed in claim 3 or any claim dependent on claim 3, further comprising:obtaining a first media data stream associated with the media component;obtaining a further media data stream associated with the further media component;encoding the first media data stream, the further media data stream, the scene description and appended bitstream.

9. The method as claimed in any of claims 1 to 8, wherein the identifier associated with the determined spatial relationship is a unique identifier.

10. The method as claimed in any of claims 1 to 9, wherein the scene description further comprises a definition of the media component.

11. A method for processing associations between audio and video media components of a scene description, the method comprising:obtaining the scene description comprising:at least a position of a media component, the media component being one of an audio or video media component; andat least one identifier associated with a spatial relationship between a further media component and the media component;obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component;determining a spatial position update for the media component;updating a spatial position of the media component based on the spatial position update; andupdating a spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship.

12. The method as claimed in claim 11, wherein the scene description further comprises a spatial or temporal offset between the further media component and the media component, and the updating the spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship further comprises updating the spatial position of the further media component further based on the spatial or temporal offset.

13. The method as claimed in any of claims 11 or 12, wherein the media component and the further media component are from a common scene.

14. The method as claimed in any of claims 11 or 12, wherein the media component and the further media component are from different scenes.

15. The method as claimed in any of claims 11 to 14, wherein the scene description is represented within a Graphics Library Transmission Format file and the further media component represented by a Graphics Library Transmission Format file extension.

16. The method as claimed in any of claims 11 to 15, further comprising: obtaining a first media data stream associated with the media component; obtaining a further media data stream associated with the further media component;wherein updating the spatial position of the media component based on the spatial position update and updating the spatial position of the further media component as identified by the identifier based on the spatial position update for the media component and the spatial relationship comprises processing the first media data stream and the further media data stream based on the updating of the spatial position of the media component based on the spatial position update andbased on the updating of the spatial position of the further media component as identified by the identifier based on the spatial position update and the spatial relationship when rendering an output audio signal.

17. The method as claimed in any of claims 1 to 16 wherein the media component or the further media component is specified according to an encoder input format or MPEG-I Immersive audio encoder input format.

18. The method as claimed in any of claims 1 to 17, wherein the media component and the further media component represent a single entity within a scene.

19. An apparatus for creating an association between audio and video media components, the apparatus comprising means configured to perform:obtaining at least a position of a media component, the media component being one of an audio or video media component;obtaining at least a position of a further media component, the further media component being the other of the audio or video media component;determining a spatial relationship between the further media component and the media component; andgenerating a scene description, the scene description defining at least one identifier associated with the determined spatial relationship.

20. An apparatus for processing associations between audio and video media components of a scene description, the apparatus comprising means configured to perform:obtaining the scene description comprising:at least a position of a media component, the media component being one of an audio or video media component; andat least one identifier associated with a spatial relationship between a further media component and the media component;obtaining a bitstream defining the further media component, the further media component being the other one of the audio or video media component;determining a spatial position update for the media component;updating a spatial position of the media component based on the spatial position update; andupdating a spatial position of the further media component as identified by 5 the identifier based on the spatial position update for the media component and the spatial relationship.

Images