Method for real-time texture adaptation

Abstract

An apparatus, comprising: a component for receiving a scene description, wherein the scene description includes data associated with the scene; a component for placing the data associated with the scene into a data buffer and creating a command buffer; a component for adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from a local media or a network media; a component for signaling information about the adaptation to update the command buffer of a command renderer; and a component for rendering the scene using the data in the data buffer and the command buffer.

Description

Technical Field

[0001] Examples and non - limiting embodiments generally relate to multimedia and software, and more particularly to a method for real - time texture adaptation. Background Art

[0002] Performing video encoding and decoding is known. Summary of the Invention

[0003] According to one aspect, an apparatus includes: means for receiving a scene description, where the scene description includes data associated with the scene; means for placing the data associated with the scene into a data buffer and creating a command buffer; means for adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from a local media or a network media; means for signaling information about the adaptation to update the command buffer of a command renderer; and means for rendering the scene using the data in the data buffer and the command buffer.

[0004] According to one aspect, an apparatus includes: at least one processor; and at least one non - transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least perform: receiving a scene description, the scene description including data associated with the scene; placing the data associated with the scene into a data buffer and creating a command buffer; adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from a local media or a network media; signaling information about the adaptation to update the command buffer of a command renderer; and rendering the scene using the data in the data buffer and the command buffer.

[0005] According to one aspect, a method includes: receiving a scene description, the scene description including data associated with the scene; placing the data associated with the scene into a data buffer and creating a command buffer; adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from a local media or a network media; signaling information about the adaptation to update the command buffer of a command renderer; and rendering the scene using the data in the data buffer and the command buffer.

[0006] According to one aspect, there is provided a machine-readable non-transitory program storage device tangibly embodying a program of machine-executable instructions for performing operations including: receiving a scene description including data associated with the scene; placing the data associated with the scene into a data buffer and creating a command buffer; adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from a local or network medium; signaling information about the adaptation to update the command buffer of a command renderer; and rendering the scene using the data in the data buffer and the command buffer. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The foregoing aspects and other features are described in connection with the accompanying drawings in the following description, in which:

[0008] Figure 1 A high-level block diagram showing an implementation of real-time texture adaptation.

[0009] Figure 2 A block diagram of a scene description.

[0010] Figure 3 An example apparatus configured to implement texture adaptation based on the examples described herein.

[0011] Figure 4 An embodiment of an example apparatus configured to implement texture adaptation based on the examples described herein.

[0012] Figure 5 An example method for implementing texture adaptation based on the examples described herein.

[0013] Figure 6 Another example method for implementing texture adaptation based on the examples described herein. DETAILED DESCRIPTION

[0014] The following abbreviations and acronyms, which may be found in the specification and / or the drawings, are defined as follows:

[0015] 2D or 2d Two-dimensional

[0016] 3D or 3d Three-dimensional

[0017] 3GP-DASH Progressive download and HTTP-based dynamic adaptive streaming

[0018] 3GPP Third Generation Partnership Project

[0019] 4CC Four-Character Code

[0020] API Application Programming Interface

[0021] assoc Association

[0022] AHS Adaptive HTTP Streaming

[0023] altr Alternate

[0024] Amd Amended

[0025] AVC Advanced Video Coding

[0026] .bin Binary file

[0027] CM Conditional Mandatory State in XML

[0028] CPU Central Processing Unit

[0029] DASH HTTP-based Dynamic Adaptive Streaming

[0030] dinf Data Information Frame

[0031] DRM Digital Rights Management

[0032] EBML Extensible Binary Meta Language

[0033] GL Graphics Library or Graphics Language

[0034] GLB Binary file format representation of a 3D model saved in GL Transmission Format

[0035] GLTF or gltf GL Transmission Format

[0036] .gltf JSON format file

[0037] GPU Graphics Processing Unit

[0038] hdlr Handler Frame

[0039] HEVC High Efficiency Video Coding

[0040] HTTP HyperText Transfer Protocol

[0041] HTTP GET Method for requesting resources from a server

[0042] ID or id Identifier

[0043] idat Item Data Frame

[0044] IEC International Electrotechnical Commission

[0045] ISO International Organization for Standardization

[0046] ISOBMFF ISO Base Media File Format

[0047] JPEG Joint Photographic Experts Group

[0048] .jpg JPEG file extension for image files

[0049] JSON JavaScript Object Notation

[0050] Leva level assignment

[0051] Mandatory state in M XML

[0052] mdat media data frame

[0053] MIME Multipurpose Internet Mail Extensions

[0054] moov movie frame

[0055] moof movie clip frame

[0056] mvex movie extension

[0057] MP4 Filename extension for MPEG-4 Part 14 files

[0058] MPD Media Presentation Description

[0059] MPEG Moving Picture Experts Group

[0060] MPEGI MPEG Immersive

[0061] Optional states in O XML

[0062] Optional with default mandatory status in OD XML

[0063] OMAF Omnidirectional Media Format

[0064] OpenGEX Open Game Engine Exchange

[0065] .png Portable Network Graphics image file

[0066] PSS Packet Switched Stream Transport

[0067] RFC Request for Comments

[0068] SMIL Synchronized Multimedia Integration Language

[0069] SRD spatial relationship description

[0070] ssix subsegment index

[0071] trak track frame

[0072] TS Technical Specification

[0073] URI Uniform Resource Identifier

[0074] URL Uniform Resource Locator

[0075] URN Uniform Resource Name

[0076] VWPT Viewpoint Information

[0077] XML Extensible Markup Language

[0078] Frame structure Hua File format

[0079] The concepts of box-structured and hierarchical file formats have been widely used for media storage and sharing. The most well-known file formats in this regard are the ISO Base Media File Format (ISOBMFF) and its variants such as the MP4 and 3GPP file formats.

[0080] ISOBMFF allows the storage of audio / video media streams captured in a timely manner (referred to as media tracks). The metadata describing the tracks is separated from the encoded bitstream itself. This format provides a mechanism to access media data in a codec-independent manner from the perspective of a file parser.

[0081] The basic building blocks in ISOBMFF are called boxes. Each box has a header and a payload. The box header indicates the type of the box and its size in bytes. The box type is usually identified by an unsigned 32-bit integer (interpreted as a four-character code (4CC)). A box can contain other boxes, and the ISO file format specifies which box types are allowed within a particular type of box. In addition, the presence of some boxes in each file can be mandatory, while the presence of others can be optional. Also, for some box types, more than one box of that type can be allowed to exist in a file. Therefore, ISOBMFF can be considered to specify a hierarchy of boxes.

[0082] In a file conforming to the ISO Base Media File Format, the media data can be provided in one or more instances of the MediaDataBox ('mdat'), and the MovieBox ('moov') can be used to include the metadata of the timed media. In some cases, for the file to be operable, the presence of both the'mdat' and'moov' boxes may be required. The'moov' box can include one or more tracks, and each track can reside in a corresponding TrackBox ('trak'). Each track is associated with a handler, identified by a four-character code specifying the track type. Video, audio, and image sequence tracks can be collectively referred to as media tracks, and they contain the basic media streams. Other track types include timed metadata tracks.

[0083] A track contains samples, such as audio or video frames. For a video track, the media samples can correspond to encoded pictures or access units. A media track reference refers to samples (which can also be referred to as media samples) that are formatted according to a media compression format (and its encapsulation in the ISO Base Media File Format). A timing metadata track can refer to samples that describe the referenced media samples.

[0084] Alternative track

[0085] ISOBMFF includes a special feature called "alternate track". This feature enables signaling of any temporally equivalent alternative of the media. This is signaled using specific fields in the track header box (from the ISOBMFF specification - ISO / IEC 14496-12):

[0086]

[0087]

[0088] alternate_group is an integer that specifies a group or set of tracks. If this field is 0, there is no information about possible relationships with other tracks. If this field is not 0, it should be the same for tracks that contain alternative data for each other, and different for tracks belonging to different such groups. At any given time, only one track in an alternate group should be played or streamed, and it must be distinguishable from other tracks in the group via attributes such as bitrate, codec, language, packet size, etc. A group can have only one member.

[0089] Typically, the alternate grouping field indicates alternatives for media tracks, such as:

[0090] - Different languages of the same audio track

[0091] - Different resolution or bitrate options of the same media track

[0092] - Different views of a 2D scene that are temporally aligned with the main 2D scene (i.e., different camera angles)

[0093] During the presentation time, only one of these alternative media tracks should be played. This restriction comes from the ISOBMFF specification and the alternate_group field definition. The playback behavior of playing multiple such media tracks is not defined.

[0094] Media players typically read alternative grouping information and create tree-structured information that groups tracks together, and then select the first track (i.e., the lowest-indexed) in the alternative tracks for initial playback. Additionally, the user can also manually switch between these alternatives.

[0095] Track group

[0096] A TrackGroupBox is contained by a TrackBox. The TrackGroupBox enables the indication of track groups, where each group shares a specific characteristic or the tracks within the group have a specific relationship. The TrackGroupBox contains zero or more boxes, and the specific characteristic or relationship is indicated by the box type of the contained boxes. The contained boxes include identifiers that can be used to determine the tracks belonging to the same track group. Tracks that contain the same type of contained boxes within a TrackGroupBox and have the same identifier value within these contained boxes belong to the same track group. Track groups are not used to indicate dependencies between tracks. Instead, the TrackReferenceBox is used for this purpose.

[0097] The syntax of the TrackGroupBox is as follows:

[0098]

[0099] Item

[0100] A file conforming to ISOBMFF can contain any non - timed object in a meta box (four - character code:'meta'), which is referred to as an item, a meta - item, or a metadata item. Although the name of the meta box refers to metadata, an item can generally contain either metadata or media data. The meta box can reside at the top - level of the file, within a movie box (four - character code:'moov'), and within a track box (four - character code: 'trak'), but at most one meta box can appear at each file - level, movie - level, or track - level. The meta box may need to contain an 'hdlr' box to indicate the structure or format of the'meta' box content. The meta box can list and characterize any number of items that can be referenced, and each item can be associated with a file name and is uniquely identified within the file by an item identifier (item_id, an integer value). Metadata items can, for example, be stored in the 'idat' box of the meta box, or in the'mdat' box, or reside in a separate file. If the metadata is located outside the file, its location can be declared by a DataInformationBox (four - character code: 'dinf'). In the specific case where the metadata is formatted using the Extensible Markup Language (XML) syntax and needs to be stored directly in the MetaBox, the metadata can be encapsulated into an XMLBox (four - character code: 'xml') or a BinaryXMLBox (four - character code: 'bxml'). An item can be stored as a contiguous byte range, or can be stored in several extents, where each extent is a contiguous byte range. In other words, an item can be stored in segments as extents, for example to enable interleaving. An extent is a contiguous subset of the bytes of a resource. A resource can be formed by concatenating multiple extents.

[0101] The ItemPropertiesBox enables any item to be associated with a set of ordered item properties. Item properties can be regarded as small data records. The ItemPropertiesBox consists of two parts: an ItemPropertyContainerBox, which contains a list of implicitly - indexed item properties; and one or more ItemPropertyAssociationBoxes, which associate items with item properties.

[0102] Entity group

[0103] Entity groups have been specified in ISO / IEC 14496 - 12:2015 / Amd.2:2018. An entity group is a grouping of items, which can also group tracks. Entities within an entity group share specific characteristics or have specific relationships, as indicated by the grouping type.

[0104] The entity group is indicated in the GroupsListBox. The entity group specified in the GroupsListBox of the file-level MetaBox refers to a track or a file-level item. The entity group specified in the GroupsListBox of the movie-level MetaBox refers to a movie-level item. The entity group specified in the GroupsListBox of the track-level MetaBox refers to a track-level item of that track. When there is a GroupsListBox in the file-level MetaBox, there is no item_ID value in any ItemInfoBox in any file-level MetaBox that is equal to the track_ID value in any TrackHeaderBox.

[0105] The GroupsListBox contains EntityToGroupBoxes, and each EntityToGroupBox specifies an entity group. The four-character box type of the EntityToGroupBox indicates the defined grouping type.

[0106] The 'altr' entity grouping type is defined as follows: The items and tracks mapped to this grouping are alternatives to each other, and only one of them should be played (when the mapped items and tracks are part of a presentation; for example, are displayable image items or tracks) or otherwise processed (when the mapped item or track is not part of a presentation; for example, is metadata). The player should select from the list of entity_id values the first entity that it can process (e.g., decode and play the mapped items and tracks that are part of a presentation) and that is suitable for the application's needs. Any entity_id value should be mapped to only one grouping of the 'altr' type. The alternative group of an entity consists of those items and tracks that are mapped to the same entity group of type 'altr'.

[0107] HTTP-based Dynamic Adaptive Streaming over HTTP (DASH)

[0108] DASH is an adaptive bitrate streaming technology that enables the delivery of high-quality media content streams over the Internet from a conventional HTTP web server.

[0109] MPEG-DASH

[0110] The Hypertext Transfer Protocol (HTTP) has been widely used to deliver real-time multimedia content over the Internet, such as in video streaming applications. Several commercial solutions for HTTP-based adaptive streaming have been introduced, such as Smooth Streaming, Adaptive HTTP Live Streaming, and Dynamic streaming has been standardized in the context of the Adaptive HTTP Streaming (AHS) which was first standardized in Release 9 of the Packet Switched Streaming (PSS) service of the 3rd Generation Partnership Project (3GPP) (3GPP TS 26.234 Release 9: "Transparent end-to-end packet switched streaming service (PSS); Protocols and codecs").

[0111] MPEG uses 3GPP AHS Release 9 as the starting point for the MPEG DASH standard (ISO / IEC 23009-1: "Dynamic Adaptive Streaming over HTTP (DASH) - Part 1: Media Presentation Description and Segment Format"). MPEG DASH and 3GPP-DASH are technically close to each other and can thus be collectively referred to as DASH. Some concepts, formats, and operations of DASH are described below as examples of a video streaming system in which embodiments can be implemented. Aspects of the examples described herein are not limited to DASH, but are described with respect to one possible basis on which the examples described herein can be partially or fully implemented.

[0112] In DASH, multimedia content can be stored on an HTTP server and delivered using HTTP. The content can be stored on the server in two parts: the Media Presentation Description (MPD), which is a manifest that describes the available content, its various alternatives, their URL addresses, and other characteristics; and segments, which contain the actual multimedia bitstream in the form of chunks in a single or multiple files. The MPD provides the necessary information for the client to establish HTTP-based dynamic adaptive streaming. The MPD contains information describing the media presentation, such as the HTTP Uniform Resource Locator (URL) for each segment for making GET segment requests.

[0113] To play the content, a DASH client can obtain the MPD, for example, by using HTTP, email, a thumb drive, broadcast, or other transport methods. By parsing the MPD, the DASH client can learn about the program timing, media content availability, media type, resolution, minimum and maximum bandwidth, and the presence, accessibility characteristics, and required Digital Rights Management (DRM) of various encoding alternatives of the multimedia components, the location of the media components on the network, and other content characteristics. Using this information, the DASH client can select a suitable encoding alternative and start streaming the content, for example, by retrieving segments using HTTP GET requests. After appropriate buffering to accommodate network throughput variations, the client can continue to retrieve subsequent segments, while also monitoring network bandwidth fluctuations. The client can decide how to adapt to the available bandwidth by retrieving segments of different alternatives (with lower or higher bitrates) to maintain sufficient buffering.

[0114] In the context of DASH, the following definitions can be used: A media content component or media component can be defined as a continuous component of media content that has a specified media component type that can be individually encoded into a media stream. Media content can be defined as a media content period or a continuous sequence of media content periods. A media content component type can be defined as a single type of media content such as audio, video, or text. A media stream can be defined as an encoded version of a media content component.

[0115] In DASH, a hierarchical data model is used to construct media presentations as described below. A media presentation consists of a sequence of one or more periods, each period containing one or more groups, each group containing one or more adaptation sets, each adaptation set containing one or more representations; each representation consists of one or more segments. A group can be defined as a set of adaptation sets that are not expected to be presented simultaneously. An adaptation set can be defined as a set of interchangeable encoded versions of one or more media content components. A representation is one of the alternative choices of media content or a subset thereof, which typically differs due to encoding choices, e.g., due to bitrate, resolution, language, codec, etc. A segment contains media data of a certain duration, as well as metadata for decoding and presenting the included media content. A segment is identified by a URI and can generally be requested via an HTTP GET request. A segment can be defined as a data unit associated with an HTTP-URL and optionally associated with a byte range specified by the MPD.

[0116] The DASH MPD conforms to the Extensible Markup Language (XML) and is thus specified by elements and attributes as defined in XML. The MPD can use the following convention for specification: Elements in the XML document can be identified by an uppercase first letter and can be shown in bold as "Element". To state that element Element1 is contained within another element Element2, it can be written as Element2.Element1. If the name of an element consists of two or more combined words, camel case can be used, e.g., ImportantElement. An element can occur once only, or the minimum and maximum number of occurrences can be specified by <minoccurs> ... <maxoccurs>Definition. Attributes in an XML document can be identified by a lowercase first letter or by prefixing them with the "@" symbol, e.g., @attribute. To refer to a specific attribute @attribute contained in an element Element, it can be written as Element@attribute. If the name of an attribute consists of two or more combined words, camel case can be used after the first word, e.g., @veryImportantAttribute. Attributes can be assigned states in XML, such as mandatory (M), optional (O), optional with a default value (OD), and conditionally mandatory (CM).

[0117] In DASH, all descriptor elements are typically constructed in the same way, as they contain the @schemeIdUri attribute that provides the URI for the identification scheme, as well as the optional attributes @value and @id. The semantics of the element are specific to the scheme being used. The URI for the identification scheme can be a URN or a URL. Some descriptors are specified in MPEG-DASH (ISO / IEC 23009-1), while some descriptors can be additionally or alternatively specified in other specifications. When specified in a specification other than MPEG-DASH, the MPD does not provide any specific information on how to use the descriptor element. Instantiating the descriptor elements with the appropriate scheme information using the DASH format depends on the application or specification. An application or specification that uses one of these elements defines the scheme identifier in the form of a URI and defines the value space for the element when using that scheme identifier. The scheme identifier appears in the @schemeIdUri attribute. If a simple set of enumerated values is needed, a text string can be defined for each value, and that string can be included in the @value attribute. If structured data is needed, any extension elements or attributes can be defined in a separate namespace. The @id value can be used to reference a unique descriptor or a group of descriptors. In the latter case, it may be required that descriptors with equivalent @id values are synonymous, i.e., it is sufficient to process one of the descriptors with equivalent @id values. Two elements of type DescriptorType are equivalent if the element name, the value of the @schemeIdUri, and the value of the @value attribute are equivalent. If the @schemeIdUri is a URN, equivalence can mean lexical equivalence as defined in clause 5 of RFC 2141. If the @schemeIdUri is a URL, equivalence can mean character-by-character equality as defined in clause 6.2.1 of RFC 3986. If the @value attribute does not exist, equivalence can be determined solely by equivalence for the @schemeIdUri. Attributes and elements in the extension namespace may not be used to determine equivalence. The @id attribute can be ignored for equivalence determination.

[0118] MPEG-DASH specifies the descriptors EssentialProperty and SupplementalProperty. For the element EssentialProperty, the media presentation author states that successful processing of the descriptor is essential for the correct use of the information in the parent element containing this descriptor, unless the element shares the same @id with another EssentialProperty element. If EssentialProperty elements share the same @id, it is sufficient to process one of the EssentialProperty elements with the same @id value. It is expected that at least one EssentialProperty element for each different @id value will be processed. If no scheme or value of the EssentialProperty descriptor is recognized, it is expected that DASH clients will ignore the parent element containing the descriptor. Multiple EssentialProperty elements with the same @id value and different @id values can exist in the MPD.

[0119] For the element SupplementalProperty, the media presentation author states that the descriptor contains supplementary information that can be used by DASH clients for optimized processing. If no scheme or value of the SupplementalProperty descriptor is recognized, it is expected that DASH clients will ignore the descriptor. Multiple SupplementalProperty elements can exist in the MPD.

[0120] MPEG-DASH specifies a view point element that is formatted as a property descriptor. The @schemeIdUri attribute of the view point element is used to identify the view point scheme in use. Adaptation sets containing non-equivalent view point element values contain different media content components. The view point element can also be applied to media content types that are not video. Adaptation sets with equivalent view point element values are intended to be presented together. This processing should also apply to recognized and unrecognized @schemeIdUri values.

[0121] SRD (Spatial Relationship Description) is specified in normative appendix H of MPEG-DASH. The following contains some excerpts of the SRD specification. The SRD scheme allows the media presentation description author to state the spatial relationship between spatial objects. Spatial objects are represented by adaptation sets or sub-presentations. For example, the spatial relationship can state that one video represents a spatial part of another full-frame video (e.g., region of interest, or tile).

[0122] SupplementalProperty and / or EssentialProperty descriptors with @schemeIdUri equal to "urn:mpeg:dash:srd:2014" are used to provide spatial relationship information associated with the included spatial objects. The SRD shall be exclusively included in these two MPD elements (AdaptationSet and SubRepresentation).

[0123] The sub - representation level SRD can be used to represent spatial objects in a presentation, such as an HEVC tile stream. In this case, the SRD descriptor can be present in both the adaptation set and the sub - representation level.

[0124] The @value of the SupplementalProperty or EssentialProperty element using the SRD scheme is a comma - separated list of the values of the SRD parameters. The SRD parameters source_id, object_x, object_y, object_width, and object_height must be present, while the SRD parameters total_width, total_height, and spatial_set_id are conditionally or optionally present.

[0125] source_id is a non - negative integer in decimal representation that provides an identifier for the source of the content. The source_id parameter provides a unique identifier for the source of the content within a period. It implicitly defines the coordinate system associated with this source. The coordinate system uses an arbitrary origin (0;0); the direction of the x - axis is from left to right, and the direction of the y - axis is from top to bottom. All SRDs sharing the same source_id value have the same origin and axis orientation. The spatial relationship of spatial objects using SRDs with different source_id values is not defined.

[0126] For a given source_id value, a reference space is defined, which corresponds to the rectangular area containing the entire source content, with its upper - left corner at the origin of the coordinate system. The total_width and total_height values in the SRD provide the size of this reference space in arbitrary units. total_width is a non - negative integer in decimal representation that represents the width of this reference space in arbitrary units. total_height is a non - negative integer in decimal representation that represents the height of this reference space in arbitrary units. For example, when the entire source content consists of two separate videos, it is allowed that there are spatial objects in the MPD that do not cover the entire content source.

[0127] object_x is a non - negative integer in decimal representation, which represents the horizontal position of the upper - left corner of the spatial object in any unit. object_y is a non - negative integer in decimal representation, which represents the vertical position of the upper - left corner of the spatial object in any unit. object_width is a non - negative integer in decimal representation, which represents the width of the spatial object in any unit. object_height is a non - negative integer in decimal representation, which represents the height of the spatial object in any unit. The object_x and object_y parameters (object_width and object_height respectively) represent the 2D position (2D size respectively) of the associated spatial object in the coordinate system associated with the source. The values of the object_x, object_y, object_width, and object_height parameters are related to the values of the total_width and total_height parameters defined as above. The positions (object_x, object_y) and sizes (object_width, object_height) of SRDs sharing the same source_id value can be compared after considering the size of the reference space, that is, after dividing the object_x and object_width values by their respective descriptors' total_width values and dividing the object_y and object_height values by their respective descriptors' total_height values. Different total_width and total_height values can be used in different descriptors to provide position and size information in different units for the same reference space.

[0128] spatial_set_id is a non - negative integer in decimal representation, which provides an identifier for a group of spatial objects. When absent, the spatial objects associated with this descriptor do not belong to any spatial set and no spatial set information is given. MPD authors can use the spatial_set_id parameter to state that, within a given source_id, some spatial objects have a specific spatial relationship. For example, MPD authors can group all adaptation sets corresponding to tiles at the same resolution level. Thus, DASH clients can use the spatial_set_id parameter to quickly select spatially - related spatial objects.

[0129] An initialization segment can be defined as a segment containing metadata that is necessary to represent the media stream encapsulated in the media segment. In the fragment format based on ISOBMFF, the initialization segment can include a movie box ('moov'), which may not include metadata for any samples, i.e., any metadata for samples is provided in the'moof' box.

[0130] A media segment contains media data of a certain duration for playback at normal speed, and this duration is referred to as the media segment duration or segment duration. The content producer or service provider can select the segment duration according to the required service characteristics. For example, a relatively short segment duration can be used in a live service to achieve short end-to-end latency. The reason is that the segment duration is typically the lower bound of the end-to-end latency perceived by the DASH client, because segments are discrete units for which media data is generated for DASH. Content generation is usually done in such a way that the entire media data segment is available to the server. In addition, many client implementations use segments as the unit of GET requests. Therefore, in a typical setup for live services, the DASH client can request segments only when the entire media segment duration is available and encoded and encapsulated into segments. For on-demand services, different strategies for selecting the segment duration can be used.

[0131] For example, a segment can be further divided into sub-segments to enable downloading the segment in multiple parts. It is required that the sub-segments contain complete access units. The sub-segments can be indexed by a segment index box that contains information mapping the presentation time range and byte range for each sub-segment. The segment index box can also signal its duration and byte offset to describe the sub-segments and stream access points in the segment. The DASH client can use the information obtained from the segment index box to issue an HTTP GET request for a specific sub-segment using a byte range HTTP request. If a relatively long segment duration is used, sub-segments can be used to keep the size of the HTTP response reasonable and flexible for bitrate adaptation. The index information of the segment can be placed in a single box at the start of the segment, or scattered in multiple index boxes within the segment. Different scattering methods are possible, such as hierarchical, daisy-chain, and hybrid. This technique can avoid adding a very large box at the start of the segment and thus can prevent possible initial download latency.

[0132] A sub - representation is embedded within a regular representation and is described by a SubRepresentation element. The SubRepresentation element is contained within a Representation element. The SubRepresentation element describes the characteristics of one or more media content components embedded within the Representation. For example, it can describe the exact characteristics of an embedded audio component (e.g., such as codec, sample rate, etc.), the exact characteristics of an embedded subtitle (e.g., such as codec), or it can describe some embedded low - quality video layers (e.g., such as some lower frame rate or others). Sub - representations and representations share some common attributes and elements. If the @level attribute exists within the SubRepresentation element, the following applies.

[0133] A sub - representation provides the ability to access a low - quality version of the representation in which it is contained. In this case, the sub - representation allows, for example, extraction of the audio track in a multiplexed representation, or if a lower frame rate is provided, it can allow for efficient fast - forward or rewind operations.

[0134] Initialization segments and / or media segments and / or index segments shall provide sufficient information to enable easy access to the data via HTTP partial GET requests. Details regarding the provision of such information are defined by the media format being used.

[0135] When using ISOBMFF segments, the following applies:

[0136] ○ The initialization segment contains a level assignment box.

[0137] ○ For each sub - segment, there is a sub - segment index box ('ssix').

[0138] ○ The attribute @level specifies the level associated with the described sub - representation in the sub - segment index. Information in the representation, sub - representation, and level assignment ('leva') box contains information about the assignment of media data to levels.

[0139] ○ The media data shall have an order such that each level provides an enhancement compared to lower levels.

[0140] If the @level attribute does not exist, the SubRepresentation element is only used to provide a more detailed description of the media stream embedded within the Representation.

[0141] ISOBMFF includes a so-called level mechanism for specifying subsets of a file. Levels follow a dependency hierarchy such that samples mapped to level n can depend on any sample at level m (where m <= n) and do not depend on any sample at level p (where p > n). For example, levels can be specified according to sublayers in time (e.g., TemporalId in HEVC). Levels can be declared in a level assignment ('leva') box contained in a movie extension ('mvex') box. Levels cannot be specified for the initial movie. When the level assignment box is present, it applies to all movie fragments after the initial movie. For the context of the level assignment box, a fraction is defined as consisting of one or more movie fragment boxes and associated media data boxes, possibly including only the initial part of the last media data box. Within a fraction, the data for each level appears continuously. The data for each level within a fraction appears in increasing order of level value. All data within a fraction is assigned to levels. The level assignment box provides a mapping from features such as scalability layers or time sublayers to levels. Features can be specified by tracks, sub-tracks within a track, or sample groupings of a track. For example, time-level sample groupings can be used to indicate the mapping of images to time levels, where these time levels are equivalent to the time sublayers in HEVC. That is, an HEVC image with a certain TemporalId value can be mapped to a specific time level using a time-level sample grouping (and the same operation can be repeated for all TemporalId values). Further, the level assignment box can reference the time-level sample grouping in the indicated mapping to levels.

[0142] The sub - segment index box ('ssix') provides a mapping from levels (as specified by the level assignment box) to the byte ranges of the sub - segments being indexed. In other words, this box provides a compact index for how data in the sub - segments is sorted into partial sub - segments according to levels. It enables clients to easily access the data of partial sub - segments by downloading the data ranges in the sub - segments. When the sub - segment index box is present, each byte in the sub - segment is assigned to a level. If a range is not associated with any information in the level assignment, any level not included in the level assignment can be used. There are 0 or 1 sub - segment index boxes per fragment index box (which only indexes leaf sub - segments, i.e., only sub - segments but no fragment index). The sub - segment index box (if any) is the next box after the associated fragment index box. The sub - segment index box records the sub - segments indicated in the just - previous fragment index box. Each level can be assigned to exactly one partial sub - segment, i.e., the byte range for a level is contiguous. The levels of partial sub - segments are assigned by incrementing the number within the sub - segment, i.e., the samples of a partial sub - segment can depend on any samples of previous partial sub - segments in the same sub - segment, but not vice versa. For example, each partial sub - segment contains samples with equivalent temporal sub - layers, and the partial sub - segments occur in increasing temporal sub - layer order within the sub - segment. When accessing partial sub - segments in this way, the final media data box may be incomplete, that is, less data is accessed than the length indication in the media data box indicates exists. The length of the media data box may need to be adjusted, or padding may be used. The padding_flag in the level assignment box indicates whether this missing data can be replaced with "zeros". If not, the sample data for samples assigned to levels that were not accessed does not exist, and care should be taken.

[0143] MPEG - DASH defines fragment container formats for both ISOBMFF and MPEG - 2 transport streams. Other specifications may specify fragment formats based on other container formats. For example, a fragment format based on the Matroska container file format has been proposed and can be summarized as follows. When a Matroska file is hosted as a DASH fragment etc., the association of DASH units with Matroska units can be specified as follows. A sub - segment (of DASH) can be defined as one or more consecutive clusters of Matroska - encapsulated content. The initialization fragment of DASH may need to include the EBML header, the (Matroska's) segment header, the (Matroska's) segment information, and tracks, and may optionally include other level - 1 elements and padding. The fragment index of DASH can include the cue elements of Matroska.

[0144] OMAF defines MPEG-DASH elements for associating various DASH elements. A SupplementalProperty element with the @schemeIdUri attribute equal to "urn:mpeg:mpegI:omaf:2018:assoc" is called an association descriptor. One or more association descriptors can exist at the adaptation set level, presentation level, and preselection level. The association descriptors included within an adaptation set / presentation / preselection element indicate that the descriptor's parent element (i.e., the adaptation set / presentation / preselection element) is associated with one or more elements in the MPD indicated by an XPath query in the omaf2:Association element and the association types signaled by the omaf2:@associationKindList.

[0145] In an OMAF DASH MPD, a view point element with the @schemeIdUri attribute equal to "urn:mpeg:mpegI:omaf:2018:vwpt" is called a Viewpoint Information (VWPT) descriptor.

[0146] At most one VWPT descriptor can exist at the adaptation set level, and no VWPT descriptors should exist at any other level. When none of the adaptation sets in a media presentation contain a VWPT descriptor, the media presentation is inferred to contain only one view point.

[0147] @value specifies the view point ID of the view point. ViewPointInfo is a container element, and its child elements and attributes provide information about the view point. The ViewPointInfo@label attribute specifies a string that provides a human-readable label for the view point. The ViewPointInfo.Position attribute of this element specifies the position information of the view point.

[0148] GLTF

[0149] The GL Transmission Format (glTF) is an API-neutral runtime asset delivery format. glTF bridges the gap between 3D content creation tools and modern 3D applications by providing an efficient, extensible, and interoperable format for the transmission and loading of 3D content.

[0150] A glTF asset is a JSON file plus supporting external data. Specifically, a glTF asset is represented by the following:

[0151] · A JSON-formatted file (.gltf) that contains a complete scene description: node hierarchy, materials, cameras, and descriptor information for meshes, animations, and other constructs

[0152] · Binary files (.bin) containing geometry and animation data as well as other buffer-based data

[0153] · Image files for textures (.jpg,.png)

[0154] Assets defined in other formats (such as images) can be stored in external files referenced via URIs, stored in GLB containers in parallel, or directly embedded in JSON using data URIs.

[0155] glTF has been designed to allow for extensibility. While the initial basic specification supports a rich feature set, there are many opportunities for growth and improvement. glTF defines a mechanism that allows for the addition of both common and vendor-specific extensions.

[0156] Due to the user's movement within the scene, the distance between the user and the objects in the scene changes. Consequently, the required level of detail of the textures of the visible objects also fluctuates.

[0157] In computer graphics, texture mapping (mipmap) is used to improve rendering efficiency. These are pre-computed optimized sequences of images, each sequence being a progressively lower-resolution rendition of the same image. High-resolution mipmap images are used for high-density samples, such as for objects close to the camera. When an object appears further away, lower-resolution images are used. Mipmaps are generated from high-resolution textures that need to be present in the target system.

[0158] Regardless of the mipmap resolution used by the renderer at a given time, the input for mipmap generation may need to be at high resolution. This computational and memory transfer time from the CPU to the GPU is a waste of resources. It should be understood that the renderer does not have to retrieve and upload high-resolution texture data, as clearly this would make no difference. Traditionally, this approach would require all images to be provided to the application before rendering begins, meaning a significant amount of data would have to be stored, transferred, and processed.

[0159] A similar issue arises when the renderer has to be ready to change the user's view direction, where, due to head movement and view blur, even for objects close to the observer, low-resolution images may be sufficient.

[0160] Part of the texture adaptation can be introduced at the network level, allowing the application to adaptively stream texture data at different levels of detail based on the required rendering precision of the object. This would also enable other styles of adaptation, such as bitrate adaptation. The current scene description specifications (e.g., glTF) do not support this technology.

[0161] In addition, it would be beneficial if the scene could use textures that change their properties over time. An example is a texture with baked information (e.g., shadows, light, and reflections). Continuously updated timed textures require mipmap calculations to be performed for each frame. In this case, the waste of bandwidth and computing resources is even more severe. Additionally, there is no possibility of using timed media in current scene description specifications (e.g., glTF).

[0162] Furthermore, traditionally, texture data is combined into larger atlases to reduce CPU-GPU loading times. By introducing timed media, texture patches can be combined, for example, using ISOBMFF, into tracks that share the same timeline. This would allow baked effects to be synchronized at the MPEG system level without having to perform complex client-side timed data synchronization. Additionally, such atlases can be generated regionally, which can enable the scaling of texture object data based on the position of the object in the scene.

[0163] Problems solved by the examples described herein include:

[0164] · When less detail is sufficient for rendering, mipmap generation for objects far from the camera still requires high-resolution textures for input.

[0165] · Computer-generated scene delivery formats are self-contained, and all data including texture information needs to be presented before rendering begins. This limitation requires that all versions of the texture for the same object must be present, which increases the size of the interchange format.

[0166] · Computer-generated scene delivery formats only support image media that provide the texture attributes of the mesh, such as jpeg and png. This limitation does not allow pre-baked time-varying lighting information.

[0167] · Even when using other formats for storing textures (including timed formats), synchronization between textures is an important component. Especially when the textures contain pre-baked lighting information.

[0168] · Due to the increase in the amount of texture data and the limitations of video codec resolution, it is impractical to encapsulate all textures in one frame (i.e., an atlas).

[0169] · When time-dependent baked textures are required for the scene, making high-resolution texture data available for distant objects wastes bandwidth and computing resources per frame, thus increasing the waste of resources multiple times.

[0170] Synchronized Multimedia Integration Language (SMIL)

[0171] SMIL is a way to choreograph and synchronize media, which focuses on the 2D domain. It does not provide tools for adapting and synchronizing texture data for 3D objects.

[0172] OpenGEX

[0173] OpenGEX is designed to share 3D scene assets. It is conceptually similar to glTF, which means it also lacks the ability to handle aspects related to timed media or streaming.

[0174] To avoid the need to use high-resolution texture data as input for mipmap calculations, the renderer retrieves appropriate data based on the adaptation element input. The adaptation input can be determined based on the viewer's position in the scene and based on the viewer's previous actions (e.g., head movement).

[0175] To ensure data synchronization between multiple objects rendered in the scene, a single timed media with multiple tracks / adaptation sets and a common timeline is used to store the data.

[0176] To minimize the initial amount of data required at the start of rendering, the adaptation of data can be introduced at the network level, allowing the application to adaptively stream data at different levels of detail based on the required rendering accuracy of the object or region. This can also enable other styles of adaptation, such as bitrate or codec adaptation.

[0177] Allowing for alternatives for the data, the lowest quality can be provided as a local file, while any additional details can be provided by the network unit and do not have to be present at the start of rendering. The alternative also allows for handling a wider range of end devices, which can differ in the supported media formats.

[0178] Figure 1 A high-level block diagram showing the implementation of real-time texture adaptation is shown. Block 100 depicts a scene description format that contains a complete scene description: a node hierarchy, materials, cameras, and descriptor information for meshes, animations, and other constructs, which can include binary files containing geometric and animation data as well as other buffer-based data.

[0179] Block 100 is presented in more detail as block 200 in Figure 2 . Figure 2 It is a block diagram of a scene description. Block 200 contains all the relevant data mentioned above in block 201. According to the example described herein, the scene description 200 also includes timing media information, which can be in the form of local binary data (e.g., ISOBMFF) or in the form of a media description (e.g., DASH MPD), which can be retrieved over the network. To facilitate this method, new blocks 202 and 203 are added to the scene description entity. Block 203 provides a list of all the timing media that can be used in the scene description. Block 203 can contain alternatives for a given media in different file forms (e.g., the file contains data encoded using different encoding tools, or files in different formats). Block 203 can also contain files that can themselves have alternatives in the form of alternative tracks (e.g., in ISOBMFF) or alternative representations (e.g., in DASH MPD). Block 203 provides a function for distinguishing these alternatives.

[0180] Block 202 allows block 203 to be linked to the existing structure of block 201 in an extensible manner.

[0181] According to the example described herein, the scene description (100, 200) is provided to rendering (101) and adaptation / synchronization (103). Based on this description, rendering (101) initializes the scene, including creating data and command buffers (102). Rendering 101 uses these buffers to retrieve and render data. Since the scene description (100 / 200) provides adaptation and synchronization in an extensible manner, the renderer can start operating before / regardless of any interaction with adaptation (103).

[0182] According to the example described herein, rendering (101) interacts with adaptation / synchronization (103) and provides information based on which adaptation is performed. Examples of such adaptation information are the user's current position in the scene and the orientation / size of the view frustum. In another embodiment, the margin information and orientation of the view frustum can also be signaled as adaptation information. Such adaptation allows the renderer (101) to update the data and command buffers (102) based on object visibility.

[0183] According to the examples described herein, the adaptation / synchronization (103) is responsible for determining which version of the media should be decoded and placing it in the data buffer (102). The adaptation / synchronization (103) can use the local media (104) or the media that should be retrieved via the network (105). The adaptation / synchronization (103) is also responsible for synchronously decoding the data according to the information provided by the local media (104) (e.g., the composition time of the ISOBMFF) or the network media (105) (e.g., the timing information of the period in the DASH MPD) and placing it in the data buffer (102). The adaptation / synchronization (103) should also indicate to the renderer about the adaptation so that the command buffer (103) can be correctly updated for the rendering pass.

[0184] The scene description (100 / 200) can be based on the glTF specification. In this case, the media list (203) can be an extension of the glTF specification.

[0185] In another embodiment, the scene description (100 / 200) can be stored as an entity in an object-based file (e.g., a file conforming to ISOBMFF, such as an MP4 file), which can be addressable and retrievable via a specific media handler. Such data can be stored as metadata within the file (e.g., items within the'meta' box and / or items referenced by the'meta' box).

[0186] The media list (203) provides an array of media items. The media items contain an array of alternatives that represent the same data. The array can have one or more entries. For each alternative, one or more of the following items of information can be provided: uri, mimeType, and tracks information.

[0187] The item uri provides the absolute or relative URI of the media. If relative paths are provided, these paths are relative to the main file location (e.g., the location of the.gltf file in the directory tree structure). Alternatively, the media can be referenced by a bufferView index. In this case, the media is stored as part of a binary blob.

[0188] The item mimeType provides the MIME type of the media. The mimeType can include a 'codecs' sub-parameter, which allows for the explicit specification of the codecs included within the file format.

[0189] The item tracks is an array that provides a list of all the tracks within the media. Each item provides track access information.

[0190] In an embodiment, in the case of DASH and ISOBMFF files, the access information is in the form of a URL fragment. To address a specific adaptation set within a DASH manifest, an MPD anchor as defined in ISO / IEC 23009-1 can be used. To address a specific track within an mp4 file, a URL fragment as specified in ISO / IEC 14496-12 can be used.

[0191] In an embodiment, one or more of the following URL fragment schemes are specified and used for ISOBMFF:

[0192] -alternate_group = <alternate_group>, which identifies an alternative group with the value of alternate_group of the TrackHeaderBox

[0193] -track_group = <track_group_type.track_group_id>, which identifies a track group with the given four-character code of track_group_type and the value of track_group_id

[0194] -entity_group = <grouping_type.group_id>, which identifies an entity group with the given four-character code of grouping_type and the value of group_id

[0195] In an embodiment, an entry reference in the media list represents a set of alternatives for the same media data as that indicated in the referenced entry. For example, the #track URL fragment for an MPD references an adaptation set that may contain multiple presentations that are alternatives to each other. In another example, any of the above-specified URL fragment schemes for ISOBMFF can be used to indicate a track group.

[0196] In an embodiment, the access information is in the form of a URL query string.

[0197] In an embodiment, in the case of a DASH MPD or any other media presentation description using XML, the access information is in the form of an XPath query string, which resolves to one or more media alternatives. For example, a given XPath query string can resolve to an adaptation set within an MPD.

[0198] In an embodiment, instead of making the track information a separate parameter of the entry or in addition to making the track information a separate parameter of the entry, the track information is embedded in the uri. For example, if the track information is a URL fragment or a URL query string, it can be embedded in the uri.

[0199] The Media List (203) can be an extension according to the glTF specification. In the following example, the Media List (203) (MPEG_media) lists two media items. The first media item contains only the item within the alternatives, which is a DASH manifest containing one track. Even without any alternatives at the media item level, the DASH manifest can still have different presentations within the adaptation set. The second media item contains two items within the alternatives. The first item lists an mp4 file containing data compressed using the AVC codec, while the second item lists an mp4 file containing data compressed using the HEVC codec. Each item within the "alternatives" array must have the same number of track items within the "tracks" object. However, each track item can contain different information depending on the structure of the MP4 file. In another embodiment, a null track Id value (e.g., equal to 0 or null) or an empty track item (e.g., "") or a 32-bit maxInt value (e.g., 0xFFFF) can be provided to indicate non-existent alternatives, or the same track ID can be used for different alternative lists.

[0200]

[0201]

[0202]

[0203] The Media Link (202) can be an extension of the source element of the texture array according to the glTF specification.

[0204] The Media Link (202) provides the possibility to link texture objects to the media listed by the Media List (203) and their corresponding tracks.

[0205] In the following example, two texture items are listed. Each texture item uses the Media Link (202) (MPEG_video_texture). The first texture item is linked to source 1 listed by the Media List (203) and track 0. The second texture item is linked to the same source 1 listed by the Media List (203) but a different track, namely track 1.

[0206]

[0207]

[0208] The Media Link (202) and Media List (203) scheme can be as follows:

[0209]

[0210]

[0211]

[0212]

[0213]

[0214] Figure 3 An example device 300 that can be implemented in hardware and is configured to perform texture adaptation based on the examples described herein. The device 300 includes a processor 302 and at least one non-transitory memory 304 including computer program code 305, wherein the at least one memory 304 and the computer program code 305 are configured to, together with the at least one processor 302, cause the device to perform texture adaptation 306 (which may be a texture adaptation circuit) based on the examples described herein. The device 300 optionally includes a display 308, which can be used to display the adapted content during rendering. The device 300 optionally includes one or more network (NW) interface(s) (I / F) 310. The NW I / F 310 can be wired and / or wireless and communicate via any communication technology over the Internet / other networks. The NW I / F 310 can include one or more transmitters and one or more receivers. The N / WI / F 310 can include components known in the art, such as amplifiers, filters, frequency converters, (de)modulators, encoder / decoder circuits, and one or more antennas.

[0215] Figure 4 is an embodiment of the device 300. As Figure 4 shown, the device 300 includes a processor 302 and at least one non-transitory memory 304 including computer program code 305, wherein the at least one memory 304 and the computer program code 305 are configured to, together with the at least one processor 302, cause the device 300 to perform a texture adaptation circuit based on the examples described herein. The computer program code 305 includes rendering 101 and adaptation / synchronization 103 to implement the Figure 1 configuration and functionality shown. The rendering circuit 101 includes a data and command buffer 102 that receives an input 114 from the adaptation / synchronization 103. The adaptation / synchronization circuit 103 includes a local medium 104 and a network medium 105. A link 112 provides an interface between the rendering 101 and the adaptation / synchronization 103. In Figure 4 In the illustrated embodiment, device 300 includes a display 308 that can be used to display adapted content during rendering. Device 300 further includes one or more network (NW) interfaces (I / F) 310. The NW I / F 310 can be wired and / or wireless and communicate over the Internet / other networks via any communication technology. The NW I / F 310 can include one or more transmitters and one or more receivers. The N / W I / F 310 can include components known in the art, such as amplifiers, filters, frequency converters, (de)modulators, encoder / decoder circuits, and one or more antennas. A bus 312 provides a communication interface and connection interface between the various components of device 300. As Figure 4 shown, device 300 receives a scene description 100 as input and provides a scene rendering 110 of the scene as output.

[0216] Figure 5 is an example method 400 for implementing texture adaptation based on the examples described herein. At 402, the method includes: receiving a scene description that includes data associated with a scene. At 404, the method includes: initializing the scene based on the scene description, wherein initializing the scene includes placing data associated with the scene into a data buffer and creating a command buffer. At 406, the method includes: adapting the data placed in the data buffer, synchronizing the data in the data buffer with information provided from local media or network media, and signaling information about the adaptation used to update the command buffer of the command renderer. At 408, the method includes: rendering the scene using the data in the data buffer and the command buffer.

[0217] Figure 6 is another example method 500 for implementing texture adaptation based on the examples described herein. At 502, the method includes: receiving a scene description that includes data associated with a scene. At 504, the method includes: placing data associated with the scene into a data buffer and creating a command buffer. At 506, the method includes: adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from local media or network media. At 508, the method includes: signaling information about the adaptation to update the command buffer of the command renderer. At 510, the method includes: rendering the scene using the data in the data buffer and the command buffer.

[0218] References to "computers", "processors", etc. should be understood to cover computers with different architectures such as single / multi-processor architectures and serial (von Neumann) / parallel architectures, as well as dedicated circuits such as field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), signal processing devices, and other processing circuits. References to computer programs, instructions, code, etc. should be understood to cover software for programmable processors, or firmware which may include instructions for a processor, such as the programmable content of a hardware device, or configuration settings for fixed function devices, gate arrays, or programmable logic devices, etc.

[0219] The memory as described herein may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The memory may include a database for storing data.

[0220] As used herein, the term "circuit" may refer to the following: (a) a hardware circuit implementation (such as an implementation of analog and / or digital circuits); (b) a combination of a circuit and software (and / or firmware), such as (if applicable): (i) a combination of processors; or (ii) portions of a processor / software, including a digital signal processor, software, and memory, which work together to enable a device to perform various functions; and (c) a circuit, such as a microprocessor or a portion of a microprocessor, which requires software or firmware to operate, even if the software or firmware is not physically present. As a further example, as used herein, the term "circuit" also covers an implementation of only a processor (or processors) or a portion of a processor and its accompanying software and / or firmware. The term "circuit" also covers (for example and if applicable to a particular claimed element) a baseband integrated circuit or an application processor integrated circuit for a mobile phone, or a similar integrated circuit in a server, a cellular network device, or another network device.

[0221] An example device includes at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the device to at least perform: receiving a scene description, the scene description including data associated with the scene; initializing the scene based on the scene description, wherein the initialization of the scene includes placing the data associated with the scene into a data buffer and creating a command buffer; adapting the data placed in the data buffer, synchronizing the data in the data buffer with information provided from a local or network medium, and signaling information about the adaptation used to update the command buffer of a command renderer; and rendering the scene using the data in the data buffer and the command buffer.

[0222] The apparatus may further comprise: wherein, the scene description further comprises: a media list of timed media, which comprises an array of data that represents alternative data to the data in the data buffer; and a media link for linking the media list to the data associated with the scene.

[0223] The apparatus may further comprise: wherein, the media link is an extension of a source element of a texture array based on a graphics library transmission format, and wherein, the media link enables linking of texture objects to the media listed in the media list of the timed media and their corresponding tracks.

[0224] The apparatus may further comprise: wherein, the access information associated with a track takes the form of a uniform resource locator fragment.

[0225] The apparatus may further comprise: wherein, the scheme associated with the uniform resource locator fragment includes an identification of at least one of an alternative group, a track group, or an entity group.

[0226] The apparatus may further comprise: wherein, the access information associated with a track takes the form of a uniform resource locator query string.

[0227] The apparatus may further comprise: wherein, the access information associated with a track takes the form of a query string that resolves to one or more media alternatives.

[0228] The apparatus may further comprise: wherein, instead of making the information associated with a track as a separate parameter of an entry or in addition to making the information associated with a track as a separate parameter of an entry, the information is embedded in a uniform resource identifier.

[0229] The apparatus may further comprise: wherein, an entry in the media list refers to a set of alternatives that represent the same media data as the media data indicated in the referenced entry.

[0230] The apparatus may further comprise: wherein, the scene description is based on a graphics library transmission format, and the timed media list is an extension of the graphics library transmission format specification.

[0231] The apparatus may further comprise: wherein, the at least one memory and the computer program code are configured to, together with at least one processor, cause the apparatus to at least perform: providing at least one of a uniform resource identifier, a Multipurpose Internet Mail Extensions type, or tracking information for each alternative.

[0232] The apparatus may further comprise: wherein, the information provided from local media or network media includes at least one of the following: the user's current position in the scene; or, the orientation, size, or margins of a view frustum.

[0233] The apparatus may further include: wherein the local media is the composition time of a base media file format; and the network media is timing information of a period in dynamic adaptive streaming based on a Hypertext Transfer Protocol Media Presentation Description.

[0234] The apparatus may further include: wherein the scene description is stored as an entity in an object-based file, and the object-based file is addressable and retrievable by a media handler.

[0235] An example method may include: receiving a scene description that includes data associated with a scene; initializing the scene based on the scene description, wherein initializing the scene includes placing data associated with the scene into a data buffer and creating a command buffer; adapting the data placed in the data buffer, synchronizing the data in the data buffer with information provided from local media or network media, and signaling information about the adaptation used to update the command buffer of a command renderer; and rendering the scene using the data in the data buffer and the command buffer.

[0236] The method may further include: wherein the scene description further includes: a media list of timed media, which includes an array of data representing alternatives to the data in the data buffer; and a media link for linking the media list to the data associated with the scene.

[0237] The method may further include: wherein the media link is an extension of a source element of a texture array based on a Graphics Library Transfer Format, and wherein the media link enables linking of texture objects to media listed in the media list of the timed media and their corresponding tracks.

[0238] The method may further include: wherein the access information associated with a track is in the form of a Uniform Resource Locator fragment.

[0239] The method may further include: wherein the scheme associated with the Uniform Resource Locator fragment includes an identification of at least one of an alternative group, a track group, or an entity group.

[0240] The method may further include: wherein the access information associated with a track is in the form of a Uniform Resource Locator query string.

[0241] The method may further include: wherein the access information associated with a track is in the form of a query string parsed into one or more media alternatives.

[0242] The method may further comprise: wherein, instead of or in addition to making information associated with an orbit as a separate parameter of an entry, the information is embedded in a uniform resource identifier.

[0243] The method may further comprise: wherein, an entry reference in the media list refers to a set of alternatives that represent the same media data as the media data indicated in the referenced entry.

[0244] The method may further comprise: wherein, the scene description is based on a graphics library transfer format, and the timed media list is an extension of the graphics library transfer format specification.

[0245] The method may further comprise: providing at least one of a uniform resource identifier, a Multipurpose Internet Mail Extensions type, or tracking information for each alternative.

[0246] The method may further comprise: wherein, the information provided from local media or network media includes at least one of the following: the user's current position in the scene; or, the orientation, size, or margin of a view frustum.

[0247] The method may further comprise: wherein, the local media is the composition time of a base media file format; the network media is the timing information of a period in dynamic adaptive streaming based on a Hypertext Transfer Protocol media presentation description.

[0248] The method may further comprise: wherein, the scene description is stored as an entity in an object-based file, and the object-based file can be addressed and retrieved by a media handler.

[0249] An example apparatus includes: components for receiving a scene description, wherein the scene description includes data associated with a scene; components for initializing the scene based on the scene description, wherein the initialization of the scene includes placing the data associated with the scene into a data buffer and creating a command buffer; components for adapting the data placed in the data buffer, synchronizing the data in the data buffer with information provided from local media or network media, and signaling information about the adaptation used to update the command buffer of a command renderer; and components for rendering the scene using the data in the data buffer and the command buffer.

[0250] A machine-readable non-transitory program storage device can be provided that tangibly embodies a machine-executable instruction program for performing operations including: receiving a scene description that includes data associated with the scene; initializing the scene based on the scene description, where the initialization of the scene includes placing the data associated with the scene into a data buffer and creating a command buffer; adapting the data placed in the data buffer, synchronizing the data in the data buffer with information provided from a local or network medium, and signaling information about the adaptation used to update the command buffer of the command renderer; and rendering the scene using the data in the data buffer and the command buffer.

[0251] Example apparatus includes: components for receiving a scene description, where the scene description includes data associated with the scene; components for placing the data associated with the scene into a data buffer and creating a command buffer; components for adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from a local or network medium; components for signaling information about the adaptation to update the command buffer of the command renderer; and components for rendering the scene using the data in the data buffer and the command buffer.

[0252] Other aspects of the apparatus may include the following. The scene description may further include: a media list for timed media, which includes an array of alternative data representing data within a data buffer; and a media link for linking the media list to data associated with the scene. The media link may be an extension of a texture format; the media link may be configured to link a graphics library transfer format texture object to media listed in the media list of the timed media and their corresponding tracks. The access information associated with the corresponding track may be in the form of a uniform resource locator fragment. The uniform resource locator fragment associated with the corresponding track may include at least one of DASH media or MP4 media. The access information associated with the track may be in the form of a uniform resource locator query string. The access information associated with the track may be in the form of a query string parsed into one or more media alternatives. Instead of making the information associated with the track as a separate parameter of an entry or in addition to making the information associated with the track as a separate parameter of an entry, the information may be embedded in a uniform resource identifier. An entry in the media list may reference one or more alternatives of the same media data as the media data indicated in the referenced entry. The scene description may be based on a graphics library transfer format, and the timed media list may be an extension of a graphics library transfer format specification. The apparatus may further include: components for providing at least one of a uniform resource identifier, a multipurpose internet mail extension type, or tracking information for each alternative. Information provided from local media or network media may include at least one of the following: translation of a scene node; rotation of a scene node; or intrinsic camera parameters of a camera object. Local media may be a media file format; network media may be dynamic adaptive streaming over hypertext transfer protocol manifest. The scene description may be stored as an entity in an object-based file, which can be addressed and retrieved by a media handler. The media list of the timed media may be an array of MPEG media extensions; the data within the data buffer may be a gltf accessor; and the media link may be at least one of an MPEG texture video extension, an MPEG viewport recommendation extension, or an MPEG animation extension. One or more alternatives of the same media may be an array.

[0253] The example apparatus includes at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least perform: receiving a scene description that includes data associated with a scene; placing the data associated with the scene into a data buffer and creating a command buffer; adapting the data placed within the data buffer and synchronizing the data within the data buffer with information provided from a local medium or a network medium; signaling information regarding the adaptation to update the command buffer of a command renderer; and rendering the scene using the data within the data buffer and the command buffer.

[0254] Other aspects of the apparatus may include the following. The scene description may further include: a media list of timed media, which includes an array of alternative data representing data within a data buffer; and a media link for linking the media list to data associated with the scene. The media link may be an extension of a texture format; the media link may be configured to link a graphics library transfer format texture object to media listed in the media list of the timed media and their corresponding tracks. The access information associated with the corresponding track may be in the form of a uniform resource locator fragment. The uniform resource locator fragment associated with the corresponding track may include at least one of DASH media or MP4 media. The access information associated with the track may be in the form of a uniform resource locator query string. The access information associated with the track may be in the form of a query string parsed into one or more media alternatives. Instead of or in addition to having the information associated with the track as a separate parameter of an entry, the information may be embedded in a uniform resource identifier. An entry in the media list may reference one or more alternatives of the same media data as the media data indicated in the referenced entry. The scene description may be based on a graphics library transfer format, and the timed media list may be an extension of a graphics library transfer format specification. The at least one memory and the computer program code may be further configured to, with the at least one processor, cause the apparatus to at least perform: providing at least one of a uniform resource identifier, a multipurpose internet mail extensions type, or tracking information for each alternative. Information provided from local media or network media may include at least one of the following: translation of a scene node; rotation of a scene node; or intrinsic camera parameters of a camera object. Local media may be a media file format; network media may be dynamic adaptive streaming over hypertext transfer protocol manifest. The scene description may be stored as an entity in an object-based file, which is addressable and retrievable by a media handler. The media list of the timed media may be an MPEG media extensions array; the data within the data buffer may be a gltf accessor; and the media link may be at least one of an MPEG texture video extension, an MPEG viewport recommendation extension, or an MPEG animation extension. One or more alternatives of the same media may be an array.

[0255] Example methods include: receiving a scene description that includes data associated with a scene; placing the data associated with the scene in a data buffer and creating a command buffer; adapting the data placed in the data buffer and synchronizing the data within the data buffer with information provided from local media or network media; signaling information about the adaptation to update the command buffer of a command renderer; and rendering the scene using the data within the data buffer and the command buffer.

[0256] Other aspects of the method may include the following. The scene description may further include: a media list of timed media, which includes an array of alternative data representing data in a data buffer; and a media link for linking the media list to data associated with the scene. The media link may be an extension of the texture format; the media link may be configured to link a graphics library transfer format texture object to the media listed in the media list of the timed media and their corresponding tracks. The access information associated with the corresponding track may be in the form of a uniform resource locator fragment. The uniform resource locator fragment associated with the corresponding track may include at least one of DASH media or MP4 media. The access information associated with the track may be in the form of a uniform resource locator query string. The access information associated with the track may be in the form of a query string parsed into one or more media alternatives. Instead of making the information associated with the track as a separate parameter of an entry or in addition to making the information associated with the track as a separate parameter of an entry, the information may be embedded in a uniform resource identifier. An entry in the media list may reference one or more alternatives of the same media data as the media data indicated in the referenced entry. The scene description may be based on a graphics library transfer format, and the timed media list may be an extension of the graphics library transfer format specification. The method may further include: providing at least one of a uniform resource identifier, a Multipurpose Internet Mail Extensions type, or tracking information for each alternative. Information provided from local media or network media may include at least one of the following: translation of a scene node; rotation of a scene node; or intrinsic camera parameters of a camera object. Local media may be a media file format; network media may be dynamic adaptive streaming over HTTP manifest. The scene description may be stored as an entity in an object-based file, which can be addressed and retrieved by a media handler. The media list of the timed media may be an MPEG media extensions array; the data in the data buffer may be a gltf accessor; and the media link may be at least one of an MPEG texture video extension, an MPEG viewport recommendation extension, or an MPEG animation extension. One or more alternatives of the same media may be an array.

[0257] A machine-readable non-transitory program storage device is provided that tangibly embodies a machine-executable instruction program for performing operations, the operations including: receiving a scene description that includes data associated with the scene; placing the data associated with the scene into a data buffer and creating a command buffer; adapting the data placed within the data buffer and synchronizing the data within the data buffer with information provided from a local media or a network media; signaling information regarding the adaptation to update the command buffer of a command renderer; and rendering the scene using the data within the data buffer and the command buffer.

[0258] Other aspects of the non - transitory program storage device may include the following. The scene description may further include: a media list for timed media, which includes an array of alternative data representing data within a data buffer; and a media link for linking the media list to data associated with the scene. The media link may be an extension of a texture format; the media link may be configured to link a graphics library transfer format texture object to media listed in the media list of the timed media and their corresponding tracks. The access information associated with the corresponding track may be in the form of a uniform resource locator fragment. The uniform resource locator fragment associated with the corresponding track may include at least one of DASH media or MP4 media. The access information associated with a track may be in the form of a uniform resource locator query string. The access information associated with a track may be in the form of a query string that resolves to one or more media alternatives. Instead of or in addition to making the information associated with a track a separate parameter of an entry, the information may be embedded in a uniform resource identifier. An entry in the media list may reference one or more alternatives of the same media data as the media data indicated in the referenced entry. The scene description may be based on a graphics library transfer format, and the timed media list may be an extension of a graphics library transfer format specification. Operations of the non - transitory program storage device may further include: providing at least one of a uniform resource identifier, a Multipurpose Internet Mail Extensions type, or tracking information for each alternative. Information provided from local media or network media may include at least one of the following: translation of a scene node; rotation of a scene node; or intrinsic camera parameters of a camera object. Local media may be a media file format; network media may be dynamic adaptive streaming over Hypertext Transfer Protocol manifest. The scene description may be stored as an entity in an object - based file, which can be addressed and retrieved by a media handler. The media list of the timed media may be an MPEG media extensions array; the data within the data buffer may be a gltf accessor; and the media link may be at least one of an MPEG texture video extension, an MPEG viewport recommendation extension, or an MPEG animation extension. One or more alternatives of the same media may be an array.

[0259] It should be understood that the foregoing description is merely illustrative. Those skilled in the art can design various alternatives and modifications. For example, the features described in the various dependent claims can be combined with each other in any suitable combination. Additionally, features from different embodiments described above can be selectively combined into new embodiments. Therefore, the above description is intended to cover all such alternatives, modifications, and variations that fall within the scope of the appended claims.< / maxoccurs> < / minoccurs>

Claims

1. An apparatus, comprising: a component for receiving a scene description, wherein the scene description includes data associated with the scene; a component for placing the data associated with the scene into a data buffer and creating a command buffer, wherein the command buffer is configured to command a renderer; a component for adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from a local media or a network media; a component for signaling information about the adaptation to update the command buffer; and a component for rendering the scene using the data in the data buffer and the command buffer.

2. The device according to claim 1, wherein The scene description further includes: a media list of timed media, which includes an array of data that represents alternatives to the data in the data buffer; and a media link for linking the media list to the data associated with the scene.

3. The apparatus according to claim 2, wherein: the media link is an extension of a texture format; and the media link is configured to link a graphics library transfer format texture object to the media listed in the media list of the timed media and their corresponding tracks.

4. The device according to claim 3, wherein The access information associated with the corresponding track is in the form of a uniform resource locator fragment.

5. The device according to claim 4, wherein, The uniform resource locator fragment associated with the corresponding track includes at least one of DASH media or MP4 media.

6. The device according to any one of claims 3 to 5, wherein The access information associated with the track is in the form of a uniform resource locator query string.

7. The apparatus according to any one of claims 3 to 5, wherein, The access information associated with the track is in the form of a query string that resolves to one or more media alternatives.

8. The device according to any one of claims 3 to 5, wherein Instead of making the information associated with the track as a separate parameter of an entry or in addition to making the information associated with the track as a separate parameter of an entry, the information is embedded in a uniform resource identifier.

9. The device according to any one of claims 2 to 5, wherein Entries in the media list reference one or more alternatives of the same media data as the media data indicated in the referenced entries.

10. The device according to any one of claims 2 to 5, wherein, The scene description is based on a graphics library transfer format, and the timed media list is an extension of the graphics library transfer format specification.

11. The apparatus according to any one of claims 2 to 5, further comprising: a component for providing at least one of a uniform resource identifier, a Multipurpose Internet Mail Extensions type, or tracking information for each alternative.

12. The device according to any one of claims 1 to 5, wherein, The information provided from the local media or the network media includes at least one of the following: translation of a scene node; rotation of a scene node; or intrinsic camera parameters of a camera object.

13. The apparatus according to any one of claims 1 to 5, wherein: the local media is a media file format; and the network media is dynamic adaptive streaming over HTTP manifest.

14. The device according to any one of claims 1 to 5, wherein The scene description is stored as an entity in an object-based file, and the object-based file can be addressed and retrieved by a media handler.

15. The apparatus according to any one of claims 2 to 5, wherein: the media list of the timed media is an MPEG media extensions array; the data in the data buffer is a gltf accessor; and The media link is at least one of an MPEG texture video extension, an MPEG viewport recommendation extension, or an MPEG animation extension.

16. The apparatus according to claim 9, wherein, The one or more alternatives of the same media are an array.

17. An apparatus, comprising: at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to at least perform: receive a scene description, the scene description including data associated with a scene; place the data associated with the scene into a data buffer and create a command buffer, the command buffer being configured to command a renderer; adapt the data placed in the data buffer and synchronize the data in the data buffer with information provided from local media or network media; signal information about the adaptation to update the command buffer; and render the scene using the data in the data buffer and the command buffer.

18. The apparatus according to claim 17, wherein The scene description further includes: a media list of timed media, which includes an array of data representing alternatives to the data in the data buffer; and a media link for linking the media list to the data associated with the scene.

19. The apparatus according to claim 18, wherein: the media link is an extension of a texture format; and the media link is configured to link a graphics library transport format texture object to the media listed in the media list of the timed media and their corresponding tracks.

20. The apparatus according to claim 19, wherein The access information associated with the corresponding track is in the form of a uniform resource locator fragment.

21. The device according to claim 20, wherein The uniform resource locator fragment associated with the corresponding track includes at least one of DASH media or MP4 media.

22. The apparatus according to any one of claims 19 to 21, wherein The access information associated with a track is in the form of a uniform resource locator query string.

23. The apparatus according to any one of claims 19 to 21, wherein, The access information associated with a track is in the form of a query string parsed into one or more media alternatives.

24. The device according to any one of claims 19 to 21, wherein Instead of or in addition to making the information associated with a track as a separate parameter of an entry, the information is embedded in a uniform resource identifier.

25. The device according to any one of claims 18 to 21, wherein, Entries in the media list refer to one or more alternatives of the same media data as the media data indicated in the referenced entries.

26. The apparatus according to any one of claims 18 to 21, wherein, The scene description is based on a graphics library transport format, and the timed media list is an extension of the graphics library transport format specification.

27. The apparatus according to any one of claims 18 to 21, wherein The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to at least perform: provide at least one of a uniform resource identifier, a Multipurpose Internet Mail Extensions type, or tracking information for each alternative.

28. The device according to any one of claims 17 to 21, wherein The information provided from the local media or the network media includes at least one of the following: translation of a scene node; rotation of a scene node; or intrinsic camera parameters of a camera object.

29. The apparatus according to any one of claims 17 to 21, wherein: the local media is a media file format; and The network media is dynamic adaptive streaming over HTTP manifest.

30. The device according to any one of claims 17 to 21, wherein The scene description is stored as an entity in an object-based file, which can be addressed and retrieved by a media handler.

31. The apparatus according to any one of claims 18 to 21, wherein: The media list of the timed media is an MPEG media extensions array; The data in the data buffer is a glTF accessor; and The media link is at least one of an MPEG texture video extension, an MPEG viewport recommendation extension, or an MPEG animation extension.

32. The apparatus according to claim 25, wherein, The one or more alternatives of the same media are an array.

33. A method, comprising: Receiving a scene description, the scene description including data associated with the scene; Placing the data associated with the scene into a data buffer and creating a command buffer configured to command a renderer; Adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from local media or network media; Signaling information about the adaptation to update the command buffer; And Rendering the scene using the data in the data buffer and the command buffer.

34. The method according to claim 33, wherein The scene description further includes: A media list of timed media, which includes an array of data representing alternatives of the data in the data buffer; and A media link for linking the media list to the data associated with the scene.

35. The method according to claim 34, wherein: The media link is an extension of a texture format; and The media link is configured to link a graphics library transmission format texture object to the media listed in the media list of the timed media and their corresponding tracks.

36. The method according to claim 35, wherein, The access information associated with the corresponding track is in the form of a uniform resource locator fragment.

37. The method according to claim 36, wherein, The uniform resource locator fragment associated with the corresponding track includes at least one of DASH media or MP4 media.

38. The method according to any one of claims 35 to 37, wherein The access information associated with the track is in the form of a uniform resource locator query string.

39. The method according to any one of claims 35 to 37, wherein The access information associated with the track is in the form of a query string that resolves to one or more media alternatives.

40. The method according to any one of claims 35 to 37, wherein, Instead of or in addition to making the information associated with the track as a separate parameter of an entry, the information is embedded in a uniform resource identifier.

41. The method according to any one of claims 34 to 37, wherein Entries in the media list refer to one or more alternatives of the same media data as the media data indicated in the referenced entries.

42. The method according to any one of claims 34 to 37, wherein, The scene description is based on a graphics library transmission format, and the timed media list is an extension of the graphics library transmission format specification.

43. The method according to any one of claims 34 to 37, further comprising: Providing at least one of a uniform resource identifier, a Multipurpose Internet Mail Extensions type, or tracking information for each alternative.

44. The method according to any one of claims 33 to 37, wherein, The information provided from the local media or the network media includes at least one of the following: Translation of a scene node; Rotation of a scene node; or Intrinsic camera parameters of a camera object.

45. The method according to any one of claims 33 to 37, wherein: the local media is a media file format; and the network media is dynamic adaptive streaming over HTTP (DASH).

46. The method according to any one of claims 33 to 37, wherein, The scene description is stored as an entity in an object-based file, which can be addressed and retrieved by a media handler.

47. The method according to any one of claims 34 to 37, wherein: the media list of the timed media is an MPEG media extensions array; the array data in the data buffer is a glTF accessor; and the media link is at least one of an MPEG texture video extension, an MPEG viewport recommendation extension, or an MPEG animation extension.

48. The method according to claim 41, wherein The one or more alternatives of the same media are arrays.

49. A machine-readable non-transitory program storage device tangibly embodying a program of machine-executable instructions for performing operations, the operations including: receiving a scene description including data associated with a scene; placing the data associated with the scene into a data buffer and creating a command buffer configured to command a renderer; adapting the data placed in the data buffer and synchronizing the data in the data buffer with information provided from local media or network media; signaling information about the adaptation to update the command buffer; and using the data in the data buffer and the command buffer to render the scene.

50. The non-transitory program storage device according to claim 49, wherein, The scene description further includes: a media list of timed media including an array of data representing alternatives to data in the data buffer; and a media link for linking the media list to the data associated with the scene.

51. The non-transitory program storage device according to claim 50, wherein: the media link is an extension of a texture format; and the media link is configured to link a graphics library transmission format texture object to media listed in the media list of the timed media and their corresponding tracks.

52. The non-transitory program storage device according to claim 51, wherein, The access information associated with the corresponding track takes the form of a uniform resource locator (URL) fragment.

53. The non-transitory program storage device according to claim 52, wherein, The URL fragment associated with the corresponding track includes at least one of DASH media or MP4 media.

54. The non-transitory program storage device according to any one of claims 51 to 53, wherein, The access information associated with a track takes the form of a URL query string.

55. The non-transitory program storage device according to any one of claims 51 to 53, wherein, The access information associated with a track takes the form of a query string that resolves to one or more media alternatives.

56. The non-transitory program storage device according to any one of claims 51 to 53, wherein, Instead of or in addition to making the information associated with a track as a separate parameter of an entry, the information is embedded in a uniform resource identifier.

57. The non-transitory program storage device according to any one of claims 50 to 53, wherein, An entry in the media list references one or more alternatives of the same media data as the media data indicated in the referenced entry.

58. The non-transitory program storage device according to any one of claims 50 to 53, wherein, The scene description is based on a graphics library transmission format, and the timed media list is an extension of the graphics library transmission format specification.

59. The non-transitory program storage device according to any one of claims 50 to 53, wherein the operation further comprises: providing at least one of a uniform resource identifier, a Multipurpose Internet Mail Extensions type, or tracking information for each alternative.

60. The non-transitory program storage device according to any one of claims 49 to 53, wherein, The information provided from the local media or the network media includes at least one of the following: translation of a scene node; rotation of a scene node; or intrinsic camera parameters of a camera object.

61. The non-transitory program storage device according to any one of claims 49 to 53, wherein: the local media is a media file format; and the network media is Dynamic Adaptive Streaming over HTTP (DASH).

62. The non-transitory program storage device according to any one of claims 49 to 53, wherein, The scene description is stored as an entity in an object-based file, and the object-based file can be addressed and retrieved by a media handler.

63. The non-transitory program storage device according to any one of claims 50 to 53, wherein: the media list of the timed media is an MPEG Media Extensions array; the data in the data buffer is a glTF accessor; and the media link is at least one of an MPEG Texture Video Extension, an MPEG Viewport Recommendation Extension, or an MPEG Animation Extension.

64. The non-transitory program storage device according to claim 57, wherein, the one or more alternatives of the same media are an array.

Images