Multi-track based immersive media playout

Abstract

A method, apparatus, and system for dynamically showing playout sets in various scenarios are disclosed. In one example aspect, the method includes determining a playout set corresponding to a volumetric video in a three-dimensional space based on playout set information, the playout set including a plurality of components, each component of a type having one or more versions, obtaining two-dimensional volumetric video content from a media track corresponding to the playout set, and reconstructing the volumetric video in the three-dimensional space based on the plurality of components with different versions.

Description

[Technical field]

[0001] This patent document relates generally to immersive technology. [Background technology]

[0002] With the recent development of human-computer interaction and artificial intelligence technology, immersive technology is revolutionizing the way we work and live. Immersive technology is applied in several fields, including marketing and advertising, health care, education, and gaming. The rapid development of new immersive technologies requires effective processing of video and image data. Summary of the Invention [Means for solving the problem]

[0003] This patent document describes, among other things, techniques for dynamically indicating playout sets in various scenarios.

[0004] In one aspect, a method of processing media content is disclosed, the method including: determining a playout set corresponding to a volumetric video in a three-dimensional space based on playout set information, the playout set including a plurality of components, each component of a type having one or more versions, obtaining two-dimensional volumetric video content from a media track corresponding to the playout set, and reconstructing the volumetric video in the three-dimensional space based on the plurality of components with different versions.

[0005] In another aspect, a method of processing media content is disclosed that includes configuring, by a server, a playout set including one or more types of components, where each component of a type has one or more versions, configuring, by the server, playout set information corresponding to the playout set, and transmitting, by the server, the playout set information in a media presentation description to a user device.

[0006] In another exemplary aspect, a wireless communications apparatus is disclosed, comprising a processor configured to implement the method described above.

[0007] In another exemplary aspect, a computer storage medium storing code for implementing the methods described above is disclosed.

[0008] These and other aspects are described herein. The present invention further provides, for example, the following: (Item 1) 1. A method for processing media content, the method comprising: determining a playout set corresponding to a volumetric video in a three-dimensional space based on the playout set information, the playout set including a plurality of components, each component of a type having one or more versions; obtaining two-dimensional volumetric video content from a media track corresponding to the playout set; reconstructing the volumetric video in a three-dimensional space based on the components with different versions; A method comprising: (Item 2) 2. The method of claim 1, wherein the component includes a visual volumetric video-based coding (V3C) component. (Item 3) 3. The method of claim 2, wherein the V3C component includes at least one of an occupancy component, a geometry component, or an attribute component. (Item 4) 4. The method of claim 3, wherein each atlas or atlas style corresponds to an attribute component, an occupancy component, and / or a geometric shape component with multiple versions. (Item 5) 4. The method of claim 3, wherein the different versions of the occupancy component, the different versions of the geometric shape component, and the different versions of the attribute component each have a dependency relationship between each other. (Item 6) 2. The method of claim 1, wherein the playout set includes V3C components from a set of multiple alternative tracks. (Item 7) 2. The method of claim 1, wherein the different versions include at least one of a different encoding method, a different bit rate, or a different resolution. (Item 8) 2. The method of claim 1, wherein the components in the playout set correspond to one or more atlases or atlas tiles. (Item 9) 2. The method of claim 1, wherein the playout group is configured based on a time-specified metadata track for configuring dynamic playout grouping information in a sample entry and / or one or more samples belonging to the sample entry. (Item 10) 10. The method of claim 9, wherein the time-specific metadata tracks are used to configure the playout set at each sample time. (Item 11) 10. The method of claim 9, wherein the sample entries include configuration information for configuring an initial playout set. (Item 12) Item 12. The method of item 11, further comprising generating a flag to indicate a change in the playout set. (Item 13) Item 12. The method of item 11, further comprising generating a flag to indicate one or more new playout sets. (Item 14) 2. The method of claim 1, wherein the playout set is configured based on a sample set including samples in video encoding component data included in a track or an atlas or atlas tile included in the track. (Item 15) 15. The method of claim 14, wherein the set of samples is configured to be used during a sampling period and updated based on the sampling period. (Item 16) 16. The method according to any of items 1-15, wherein the playout group information is described by a syntax of a playout group structure. (Item 17) 1. A method for processing media content, the method comprising: configuring, by a server, a playout set including one or more types of components, each component of a type having one or more versions; configuring, by the server, playout group information corresponding to the playout group; transmitting, by the server, to a user device, the playout set information in a media presentation description; A method comprising: (Item 18) Item 18. The method of item 17, wherein the component includes a visual volumetric video-based coding (V3C) component. (Item 19) 20. The method of claim 18, wherein the V3C components include at least one of an occupancy component, a geometry component, or an attribute component. (Item 20) 20. The method of claim 19, wherein each of the V3C components is represented as a separate conformance set in the media presentation description. (Item 21) 20. The method of claim 17, further comprising adding at least one of a playout group identifier or an alternative group identifier corresponding to the playout group to the media presentation description to indicate the playout group information. (Item 22) 22. The method of claim 21, wherein the playout group identifier corresponds to one or more playout groups associated with a V3C component in a presentation set or a V3C component in a conformance set. (Item 23) 22. The method of claim 21, wherein the alternative set identifier corresponds to one or more alternative sets associated with a V3C component in a conformance set. (Item 24) 20. The method of claim 17, further comprising configuring a playout set descriptor to include playout set information in the media presentation description. (Item 25) 25. The method of claim 24, wherein the playout group descriptor corresponds to a V3C component in a representation set associated with one or more playout groups. (Item 26) 26. The method according to any of items 24-25, wherein the playout set descriptor includes at least one of one or more identifiers of a playout set, one or more quality levels of a playout set, or one or more identifiers of V3C components in a conformance set. (Item 27) An apparatus for wireless communication, the apparatus comprising a memory and a processor, the processor reading a code from the memory and implementing a method defined in any of items 1-26. (Item 28) A computer readable program storage medium storing code which, when executed by a processor, causes the processor to implement a method defined in any of items 1-26. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a flow chart illustrating an example of volumetric video data processing based on a playout set.

[0010] [Diagram 2] FIG. 2 is a diagram illustrating an example of a media track structure of an alternative V3C component consistent with some embodiments of the disclosed technology.

[0011] [Diagram 3] FIG. 3 is a diagram illustrating an example structure of a playout group of V3C component tracks according to some embodiments of the disclosed technology.

[0012] [Figure 4] FIG. 4 is a diagram illustrating an example of a playout group structure of a V3C component in a media presentation description (MPD) according to some embodiments of the disclosed technology.

[0013] [Diagram 5] FIG. 5 is an example block diagram of a volumetric video data representation and processing device according to some embodiments of the disclosed techniques.

[0014] [Figure 6] FIG. 6 illustrates an example method for processing media content according to some exemplary embodiments of the disclosed technology.

[0015] [Figure 7]FIG. 7 illustrates another example of a method for processing media content according to some exemplary embodiments of the disclosed technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Section headings are used herein for ease of understanding only and do not limit the scope of the embodiments to the section in which they are described.

[0017] The disclosed technology can be implemented in some embodiments to provide a playout representation and processing method based on multi-track encapsulated volumetric video that can improve transmission efficiency and rendering process. Some embodiments of the disclosed technology can be used to effectively transmit volumetric video content and reconstruct the volumetric video with suitable quality under different network conditions.

[0018] Immersive media allows users to experience a highly immersive experience in realistic virtual spaces through audio and video technologies. Currently, immersive media systems use volumetric video to represent 3D scenes / objects, i.e., any object in the scene can be viewed through six degrees of freedom of movement.

[0019] In the immersive media system, the volumetric video processing process includes: the system platform creates volumetric video, projects the three-dimensional volumetric video onto a two-dimensional plane, represents the volumetric video by atlas data (projection relationship between three-dimensional space and two-dimensional space) and two-dimensional components (geometry, attributes, and occupancy map data), compresses the atlas data and component data, and encapsulates them into media files. The volumetric video is stored on the platform side and transferred to the user side based on viewing requirements.

[0020] Under current technology with the representation of volumetric video on a two-dimensional plane, a volumetric video media file can contain multiple tracks, each track separately containing volumetric video parameters, atlas parameters, atlas data, and / or V3C components. The V3C (visual volumetric video based coding) component data can be compressed using video encoding methods and generated in different versions such as AVC (Advanced Video Coding), HEVC (High Efficiency Video Coding) encoding methods, different bit rates, resolutions, etc.

[0021] Different versions of occupancy, geometry, and attribute components may have relationships or dependencies between each other. The geometry component data is generated based on the atlas data and the occupancy component. Furthermore, the attribute component data is also generated based on the occupancy component data, the geometry component data, and other information. When the dependencies are not kept, visible rendering artifacts may occur. For example, a boundary between geometry and attributes that does not match exactly may cause "flying pixels". Even more visible errors may occur when the exact occupancy track is not paired with the matching geometry track. Combinations of occupancy maps and geometry of different quality may eventually easily generate distorted edges around patch boundaries. In this case, only certain combinations of V3C components from the alternative set can be played out together.

[0022] According to the dependency between different versions and types of V3C components, there are many playout combinations that form alternative or modified groups, which include the same type of V3C components with different versions. This specification defines a mechanism for indicating alternative playout groups by tracks or entities. The combination of each playout group is constant in the playout process, and it is required to list as many all playout combinations as possible, so that the optimal playout scheme can be selected by the user terminal. If there are multiple atlases or atlas styles, the number of such groups will be large.

[0023] For volumetric video supporting six degrees of freedom, the director may consider controlling and changing the user's viewing state during the production of the volumetric video to induce the user to view the optimal visual content. The volumetric video content can be a partial transmission, for example, a high-quality version of component data corresponding to the field of view (FOV) and a low-quality version of component data outside the FOV are transmitted, or the content within the viewport is transmitted. As the scenes / objects in the volumetric video are dynamically changed, the playout set should change accordingly. A mechanism for dynamically indicating the playout set is needed to fit more scenarios.

[0024] It should be noted that in some embodiments of the disclosed technology, the implementation manner is for storing volumetric video data in a file based on the ISO (International Organization for Standardization, International Organization for Standardization) Basic Media File Format. In particular, the ISO Basic Media File Format (such as the Restricted Scheme Information Box, the Track Reference Box, and the Track Group Box) may refer to MPEG-4 (established by ISO / IEC JTC1 / SC29 Moving Image Experts Group (abbreviated as MPEG) Part 12 ISO Base Media File Format (ISO / IEC14496-12)) to operate. The projection, packaging step, and basic format of the volumetric video can be operated with reference to MPEG-I Part 10 (ISO / IEC23090-10) (established by ISO / IEC JTC1 / SC29 Moving Image Experts Group (MPEG)). The transmission of the encapsulated volumetric video and its basic format may refer to MPEG-DASH (established by ISO / IEC JTC1 / SC29 / Moving Image Experts Group (MPEG)).

[0025] FIG. 1 is a flow chart illustrating an example of volumetric video data processing based on a playout set.

[0026] Some embodiments of the disclosed technology can be used to provide a playout method for multi-track-based volumetric video. As shown in FIG. 1, the volumetric video processing based on the playout group may include the following operations. S201 determines, according to the information of the playout set, a playout set corresponding to the volumetric video reconstructed in the three-dimensional space. - S202 acquires two-dimensional volumetric video content in a media track corresponding to a playout set. - The S203 reconstructs volumetric video into 3D space.

[0027] FIG. 2 is a diagram illustrating an example of a media track structure of an alternative V3C component consistent with some embodiments of the disclosed technology.

[0028] A media track stores volumetric video data projected from a three-dimensional space onto a two-dimensional plane, and includes atlas data and its corresponding geometry, attribute, and / or occupancy components, where each component may have alternative versions. V3C components with different versions are stored in different tracks, as shown in FIG. 2, and the media track structure is stored in the media file.

[0029] If an alternative scheme for V3C component tracks exists, the V3C component tracks in which the alternative components are located may form an alternative group, i.e. a playout group. There are multiple playout groups that may be selected during transmission, decoding, and / or reconstruction.

[0030] In one implementation, media tracks in which V3C components of the same type with different versions corresponding to the same atlas or atlas style are located are alternate tracks, indicated through the mechanism defined in ISO / IEC 14496-12. In the TrackHeaderBox of an alternate track, an alternate_group field is present. V3C component tracks with the same alternate_group value are alternate tracks if they contain V3C components of alternate versions.

[0031] In another implementation, an alternative track group can also be described by an EntityToGroupBox with a type of 'altr' defined by ISO / IEC 14496-12. The media tracks in which the alternative V3C components are located form an entity group. There can be multiple such groups, each entity group having a group_id.

[0032] FIG. 3 is a diagram illustrating an example structure of a playout group of V3C component tracks according to some embodiments of the disclosed technology.

[0033] In some implementations, a playout group contains V3C components from multiple alternative track groups, and the content of each V3C component in a playout group is different, i.e., two different versions of the same type of V3C component corresponding to the same atlas or atlas tile cannot exist within the same playout group. An example of a playout group structure is shown in Figure 3.

[0034] In order to guarantee the quality of the 3D volumetric video generated by reconstruction and rendering, different types and versions of V3C components corresponding to atlases or atlas styles are not arbitrarily selected for a certain playout set. The user's viewing state and the director's positioning should be taken into account.

[0035] The playout set includes some of the alternative V3C components that correspond to parts of the atlases or atlas tiles that correspond to volumetric videos in partial 3D spatial regions, i.e., V3C components that correspond to some atlases or atlas tiles may not be included in the playout set at this moment.

[0036] The structure of a playout group is described by a PlayoutGroupStruct, which contains the playout group identifier, the number of V3C components, the type of V3C component (such as a geometry component, an attribute component, or an occupancy component), and the identifier of the corresponding V3C component track.

[0037] [Table 1]

[0038] Here, "group_id" indicates the identifier of the playout group, "num_playout_member" indicates the number of V3C components in the playout group, "atlas_id" indicates the ID of the atlas corresponding to the V3C component, "atlas_tile_flag" indicates whether the component track includes an atlas tile, and if its value is 0, it indicates that the component track does not include an atlas tile, and if its value is 1, it indicates that the component track includes one or more atlas tiles, "tile_id" indicates the ID of the atlas tile, "component_track_id" indicates the identifier of the V3C component track, and "V3C_component_type" indicates the type of V3C component stored in the V3C component track.

[0039] (Solution 1)

[0040] In some embodiments, the playout set corresponding to the user's viewport may have different frequency of changes due to various factors such as scene changes, director's intent, etc. The V3C components that need to be transmitted may correspond to all of the entire three-dimensional space region or a part of the three-dimensional space region, and thus the playout set can be dynamically changed according to the actual viewing scene.

[0041] This solution provides a way to describe frequently and dynamically changing playout sets in a media file.

[0042] The configuration information of a playout group is described by a DynamicPlayoutGroupConfigBox, which is identified by the box type 'dpog'.

[0043] Box type: 'dpog' Container: Sample Description Box ('stsd') Required: No Quantity: 0 or 1

[0044] [Table 2]

[0045] where "num_playout_group" indicates the number of playout groups, "viewport_match_flag" indicates whether the playout group corresponds to a viewport, "viewport_id" indicates the ID of the viewport that the playout group corresponds to, and "effect_level" indicates the quality level of the V3C component in the playout group. Additionally, playout_group_description is a null-terminated UTF-8 string that provides a text description of the playout group.

[0046] A volumetric video media file contains a timed metadata media track identified by a sample entry of type 'plgs'. The sample entry contains a data box of type 'dpog', which describes the configuration information of the dynamic playout set.

[0047] An example syntax for a sample entry of a timed metadata media track in a V3C playout set is as follows:

[0048] [Table 3]

[0049] The syntax sample format for a dynamic timed metadata track for the V3C playout suite is as follows:

[0050] [Table 4]

[0051] Here, "viewport_pos_update" indicates whether the viewport to which the playout group corresponds has been changed, "num_addition_playout" indicates the number of newly added playout groups, and "viewport_index" indicates the index of the viewport.

[0052] Additionally, timed metadata tracks are a mechanism in the ISO Basic Media File Format (ISOBMFF) for establishing timed metadata associated with a particular sample. Timing metadata is largely uncoupled from the media data and is typically "descriptive".

[0053] (Solution 2)

[0054] In some embodiments, the content of the volumetric video is dynamic, but such changes are infrequent. One or more playout sets may be used during one sampling period and changed in the next. This solution provides a way to describe that one or more playout sets change from time to time.

[0055] The samples in the media tracks in which the atlas data and / or V3C parameter sets are located are grouped through the sample group method: the playout group information corresponding to the V3C component tracks referenced by the atlas track is described in the sample group box of the media track.

[0056] Information on a sample group based on a playout group is described through a PlayoutSampleGroupDescriptionEntry, which is included in a SampleGroupDescriptionBox defined in ISO / IEC14496-12 and ISO / IEC23090-10.

[0057] [Table 5]

[0058] (Solution 3)

[0059] FIG. 4 is a diagram illustrating an example of playout group information of a V3C component in a media presentation description (MPD) according to some embodiments of the disclosed technology.

[0060] In some embodiments, the V3C components in a playout group are transmitted to the user side according to the playout group information, and the terminal only needs to process the V3C components in the playout group. The information of the transmitted media files stored in the server is described in a DASH (Dynamic Adaptive Streaming over HTTP) Manifest (MPD), and the user side can obtain the corresponding V3C component data from the server by referring to the information.

[0061] Each V3C component is represented as a separate adaptation set in the MPD. The adaptation set of the atlas information is used as the main adaptation set. If a geometry or attribute component has multiple maps corresponding to the atlas style, a separate AdaptationSet or presentationSet element is used to signal each V3C component.

[0062] In some implementations, playout_group_ids or / and alternative_group_id may be added at the AdaptationSet or presentationSet level to represent playout group information in the MPD.

[0063] Here, "playout_group_ids" indicates the identifiers of all playout groups to which a V3C component in a presentation set or an alternative V3C component in a conformance set belongs. A V3C component can belong to one or more playout groups at the same time, and "alternative_group_id" indicates the identifier of the alternative group to which a V3C component in a conformance set belongs.

[0064] In implementation, a V3CComponent descriptor is defined in ISO / IEC23090-10 for each V3C component present in the AdaptationSet, and playout_group_ids or / and alternative_group_id can be added to the V3CComponent descriptor.

[0065] In another implementation, the PlayoutGroup descriptor is defined as a SupplementalProperty element for describing playout group information in the MPD. The PlayoutGroup descriptor indicates that the corresponding V3C component in the representation set belongs to one or more playout groups.

[0066] One or more PlayoutGroup descriptors are present at the representation set level of the MPD. The PlayoutGroup descriptor may be present at the MPD level or the AdaptationSet level. If a PlayoutGroup descriptor appears at the MPD or AdaptationSet level, the higher level descriptor should be used.

[0067] In some implementations, a PlayoutGroup descriptor that appears in the MPD or AdaptationSet level includes the elements and attributes defined in the table below.

[0068] [Table 10]

[0069] In some other implementations, the PlayoutGroup descriptor, which appears within the RepresentationSet level, includes the elements and attributes defined in the table below.

[0070] [Table 20]

[0071] FIG. 5 is an example block diagram of a volumetric video data representation and processing device according to some embodiments of the disclosed techniques.

[0072] In some embodiments of the disclosed technology, an immersive media system includes an immersive media system platform 10 and a player terminal 20.

[0073] In some implementations, the immersive media system platform 10 includes a content creation module 101, a storage module 102, and a transmission module 103. The content creation module 101 may be used for encoding and encapsulation processing of captured volumetric video. The storage module 102 is used to store encapsulated media files and describe them by MPD. In addition, the transmission module 103 is used to receive request messages from client terminals or to transmit data (media files, MPD, etc.) stored in 102. The above-mentioned reception or transmission can be realized through a wireless network provided by a communication provider, a local wireless area network, or a wired method. The display 20 includes at least a transmission module 201, a media processing module 202, and a media presentation module 203. The transmission module 201 is used to receive data (media files, etc.) transmitted by the transmission module 103, or to transmit messages to 103, such as requests for media files, reports of user status, etc. The media processing module 202 is used to decode and decapsulate the received media files, reconstruct the volumetric video according to the user's current viewing state (viewing position, viewing direction, etc.), and perform the rendering process. In addition, the media presentation module 203 is used to present the reconstructed volumetric video within the user's viewport.

[0074] 6 illustrates an example of a method 600 for processing media content according to some example embodiments of the disclosed technology. At 610, the method includes determining a playout set corresponding to a volumetric video in a three-dimensional space based on the playout set information, the playout set including a plurality of components, each component of a type having a different version from another component of that type. At 620, the method includes obtaining two-dimensional volumetric video content from a media track corresponding to the playout set. At 630, the method includes reconstructing the volumetric video in the three-dimensional space based on the plurality of components with different versions.

[0075] 7 shows another example 700 of a method for processing media content according to some example embodiments of the disclosed techniques. At 710, the method includes configuring, by the server, a playout set including one or more types of components, where each component of a type has a different version from another component of that type. At 720, the method includes configuring, by the server, playout set information corresponding to the playout set. At 730, the method includes transmitting, by the server, the playout set information in the media presentation description to the user device.

[0076] It should be appreciated that this document discloses techniques that may be embodied in various embodiments and establish and manage multicast sessions in various scenarios. The disclosed and other embodiments, modules, and functional operations described herein can be implemented in digital electronic circuitry, including the structures disclosed herein and their structural equivalents, or in computer software, firmware, or hardware, or in a combination of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or to control the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition that produces a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include code that creates an execution environment for the computer program, such as processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these. A propagated signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to a suitable receiver device.

[0077] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple collaborative files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program can be deployed to run on one computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0078] The processes and logic flows described herein may be implemented by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be implemented by, and apparatus may also be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0079] Processors suitable for executing computer programs include, by way of example, both general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for carrying out instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or be operatively coupled to receive data therefrom or transfer data thereto, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include, by way of example, all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices, magnetic disks, such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0080] Some embodiments may preferably implement one or more of the following solutions, which are listed in appendix form. The following appendixes are supported and further described in the examples above and throughout this document.

[0081] Addendum 1. A method for processing media content, the method including: determining a playout set corresponding to a volumetric video in a three-dimensional space based on playout set information, the playout set including a plurality of components, each component of a type having one or more versions; obtaining two-dimensional volumetric video content from a media track corresponding to the playout set; and reconstructing the volumetric video in the three-dimensional space based on the plurality of components with different versions.

[0082] Addendum 2. The method of Addendum 1, wherein the components include a visual volumetric video-based coding (V3C) component.

[0083] Addendum 3. The method of Addendum 2, wherein the V3C component includes at least one of an occupancy component, a geometry component, or an attribute component.

[0084] Addendum 4. The method of Addendum 3, wherein each atlas or atlas style corresponds to an attribute component, an occupancy component, and / or a geometric shape component with multiple versions.

[0085] Addendum 5. The method of Addendum 3, wherein the different versions of the occupancy component, the different versions of the geometry component, and the different versions of the attribute component each have a dependency between each other.

[0086] Appendix 6. The method of Appendix 1, wherein the playout set includes V3C components from a set of multiple alternative tracks.

[0087] Appendix 7. The method of Appendix 1, wherein the different versions include at least one of a different encoding method, a different bit rate, or a different resolution.

[0088] Addendum 8. The method of Addendum 1, wherein the components in the playout group correspond to one or more atlases or atlas tiles.

[0089] Addendum 9. The method of Addendum 1, wherein the playout group is configured based on a time-specific metadata track for configuring dynamic playout grouping information in the sample entry and / or one or more samples belonging to the sample entry.

[0090] Clause 10. The method of clause 9, wherein time-specific metadata tracks are used to configure a playout set at each sample time.

[0091] Addendum 11. The method of Addendum 9, wherein the sample entries include configuration information for configuring an initial playout set.

[0092] Clause 12. The method of clause 11, further comprising generating a flag to indicate a change in the playout set.

[0093] Clause 13. The method of clause 11, further comprising generating a flag to indicate one or more new playout sets.

[0094] Addendum 14. The method of Addendum 1, wherein the playout set is constructed based on a sample set including samples in video encoded component data contained in the track or an atlas or atlas tile contained in the track.

[0095] Clause 15. The method of clause 14, wherein the set of samples is configured to be used during a sampling period and updated based on the sampling period.

[0096] Addendum 16. The method of any of Addendums 1-15, wherein the playout group information is described by a syntax of a playout group structure.

[0097] Addendum 17. A method of processing media content, the method comprising: configuring, by a server, a playout set including one or more types of components, each component of a type having one or more versions; configuring, by the server, playout set information corresponding to the playout set; and transmitting, by the server, the playout set information in a media presentation description to a user device.

[0098] Addendum 18. The method of Addendum 17, wherein the components include a visual volumetric video-based coding (V3C) component.

[0099] Addendum 19. The method of Addendum 18, wherein the V3C component includes at least one of an occupancy component, a geometry component, or an attribute component.

[0100] Appendix 20. The method of Appendix 19, wherein each of the V3C components is represented as a separate conformance set in the media presentation description.

[0101] Appendix 21. The method of Appendix 17, further comprising adding at least one of a playout group identifier or an alternative group identifier corresponding to the playout group to the media presentation description to indicate the playout group information.

[0102] Clause 22. The method of clause 21, wherein the playout group identifier corresponds to one or more playout groups associated with a V3C component in the presentation set or a V3C component in the conformance set.

[0103] Clause 23. The method of Clause 21, wherein the alternative set identifier corresponds to one or more alternative sets associated with a V3C component in the conformance set.

[0104] Clause 24. The method of claim 17, further comprising constructing a playout set descriptor to include playout set information in the media presentation description.

[0105] Clause 25. The method of clause 24, wherein the playout set descriptor corresponds to a V3C component in a representation or conformance set associated with one or more playout sets.

[0106] Addendum 26. A method as described in any of Addendums 24-25, wherein the playout group descriptor includes at least one of one or more identifiers (IDs) of the playout group, one or more quality levels of the playout group, or one or more identifiers (IDs) of V3C components in the conformance set.

[0107] Addendum 27. An apparatus for wireless communication, the apparatus comprising a memory and a processor, the processor reading code from the memory and implementing a method defined in any of Addendums 1-26.

[0108] Addendum 28. A computer-readable program storage medium storing code which, when executed by a processor, causes the processor to implement a method defined in any of Addendums 1-26.

[0109] Although this patent document contains many details, these should not be construed as limitations on the scope of any invention or claimed subject matter, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as operating in a combination and may even be initially claimed as such, one or more features from a claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0110] Similarly, although operations are depicted in the figures in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown, or in a sequential order, or that all of the illustrated operations be performed, to achieve desirable results. Further, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0111] Only some implementations and examples have been described; other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. 1. A method for processing media content, the method comprising: determining a playout set corresponding to a volumetric video in a three-dimensional space based on the playout set information, the playout set including a plurality of components, each component of a type having a different version than another component of the type; Obtaining two-dimensional volumetric video content from a media track corresponding to the playout set, wherein the components in the playout set include components having different versions, the components having different versions corresponding to a same atlas or a same atlas tile, and being stored in different media tracks, respectively; reconstructing the volumetric video in a three-dimensional space based on the plurality of components having the plurality of different versions; A method comprising:

2. 2. The method of claim 1, wherein the plurality of components comprises a plurality of visual volumetric video-based coding (V3C) components, the plurality of V3C components comprising at least one of a plurality of occupancy components, a plurality of geometric components, or a plurality of attribute components, and wherein the plurality of different versions of the plurality of occupancy components and the plurality of different versions of the plurality of geometric components and the plurality of different versions of the plurality of attribute components each have a dependency between each other.

3. The method of claim 1 , wherein the playout set includes a plurality of V3C components from a plurality of alternative track sets.

4. The method of claim 1 , wherein the different versions include at least one of: different encoding methods, different bit rates, or different resolutions.

5. The method of claim 1 , wherein the components in the playout set correspond to one or more atlases or one or more atlas tiles, each atlas or each atlas tile corresponding to an attribute component, an occupancy component, and / or a geometric shape component having multiple versions.

6. The method of claim 1 , wherein the playout group is configured based on a time-specific metadata track for configuring dynamic playout grouping information in a sample entry and / or one or more samples belonging to the sample entry.

7. The method of claim 1 , wherein the playout set is constructed based on a sample set including samples in video encoding component data contained in a track or an atlas or atlas tile contained in a track.

8. The method of claim 1 , wherein the playout group configuration information is described by dynamic playout group configuration boxes identified by box types.

9. The method of claim 8 , wherein the dynamic playout group configuration box includes at least one of: one or more playout group structures, one or more identifiers of a viewport, and one or more quality levels of a playout group.

10. 1. A method for processing media content, the method comprising: A server configures a playout set including one or more types of components, the playout set including a plurality of components having a plurality of different versions, the plurality of components having a plurality of different versions corresponding to a same atlas or a same atlas style and respectively stored in a plurality of different media tracks; said server configuring playout set information corresponding to said playout set; the server transmitting the playout group information to a user device in a media presentation description; A method comprising:

11. 11. The method of claim 10, wherein the plurality of components comprises a plurality of visual volumetric video-based coding (V3C) components, the plurality of V3C components comprising at least one of a plurality of occupancy components, a plurality of geometry components, or a plurality of attribute components.

12. The method of claim 10, further comprising adding at least one of a playout group identifier or an alternative group identifier corresponding to the playout group to the media presentation description to indicate the playout group information.

13. The method of claim 10, further comprising constructing a playout set descriptor to include playout set information within the media presentation description.

14. The method of claim 13 , wherein the playout set descriptor comprises at least one of one or more identifiers of a playout set, one or more quality levels of a playout set, or one or more identifiers of V3C components in a conformance set.

15. An apparatus for wireless communication, said apparatus comprising a memory and a processor, said processor reading code from said memory and performing the method according to any of claims 1 to 14.

Images