Volumetric media processing method and device
Through visual volumetric video coding (V3C) technology, three-dimensional scenes are converted into volumetric media, and existing video coding components are used for encoding and decoding, which solves the problem of low efficiency of three-dimensional visual scenes in network storage and transmission, realizes efficient three-dimensional scene encoding and transmission, and supports six-degree-of-freedom immersive media experience.
Patent Information
- Application Number
- CN202080094057.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-24
- Publication Date
- 2025-09-09
- Estimated Expiration
- 2040-06-24
AI Technical Summary
Existing video coding technologies have difficulty in efficiently processing and transmitting visual information of three-dimensional visual scenes, especially in the process of network storage and transmission.
It uses technology based on visual volumetric video coding (V3C) to convert three-dimensional scenes into volumetric media, uses existing video coding components for encoding and decoding, and transmits them through multi-track packaging and adaptive bitrate streaming technology, supporting DASH and HLS protocols.
It achieves efficient encoding and transmission of three-dimensional visual scenes, supports six-degree-of-freedom immersive media experience, reduces storage and transmission costs, and adapts to different access requirements.
Smart Images

Figure CN115004716B_ABST
Abstract
Description
Technical Field
[0001] This patent document relates to volumetric media processing and transmission technology. Background Art
[0002] Video coding uses compression tools to encode two-dimensional video frames into a compressed bitstream representation that is more efficient for storage or transmission over a network. Traditional video coding techniques that use two-dimensional video frames for encoding are sometimes inefficient for representing the visual information of three-dimensional visual scenes. Summary of the Invention
[0003] This patent document describes, among other things, techniques for encoding and decoding digital video that carries visual information related to visual volumetric media.
[0004] In one example aspect, a method for processing visual volumetric data is disclosed. The method includes determining one or more preselected elements corresponding to a preselection of volumetric media from a media presentation description (MPD) file, accessing one or more atlas data components and associated video encoding components of the volumetric media using the one or more preselected elements, and reconstructing the volumetric media from the one or more atlas data components and the associated video encoding components.
[0005] In another example aspect, a method for generating a bitstream including visual volumetric data is disclosed. The method includes generating volumetric media from a three-dimensional scene, the volumetric media including one or more atlas data components representing the three-dimensional scene and an associated video encoding component; and including one or more preselected elements in a media presentation description (MPD) file indicating a preselection of the volumetric media. The one or more preselected elements provide access to the one or more atlas data components associated with the three-dimensional scene and the video encoding component.
[0006] In another exemplary aspect, a device for implementing one or more of the above methods is disclosed, wherein the device may include a processor configured to implement the encoding or decoding method described.
[0007] In yet another example aspect, a computer program storage medium is disclosed. The computer program storage medium includes code stored thereon. The code, when executed by a processor, causes the processor to implement the described method.
[0008] These and other aspects are described herein. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1A is a flow chart of an example method of volumetric media processing.
[0010] Figure 1B is a flow chart of an example method of volumetric media processing.
[0011] Figures 2A-2C An example of a flowchart of an example process for parsing a bitstream is shown.
[0012] Figure 3-6 Various exemplary DASH configurations for volumetric media delivery are shown.
[0013] Figure 7 An example of a flow chart for preselecting volumetric media using preselection elements is shown.
[0014] Figure 8 is a block diagram of an example of an encoder device.
[0015] Figure 9 is a block diagram of an example of a decoder device.
[0016] Figure 10 An example of a video system is shown.
[0017] Figure 11 is a block diagram of a hardware platform for implementing the techniques disclosed in this document. DETAILED DESCRIPTION
[0018] The section headings used in this document are for readability purposes only and do not limit the scope of the embodiments and techniques disclosed in each section to that section alone. Certain features are described using examples from the Moving Picture Experts Group (MPEG) standards such as H.264 / AVC, H.265 / HEVC, and MPEG-DASH. However, the applicability of the disclosed techniques is not limited to these standards.
[0019] Throughout this document, various syntax elements for point cloud data processing are disclosed in different sections. However, it should be understood that, unless otherwise noted, syntax elements with the same name used in different sections will have the same format and syntax. Furthermore, in various embodiments, the different syntax elements and structures described under different section headings may be combined. Furthermore, while specific structures are described as implementation examples, it should be understood that the order of the various entries in the syntax structure may vary unless otherwise noted in this document.
[0020] Traditionally, the capture, processing, storage, and presentation of digital visual media, such as images and videos, have used two-dimensional frame-based visual scene capture. Recent advances in 3D capture and rendering technologies have triggered a new wave of innovation in virtual / augmented / mixed reality (VR / AR / MR) content, also known as extended reality (XR) services and applications.
[0021] For example, point clouds have emerged as one of the main representations for such applications. A point cloud frame consists of a set of independent three-dimensional (3D) points. In addition to having a 3D position (e.g., spatial properties), each point can also be associated with several other properties (e.g., color, reflectivity, surface normal, etc.). A point cloud consists of a series of point cloud frames. The number, position, and properties of the points may vary from frame to frame. Another example of visual volumetric data is multi-view video content, in which a real or virtual 3D scene is captured by multiple real or virtual cameras. Such representations require a large amount of data, which can be costly in terms of storage and transmission.
[0022] Various industry standards have begun to address issues related to the capture, carrying, and presentation of 3D visual scenes. The Moving Picture Experts Group (MPEG) has been developing coded representations of immersive media under the MPEG-I standardization project. One of the results, the Visual Volumetric Video Coding (V3C) standard, leverages traditional two-dimensional (2D) frame-based video coding tools to encode 3D visual information by projecting 3D information onto a 2D plane. This is expected to be reused by other MPEG-I volumetric codecs currently under development, such as Video-based Point Cloud Compression (V-PCC) and MPEG Immersive Video (MIV).
[0023] Video-based point cloud compression (V-PCC) represents the volumetric encoding of point cloud visual information and uses MPEG video codecs (such as AVC, HEVC) to achieve efficient capture, compression, reconstruction and rendering of point cloud data. The V-PCC bitstream containing the coded point cloud sequence (CPCS) consists of VPCC units, which carry sequence parameter set (SPS) data, atlas information bitstream, 2D video coding occupancy map bitstream, 2D video coding geometry bitstream and zero or more 2D video coding attribute bitstreams. Each V-PCC unit has a V-PCC unit header and a V-PCC unit payload that describes the type of the V-PCC unit. The payloads of the occupancy, geometry and attribute V-PCC units correspond to video data units that can be decoded by the corresponding video decoder (e.g., High Efficiency Video Coding Network Abstraction Layer or High Efficiency Video Coding Network Abstraction Layer HEVC NAL unit).
[0024] MPEG is developing an international standard (ISO / IEC 23090-12), MPEG Immersive Video (MIV), to support the compression of multi-view video content, where a real or virtual 3-D scene is captured by multiple real or virtual cameras. MIV content supports playback of three-dimensional (3D) scenes with 6 degrees of freedom (6DoF) within a limited range of viewing positions and orientations.
[0025] Despite differences in input data formats and rendering, 23090-5 Video-based Point Cloud Compression (V-PCC) and 23090-12 MPEG Immersive Video (MIV) share the same core tools to represent information in the coding domain, i.e., splitting 3D spatial data into 2D patch maps and encoding them into 2D atlas frames. Therefore, it would be beneficial to use the common V3C elementary bitstream as the base bitstream syntax to carry V-PCC and MIV content.
[0026] A V3C bitstream containing a coded V3C sequence (CVS) may consist of V3C units carrying V3C parameter set (VPS) data, a coded atlas bitstream, a 2D video coded occupancy map bitstream, a 2D video coded geometry bitstream, and zero or more 2D video coded attribute bitstreams.
[0027] In general, embodiments based on the disclosed technology can be used for volumetric media processing. In some embodiments, a technical solution is provided for reusing existing tools to store video encoding components to allow visual volumetric media such as V-PCC data and MIV content to be represented in files compatible with traditional 2D video formats.
[0028] In some embodiments, the visual volumetric video-based coding (V3C) data is stored in a file based on the ISO / IEC 14496-12 ISO (International Organization for Standardization) Base Media File Format (ISOBMFF).
[0029] V3C data storage that conforms to ISOBMFF (such as MP4 files) consists of a hierarchical data structure called boxes, each of which has a type and length. A box that contains another box is called a container box. V3C data storage based on ISOBMFF has a box of type "ftyp" that contains the file type and compatibility information. There is a separate box of type "moov" (movie box), which is a container box whose child boxes contain all the metadata information for the file. The media data of an ISOBMFF file is contained in a media box of type "mdat" (media data box), which is also a container box and can reference other files.
[0030] In some embodiments, V3C data can be encapsulated as a single track in the case of simple ISOBMFF encapsulation. A V3C bitstream sample contains one or more V3C units belonging to the same presentation time, that is, a V3C access unit. The V3C unit header and V3C payload data structure can be retained in the bitstream without further processing.
[0031] In some embodiments, different V3C substreams (e.g., one or more coded atlas substreams, 2D video coding substreams for geometry, occupancy maps, and attributes) can be encapsulated as independent tracks within an ISOBMFF container file. Multi-track encapsulation of V3C data requires a multi-track declaration to represent the V3C bitstream, i.e., V3C units in a V-PCC elementary stream are mapped to independent tracks within the ISOBMFF container file based on their type. There are two types of tracks in a multi-track encapsulated V3C data ISOBMFF container: V3C tracks and V3C component tracks.
[0032] The V3C track represents the volumetric vision track, which carries the atlas data of the V3C bitstream, including patch information and sequence parameter sets. The video coding V3C component track represents the video track, which carries the 2D video coding data of any one of the occupancy map, geometry, or attribute component video bitstreams of the V3C bitstream. Based on this layout, the V3C ISOBMFF container can include the following:
[0033] - A V3C track, which contains the V3C parameter sets and atlas sub-bitstream parameter sets and samples that carry atlas sub-bitstream NAL units. The V3C track also includes a track reference to the video coding V3C track that carries the payload of the video compression V3C unit.
[0034] - a constrained video scheme track, where the samples contain access units of a video coding elementary stream with occupancy map data.
[0035] - One or more constrained video scheme tracks, where samples contain access units of a video coding elementary stream of geometry data.
[0036] - Zero or more constrained video scheme tracks, where samples contain access units of a video coding elementary stream of attribute data.
[0037] Unlike traditional media content, volumetric media designated as V3C data consists of several V3C components, including atlases, occupancy maps, geometry, or attributes. In order to reconstruct and render V3C data, the different V3C components need to be collectively retrieved and decoded.
[0038] On the other hand, volumetric media encoded as V3C data can provide users with an immersive six-degrees-of-freedom (6-DoF) media experience. At any given time, only a portion of the volumetric media is visible, depending on the user's view position, orientation, and other factors. For many applications, delivering, decoding, and rendering the entire volumetric media data is unnecessary.
[0039] In general, embodiments based on the disclosed technology can be used for volumetric media processing. In some embodiments, a technical solution for configuring and streaming V3C components related to volumetric media is provided to support volumetric media delivery based on adaptive bitrate streaming technologies such as Dynamic Adaptive Streaming over HTTP (DASH) and HTTP Live Streaming (HLS).
[0040] Figure 1A A flowchart depicts an example method 100 for volumetric media processing. The method 100 may be implemented by a decoder device that decodes a volumetric media bitstream to reconstruct a volumetric scene therefrom. The method 100 includes, at 102, determining one or more preselected elements corresponding to a preselection of volumetric media from a media presentation description (MPD) file. The method 100 includes, at 104, accessing one or more atlas data components and associated video encoding components of the volumetric media using the one or more preselected elements. The method 100 includes, at 106, reconstructing the volumetric media using results of decoding the one or more atlas data components and results of decoding the associated video encoding components. Additional details of the method 100 and the format of the volumetric media bitstream are provided herein.
[0041] Figure 1B A flowchart of an example method 150 for encoding volumetric media is depicted. The method may be implemented by an encoder apparatus as described herein. The method 150 may be used to convert a 3D scene into a bitstream representation conforming to a bitstream format as described herein. The method 150 includes generating (152) volumetric media from a three-dimensional scene, the volumetric media including one or more atlas data components representing the three-dimensional scene and associated video encoding components. Various embodiments of the arrangement of the atlas data components and the video encoding components are disclosed herein. The method 150 may also include including (154) one or more preselection elements in a media presentation description (MPD) file indicating a preselection of the volumetric media, wherein the one or more preselection elements provide access to one or more atlas data components associated with the three-dimensional scene and the video encoding components. Various embodiments disclosed herein relate to formats of MPD files and volumetric media bitstreams. Preselection may represent a logical grouping of media components that are bundled together and intended to be used together at a decoder.
[0042] In order to partially access and decode a portion of volumetric media, such as one or more atlases within a 3D scene, in some embodiments, a technical solution is provided to support multi-track encapsulation of V3C bitstreams, where each atlas of the V3C bitstream is encapsulated in a separate track.
[0043] V3C track sample entry
[0044] Sample entry type: 'v3c1', 'v3cg', 'v3cb', 'v3a1', or 'v3ag'
[0045] Container: SampleDescriptionBox
[0046] Mandatory: 'v3c1' or 'v3cg', or 'v3cb' sample entry is mandatory
[0047] Quantity: There can be one or more sample entries
[0048] V3C tracks shall use V3CSampleEntry, which extends VolumetricVisualSampleEntry with sample entry type 'v3c1', 'v3cg', 'v3cb', 'v3a1', or 'v3ag'.
[0049] When a V3C bitstream contains a single atlas, a V3C track with sample entry type 'v3c1' or 'v3cg' should be used. When a V3C bitstream contains multiple atlases, each atlas bitstream should be encapsulated in a separate V3C track. One of the tracks should use sample entry type 'v3cb', while the other tracks should use sample entry type 'v3a1' or 'v3ag'.
[0050] The V3C track sample entry should contain V3CConfigurationBox and V3CUnitHeaderBox.
[0051] grammar
[0052]
[0053] Semantics
[0054] The V3CConfigurationBox provides decoding specific information for a V3C bitstream for configuration and initialization of a V3C decoder. The V3CConfigurationBox shall contain the V3C parameter set for the V3C bitstream and shall contain only non-ACL NAL units that are common to all V3C tracks of the V3C data.
[0055] V3CUnitHeaderBox contains the V3C unit header of the V3C data of a V3C track.
[0056] In order to partially access and decode a portion of volumetric media, such as one or more tiles within a 3D scene, in another embodiment, a technical solution is provided to support multi-track packaging of V3C bitstreams, wherein one or more tiles belonging to an atlas are packaged into separate atlas tile tracks.
[0057] V3C atlas tile track sample entry
[0058] Sample entry type: 'v3t1'
[0059] Container: SampleDescriptionBox
[0060] Mandatory: Yes
[0061] Quantity: There can be one or more sample entries
[0062] Atlas tile tracks should use AtlasTileSampleEntry, which extends VolumetricVisualSampleEntry with sample entry type 'v3t1'.
[0063] An atlas tile track shall contain only ACL NAL units that belong to the same atlas. An atlas tile track shall contain the ACL NAL unit of at least one tile.
[0064] grammar
[0065]
[0066] Semantics
[0067] num_tiles is the number of tiles contained in this track.
[0068] tile_id specifies the tile address of a tile present in this track.
[0069] In order to correctly access volumetric media component files for presenting volumetric media, in one embodiment, a streaming client (software and / or hardware) (such as a DASH client) of a user terminal receives a manifest, such as a Media Presentation Description (MPD) file for presenting volumetric media.
[0070] like Figure 1A As shown, at 102, the method 100 is facilitated by an MPD file including one or more preselected elements for preselection of volumetric media.
[0071] Each of the one or more preselected elements may include a V3C descriptor identifying the volumetric media. The V3C descriptor may also include an identifier of the atlas data component of the volumetric media. The V3C descriptor may also include tile addresses of all tiles belonging to the atlas data component of the volumetric media.
[0072] The SupplementalProperty element with the @schemeIdUri attribute equal to "urn:mpeg:mpegI:v3c:2020:vpc" is called a V3C descriptor. At the adaptation set level of the primary adaptation set of a V3C media, there is at most one V3C descriptor.
[0073] Due to the nature of V3C content, the atlas data V3C component serves as the entry point for V3C data access. The V3C descriptor should be able to describe the atlas data V3C component represented by the adaptation set to support any flexible V3C data access, including single atlas access, multi-atlas access, and tile access.
[0074] First, to support multiple versions of the same volumetric media, the V3C descriptor should include the @vId attribute. For the case where multiple versions of the same atlas substream of the same volumetric media each have a separate adaptation set, the V3C descriptor should include the @atlas_id attribute. For the case where multiple versions of the same tile of the same atlas each have a separate adaptation set, the V3C descriptor should include the @tile_addresses attribute.
[0075] An adaptation set with such a V3C descriptor is applicable to atlas data V3C components with any sample entry type, i.e. any atlas substream carried by a V3C track or a V3C atlas tile track.
[0076] Table 1 Attributes of V3C descriptor
[0077]
[0078] Each of the one or more pre-selected elements may include properties of an atlas data component and an associated video encoding component of a pre-selected volumetric media.
[0079] For example, preselection of volumetric media in MPD uses a preselection element with a @PreselectionComponents attribute that has an id list consisting of the id of the primary adaptation set for the volumetric media followed by the ids of the adaptation sets corresponding to the video coding components.
[0080] In the case of multi-track encapsulation of V3C data, a special adaptation set of atlas information with vuh_atlas_id equal to 0x3F can be used as the main adaptation set for V3C content in the DASH manifest (MPD). If there is more than one coded atlas substream in the V3C bitstream and each atlas substream is encapsulated as a separate atlas track, a separate adaptation set can be used to signal each atlas track.
[0081] In the case of multi-track encapsulation of V3C data, if one or more tiles belonging to one atlas are encapsulated as separate atlas tile tracks, each atlas tile track can be signaled in the DASH manifest (MPD) file using a separate adaptation set.
[0082] When a V3C bitstream contains multiple atlases, each atlas bitstream can be encapsulated in a separate V3C track. Since the special atlas V3C track (i.e., atlas_id equals 0x3F) stores the V3C parameter sets of all other V3C tracks, any access to V3C content with multiple atlases should always retrieve the adaptation set representing this spatial atlas data V3C component. On the other hand, since this special atlas V3C track does not include any ACL NAL unit and corresponds to any specific video coding V3C track, any preselection element should not select the adaptation set representing this spatial atlas data V3C component.
[0083] When an atlas sub-bitstream contains multiple tiles, one or more tiles belonging to the same atlas can be encapsulated in separate atlas tile tracks. Since the atlas V3C track stores the atlas sub-bitstream parameter sets associated with the same atlas_id for all atlas tile tracks, the adaptation set representing this atlas data V3C component should always be retrieved for any access to an atlas with multiple tiles. When the 'v3t1' track is present, the atlas V3C track does not include any ACL NAL units and corresponds to any specific video coding V3C track, in which case any preselection element should not select the adaptation set representing this atlas data V3C component.
[0084] In case of preselection of V3C content with multiple atlases, any preselection element should not select the primary adaptation set representing the atlas data V3C component.
[0085] In case of preselection of one or more tiles of V3C content, any preselection element shall not select an adaptation set of atlas data V3C components representing the atlas to which the one or more tiles belong.
[0086] The V3C component video track may be represented in the file as a constrained video, which includes, for example, at least: a 2D video coding occupancy map track, a 2D video coding geometry track, and zero or more 2D video coding attribute tracks.
[0087] In the case of multi-track encapsulation of V3C data, each V3C component can be represented as a separate adaptation set in the DASH manifest (MPD) file. If a geometry or attribute component has multiple graphs, a separate adaptation set can be used to signal each graph.
[0088] To identify the type of V3C component in an adaptation set, the V3CComponent descriptor can be used. A V3CComponent is an EssentialProperty descriptor with the @schemeIdUri attribute equal to "urn:mpeg:mpegI:v3c:2020:component".
[0089] At the adaptation set level, one V3CComponent descriptor may be signaled for each V3C component present in the representation of the adaptation set.
[0090] The @value attribute of the V3CComponent descriptor may not be present. The V3CComponent descriptor may include one or more elements and attributes specified in the table.
[0091] Table 2 Elements and attributes of V3CComponent descriptor
[0092]
[0093]
[0094] like Figure 1A As shown, at 104 , the DASH client may access a volumetric media component corresponding to a preselection of volumetric media using one or more preselection elements.
[0095] Figures 2A-2C An example method 200 for accessing volumetric media is depicted.
[0096] In case of atlas-based partial access to V3C content, in addition to the atlas data V3C component and the associated video coding V3C component signaled by V3C preselection using one or more preselection elements, an adaptation set representing the entry atlas or main atlas data V3C component should also be retrieved.
[0097] The entry atlas may represent a special atlas V3C track defined according to the bitstream format (i.e., atlas_id equal to 0x3F) to store V3C parameter sets for all other V3C tracks. The entry atlas may have a pre-assigned id, e.g., atlas_id equal to 0x3F. For any access to V3C content with a multi-atlas representation, the adaptation set representing the V3C component of this spatial atlas data may be retrieved first.
[0098] In case of tile-based partial access to V3C content, in addition to the atlas data V3C component and the associated video coding V3C component signaled by V3C preselection using one or more preselection elements, an adaptation set of atlas data V3C components representing the atlas to which one or more tiles belong should be retrieved simultaneously.
[0099] like Figure 2A As shown, at 202, one or more preselection elements (syntax structures) may be located as described above, indicating preselection of volume data. At 204, a determination may be performed to check whether the bitstream format includes multi-atlas access to volume data or single-atlas access.
[0100] like Figure 2B As shown, if the bitstream is configured for multi-atlas access, a determination is made at 206 whether partial or full access to the volumetric data is available. If full access is available, at 208, access is required to the volumetric media file for the following V3C components, including all atlases, all tiles belonging to all of these atlases (if any), and all associated video coding V3C components, in order to decode and reconstruct the entire volumetric media.
[0101] Alternatively, if it is determined at 206 that the access is partial, it is determined at 210 whether the access is atlas-based or tile-based. If the access is tile-based, at 212, in order to decode and reconstruct one or more tiles of the volumetric media, it is necessary to access the volumetric media file of the following V3C components, including a specific type of atlas (e.g., atlas id is equal to 0x3F), the atlas to which the pre-selected tile belongs, the pre-selected tile, and all associated video coding V3C components.
[0102] Alternatively, at 214, in order to decode and reconstruct one or more atlases of the volumetric media, access is required to the volumetric media file of the following V3C components, including a specific type of atlas (e.g., atlas id equal to 0x3F), a pre-selected atlas or atlases, all tiles belonging to the one or more atlases (if any), and all associated video coding V3C components.
[0103] like Figure 2C As shown, if it is determined at 204 that single atlas access is used, a determination is made at 220 whether full or partial access to the volumetric data is used for scene generation. If full access to the volumetric data is used, then at 222, the atlas, all tiles (if any), and all associated volumetric media files of the video coding (V3C) component are accessed for reconstruction and scene generation. Alternatively, at 224, the atlas to which the preselected tile belongs, the preselected tile, and the corresponding video data are partially accessed for reconstruction and scene generation.
[0104] like Figure 1A As shown, volumetric media may be reconstructed using the one or more atlas data components and the associated video encoding components at 106. For example, the results of decoding the one or more atlas data components may be used along with the results of decoding the associated video encoding components.
[0105] For example, the reconstruction process may use one or more of the following. In some embodiments, a 3D scene may be encoded using a coding technique in which a 2D scene may be encoded as several 2D video images or views. A 3D scene may be represented as a collection of views, sometimes referred to as patches. The collection may be organized as an atlas, which includes one or more patches and may have corresponding content texture and / or depth information. The various components representing visual information may be formatted using a predefined format known to both the encoder and decoder sides. For example, a V3C sub-bitstream component may be used to represent a visual component. The visual component includes the aforementioned atlas, an occupancy map representing a 2D array corresponding to the atlas, so that each sample position indicates the identity of a corresponding patch containing visual information. The bitstream format may further specify that the visual component includes a specific type of geometry or attribute associated with the representation of the volumetric media, which enables the volumetric media to be reconstructed when decoded.
[0106] During reconstruction of a 3D scene, the decoder may perform the following operations:
[0107] - Parse the bitstream to recover atlas parameters and camera parameter lists.
[0108] - Localize and decode the occupancy map using the patch IDs generated from the atlas parameter list and the decoded depth atlas.
[0109] Checks whether the viewer (viewpoint) position and orientation are specified.
[0110] Decode textures and depth atlases, decode video and apply textures and depth atlases.
[0111] The 3D scene is synthesized from the decoded information mixed together according to parameters in the bitstream and the viewing position and direction.
[0112] This document provides various embodiments of the format of an MPD file or volumetric media file, which can be used by a 3D content encoder to encode 3D content, or by a 3D content decoder to decode and reconstruct 3D content from an encoded bitstream.
[0113] In order to efficiently process visual volumetric media, in one embodiment, a technical solution is provided that supports delivering V3C components belonging to an atlas of volumetric media in 3D space.
[0114] Figure 3 An exemplary DASH configuration for grouping V3C components belonging to one atlas of volumetric media within an MPEG-DASH MPD file is shown.
[0115] In this embodiment, a V3C sequence comprises a single atlas data V3C component and associated video coding components of occupancy, geometry and attributes.
[0116] The V3C track containing the atlas sub-bitstream is signaled by the primary adaptation set. The primary adaptation set has a V3C descriptor with the @vId attribute set to the identifier of the volumetric media. Since there is only one atlas within the volumetric media, @atlas_id is not present in the V3C descriptor.
[0117] Each of the video coding occupancy, geometry and attributes components is signaled by a separate component adaptation set. Each component adaptation set for the occupancy, geometry and attributes components includes a VPCCComponent descriptor with its @component_type attribute set to the corresponding value.
[0118] The atlas data components carried in the master adaptation set and the associated video coding components carried in the respective component adaptation sets are grouped together to preselect volumetric media using the preselection element in the MPD. Figure 3 As shown, the @preselectionComponents attribute of the preselection element includes the id list "1 2 3 4". The preselection element includes a V3C descriptor, which contains a @vId attribute to identify the volumetric media.
[0119] In order to be able to efficiently process visual volumetric media, in another embodiment, a technical solution is provided for a V3C component that supports partial access and delivery of individual tiles belonging to an atlas of volumetric media.
[0120] Figure 4 An exemplary DASH configuration for grouping V3C components of individual tiles belonging to an atlas of volumetric media within an MPEG-DASH MPD file is shown.
[0121] In this embodiment, a V3C sequence comprises an atlas data V3C component of a single atlas, one or more atlas data V3C components of tiles belonging to the atlas, and associated video coding components of occupancy, geometry and attributes.
[0122] The V3C track containing the atlas sub-bitstream is signaled by the main adaptation set. Each atlas tile track containing the tile sub-bitstream is signaled by a separate adaptation set. The main adaptation set and each tile adaptation set have V3C descriptors with the @vId attribute set to the identifier of the volumetric media. Each tile adaptation set has a V3C descriptor with the @tile_addresses attribute set to the tile addresses of all tiles. Since there is only one atlas within the volumetric media, @atlas_id is not present in the V3C descriptor.
[0123] Atlas data components carried in tile adaptation sets and associated video coding components carried in respective component adaptation sets are grouped together to preselect one or more tiles of volumetric media using one or more preselection elements in the MPD.
[0124] like Figure 4 As shown, for example, the @preselectionComponents attribute of one such preselection element includes the id list "2 5 6 7". The preselection element includes a V3C descriptor, which contains a @vId attribute to identify the volumetric media.
[0125] In order to be able to efficiently process visual volumetric media, in another embodiment, a technical solution is provided for supporting partial access and delivery of V3C components belonging to more than one atlas of the volumetric media.
[0126] Figure 5 An exemplary DASH configuration for grouping V3C components belonging to more than one atlas of volumetric media within an MPEG-DASH MPD file is presented.
[0127] In this embodiment, a V3C sequence includes more than one atlas data V3C component of multiple atlases and associated video coding components of occupancy, geometry and attributes.
[0128] The V3C track containing the atlas sub-bitstream for a particular atlas (e.g., atlas id equal to 0x3F) is signaled by the main adaptation set. Other V3C tracks containing atlas sub-bitstreams for other atlases are signaled by separate adaptation sets. The main adaptation set and each of the other atlas adaptation sets have a V3C descriptor with the @vId attribute set to the identifier of the volumetric media. Since there is more than one atlas in the volumetric media, @atlas_id is present in the V3C descriptor to signal the identifier of the respective atlas data component.
[0129] Atlas data components carried in respective atlas adaptation sets and associated video coding components carried in respective component adaptation sets are grouped together to preselect one or more atlases for volumetric media using one or more preselection elements in the MPD.
[0130] like Figure 5 As shown, for example, the @preselectionComponents attribute of one such preselection element includes the id list "2 5 6 7". The preselection element includes a V3C descriptor, which contains a @vId attribute to identify the volumetric media.
[0131] In order to be able to efficiently process visual volumetric media, in another embodiment, a technical solution is provided for a V3C component that supports partial access and delivery of individual tiles belonging to more than one atlas of the volumetric media.
[0132] Figure 6 An exemplary DASH configuration is presented for grouping V3C components of individual tiles belonging to more than one atlas of volumetric media within an MPEG-DASH MPD file.
[0133] In this embodiment, a V3C sequence includes more than one atlas data V3C components for multiple atlases, one or more atlas data V3C components for tiles belonging to one or more atlases, and associated video coding components of occupancy, geometry and attributes.
[0134] The V3C tracks for atlas subbitstreams that include special atlases (i.e., atlas_id equal to 0x3F) are signaled by the main adaptation set. Other V3C tracks for atlas subbitstreams that include other atlases are signaled by separate adaptation sets. Atlas tile tracks that include tile subbitstreams are signaled by separate adaptation sets. The main adaptation set, each atlas adaptation set, and each tile adaptation set have V3C descriptors with the @vId attribute set to the identifier of the volumetric media. Since there is more than one atlas in the volumetric media, @atlas_id is present in the V3C descriptor to signal the identifier of the respective atlas data component. Each tile adaptation set has a V3C descriptor with the @tile_addresses attribute set to the value of the tile addresses of all tiles.
[0135] The atlas data components carried in the respective atlas adaptation sets and tile adaptation sets and the associated video coding components carried in the respective component adaptation sets are grouped together to preselect one or more tiles from one or more atlases of the volumetric media using one or more preselect elements in the MPD. Figure 6 As shown, for example, the @preselectionComponents attribute of one such preselection element includes the id list "6 8 9 10". The preselection element includes a V3C descriptor, which contains a @vId attribute to identify the volumetric media.
[0136] The preselection mechanism uses the preselection element and V3C content-related descriptors to support partial access and delivery of volumetric media in DASH. Its usage is as follows:
[0137] Figure 7 An example process flow for preselection of volumetric media is depicted.
[0138] like Figure 7As shown in
[15] , when multiple versions of the same volumetric media are available, the @vId value of the V3C or V3CTile descriptor is used to identify the volumetric media. The DASH client should parse the MPD file to check for all preselected elements containing a V3C descriptor or V3CTile descriptor where the @vId value of the descriptor is the same as the volumetric media to be retrieved.
[0139] If such preselected element does not exist, volumetric media is not available in the current MPD file. If such preselected element exists, volumetric media is available in the current MPD file, where
[0140] If the @tag value of such a preselection element is equal to a specific value, for example, tag="0", then the preselection element shall contain a V3C descriptor to indicate preselection for volumetric media with multiple atlases. The @preselectionComponents attribute of the preselection element shall contain the id of the primary adaptation set for the particular atlas (atlas id equal to 0x3F), followed by the id of the adaptation set corresponding to each additional atlas of the volumetric media.
[0141] If the @tag value of such a preselection element is equal to another specific value, for example, tag="1", then the preselection element should contain a V3C descriptor to indicate the preselection of an atlas with multiple tiles. The @preselectionComponents attribute of the preselection element includes the id of the adaptation set of the atlas to which the multiple tiles belong, which in case of preselection of volumetric media with a single atlas is the id of the main adaptation set of the volumetric media, followed by the id of the adaptation set corresponding to each atlas tile track.
[0142] If the @tag value of such a preselection element is equal to another specific value, for example, tag="2", and if the preselection element contains a V3C descriptor, then it indicates the preselection of an atlas and its associated video coding V3C component. The @preselectionComponents attribute of the preselection element includes the id of the adaptation set of the atlas, which in the case of preselection of volumetric media with a single atlas is the id of the primary adaptation set of the volumetric media, followed by the id of the adaptation set corresponding to the video coding V3C component.
[0143] If the @tag value of such a preselection element is equal to another specific value, for example, tag="2", and if the preselection element contains a V3CTile descriptor, then it indicates the preselection of an atlas tile track and its associated video coding V3C component. The @preselectionComponents attribute of the preselection element includes the id of the adaptation set of the atlas tile track, followed by the id of the adaptation set corresponding to the video coding V3C component.
[0144] Examples of encoder, decoder, and system implementations
[0145] Figure 8 is a diagram illustrating a first example device including at least an example video encoder, picture encoder, or volumetric media encoder.
[0146] Acquisition unit 1001 captures videos and images. Acquisition unit 1001 may be equipped with one or more cameras to capture videos or images of natural scenes. Alternatively, acquisition unit 1001 may be implemented using a camera to obtain depth videos or depth images. Alternatively, acquisition unit 1001 may include components of an infrared camera. Alternatively, acquisition unit 1001 may be configured with a remote sensing camera. Acquisition unit 1001 may also be a device or apparatus that generates videos or images by scanning an object using radiation.
[0147] Optionally, the acquisition unit 1001 can perform preprocessing on the video or picture, such as automatic white balance, automatic focus, automatic exposure, backlight compensation, sharpening, denoising, stitching, upsampling / downsampling, frame rate conversion, virtual view synthesis, etc.
[0148] The acquisition unit 1001 may also receive a video or picture from another device or processing unit. For example, the acquisition unit 1001 may be a component unit in a transcoder. The transcoder feeds one or more decoded (or partially decoded) pictures to the acquisition unit 1001. Another example is that the acquisition unit 1001 obtains the video or picture from another device via a data link to the device.
[0149] Note that the acquisition unit 1001 can be used to capture other media information besides videos and pictures, such as audio signals. The acquisition unit 1001 can also receive artificial information, such as characters, text, computer-generated videos or pictures, etc.
[0150] Encoder 1002 is an example implementation of an encoder. The input of encoder 1002 is the video or picture output by acquisition unit 1001. Encoder 1002 encodes the video or picture and outputs the generated video or picture bitstream.
[0151] The storage / transmission unit 1003 receives the video or picture bitstream from the encoder 1002 and performs system layer processing on the bitstream. For example, the storage / transmission unit 1003 encapsulates the bitstream according to the transmission standard and media file format (e.g., MPEG-2 TS, ISOBMFF, DASH, MMT, etc.). The storage / transmission unit 1003 stores the encapsulated transport stream or media file in the memory or disk of the first example device, or transmits the transport stream or media file via a wired or wireless network.
[0152] Note that in addition to the video or picture bitstream from the encoder 1002, the input of the storage / transmission unit 1003 may also include audio, text, images, graphics, etc. The storage / transmission unit 1003 generates a transmission or media file by encapsulating these different types of media bitstreams.
[0153] The first example device described in this embodiment can be a device capable of generating or processing video (or picture) bit streams in video communication applications, such as a mobile phone, a computer, a media server, a portable mobile terminal, a digital camera, a broadcasting device, a CDN (content distribution network) device, a surveillance camera, a video conferencing device, etc.
[0154] Figure 9 is a diagram illustrating a second example device including at least an example video decoder, picture decoder, or volumetric media decoder.
[0155] The receiving unit 1101 receives a video or picture bit stream by obtaining the bit stream from a wired or wireless network, by reading a memory or disk in an electronic device, or by acquiring data from other devices via a data link.
[0156] The input of the receiving unit 1101 may also include a transport stream or a media file containing a video or picture bitstream. The receiving unit 1101 extracts the video or picture bitstream from the transport stream or the media file according to the specification of the transport or media file format.
[0157] Receiving unit 1101 outputs a video or picture bitstream and passes it to decoder 1102. Note that in addition to the video or picture bitstream, the output of receiving unit 1101 may also include an audio bitstream, characters, text, images, graphics, etc. Receiving unit 1101 passes the output to the corresponding processing unit in the second example device. For example, receiving unit 1101 passes the output audio bitstream to the audio decoder in the device.
[0158] Decoder 1102 is an example implementation of a decoder. The input of decoder 1102 is the video or picture bitstream output by receiving unit 1101. Decoder 1102 decodes the video or picture bitstream and outputs the decoded video or picture.
[0159] Rendering unit 1103 receives the decoded video or picture from decoder 1102. Rendering unit 1103 presents the decoded video or picture to the viewer. Rendering unit 1103 may be a component of the second example device, such as a screen. Rendering unit 1103 may also be a device separate from the second example device that has a data link to the second example device, such as a projector, a display, a television, etc. Optionally, rendering unit 1103 performs post-processing on the decoded video or picture before presenting it to the viewer, such as automatic white balance, automatic focus, automatic exposure, backlight compensation, sharpening, denoising, stitching, upsampling / downsampling, frame rate conversion, virtual view synthesis, etc.
[0160] Note that in addition to decoded video or images, the input to rendering unit 1103 can be other media data from one or more units of the second example device, such as audio, characters, text, images, graphics, etc. The input to rendering unit 1103 can also include manual data, such as lines and marks drawn by a local teacher on a slide to draw attention in a distance education application. Rendering unit 1103 combines different types of media and then presents the combination to the audience.
[0161] The second example device described in this embodiment can be a device capable of decoding or processing video (or picture) bit streams in video communication applications, such as mobile phones, computers, set-top boxes, televisions, HMDs, displays, media servers, portable mobile terminals, digital cameras, broadcasting equipment, CDN (content distribution network) equipment, monitoring, video conferencing equipment, etc.
[0162] Figure 10 It shows the inclusion Figure 8 The first example device and Figure 9 A diagram of an electronic system of a second example device in FIG.
[0163] Service device 1201 is Figure 8 The first example device in .
[0164] Storage media / transmission network 1202 can include internal memory resources of a device or electronic system, external memory resources accessible via data links, and data transmission networks consisting of wired and / or wireless networks. Storage media / transmission network 1202 provides storage resources or a data transmission network for storage / transmission unit 1203 in service device 1201.
[0165] The target device 1203 is Figure 9 The receiving unit 1201 in the target device 1203 receives a video or picture bitstream, a transport stream containing the video or picture bitstream, or a media file containing the video or picture bitstream from a storage medium / transmission network 1202 .
[0166] The electronic system described in this embodiment can be a device or system that can generate, store or transmit and decode video (or picture) bit streams in video communication applications, such as mobile phones, computers, IPTV systems, OTT systems, multimedia systems on the Internet, digital television broadcasting systems, video surveillance systems, portable mobile terminals, digital cameras, video conferencing systems, etc.
[0167] Figure 11 An example apparatus 1400 is shown that can be used to implement the encoder-side or decoder-side techniques described in this document. Apparatus 1400 includes a processor 1402 that can be configured to perform the encoder-side or decoder-side techniques, or both. Apparatus 1400 may also include a memory (not shown) for storing processor-executable instructions and for storing video bitstreams and / or display data. Apparatus 1400 may include video processing circuitry (not shown), such as transform circuitry, arithmetic encoding / decoding circuitry, lookup table-based data encoding techniques, and the like. The video processing circuitry may be partially included in the processor and / or partially included in other specialized circuitry, such as a graphics processor, a field programmable gate array (FPGA), and the like.
[0168] The technical problems described in this document regarding video encoding and decoding technology can be solved by some embodiments by preferably combining one or more of the following solutions.
[0169] 1. A volumetric media processing method (e.g., Figure 1A The method 100 shown comprises: determining (102) one or more pre-selected elements corresponding to a pre-selection of volumetric media from a media presentation description (MPD) file, accessing (104) one or more atlas data components and associated video coding components of the volumetric media using the one or more pre-selected elements; and reconstructing (106) the volumetric media from the one or more atlas data components and the associated video coding components.
[0170] 2. The method of claim 1, wherein each of the one or more pre-selected elements comprises a descriptor identifying the volumetric media and attributes pre-selecting an atlas data component and the associated video coding component of the volumetric media.
[0171] 3. The method of claim 2, wherein the descriptor identifying the volumetric media comprises an identification of the atlas data component of the volumetric media.
[0172] 4. The method of claim 2, wherein the descriptor identifying the volumetric media comprises one or more tile addresses of tiles belonging to the atlas data component of the volumetric media.
[0173] 5. The method of claim 1 , wherein, since the preselection of the volumetric media indicates access to one or more atlases of the volumetric media, the method further comprises: accessing the one or more atlas data components and all associated video coding components of the one or more atlases.
[0174] 6. A method according to Option 1, wherein, due to the preselection of the volumetric media indicating access to one or more tiles of the volumetric media, the method further comprises: accessing the one or more atlas data components of the one or more atlases to which the one or more tiles belong, the one or more atlas data components of the one or more tiles, and all associated video encoding components.
[0175] 7. The method of claim 1, 5 or 6, wherein if the preselection of the volumetric medium indicates access to the volumetric medium having a plurality of atlases: accessing the atlas data component of a primary atlas of the volumetric medium.
[0176] 8. A method according to any one of options 1 to 7, wherein the atlas data component carries the atlas data of the volumetric media and the video encoding component carries 2D video encoding data of any one of the occupancy map, geometry or attribute component video bitstream of the volumetric media.
[0177] 9. A volumetric media encoding method (e.g., comprising Figure 1B The method 150 comprises: generating (152) volumetric media from a three-dimensional scene, the volumetric media comprising one or more atlas data components representing the three-dimensional scene and an associated video encoding component; including (154) one or more preselected elements in a media presentation description (MPD) file indicating a preselection of the volumetric media, wherein the one or more preselected elements provide access to one or more atlas data components associated with the three-dimensional scene and the video encoding component.
[0178] 10. The method of claim 9, wherein each of the one or more pre-selected elements comprises a descriptor identifying the volumetric media and attributes pre-selecting an atlas data component and an associated video coding component of the volumetric media.
[0179] 11. The method of claim 8, wherein the descriptor identifying the volumetric media comprises an identification of the atlas data component of the volumetric media.
[0180] 12. The method of claim 8, wherein the descriptor identifying the volumetric media comprises one or more tile addresses of tiles belonging to the atlas data component of the volumetric media.
[0181] 13. A method according to Option 9, wherein, since the pre-selection of the volumetric media indicates access to one or more atlases of the volumetric media, the method further comprises: configuring access to the one or more atlas data components and all associated video encoding components of the one or more atlases.
[0182] 14. A method according to Option 9, wherein, due to the preselection of the volumetric media indicating access to one or more tiles of the volumetric media, the method includes: configuring access to the one or more atlas data components of the one or more atlases to which the one or more tiles belong, the one or more atlas data components of the one or more tiles, and all associated video encoding components.
[0183] 15. The method of claim 9, 13 or 14, wherein if the preselection of the volumetric medium indicates access to the volumetric medium having a plurality of atlases: configuring the atlas data component of a primary atlas of the volumetric medium.
[0184] 16. An apparatus for processing volumetric media, comprising a processor configured to implement the method described in one or more of embodiments 1 to 15.
[0185] 17. A computer program product comprising a computer-readable medium having codes stored thereon, the codes causing a processor to implement the method described in one or more of schemes 1 to 15 when the codes are executed by the processor.
[0186] The disclosed and other embodiments, the modules and functional operations described in this document can be implemented in digital electronic circuitry or in computer software, firmware, or hardware (including the structures disclosed in this document and their structural equivalents), or a combination of one or more thereof. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by a data processing apparatus or for controlling the operation of the data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials that effects a machine-readable propagated signal, or a combination of one or more thereof. The term "data processing apparatus" encompasses apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may include code that creates an execution environment for the computer program in question, such as code comprising processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a suitable receiver device.
[0187] A computer program (also referred to as a program, software, software application, script or code) can be written in any form of programming language, including compiled or interpreted languages, and the computer program can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other units that are applicable to a computing environment. A computer program does not necessarily correspond to a file in a file system. The program can be stored in a part of a file that keeps other programs or data (for example, one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordination files (for example, files that store one or more modules, subroutines, or parts of code). A computer program can be deployed to execute on a computer or on multiple computers that are located at a site or distributed across multiple sites and interconnected by a communication network.
[0188] The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by an apparatus, and the apparatus can also be implemented as special-purpose logic circuitry, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).
[0189] Processors suitable for executing computer programs include, by way of example, both general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magnetic, magneto-optical, or optical disks) for storing data, or be operatively coupled to receive data from or transfer data to, or both, the mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and storage devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated within, special-purpose logic circuitry.
[0190] Although this patent document contains many specific contents, these specific contents should not be interpreted as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described in this patent document in the context of various embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, in some cases, one or more features in a claimed combination may be deleted from that combination, and a claimed combination may point to a sub-combination or a variant of a sub-combination.
[0191] Similarly, while operations are described in a particular order in the drawings, this should not be understood as requiring that the operations be performed in the particular order or sequence shown, or that all illustrated operations be performed, in order to achieve desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.
[0192] Only some implementations and examples are described, and other implementations, improvements, and variations may be made based on what is described and illustrated in this patent document.
Claims
1. A volumetric media processing method, comprising: determining, from a Media Presentation Description (MPD) file, one or more preselected elements corresponding to a preselection of volumetric media, each of the one or more preselected elements comprising a descriptor identifying the volumetric media and attributes preselecting an atlas data component and an associated video coding component of the volumetric media; accessing one or more atlas data components and associated video encoding components of the volumetric media using the one or more preselected elements; as well as reconstructing the volumetric media from the one or more atlas data components and the associated video encoding components; The descriptor identifying the volumetric media includes an identification of an atlas data component of the volumetric media.
2. The method according to claim 1, wherein The descriptor identifying the volumetric media includes one or more tile addresses of tiles belonging to the atlas data component of the volumetric media.
3. The method according to claim 1, wherein As a result of the preselection of the volumetric media indicating access to one or more atlases of the volumetric media, the method further comprises: The one or more atlas data components and all associated video coding components of the one or more atlases are accessed.
4. The method according to claim 1, wherein As a result of the preselection of the volumetric media indicating access to one or more tiles of the volumetric media, the method further comprises: The one or more atlas data components of the one or more atlases to which the one or more tiles belong, the one or more atlas data components of the one or more tiles, and all associated video coding components are accessed.
5. The method according to claim 1, 3 or 4, wherein: If the preselection of the volumetric medium indicates access to the volumetric medium having a plurality of atlases: The atlas data component of the entry atlas of the volumetric media is accessed.
6. The method according to any one of claims 1 to 4, wherein The atlas data component carries atlas data for the volumetric media, and the video encoding component carries 2D video encoding data for any one of an occupancy map, geometry, or attribute component video bitstream for the volumetric media.
7. The method according to claim 5, wherein: The atlas data component carries atlas data for the volumetric media, and the video encoding component carries 2D video encoding data for any one of an occupancy map, geometry, or attribute component video bitstream for the volumetric media.
8. A volumetric media encoding method, comprising: generating volumetric media from a three-dimensional scene, the volumetric media comprising one or more atlas data components representing the three-dimensional scene and associated video encoding components; Including one or more preselection elements indicating a preselection of the volumetric media in a Media Presentation Description MPD file, wherein each of the one or more preselected elements comprises a descriptor identifying the volumetric media and attributes preselecting an atlas data component and an associated video coding component of the volumetric media; wherein the one or more preselected elements provide access to one or more atlas data components associated with the three-dimensional scene and the video encoding component; and The descriptor identifying the volumetric media includes an identification of an atlas data component of the volumetric media.
9. The method according to claim 8, wherein The descriptor identifying the volumetric media includes one or more tile addresses of tiles belonging to the atlas data component of the volumetric media.
10. The method according to claim 8, wherein As a result of the preselection of the volumetric media indicating access to one or more atlases of the volumetric media, the method further comprises: Access to the one or more atlas data components and all associated video encoding components of the one or more atlases is configured.
11. The method according to claim 8, wherein Due to the preselection of the volumetric media indicating access to one or more tiles of the volumetric media, the method comprises: Access is configured to the one or more atlas data components of the one or more atlases to which the one or more tiles belong, the one or more atlas data components of the one or more tiles, and all associated video coding components.
12. The method according to claim 8, 10 or 11, wherein If the preselection of the volumetric medium indicates access to the volumetric medium having a plurality of atlases: The atlas data component of the entry atlas of the volumetric media is configured.
13. An apparatus for processing volumetric media, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the method according to any one of claims 1 to 12 when executing the program.
14. A computer program product comprising a computer readable medium having codes stored thereon, the codes, when executed by a processor, causing the processor to implement the method as claimed in any one of claims 1 to 12.
Citation Information
Patent Citations
Incremental quality delivery and compositing processing
US20190364259A1
Processing video patches for three-dimensional content
WO2019202207A1