MMT signaling for streaming of Visual Volumetric Video-Based (V3C) and Geometry-Based Point Cloud (G-PCC) media.

Abstract

The present invention provides a method, system, and apparatus for streaming visual volumetric video-based coding (V3C) media and geometry-based point cloud coding (G-PCC) media. [Solution] In a communication system, a method performed by a receiving device includes receiving a first message from a transmitting device containing a list of media assets available for streaming, or one or more messages describing media assets, respectively, and sending a second message from the transmitting device indicating a request for a subset of media assets to be streamed. The requested subset of media assets is determined based on the viewport of the receiving device. The method also includes processing packets to reconstruct at least a portion of the requested subset of media assets.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of U.S. Provisional Application No. 63 / 134,038, filed on January 5, 2021, and U.S. Provisional Application No. 63 / 134,143, filed on January 5, 2021, the contents of which are incorporated herein by reference.

Background Art

[0002] High - quality three - dimensional (3D) point clouds and other visual volumetric media, such as immersive video content in which a real or virtual 3D scene is captured by a plurality of real or virtual cameras, have recently emerged as an advanced representation of immersive media.

[0003] Recent advances in the technology of capturing and rendering 3D points could enable novel applications in the fields of telepresence, virtual reality, and large-scale dynamic 3D maps. The 3D Graphics Subgroup of the ISO / IEC JTC1 / SC29 / WG11 Moving Picture Experts Group (MPEG) is currently working on developing two 3D point cloud compression (PCC) standards: a geometry-based compression standard for static point clouds and a video-based compression standard for dynamic point clouds. The goal of these standards may be to support the efficient and interoperable storage and transmission of 3D point clouds. One requirement of these standards may be to support the lossy and / or lossless coding of point cloud geometry coordinates and attributes. MPEG-I Visual is another MPEG subgroup working on developing a standard for compressing immersive video content to support 6DoF virtual walkthroughs with correct motion parallax within bounded volumes. Since both video-based point cloud compression with limited 6 degrees of freedom (6DoF) and immersive video may rely on video-encoded components, these codings of these two types of immersive media may be collectively referred to as visual volumetric video-based coding (V3C), and the same bitstream format may be used to represent their encoded information. [Overview of the project]

[0004] Methods, systems, and apparatus for streaming visual volumetric video-based coding (V3C) media and geometry-based point cloud coding (G-PCC) media are described herein. A method performed on a receiving device may include receiving a first message containing a list of media assets available for streaming from a transmitting device, or one or more messages describing each media asset. The method may further include sending a second message indicating a request for a subset of media assets to be streamed from the transmitting device. The requested subset of media assets may be determined based on the viewport of the receiving device. The method may further include receiving a Motion Picture Experts Group (MPEG) Media Transport Protocol (MMTP) packet and processing the packet to reconstruct at least a portion of the requested subset of media assets. [Brief explanation of the drawing]

[0005] A more detailed understanding can be obtained from the following description, which is given as an example in conjunction with the attached drawings, where similar reference numbers in the drawings indicate similar elements. [Figure 1A] This is a system diagram showing an exemplary communication system in which one or more disclosed embodiments may be implemented. [Figure 1B] This is a system diagram showing an exemplary wireless transmit / receive unit (WTRU) that may be used in the communication system shown in Figure 1A, according to one embodiment. [Figure 1C] This is a system diagram showing an exemplary radio access network (RAN) and an exemplary core network (CN) that may be used in the communication system shown in Figure 1A according to one embodiment. [Figure 1D]This is a system diagram showing further exemplary RAN and further exemplary CN that may be used in the communication system shown in Figure 1A according to one embodiment. [Figure 2] This is a diagram illustrating an example of a video encoder. [Figure 3] This is a diagram illustrating an example of a video encoder. [Figure 4] This figure illustrates an example of a system in which various aspects and embodiments described herein may be implemented. [Figure 5] This figure shows an example system interface between a server and a client. [Figure 6] This figure shows another exemplary system interface between a server and a client. [Figure 7] This figure shows an example of a V3C bitstream structure. [Figure 8] This table illustrates examples of supported V3C attribute types. [Figure 9] This figure shows an exemplary structure of a V3C container that can be implemented according to the ISOBMFF standard. [Figure 10] This figure shows an exemplary multi-track container with two or more Atlas and multiple Atlas styles. [Figure 11] This diagram illustrates an example of a bitstream structure. [Figure 12] This table provides an example syntax structure of a G-PCC TLV encapsulation unit. [Figure 13] This is a table providing possible values ​​and corresponding descriptions for TLV type parameters. [Figure 14] This table provides an example syntax structure for a G-PCC TLV unit payload. [Figure 15] This figure illustrates an exemplary sample structure in which a bitstream providing G-PCC geometry information and attribute information is stored on a single track. [Figure 16]It is a diagram showing an exemplary structure of a multi-track ISOBMFF G-PCC container. [Figure 17] It is a diagram depicting an exemplary end-to-end architecture of a system where MMT signaling is executed. [Figure 18] It is a diagram exemplifying a package structure according to some embodiments. [Figure 19] It is a table providing a list of defined application message types. [Figure 20] It is a table providing an exemplary syntax structure of a V3C asset descriptor. [Figure 21] It is a table exemplifying an exemplary syntax of a V3CAssetGroupMessage. [Figure 22] It is a table exemplifying exemplary V3C data type values that can be used in the Data_type field. [Figure 23] It is a table showing an exemplary syntax of a V3CSelectionMessage. [Figure 24] It is a table providing a definition of the switching_mode field. [Figure 25] It is a table exemplifying an exemplary syntax of a V3CViewChangeFeedbackMessage. [Figure 26] It is a table providing an exemplary syntax structure of a G-PCC asset descriptor. [Figure 27] It is a table exemplifying examples of defined G-PCC application message types. [Figure 28] It is a table exemplifying an exemplary syntax of a group message. [Figure 29] It is a table exemplifying exemplary G-PCC data type values that can be used in the Data_type field. [Figure 30] It is a table exemplifying an exemplary syntax of a GPCC selection feedback message. [Figure 31]A table providing the definition of the switching_mode field. [Figure 32] A table illustrating an exemplary syntax of a G-PCC view change feedback message (e.g., "GPCCViewChangeFeedback").

Best Mode for Carrying Out the Invention

[0006] FIG. 1A is a diagram showing an exemplary communication system 100 in which one or more of the disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to a plurality of wireless users. The communication system 100 may enable a plurality of wireless users to access the above-described content through sharing of system resources including wireless bandwidth. For example, the communication system 100 may use one or more channel access methods such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tail unique-word discrete Fourier transform spread OFDM (ZT-UW-DFT-S-OFDM), unique word OFDM (UW-OFDM), resource block filter type OFDM, filter bank multicarrier (FBMC).

[0007] As shown in Figure 1A, the communication system 100 may include radio transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, a radio access network (RAN) 104, a core network (CN) 106, a public switched telephone network (PSTN) 108, the internet 110, and other networks 112, but it will be understood that the disclosed embodiments intend any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, and 102d may be any type of device configured to operate and / or communicate in a radio environment. For example, WTRU102a, 102b, 102c, and 102d, all of which may be referred to as stations (STA), may be configured to transmit and / or receive radio signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscriber-based units, pagers, mobile phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearables, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated processing chain context), consumer electronic devices, and devices operating on commercial and / or industrial wireless networks. Any of WTRU102a, 102b, 102c, and 102d may interchangeably be referred to as UE.

[0008] The communication system 100 may also include base stations 114a and / or base stations 114b. Each of the base stations 114a and 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks such as CN 106, the Internet 110, and / or other networks 112. As an example, base stations 114a and 114b may be next-generation node B such as base transceiver station (BTS), node B, eNode B (eNB), home node B, home eNode B, gNode B (gNB), new radio (NR) node B, site controller, access point (AP), wireless router, etc. Although base stations 114a and 114b are shown as single elements, it will be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0009] Base station 114a may be part of RAN 104, which may also include other base stations such as a base station controller (BSC), a radio network controller (RNC), relay nodes, and / or network elements (not shown). Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies which may be referred to as cells (not shown). These frequencies may be licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of radio services to a particular geographic area which may be relatively fixed or change over time. A cell may be further divided into cell sectors. For example, a cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver per sector of the cell. In one embodiment, the base station 114a may use multiple-input multiple output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0010] Base stations 114a and 114b may communicate with one or more WTRUs 102a, 102b, 102c, and 102d via an air interface 116, which may be any suitable radio communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0011] More specifically, as described above, the communication system 100 may be a multiple access system and may use one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a of RAN 104 and WTRU 102a, 102b, 102c may implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish an air interface 116 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink Packet Access (HSDPA) and / or High-Speed ​​Uplink Packet Access (HSUPA).

[0012] In one embodiment, base stations 114a and WTRUs 102a, 102b, and 102c may implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish an air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0013] In one embodiment, the base station 114a and WTRUs 102a, 102b, and 102c may implement radio technologies such as NR radio access, which may establish an air interface 116 using NR.

[0014] In one embodiment, base station 114a and WTRU 102a, 102b, 102c may implement multiple radio access technologies. For example, base station 114a and WTRU 102a, 102b, 102c may implement LTE radio access and NR radio access together, for example, using the dual connectivity (DC) principle. Thus, the air interface utilized by WTRU 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions transmitted to and from multiple types of base stations (e.g., eNB and gNB).

[0015] In other embodiments, base stations 114a and WTRUs 102a, 102b, and 102c may implement wireless technologies such as IEEE 802.11 (i.e., Wireless Fidelity, WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access, WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), and GSM EDGE (GERAN).

[0016] The base station 114b in Figure 1A may be, for example, a wireless router, home node B, home e-node B, or access point, and may utilize any suitable RAT to facilitate wireless connectivity in local areas such as offices, homes, vehicles, campuses, industrial facilities, aerial corridors (for use by drones), roads, etc. In one embodiment, the base station 114b and WTRU 102c, 102d may implement wireless technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and WTRU 102c, 102d may implement wireless technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base stations 114b and WTRUs 102c, 102d may establish picocells or femtocells using cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.). As shown in Figure 1A, base station 114b may have a direct connection to the internet 110. Therefore, base station 114b may not need to access the internet 110 via CN 106.

[0017] RAN104 may communicate with CN106, which may be any type of network configured to provide voice, data, applications, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRU102a, 102b, 102c, and 102d. The data may have various quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, and mobility requirements. CN106 may provide call control, billing services, mobile location-based services, prepaid calls, internet connectivity, video distribution, etc., and / or high-level security functions such as user authentication. Although not shown in Figure 1A, it will be understood that RAN104 and / or CN106 may communicate directly or indirectly with other RANs using the same RAT or different RAT as RAN104. For example, in addition to being connected to RAN104 which may utilize NR radio technology, CN106 may also communicate with another RAN (not shown) using GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0018] CN106 may also function as a gateway to WTRU102a, 102b, 102c, and 102d for access to PSTN108, the Internet 110, and / or other networks 112. PSTN108 may include a public switched telephone network providing plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices, which use common communication protocols such as the transmission control protocol (TCP), the user datagram protocol (UDP), and / or the Internet protocol (IP) of the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs that may use the same RAT as RAN104 or a different RAT.

[0019] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multimode capability (for example, WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different radio networks via different radio links). For example, WTRU 102c shown in Figure 1A may be configured to communicate with base station 114a, which may use cellular-based radio technology, and base station 114b, which may use IEEE 802 radio technology.

[0020] Figure 1B is a system diagram showing an exemplary WTRU102. As shown in Figure 1B, the WTRU102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be understood that the WTRU102 may include any partial combination of the aforementioned elements while maintaining consistency with one embodiment.

[0021] The processor 118 may be a general-purpose processor, a dedicated processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to a transceiver 120 which may be coupled to a transmit / receive element 122. Figure 1B shows the processor 118 and transceiver 120 as separate components, but it will be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0022] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF signals and optical signals. It will be understood that the transmit / receive element 122 may be configured to transmit and / or receive any combination of radio signals.

[0023] Although the transmit / receive element 122 is shown as a single element in Figure 1B, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may utilize MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving radio signals via the air interface 116.

[0024] The transceiver 120 may be configured to modulate the signal transmitted by the transmit / receive element 122 and demodulate the signal received by the transmit / receive element 122. As described above, the WTRU 102 may have multimode capability. Therefore, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0025] The processor 118 of the WTRU102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light-emitting diode (OLED) display unit) and may receive user input from these. The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from any type of suitable memory, such as non-removable memory 130 and / or removable memory 132, and store data in such memory. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from memory not physically located on the WTRU 102, such as on a server or home computer (not shown), and store data in such memory.

[0026] The processor 118 may receive power from the power supply 134, but may be configured to distribute and / or control power to other components in the WTRU 102. The power supply 134 may be any suitable device for supplying power to the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), a solar cell, a fuel cell, etc.

[0027] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to or instead of the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be understood that the WTRU 102 may acquire location information by any preferred location determination method while maintaining consistency with one embodiment.

[0028] The processor 118 may be further coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functions, and / or wired or wireless connectivity. For example, peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, and the like. Peripherals 138 may include one or more sensors. The sensor may be one or more of the following: gyroscope, accelerometer, Hall effect sensor, magnetometer, orientation sensor, proximity sensor, temperature sensor, time sensor, geolocation sensor, altimeter, light sensor, touch sensor, barometer, gesture sensor, biometric sensor, humidity sensor, etc.

[0029] WTRU102 may include a full-duplex radio in which the transmission and reception of some or all of a signal (for example, associated with specific subframes of both UL (for example, for transmission) and DL (for example, for reception) may occur simultaneously and / or together. The full-duplex radio may include an interference management unit for reducing and / or substantially eliminating self-interference via hardware (e.g., chokes) or signal processing via a processor (e.g., via a separate processor (not shown) or processor 118). In one embodiment, WTRU102 may include a half-duplex radio for the transmission and reception of some or all of a signal (for example, associated with specific subframes of either UL (for example, for transmission) or DL ​​(for example, for reception)).

[0030] Figure 1C is a system diagram illustrating RAN104 and CN106 according to one embodiment. As described above, RAN104 can communicate with WTRU102a, 102b, and 102c via the air interface 116 using E-UTRA wireless technology. RAN104 can also communicate with CN106.

[0031] RAN104 may include e-nodes B160a, 160b, and 160c, but it will be understood that RAN104 may include any number of e-nodes B while maintaining consistency with one embodiment. Each of e-nodes B160a, 160b, and 160c may include one or more transceivers for communicating with WTRU102a, 102b, and 102c via the air interface 116. In one embodiment, e-nodes B160a, 160b, and 160c may implement MIMO technology. Thus, e-node B160a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU102a.

[0032] Each of the e-nodes B160a, 160b, and 160c may be associated with a specific cell (not shown) and may be configured to handle wireless resource management decisions, handover decisions, user scheduling, etc., in UL and / or DL. As shown in Figure 1C, the e-nodes B160a, 160b, and 160c may communicate with each other via the X2 interface.

[0033] The CN106 shown in Figure 1C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (PGW) 166. Although these elements are shown as part of CN106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0034] The MME162 can be connected to each of the e-nodes B162a, 162b, and 162c in RAN104 via the S1 interface and can function as a control node. For example, the MME162 may perform roles such as authenticating users of WTRU102a, 102b, and 102c, activating / deactivating bearers, and selecting gateways for specific services during the initial attachment of WTRU102a, 102b, and 102c. The MME162 may provide control plane functionality for switching between RAN104 and other RANs (not shown) employing other radio technologies such as GSM and / or WCDMA.

[0035] The SGW164 can be connected to each of the e-nodes-B160a, 160b, and 160c in RAN104 via the S1 interface. The SGW164 can generally route and forward user data packets to and from WTRU102a, 102b, and 102c. The SGW164 can perform other functions, such as anchoring the user plane during e-node-B handovers, triggering paging when DL data is available to WTRU102a, 102b, and 102c, and managing and remembering the context of WTRU102a, 102b, and 102c.

[0036] SGW164 may be connected to PGW166, which may provide WTRU102a, 102b, and 102c with access to a packet-switched network such as the Internet 110 to facilitate communication between WTRU102a, 102b, and 102c and IP-enabled devices.

[0037] CN106 can facilitate communication with other networks. For example, CN106 can provide WTRU102a, 102b, and 102c with access to a circuit-switched network such as PSTN108 to facilitate communication between WTRU102a, 102b, and 102c and conventional terrestrial line communication devices. For example, CN106 may include or communicate with an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that acts as an interface between CN106 and PSTN108. In addition, CN06 may provide WTRU102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0038] Although the WTRU is shown as a wireless terminal in Figures 1A to 1D, in certain representative embodiments, such a terminal is intended to be able to use a wired communication interface (e.g., temporary or permanent) with a communication network.

[0039] In a typical embodiment, the other network 112 may be a WLAN.

[0040] A WLAN in Infrastructure Basic Service Set (BSS) mode may have access points (APs) of the BSS and one or more stations (STAs) associated with the APs. APs may have access to or interfaces with a Distribution System (DS) or another type of wired / wireless network that carries traffic within and / or outside the BSS. Traffic originating outside the BSS and destined for an STA may reach and be delivered to the STA via an AP. Traffic originating from an STA to a destination outside the BSS may be sent to an AP and then delivered to its respective destination. Traffic between STAs within the BSS may be transmitted, for example, via an AP; a source STA may send traffic to an AP, and the AP may deliver the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic may be transmitted between a source STA and a destination STA (for example, directly between them) via a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunneled DLS (TDLS). A WLAN using Independent BSS (IBSS) mode may not have APs, and STAs within or using IBSS (e.g., all STAs) may communicate directly with each other. The IBSS mode of communication may be referred to herein as “ad hoc” communication mode.

[0041] When using the 802.11ac infrastructure operating mode or a similar operating mode, an AP may transmit beacons on a fixed channel, such as the primary channel. The primary channel may have a fixed width (e.g., a 20 MHz bandwidth) or a dynamically set width. The primary channel may be the operating channel of the BSS and may be used by the STA to establish a connection with the AP. In certain typical embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA / CA) may be implemented, for example, in an 802.11 system. In the case of CSMA / CA, the STA, including the AP (e.g., all STAs), may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, that STA may be backed off. A single STA (e.g., only one station) may transmit at any given time on a given BSS.

[0042] High-throughput (HT) STAs may use a 40 MHz wide channel for communication, which may be formed, for example, through a combination of a primary 20 MHz channel and adjacent or non-adjacent 20 MHz channels.

[0043] Very High Throughput (VHT) STAs can support channels with widths of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz. The 40 MHz and / or 80 MHz channels can be formed by combining consecutive 20 MHz channels. A 160 MHz channel can be formed by combining eight consecutive 20 MHz channels, or by combining two non-contiguous 80 MHz channels, which may be referred to as an 80+80 configuration. In the 80+80 configuration, after channel coding, the data can pass through a segment parser that can split the data into two streams. Inverse Fast Fourier Transform (IFFT) and time-domain processing can be performed separately for each stream. The streams may be mapped to two 80 MHz channels, and the data can be transmitted by a transmitting STA. At the receiver of a receiving STA, the operation described above for the 80+80 configuration may be reversed, and the combined data may be transmitted to Medium Access Control (MAC).

[0044] Sub-1 GHz operating modes are supported by 802.11af and 802.11ah. Channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports bandwidths of 5 MHz, 10 MHz, and 20 MHz in the TV White Space (TVWS) spectrum, while 802.11ah supports bandwidths of 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz using the non-TVWS spectrum. According to a typical embodiment, 802.11ah may support meter-type control / machine-type communications (MTC), such as MTC devices in a macro coverage area. MTC devices may have certain capabilities, including support for specific and / or limited bandwidths (e.g., support only for that). MTC devices may include batteries with battery life exceeding a threshold (e.g., to maintain very long battery life).

[0045] A WLAN system capable of supporting multiple channels and channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah includes a channel that can be designated as the primary channel. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by an STA from among all STAs operating in a BSS that support the minimum bandwidth operating mode. In the 802.11ah example, the primary channel may be 1 MHz wide for an STA (e.g., an MTC type device) that supports (e.g., only) the 1 MHz mode, even if other STAs in the AP and BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or Network Allocation Vector (NAV) settings may depend on the state of the primary channel. For example, if the primary channel is busy, an STA (which only supports 1MHz operating mode) sending to the AP may consider the entire available frequency band to be busy, even if most of the available frequency band is idle.

[0046] In the United States, the available frequency band that can be used by 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. The total bandwidth available for 802.11ah is 6MHz to 26MHz, depending on the country code.

[0047] Figure 1D is a system diagram illustrating RAN104 and CN106 according to one embodiment. As described above, RAN104 can communicate with WTRU102a, 102b, and 102c via the air interface 116 using NR radio technology. RAN104 can also communicate with CN106.

[0048] RAN104 may include gNB180a, 180b, and 180c, but it will be understood that RAN104 may include any number of gNBs while maintaining consistency with one embodiment. Each of gNB180a, 180b, and 180c may include one or more transceivers for communicating with WTRU102a, 102b, and 102c via the air interface 116. In one embodiment, gNB180a, 180b, and 180c may implement MIMO technology. For example, gNB180a and 180b may use beamforming to transmit and / or receive signals to gNB180a, 180b, and 180c. Thus, gNB180a may, for example, use multiple antennas to transmit and / or receive radio signals from WTRU102a. In one embodiment, gNB180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB180a may transmit multiple component carriers to WTRU102a (not shown). A subset of these component carriers may be on the unauthorized spectrum, and the remaining component carriers may be on the authorized spectrum. In one embodiment, gNB180a, 180b, and 180c may implement coordinated multi-point (CoMP) technology. For example, WTRU102a may receive coordinated transmissions from gNB180a and gNB180b (and / or gNB180c).

[0049] WTRU102a, 102b, and 102c may communicate with gNB180a, 180b, and 180c using transmissions associated with an expandable numerology. For example, OFDM symbol intervals and / or OFDM subcarrier intervals may vary for different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRU102a, 102b, and 102c may communicate with gNB180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of varying or expandable lengths (e.g., varying numbers of OFDM symbols and / or varying durations of absolute time).

[0050] gNB180a, 180b, and 180c can be configured to communicate with WTRU102a, 102b, and 102c in standalone and / or non-standalone configurations. In a standalone configuration, WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c without accessing other RANs (e.g., e-nodes B160a, 160b, and 160c). In a standalone configuration, WTRU102a, 102b, and 102c can utilize one or more of gNB180a, 180b, and 180c as mobility anchor points. In a standalone configuration, WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using signals in unlicensed bands. In a non-standalone configuration, WTRU102a, 102b, and 102c can communicate with and connect to gNB180a, 180b, and 180c, while also communicating with and connecting to other RANs such as enodes B160a, 160b, and 160c. For example, WTRU102a, 102b, and 102c can implement DC principles for substantially simultaneous communication with one or more gNB180a, 180b, and 180c and one or more enodes B160a, 160b, and 160c. In a non-standalone configuration, e-nodes B160a, 160b, and 160c can function as mobility anchors for WTRU102a, 102b, and 102c, while gNB180a, 180b, and 180c can provide additional coverage and / or throughput to service WTRU102a, 102b, and 102c.

[0051] Each of the gNB180a, 180b, and 180c may be associated with a specific cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slice support, interaction between DC, NR and E-UTRA, routing of user plane data to User Plane Functions (UPFs) 184a and 184b, routing of control plane information to Access and Mobility Management Functions (AMFs) 182a and 182b, and so on. As shown in Figure 1D, the gNB180a, 180b, and 180c may communicate with each other via the Xn interface.

[0052] The CN106 shown in Figure 1D may include at least one AMF182a, 182b, at least one UPF184a, 184b, at least one Session Management Function (SMF)183a, 183b, and possibly a Data Network (DN)185a, 185b. Although the aforementioned elements are shown as part of CN106, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0053] AMF182a and 182b may be connected to one or more of gNB180a, 180b, and 180c in RAN104 via the N2 interface and may function as control nodes. For example, AMF182a and 182b may play roles such as user authentication for WTRU102a, 102b, and 102c, support for network slicing (e.g., handling different protocol data unit (PDU) sessions with different requirements), selection of SMF183a and 183b for registration, management of registration areas, termination of non-access stratum (NAS) signaling, and mobility management. Network slicing may be used by AMF182a and 182b to customize CN support for WTRU102a, 102b, and 102c based on the type of service utilizing WTRU102a, 102b, and 102c. For example, different network slices may be established for different use cases, such as services that rely on ultra-reliable low latency (URLLC) access, services that rely on enhanced massive mobile broadband (eMBB) access, and services for MTC access. AMF182a, 182b may provide control plane functionality for switching between RAN104 and other RANs (not shown) using other radio technologies such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as WiFi.

[0054] SMF183a and 183b may be connected to AMF182a and 182b in CN106 via the N11 interface. SMF183a and 183b may also be connected to UPF184a and 184b in CN106 via the N4 interface. SMF183a and 183b may select and control UPF184a and 184b and configure the routing of traffic through UPF184a and 184b. SMF183a and 183b may perform other functions such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing DL data notifications. PDU session types may be IP-based, non-IP-based, Ethernet-based, etc.

[0055] UPF184a and 184b may be connected via the N3 interface to one or more of gNB180a, 180b, and 180c in RAN104, thereby providing WTRU102a, 102b, and 102c with access to a packet-switched network such as the Internet 110 to facilitate communication between WTRU102a, 102b, and 102c and IP-enabled devices. UPF184 and 184b may perform other functions such as packet routing and forwarding, enforcement of user plane policies, support for multi-homed PDU sessions, processing of user plane QoS, buffering of DL packets, and mobility anchoring.

[0056] CN106 can facilitate communication with other networks. For example, CN106 may include or communicate with an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that functions as an interface between CN106 and PSTN108. In addition, CN06 may provide WTRU102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRU102a, 102b, 102c may be connected to local DN185a, 185b via UPF184a, 184b through N3 interfaces to UPF184a, 184b and N6 interfaces between UPF184a, 184b and DN185a, 185b.

[0057] With regard to Figures 1A-1D and the corresponding descriptions in Figures 1A-1D, one or more of the functions described herein with respect to one or more of the WTRU102a-d, base stations 114a-b, e-nodes B160a-c, MME162, SGW164, PGW166, gNB180a-c, AMF182a-b, UPF184a-b, SMF183a-b, DN185a-b, and / or any other devices described herein may be performed by one or more emulation devices (not shown). An emulation device may be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0058] Emulation devices may be designed to implement testing of one or more other devices in a laboratory and / or operator network environment. For example, one or more emulation devices may perform one or more or all functions while fully or partially implemented and / or deployed as part of a wired and / or wireless network to test other devices in a communications network. One or more emulation devices may perform one or more or all functions while temporarily implemented / deployed as part of a wired and / or wireless network. Emulation devices may be directly coupled to another device for the purpose of testing and / or performing testing using over-the-air wireless communication.

[0059] One or more emulation devices may perform one or more functions, including all of the above, while not implemented / deployed as part of a wired and / or wireless communication network. For example, an emulation device may be used in a test laboratory test scenario, and / or in a wired and / or wireless communication network that is not deployed (e.g., for testing purposes), to implement testing of one or more components. One or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation device to transmit and / or receive data.

[0060] Various methods and other embodiments described herein may be used, for example, to modify modules of a video encoder 200 and a decoder 300, as shown in Figures 2 and 3. Furthermore, the subject matter disclosed herein presents embodiments that are not limited to V3C, G-PCC, and may be applied, for example, to any type, format, or version of video coding, whether existing or future, whether described in a standard or recommendation, and to any extensions of such standards and recommendations (including, for example, V3C and G-PCC). Unless otherwise indicated or technically excluded, embodiments described herein may be used individually or in combination.

[0061] The examples described in this application use various numerical values, such as the number of bits reserved for fields in V3C application messages or G-PCC application messages. These and other specific values ​​are for illustrative purposes only, and the embodiments described are not limited to these specific values.

[0062] Figure 2 shows an example of a video encoder. While various modifications of the exemplary encoder 200 are intended, the encoder 200 is described below for clarity without describing all possible modifications.

[0063] Before encoding, the video sequence may undergo pre-encoding processing (201), such as applying a color conversion to the input color picture (e.g., converting from RGB4:4:4 to YCbCr4:2:0), or performing a remapping of the input picture components to obtain a signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing, and such metadata may be attached to the bitstream.

[0064] In encoder 200, the picture may be encoded by encoder elements as described below. The picture to be encoded may be divided (202) and processed in units of coding units (CUs), for example. Each unit may be encoded using either intra-mode or inter-mode, for example. When a unit is encoded in intra-mode, the unit performs intra-prediction (260), while in inter-mode, motion estimation (275) and motion compensation (270) are performed. The encoder may decide whether to use intra-mode or inter-mode to encode a unit (205), and the intra / inter decision may be indicated, for example, by a prediction mode flag. The prediction residual may be calculated, for example, by subtracting the predicted blocks from the original image blocks (210).

[0065] Next, the predicted residual may be transformed (225) and quantized (230). The quantized transformation coefficients, as well as the motion vector and other syntax elements, may be entropy coded to output a bitstream (245). The encoder may skip the transformation and apply quantization directly to the untransformed residual signal. The encoder may bypass both the transformation and quantization, i.e., the residual is coded directly without applying either the transformation or quantization process.

[0066] The encoder decodes the encoded blocks to provide a reference for further prediction. The quantized transformation coefficients are inversely quantized (240) and inversely transformed (250) to decode the prediction residuals. Combining the decoded prediction residuals with the prediction blocks (255) reconstructs the image blocks. The reconstructed picture is then subjected to an ln-loop filter (265) to reduce encoding artifacts, for example, to perform deblocking / SAO (sample adaptive offset) filtering. The filtered image is stored in a reference picture buffer (280).

[0067] Figure 3 shows an embodiment of a video decoder. In the exemplary decoder 300, the bitstream is decoded by the decoder elements as described below, and the video decoder 300 generally performs an encoded path and a decoded path in reverse, as described in Figure 2. The encoder 200 also generally performs video decoding as part of encoding the video data. In particular, the input to the decoder may include a video bitstream that can be generated by the video encoder 200. The bitstream may first be entropy-decoded to obtain transformation coefficients, motion vectors, and other coded information (330). Picture segmentation information indicates how the picture is segmented, and therefore the decoder may segment the picture according to the decoded picture segmentation information (335). The transformation coefficients are inversely quantized (340) and inversely transformed (350) to decode the predicted residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct the image block, and the prediction block may be obtained from intra-prediction (360) or motion-compensated prediction (i.e., inter-prediction) (375) (370). An intra-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0068] The decoded picture may undergo further post-decoded processing (385), such as inverse color transformation (e.g., conversion from YCbCr4:2:0 to RGB4:4:4), or inverse remapping, which performs the reverse of the remapping process performed in pre-encoded processing (201). The post-decoded processing may use metadata derived in pre-encoded processing and signaled in the bitstream.

[0069] Figure 4 shows an example of a system in which various aspects and embodiments described herein may be implemented. System 400 may be embodied as a device including various components described below and may be configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected consumer electronics, and servers. The elements of System 400 may be embodied individually or in combination as a single integrated circuit (IG), multiple ICs, and / or separate components. For example, in at least one example, the processing and encoder / decoder elements of System 400 are distributed across multiple ICs and / or separate components, and in various embodiments, System 400 is communicably coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, System 400 is configured to implement one or more of the aspects described herein.

[0070] System 400 includes, for example, at least one processor 410 configured to execute instructions loaded therein to implement various embodiments described in this document, the processor 410 may include embedded memory, input / output interfaces, and various other circuits as known in the Art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440, which may include, but is not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, magnetic disk drives, and / or optical disk drives, as well as non-volatile and / or volatile memory. The storage device 440 may, in non-limiting examples, include internal storage devices, mounted storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0071] System 400 includes, for example, an encoder / decoder module 430 configured to process data and provide encoded or decoded video, the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents a module that may be included in the device to perform encoding and / or decoding functions. As is known, the device may include one or both of the encoding module and the decoding module. In addition, the encoder / decoder module 430 may be implemented as a separate element of System 400 or may be incorporated into the processor 410 as a combination of hardware and software, as is known to those skilled in the art.

[0072] Program code to be loaded into the processor 410 or encoder / decoder 430 to perform various embodiments described herein may be stored in the storage device 440 and then loaded into the memory 420 for execution by the processor 410. According to various embodiments, one or more of the processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more of various items during the execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video, or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, actions, and operational logic.

[0073] In some embodiments, internal memory of the processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory of the processing device (e.g., the processing device may be either the processor 410 or the encoder / decoder module 430) may be used for one or more of these functions. The external memory may be memory 420 and / or storage device 440, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, external non-volatile flash memory is used to store, for example, the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video coding and video decoding operations such as MPEG-2. MPEG refers to the Moving Picture Experts Group, and MPEG-2 may be called ISO / IEC 13818. ISO / IEC 13818-1 is also known as H.222, and 13818-2 is H.262), HEVG (HEVC, also known as H.265 and MPEG-H Part 2, refers to High Efficiency Video Coding), or VVC (Multipurpose Video Coding, a new standard being developed by JVET, the Joint Video Expert Team).

[0074] Inputs to the elements of System 400 may be provided through various input devices, as shown in Block 445. Such input devices include, but are not limited to, (i) a Radio Frequency (RF) section for receiving RF signals transmitted over the entire broadcast by a broadcaster, (ii) a Component (COMP) input terminal (or set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other embodiments, not shown in Figure 4, may include composite video.

[0075] In various embodiments, the input devices of block 445 may have associated input processing elements known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower frequency band in order to select a signal frequency band that may be referred to as a channel in a particular embodiment, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) multiplexing to select a desired stream of data packets. The RF portion of various embodiments may include one or more elements for performing these functions, e.g., frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion may include tuners for performing various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to the baseband) or the baseband. In one set-top box embodiment, the RF section and its associated input processing element receive an RF signal transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. In various embodiments, the order of the elements described above (and others) may be rearranged, some of these elements may be removed, and / or other elements performing similar or different functions may be added. Adding elements may include, for example, inserting elements between existing elements, such as amplifiers and analog-to-digital converters, and in various embodiments, the RF section may include an antenna.

[0076] Furthermore, the USB and / or HDMI terminals may include their respective interface processors for connecting the system 400 to other electronic devices via USB and / or HDMI connections, and it should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be performed, for example, within a separate input processing unit 1C or, if necessary, within the processor 410. Similarly, aspects of USB or HDMI interface processing may be implemented, if necessary, within separate interface ICs or within the processor 410. Demodulated, error-corrected, and multiplexed streams are provided to various processing elements, for example, the processor 410, and an encoder / decoder 430, which operates in conjunction with memory and storage elements to process the data stream as necessary for presentation on an output device.

[0077] Various elements of system 400 may be provided within an integrated housing. Within the integrated housing, the various elements are interconnected and can transmit data between them using an appropriate connection array 425, for example, an internal bus known in the art, including an Inter-IC (I2C) bus, wiring, and a printed circuit board.

[0078] System 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or a network card, and the communication channel 460 may be implemented, for example, in a wired and / or wireless medium.

[0079] In various embodiments, the data may be streamed to the system 400 or otherwise provided using a wireless network such as a Wi-Fi network, for example, IEEE 802.11 (IEEE stands for Institute of Electrical and Electronics Engineers). The Wi-Fi signals in these embodiments are received via a communication channel 460 and a communication interface 450 adapted for Wi-Fi communication. The communication channel 460 in these embodiments is typically connected to an access point or router that provides access to an external network, including the Internet, to enable streaming applications and other over-the-top communications. In other embodiments, streaming data is provided to the system 400 using a set-top box that distributes data via an HDMI connection on the input block 445. Yet another embodiment provides the streamed data to the system 400 using an RF connection on the input block 445. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments may use wireless networks other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0080] System 400 can provide output signals to various output devices, including a display 475, a speaker 485, and other peripheral devices 495. In various embodiments, the display 475 includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 475 may be for a television, tablet, laptop, mobile phone, or other device. The display 475 may also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop). In various examples of embodiments, the other peripheral devices 495 include one or more of a standalone digital video disc (or digital multi-purpose disc) (DVR in both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 495 that provide functions based on the output of System 400. For example, a disc player performs the function of playing back the output of System 400.

[0081] In various embodiments, control signals may be communicated between the system 400 and the display 475, speaker 485, or other peripheral devices 495 using signaling such as AVLink, Consumer Electronics Control (CEC), or other communication protocols, enabling inter-device control with or without user intervention. Output devices may be communicably coupled to the system 400 via dedicated connections through their respective interfaces 470, 480, and 490. Alternatively, output devices may be connected to the system 400 using a communication channel 460 via a communication interface 450. The display 475 and speaker 485 may be integrated into a single unit with other components of the system 400 in an electronic device such as a television, and in various embodiments, the display interface 470 may include a display driver, such as a timing controller (TCon) chip.

[0082] The display 475 and speaker 485 can, alternatively, be isolated from one or more of the other components, for example, if the RF portion of input 445 is part of a separate set-top box. In various embodiments where the display 475 and speaker 485 are external components, the output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0083] The embodiments may be implemented by a processor 410, by hardware, or by computer software implemented by a combination of hardware and software. In a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 420 may be of any type appropriate for the technical environment, and in a non-limiting example, may be implemented using any suitable data storage technology such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 410 may be of any type appropriate for the technical environment, and in a non-limiting example, may include one or more of a microprocessor, a general-purpose computer, a dedicated computer, and a processor based on a multi-core architecture.

[0084] Various implementations involve decoding. As used in this application, “decoding” can encompass all or part of the processes performed on, for example, a received encoded sequence to produce a final output suitable for display, and in various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also include, or alternatively, processes performed by the decoder in the various implementations described in this application, such as decoding a portion of a coded point cloud sequence (e.g., encapsulated in an ISOBMFF container using one or more file format structures, such as those disclosed herein) to provide partial access to the coded point cloud sequence (e.g., encapsulated in an ISOBMFF container).

[0085] In further embodiments, in some examples, “decoding” may refer only to entropy decoding; in other embodiments, “decoding” may refer only to differential decoding; and in other embodiments, “decoding” may refer to a combination of entropy decoding and differential decoding. Whether the term “decoding process” is intended to refer specifically to a subset of operations or to a broader decoding process in general will be evident from the context of the particular description and should be well understood by those skilled in the art.

[0086] Various implementations involve encoding. Similar to the above consideration of "decoding," "encoding" as used in this application may encompass all or part of the processes performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, such as partitioning, differential coding, transformation, quantization, and entropy coding. In various embodiments, such processes may similarly or alternatively include the processes performed by the encoders of the various embodiments described in this application, such as encoding a video-based point cloud bitstream that includes one or more file format structures (such as those disclosed herein) to provide partial access support to different parts of an encoded point cloud sequence (e.g., encapsulated in an ISOBMFF container).

[0087] As further examples, in one embodiment, “encoding” refers only to entropy coding; in another embodiment, “encoding” refers only to differential coding; and in yet another embodiment, “encoding” refers to a combination of differential coding and entropy coding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or to a broader encoding process in general will become clear from the context of the particular description and should be easily understood by those skilled in the art.

[0088] Please note that the syntax elements used herein, such as V3CSelectionMessage, V3CAssetGroupMessage, and V3CViewChangeFeedbackMessage, are descriptive terms. Therefore, they do not preclude the use of other syntax element names.

[0089] If a diagram is presented as a flowchart, it should be understood that the diagram also provides a block diagram of the corresponding device. Similarly, if a diagram is presented as a block diagram, it should be understood that the diagram also provides a flowchart of the corresponding method / process.

[0090] The implementations and embodiments described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if considered only in the context of a single implementation (for example, considered only as a method), the implementations of the considered features may also be implemented in other forms (for example, apparatus or programs). Apparatus may be implemented, for example, with appropriate hardware, software, and firmware. The method may be implemented, for example, in a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor may also include communication devices, such as, for example, computers, mobile phones, personal digital assistants (PDAs), and other devices that facilitate the communication of information between end users.

[0091] The terms “one embodiment,” “embodiment,” “example,” “one implementation,” or “implementation,” as well as other variations thereof, mean that certain features, structures, characteristics, etc., described in relation to an embodiment are included in at least one embodiment. Therefore, the appearance of the phrases “in one embodiment,” “in an example,” “in one implementation,” or “in an implementation,” as well as any other variations thereof, appearing in various places throughout this application, does not necessarily all refer to the same embodiment or example.

[0092] Furthermore, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of the following: estimating information, calculating information, predicting information, or retrieving information from memory. Obtaining information may include receiving, retrieving, constructing, generating, and / or determining.

[0093] Furthermore, this application may refer to accessing various parts of information. Accessing information may include, for example, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, inferring information, or estimating information.

[0094] In addition, this application may refer to "receiving" various types of information. Receiving is intended to be a broad term, similar to "accessing." Receiving information may include, for example, accessing information or retrieving information (for example, from memory). Furthermore, "receiving" typically involves, in some or other way, in operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0095] Please understand that the use of "and / or" and "for example, "A / B", "A and / or B", and "at least one of A and B" below is intended to encompass the selection of only the first listed option (A), only the second listed option (B), or both options (A and B). As further examples, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such expressions are intended to encompass the selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or the selection of all three options (A, B, and C). This can be extended to the number of listed items, as will be apparent to those skilled in the art in this and related fields.

[0096] Furthermore, as used herein, the word “signaling” refers, among other things, to indicating something to the corresponding decoder. In some embodiments, the encoder may signal (e.g., in an encoded bitstream and / or encapsulated file such as an ISOBMFF container) such as a parameter set, SEI messages, metadata, edit lists, post-decoder requirements, signals enabling flexible partial access to different parts of the encoded point cloud sequence encapsulated in the ISOBMFF container, dependency lists for each signaled object, mapping to spatial domains, 3D bounding box information, etc. Thus, in one embodiment, the same parameters are used on both the encoder and decoder sides. For example, the encoder can send certain parameters to the decoder (explicit signaling) so that the decoder can use the same particular parameters. Conversely, if the decoder already has certain parameters as well as other parameters, signaling can be used to allow the decoder to know and select certain parameters simply without transmission (implicit signaling). It should be understood that bit saving is achieved in various embodiments by avoiding the transmission of any actual functions, and signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc., are used in various embodiments to signal information to the corresponding decoder. The above relates to the verb form of the word “signaling,” but the word “signal” may also be used as a noun in this specification.

[0097] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the implementations described. For example, a signal may be formatted to carry a bitstream of the embodiment described. Such a signal may be formatted, for example, as an electromagnetic wave (using, for example, the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The signal carried by the signal may be, for example, analog or digital information. The signal may be transmitted by various different wired or wireless links, as is known. The signal may be stored in a processor-readable medium.

[0098] Capturing and rendering three-dimensional (3D) images (e.g., using 3D point clouds) can have many applications, such as telepresence, virtual reality, and large-scale dynamic 3D maps. 3D point clouds may be used to represent immersive media. A 3D point cloud may contain a set of points represented in 3D space. Each point may contain coordinates and / or one or more attributes. Coordinates may indicate the location of each point. Attributes may include one or more of the following associated with each point: color, transparency, acquisition time, laser or material properties, etc. Point clouds may be captured or unfolded in several ways. Point clouds can be captured or unfolded using, for example, multiple cameras and depth sensors, light detection and ranging (LIDAR) laser scanners, etc., to sample 3D space. Points (e.g., represented by coordinates and / or attributes) can be generated, for example, by sampling objects in 3D space. A point cloud can contain multiple points, each of which may be represented by a set of coordinates (e.g., x, y, z coordinates) that map to 3D space. For example, a 3D object or scene may be represented or reconstructed as a point cloud containing millions or billions of sampled points. A 3D point cloud can represent a static and / or dynamic (moving) 3D scene.

[0099] Point cloud data can be represented and / or compressed (e.g., point cloud compression (PCC)) for, for example, to store and / or transmit the point cloud data (e.g., efficiently). For example, geometry-based compression can be used to encode and decode static point clouds, and video-based compression can be used to encode and decode dynamic point clouds, in order to support efficient and interoperable storage and transmission of 3D point clouds. Point cloud sampling, representation, compression, and / or rendering can support lossy coding and / or lossless coding (e.g., encoding or decoding) of the geometric coordinates and / or attributes of the point cloud.

[0100] Figure 5 shows the system interface 500 for server 502 and client 510. Server 502 may be a point cloud server connected to the Internet 504 and other networks 506. Client 510 may also be connected to the Internet 504 and other networks 506 and enable communication between nodes (e.g., server 502 and client 510). Each node may include a processor, a non-temporary computer-readable memory storage medium, and executable instructions stored in the storage medium that are executable by the processor to perform the method or part of the method disclosed herein. One or more nodes may further include one or more sensors. Client 510 may include a graphics processor 512 for rendering 3D video for a display such as a head-mounted display (HMD) 508 (e.g., may also include a graphics processor 512). Any or all nodes may have a WTRU and communicate over the network as described above with respect to Figures 1A to 1D.

[0101] Figure 6 shows a system interface 600 for a server 602 and a client 604. The server 602 may be a point cloud content server 602 and may include a database of point cloud content, logic for processing detail levels, and server management functions. In some embodiments, processing detail may reduce the resolution for transmission to the client 604 (e.g., a viewing client 604) so ​​that it is possible due to bandwidth limitations or because the viewing distance is sufficient to allow reduction. The point cloud content server 602 may communicate with the client 604 and exchange point cloud data and / or point cloud metadata. In some examples, the point cloud data rendered for the viewer may undergo a data construction process to reduce and / or increase the level of detail from the point cloud data and / or point cloud metadata (e.g., streamed from the point cloud server 602 to the viewing client 604), etc. The point cloud server 602 may stream the point cloud data at the resolution provided by the spatial capture, or in some embodiments, it may downsample to comply with bandwidth constraints or viewing distance tolerances, for example. The point cloud server 602 may dynamically reduce the level of detail, and in some examples, the point cloud server 602 may segment the point cloud data (for example, similarly) to identify objects within the point cloud. In some examples, points in the point cloud data corresponding to selected objects may be replaced with lower-resolution data.

[0102] A client 604 (for example, a client 604 with an HMD) may request a portion and / or tile of a point cloud from a point cloud content server 602 via a bitstream, for example, a video-based point cloud compression (V-PCC) encoded bitstream. For example, a portion and / or tile of a point cloud may be retrieved based on the location and / or orientation of the HMD.

[0103] A point cloud can consist of a set of points represented in 3D space using coordinates that indicate the position of each point, along with one or more attributes associated with each point, such as color, transparency, acquisition time, laser reflectivity, or material properties. Point clouds may be captured in several ways. For example, one technique for capturing a point cloud may use multiple cameras and depth sensors. Light detection and ranging (LiDAR) laser scanners may also be used to capture point clouds. The number of points required to realistically reconstruct objects and scenes using a point cloud can be in the millions (or even billions). Therefore, efficient representation and compression may be essential for storing and transmitting point cloud data. Similar to point clouds, some immersive video types may also be capable of representing visual volumetric content, for example, with 6 degrees of freedom (6DoF), which may be able to provide support for playback of 3D scenes within a limited range of viewing positions and orientations.

[0104] As substantially explained in the paragraph above, at least two 3D point cloud compression (PCC) standards are proposed: a geometry-based compression standard for static point clouds and a video-based compression standard for dynamic point clouds. Regarding the video-based compression standard for dynamic point clouds, Visual Volumetric Video-Based Coding (V3C) is one example, and various forms of V3C-based implementations can be described as follows.

[0105] Figure 7 shows an example of the structure of an exemplary V3C bitstream. As shown in Figure 7, the bitstream may include a V3C sample stream 701, which may include a set of V3C units, each having a V3C unit header and a V3C unit payload. The V3C unit header may describe a V3C unit type. For example, V3C unit types may include V3C_OVD, V3C_GVD, and / or V3C_AVD. V3C units having unit types V3C_OVD, V3C_GVD, and V3C_AVD may be an occupied video data unit, a geometry attribute video data unit, and an attribute video data unit, respectively. These data units may represent three main components required to reconstruct visual volumetric media content. The occupied V3C unit payload, geometry V3C unit payload, and attribute V3C unit payload may correspond to video data units (e.g., NAL units) that can be decoded by a suitable video decoder. The V3C bitstream may also include one or more V3C_VPS units, which may provide a set of parameters defining syntax elements that may be used in the V3C unit header. The V3C bitstream may further include an atlas sub-bitstream (e.g., indicated by the V3C unit header V3C_AD), which may carry a Network Abstraction Layer (NAL) sample stream 702, which includes a unit containing at least a NAL unit header and a unit encapsulating data that defines (or partially defines) the encoded atlas. For example, as shown in Figure 7, the NAL unit may include a payload for an atlas style group layer 703 (e.g., a raw byte sequence payload (RBSP)) corresponding to an atlas style group, which may include a header and data describing a patch (i.e., a region in the atlas associated with volumetric information).

[0106] Figure 8 is a table illustrating examples of supported V3C attribute types. A V3C attribute unit header may specify an attribute type in addition to the V3C unit type. A V3C attribute unit header may also specify an index, allowing for support of multiple instances of the same attribute type. For example, supported attribute types may include texture, material, transparency, reflectivity, or surface normal.

[0107] This specification describes the V3C container file format.

[0108] Figure 9 shows an exemplary structure of a V3C container that may be implemented according to the ISOBMFF standard. Generally, a V3C container may contain volumetric video data 900, further defined by atlas data, geometry data, attribute data, and occupancy data. More specifically, the container may include a V3C atlas track 910 containing the V3C parameter set and atlas parameter set in the sample entry, and the atlas component bitstream NAL units in the sample. The V3C atlas track may also include track references to other tracks 920, 930, and 940, or V3C atlas style tracks, that carry payloads of video compression V3C units (i.e., V3C unit types equivalent to V3C_OVD, V3C_GVD, and V3C_AVD).

[0109] The container may contain one or more V3C video component tracks in which the sample contains access units for a video-coded elementary stream for geometry data (i.e., the payload of a V3C unit of type equal to V3C_GVD), as illustrated in Figure 9, 920. The container may contain zero or more V3C video component tracks in which the sample contains access units for a video-coded elementary stream for attribute data (i.e., the payload of a V3C unit of type equal to V3C_AVD), as illustrated in Figure 9, 930. The container may contain zero or more V3C video component tracks in which the sample contains access units for a video-coded elementary stream for occupancy data (i.e., the payload of a V3C unit of type equal to V3C_OVD), as illustrated in Figure 9, 940.

[0110] Figure 10 illustrates an exemplary multi-track container having two or more atlases and multiple atlas styles. If multiple atlases are present on the V3C media, these atlases may be carried on separate atlas tracks having track references to their associated V3C component tracks (i.e., tracks carrying associated occupancy maps, geometry, and attribute information). If the atlas data includes two or more atlas styles, these atlas styles may be stored on separate atlas style tracks referenced by the atlas tracks, with additional track references stored from the atlas style tracks to tracks carrying associated V3C video component information for the atlas styles carried by the atlas style tracks. This may be illustrated, for example, in Figure 10. As illustrated in 1001, the V3C track "v3cb" may contain multiple atlases. The atlases may be stored on separate V3C tracks 1010 and 1020, each having, for example, sample entries for "v3a1" or "v3ag". V3C tracks 1010 and 1020 may each include multiple Atlas Style tracks 1011 and 1012, and each of Atlas Style tracks 1011 and 1012 may each include V3C Component tracks 1013 and 1014.

[0111] As substantially described in the paragraphs above, geometry-based compression (G-PCC) standards for static point clouds can also be defined to support efficient and interoperable storage and transmission of 3D point clouds. Methods, apparatus, and systems that can be performed and / or implemented in accordance with such geometry-based compression standards are proposed herein.

[0112] Figure 11 illustrates an example of the structure of a bitstream encoded in the G-PCC standard. As shown in Figure 11, the G-PCC bitstream 1100 may carry a set of G-PCC units, also known as type-length-value (TLV) encapsulation structures. As shown in 1110, (i.e., each) G-PCC TLV unit may contain information indicating the TLV type 1111 and the G-PCC TLV unit payload 1112. Although not shown in Figure 11, the G-PCC TLV unit may further contain information indicating the G-PCC TLV unit payload length, which can be expressed, for example, in bytes or bits. The G-PCC TLV unit payload 1112 may contain information of a given type. For example, the G-PCC TLV unit payload may carry information of a given type, which may be, for example, a sequence parameter set, a geometry parameter set, a geometry data unit, an attribute parameter set, an attribute data unit, a tile inventory, a frame boundary marker, or a default attribute data unit.

[0113] Figure 12 is a table providing an exemplary syntax structure for a G-PCC TLV encapsulation unit, which may be defined, for example, according to the MPEG standard. As shown in Figure 12, a TLV encapsulation unit may indicate the payload type using a first number of bits (or bytes), e.g., 8 bits. The TLV encapsulation unit payload length may be represented by a second number of bits (e.g., 32 bits). A G-PCC TLV encapsulation unit may contain a payload having the indicated payload type and payload length.

[0114] Figure 13 is a table providing possible values ​​for TLV type parameters and corresponding descriptions for each of these possible values. As shown in Figure 13, the TLV payload type may be a sequence parameter set, a geometry parameter set, a geometry data unit, an attribute parameter set, an attribute data unit, a tile inventory, a frame boundary marker, or a default attribute data unit. G-PCC TLV units having unit types "2" and "4" may be a geometry data unit and an attribute data unit, respectively.

[0115] Figure 14 is a table providing an exemplary syntax structure for a G-PCC TLV unit payload. The exemplary syntax shown in Figure 14 may correspond to a syntax structure defined, for example, in MPEG-I Part 9 (ISO / IEC 23090-9). The payload information for geometry G-PCC units and attribute G-PCC units may be decoded by a G-PCC decoder and may correspond to media data units (e.g., TLV units) specified in the corresponding geometry G-PCC units and attribute parameter set G-PCC units.

[0116] The high-level syntax (HLS) of G-PCC files may support the concepts of slices and tile groups in geometry and attribute data. A frame may be divided into multiple tiles and slices. A slice may be understood as a set of points that can be independently encoded or decoded. A slice may contain, for example, one geometry data unit and zero or more attribute data units. The information in an attribute data unit may depend on the corresponding information in a geometry data unit within the same slice. Within a slice, a geometry data unit may necessarily appear before its associated attribute unit. Data units in a slice may be contiguous. The ordering of slices within a frame is not necessarily specified.

[0117] In some schemes, groups of slices may be identified by a common tile identifier. A tile inventory describing the bounding box of each tile may be provided, consistent with some standards. Tiles may overlap with other tiles within their bounding box. Each slice may contain an index identifying the tile to which the slice belongs.

[0118] This specification describes the G-PCC container file format. When a G-PCC bitstream is carried on a single track, it may be necessary for the G-PCC encoded bitstream to be represented by a single-track declaration. Single-track encapsulation of G-PCC data can, in some cases, utilize simple ISOBMFF encapsulation, where the G-PCC bitstream is stored on a single track without further processing. Each sample within such a track may contain one or more G-PCC components. In other words, each sample may contain one or more TLV encapsulation structures.

[0119] Figure 15 illustrates an exemplary sample structure in which a bitstream providing G-PCC geometry information and attribute information is stored on a single track. As shown in Figure 15, sample 1500 of the track carrying the G-PCC bitstream may include at least one of a first TLV 1510 providing a parameter set, a second TLV 1520 providing geometry data, and a third TLV 1530 providing attribute data corresponding to the geometry data in the second TLV 1520.

[0120] When one or more encoded G-PCC geometry bitstreams and one or more encoded G-PCC attribute bitstreams are stored on separate tracks, each sample in the track may contain at least one TLV encapsulation structure that carries a single G-PCC component data.

[0121] Figure 16 shows an exemplary structure of a multi-track ISOBMFF G-PCC container that may be implemented according to several standards (such as MPEG-I Part 18 (ISO / IEC 23090-18)). The multi-track G-PCC container may include information units known as “boxes” shown in Figure 16 by the ftyp, moov, and mdat structures 1610, 1620, and 1630, respectively, which may correspond to the base media file format defined in ISO / IEC 14496-12. The ftyp box 1610 may provide, for example, file type description information and common data structures used in media files. The moov box 1620 and mdat box 1630 may include G-PCC tracks 1621 and 1631 that together contain geometry bitstream samples carrying geometry parameter sets, sequence parameter sets, and geometry data TLV units. Tracks may also include track references to other tracks carrying payloads of G-PCC attribute components. The moov box 1620 and mdat box 1630 may collectively include G-PCC tracks 1622 and 1632, which may contain attribute parameter sets for their respective attributes, and attribute bitstream samples that carry attribute data TLV units.

[0122] When a G-PCC bitstream is carried across multiple tracks, G-PCC component tracks may be linked using a track reference tool, which may be implemented according to several standards (such as ISO / IEC 14496-12). In some cases, one or more TrackReferenceTypeBoxes may be added to the TrackReferenceBox within the TrackBox of a G-PCC track. The TrackReferenceTypeBox may contain an array of track_IDs that specify the track referenced by the G-PCC track. To link a G-PCC geometry track to a G-PCC attribute track, the reference_type of the TrackReferenceTypeBox of the G-PCC geometry track may identify the associated attribute track. The four-character code (4CC) associated with these track reference types may be "gpca", which may indicate that the referenced track contains an encoded bitstream of G-PCC attribute data.

[0123] If the geometry stream of a G-PCC bitstream contains multiple tiles, each tile or group of tiles may be encapsulated in a separate track, which may be called a geometry tile track. The geometry tile track may carry TLV units of one or more geometry tiles, thus enabling direct access to these tiles. Similarly, the attribute stream of a G-PCC bitstream containing multiple tiles may also be carried in multiple attribute tile tracks.

[0124] Data for one or more G-PCC tiles may be carried in separate geometry tile tracks and attribute tile tracks within the container. To support partial access in an ISOBMFF container for G-PCC encoded streams, tiles corresponding to spatial regions within a point cloud scene may be signaled in a sample of a time-dependent metadata track, such as a track with Dynamic3DSpatialRegionSampleEntry, which may be defined in accordance with several MPEG standards, or in a GPCCSpatialRegionInfoBox box, which may similarly be defined in several MPEG standards. This may allow players and streaming clients to retrieve a set of tile tracks carrying the information necessary to render a particular spatial region or tile within a point cloud scene.

[0125] The G-PCC base track may carry TLV encapsulated structures containing only SPS, GPS, APS, and tile inventory information, as described in ISO / IEC 23090-9. To link the G-PCC base track to geometry tile tracks, a track reference with a new track reference type may be defined using 4CC "gpbt". The G-PCC base track may be linked to each geometry tile track using the new type of track reference.

[0126] Each geometry tile track can be linked to one or more other attributes of a G-PCC tile track that carries attribute information for each tile or tile group, using a track reference tool that may be implemented in accordance with, for example, ISO / IEC 14496-12. The 4CCs of these track reference types may be, for example, "gpca" as defined in accordance with the MPEG standard.

[0127] Point cloud scenes may be coded in alternative forms. In such cases, the alternative forms of coded G-PCC data may be indicated by an alternative track mechanism that can be implemented in accordance with ISO / IEC 14496-12. For example, the alternate_group field of a TrackHeaderBox may be used to indicate alternatives of coded G-PCC data. When each alternative G-PCC bitstream is stored in a single track, G-PCC tracks containing coded G-PCC bitstreams that may be alternative forms of each other may have the same alternate_group value in their TrackHeaderBox. When each alternative G-PCC bitstream is stored in a multi-track container, i.e., when the different component bitstreams of each alternative G-PCC bitstream are carried on separate tracks, the G-PCC geometry tracks of the alternative G-PCC bitstreams may have the same alternate_group value in their TrackHeaderBox.

[0128] Methods, procedures, apparatus, and systems for MPEG media transport (MMT) are described herein. Generally speaking, a set of tools can be used to enable advanced media transport and distribution services. The tools may be distributed across three different functional areas: media processing unit (MPU) formatting, distribution, and signaling. Such tools may be designed to be used efficiently together, but they may also be used independently.

[0129] The Media Processing Unit (MPU) functional area may define the logical structure of media content, the package and format of data units processed by MMT entities, and their instantiation using, for example, an ISO-based media file format. The package may specify components, including media content and their relationships, to provide information necessary for advanced distribution. The data format may be defined to encapsulate encoded media data for either storage or distribution, and to allow for easy conversion between data to be stored and data to be distributed.

[0130] The distribution function domain may define an application layer transport protocol and payload format called the MMT protocol (MMTP). The application layer transport protocol may provide extensions for the distribution of multimedia data, such as multiplexing and support for mixed use of streaming and download distribution in a single packet flow. The payload format may enable the transport of encoded media data that is independent of media type and encoding method.

[0131] The signaling function area may define the format of signaling messages for managing the distribution and consumption of media data. Signaling messages for consumption management may be used to signal the package structure, and signaling messages for distribution management may be used to signal the payload format and protocol configuration structure.

[0132] The MMT protocol can support the multiplexing of different media data, such as media processing units (MPUs) from various assets, over a single MMTP packet flow. It can deliver multiple types of data to a receiving entity in the order of consumption to aid in synchronization between different types of media data without introducing significant latency or requiring large buffers. MMTP may also support the multiplexing of media data and signaling messages within a single packet flow.

[0133] In some embodiments, an MMTP payload may be carried in only one MMTP packet. Fragmentation and aggregation may be provided by the payload format, or not by MMTP itself. MMTP may define two packetization modes: General File Delivery (GFD) mode and MPU mode. GFD mode may identify data units using their byte locations within the transport object. MPU mode may identify data units using their roles and media locations within the MPU. The MMT protocol may support mixed use of packets having two different modes in a single delivery session. A single packet flow of MMT packets may optionally consist of two different payloads.

[0134] Figure 17 depicts an exemplary end-to-end architecture of a system in which MMT signaling is performed. The architecture may include, but is not limited to, a package provider 1710, one or more asset providers 1721 and 1722, an MMT transmit entity 1730, and an MMT receive entity 1740. As shown in Figure 17, the MMT transmit entity 1730 may receive a package from the package provider 1710. The MMT transmit entity 1730 may also be responsible for sending the package to the MMT receive entity 1740 as an MMTP packet flow. The MMT transmit entity 1730 may be required to collect media content from a content provider based on the presentation information of the package provided by the package provider 1710. The media content may be provided as an asset segmented into a set of encapsulated MMT processing units that form an MMTP packet flow. Thus, the MMT transmit entity 1730 may collect asset information from one or more of the asset providers 1721 and / or 1722.

[0135] Signaling messages may be used to manage the delivery and consumption of packages. The interface between the MMT sending entity 1730 and the MMT receiving entity 1740, and their operation, may be standardized. The MMT protocol (MMTP) may be used by the MMT receiving entity 1740 to receive and multiplex decrypt streamed media based on the packet_id and payload type. The decapsulation procedure performed by the MMT receiving entity 1740 may depend on the type of payload being carried and may be handled separately, for example, in the scenario shown in Figure 17.

[0136] This specification describes various aspects of the MMT data model. The MMT protocol can provide both streaming and downloadable delivery of encoded media data. For streaming delivery, the MMT protocol may assume a specific data model including MPUs, assets, and packages. The MMT protocol may preserve the data model during delivery by using signaling messages to indicate the structural relationships between MPUs, assets, and packages.

[0137] A collection of encoded media data and its associated metadata may form a package. A package may be delivered from one or more MMT sending entities to one or more MMT receiving entities. One or more portions of the encoded media data in a package, such as a portion of audio or video content, may constitute an asset.

[0138] Assets may be associated with identifiers that do not depend on the actual physical location or service provider providing the asset, so that the asset can be identified globally and uniquely. Assets with different identifiers may not be interchangeable. For example, two different assets may carry two different encodings of the same content, but they may not be interchangeable. While MMT cannot specify a particular identification mechanism, it may allow the use of URIs or UUIDs for this purpose. Each asset may have its own timeline, which may have a different duration from the overall timeline of the presentation created by the package.

[0139] Each MPU can constitute a non-overlapping portion of an asset; that is, two consecutive MPUs of the same asset may not contain the same media samples. Each MPU can be consumed independently by the presentation engine of the MMT receiving entity.

[0140] Figure 18 illustrates package structures according to several embodiments. As shown in Figure 18, package 1800 may be a logical entity. Package 1800 may contain one or more presentation information documents 1810, one or more assets 1820, and asset delivery characteristics (ADCs) associated with each asset. Each asset 1820 may contain one or more MPUs 1830. Package processing may be performed on an MPU basis, and each MPU may share the same asset ID.

[0141] MMT assets are described further herein according to several embodiments. An asset may be any multimedia data used to construct a multimedia presentation. An asset may be a logical grouping of MPUs that share the same asset ID for carrying encoded media data. The encoded media data of an asset may be timed or non-timed data. Timed data may include encoded media data having its own timeline and may require synchronized decoding and presentation of data units at specified times. Non-timed data may include any other type of data that does not have its own timeline for decoding and presentation of its media content. The decoding and presentation times of each item of non-timed data may not necessarily be related to the decoding and presentation times of other items of the same non-timed data. For example, these may be determined by user interaction or presentation information.

[0142] Two MPUs carrying the same asset with time-sensitive media data may not overlap in their presentation times. Any type of data referenced by presentation information may be considered an asset. Examples of media data types that may be considered individual assets may include audio data, video data, or web page data.

[0143] The features and characteristics of a Media Processing Unit (MPU) are described herein. A Media Processing Unit (MPU) may be a media data item that is processed by an MMT entity and can be consumed by the presentation engine independently of other MPUs.

[0144] Processing of the MPU by MMT entities may include encapsulation / decapsulation and packetization / depacketization. The MPU may include MMT hint tracks indicating the boundaries of the MFU for media-aware packetization. Consumption of the MPU may include media processing (e.g., encoding / decoding) and presentation.

[0145] For packetization purposes, an MPU may be fragmented into data units that are smaller than access units (AUs). The syntax and semantics of an MPU may be independent of the type of media data carried within it. A single-asset MPU may have either timed or non-timed media. An MPU may contain a portion of data formatted according to one or more standards, such as MPEG-4 AVC (ISO / IEC 14496-10) or MPEG-2 TS.

[0146] A single MPU may contain an integer number of AUs or non-time-limited data. For time-limited data, a single AU may not be fragmented into multiple MPUs. For non-time-limited data, a single MPU may contain one or more non-time-limited data items consumed by the presentation engine. MPUs may be identified by their associated asset identifier (asset_id) and / or sequence number.

[0147] Aspects of MMTP payloads are described herein. An MMTP payload may be a general-purpose payload used to packetize and transmit media data such as MPUs, general-purpose objects, and other information for consuming the package via the MMT protocol. A suitable MMTP payload format may be used to packetize the MPUs, general-purpose objects, and signaling messages.

[0148] An MMTP payload may carry a complete MPU or fragments of an MPU, a signaling message, a generic object, an AL-FEC-style repair symbol, or other data units or structures. The type of payload may be indicated by the type field in the MMT protocol packet header. For each payload type, one or more data units for delivery and, additionally or alternatively, a type-specific payload header may be defined. For example, when an MMTP payload carries fragments of an MPU, the fragments of the MPU (e.g., an MFU) may be considered a single data unit. The MMT protocol may aggregate multiple data units of the same data type into a single MMTP payload. Alternatively, a single data unit may be fragmented into multiple MMTP packets.

[0149] An MFU may be a sample or subsample of timed data, or an item of non-timed data. An MFU may contain media data which may be smaller than the AU for timed data, and the contained media data may be processed by a media decoder. An MFU may contain an MFU header which contains information about the boundaries of the media data being carried. An MFU may contain an identifier which uniquely distinguishes the MFU within an MPU. It may also provide dependency and priority information for other MFUs within the same MPU.

[0150] An MMTP payload may include a payload header and payload data. Some data types may allow fragmentation and aggregation, in which case a single data unit may be divided into multiple fragments, or a set of data units may be delivered in a single MMTP packet.

[0151] In recent years, there has been considerable interest in emerging media types such as virtual reality (VR), immersive video, and 3D graphics. High-quality 3D point clouds have emerged as a sophisticated representation of immersive media, enabling new forms of interaction and communication with virtual worlds. The large amount of information required to represent such point clouds necessitates efficient coding algorithms. New standards for video-based point cloud compression are currently under development and will form the basis for visual volumetric video-based coding (V3C). Standards for geometry-based point cloud compression are also being developed, which may define bitstreams for compressed static point clouds. In parallel, standards defining the transport of V3C media and geometry-based point cloud data are also under development.

[0152] Discussions surrounding V3C carriage and point cloud standards may address the storage and signaling modes of V3C and point cloud data, but such discussions may be limited to relating only to signaling for dynamic adaptive streaming over HTTP based on the MPEG-DASH standard, for example. Another important candidate standard for enabling different streaming and delivery applications is MPEG Media Transport (MMT). However, the MMT standard currently does not always provide a signaling mechanism for V3C media. Therefore, new signaling elements are desired that enable streaming clients to identify V3C streams and their component substreams. In addition, it may be necessary to signal different types of metadata associated with V3C components so that streaming clients can select the optimal version(s) of V3C content or its components that they can support or deliver to the user's viewport under given specific network constraints or at any given time.

[0153] Furthermore, actual point cloud applications are expected to require streaming point cloud data over a network. Such applications may perform either live streaming or on-demand streaming of point cloud content, depending on how the content was generated. Due to the large amount of information required to represent the point cloud, such applications may need to support adaptive streaming techniques to avoid network overload and to provide an optimal viewing experience at any given moment, for example, with respect to the network capacity at that moment. Components of point cloud content may be divided into multiple tiles. One or more streaming clients may (e.g., simply) want to stream a specific tile portion of a geometric component (e.g., instead of the entire point cloud data) based on bandwidth availability, for example. G-PCC component tile data may be encapsulated in different G-PCC tile tracks. (e.g., each) tile track may represent a set of G-PCC component tiles or all G-PCC component tiles.

[0154] Currently, MMT does not provide a signaling mechanism for point cloud media, including point cloud streams based on the MPEG G-PCC standard. Therefore, it is important to define new signaling elements that enable streaming clients to identify point cloud streams and their component substreams. It is also necessary to signal different types of metadata associated with point cloud components to enable streaming clients to select the optimal version of the point cloud or its components that they can support.

[0155] The solutions described herein may provide novel signaling elements that enable MMT streaming clients to identify different components and metadata associated with V3C and GPCC media content and to select the media data that the client needs to retrieve from the content server at any point during a streaming session. Furthermore, the solutions described herein may provide various methods for encapsulating G-PCC data for MMT streaming with MMT signaling messages necessary to support the delivery of G-PCC data via MMT.

[0156] The MMT delivery of V3C content is further described herein. V3C content may assist MMT transmission entities during the streaming process. For example, presentation information may include information describing a V3C-compliant MPU to enable appropriate processing by the application.

[0157] The player may receive information about the current viewing direction, the current viewport, and the display characteristics of the device on which the player is running. Based on this information, view-dependent streaming may be used to reduce the bandwidth required in the streaming session. In the case of MMT, view-dependent streaming may be achieved by one or more methods.

[0158] In some client-based streaming techniques, the MMT receiving entity may be instructed by the player to select a subset of assets that carry the V3C information required to render the portion of V3C content contained within (or intersecting with) the current viewport. An MMT session control procedure may be used to request the selected set of assets from the MMT sending entity. The player may use V3C application-specific signaling messages from the server to select the appropriate assets to switch to for view-dependent streaming.

[0159] In some server-based approaches, an MMT receiving entity may depend on an MMT sending entity to select the correct subset of assets that provide V3C information for rendering the portion of V3C content covering the current viewport. The receiving entity may use V3C application-specific signaling to send information about the current viewport to the sending entity.

[0160] Methods and procedures for mapping V3C containers to MMT assets are described herein. To support the delivery of V3C content using MMT, each track within a multi-track ISOBMFF V3C container may be encapsulated as a separate asset. Thus, the number of assets may be equal to the number of tracks in the container. Assets belonging to the same V3C component may be logically grouped into asset groups. These asset groups may be signaled to receiving entities to enable streaming clients to determine which asset group they should request. V3C application-specific MMT signaling is described herein.

[0161] For the purpose of streaming V3C-encoded data using MMT, several V3C-specific MMT messages are defined. For example, V3C application-specific signaling may include sending group messages such as V3CAssetGroupMessage, selection messages such as V3CSelectionMessage, or view change feedback messages such as V3CViewChangeFeedbackMessage. In some embodiments, these messages may include an application identifier having a uniform resource name (URN) "urn:mpeg:mmt:app:v3c:2020", which may allow a sending entity to associate the signaling with a V3C application.

[0162] Figure 19 is a table providing a list of defined application message types. In the proposed MMT V3C signaling, a set of application message types may be defined, and each message type in the set may be associated with an application message name, as shown in Figure 19. Via V3CAssetGroupMessage, the sending entity may notify the client of a set of assets available on the server and provide a list of assets being streamed to the receiving entity. With V3CSelectionMessage, the client may request that a set of assets be streamed to the receiving entity by the sending entity. With V3CViewChangeFeedbackMessage, the client may send instructions to the server regarding the user's current viewing direction and viewport in a server-based view-dependent streaming session.

[0163] When transmitting V3C content via MMT, in some embodiments, the V3CAssetGroupMessage may be mandatory and may provide the receiving entity with a list of assets available on the server associated with the V3C content. This message may also be used to inform the receiving entity which of these assets are currently being streamed to it. From this list, a client operating on the receiving entity may request a unique subset of these V3C assets using the V3CSelectionMessage message.

[0164] For view-dependent delivery of V3C content via MMT, a client may use the V3CViewChangeFeedbackMessage message to send its current viewport information to a server, which may then select and deliver assets corresponding to that viewport to the client. The V3CAssetGroupMessage may also be used to update a client with a selected subset of assets. Figure 20 is a table providing an exemplary syntactic structure for a V3C asset descriptor. Asset descriptors can be used to inform receiving entities and consuming applications about the content of assets that carry V3C content. The semantics of V3C asset descriptors are provided herein. A descriptor tag, e.g., "Descriptor_tag", may indicate the type of descriptor. A descriptor length, e.g., "Descriptor_length", may specify the length in bytes, counting from the byte following this field to the last byte of the descriptor. A data type, e.g., "Data_type", may indicate the type of V3C data present in this asset. The values ​​of this field are further shown in Figure 22 and introduced and substantially explained in the following paragraphs. The dependency flag, for example, "Dependency_flag," may indicate whether a V3C asset depends on data in another V3C asset for decryption. A value of 0 may indicate that this V3C component asset group data can be decrypted independently. A value of 1 may indicate that this V3C asset depends on other V3C asset data for decryption. The alternate group flag, for example, "Alternate_group_flag," may indicate whether this V3C asset has alternate versions. A value of 0 may indicate that this V3C component asset has no alternate assets. A value of 1 may indicate that this V3C asset has one or more alternates. The alternate group ID, for example, "Alternate_group_id," may indicate an ID that identifies a group of alternate assets. Different encoded versions of the same V3C asset may have the same value for this field.The dependent asset ID, for example, "Dep_asset_id," may indicate the value of the asset ID on which the decryption of this asset depends. In some cases, this value may only exist when dependency_flag is set to 1. For example, a V3C video component asset may use the corresponding V3C atlas component asset ID for this field. "Num_tiles" may indicate the number of tiles carried by this asset. "tile_id" may indicate a unique identifier for a particular atlas style.

[0165] Figure 21 is a table showing an example of the exemplary syntax of V3CAssetGroupMessage. In accordance with the table in Figure 21, the semantics of V3CAssetGroupMessage can be described as follows: "Message_id" may indicate the identifier of the V3C application message. "Version" may indicate the version of the V3C application message. "Length" may indicate the length of the V3C application message in bytes, counting from the start of the following field to the last byte of the message. The value of this field does not have to be equal to 0. The application identifier, for example, "Application_identifier", may indicate the application identifier as a URN that uniquely identifies the application that will consume the contents of this message. "App_message_type" may indicate the application-specific message type, as substantially described above with respect to Figure 19. "Num_V3C_asset_groups" may indicate the number of V3C asset groups, each group containing assets associated with a V3C component. "asset_group_id" may indicate the identifier of the asset group associated with the V3C component. "Num_assets" indicates the number of assets in the asset group associated with the V3C component. "Start_time" may indicate the presentation time of the V3C component to which the state of the assets listed in this message is applicable. "Data_type" may indicate the type of V3C data present in this asset group. Examples of values ​​for this field are illustrated in the context of Figure 22 and introduced and substantially explained in the following paragraphs. "Pending_flag" may indicate whether all data components are ready for rendering the asset group. For example, if set to "1", it may indicate that the data is ready; otherwise, the flag may be "0". "asset_id" may provide the asset identifier of the asset. "state_flag" may indicate the delivery status of the asset.When set to 1 ("1"), this may indicate that the sending entity is actively sending assets to the receiving entity. When set to 0 ("0"), this may indicate that the sending entity is not actively sending assets to the receiving entity. "Sending_time_flag" may indicate the presence of "sending_time" for the first MMTP packet containing the first MPU in the asset stream. The default value may be "0". "alternate_group_flag" may indicate whether this V3C component asset has alternate versions. A value of 0 may indicate that this V3C asset has no alternate assets. A value of 1 may indicate that this V3C asset has alternate assets. Dependency flags, such as "Dependency_flag", may indicate whether this V3C component asset depends on data in other V3C assets for decoding. A value of 0 may indicate that this V3C component asset group data can be decoded independently. A value of 1 may indicate that this V3C asset depends on other V3C asset data for decoding. The transmission time, for example "Sending_time," may indicate the transmission time for the first MMTP packet containing the first MPU in the asset stream. Using this information, the client may prepare a new packet processing pipeline for the new asset stream. "alternate_group_id" may indicate the identifier of an alternate V3C component asset. Different encoded versions of the same V3C asset may have the same value for this field. "Dep_asset_group_id" may indicate the ID for the asset on which the decoding of this asset depends. In some cases, this value may only exist when, for example, dependency_flag is set to 1. For example, a V3C attribute component asset may use the V3C atlas component asset ID corresponding to this field. "all_tiles_present_flag" may indicate whether all tiles of an atlas component are part of the asset.A value of 1 may indicate that data for all Atlas styles is available in the asset. A value of 0 may indicate that data for a subset of Atlas styles is available in the asset. "Num_tiles" may indicate the number of tiles carried in this asset. "tile_id" may provide a unique identifier for a particular Atlas style.

[0166] Figure 22 is a table illustrating exemplary V3C data type values ​​that may be used in the Data_type field. As shown in Figure 22, the value of the Data_type field can represent all V3C component data, atlas component data, occupied component data, geometry component data, attribute component data, codec initialization data, dynamic volumetric time-limited metadata information, or viewport time-limited metadata information.

[0167] Figure 23 is a table showing exemplary syntax for V3CSelectionMessage. In accordance with the table in Figure 23, the semantics of V3CSelectionMessage can be described as follows: "Message_id" may indicate the identifier of the V3C application message. "Version" may indicate the version of the V3C application message. "Length" may indicate the length of the V3C application message in bytes, for example, counting from the beginning of the following field to the last byte of the message. The value of this field does not have to be equal to 0. "Application_identifier" may indicate the application identifier as a URN that uniquely identifies the application that consumes the content of this message. "App_message_type" may indicate the application-specific message type, as substantially described above in the paragraph above with respect to Figure 19. "Num_selected_asset_groups" may indicate the number of asset groups for which the receiving entity has associated state change requests. "asset_group_id" may indicate the identifier of the asset group associated with the V3C content. "switching_mode" may indicate the switching mode used for selecting assets requested by the receiving entity. A list of values ​​for "switching_mode" may be defined in accordance with the following paragraphs, which introduce and explain Figure 23, for example. "Num_assets" may indicate the number of assets signaled for state changes due to the specified switching mode. "Asset_id" may indicate the identifier of the asset for state changes due to the specified switching mode.

[0168] Figure 24 is a table that provides the definition of the switching_mode field. As shown in Figure 24, the "switching_mode" field can indicate the switching mode used for asset selection. For example, if the switching mode is set to refresh, the State_flag of each asset listed in V3CSelectionMessage will be set to "1", and the State_flag of all assets not listed in V3CSelectionMessage will be set to "0". If the switching mode is set to toggle, the State_flag of each asset listed in V3CSelectionMessage will change, for example, from "0" to "1", and from "1" to "0", but the State_flag of all assets not listed in V3CSelectionMessage will remain unchanged. If the switching mode is set to send all for all assets in the asset group specified in V3CSelectionMessage, the State_flag of each asset will be set to "1".

[0169] Figure 25 is a table showing exemplary syntax for V3CViewChangeFeedbackMessage. In accordance with the table in Figure 25, the semantics of V3CViewChangeFeedbackMessage may be described as follows: “Message_id” may indicate the identifier of the V3C application message. “Version” may indicate the version of the V3C application message. “Length” may indicate the length of the V3C application message in bytes, counting from the start of the following field to the last byte of the message. The value of this field shall not be equal to 0. “Application_identifier” may indicate the application identifier as a URN that uniquely identifies the application consuming the content of this message. “App_message_type” may indicate the application-specific message type, as substantially described above in the paragraph above with respect to Figure 19. “Vp_pos_x”, “vp_pos_y”, and “vp_pos_z” may indicate the x, y, and z coordinates of the viewport position in the global reference coordinate system in meters, respectively. The values ​​may be, for example, 2 -16 It may be provided in units of meters. "Vp_quat_x", "vp_quat_y", and "vp_quat_z" may represent the x, y, and z components of the rotation of the viewport region using quaternion representation, respectively. The coordinate values ​​may be floating-point values ​​in the range of -1 to 1, including the endpoints. These values ​​may specify the x, y, and z components of the rotation, i.e., qX, qY, and qZ, applied to transform the global coordinate axes to the camera's local coordinate axes using quaternion representation. The fourth component of the quaternion qW may be generated according to Equation 1,

[0170]

number

[0171]

number

[0172] "clipping_near_plane" and "clipping_far_plane" can indicate the near and far depth (or distance) in meters based on the viewport's near and far clipping planes. "Horizontal_fov" may specify a longitude range corresponding to the horizontal size of the viewport area, for example, in radians. This value may be within the range of 0 to 2π. "vertical_fov" may specify a latitude range corresponding to the vertical size of the viewport area, for example, in radians. This value may be within the range of 0 to π.

[0173] Methods and apparatus relating to streaming client behavior are described herein. The MMT client may be guided by information provided in application-specific signaling messages. The following is an example of client behavior for streaming V3C content using the MMT signaling presented herein.

[0174] In some cases, an MMT sending entity may send a "V3CAssetGroupMessage" application message to interested clients. Receiving clients can parse the "V3CAssetGroupMessage" application message and identify the V3C media assets present in the MMT content sending entity. To identify available V3C media content, streaming clients may check the "application_identifier" field in the "V3CAssetGroupMessage" application message, which is set to "urn:mpeg:mmt:app:v3c:2020". All or some of the available V3C assets in the V3C content can be identified by checking the asset IDs signaled in the "V3CAssetGroupMessage" application message. Clients may select the desired assets to be streamed based on the user's current viewport. MMT clients may send a "V3CSelectionMessage" application message to the sending entity requesting V3C assets of interest from a list of available V3C assets. The MMT sending entity may form an MMTP packet with MTP and send the MTTP packet to the client.

[0175] In several ways, an MMT client may receive an MMTP packet and depacket the MPU or MFU. The MPU / MFU may contain timed media content or non-timed V3C media content. When an MMT client receives an MMTP packet with asset group "data_type" set to "0x05", this V3C asset data represents initialization information such as VPS, ASPS, AAPS, AFPS, and SEI messages. When an MMT client receives an MMTP packet with asset group "data_type" set to "0x06", this V3C asset data may represent 3D spatial domain timed metadata information. The information within this asset may be used for partial access to V3C content. When an MMT client receives an MMTP packet with asset group "data_type" set to "0x07", this V3C asset data may indicate initial or recommended viewport information. This information can be used to enable automatic viewport changes based on different criteria. The MMT client may select the necessary V3C assets based, for example, on the user's viewport or a recommended viewport and one or more corresponding 3D spatial regions. The MMT client may send a "V3CSelectionMessage" application message to the sending entity requesting the target V3C assets.

[0176] In some cases, when the user's viewport changes in a client-based streaming method, the MMT client may request a different set of V3C assets using the "V3CSelectionMessage" application message. When the user's viewport changes in a server-based streaming method, the MMT client may signal the user's current viewport by sending a "V3CViewChangeFeedbackMessage" message to the sending entity. Upon receiving this message, the MMT sending entity selects a new set of V3C assets based on the user's new viewport information and sends a "V3CAssetGroupMessage" application message to the MMT client along with the corresponding V3C assets. The MMT sending entity may stream the V3C asset data as MMTP packets. The MMT client may begin receiving MMTP packets for all requested V3C assets and extract the MPU and MFU from the MMTP payload. The MPU and MFU may contain media samples directly or may contain media segments. The MMT client may initiate parsing of a media segment container (e.g., ISOBMFF) to extract elementary stream information and structure it into a V3C bitstream according to the V3C standard. The bitstream may then be passed to a V3C decoder. If the MMTP payload contains a V3C media sample, the elementary stream data is extracted and structured according to the V3C bitstream standard. The bitstream may then be passed to a V3C decoder.

[0177] Embodiments relating to the encapsulation and signaling of G-PCC data in MMT are described herein. Unlike conventional media content, G-PCC media content may contain numerous components, such as geometry and attributes. Each component may be encoded separately as a substream of the G-PCC bitstream. Components such as geometry and attributes may be encoded, for example, using a G-GPCC encoder. However, these substreams may need to be decoded together with additional metadata in order to render a point cloud.

[0178] G-PCC encoded content may be delivered over the network using MMT. When G-PCC components within ISOBMFF are signaled using multiple tracks, each track may be proposed to be encapsulated in a separate asset, which may then be packetized into MMTP packets in the usual manner. G-PCC defined application messages are also proposed to enable servers and clients to identify groups of multiple assets for a particular G-PCC component.

[0179] G-PCC media content may contain one or more (e.g., multiple) components, such as geometry and attributes. Each component (e.g.) may be encoded separately as a substream of the G-PCC bitstream. Components such as geometry and attributes may be encoded using, for example, a G-GPCC encoder. The substreams may be decoded together with additional metadata, for example, to render a point cloud.

[0180] G-PCC data may be encapsulated and signaled in MMT. G-PCC encoded content may be delivered over a network using MMT. G-PCC data may be encapsulated for MMT streaming using various encapsulation methods (e.g., as described herein). MMT signaling messages may support the delivery of G-PCC data over MMT (e.g., they may be generated and transmitted).

[0181] G-PCC components within ISOBMFF may be signaled using multiple tracks. Each track (for example, one of the multiple tracks) may be encapsulated in a separate asset, and the separate asset may then be packetized into an MMTP packet. G-PCC defined application messages may be configured / deployed (for example, also) to identify a group of assets to or for a particular G-PCC component, for example, by the server and client.

[0182] In some cases (for example, to support the delivery of G-PCC content using MMT), each track within a multi-track ISOBMFF G-PCC container may be encapsulated in a separate asset. The number of assets may be equal to the number of tracks within the multi-track ISOBMFF G-PCC container. In some cases, multiple assets corresponding to a single G-PCC component may be grouped together and signaled as an asset group in a message (for example, a "GPCCAssetGroupMessage" application message). Alternate component tracks may be exposed in a message (for example, using a "GPCCAssetGroupMessage" message) to enable server and client selection (for example, efficiently) (for example, without first parsing the ISOBMFF file in the MMTP packet).

[0183] MMT may define application-specific signaling messages that support (e.g., allow) the delivery of application-specific information. G-PCC-specific signaling messages may be defined (e.g., configured) to stream G-PCC encoded data using MMT. G-PCC-specific signaling messages may have an application identifier with a Uniform Resource Name (URN) value (e.g., "urn:mpeg:mmt:app:gpcc:2020").

[0184] Figure 26 is a table providing an exemplary syntactic structure for a G-PCC asset descriptor. Asset descriptors can be used to inform receiving entities and consuming applications about the content of the asset that carries the G-PCC content. The semantics of G-PCC asset descriptors are provided herein. "descriptor_tag" may indicate the type of descriptor. "Descriptor_length" may specify the length in bytes, counting from the byte following this field to the last byte of the descriptor. "Data_type" may indicate the type of G-PCC data present in this asset group. The values ​​of this field are further shown in Figure 29 and introduced and substantially explained in the following paragraphs. "Dependency_flag" may indicate whether the G-PCC asset depends on data in another G-PCC asset for decoding. A value of 0 may indicate that this G-PCC component asset group data can be decoded independently. A value of 1 may indicate that this G-PCC asset depends on other G-PCC asset data for decoding. "alternate_group_flag" may indicate whether this G-PCC asset has alternate versions. A value of 0 may indicate that this G-PCC component asset has no alternate assets. A value of 1 may indicate that this G-PCC asset has one or more alternates. "alternate_group_id" may indicate an ID that identifies a group of alternate assets. Different encoded versions of the same G-PCC asset may have the same value for this field. "Dep_asset_id" may indicate the value of the asset ID on which the decoding of this asset depends. In some cases, this value may only exist when dependency_flag is set to 1. For example, a G-PCC attribute component asset may use the corresponding G-PCC geometry component asset ID for this field. "Num_tiles" may indicate the number of tiles carried by this asset. tile_id indicates a unique identifier for a particular tile in the tile inventory.When dynamic_tile_id_flag is set to value 0, tile_id may represent one of the tile ID values ​​present in the tile inventory.

[0185] MMT G-PCC signaling may include one or more of a set of application message types (e.g., defined), such as group messages like GPCCAssetGroupMessage, selection feedback messages like GPCCSelectionMessageFeedback, and / or change view feedback messages like GPCCViewChangeFeedback.

[0186] Figure 27 is a table illustrating examples of defined G-PCC application message types. As shown in Figure 27, an application message type may indicate that the message is a GPCCAssetGroupMessage, a GPCCSelectionMessageFeedback message, or a GPCCViewChangeFeedback message. In the example of the GPCCAssetGroupMessage message type, the sending entity may send a group message (e.g., a GPCCAssetGroupMessage message) to inform the client about a set of assets available on the server and / or a list of assets that can be streamed to (e.g., are being streamed) to the receiving entity. In the example of the selection feedback message type (e.g., a GPCCSelectionMessageFeedback message type), the client may use the selection feedback message to request that a set of assets be streamed to the receiving entity by the sending entity. In the example of the change view feedback message (e.g., a GPCCViewChangeFeedback message), the client may use the view change feedback message to send instructions to the server regarding the user's current viewing space.

[0187] Group messages (e.g., GPCCAssetGroupMessage messages) may be used to transmit G-PCC encoded content via MMT. Group messages (e.g., GPCCAssetGroupMessage messages) may provide the client with a list of G-PCC data type assets available on the server and / or inform the client which assets can be streamed to the receiving entity (e.g., which are currently being streamed). The client may request a unique subset of G-PCC data type assets (e.g., from the list). Requests may be made, for example, using GPCCSelectionFeedback messages.

[0188] The client may use the GPCCViewChangeFeedback message (for example, for view-dependent delivery of G-PCC content via MMT) to send information to the server, for example, the current viewing space (e.g., frustum). The server may select assets corresponding to the viewing space and deliver them to the client. The GPCCAssetGroupMessage may also be updated and sent to the client. Table 4 provides examples of defined G-PCC application message types.

[0189] Figure 28 is a table illustrating exemplary syntax for group messages such as GPCCAssetGroupMessage. Consistent with the table in Figure 28, the semantics of GPCCAssetGroupMessage may be as follows: "Message_id" may indicate the identifier of the G-PCC application message. "Version" may indicate the version of the G-PCC application message. "Length" may indicate the length of the G-PCC application message (e.g., in bytes counting from the beginning of the following field to the last byte of the message). The value of the length field does not have to be equal to 0. The application identifier (e.g., "application_identifier") may indicate the application identifier, for example, as a URN that uniquely identifies the type of application consuming the message content. The application message type (e.g., "app_message_type") may define an application-specific message type (e.g., as given by the examples in Table 4). The length of the application message type field may be, for example, 8 bits. The number of G-PCC asset groups (e.g., "num_gpcc_asset_groups") may indicate the number of G-PCC asset groups. Each asset group may contain assets associated with a G-PCC component. The asset group identifier (e.g., "asset_group_id") may indicate the identifier of the asset group associated with a G-PCC component. The number of assets (e.g., "num_assets") may indicate the number of assets in the asset group associated with a G-PCC component. The start time (e.g., "start_time") may indicate the presentation time of the G-PCC component to which the status of the assets listed in the message may be applicable. The data type (e.g., "data_type") may indicate the type of G-PCC point cloud data present within the asset group, as further described in the following paragraphs with respect to Figure 29.The pending flag (e.g., "pending_flag") may indicate whether, for example, all data components are ready to render for an asset group. A pending flag set to "1" may indicate that the data is ready. A pending flag set to 0 ("0") may indicate that the data is not ready. The dependency flag (e.g., "dependency_flag") may indicate whether a G-PCC component asset group depends on other G-PCC component asset group data for decryption. A value of 0 ("0") may indicate that the G-PCC component asset group data can be decrypted independently. A value of 1 ("1") may indicate that the G-PCC component asset group depends on other G-PCC component asset group data for decryption. The dependent asset group ID (e.g., "dep_asset_group_id") may indicate the value of the asset group ID on which asset group content decryption depends. The value may exist, for example, when / when dependency_flag is set to 1 (for example, only in such cases). For example, a G-PCC attribute component asset group may use the corresponding G-PCC geometry component asset group ID for the dependent asset group ID field. The asset ID (e.g., "asset_id") may provide the asset identifier of the asset. The alternate asset group flag (e.g., "alternate_asset_group_flag") may indicate whether a G-PCC component asset has an alternate version. A value of 0 ("0") may indicate that a G-PCC component asset does not have an alternate version. A value of 1 ("1") may indicate that a G-PCC component asset has an alternate version. The value of the alternate group flag field may be set to 1 ("1"), for example, when different encoded versions of the same G-PCC component and / or asset are available in the bitstream.The value of the alternate group flag field may be set to 0 ("0"), for example, when a different encoded version of the same G-PCC component and / or asset is not available in the bitstream. The alternate asset group ID (e.g., "alternate_asset_group_id") may indicate a value (e.g., a unique value) for the alternate G-PCC component asset. Different encoded versions of a G-PCC component or asset may represent the same value for the alternate asset group ID field. The state flag (e.g., "state_flag") may indicate the delivery status of the asset. A state flag set to 1 ("1") may indicate that the sending entity is actively sending the asset to the receiving entity. A state flag set to 0 ("0") may indicate that the sending entity is not actively sending the asset to the receiving entity. The send time flag (e.g., "sending_time_flag") may indicate the existence of a send time (e.g., sending_time) for the first MMTP packet containing the first MPU in the asset stream. The default value may be, for example, 0 ("0"). The transmission time (e.g., "sending_time") may indicate the transmission time of the first MMTP packet containing the first MPU in the asset stream. The client may prepare a new packet processing pipeline for a new asset stream (e.g., using the transmission time information). The dynamic tile flag (e.g., "dynamic_tile_flag") may indicate whether the number of tiles and / or tile identifiers may change dynamically within the asset. A value of 0 ("0") may indicate that the number of tiles and tile identifiers in the asset do not change throughout the bitstream, and / or that the number of tiles (e.g., "num_tiles") and tile ID (e.g., "tile_id") are signaled. A value of 1 ("1") may indicate the number of tiles, where the tile identifier may change within the asset. A value of 1 ("1") may indicate that the tile IDs present in the tile track change dynamically over time in the bitstream.The number of tiles (e.g., "num_tiles") may indicate the number of tiles transported within the asset. The tile ID (e.g., "tile_id") may indicate a (e.g., unique) identifier for a particular tile in the tile inventory. The tile ID (e.g., "tile_id") may represent a tile ID value (e.g., one of the tile ID values) that exists in the tile inventory when, for example, the dynamic tile flag (e.g., "dynamic_tile_flag") is set to a value of 0 ("0").

[0190] Figure 29 is a table showing exemplary G-PCC data type values ​​that may be used in the Data_type field. As shown in Figure 24, the value of the Data_type field may represent all G-PCC component data, geometry data, attribute data, SPS, GPS, APS, and tile inventory data, or 3D spatial domain time-limited metadata information.

[0191] Figure 30 is a table showing exemplary syntax for a GPCC selection feedback message (e.g., "GPCCSelectionFeedback"). Consistent with the table in Figure 30, the semantics of a GPCCSelectionFeedback message may be as follows: The message ID (e.g., "message_id") may indicate the identifier of the G-PCC application message. The version (e.g., "version") may indicate the version of the G-PCC application message. The length (e.g., "length") may indicate the length of the G-PCC application message (e.g., in bytes counting from the beginning of the following field to the last byte of the message). The value of the length field does not have to be equal to 0. The application identifier (e.g., "application_identifier") may indicate the application identifier, for example, as a URN that uniquely identifies the type of application consuming the message's content. The application message type (e.g., "app_message_type") may define an application-specific message type (e.g., substantially described in the paragraph above with respect to Figure 27). The length of the application message type field may be, for example, 8 bits. The number of selected asset groups (e.g., "num_selected_asset_groups") may indicate the number of asset groups for which an associated state change request exists by the receiving entity. The asset group ID (e.g., "asset_group_id") may indicate the identifier of the asset group associated with the G-PCC content. The switching mode (e.g., "switching_mode") may indicate the switching mode used for asset selection (e.g., as requested by the receiving entity). The number of assets (e.g., "num_assets") may indicate the number of assets signaled for state changes (e.g., according to the specified switching mode).The asset ID (e.g., "asset_id") may indicate the identifier of the asset for a state change (e.g., according to a specified switching mode).

[0192] Figure 31 is a table showing the definition of the switching_mode field. As shown in Figure 31, the "switching_mode" field can indicate the switching mode used for asset selection. For example, if the switching mode is set to refresh, the State_flag of each asset listed in GPCCSelectionMessageFeedback will be set to "1", and the State_flag of all assets not listed in GPCCSelectionMessageFeedback will be set to "0". If the switching mode is set to toggle, the State_flag of each asset listed in GPCCSelectionMessageFeedback will be changed, for example, from "0" to "1", and from "1" to "0", but the State_flag of all assets not listed in GPCCSelectionMessageFeedback will remain unchanged. If the switching mode is set to send all to all assets in the asset group specified in GPCCSelectionMessageFeedback, the State_flag of each asset will be set to "1".

[0193] Figure 32 is a table showing exemplary syntax for a G-PCC view change feedback message (e.g., "GPCCViewChangeFeedback"). Consistent with the table in Figure 32, the semantics of a GPCCViewChangeFeedback message may be as follows: The message ID (e.g., "message_id") may indicate the identifier of the G-PCC application message. The version may indicate the version of the G-PCC application message. The length may indicate the length of the G-PCC application message (e.g., in bytes, counting from the beginning of the following field to the last byte of the message). The value of the length field does not have to be equal to 0. The application identifier (e.g., "application_identifier") may indicate the application identifier, for example, as a URN that uniquely identifies the type of application consuming the message content. The application message type (e.g., "app_message_type") may define an application-specific message type (e.g., as given by the examples in Table 4). The length of the application message type field may be, for example, 8 bits. Viewport position coordinates (e.g., vp_pos_x, vp_pos_y, vp_pos_z) can represent the x, y, and z coordinates of the viewport's position in the global reference coordinate system in meters. The values ​​are, for example, 2 -16The units may be meters. Viewport rotations (e.g., vp_quat_x, vp_quat_y, vp_quat_z) may represent the x, y, and z components of the rotation of the viewport region (e.g., using quaternion representation). The values ​​may be floating-point values ​​in the range of -1 to 1, including the endpoints. The values ​​may specify the x, y, and z components (e.g., qX, qY, and qZ) of the rotation applied to transform the global coordinate axes to the camera's (e.g., using quaternion representation) local coordinate axes. The fourth component of the quaternion qW may be calculated, for example, according to Equation 1 substantially described in the paragraph above. The point (w, x, y, z) may represent a rotation around the axis oriented by the vector (x, y, z) by an angle determined according to Equation 2, which is also substantially described in the paragraph above.

[0194] Clipping in the near plane (e.g., clipping_near_plane) and clipping in the far plane (e.g., clipping_far_plane) may indicate near and far depth or distance based on, for example, the viewport's near and far clipping planes (e.g., in meters).

[0195] The horizontal field of view (FOV) (e.g., horizontal_fov) may specify a range of longitude (e.g., in radians) corresponding to the horizontal size of the viewport area. This value may be within the range of 0 to 2π.

[0196] The vertical FOV (e.g., vertical_fov) may specify a latitude range (e.g., in radians) corresponding to the vertical size of the viewport area. This value may be within the range of 0 to π.

[0197] Streaming client behavior may be provided (e.g., defined or configured). The MMT client may be guided, for example, by information provided in application-specific signaling messages. An example of client behavior is provided for streaming geometry-based point cloud compressed content (e.g., using the MMT signaling examples disclosed herein).

[0198] An MMT sending entity may send a "GPCCAssetGroupMessage" application message to interested clients. A receiving client may parse the "GPCCAssetGroupMessage" application message and identify the G-PCC media assets present in the MMT content sending entity. A streaming client may identify available G-PCC media content, for example, by checking the "application_identifier" field in the "GPCCAssetGroupMessage" application message (e.g., set to "urn:mpeg:mmt:app:gpcc:2020"). Available G-PCC assets within G-PCC point cloud content (e.g., all G-PCC assets) may be identified, for example, by checking the asset_id present in the "GPCCAssetGroupMessage" application message. A client may select (e.g., choose) an asset_id to be streamed, for example, based on the user's current viewport. An MMT client may send a "GPCCSelectionFeedback" application message to the sending entity requesting G-PCC assets of interest from the list of available G-PCC assets. An MMT transmitting entity may form an MMTP packet using MTP. The MMT transmitting entity may send the MTTP packet to a client. The MMT client may receive the MMTP packet. The MMT client may depacket the MPU or MFU. The MPU / MFU may contain timed or non-timed G-PCC media content.

[0199] G-PCC asset data may represent initialization information (e.g., SPS, GPS, APS, and / or tile inventory) when an MMT client receives an MMTP packet with asset group "data_type" set to "3". G-PCC asset data may represent 3D spatial domain time-limited metadata information when an MMT client receives an MMTP packet with asset group "data_type" set to "4". G-PCC asset information may be used for partial access to G-PCC data.

[0200] An MMT client may select G-PCC assets based on the user viewport and the corresponding 3D spatial region. The MMT client may send a "GPCCSelectionFeedback" application message to the sending entity requesting the target G-PCC assets. The MMT client may request a different set of G-PCC assets (for example, using the "GPCCSelectionFeedback" application message) when the user viewport changes.

[0201] An MMT client may send a "GPCCViewChangeFeedback" message to the sending entity (for example, to signal the user's current viewport) when / when the user viewport changes. The MMT sending entity may select a G-PCC asset (for example, based on the user's new viewport information) (for example, upon receiving a message from the MMT client). The MMT sending entity may send a "GPCCAssetGroupMessage" application message to the MMT client along with the corresponding G-PCC asset. The MMT sending entity may stream the G-PCC asset data as an MMTP packet.

[0202] The MMT client may begin receiving MMTP packets for the requested G-PCC assets (e.g., all). The MMT client may extract the MPU and MFU from the MMTP payload. The MPU and MFU may contain media samples (e.g., direct) or media segments.

[0203] The MMT client may initiate parsing of the media segment container (e.g., ISOBMFF), extract elementary stream information, structure the G-PCC bitstream, and pass the bitstream to the G-PCC decoder. For example, if the MMTP payload contains a G-PCC media sample, the elementary stream data may be extracted and structured, and the bitstream may be passed to the G-PCC decoder.

[0204] Systems, methods, and apparatus for MPEG Media Transport (MMT) streaming of geometry-based point clouds (G-PCCs) are described herein. G-PCC encoded content may be delivered over a network using MMT. G-PCC data may be encapsulated for MMT streaming. MMT signaling messages may support the delivery of G-PCC data over MMT. Each track may be encapsulated into a separate asset that can be packetized into an MMTP packet, for example, when a G-PCC component in an International Organization for Standardization-Based Media File Format (ISOBMFF) is signaled using multiple tracks. G-PCC definition application messages may enable servers and clients to identify a group of multiple assets for a G-PCC component.

[0205] While features and elements are described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in computer programs, software, or firmware embedded in computer-readable media for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor associated with software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A method performed in a receiving device, wherein the method is From the transmitting device, A first message containing a list of media assets available for streaming from the transmitting device to the receiving device, or Receiving at least one of one or more messages describing each of the media assets available for streaming from the transmitting device to the receiving device; sending a second message to the transmitting device requesting a subset of the media assets to be streamed from the transmitting device to the receiving device, the requested subset of the media assets being determined based on the viewport of the receiving device, and including information indicating the requested subset of the media assets. The transmitting device receives one or more Motion Picture Experts Group (MPEG) Media Transport Protocol (MMTP) packets in response to the second message, A method comprising processing one or more MMTP packets to restore at least a portion of the requested subset of the media asset.

2. The method according to claim 1, further comprising sending a third message to the transmitting device, which includes information indicating a request for the requested updated subset of the media assets, the requested updated subset of the media assets being streamed from the transmitting device to the receiving device, wherein the requested updated subset of the media assets is determined based on an updated viewport of the receiving device.

3. The method according to claim 1, wherein the first message received from the transmitting device further includes information identifying an application associated with the list of media assets.

4. The method according to claim 3, wherein the information identifying the application indicates that the application consumes visual volumetric video-based coding (V3C) data.

5. The method according to claim 3, wherein the information identifying the application indicates that the application consumes geometry-based point cloud compression (G-PCC) data.

6. The method according to claim 1, wherein the first message includes information indicating one or more of the following: the dependency of a media asset on another media asset for decoding; an indication of the other media asset on which the media asset depends; whether the media asset has an alternate version; and an identification of the alternate version of the media asset.

7. A receiving device, Processor and Equipped with a communication interface, The processor and the communication interface are transmitted from the transmitting device. A first message containing information indicating a list of media assets available for streaming from the transmitting device to the receiving device, or The device is configured to receive at least one of one or more messages describing the media assets available for streaming from the transmitting device to the receiving device. The processor and the communication interface are configured to send to the transmitting device a second message to the transmitting device which includes information indicating a request for a subset of the media assets to be streamed from the transmitting device to the receiving device, wherein the requested subset of the media assets is determined based on the viewport of the receiving device. The processor and the communication interface are configured to receive one or more Motion Picture Experts Group (MPEG) Media Transport Protocol (MMTP) packets from the transmitting device in response to the second message. A receiving device, wherein the processor is configured to process one or more MMTP packets to restore at least a portion of the requested subset of the media assets.

8. The receiving device according to claim 7, further comprising sending to the transmitting device an updated subset of the media assets to be streamed from the transmitting device to the receiving device, the requested updated subset of the media assets being determined based on an updated viewport of the receiving device, the requested updated subset of the media assets being a third message indicating a request for the requested updated subset of the media assets.

9. The receiving device according to claim 7, wherein the first message received from the transmitting device further includes information identifying an application associated with the list of media assets.

10. The receiving device according to claim 9, wherein the information identifying the application indicates that the application consumes visual volumetric video-based coding (V3C) data.

11. The receiving device according to claim 9, wherein the information identifying the application indicates that the application consumes geometry-based point cloud compression (G-PCC) data.

12. The receiving device according to claim 7, wherein the first message includes information indicating one or more of the following: the dependency of a media asset on another media asset for decoding; an indication of the other media asset on which the media asset depends; whether the media asset has an alternate version; and an identification of the alternate version of the media asset.

13. A receiving device, Processor and Equipped with a communication interface, The processor and the communication interface are transmitted from the transmitting device. A first message containing information indicating a set of media assets available for streaming from the transmitting device to the receiving device, or The device is configured to receive at least one of one or more messages describing the media assets available for streaming from the transmitting device to the receiving device. The processor and the communication interface are configured to send a second message to the transmitting device, which includes information indicating the viewport of the receiving device. The processor and the communication interface are configured to receive from the transmitting device a third message containing information indicating the indicated subset of the media assets, which is to be streamed from the transmitting device to the receiving device, and which is determined based on the indicated viewport of the receiving device. The processor and the communication interface are configured to receive one or more Motion Picture Experts Group (MPEG) Media Transport Protocol (MMTP) packets from the transmitting device in response to the third message. The receiving device is configured such that the processor processes one or more MMTP packets to reconstruct at least a portion of the indicated subset of the media asset.

14. The processor and the communication interface are configured to send a fourth message to the transmitting device, which includes information indicating the updated viewport of the receiving device. The processor and the communication interface are configured to receive a fourth message from the transmitting device, which includes information indicating an updated set of media assets associated with the updated viewport. The processor and the communication interface are configured to receive one or more other MMTP packets from the transmitting device. The receiving device according to claim 13, wherein the processor is configured to process one or more other MMTP packets to restore at least a portion of the updated set of media assets associated with the updated viewport of the receiving device.

15. The receiving device according to claim 13, wherein the first message received from the transmitting device further includes information identifying an application associated with the list of media assets.

16. The receiving device according to claim 15, wherein the information identifying the application indicates that the application consumes visual volumetric video-based coding (V3C) data.

17. The receiving device according to claim 15, wherein the information identifying the application indicates that the application consumes geometry-based point cloud compression (G-PCC) data.

18. The receiving device according to claim 13, wherein the first message includes information indicating one or more of the following: the dependency of a media asset on another media asset for decoding; an indication of the other media asset on which the media asset depends; whether the media asset has an alternate version; and an identification of the alternate version of the media asset.

Images